Blood vessel segmentation device, blood vessel segmentation method, and medical image processing device

Abstract

The invention provides a blood vessel segmentation apparatus, a blood vessel segmentation method, and a medical image processing apparatus. A blood vessel segmentation device is provided with: an image acquisition unit that acquires a plurality of medical images; the deep learning model acquisition part is used for carrying out deep learning model training by using a center line enhanced loss function and a shape statistic constrained loss function based on the medical image to obtain a trained deep learning model; and a segmentation unit which, on the basis of the trained deep learning model, segments the acquired medical image of the subject, and segments the blood vessel to obtain a segmentation result.

Description

Technical Field

[0001] The present invention relates to a blood vessel segmentation device, a blood vessel segmentation method and a medical image processing device for performing blood vessel segmentation on medical images based on deep learning. Background Art

[0002] In clinical research and application of medicine, three-dimensional reconstruction of blood vessels based on medical images acquired by medical imaging devices can assist doctors in quickly locating organs, tissues, tumors, etc. Therefore, fast and accurate segmentation of blood vessels is an important task in medical image processing.

[0003] With the continuous development of computer technology, the existing algorithms used for blood vessel segmentation generally include algorithms based on blood vessel enhancement filtering, algorithms based on deformable models, algorithms based on tracking, and algorithms based on deep learning.

[0004] The algorithm based on vascular enhancement filtering can enhance the vascular part in the original image, but it will introduce noise. In addition, the algorithm based on the deformable model needs to initialize the seed point, for example, the point with a larger gray value after the vascular enhancement is used as the initial seed point, and then the vascular segmentation is performed in combination with the watershed and region growing algorithms. Seed point initialization will affect the accuracy of vascular segmentation, and the calculation is complex. In addition, the algorithm based on tracking needs to use multiple preset templates and combine the initial position and tracking direction to achieve vascular segmentation. Multiple preset templates are a complicated process. In addition, the algorithm based on deep learning uses the method of classifying the original image pixel by pixel to achieve vascular segmentation, but the algorithm based on deep learning is too dependent on the amount of labeled data and the quality of the annotation. In addition, due to the contrast between blood vessels and other organ tissues in medical images, it is difficult to achieve high-precision vascular annotation, so it is not easy for the algorithm based on deep learning to achieve high-precision vascular segmentation, and the connectivity of the blood vessels in the segmentation results is poor.

[0005] It can be seen that the existing blood vessel segmentation algorithm needs to further improve the accuracy of blood vessel segmentation, while reducing the situation where the blood vessels are actually connected but not connected in the segmentation results due to the contrast between blood vessels and tissues. Summary of the invention

[0006] In view of this situation, an object of the present invention is to provide a blood vessel segmentation device, a blood vessel segmentation method and a medical image processing device that can perform blood vessel segmentation with high accuracy even when the contrast between blood vessels and tissues in medical images is not high, and improve the connectivity of blood vessels in the segmentation results.

[0007] In order to achieve the above-mentioned purpose, the blood vessel segmentation device of the present invention comprises: an image acquisition unit, which acquires multiple medical images; a deep learning model acquisition unit, which acquires a trained deep learning model obtained by training the deep learning model based on the medical images using a centerline enhancement loss function and a shape statistic constraint loss function; and a segmentation unit, which segments the acquired object medical image based on the trained deep learning model, segments the blood vessels and obtains a segmentation result.

[0008] In addition, the blood vessel segmentation device of the present invention further comprises a feature domain enhancement unit, which applies tubular filtering to extract tubular features and obtains a feature domain of enhanced tubular features for the segmentation result, and applies regional growing to connect the tubular features to obtain a segmentation result after feature domain enhancement.

[0009] In addition, the blood vessel segmentation device of the present invention further comprises an image domain enhancement unit, which performs connectivity restoration processing on the free blood vessel segments in the segmentation result in the image domain to obtain the segmentation result after image domain enhancement.

[0010] In addition, the blood vessel segmentation device of the present invention further comprises a morphological processing unit, which performs morphological image processing on the segmentation result, the segmentation result after feature domain enhancement and / or the segmentation result after image domain enhancement to obtain a final segmentation result.

[0011] In addition, the blood vessel segmentation device of the present invention also includes a preprocessing unit, which preprocesses the multiple medical images obtained to obtain standardized medical images; the deep learning model acquisition unit obtains a trained deep learning model based on the standardized medical images by training the deep learning model using a centerline enhancement loss function and a shape statistics constraint loss function.

[0012] In addition, in the blood vessel segmentation device of the present invention, the deep learning model acquisition unit uses an integrated loss function obtained by weighted sum of a Dice coefficient loss function, a cross entropy loss function, a centerline enhancement loss function, and a shape statistic constraint loss function to train the deep learning model to obtain a trained deep learning model.

[0013] In addition, in the blood vessel segmentation device of the present invention, the feature domain enhancement unit applies a Hessian matrix to extract tubular features based on the segmentation result.

[0014] In addition, the blood vessel segmentation method in the blood vessel segmentation device of the present invention includes: an image acquisition step, acquiring multiple medical images; a deep learning model acquisition step, acquiring a trained deep learning model obtained by training the deep learning model based on the medical images using a centerline enhancement loss function and a shape statistic constraint loss function; and a segmentation step, based on the trained deep learning model, segmenting the acquired object medical image to segment the blood vessels and obtain a segmentation result.

[0015] Furthermore, the medical image processing apparatus of the present invention comprises: a target image acquisition unit that acquires a plurality of target medical images; any one of the above-mentioned blood vessel segmentation devices; and an output unit that outputs the segmentation result.

[0016] [Effects of the invention]

[0017] As described above, according to the blood vessel segmentation device, blood vessel segmentation method and medical image processing device of the present invention, blood vessel segmentation can be performed with high accuracy even when the contrast between blood vessels and tissues in medical images is not high, and the connectivity of blood vessels in the segmentation results can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram showing the structure of the blood vessel segmentation device according to the first embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of general deep learning model training.

[0020] Figure 3 This is an operation flow chart of the blood vessel segmentation device 1 according to the first embodiment of the present invention.

[0021] Figure 4A to Figure 4C It is a schematic diagram of a specific example of the blood vessel segmentation device according to the first embodiment of the present invention.

[0022] Figure 5 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a second embodiment of the present invention.

[0023] Figure 6 This is the operation flow of the blood vessel segmentation device according to the second embodiment of the present invention.

[0024] Figure 7 FIG. 1 is a schematic diagram of a region growing process performed by the blood vessel segmentation device according to the second embodiment of the present invention.

[0025] Figure 8 It is a schematic diagram of a specific example of the blood vessel segmentation device according to the second embodiment of the present invention.

[0026] Fig. 9 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a third embodiment of the present invention.

[0027] Fig.10 It is a schematic diagram for explaining the process of the image domain enhancement unit performing connectivity repair on the segmentation result.

[0028] Fig.11 This is the operation flow of the blood vessel segmentation device according to the third embodiment of the present invention.

[0029] Fig.12 It is a schematic diagram of a specific example of the blood vessel segmentation device according to the third embodiment of the present invention.

[0030] Fig.13 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a fourth embodiment of the present invention.

[0031] Fig.14 This is an operation flow chart of the blood vessel segmentation device according to the fourth embodiment of the present invention.

[0032] Fig.15 It is a schematic diagram of a specific example of the blood vessel segmentation device according to the fourth embodiment of the present invention.

[0033] Fig.16 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a fifth embodiment of the present invention.

[0034] Fig.17 This is an operation flow chart of the blood vessel segmentation device according to the fifth embodiment of the present invention.

[0035] Fig.18 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a sixth embodiment of the present invention.

[0036] Fig.19 This is an operation flow chart of the blood vessel segmentation device according to the sixth embodiment of the present invention. DETAILED DESCRIPTION

[0037] Hereinafter, a blood vessel segmentation device, a blood vessel segmentation method, and a medical image processing device according to the present invention will be described with reference to the accompanying drawings.

[0038] (First Embodiment)

[0039] ※ Configuration of the blood vessel segmentation device 100

[0040] First, refer to Figure 1 The structure of the blood vessel segmentation device 100 according to the first embodiment of the present invention is described. In addition, the blood vessel segmentation device includes various components. Figure 1 Only components related to the technical concept of the present invention are shown, while other components are omitted.

[0041] like Figure 1As shown in FIG. 1 , the blood vessel segmentation device 100 of the present invention includes an image acquisition unit 101 , a deep learning model acquisition unit 102 , and a segmentation unit 103 .

[0042] The image acquisition unit 101 may be composed of a component or module with information processing functions such as a CPU or an MCU, and is used to acquire multiple medical images. The image acquisition unit 101 may acquire multiple medical images from a medical imaging device through communication, or may read a storage device pre-stored with multiple medical images to acquire multiple medical images, or may read multiple medical images processed by other components.

[0043] Here, the medical image may be an ultrasonic image, a CT image, an MR image or the like captured by a medical imaging device such as an ultrasonic diagnostic device, a CT (computed tomography) imaging device, or an MR (magnetic resonance) imaging device. That is, the medical image may be a three-dimensional image or a two-dimensional image.

[0044] The deep learning model acquisition unit 102, for example, can be composed of a component or module with information processing functions such as a CPU or MCU, and is used to obtain a trained deep learning model obtained by training a deep learning model based on medical images. The deep learning model acquisition unit 102 obtains a trained deep learning model obtained by training a deep learning model using at least a centerline enhancement loss function and a shape statistic constraint loss function. In addition, the deep learning model acquisition unit 102 can, for example, obtain a trained deep learning model obtained by training a deep learning model using an integrated loss function obtained by weighted summing a Dice coefficient loss function, a cross entropy loss function, a centerline enhancement loss function, and a shape statistic constraint loss function.

[0045] Figure 2 The following is a schematic diagram of general deep learning model training. Figure 2 As shown, the deep learning model training is performed based on the pre-annotated annotations (GT, Ground Truth) and multiple medical images obtained by the image acquisition unit 101. Specifically, the medical image is input as an input image (input volume) to the deep learning network to obtain the prediction result (Predication). The value L of the loss function (LOSS) is total Used to measure the degree of deviation between the prediction result (Predication) and GT (Ground Truth).

[0046] In the present embodiment, the annotation GT can be acquired by, for example, the image acquisition unit 1 .

[0047] In this embodiment, the centerline enhancement loss function L is introduced. CLand shape statistics constraint loss function L AC The trained deep learning model obtained by training the deep learning model can enhance the connectivity and geometric topological structure of the blood vessels in the segmentation results.

[0048] In this embodiment, a trained deep learning model is obtained by training the deep learning model using an integrated loss function obtained by weighted summing the Dice coefficient loss function, cross entropy loss function, centerline enhancement loss function, and shape statistic constraint loss function as shown in Formula 1.

[0049] (Formula 1)

[0050]

[0051] Among them, L total represents the loss function (LOSS), L DICE represents the Dice coefficient loss function, L CE represents the cross entropy loss function, L CL represents the centerline enhancement loss function, L AC represents the shape statistics constraint loss function.

[0052] Here, the weights of the Dice coefficient loss function, the cross entropy loss function, the centerline enhancement loss function, and the shape statistics constraint loss function are set to 0.3:0.3:0.3:1, respectively.

[0053] In addition, the centerline enhancement loss function L CL As shown in formula 2.

[0054] (Formula 2)

[0055]

[0056] in,

[0057]

[0058] V L Indicates the blood vessels in the labeled GT, V p represents the liver blood vessels in the prediction results, S L Indicates the centerline structure of the blood vessels in the annotated GT, S P Represents the centerline structure of the blood vessel in the prediction result.

[0059] As mentioned above, the centerline enhancement loss function takes topology into account and is calculated on the centerline of the vessel segmentation mask (vessel GT). It can segment tubular objects while maintaining the topological structure, thereby improving the segmentation effect of small vessel branches.

[0060] In addition, the statistic constrains the loss function L AC As shown in formula 3.

[0061] (Formula 3)

[0062]

[0063] in,

[0064]

[0065] c1 and c2 represent the means inside and outside the segmentation contour C, respectively. f(x) represents the pixel intensity of the image at position x. Length(C) represents the length of the segmentation contour C. λ represents the parameter used to balance the weights.

[0066] The shape statistics constraint loss function regards the segmentation problem as an energy minimization problem, and energy is a function of the contour line. The energy function is minimized using a partial differential method to make the contour line evolve continuously toward the target boundary. The curve position when the energy is minimized is the contour of the target, and the relationship between the segmentation contour and the volume is considered. By referencing the shape statistics constraint loss function, the prior knowledge of the shape can be added to the regularization part, making it more consistent with the shape characteristics of the segmentation target and improving the image segmentation results.

[0067] In addition, regarding L DICE and L CE , which is the prior art and will not be described in detail here.

[0068] By applying the trained deep learning model obtained by deep learning training using the above-mentioned integrated loss function, the deficiency of the existing technology that only considers grayscale or texture information but not geometric information when only considering the Dice coefficient loss function and the cross entropy loss function is made up for, and the connectivity and geometric topological structure of blood vessels in the segmentation results can be enhanced.

[0069] Return to Figure 1 The segmentation unit 103 may be composed of a component or module having information processing functions, such as a CPU or an MCU, and may segment the acquired object medical image based on the trained deep learning model obtained by the deep learning model acquisition unit 102 to segment the blood vessels and obtain a segmentation result.

[0070] Here, the target medical image may be acquired by, for example, the image acquisition unit 101 as described above, or may be acquired by other components not shown.

[0071] ※Operation of the blood vessel segmentation device 100

[0072] Next, a blood vessel segmentation method applied by the blood vessel segmentation device 100 according to the first embodiment of the present invention will be described. Figure 3 This is an operation flow chart of the blood vessel segmentation device 100 according to the first embodiment of the present invention.

[0073] like Figure 3 As shown, after the blood vessel segmentation device 100 starts processing, in step S100, the image acquisition unit 101 acquires multiple medical images, and then enters step S200. Figure 4A Liver images such as those shown Figure 4B The annotation GT is shown. Here, Figure 4A Only one liver image is shown, and other liver images are omitted. Figure 4B In the figure, the portion marked with hatching indicates the blood vessel region.

[0074] In step S200, the deep learning model acquisition unit 102 acquires a trained deep learning model obtained by deep learning model training based on the multiple medical images acquired by the image acquisition unit 101 in step S100, and then proceeds to step S300. Here, in the deep learning model training, the centerline enhancement loss function and the shape statistics constraint loss function are introduced.

[0075] In step S300, the segmentation unit 103 segments the acquired object medical image based on the trained deep learning model obtained by the deep learning model acquisition unit 102, and segments the blood vessels to obtain Figure 4C Here, it is assumed that the target medical image is Figure 4A A portion of a plurality of medical images acquired by the image acquisition unit 101 as shown in FIG. Figure 4C In the figure, the section line part is the segmented blood vessel, and the solid line frame marks the relative Figure 4B The portion further segmented out is shown as marked GT.

[0076] contrast Figure 4A The object image shown, Figure 4B The labeled GT images shown and Figure 4C From the segmentation results shown, it can be seen that according to the blood vessel segmentation device of this embodiment, since a trained deep learning model obtained by deep learning training with reference to the centerline enhancement loss function and the shape statistics constraint loss function is applied, more blood vessels can be segmented even when the contrast between blood vessels and tissues in medical images is not high, blood vessel segmentation can be performed with high precision, and the connectivity of blood vessels in the segmentation results can be improved.

[0077] (Second Embodiment)

[0078] ※ Configuration of the blood vessel segmentation device 100A

[0079] First, refer to Figure 5 , the structure of a blood vessel segmentation device 100A according to a second embodiment of the present invention will be described. Figure 5 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a second embodiment of the present invention.

[0080] like Figure 5 As shown, the blood vessel segmentation device 100A of this embodiment includes an image acquisition unit 101, a deep learning model acquisition unit 102, and a segmentation unit 103, and further includes a feature domain enhancement unit 104. Here, the image acquisition unit 101, the deep learning model acquisition unit 102, and the segmentation unit 103 are respectively the same as the image acquisition unit 101, the deep learning model acquisition unit 102, and the segmentation unit 103 in the first embodiment. No further description is given here.

[0081] In addition, the feature domain enhancement unit 104 applies vesselness filtering to the segmentation result of the segmentation unit 103 to extract the tubular features and obtain the feature domain of the enhanced tubular features, and applies region growing to connect the tubular features to obtain the segmentation result after the feature domain enhancement.

[0082] Here, the feature domain enhancement unit 104 applies, for example, the Hessian Matrix to extract the tubular features based on the segmentation result. This is because the eigenvalues ​​of the Hessian Matrix can measure the information of the tubular structure. However, the method of extracting the tubular features is not limited to the method of applying the Hessian Matrix, and various methods in the known technology can be applied, which will not be described here.

[0083] In addition, the feature region enhancement unit 104 uses region growing to connect the broken blood vessels that are actually connected but appear broken in the segmentation result of the segmentation unit.

[0084] Specifically, the feature domain enhancement unit 104 designs a tubular metric to describe the probability of whether a voxel in the image is tubular, so as to identify blood vessels with relatively low contrast with tissues (e.g., the substantial part outside the blood vessels in the organ) in the image. For example, the diameter of the target blood vessel is set to be approximately 0.5 mm to 6 mm, and 0.5 mm is selected as the step length. The second-order gradient information of the image as the segmentation result of the segmentation unit 103 is extracted by using the Hessian matrix analysis, that is, the operation based on the Hessian matrix is ​​performed for each voxel. When the operation result based on the Hessian matrix is ​​that there are only two relatively large eigenvalues, the voxel is considered to belong to the tubular structure. Here, the relatively large eigenvalue can be appropriately set according to the organ, image, etc. In addition, the setting of the metric value of the diameter and step length as the target blood vessel is only an example, and is not limited to this, and can be arbitrarily set according to actual conditions.

[0085] In addition, regarding region growing, points with larger probability values ​​in the probability map (mask) included in the segmentation result of the segmentation unit 103 are used as seed points, and 60, for example, is used as the lower threshold to perform region growing in the tubular feature domain to repair broken blood vessels and supplement small blood vessels.

[0086] ※Operation of the blood vessel segmentation device 100A

[0087] Next, a blood vessel segmentation method applied by the blood vessel segmentation device 100A according to the second embodiment of the present invention will be described. Figure 6 This is an operation flow chart of the blood vessel segmentation device 100A according to the second embodiment of the present invention.

[0088] like Figure 6 As shown, after the blood vessel segmentation device 100A starts processing, the processing of steps S100 to S300 is performed. The processing of steps S100 to S300 is similar to that of Figure 3 The processing of steps S100 to S300 is the same, and the detailed description is omitted here. In addition, in this embodiment, the segmentation result also includes a probability map.

[0089] In step S301, the feature domain enhancement unit 104 applies tubular filtering to extract tubular features and obtains a feature domain of enhanced tubular features based on the segmentation result of the segmentation unit 103 in step S300, and applies regional growing to connect the tubular features to obtain a segmentation result after feature domain enhancement.

[0090] Specifically, in step S301, Figure 7 As shown, based on the medical image acquired by image acquisition 101 and the probability map included in the segmentation result of step S300, based on the Hessian matrix, tubular filtering is applied to extract tubular features and obtain the feature domain of the enhanced tubular features, and the points with larger probability values ​​in the probability map are used as seed points, and regional growth is performed in the tubular feature domain to connect the tubular features.

[0091] This embodiment also applies the specific example of the first embodiment. Figure 4A The object image shown, Figure 4B The labeled GT image shown, Figure 4C The segmentation result of the first embodiment shown and Figure 8 As can be seen from the segmentation results of the present embodiment, the blood vessel segmentation device according to the present embodiment has the technical effect of the first embodiment. In addition, due to the feature domain enhancement unit 104, based on tubular filtering and region growing, the broken blood vessels in the segmentation results based on the deep learning model are repaired in the tubular feature domain, which can also improve the segmentation quality of small blood vessels and supplement small blood vessels.

[0092] (Third Embodiment)

[0093] ※ Configuration of the blood vessel segmentation device 100B

[0094] First, refer to Fig. 9 , the structure of a blood vessel segmentation device 100B according to a third embodiment of the present invention will be described. Fig. 9 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a third embodiment of the present invention.

[0095] like Fig. 9 As shown, the blood vessel segmentation device 100B of this embodiment includes an image domain enhancement unit 105 in addition to the image acquisition unit 101, the deep learning model acquisition unit 102 and the segmentation unit 103. Here, the image acquisition unit 101, the deep learning model acquisition unit 102 and the segmentation unit 103 are respectively the same as the image acquisition unit 101, the deep learning model acquisition unit 102 and the segmentation unit 103 in the first embodiment. No further description is given here.

[0096] In addition, the image domain enhancement unit 105 performs connectivity restoration processing on the free blood vessel segments in the segmentation result of the segmentation unit 103 in the image domain to obtain the segmentation result after image domain enhancement. For example, the image domain enhancement unit 105 performs connectivity restoration on the segmentation result of the segmentation unit 103 using a super voxel method.

[0097] Here, combined Fig.10 , the process of the image domain enhancement unit 105 using the super voxel method to repair the connectivity of the segmentation result is described. Fig.10 2 is a schematic diagram for explaining the process of the image domain enhancement unit performing connectivity restoration on the segmentation result. In addition, for the convenience of explanation, only a part of the segmentation result is shown.

[0098] First, the image domain enhancement unit 105 performs free branch analysis on the segmentation result of the segmentation unit 103 to determine the main body of the blood vessel in the segmentation result and the free branches that are disconnected parts. Fig.10 As shown in FIG. 1 , the connected domain with the largest volume in the segmentation result is taken as the main vessel M, and the vessel segments other than the main vessel M that are not connected to the main vessel and have a smaller volume are taken as free branches D. Fig.10 In the figure, for the sake of convenience, only the main vessel M and one free branch D are shown, but there may be multiple free branches. The image domain enhancement unit 105 performs the same process of connectivity restoration on each free branch. Fig.10 The connectivity repair process of the free branch D shown in FIG. 1 is used to illustrate. In addition, when there are multiple free branches, the image domain enhancement unit 105 can perform connectivity repair on each free branch in sequence or in parallel.

[0099] Next, the image domain enhancement unit 105 extracts the center line of the free branch D, determines the two endpoints (star marks) P1 and P2 of the center line of the free branch D, calculates the distance between the two endpoints P1 and P2 and the blood vessel main body M, and takes the endpoint with the smaller distance as the proximal endpoint of the free branch D. Here, the shortest distance between the two endpoints P1 and P2 and the blood vessel main body M (i.e., the shortest distance from a point to a volume) is defined as the distance between the endpoint and the blood vessel main body M. Fig.10 In the case shown, the endpoint P2 is the proximal endpoint of the free branch D.

[0100] Then, the image domain enhancement unit 105 connects the proximal end point P2 of the free branch D to the vascular trunk M based on supervoxel analysis. The method of connecting the proximal end point P2 of the free branch D to the vascular trunk M based on supervoxel analysis is a conventional technique and its description is omitted here.

[0101] ※Operation of the blood vessel segmentation device 100B

[0102] Next, a blood vessel segmentation method applied by the blood vessel segmentation device 100B according to the third embodiment of the present invention will be described. Fig.11 This is an operation flow chart of the blood vessel segmentation device 100B according to the third embodiment of the present invention.

[0103] like Fig.11 As shown, after the blood vessel segmentation device 100B starts processing, it executes the processing of steps S100 to S300. The processing of steps S100 to S300 is similar to that of Figure 3 The processing of steps S100 to S300 is the same, and the detailed description is omitted here.

[0104] In step S302, the image domain enhancement unit 105 performs the following steps in the image domain based on the segmentation result of the segmentation unit 103 in step S300: Fig.10 This illustrates that the connectivity of the free blood vessel segments is restored, and the segmentation result after image domain enhancement is obtained.

[0105] This embodiment also applies the specific example of the first embodiment. Figure 4A The object image shown, Figure 4B The labeled GT image shown, Figure 4C The segmentation result of the first embodiment shown and Fig.12As can be seen from the segmentation results of the present embodiment shown, the blood vessel segmentation device according to the present embodiment has the technical effect of the first embodiment. In addition, due to the image domain enhancement unit 105, the blood vessel segmentation result is analyzed based on the supervoxel method, and the connectivity of the disconnected blood vessels is repaired in the image domain according to the distance between the end point of the center line of the free segment and the main branch. In this way, the connectivity problem of the free blood vessels without protruding tubular structures that cannot be processed by the feature domain enhancement unit can be solved.

[0106] (Fourth Embodiment)

[0107] ※ Configuration of the blood vessel segmentation device 100C

[0108] First, refer to Fig.13 , the structure of a blood vessel segmentation device 100C according to a fourth embodiment of the present invention will be described. Fig.13 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a fourth embodiment of the present invention.

[0109] like Fig.13 As shown, the blood vessel segmentation device 100C of this embodiment includes, in addition to the image acquisition unit 101, the deep learning model acquisition unit 102 and the segmentation unit 103, a feature domain enhancement unit 104 and an image domain enhancement unit 105a. Here, the image acquisition unit 101, the deep learning model acquisition unit 102 and the segmentation unit 103 are respectively the same as the image acquisition unit 101, the deep learning model acquisition unit 102 and the segmentation unit 103 in the first embodiment, the feature domain enhancement unit 104 is the same as the feature domain enhancement unit 104 in the second embodiment, and the image domain enhancement unit 105a is different from the image domain enhancement unit 105 in the second embodiment only in that the object processed by the image domain enhancement unit 105a is the segmentation result after the feature domain enhancement by the feature domain enhancement unit 104, rather than the segmentation result of the segmentation unit 103, and repeated descriptions are omitted here.

[0110] ※Operation of the blood vessel segmentation device 100C

[0111] Next, a blood vessel segmentation method applied by the blood vessel segmentation device 100C according to the fourth embodiment of the present invention will be described. Fig.14 This is an operation flow chart of the blood vessel segmentation device 100C according to the fourth embodiment of the present invention.

[0112] like Fig.14 As shown, after the blood vessel segmentation device 100C starts processing, the processing of steps S100 to S301 is performed. The processing of steps S100 to S301 is similar to that of Figure 6 The processing of steps S100 to S301 is the same, and the detailed description is omitted here.

[0113] In step S302, the image domain enhancement unit 105 performs the segmentation process in the image domain as shown in the reference numeral 104 according to the segmentation result processed by the feature domain enhancement unit 104 in step S301. Fig.10 This illustrates that the connectivity of the free blood vessel segments is restored, and the segmentation result after image domain enhancement is obtained.

[0114] This embodiment also applies the specific example of the first embodiment. Figure 4A The object image shown, Figure 4B The labeled GT image shown, Figure 4C The segmentation result of the first embodiment shown and Fig.15 It can be seen from the segmentation results of this embodiment shown that the blood vessel segmentation device according to this embodiment has the technical effects of the first to third embodiments, which will not be described in detail here.

[0115] (Fifth Embodiment)

[0116] ※ Configuration of the blood vessel segmentation device 100D

[0117] First, refer to Fig.16 , the structure of a blood vessel segmentation device 100D according to a fifth embodiment of the present invention will be described. Fig.16 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a fifth embodiment of the present invention.

[0118] like Fig.16 As shown, the blood vessel segmentation device 100D of this embodiment includes a pre-processing unit 106 in addition to the image acquisition unit 101 , the deep learning model acquisition unit 102 , and the segmentation unit 103 .

[0119] Here, the image acquisition unit 101 , the deep learning model acquisition unit 102 , and the segmentation unit 103 are respectively the same as the image acquisition unit 101 , the deep learning model acquisition unit 102 , and the segmentation unit 103 in the first embodiment, and will not be described in detail here.

[0120] In addition, the preprocessing unit 106 performs preprocessing on the plurality of medical images acquired by the image acquisition unit 101 to obtain standardized medical images.

[0121] Furthermore, the deep learning model acquisition unit 102 acquires a trained deep learning model obtained by performing deep learning model training using a centerline enhancement loss function and a shape statistics constraint loss function based on the standardized medical image preprocessed by the preprocessing unit 106 .

[0122] The preprocessing performed by the preprocessing unit 106 on the plurality of medical images may be, for example, one or more of data format conversion, cropping, resampling, and normalization.

[0123] Regarding data format conversion preprocessing, common medical image formats include DICOM (suffix .dcm), MHD (suffix .mhd and .raw), and NIFTY (suffix .nii or .nii.gz), but these formats are not suitable for direct deep learning operations and usually require data format conversion.

[0124] In addition, the image cropping preprocessing is to crop the three-dimensional medical image to its non-zero area. Compared with the image before cropping, the cropped medical image has no effect on the final segmentation result, but it can reduce the image size, avoid useless calculations, and improve calculation efficiency.

[0125] In addition, the purpose of resampling preprocessing is to solve the problem that in some three-dimensional medical image data sets, the actual space size (spacing) represented by a single voxel in different medical images is inconsistent. In order to avoid this difference, it is necessary to reset the size of different medical image data in the voxel space to ensure that the actual physical space represented by each voxel in different image data is consistent.

[0126] In addition, regarding normalization preprocessing, the purpose of normalization preprocessing for medical images is to make the grayscale values ​​of each medical image in the training set have the same distribution.

[0127] Here, the preprocessing unit 106 performs at least cropping preprocessing and normalization preprocessing, for example.

[0128] ※Operation of the blood vessel segmentation device 100D

[0129] Next, a blood vessel segmentation method applied by the blood vessel segmentation device 100D according to the fifth embodiment of the present invention will be described. Fig.17 This is an operation flow chart of the blood vessel segmentation device 100D according to the fifth embodiment of the present invention.

[0130] like Fig.17 As shown, after the blood vessel segmentation device 100D starts processing, it executes the processing of step S100 and then proceeds to step S101.

[0131] In step S101 , the preprocessing unit 106 performs at least cropping preprocessing and normalization preprocessing on the acquired medical image, and then proceeds to step S200 .

[0132] The processing after step S200 is the same as Figure 3 The processing of steps S200 to S300 is the same, and the detailed description is omitted here.

[0133] The blood vessel segmentation device of this embodiment has the technical effects of the first embodiment. In addition, since the pre-processing unit 106 is provided, blood vessel segmentation can be performed quickly and efficiently.

[0134] (Sixth Embodiment)

[0135] ※ Configuration of the blood vessel segmentation device 100E

[0136] First, refer to Fig.18 , the structure of a blood vessel segmentation device 100E according to a sixth embodiment of the present invention will be described. Fig.18 It is a schematic diagram showing the structure of a blood vessel segmentation device according to a sixth embodiment of the present invention.

[0137] like Fig.18 As shown, the blood vessel segmentation device 100E of this embodiment includes a morphological processing unit 107 in addition to the image acquisition unit 101 , the deep learning model acquisition unit 102 , the segmentation unit 103 , and the pre-processing unit 106 .

[0138] Here, the image acquisition unit 101, the deep learning model acquisition unit 102, the segmentation unit 103 and the preprocessing unit 106 are respectively the same as those in the fifth embodiment, and will not be described in detail here.

[0139] In addition, the morphological processing unit 107 performs morphological image processing on the segmentation result to obtain a final segmentation result. Here, the segmentation result may be the segmentation result of the segmentation unit 103, the segmentation result after feature domain enhancement and / or the segmentation result after image domain enhancement.

[0140] Image morphological processing includes, for example, erosion, dilation, opening, closing, skeleton extraction, epipolar erosion, hit-hit-miss transformation, Top-hat transformation, particle analysis, watershed transformation, morphological gradient, etc. The most basic morphological operations are dilation and erosion. The main functions of erosion and dilation are to eliminate noise, segment independent image elements, and connect adjacent elements in the image.

[0141] ※Operation of the blood vessel segmentation device 100E

[0142] Next, a blood vessel segmentation method applied by the blood vessel segmentation device 100E according to the sixth embodiment of the present invention will be described. Fig.19 This is an operation flow chart of the blood vessel segmentation device 100E according to the sixth embodiment of the present invention.

[0143] like Fig.19As shown in FIG. 1 , after the blood vessel segmentation device 100E starts processing, it executes the processing of steps S100 to S300 and then proceeds to step S101. Here, the processing of steps S100 to S300 is the same as that of Fig.17 The processing of steps 100 to S300 is the same, and the detailed description is omitted here.

[0144] In step S400, the morphological processing unit 107 performs morphological image processing on the segmentation result to obtain a final segmentation result, and then ends the processing.

[0145] The blood vessel segmentation device of this embodiment has the technical effect of the fifth embodiment. In addition, since the morphological processing unit 107 is provided, small-area free blood vessels caused by noise or the like in the segmentation result can be further removed, thereby improving the accuracy of blood vessel segmentation.

[0146] The first to sixth embodiments of the present invention have been described above. The above-described embodiments may be omitted, replaced, combined, or modified within the scope of the gist of the invention.

[0147] For example, the feature domain enhancement unit 104 and the image domain enhancement unit 105 a according to the fourth embodiment may be applied to the sixth embodiment to form the seventh embodiment.

[0148] The same specific example of hepatic vascular segmentation was applied to the conventional deep learning-based technology and the fifth embodiment, and the segmentation results are shown in Table 1.

[0149] (Table 1)

[0150]

[0151] According to Table 1, the hepatic vascular segmentation based on deep learning using an integrated loss function according to the present invention has a higher centerline matching degree for both the hepatic vein and the portal vein than the prior art. The centerline matching degree is an indicator used to characterize the accuracy of vascular segmentation. The higher the centerline matching degree, the higher the surface vascular segmentation accuracy.

[0152] In addition, the same specific example of hepatic blood vessel segmentation is applied to the fifth embodiment and the seventh embodiment, and the segmentation results are shown in Table 2.

[0153] (Table 2)

[0154]

[0155] According to Table 2, when the blood vessel segmentation device of the fifth embodiment of the present invention is used to segment liver blood vessels, the segmentation results of the hepatic vein and portal vein both have disconnected regions outside the main blood vessel tree. Here, the disconnected region outside the main blood vessel tree refers to an area that is actually connected but becomes disconnected in the segmentation result.

[0156] In addition, when the blood vessel segmentation device of the seventh embodiment of the present invention having the feature domain enhancement unit 104 and the image domain enhancement unit 105a is used to segment the hepatic blood vessels, the segmentation results of the hepatic vein and the portal vein do not have disconnected domains outside the main blood vessel tree. In other words, the feature domain enhancement unit 104 and the image domain enhancement unit 105a can achieve high-precision blood vessel segmentation even when the contrast between the blood vessels and tissues in the medical image is not high, and improve the connectivity of the blood vessels in the segmentation results.

[0157] In addition, the same specific example of hepatic vascular segmentation is applied to the fifth embodiment and the seventh embodiment, and the statistics of hepatic vein vascular branches and portal vein vascular branches involved in the segmentation results are shown in Table III.

[0158] (Table 3)

[0159]

[0160] According to Table 3, compared with the case where the blood vessel segmentation device of the fifth embodiment of the present invention is used to segment liver blood vessels, when the blood vessel segmentation device of the seventh embodiment of the present invention having the feature domain enhancement unit 104 and the image domain enhancement unit 105a is used to segment liver blood vessels, the detection ratios of the predicted labeled blood vessels related to the hepatic vein and the portal vein relative to the GT are 95% and 97% respectively.

[0161] It can be seen that, through the feature domain enhancement unit 104 and the image domain enhancement unit 105a, blood vessel segmentation can be performed with high accuracy even when the contrast between blood vessels and tissues in the medical image is not high, and the connectivity of blood vessels in the segmentation results can be improved.

[0162] In addition, according to the present invention, when training and segmenting based on a deep learning model, it is possible to predict both blood vessels that have been annotated in GT and blood vessels that have not been annotated in GT. This is because it is actually difficult to achieve complete blood vessel annotating without omission.

[0163] (Variation Example)

[0164] The above embodiments are described, but the present invention is not limited to the above embodiments. For example, those skilled in the art may appropriately add, delete, or change the design of the structural elements of the above embodiments, or appropriately combine the features of the embodiments, which are all within the scope of the present invention as long as they conform to the technical concept of the present invention.

[0165] In addition, although the liver blood vessel segmentation is mainly described as an example, the technical solution of the present invention can be applied to the blood vessel segmentation of various organs. In other words, as long as the contrast between blood vessels and tissues in medical images is not high, the technical solution of the present invention can be effectively applied.

[0166] In addition, the numerical range involved in the present invention is only an example and can be appropriately set or changed according to actual needs.

[0167] In addition, the present invention describes in detail the specific forms of the vascular segmentation device and the vascular segmentation method it performs, but the specific forms of the present invention are not limited to this, and can also be expressed in various forms including a medical image processing device, an integrated circuit, a program, and a medium recording the program.

[0168] For example, the medical image processing apparatus of the present invention includes: a target image acquisition unit that acquires a plurality of target medical images; any one of the above-mentioned blood vessel segmentation devices; and an output unit that outputs the segmentation result.

Claims

1. A blood vessel segmentation device, comprising: An image acquisition unit, for acquiring a plurality of medical images; A deep learning model acquisition unit acquires a trained deep learning model obtained by performing deep learning model training based on the medical image using a centerline enhancement loss function and a shape statistics constraint loss function; and The segmentation unit segments the acquired target medical image based on the trained deep learning model, segments the blood vessels and obtains a segmentation result.

2. The blood vessel segmentation device according to claim 1, wherein: It also includes a feature domain enhancement unit, which applies tubular filtering to extract tubular features and obtains a feature domain of enhanced tubular features for the segmentation result, and applies regional growing to connect the tubular features to obtain a segmentation result after feature domain enhancement.

3. The blood vessel segmentation device according to claim 1, wherein: The invention also comprises an image domain enhancement unit, which performs connectivity restoration processing on the free blood vessel segments in the segmentation result in the image domain to obtain the segmentation result after image domain enhancement.

4. The blood vessel segmentation device according to claim 2, wherein: The invention also comprises an image domain enhancement unit, which performs connectivity restoration processing on the free blood vessel segments in the segmentation result in the image domain to obtain the segmentation result after image domain enhancement.

5. The blood vessel segmentation device according to any one of claims 1 to 4, wherein: It also includes a morphological processing unit, which performs morphological image processing on the segmentation result, the segmentation result after feature domain enhancement and / or the segmentation result after image domain enhancement to obtain a final segmentation result.

6. The blood vessel segmentation device according to claim 1, wherein: It also includes a preprocessing unit, which preprocesses the acquired multiple medical images to obtain standardized medical images; The deep learning model acquisition unit performs deep learning model training based on the standardized medical image using a centerline enhancement loss function and a shape statistics constraint loss function to obtain a trained deep learning model.

7. The blood vessel segmentation device according to claim 1, wherein: During deep learning model training, the deep learning model acquisition unit uses an integrated loss function obtained by weighted summing of a Dice coefficient loss function, a cross entropy loss function, a centerline enhancement loss function, and a shape statistics constraint loss function.

8. The blood vessel segmentation device according to claim 2, wherein: The feature domain enhancement unit applies a Hessian matrix to extract tubular features based on the segmentation result.

9. A blood vessel segmentation method in a blood vessel segmentation device, comprising: An image acquisition step, acquiring a plurality of medical images; A training step, based on the medical image, performing deep learning model training using a centerline enhancement loss function and a shape statistics constraint loss function to obtain a trained deep learning model; as well as The segmentation step is to segment the acquired object medical image based on the trained deep learning model, segment the blood vessels and obtain the segmentation result.

10. A medical image processing device comprising: An object image acquisition unit, which acquires a plurality of object medical images; The blood vessel segmentation device according to any one of claims 1 to 8; and The output unit outputs the segmentation result.

Images