Artificial intelligence model for classifying liver lesion by using liver ultrasound image, method for constructing model, and method for classifying liver lesion by using artificial intelligence model

Abstract

The present invention relates to classification of liver lesions by a liver ultrasound image and, more specifically, to an artificial intelligence model for classifying liver lesions by using a liver ultrasound image, a method for constructing the model, and a method for classifying liver lesions by using the artificial intelligence model. To this end, the present invention provides a method for constructing an artificial intelligence model for classifying liver lesions by using a liver ultrasound image, the method comprising: a preparation step of excluding an image corresponding to a predetermined exclusion criterion, among a plurality of liver ultrasound images (S100); a step, executed by a computer, of receiving by the computer, the plurality of prepared liver ultrasound images as original images (S120); a step of cross-allocating the original images to a plurality of folds according to liver lesions (S140); a step of image-processing each original image for training (S160); and a step of training an artificial intelligence model by using the image-processed image (S180).

Description

An artificial intelligence model for classifying liver lesions using liver ultrasound images, a method for constructing the model, and a method for classifying liver lesions using the artificial intelligence model

[0001] The present invention relates to classification of liver lesions using liver ultrasound images, and more particularly, to an artificial intelligence model for classifying liver lesions using liver ultrasound images, a method for constructing the model, and a method for classifying liver lesions using the artificial intelligence model.

[0002] Liver ultrasound is commonly used to diagnose liver disease or lesions. Liver ultrasound images are analyzed to classify hepatocellular carcinoma (HCC), hemangiomas, and cysts.

[0003] Traditionally, these image analysis tasks were performed by people (e.g., radiologists), but with the recent development of artificial intelligence, cases where assistance is provided by artificial intelligence models are gradually increasing.

[0004] However, segmenting and detecting the boundaries of liver lesions from full-length ultrasound images is challenging, and detecting their locations is also challenging. Furthermore, to accurately infer and classify the specific liver lesions, it is essential to carefully organize the original dataset and design a detailed artificial intelligence model.

[0005] Accordingly, the present invention has been devised to solve the above problems, and the problem to be solved by the present invention is to provide an artificial intelligence model for classifying liver lesions using liver ultrasound images, a method for constructing the model, and a method for classifying liver lesions using the artificial intelligence model.

[0006] However, the technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0007] In order to achieve the above technical task, a method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images is provided, characterized in that it includes a preparatory step (S100) of excluding images corresponding to predetermined exclusion criteria from among a plurality of liver ultrasound images; a step (S120) executed by a computer, wherein the computer receives the prepared plurality of liver ultrasound images as original images; a step (S140) of cross-assigning the original images to a plurality of folds according to liver lesions; a step (S160) of image-processing each original image for training; and a step (S180) of training an artificial intelligence model using the image-processed images.

[0008] Additionally, the exclusion criteria were liver lesions with a maximum size of 10 cm or more or a minimum size of 1 cm or less on the liver ultrasound image.

[0009] Additionally, the liver lesions are at least two of the following: hepatocellular carcinoma (HCC), hemangioma, or cyst.

[0010] Additionally, in the allocation step (S140), each fold is distinguished by the liver lesion, and the original image is allocated by the liver lesion for each fold, but all liver lesions of the same subject are allocated to the same fold.

[0011] Additionally, there are five multiple folds, and liver lesions are classified into hepatocellular carcinoma (HCC), hemangioma, and cyst.

[0012] In addition, the image processing step (S160) includes a step of applying a minimum / maximum curvature flow filter to the original image to reduce noise (S162); a step of performing adaptive histogram equalization on the original image to enhance the image (S164); a step of cropping and deleting a portion of the original image that is not related to the image (S166); and a step of zero-padding the original image into a square and resizing it to a predetermined size (S168).

[0013] The above-described object of the present invention can be achieved by a computer-readable recording medium that is readable by a computer and records an artificial intelligence model constructed using the aforementioned construction method.

[0014] The purpose of the present invention as described above is another embodiment, comprising: a preparatory step (S100) of excluding an image corresponding to a predetermined exclusion criterion from among a plurality of liver ultrasound images; a step (S120) executed by a computer, wherein the computer receives the prepared plurality of liver ultrasound images as original images; a step (S140) of cross-assigning the original images to a plurality of folds according to liver lesions; a step (S160) of image-processing each original image for training; a step (S180) of training an artificial intelligence model using the image-processed image; a step (S200) of calculating an IoU value between a bounding box corresponding to a liver lesion among actual labeled images and a bounding box of a liver lesion inferred by an artificial intelligence model for the labeled images; a step (S220) of determining whether the IoU value is greater than or equal to a threshold value; a step (S240) of cropping and resizing the bounding box inferred by the artificial intelligence model into an image of a predetermined size if the IoU value is greater than or equal to the threshold value; And it can be achieved by a method for classifying liver lesions using an artificial intelligence model, characterized in that it includes a step (S260) of classifying the cropped image as one of a plurality of liver lesions by an artificial intelligence model.

[0015] In addition, the IoU value calculation step (S200) includes a step (S202) of segmenting the label image to generate a prediction mask representing a liver lesion; and a detection step (S204) of drawing a bounding box including the prediction mask.

[0016] Also, the threshold is 0.2.

[0017] Additionally, the cropping and resizing step (S240) includes a step of converting the bounding box into a square (S242); a step of cropping the square bounding box (S246); and a step of resizing the cropped bounding box to a predetermined size (S248).

[0018] According to one embodiment of the present invention, an artificial intelligence model quickly and accurately classifies the type of liver lesion from liver ultrasound images. Therefore, this model can aid in the analysis of liver ultrasound images and prevent misdiagnosis or underdiagnosis.

[0019] However, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0020] The following drawings attached to this specification illustrate preferred embodiments of the present invention, and together with the detailed description of the invention described below, serve to further understand the technical idea of ​​the present invention, and therefore, the present invention should not be interpreted as being limited to matters described in such drawings.

[0021] Figure 1 is an explanatory diagram showing the process of processing an ultrasound image of the liver according to the present invention.

[0022] Figure 2a is a diagram schematically showing a step in which an artificial intelligence model according to the present invention generates a predicted mask from an actual label image.

[0023] Figure 2b is a state diagram of drawing a bounding box on the prediction mask generated in Figure 2a.

[0024] Figure 2c is a state diagram showing the process of classifying liver lesions using the image of the bounding box drawn in Figure 2b.

[0025] Figure 3a is an example of an image in which the predicted mask is inferred as a true positive prediction when the IoU (Intersection Over Union) threshold between the bounding box of the actual label image and the bounding box inferred by the artificial intelligence model is 0.2 or higher.

[0026] Figure 3b is an example of an image in which the predicted mask is inferred as a miss-detection when the IoU threshold between the bounding box of the actual label image and the bounding box inferred by the artificial intelligence model is less than 0.2.

[0027] Figure 4a is an example of an image showing a bounding box predicted as a liver lesion among liver ultrasound images.

[0028] Figure 4b is an example of an image in which the bounding box predicted in Figure 4a is converted into a square.

[0029] Figure 4c is an example of a cropped image of the square bounding box converted in Figure 4b.

[0030] Figure 5a is an example showing the segmentation and location detection between the actual label image and the inferred image of the artificial intelligence model when the IoU of the bounding box is 0.121, 0.261, and 0.397, respectively.

[0031] Figure 5b is an example showing the segmentation and location detection between the actual label image and the inferred image of the artificial intelligence model when the IoU of the bounding box is 0.487, 0.577, and 0.624, respectively.

[0032] Figure 5c is an example showing the segmentation and location detection between the actual label image and the inferred image of the artificial intelligence model when the IoU of the bounding box is 0.790, 0.834, and 0.964, respectively.

[0033] Figures 6a to 6e are examples of confusion matrices showing classification results for folds 1 to 5.

[0034] Figures 7a to 7e are examples of graphs showing ROC (Receiver Operating Characteristic) curves for folds 1 to 5.

[0035] Figure 8 is a confusion matrix of classification results that synthesizes the confusion matrices shown in Figures 6a to 6e.

[0036] Figure 9 is a graph of an ROC curve that synthesizes the ROC curves shown in Figures 7a to 7e.

[0037] Figures 10a to 10i are images comparing liver lesions of actual labels with liver lesions classified by the artificial intelligence model of the present invention.

[0038] Figure 11a is a confusion matrix representing the mutual classification between HCC and hemangioma among liver lesions.

[0039] Figure 11b is a confusion matrix representing the mutual classification between HCC and cyst among liver lesions.

[0040] Figure 11c is a confusion matrix representing the mutual classification between hemangiomas and cysts among liver lesions.

[0041] FIG. 12 is a flowchart schematically illustrating a method for constructing and classifying an artificial intelligence model for classifying liver lesions using liver ultrasound images according to one embodiment of the present invention.

[0042] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily practice the present invention. However, the description of the present invention is merely an embodiment for structural and functional explanation, and therefore the scope of the present invention should not be construed as being limited by the embodiments described in the text. In other words, since the embodiments can be modified in various ways and can have various forms, the scope of the present invention should be understood to include equivalents that can realize the technical idea. In addition, the purposes or effects presented in the present invention do not mean that a specific embodiment must include all of them or only such effects, and therefore the scope of the present invention should not be construed as being limited thereby.

[0043] The meanings of terms described in the present invention should be understood as follows.

[0044] Terms such as "first" and "second" are intended to distinguish one component from another, and the scope of the rights should not be limited by these terms. For example, a first component could be referred to as a second component, and similarly, a second component could also be referred to as a first component. When a component is referred to as being "connected" to another component, it should be understood that it may be directly connected to that other component, but there may also be other components in between. Conversely, when a component is referred to as being "directly connected" to another component, it should be understood that there are no other components in between. Meanwhile, other expressions describing the relationship between components, such as "between" and "immediately between" or "adjacent to" and "directly adjacent to", should be interpreted similarly.

[0045] Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "comprises" or "has" should be understood to specify the presence of stated features, numbers, steps, operations, components, parts, or combinations thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0046] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted to be consistent with their meaning within the context of the relevant technology, and should not be interpreted as having ideal or overly formal meanings unless explicitly defined herein.

[0047] Hereinafter, a preferred embodiment will be described in detail with reference to the attached drawings. Fig. 12 is a flowchart schematically showing a method for constructing and classifying an artificial intelligence model for classifying liver lesions using liver ultrasound images according to one embodiment of the present invention. As illustrated in Fig. 12, first, as a preparatory step, images that meet a predetermined exclusion criterion are excluded from a plurality of liver ultrasound images (S100). Liver ultrasound images are black and white gray images that include liver lesions, and only the images remaining after being excluded according to the exclusion criterion are used. The exclusion criterion is that the maximum size of the liver lesion in the liver ultrasound image is 10 cm or more, or the minimum size is 1 cm or less. This is because if the liver lesion is too large, it is difficult to segment it, and if it is too small, it may result in incorrect inference or take a long time to learn.

[0048] Based on the exclusion criteria, the images excluded and used for training included 3,714 images of hepatocellular carcinoma (HCC) from 1,258 patients and 3,739 liver lesions. Hemangioma included 2,018 images of hepatoma from 834 patients and 2,044 liver lesions. Cysts included 980 images of hepatoma from 480 patients and 1,039 liver lesions. These images are categorized by liver lesion type, and depending on the patient, multiple liver lesions or multiple images are included. This preparatory step can be performed in batches on a computer or manually by an operator using liver lesion size data output from the ultrasound equipment.

[0049] Next, the prepared multiple liver ultrasound images are input into a computer as raw images and stored (S120). For this purpose, the computer typically includes a CPU, RAM, hard disk, input ports (e.g., USB ports), etc., and can be equipped with an operating system (e.g., Windows, Linux) and an artificial intelligence program (e.g., a Python package program including PyTorch) that can be installed and executed.

[0050] Next, the original image is divided into five parts and cross-assigned to five folds for each liver lesion, as shown in [Table 1] (S140). In [Table 1], Training represents the images used for learning the AI ​​model (corresponding to 3 / 5 of the total images), Validation represents the images used to verify the completion of AI model learning (corresponding to 1 / 5 of the total images), and Test represents the images used to verify the accuracy of the AI ​​model's inference (corresponding to 1 / 5 of the total images). Therefore, five cross-validation (CV) images are defined for the five folds.

[0051] Fold 1 Fold 2 Fold 3 Fold 4 Fold 5 CV 1 Training Training Training Verification Test CV 2 Training Training Verification Test Training CV 3 Training Verification Test Training Training CV 4 Verification Test Training Training Training CV 5 Test Training Training Training Verification

[0052] As shown in [Table 2], each fold is classified by liver lesion, and the original image is assigned to each liver lesion for each fold, but all liver lesions of the same subject are assigned to the same fold. Here, liver lesions are classified as hepatocellular carcinoma (HCC), hemangioma, and cyst.

[0053] Fold Lesion Image Target Fold 1 Total 1,1231,104,520 Hepatocellular carcinoma (HCC) 563,556,252 Hemangioma 370,367,169 Cyst 190,18199 Fold 2 Total 1,2941,275,520 Hepatocellular carcinoma (HCC) 660,654,252 Hemangioma 411,411,169 Cyst 223,21099 Fold 3 Total 1,5031,479,520 Hepatocellular carcinoma (HCC) 904,901,252 Hemangioma 395,391,169 Cyst 204,18799 Fold 4 Total 1,4341,417,520 Hepatocellular carcinoma (HCC) 8,168,152,520 Hemangioma 4,484,381,690 Cyst 1,701,649,9 Fold 5 Total 1,4681,437,520 Hepatocellular carcinoma (HCC) 7,967,882,540 Hemangioma 4,204,111,700 Cyst 2,522,389,9 Fold

[0054] As shown in [Table 2], each fold is assigned to 520 subjects (patients), and it can be seen that the training set, validation set, and test set are independent of each other. Then, the computer performs image processing on each original image for training (S160). Fig. 1 is an explanatory diagram showing the process of image processing a liver ultrasound image according to the present invention. As shown in Fig. 1, the image processing step (S160) applies a minimum maximum curvature flow filter to the original image to reduce noise (S162). Then, adaptive histogram equalization is performed on the original image to enhance the image (S164). Then, a portion of the original image that is not related to the image (e.g., a text portion) is cropped and deleted (S166). Next, the original image is zero-padded into a square and resized to a predetermined size (e.g., 512 x 512) (S168). Zero-padding is a method of filling all bytes to be padded with 0. For the image processing step (S160), a general image processing program can be installed and executed on the computer. Then, an artificial intelligence model is trained using the image processed as described above (S180). The artificial intelligence program used was a commercial "Python package program with Pytorch" and the Deep lab V3-Resnet 101 model or Efficientnet-b2 model as the AI ​​system architecture.

[0055] For training the AI ​​model, the initial learning rate was set to 0.001, and the batch size was 4. The loss function for training was a combination of dice loss and binary cross entropy (BCE) loss. Stochastic gradient descent (SGD) with a momentum of 0.9 and a weight decay of 0.0001 was used to optimize the loss function. SGD is an algorithm that calculates the predicted gradient at each step using a single example randomly selected from the entire data set.

[0056] Additionally, a rectangular bounding box was created to indicate the location of the liver lesion during the training process, and on-the-fly data augmentation strategies including rotation, translation, flip, scaling, and color jitter were implemented.

[0057] As a result of training, when the loss on the validation data did not decrease for 8 epochs, the learning rate was reduced to 0.5, and when the loss on the validation data did not decrease for 20 epochs, the training process was terminated early. The learning rate is a measure of the learning process of the AI ​​model, meaning that the AI ​​model gradually approaches the optimal solution during the optimization process. After the training process was completed, 0.1 cm was used in the post-processing process. 2 Smaller objects were removed, and a 7 x 7 median filter was run on the predicted mask. This training process completes the AI ​​model.

[0058] Next, as a testing process, the computer calculates the IoU value between the bounding box corresponding to the liver lesion among the actual labeled images and the bounding box of the liver lesion inferred by the artificial intelligence model for the labeled images (S200).

[0059] First, the process of generating a bounding box is as follows. Fig. 2a is a diagram schematically illustrating a step of generating a predicted mask from an actual label image by an artificial intelligence model according to the present invention, and Fig. 2b is a diagram illustrating a state of drawing a bounding box on the predicted mask generated in Fig. 2a. As shown in Figs. 2a and 2b, 101 layer operations are performed on a liver lesion image, and the first layer is formed into 64 channels through a convolution operation on a 7x7 matrix. Next, Atrous spatial pyramid pooling (ASPP) is performed. This constructs Atrous pooling in a pyramid shape. The output of this Atrous spatial pyramid pooling network is up-sampled through 1x1 convolution to generate a predicted mask on the liver lesion image (S202).

[0060] As illustrated in Figure 2b, a minimum square containing the prediction mask is defined and generated as a bounding box (S204). This bounding box is derived from the bounding box corresponding to the liver lesion in the actual labeled image and the bounding box of the liver lesion inferred by the AI ​​model for the labeled image.

[0061] It is determined whether the IoU value calculated in step S200 is 0.2 or greater (S220). Fig. 3a is an example of an image in which the prediction mask is inferred as a true positive prediction when the IoU (Intersection Over Union) threshold between the bounding box of the actual label image and the bounding box inferred by the artificial intelligence model is 0.2 or greater, and Fig. 3b is an example of an image in which the prediction mask is inferred as a miss-detection when the IoU threshold between the bounding box of the actual label image and the bounding box inferred by the artificial intelligence model is less than 0.2. Therefore, high-sensitivity detection of liver lesions can be achieved with a lower IoU threshold (e.g., 0.1, 0.15, etc.).

[0062] Fig. 2c is a state diagram showing the process of classifying liver lesions using the image of the bounding box drawn in Fig. 2b when it is determined that it is a true positive prediction. As shown in Fig. 2c, first, the bounding box is converted into a square (S242), the square bounding box is cropped (S246), and then the cropped square bounding box is resized to a size of 96 x 96 (S248). As an example, Fig. 4a is an image showing a bounding box predicted as a liver lesion among liver ultrasound images, Fig. 4b is an image obtained by converting the predicted bounding box in Fig. 4a into a square, and Fig. 4c is an image obtained by cropping the square bounding box converted in Fig. 4b. In step S242, the rectangular bounding box is converted into a square based on the length of the long side.

[0063] Next, the process of classifying the resized image into one of Hepatocellular Carcinoma (HCC), Hemangioma, and Cyst is as follows. As shown in Fig. 2c, first, a convolutional neural network consisting of 23 mobile inverted bottle neck convolutions (MBConv) extracts eight features. MBConv extracts image features for liver lesion classification through depthwise convolution, which separates the feature map by channel and performs operations to extract features with fewer parameters than the existing convolution operation, pointwise convolution, which merges these back into one channel through a 1 x 1 convolution operation, a squeeze network that converts each channel's feature map into a single feature value, and an excitation network that calculates the importance of the corresponding feature values. Simultaneously with feature extraction through this convolutional neural network, feature extraction using the radiomics method is performed. In the radiomics method, first-order features representing the pixel intensity distribution of the image and texture features representing the signal relationship between adjacent pixels are extracted.Among these, the texture features are applied to the GLCM (Gray Level Co-occurrence Matrix), which calculates the frequency with which pairs of pixels with a specific value occur in a specific space in the image; the GLRLM (Gray Level Run Length Matrix), which calculates the length of consecutive pixels with the same intensity in the image; the GLSZM (Gray Level Size Zone Matrix), which calculates the size of adjacent pixels with the same intensity in the image; the NGTDM (Neighboring Tone Difference Matrix), which calculates the difference between the intensity of a specific pixel and the average intensity of adjacent pixels in the image; and the GLDM (Gray Level Dependence Matrix), which calculates the occurrence frequency of pixel intensities between pixels that are a specific distance apart in the image. Next, in order to prevent overfitting in the classification model, feature selection is performed through Lasso regression, which constrains the sum of the absolute values ​​of the weights to be 0. Therefore, important features are selected, as shown in [Table 3]. The selected features are input into a multilayer perceptron with a layer structure of 32-16-16-8, converted into 8 features again, and combined with 8 features extracted using a convolution method, a probability value for classifying into one of HCC, Hemangioma, and Cyst is derived (S260).

[0064] Feature DefinitionFirst orderInterquartile rangeThe difference between the 25th and 75th percentiles of image intensity valuesGLCMCluster shadeSkewness and uniformity of GLCMGLDMDependence non-uniformity normalizedDependency similarity of an imageLarge dependence low gray level emphasisJoint distribution of low intensity and large dependenceNGTDMCoarsenessThe average difference between the center pixel and its neighborsStrengthThe difference in the rate of change and coarseness of image pixel intensities

[0065] As shown in FIGS. 3A to 3C and [Table 4], the artificial intelligence model of the present invention classified 6,077 cropped lesions and 6,038 images from 2,444 patients into three types of liver lesions.

[0066] Fold Lesion (Crop) Image Target Fold 1 Total 1,2111,191486 Hepatocellular carcinoma (HCC) 655655233 Hemangioma 368368160 Cyst 18816893 Fold 2 Total 1,2391,233486 Hepatocellular carcinoma (HCC) 656656233 Hemangioma 391389160 Cyst 19218893 Fold 3 Total 1,1801,178486 Hepatocellular carcinoma (HCC) 645645233 Hemangioma 354354160 Cyst 18117993 Fold 4 Total 1,2391,229,486 Hepatocellular carcinoma (HCC) 658,681,233 Hemangioma 374,370,160 Cyst 180,178,93 Fold 5 Total 1,2081,207,500 Hepatocellular carcinoma (HCC) 640,640,232 Hemangioma 370,369,167 Cyst 198,198,101

[0067]

[0068] Comparative experiment

[0069] Below are the results of various comparative experiments of the artificial intelligence models constructed as described above. First, Fig. 5a is an example that contrastively shows the segmentation and location detection between the actual label image and the inferred image of the artificial intelligence model when the IoU of the bounding box is 0.121, 0.261, and 0.397, respectively; Fig. 5b is an example that contrastively shows the segmentation and location detection between the actual label image and the inferred image of the artificial intelligence model when the IoU of the bounding box is 0.487, 0.577, and 0.624, respectively; and Fig. 5c is an example that contrastively shows the segmentation and location detection between the actual label image and the inferred image of the artificial intelligence model when the IoU of the bounding box is 0.790, 0.834, and 0.964, respectively. In Figs. 5a to 5c, the first column is the original image, the red color in the second column indicates a lesion in the label image, and the blue color indicates the predicted mask inferred by the artificial intelligence model. In the third column, red is the bounding box containing the lesion in the label image, and blue is the bounding box containing the predicted mask inferred by the artificial intelligence model.

[0070] In Figure 5a, when the IoU is 0.1211, the larger blue bounding box only encompasses half of the red bounding box. This indicates that the AI ​​model accurately infers the location of the liver lesion. However, when the IoU is 0.2 or higher, the locations of the red and blue bounding boxes are found to be similarly close. In particular, as the IoU value approaches 1, the locations of the bounding boxes become very close.

[0071] Figures 6a to 6e are examples of confusion matrices representing classification results for folds 1 to 5. As illustrated in Figure 6a, for fold 1, the AI ​​model correctly classified HCC in the labeled image as HCC in 461 cases, which is a very high number, while incorrectly classified it in 151 + 28 = 179 cases. It can be confirmed that similar classification results are shown across folds 1 to 5.

[0072] Figures 7a to 7e are graphs showing Receiver Operating Characteristic (ROC) curves for folds 1 to 5. It can be confirmed that similar classification results are shown across folds 1 to 5.

[0073] And, Fig. 8 is a confusion matrix of the classification result that synthesizes the confusion matrices shown in Figs. 6a to 6e, and Fig. 9 is a graph of an ROC curve that synthesizes the ROC curves shown in Figs. 7a to 7e. As shown in Fig. 8, the artificial intelligence model classified cyst lesions the best, and the classification accuracy of HCC and Hemangioma was relatively low. In Fig. 9, the ROC curves of HCC and Hemangioma showed similar shapes (AUC of HCC = 0.865, AUC of Hemangioma = 0.850), and the AUC of Cyst was high at 0.990.

[0074] [Table 5] is a graph showing the segmentation performance (boundary detection) of liver lesions, and the average value is indicated with the standard deviation in parentheses.

[0075] Type segmentation performance DSC(%) IoU(%) Total Interleukin 0.728(0.272)0.629(0.269)CV 10.706(0.290)0.608(0.283)CV 20.716(0.278)0.616(0.274)CV 30.740(0.259)0.640(0.258)CV 40.720(0.280)0.622(0.276)CV 50.755(0.246)0.655(0.249)

[0076] DSC (Dice Similarity coefficient) / IoU (Intersection Over union).

[0077] Here, the dice similarity coefficient (DSC) is twice the overlap area between the labeled and predicted masks divided by the total area of ​​the labeled and predicted masks. The IoU score is the overlap area between the labeled and predicted bounding boxes divided by the union of the label and predicted boxes.

[0078] As shown in [Table 5], CV1 through CV5 demonstrate consistent DSC and IoU performance, demonstrating 72.8% DSC performance and 62.9% cross-correlation performance. Considering the difficulty in clearly defining the boundaries of liver disease, the segmentation performance of the AI ​​model is exceptionally high.

[0079] [Table 6] is a graph showing the detection performance (location detection) of liver lesions, and the parentheses indicate a 95% confidence interval.

[0080] Type detection performance Detection rate (%)FPPI (%) Total liver lesions 0.891 (0.879-0.903) 0.223 (0.212-0.234) CV 10.868 (0.839-0.897) 0.276 (0.250-0.302) CV 20.878 (0.848-0.908) 0.236 (0.208-0.264) CV 30.903 (0.878-0.928) 0.205 (0.180-0.230) CV 40.883 (0.857-0.909) 0.216 (0.192-0.240) CV 50.921(0.899-0.943)0.183(0.160-0.206)

[0081] FPPI: False Positive Per Image

[0082] If the segmentation mask is not output for the area containing the liver lesion, the number of true and false positives (false positives) is counted, and the false positives per image (FPPI) is the number of false positives divided by the total number of images.

[0083] In terms of detection performance, the AI ​​model of the present invention exhibited a detection rate of 0.891 (95% CI: 0.879–0.903) and an FPPI of 0.223 (95% CI: 0.212–0.234) across all folds. The AI ​​model exhibited a detection rate of 0.868–0.921 and an FPPI of 0.183–0.276 in each cross-validation fold.

[0084] [Table 7] shows the overall classification performance of the artificial intelligence model in liver ultrasound images.

[0085] Classification Performance Type Recall (%) Precision (%) F1-score AUC Total HCC Emangioma Cyst 0.753 (0.738-0.768) 0.691 (0.670-0.712) 0.930 (0.911-0.945) 0.817 (0.803-0.831) 0.633 (0.612-0.654) 0.851 (0.828-0.872) 0.784 (0.774-0.794) 0.661 (0.646-0.676) 0.889 (0.874-0.902) 0.865 (0.856-0.874) 0.850 (0.838-0.861) 0.990 (0.985-0.994)CV 1HCCHMangiomaCyst0.720 (0.684-0.754)0.735 (0.688-0.779)0.879 (0.826-0.919)0.812 (0.777-0.842)0.631 (0.584-0.676)0.833 (0.776-0.879)0.763 (0.738-0.786)0.679 (0.646-0.711)0.855 (0.817-0.887)0.860 (0.839-0.880)0.861 (0.835-0.886)0.987 (0.975-0.998)CV 2HCCHMangiomaCyst0.745 (0.710-0.777)0.701 (0.652-0.746)0.973 (0.940-0.989)0.831 (0.798-0.860)0.637 (0.589-0.683)0.836 (0.779-0.880)0.786 (0.762-0.808)0.668 (0.633-0.700)0.899 (0.866-0.925)0.881 (0.862-0.900)0.861 (0.836-0.887)0.992 (0.983-1.000)CV 3HCCHMangiomaCyst0.787 (0.753-0.816)0.665 (0.616-0.711)0.927 (0.881-0.957)0.805 (0.772-0.834)0.658 (0.610-0.704)0.877 (0.823-0.917)0.796 (0.773-0.817)0.662 (0.627-0.694)0.901 (0.867-0.928)0.863 (0.842-0.883)0.840 (0.813-0.866)0.991 (0.981-1.000)CV 4HCCHMangiomaCyst0.758 (0.722-0.789)0.706 (0.656-0.752)0.961 (0.921-0.982)0.842 (0.809-0.869)0.635 (0.585-0.682)0.849 (0.793-0.893)0.789 (0.774-0.819)0.668 (0.634-0.702)0.902 (0.867-0.928)0.874 (0.854-0.894)0.850 (0.823-0.876)0.993 (0.985-1.000)CV 5HCCHMangiomaCyst0.755 (0.721-0.786)0.652 (0.602-0.699)0.911 (0.859-0,945)0.800 (0.767-0.830)0.605 (0.556-0.653)0.863 (0.806-0.906)0.777 (0.754-0.799)0.628 (0.593-0.662)0.886 (0.850-0.915)0.854 (0.833-0.875)0.840 (0.813-0.867)0.990 (0.979-1.000).

[0086] AUC: Area Under the receiver operating Characteristic Curve

[0087] In [Table 7], recall is the number of true positive lesions divided by the total number of positive lesions, and plays an important role in assisting clinicians in making decisions.

[0088] Precision is defined as the number of true positive lesions divided by the total number of lesions predicted as positive (the sum of the numbers of true positive and false positive lesions).

[0089] The F1-score is defined as the harmonic mean of recall and precision. The area under the receiver operating characteristic (AUC) curve is defined as the area under the receiver operating characteristic (ROC) curve, which represents the recall versus the false positive rate for a decision threshold.

[0090] As shown in [Table 7], for example, the classification of HCC showed a recall of 0.753 (95% CI: 0.738-0.768), a precision of 0.817 (95% CI: 0.803-0.831), an F1-score of 0.784 (95% CI: 0.774-0.794), and an AUC score of 0.865 (95% CI: 0.856-0.874).

[0091] Figures 10a to 10i are images comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. In Figure 10a, a cyst in the labeled image was correctly classified as a cyst, but in Figure 10b, a cyst in the labeled image was incorrectly classified as a hemangioma.

[0092] Figure 11a is a confusion matrix showing the mutual classification between HCC and hemangioma among liver lesions, and [Table 8] is the classification performance showing the mutual classification between HCC and hemangioma.

[0093] Classification Performance Type Recall (%) Precision (%) F1-score AUC Total HCC emangioma 0.724 (0.708-0.739) 0.749 (0.729-0.768) 0.836 (0.822-0.849) 0.606 (0.585-0.625) 0.776 (0.765-0.786) 0.670 (0.655-0.684) 0.807 (0.794-0.820)

[0094] As shown in Figure 11a, in the labeled images, HCC was correctly classified as HCC in 2,375 cases, while it was incorrectly classified as a hemangioma in 906 cases. Furthermore, in the labeled images, hemangiomas were correctly classified as hemangiomas in 1,391 cases, while it was incorrectly classified as a HCC in 466 cases. Accordingly, as shown in [Table 8], the recall between HCC and hemangioma ranged from 0.724 to 0.749.

[0095] Figure 11b is a confusion matrix showing the mutual classification between HCC and cyst among liver lesions, and [Table 9] is the classification performance showing the mutual classification between HCC and cyst.

[0096] Classification Performance Type Recall (%) Precision (%) F1-score AUC Total HCC Cyst 0.949 (0.941-0.956) 0.958 (0.944-0.970) 0.988 (0.985-0.991) 0.843 (0.820-0.864) 0.968 (0.963-0.972) 0.897 (0.883-0.910) 0.987 (0.982-0.993)

[0097] As shown in Figure 11b, there were 3,114 cases where HCC in the labeled image was correctly classified as HCC, and 167 cases where it was incorrectly classified as a cyst. In addition, there were 900 cases where cysts in the labeled image were correctly classified as cysts, and 39 cases where they were incorrectly classified as HCC. Accordingly, as shown in [Table 9], it can be seen that the recall between HCC and cysts is between 0.949 and 0.958. This indicates that the classification performance between HCC and cysts is much higher than that between HCC and hemangioma. In other words, it means that the AI ​​model can distinguish between HCC and cysts well.

[0098] Figure 11c is a confusion matrix showing the mutual classification between hemangiomas and cysts among liver lesions, and [Table 10] is the classification performance showing the mutual classification between hemangiomas and cysts.

[0099] Classification Performance Type Recall (%) Precision (%) F1-score AUC Total Hemangioma Cyst 0.954 (0.943-0.963) 0.967 (0.953-0.977) 0.983 (0.976-0.988) 0.913 (0.894-0.930) 0.968 (0.962-0.973) 0.939 (0.928-0.949) 0.991 (0.986-0.995)

[0100] As shown in Figure 11c, there were 1,771 cases where hemangiomas in the labeled images were correctly classified as hemangiomas, and 86 cases where they were incorrectly classified as cysts. Furthermore, there were 908 cases where cysts in the labeled images were correctly classified as cysts, and 31 cases where they were incorrectly classified as hemangiomas. Accordingly, as shown in [Table 10], the recall between hemangiomas and cysts was between 0.954 and 0.967. This indicates that the classification performance between cysts and hemangiomas was much higher than that between HCC and hemangiomas. In other words, the AI ​​model distinguishes between hemangiomas and cysts well.

[0101] The detailed description of the preferred embodiments of the present invention disclosed above has been provided to enable those skilled in the art to implement and practice the present invention. While the above description has been made with reference to preferred embodiments of the present invention, those skilled in the art will appreciate that various modifications and variations can be made to the present invention without departing from the scope of the present invention. For example, those skilled in the art can utilize the individual components described in the above-described embodiments in combination with each other. Accordingly, the present invention is not intended to be limited to the embodiments described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0102] The present invention may be embodied in other specific forms without departing from the spirit and essential characteristics thereof. Therefore, the above detailed description should not be construed in any way as limiting but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes coming within the equivalent scope of the claims are intended to be embraced therein. The present invention is not intended to be limited to the embodiments set forth herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. Furthermore, claims that are not explicitly cited in the claims may be combined to form an embodiment or incorporated into a new claim by post-application amendment.

[0103] The present invention relates to classification of liver lesions using liver ultrasound images, and more particularly, to an artificial intelligence model for classifying liver lesions using liver ultrasound images, a method for constructing the model, and a method for classifying liver lesions using the artificial intelligence model.

Claims

1. A preparatory step (S100) for excluding images that meet certain exclusion criteria from multiple liver ultrasound images; It is executed by a computer, and said computer, Step (S120) of receiving multiple prepared liver ultrasound images as original images; A step of cross-allocating the above original image to multiple folds for each liver lesion (S140); Step (S160) of image processing each of the above original images for training; A method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images, characterized in that it comprises a step (S180) of training an artificial intelligence model using image-processed images.

2. In paragraph 1, The above exclusion criteria are a method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images, characterized in that the maximum size of the liver lesion in the liver ultrasound image is 10 cm or more or the minimum size is 1 cm or less.

3. In paragraph 1, A method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images, wherein the liver lesions are characterized by at least two of hepatocellular carcinoma (HCC), hemangioma, and cyst.

4. In paragraph 1, In the above allocation step (S140), Each of the above folds is distinguished by the above liver lesions, The above original image is assigned to each liver lesion for each fold, A method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images, characterized in that all liver lesions of the same subject are assigned to the same fold.

5. In paragraph 4, The above multiple folds are 5, A method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images, wherein the above liver lesions are classified into hepatocellular carcinoma (HCC), hemangioma, and cyst.

6. In paragraph 1, The above image processing step (S160) is A step of reducing noise by applying a minimum / maximum curvature flow filter to the original image (S162); A step of enhancing the image by performing adaptive histogram equalization on the original image (S164); Step (S166) of cropping and deleting a portion of the original image that is not related to the image; and A method for constructing an artificial intelligence model for classifying liver lesions using a liver ultrasound image, characterized in that it includes a step (S168) of zero-padding the original image into a square and resizing it to a predetermined size.

7. Computer readable, A computer-readable recording medium recording an artificial intelligence model constructed by a construction method according to any one of claims 1 to 6.

8. A preparatory step (S100) for excluding images that meet certain exclusion criteria from multiple liver ultrasound images; It is executed by a computer, and said computer, Step (S120) of receiving multiple prepared liver ultrasound images as original images; A step of cross-allocating the above original image to multiple folds for each liver lesion (S140); Step (S160) of image processing each of the above original images for training; Step of training an artificial intelligence model using image-processed images (S180); A step (S200) of calculating an IoU (Intersection over Union) value between a bounding box corresponding to the liver lesion among the actual label images and a bounding box of the liver lesion inferred by the artificial intelligence model for the label images; Step (S220) of determining whether the above IoU value is greater than or equal to a threshold; If the IoU value is greater than the threshold, a step (S240) of cropping and resizing the bounding box inferred by the artificial intelligence model into an image of a predetermined size; and A method for classifying liver lesions using an artificial intelligence model, characterized in that it includes a step (S260) of classifying the cropped image by the artificial intelligence model into one of a plurality of liver lesions.

9. In paragraph 8, The above IoU value calculation step (S200) is A step (S202) of segmenting the label image to generate a prediction mask representing the liver lesion; and A method for classifying liver lesions using an artificial intelligence model, characterized by including a detection step (S204) of drawing a bounding box including the above prediction mask.

10. In paragraph 8, A method for classifying liver lesions using an artificial intelligence model, wherein the threshold value is 0.

2.

11. In paragraph 8, The above cropping and resizing step (S240) is Step of converting the above bounding box into a square (S242); A step of cropping the above square bounding box (S246); and A method for classifying liver lesions using an artificial intelligence model, characterized in that it includes a step (S248) of resizing the cropped bounding box to a predetermined size.

12. In paragraph 8, A method for classifying liver lesions using an artificial intelligence model, characterized in that in the above classification step (S160), the liver lesions are at least two of hepatocellular carcinoma (HCC), hemangioma, and cyst.

Images