Artificial intelligence model for classifying liver lesions using liver ultrasound images, method for constructing said model, and method for classifying liver lesions using said artificial intelligence model

Abstract

The present invention relates to the classification of liver lesions using liver ultrasound images, and more particularly to an artificial intelligence model for classifying liver lesions using liver ultrasound images, a method for constructing the model, and a method for classifying liver lesions using the artificial intelligence model. To this end, the present invention provides a method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images, the method comprising: a preparation step (S100) of excluding images that meet predetermined exclusion criteria from among a plurality of liver ultrasound images; a step (S120) of receiving the prepared plurality of liver ultrasound images as original images; a step (S140) of assigning the original images to a plurality of folds alternately for each liver lesion; a step (S160) of image processing each original image for training; and a step (S180) of training the artificial intelligence model using the image-processed images.

Description

[Technical Field]

[0001] The present invention relates to the classification of liver lesions using liver ultrasound images, and more particularly to an artificial intelligence model that classifies liver lesions using liver ultrasound images, a method for constructing the model, and a method for classifying liver lesions using the artificial intelligence model. [Background technology]

[0002] Liver ultrasound imaging devices are commonly used to diagnose liver diseases or liver lesions. Liver ultrasound images are analyzed to classify liver tumors into hepatocellular carcinoma (HCC), hemangioma, and cysts.

[0003] Traditionally, this type of image analysis work has been performed by humans (e.g., radiology doctors), but recently, with the development of artificial intelligence, there has been a gradual increase in cases where this work is supported by artificial intelligence models.

[0004] However, it is not easy to detect and segment the boundaries of liver lesions from the entire liver ultrasound image, nor is it easy to detect their locations.Furthermore, in order to accurately infer and classify which liver lesions correspond to which, it is essential to carefully organize the original dataset and design a precise artificial intelligence model. Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, the present invention has been made to solve the above problems, and the problem that the present invention aims to solve is to provide an artificial intelligence model that classifies liver lesions using liver ultrasound images, a method for constructing such a model, and a method for classifying liver lesions using such an artificial intelligence model.

[0006] However, the technical problems that the present invention aims to solve are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those having ordinary skill in the technical field to which the present invention pertains from the following description. [Means for solving the problem]

[0007] In order to solve the above technical problems, a method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images is provided, which comprises: a preparation step (S100) of excluding images that meet predetermined exclusion criteria from a plurality of liver ultrasound images; a step (S120) executed by a computer in which the computer receives the prepared plurality of liver ultrasound images as input as original images; a step (S140) of alternately allocating the original images to a plurality of folds for each liver lesion; a step (S160) of image processing each original image for training; and a step (S180) of training the artificial intelligence model using the image-processed images.

[0008] The exclusion criteria are liver lesions with a maximum size of 10 cm or more or a minimum size of 1 cm or less on liver ultrasound images.

[0009] Furthermore, the liver lesions are at least two of the following types: hepatocellular carcinoma (HCC), hemangioma, and cyst.

[0010] Furthermore, in the allocation step (S140), each fold is divided into sections for each of the liver lesions, and original images are allocated for each of the liver lesions in each fold, so that all liver lesions of the same subject are allocated to the same fold.

[0011] Furthermore, there are five multiple folds, and liver lesions are classified into hepatocellular carcinoma (HCC), hemangioma, and cyst.

[0012] Furthermore, the image processing step (S160) includes a step of applying a minimum-maximum curvature flow filter to the original image to reduce noise (S162), a step of performing adaptive histogram equalization on the original image to enhance the image (S164), a step of cropping and removing parts of the original image that are not related to the image (S166), and a step of zero-padding the original image into a square and resizing it to a predetermined size (S168).

[0013] The object of the present invention is achieved by a computer-readable recording medium that can be read by a computer and that records an artificial intelligence model constructed by the above-described construction method.

[0014] In another embodiment, the object of the present invention is to provide a method for detecting liver lesions, the method comprising: a preparation step (S100) of excluding images that meet predetermined exclusion criteria from a plurality of liver ultrasound images; a step (S120) of receiving the prepared plurality of liver ultrasound images as original images, the step (S140) of alternately allocating the original images to a plurality of folds for each liver lesion; a step (S160) of image processing each original image for training; a step (S180) of training an artificial intelligence model using the image-processed images; and a step (S190) of actually labeling the original images. This is achieved by a method for classifying liver lesions using an artificial intelligence model, comprising the steps of: calculating a bounding box corresponding to a liver lesion in an image and an IoU value of the bounding box of the liver lesion inferred by the artificial intelligence model for the labeled image (S200); determining whether the IoU value is equal to or greater than a threshold (S220); if the IoU value is equal to or greater than the threshold, cropping and resizing the bounding box inferred by the artificial intelligence model to an image of a predetermined size (S240); and classifying the image cropped by the artificial intelligence model into one of a plurality of liver lesions (S260).

[0015] The IoU value calculation step (S200) also includes a segmentation step (S202) of the labeled image to generate a predicted mask indicating liver lesions, and a detection step (S204) of drawing a bounding box including the predicted mask.

[0016] Furthermore, the threshold is 0.2.

[0017] Furthermore, the cropping and resizing step (S240) includes a step of converting the bounding box to a square (S242), a step of cropping the square bounding box (S246), and a step of resizing the cropped bounding box to a predetermined size (S248). [Effects of the Invention]

[0018] According to one embodiment of the present invention, an artificial intelligence model quickly and accurately classifies the type of liver lesion from liver ultrasound images, which is useful for analyzing liver ultrasound images and can prevent misdiagnosis or non-diagnosis in advance.

[0019] However, the effects obtained from the present invention are not limited to those described above, and other effects not mentioned will be clearly understood by those having ordinary skill in the technical field to which the present invention pertains from the following description. [Brief explanation of the drawings]

[0020] In this specification, the accompanying drawings show preferred embodiments of the present invention and, together with the detailed description of the invention to be given later, aid in understanding the technical concept of the present invention, and therefore the present invention should not be interpreted as being limited to the matters shown in these drawings. [Figure 1] 1 is an explanatory diagram showing the process of image processing of a liver ultrasound image according to the present invention. [Figure 2a] 1 is a state diagram showing the steps by which an artificial intelligence model according to the present invention generates a predicted mask from an actual label image. [Figure 2b]FIG. 2b is a state diagram for drawing a bounding box on the predicted mask generated in FIG. 2a. [Figure 2c] FIG. 2B is a state diagram illustrating the process of classifying liver lesions using the bounding box images depicted in FIG. 2B. [Figure 3a] This is an example of an image in which if the IoU (Intersection Over Union) threshold between the bounding box of the actual labeled image and the bounding box inferred by the artificial intelligence model is 0.2 or higher, the predicted mask is inferred to be a true positive prediction. [Figure 3b] This is an example of an image where if the IoU threshold between the bounding box of the actual labeled image and the bounding box inferred by the AI ​​model is less than 0.2, the predicted mask is inferred to be a false positive (Miss-detection). [Figure 4a] 1 is an example of an image showing a bounding box predicted to be a liver lesion in a liver ultrasound image. [Figure 4b] This is an example of an image in which the predicted bounding box in Figure 4a is converted into a square. [Figure 4c] This is an example of an image cropped from the square bounding box converted in Figure 4b. [Figure 5a] This is an example showing the comparison of segmentation and detection between the actual labeled image and the inferred image of the AI ​​model when the IoU of the bounding box is 0.121, 0.261, and 0.397, respectively. [Figure 5b] This is an example showing the segmentation and localization of an actual labeled image and an inferred image of an artificial intelligence model when the IoU of the bounding box is 0.487, 0.577, and 0.624, respectively. [Figure 5c] This is an example showing the segmentation and localization of an actual labeled image and an inferred image of an artificial intelligence model when the IoU of the bounding box is 0.790, 0.834, and 0.964, respectively. [Figure 6a] This is an example of a confusion matrix showing the classification results in fold 1. [Figure 6b] This is an example of a confusion matrix showing the classification results in fold 2. [Figure 6c] This is an example of a confusion matrix showing classification results in fold 3. [Figure 6d] This is an example of a confusion matrix showing classification results in fold 4. [Figure 6e] This is an example of a confusion matrix showing classification results in fold 5. [Figure 7a] 10 is an example of a graph showing a receiver operating characteristic (ROC) curve for fold 1. [Figure 7b] 10 is an example of a graph showing a receiver operating characteristic (ROC) curve in fold 2. [Figure 7c] 10 is an example of a graph showing a receiver operating characteristic (ROC) curve in fold 3. [Figure 7d] 10 is an example of a graph showing a receiver operating characteristic (ROC) curve for fold 4. [Figure 7e] 10 is an example of a graph showing a receiver operating characteristic (ROC) curve for fold 5. [Figure 8] 6A to 6E are confusion matrices showing classification results obtained by combining the confusion matrices of FIGS. 6A to 6E. [Figure 9] 7 is a graph showing an ROC curve obtained by combining the ROC curves of FIGS. 7a to 7e. [Figure 10a] This is an image comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. [Figure 10b] This is an image comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. [Figure 10c] This is an image comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. [Figure 10d] This is an image comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. [Figure 10e] This is an image comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. [Figure 10f] This is an image comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. [Figure 10g] This is an image comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. [Figure 10h] This is an image comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. [Figure 10i] This is an image comparing actual labeled liver lesions with liver lesions classified by the artificial intelligence model of the present invention. [Figure 11a] This is a confusion matrix showing the mutual classification of HCC and hemangioma among liver lesions. [Figure 11b] This is a confusion matrix showing the mutual classification of HCC and cysts among liver lesions. [Figure 11c] 1 is a confusion matrix showing the mutual classification of hemangiomas and cysts among liver lesions. [Figure 12] 1 is a flowchart illustrating an outline of a method for constructing an artificial intelligence model and classifying liver lesions using liver ultrasound images according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, with reference to the accompanying drawings, detailed descriptions of embodiments of the present invention will be given so that those skilled in the art can easily understand and practice the present invention. However, since the description of the present invention is merely an embodiment for the purpose of describing the structure or function of the present invention, the scope of the present invention should not be construed as being limited to the embodiments of the present invention. In other words, since the embodiments can be modified in various ways and can be implemented in various forms, the scope of the present invention should be understood to include equivalents that realize the technical ideas. Furthermore, the objectives or effects presented by the present invention do not mean that a particular embodiment must include all of the objectives or effects, or that only such effects must be included, and the scope of the present invention should not be construed as being limited thereto.

[0022] The meanings of the terms used in the present invention should be understood as follows.

[0023] Terms such as "first" and "second" are used to distinguish one component from another and do not limit the scope of the rights. For example, the first component may be the second component, and similarly, the second component may be the first component. When a component is "connected" to another component, it should be understood that it may be directly connected to the other component, or that there may be other components between them. Conversely, when a component is "directly connected" to another component, it should be understood that there are no other components between them. Other expressions describing the relationship between components, such as "between" and "directly between," "adjacent to" and "directly adjacent to," should be understood in the same way.

[0024] Singular expressions should be understood to include plural expressions unless otherwise specified, and terms such as "comprise", "have", etc. should be understood to indicate the presence of stated features, numbers, steps, operations, components, parts or combinations thereof, but not to preclude the presence or possible addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0025] Unless otherwise specified, all terms used herein have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted in accordance with the context of the relevant art, and should not be interpreted in an ideal or overly formal sense unless otherwise specified.

[0026] A preferred embodiment will now be described in detail with reference to the accompanying drawings. FIG. 12 is a flowchart outlining a method for constructing an artificial intelligence model and classifying liver lesions using liver ultrasound images according to one embodiment of the present invention. As shown in FIG. 12, first, as a preparatory step, images that meet predetermined exclusion criteria are excluded from multiple liver ultrasound images (S100). Liver ultrasound images are black and white gray images that contain liver lesions, and only the images remaining after exclusion according to the exclusion criteria are used. The exclusion criteria are that liver lesions in the liver ultrasound images have a maximum size of 10 cm or more or a minimum size of 1 cm or less. This is because liver lesions that are too large are difficult to segment, and those that are too small may result in erroneous inference and require a long learning time.

[0027] The images used for training after exclusion according to the exclusion criteria included 3,714 images of 1,258 patients with 3,739 liver lesions for hepatocellular carcinoma (HCC). 2,018 images of 834 patients with 2,044 liver lesions for hemangioma. 980 images of 480 patients with 1,039 liver lesions for cysts. These images were categorized by liver lesion type, and some patients may contain multiple liver lesions, resulting in multiple images. This preparation step may be performed collectively by a computer using size data of liver lesions output from the ultrasound system, or by an operator.

[0028] Next, the prepared liver ultrasound images are input as original images into a computer and stored (S120). To this end, the computer may include a conventional CPU, RAM, hard disk, input port (e.g., USB port), etc., and may have an operating system (e.g., Windows, Linux) program and an artificial intelligence program (e.g., Python package program including Pytorch) installed and running.

[0029] Next, the original image is divided into five equal parts, and then each liver lesion is assigned to one of five folds (S140) in a staggered manner, as shown in Table 1. In Table 1, Training indicates the images used to train the AI ​​model (equivalent to 3 / 5 of the total images), Validation indicates the images used to verify the completion of AI model training (equivalent to 1 / 5 of the total images), and Test indicates the images used to verify the accuracy of the AI ​​model's inference (equivalent to 1 / 5 of the total images). Thus, five cross validations (CVs) are defined for the five folds.

[0030] [Table 1]

[0031] As shown in Table 2, each fold is classified by liver lesion, and the original image is assigned to each liver lesion in each fold, so that all liver lesions of the same subject are assigned to the same fold. Here, liver lesions are classified into hepatocellular carcinoma (HCC), hemangioma, and cyst.

[0032] [Table 2]

[0033] As shown in Table 2, 520 subjects (patients) were assigned to each fold, and the training set, validation set, and test set were independent of each other. Next, the computer processes each original image for training (S160). FIG. 1 is an explanatory diagram showing the process of image processing of liver ultrasound images according to the present invention. As shown in FIG. 1, the image processing step (S160) involves applying a min-max curvature flow filter to the original image to reduce noise (S162). Next, adaptive histogram equalization is performed on the original image to enhance the image (S164). Next, parts of the original image that are not related to the image (e.g., text) are cropped and deleted (S166). Next, the original image is zero-padded to a square and resized to a predetermined size (e.g., 512 x 512) (S168). Zero padding is a method of filling all padding bytes with zeros. For the image processing step (S160), a standard image processing program is installed and executed on the computer. Next, an AI model is trained using the processed images as described above (S180). A commercial "Python package program including Pytorch" was used as the AI ​​program, and the DeepLab V3-Resnet 101 model or the Efficientnet-b2 model was used as the AI ​​system architecture.

[0034] To train the artificial intelligence model, the initial learning rate was set to 0.001 and the batch size was 4. The loss function for training was a combination of Dice loss and Binary Cross Entropy (BCE) loss. To optimize the loss function, we used Stochastic Gradient Descent (SGD) with momentum of 0.9 and weight decay of 0.0001. Stochastic Gradient Descent (SGD) is an algorithm that calculates the predicted gradient at each step using one example probabilistically selected from the entire dataset.

[0035] During the training process, rectangular bounding boxes were generated to indicate the locations of liver lesions, and on-the-fly data augmentation strategies, including rotation, translation, flip, scaling, and color jitter, were performed.

[0036] As a result of the training, if the loss of validation data did not decrease for 8 epochs in the AI ​​model, the learning rate was reduced to 0.5, and if the loss of validation data did not decrease for 20 epochs, the training process was terminated early. The learning rate is a measure of the learning process of the AI ​​model, which means that the AI ​​model is gradually approaching the optimal solution as it is optimized. In the post-processing process after the training process is completed, the learning rate was reduced to 0.1 cm. 2 Smaller objects were removed and a 7x7 median filter was applied to the predicted mask. This training process resulted in the creation of an artificial intelligence model.

[0037] Next, during the testing process, the computer calculates the IoU value between the bounding box corresponding to the liver lesion in the actual labeled image and the bounding box of the liver lesion inferred by the artificial intelligence model for the labeled image (S200).

[0038] First, the bounding box generation process is as follows. Figure 2a is a state diagram showing the steps of an artificial intelligence model according to the present invention generating a predicted mask from an actual labeled image, and Figure 2b is a state diagram showing how a bounding box is drawn on the predicted mask generated in Figure 2a. As shown in Figures 2a and 2b, 101 layer operations are performed on the liver lesion image, with the first layer consisting of 64 channels generated by convolution operations on a 7x7 matrix. Next, atrous spatial pyramid pooling (ASPP) is performed, which constructs atrous pooling in the form of a pyramid. The output of this atrous spatial pyramid pooling network is upsampled by 1x1 convolution, generating a predicted mask on the liver lesion image (S202).

[0039] As shown in Figure 2b, the smallest square that contains the predicted mask is defined and generated as a bounding box (S204). These bounding boxes are calculated as the bounding box corresponding to the liver lesion in the actual labeled image and the bounding box of the liver lesion inferred from the labeled image by the AI ​​model.

[0040] In step S200, it is determined whether the calculated IoU value is 0.2 or greater (S220). Figure 3a shows an example of an image in which, if the IoU (Intersection Over Union) threshold between the bounding box of the actual labeled image and the bounding box inferred by the AI ​​model is 0.2 or greater, the predicted mask is inferred to be a true positive prediction. Figure 3b shows an example of an image in which, if the IoU threshold between the bounding box of the actual labeled image and the bounding box inferred by the AI ​​model is less than 0.2, the predicted mask is inferred to be a false positive. Therefore, a lower IoU threshold (e.g., 0.1, 0.15, etc.) can be used to achieve highly sensitive detection of liver lesions.

[0041] Figure 2c is a state diagram showing the process of classifying liver lesions using the bounding box image depicted in Figure 2b, once it is determined to be a true positive prediction. As shown in Figure 2c, first, the bounding box is converted to a square (S242), the square bounding box is cropped (S246), and then the cropped square bounding box is resized to a size of 96 x 96 (S248). For example, Figure 4a shows an image of a bounding box predicted to be a liver lesion in a liver ultrasound image. Figure 4b shows an image of the predicted bounding box in Figure 4a converted to a square, and Figure 4c shows an image of the square bounding box converted in Figure 4b cropped. In step S242, the rectangular bounding box is converted to a square based on the length of its long side.

[0042] Next, the resized image is inferred to classify it as hepatocellular carcinoma (HCC), hemangioma, or cyst (Cyst) as follows: As shown in Figure 2c, a convolutional neural network consisting of 23 mobile inverted bottleneck convolutions (MBConv) extracts eight features. MBConv performs depthwise convolution, which separates the feature map into channels and performs calculations to extract features with fewer parameters than conventional convolution operations. Then, pointwise convolution combines the channels into a single channel again using a 1x1 convolution operation. Finally, a squeeze network converts each channel's feature map into a single feature value, and an excitation network calculates the importance of the feature value. Finally, image features for liver lesion classification are extracted through these processes. In addition to this feature extraction using a convolutional neural network, we also perform feature extraction using the radiomics method, which extracts first-order features that indicate the pixel intensity distribution of an image and texture features that indicate the signal relationship between adjacent pixels.Among these, texture features are applied using the Gray Level Co-occurrence Matrix (GLCM), which calculates the frequency of pixel pairs with specific values ​​occurring in a specific space in an image; the Gray Level Run Length Matrix (GLRLM), which calculates the length of consecutive pixels with the same intensity in an image; the Gray Level Size Zone Matrix (GLSZM), which calculates the size of adjacent pixels with the same intensity in an image; the Neighboring Tone Difference Matrix (NGTDM), which calculates the difference between the intensity of a specific pixel and the average intensity of its neighboring pixels in an image; and the Gray Level Dependence Matrix (GLDM), which calculates the frequency of pixel intensities occurring between pixels separated by a specific distance in an image. Next, to prevent overfitting in the classification model, feature selection is performed using a constrained lasso regression in which the sum of the absolute values ​​of the weights is zero. This allows important features to be selected, as shown in Table 3. The selected features are input to a multilayer perceptron with a 32-16-16-8 layer structure and converted back into eight features. This is combined with eight features extracted using the convolution method to derive a probability value for classifying the tumor as HCC, Hemangioma, or Cyst (S260).

[0043] [Table 3]

[0044] As shown in Figures 3a-3c and Table 4, the artificial intelligence model of the present invention classified 6,077 cropped lesions and 6,038 images from 2,444 patients into three types of liver lesions.

[0045] [Table 4] [Example]

[0046] Comparative experiment The following are the results of various comparative experiments using the AI ​​model constructed as described above. Figure 5a shows an example of the segmentation and localization of an actual labeled image compared to an inferred image using an AI model when the IoU of the bounding box is 0.121, 0.261, and 0.397, respectively. Figure 5b shows an example of the segmentation and localization of an actual labeled image compared to an inferred image using an AI model when the IoU of the bounding box is 0.487, 0.577, and 0.624, respectively. Figure 5c shows an example of the segmentation and localization of an actual labeled image compared to an inferred image using an AI model when the IoU of the bounding box is 0.790, 0.834, and 0.964, respectively. In Figures 5a-5c, the first column shows the original image, the red in the second column shows the lesion in the labeled image, and the blue shows the predicted mask inferred by the AI ​​model. In the third column, the red color indicates the bounding box containing the lesion in the labeled image, and the blue color indicates the bounding box containing the predicted mask inferred by the artificial intelligence model.

[0047] In Figure 5a, when the IoU is 0.1211, the relatively large blue bounding box only encompasses half of the red bounding box. This indicates that the AI ​​model accurately infers the location of the liver lesion. However, when the IoU is 0.2 or greater, the red and blue bounding boxes are quite close to each other. In particular, the closer the IoU value is to 1, the closer the bounding boxes are to each other.

[0048] Figures 6a to 6e are examples of confusion matrices showing the classification results for folds 1 to 5. As shown in Figure 6a, in fold 1, the AI ​​model classified HCC in the labeled image as HCC in 461 cases (correct classification), and incorrectly classified it in 151 + 28 = 179 cases. It can be seen that similar classification results are shown across folds 1 to 5.

[0049] 7a to 7e are graphs showing ROC (Receiver Operating Characteristic) curves for folds 1 to 5. It can be seen that similar classification results are shown across folds 1 to 5.

[0050] Figure 8 shows the confusion matrix representing the classification results obtained by combining the confusion matrices in Figures 6a to 6e, and Figure 9 shows the ROC curve representing the ROC curves in Figures 7a to 7e. As shown in Figure 8, the AI ​​model performed best in classifying cysts, but the classification accuracy for HCC and hemangioma was relatively low. In Figure 9, the ROC curves for HCC and hemangioma showed similar patterns (AUC for HCC = 0.865, AUC for hemangioma = 0.850), while the AUC for cysts was high at 0.990.

[0051] Table 5 is a graph showing the segmentation performance (boundary detection) of liver lesions, with the average values ​​shown in parentheses with standard deviations.

[0052] [Table 5]

[0053] Here, the Dice Similarity Coefficient (DSC) is twice the overlap area between the label-specified mask and the predicted mask divided by the total area of ​​the label-specified mask and the predicted mask, and the IoU score is the overlap area between the label's bounding box and the predicted bounding box divided by the union of the label and the prediction.

[0054] As shown in Table 5, it was confirmed that consistent DSC and IoU performance was achieved across CV1 to CV5, with a DSC performance of 72.8% and an IoU performance of 62.9%. Considering the difficulty of clearly defining the boundaries of liver lesions, the segmentation performance of the artificial intelligence model is extremely excellent.

[0055] Table 6 is a graph showing the detection performance (location detection) of liver lesions, with the 95% confidence interval in parentheses.

[0056] [Table 6]

[0057] If no segmentation mask is output for the region containing liver lesions, the number of true and false positives (false positives) is counted. False positives per image (FPPI) is the number of false positives divided by the total number of images.

[0058] In terms of detection performance, the AI ​​model of the present invention showed a detection rate of 0.891 (95% CI: 0.879-0.903) and an FPPI of 0.223 (95% CI: 0.212-0.234) across all folds. The AI ​​model showed a detection rate of 0.868-0.921 and an FPPI of 0.183-0.276 across each cross-validation fold.

[0059] Table 7 shows the overall classification performance of the artificial intelligence model on liver ultrasound images.

[0060] [Table 7] JPEG2025533371000009.jpg111150

[0061] In Table 7, recall is the number of true positive lesions divided by the total number of positive lesions, and plays an important role in supporting clinical decision-making.

[0062] Accuracy is defined as the number of true-positive lesions divided by the total number of lesions predicted to be positive (the sum of the number of true-positive and false-positive lesions).

[0063] The F1 score is defined as the harmonic mean of recall and precision. The area under the receiver operating characteristic curve (AUC) score is defined as the area under the receiver operating characteristic (ROC) curve, which indicates the number of false positives at the decision threshold.

[0064] As shown in Table 7, for example, the classification of HCC had a recall of 0.753 (95% CI: 0.738-0.768), a precision of 0.817 (95% CI: 0.803-0.831), an F1 score of 0.784 (95% CI: 0.774-0.794), and an AUC score of 0.865 (95% CI: 0.856-0.874).

[0065] 10a to 10i are images comparing actual labeled liver lesions with liver lesions classified by the AI ​​model of the present invention. In Fig. 10a, the cyst in the labeled image was correctly classified as a cyst, but in Fig. 10b, the cyst in the labeled image was incorrectly classified as a hemangioma.

[0066] Figure 11a is a confusion matrix showing the mutual classification of HCC and hemangioma among liver lesions, and Table 8 shows the classification performance showing the mutual classification of HCC and hemangioma.

[0067] [Table 8]

[0068] As shown in Figure 11a, there were 2,375 cases in which HCC in the labeled image was correctly classified as HCC, and 906 cases in which it was incorrectly classified as hemangioma. There were also 1,391 cases in which hemangioma in the labeled image was correctly classified as hemangioma, and 466 cases in which it was incorrectly classified as HCC. Therefore, as shown in Table 8, the recall rate between HCC and hemangioma ranged from 0.724 to 0.749.

[0069] Figure 11b is a confusion matrix showing the mutual classification of HCC and cysts among liver lesions, and Table 9 shows the classification performance showing the mutual classification of HCC and cysts.

[0070] [Table 9]

[0071] As shown in Figure 11b, 3,114 cases of HCC in the labeled image were correctly classified as HCC, while 167 cases were incorrectly classified as cysts. Also, 900 cases of cysts in the labeled image were correctly classified as cysts, while 39 cases were incorrectly classified as HCC. Therefore, as shown in Table 9, the recall rate between HCC and cysts was 0.949-0.958. This indicates that the classification performance between HCC and cysts is much higher than that between HCC and hemangiomas. This indicates that the AI ​​model can effectively distinguish between HCC and cysts.

[0072] Figure 11c is a confusion matrix showing the mutual classification of hemangiomas and cysts among liver lesions, and Table 10 shows the classification performance showing the mutual classification of hemangiomas and cysts.

[0073] [Table 10]

[0074] As shown in Figure 11c, there were 1,771 cases in which hemangiomas in the labeled images were correctly classified as hemangiomas, and 86 cases in which they were incorrectly classified as cysts. There were also 908 cases in which cysts in the labeled images were correctly classified as cysts, and 31 cases in which they were incorrectly classified as hemangiomas. Therefore, as shown in Table 10, the recall between hemangiomas and cysts was 0.954-0.967. This indicates that the classification performance between cysts and hemangiomas is much higher than that between HCC and hemangiomas. This indicates that the AI ​​model can effectively distinguish between hemangiomas and cysts.

[0075] The detailed description of the preferred embodiments of the present invention described above is provided to enable those skilled in the art to realize and practice the present invention. Although the present invention has been described with reference to the preferred embodiments, those skilled in the art will understand that various modifications and variations are possible without departing from the spirit and scope of the present invention. For example, those skilled in the art will understand that the configurations described in the above embodiments can be used in combination with each other. Therefore, the present invention is not limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0076] The present invention may be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention. Therefore, the above detailed description should not be construed as limiting in any respect, but should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the claims, and all modifications within the scope of equivalents of the present invention are intended to be included in the present invention. The present invention is not limited to the embodiments set forth in this specification, but is to be accorded the widest scope consistent with the principles and novel features set forth in this specification. Furthermore, in the claims, embodiments may be constructed by combining claims that are not explicitly cited, and new claims may be included by amendment after filing. [Industrial Applicability]

[0077] The present invention relates to the classification of liver lesions using liver ultrasound images, and more particularly to an artificial intelligence model that classifies liver lesions using liver ultrasound images, a method for constructing the model, and a method for classifying liver lesions using the artificial intelligence model.

Claims

1. A preparation step (S100) of excluding images that meet predetermined exclusion criteria from among a plurality of liver ultrasound images; The method is executed by a computer, and the computer A step (S120) of receiving input of a plurality of prepared liver ultrasound images as original images; A step (S140) of assigning the original image to a plurality of folds alternately for each liver lesion; A step (S160) of image processing each of the original images for training; and a step (S180) of training the artificial intelligence model using the processed images.

2. 2. The method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images according to claim 1, characterized in that the exclusion criteria are that the maximum size of the liver lesion in the liver ultrasound image is 10 cm or more, or the minimum size is 1 cm or less.

3. 2. The method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images according to claim 1, characterized in that the liver lesions are at least two types of hepatocellular carcinoma (HCC), hemangioma, and cyst.

4. In the allocation step (S140), Each fold is divided into liver lesions, The original image is assigned to each of the liver lesions in each fold; The method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images according to claim 1, characterized in that all liver lesions of the same subject are assigned to the same fold.

5. said plurality of folds being five; The method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images according to claim 4, characterized in that the liver lesions are classified into hepatocellular carcinoma (HCC), hemangioma, and cyst.

6. The image processing step (S160) applying a minimum-maximum curvature flow filter to the original image to reduce noise (S162); performing adaptive histogram equalization on the original image to enhance the image (S164); a step (S166) of cropping and deleting portions of the original image that are not related to the image; The method for constructing an artificial intelligence model for classifying liver lesions using liver ultrasound images according to claim 1, further comprising a step (S168) of zero-padding the original image into a square and resizing it to a predetermined size.

7. is computer readable; A computer-readable recording medium on which an artificial intelligence model constructed by the construction method according to any one of claims 1 to 6 is recorded.

8. A preparation step (S100) of excluding images that meet predetermined exclusion criteria from among a plurality of liver ultrasound images; The method is executed by a computer, and the computer A step (S120) of receiving input of a plurality of prepared liver ultrasound images as original images; A step (S140) of assigning the original image to a plurality of folds alternately for each liver lesion; A step (S160) of image processing each of the original images for training; training an artificial intelligence model using the processed images (S180); Calculating an Intersection over Union (IoU) value between a bounding box corresponding to the liver lesion in the actual labeled image and a bounding box of the liver lesion inferred by the AI ​​model for the labeled image (S200); determining whether the IoU value is equal to or greater than a threshold (S220); If the IoU value is greater than or equal to a threshold, resizing the bounding box inferred by the artificial intelligence model by cropping it to a predetermined size image (S240); and a step (S260) in which the artificial intelligence model classifies the cropped image as one of a plurality of liver lesions.

9. The IoU value calculation step (S200) A step (S202) of segmenting the labeled image to generate a predicted mask indicative of the liver lesion; and a detection step (S204) of drawing a bounding box including the predicted mask.

10. The method for classifying liver lesions using an artificial intelligence model according to claim 8, wherein the threshold value is 0.

2.

11. The cropping and resizing step (S240) includes: converting the bounding box into a square (S242); Cropping the square bounding box (S246); and resizing the cropped bounding box to a predetermined size (S248).

12. 9. The method for classifying liver lesions using an artificial intelligence model according to claim 8, wherein in the classification step (S160), the liver lesions are at least two types of hepatocellular carcinoma (HCC), hemangioma, and cyst.

Images