Breast cancer image classification method based on artificial intelligence model

Abstract

The present invention relates to an artificial intelligence-based method for determining breast cancer histopathology images, specifically comprising the steps of: inputting a breast cancer histopathology image dataset in TIFF format with a pixel size of 2048x1536; stain normalizing the breast cancer histopathology image dataset; converting the stain-normalized image dataset into patches with a pixel size of 512x512; oversampling the converted image dataset using the Synthetic Minority Oversampling Technique (SMOTE) to process data imbalance; extracting features from a pre-trained ResNet-34 model using transfer learning; fine-tuning the final layer of the ResNet-34 model from which all features have been extracted as a discriminator; applying the fine-tuned ResNet-34 model to the oversampled image dataset; and determining the breast cancer histopathology image and outputting it as normal or invasive breast cancer.

Description

Technology Field

[0001] The present invention relates to a method for determining breast cancer histopathology images. Specifically, it relates to a method for determining breast cancer histopathology images based on an artificial intelligence model, which analyzes breast cancer histopathology images and determines whether the breast cancer is normal or invasive, thereby reducing the time and effort required for clinicians to locate invasive breast cancer slides. Background Technology

[0003] Breast cancer is the most common type of cancer affecting women worldwide. While it can occur in both men and women, it is much more common in women and is considered one of the leading causes of death from cancer.

[0004] To correctly diagnose the various types of breast cancer mentioned above, medical examinations (generally performed by a surgeon) must be conducted, followed by microscopic analysis of the breast tissue. To do this, the biopsy material must first be excised and stained using hematoxylin and eosin. The hematoxylin solution binds to deoxyribonucleic acid (DNA) and highlights the nucleus, while eosin binds to proteins and highlights other structures. Subsequently, a specialist pathologist evaluates the tissue biopsy by visualizing the highlighted areas in the digitized image using a microscope. The evaluation of the tissue biopsy enables the identification of initial clues. However, performing this task requires a significant amount of time and effort from the specialist pathologist.

[0005] For the general public to be diagnosed with breast cancer, not only is it time-consuming and expensive, but they also have no choice but to rely heavily on the pathologist's prior knowledge and the consistency of pathological reports. The average diagnostic accuracy of professional pathologists is known to be about 75%.

[0006] Meanwhile, Computer-Aided Diagnosis (CAD) systems have been developed to help physicians diagnose patients more quickly and accurately in many hospitals. These CAD systems are integrated into the diagnostic process for breast cancer detection to reduce inter-observer variability, effectively provide biopsy recommendations, and distinguish between normal tissues and invasive carcinomas.

[0007] However, while the above CAD system contributed to increasing the diagnostic efficiency of biopsy tissues by using images stained with hematoxylin and eosin, most CAD systems traditionally used inefficient and time-consuming manual feature extraction methods.

[0008] Furthermore, deep learning networks have recently been developed to extract the most distinct features and enhance the effectiveness of medical image analysis. The use of deep learning networks for feature extraction offers two advantages. First, using deep learning models allows for the automatic extraction of more complex feature sets compared to other machine learning tools. Second, since collaborative and layered learning features can be extracted from multiple layers of the deep learning network, the network is consequently utilized efficiently in the feature selection stage.

[0009] However, breast tissue biopsy images stained with hematoxylin and eosin are difficult to train using deep learning models because of their large image size.

[0010] Furthermore, another difficult problem in breast cancer histopathology image analysis is related to the issue of imbalanced data. For example, Deep Convolutional Neural Networks (DCNNs) are effective only when the number of available training samples is sufficiently large during the training phase. Conversely, if training samples are limited, these networks often suffer from overfitting. Unlike natural image recognition tasks, there are far fewer medical images available for the effective training of deep learning networks for medical applications such as breast cancer detection. This is due to data privacy concerns and the increasing cost of data collection.

[0011] Furthermore, as with many other medical imaging applications, breast cancer detection methods have faced the problem of unbalanced training data because collecting data from patients is not an easy task. The problem to be solved

[0013] The purpose of the present invention is to provide an artificial intelligence-based breast cancer histopathology image identification method that solves the problem of unbalanced training data as described above and improves system performance and identification accuracy by inputting the minimum possible load into a very large pathology image dataset. means of solving the problem

[0015] To achieve the above objective, the artificial intelligence-based breast cancer histopathology image discrimination method of the present invention may be an artificial intelligence-based breast cancer histopathology image discrimination method comprising the steps of: inputting a breast cancer histopathology image dataset in TIFF format with a pixel size of 2048x1536; stain normalizing the breast cancer histopathology image dataset; converting the stain-normalized image dataset into a patch with a pixel size of 512x512; oversampling the converted image dataset using the SMOTE (Synthetic Minority Oversampling Technique) technique to process data imbalance; extracting features from a pre-trained ResNet-34 model using transfer learning; fine-tuning the final layer of the ResNet-34 model from which all features have been extracted as a discriminator; applying the fine-tuned ResNet-34 model to the oversampled image dataset; and discriminating the breast cancer histopathology image and outputting it as normal or invasive breast cancer.

[0016] As another embodiment of the present invention, when a pre-learned feature is extracted in the transfer learning, the convolutional layer is fixed and only the final layer has a sigmoid function activated to determine the output, which may be a breast cancer histopathology image discrimination method.

[0017] As another embodiment of the present invention, the breast cancer histopathology image identification method may further include a verification and evaluation step following the step of identifying and outputting the breast cancer histopathology image.

[0018] In another embodiment of the present invention, the verification and evaluation step may be a breast cancer histopathology image discrimination method in which a confusion matrix consisting of true positives, true negatives, false positives, and false negatives is obtained by evaluating the discrimination value, and accuracy, precision, recall, and F-score are measured using the confusion matrix. Effects of the invention

[0020] The present invention can improve system performance and discrimination accuracy in a breast cancer histopathology image discrimination method by solving the problem of unbalanced training data and minimizing the load of a very large pathology data set. Brief explanation of the drawing

[0022] Figure 1 is a flowchart of the breast cancer histopathology image identification method of the present invention. Figure 2 is a diagram illustrating the blot normalization process of the present invention using the Vahadane blot normalization algorithm. Figure 3 compares the structure of the artificial intelligence model-based neural network of the present invention and the structure of the conventional VGG19 artificial intelligence model-based neural network. Figure 4 is a flowchart of a method for performing oversampling for data imbalance processing in the present invention. Figure 5 shows the form of a residual block in which the input and output of a weight layer are connected through an ID connection. Figure 6 is an algorithm for fine-tuning the final layer in the present invention. Figure 7 is a flowchart showing the transfer learning process of the model. Figure 8 is a diagram showing the confusion matrix obtained by verifying the determination results of the embodiment of the present invention and Comparative Example 1. Figure 9 is a screen of an application implementing the breast cancer histopathology image algorithm of the present invention. Figure 10 is an output image obtained through the artificial intelligence model-based breast cancer histopathology image identification method of the present invention. Specific details for implementing the invention

[0023] Hereinafter, preferred embodiments of the method for identifying breast cancer histopathology images based on an artificial intelligence model according to the present invention will be described in detail with reference to the attached drawings. Identical reference numerals in each drawing indicate identical components. Furthermore, specific structural or functional descriptions regarding embodiments of the present invention are illustrative of embodiments according to the present invention, and unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and unless explicitly defined in this specification, they should not be interpreted in an ideal or overly formal sense.

[0024] FIG. 1 is a diagram showing the overall flow of the breast cancer histopathology image identification method of the present invention, showing each step of the breast cancer histopathology image identification method.

[0025] Specifically, the artificial intelligence-based breast cancer histopathology image identification method of the present invention includes the steps of inputting a breast cancer histopathology image dataset (S10), image stain normalization (S11), converting the stain-normalized image dataset into a patch with a pixel size of 512x512 (S12), oversampling the converted image dataset using the Synthetic Minority Oversampling Technique (SMOTE) (S13), extracting features using transfer learning (S14), fine-tuning the final layer of the learning model (S15), applying the transferred-learned model to the oversampled image dataset (S16), and identifying and outputting the breast cancer histopathology image (S17).

[0026] The step of inputting a breast cancer histopathology image dataset (S10) is a step of inputting breast biopsy image data stained with hematoxylin and eosin, and the dataset, which has two classes of normal and invasive breast cancer, is a microscopic image in TIFF (Tag Image File Format) format of 2048x1536 pixels, and the pixel scale is 0.41㎛ x 0.42㎛.

[0027] Generally, pathologists primarily use microscopic images in TIFF format. The present invention is a discrimination method for pathologists, and it is desirable to use the TIFF format because it is convenient for storing high-quality images and performing learning in a neural network model while avoiding data loss.

[0028] The image blot normalization step (S11) is a preprocessing step for the breast cancer histopathology image dataset.

[0029] Breast cancer histopathology images are stained with hematoxylin and eosin to highlight regions of interest. However, differences in stain color values ​​may occur among breast cancer histopathology images due to various environmental factors, such as variations in staining concentrations during the staining process or tissue slide image acquisition, and the influence of light during scanning. Since differences in stain color values ​​in breast cancer histopathology image data can affect the discrimination results of models trained on the corresponding image dataset, an image stain normalization process is required.

[0030] To compensate for the disadvantage of color pixels scattering and image quality being distorted while normalizing the color of the blot, the Vahadane blot normalization algorithm, which has the characteristic of preserving structure, is used.

[0031] Through the Vahadane blot normalization algorithm, blot density maps of images in the dataset can be generated and combined with the blot colors of the target images to modify only the image colors while preserving the structure, thereby enabling normalization with minimal loss compared to other normalization techniques.

[0032] The stain normalization preprocessing method applied in the present invention includes the steps of generating a stain density map of a breast cancer histopathology image dataset, setting a stain of a target image according to the stain details of the stain density map, and generating a normalized stain by combining the stain of the stain density map with the set stain of the target image.

[0033] Figure 2 is a diagram illustrating the stain normalization process of the present invention using the Vahadane stain normalization algorithm. When a stain-normalized image in PNG format with a size of 1388x1040 pixels and a source image in TIFF format with a size of 2048x1536 pixels, provided by the Vahadane stain normalization algorithm, are normalized, the characteristics of the two images are merged to generate an image of 1296x1296 pixels. It can be confirmed that only the image color is changed and the structure is preserved by combining with the stain color of the stain-normalized image.

[0034] The step (S12) of converting the blot-normalized image dataset into patches with a pixel size of 512x512 is a step of converting the images into small patches to highlight details so that the Convolutional Neural Network (CNN) can accurately detect features.

[0035] The reason for setting the size of the transformation patch to 512x512 pixels is that this patch size encompasses a large Field of View (FOV) and allows for greater spatial resolution. Commonly used patch sizes of 64x64 and 256x256 provide output results with fewer pixels and blurry images, which can lead to reduced accuracy. This is because larger patch sizes integrate more samples into each prediction generated by the network, thereby providing better results. It is advisable to select a size of 512x512 to improve model accuracy and provide clear output results.

[0036] The step (S13) of oversampling the transformed image dataset using the Synthetic Minority Oversampling Technique (SMOTE) to handle data imbalance is a step of making the distribution of the two classes equal by generating synthetic or dummy samples of the minority class because it is difficult for the model to distinguish between two classes of unbalanced size.

[0037] For example, in a dataset, breast cancer is a minority class with 44 samples and normal is a majority class with 48 images. The SMOTE algorithm can balance the two classes to have 48 samples per class by generating 4 synthetic samples for the breast cancer class and increasing the sample level to 48.

[0038] FIG. 4 illustrates an example of a modeling method for performing oversampling for data imbalance processing in the present invention. The oversampling method is a method of generating invasive breast cancer image data with a small amount of quantity as virtual data to create a balanced state when the quantity of invasive breast cancer image data is in an imbalanced state that is not suitable for use as a training dataset for machine learning for image recognition.

[0039] As shown in Fig. 4, an imbalanced ratio (IR) based on the number of samples in a breast cancer histopathology image dataset can be calculated (S130). Here, the imbalanced ratio is the value obtained by dividing the number of data in the majority class by the number of data in the minority class in the breast cancer histopathology image dataset.

[0040] Before calculating the imbalance ratio, a threshold ratio that can serve as a criterion for judging imbalance can be set. Given that the distinction between positive and negative is relatively clear through image samples, it is desirable to set the threshold ratio to 1.0 to 1.5, and most preferably to 1.0.

[0041] In a modeling method that performs oversampling for data imbalance processing, the calculated imbalance ratio can be determined whether it is a threshold ratio, for example 1.0, which serves as a criterion for judging imbalance (S131). Here, the threshold ratio of 1.0 is merely one example and is set differently depending on the condition of the breast cancer histopathology image sample, etc.

[0042] If the calculated imbalance ratio exceeds a threshold ratio of 1.0, and breast cancer images are in the minority class and normal images are in the majority class, the dataset of breast cancer histopathology images is determined to be in an imbalanced state, and samples of breast cancer images in the minority class are generated using a virtual data augmentation method (S132). The virtual data augmentation method uses SMOTE, which is a well-known oversampling algorithm that synthesizes and augments data of the minority class.

[0043] If the calculated imbalance ratio is 1.0, which is the threshold ratio, the dataset of breast cancer histopathology images is determined to be in a balanced state and the virtual data augmentation method is stopped (S133).

[0044] The feature extraction step (S14) using transfer learning is a step of extracting learned features from a pre-trained model and transferring them to a new model.

[0045] A neural network model for transfer learning includes an input layer, multiple convolutional layers, a pooling layer, and a final layer.

[0046] The input layer can be composed of a convolutional neural network layer with 64 filters of size 7x7, and an image of an oversampled dataset is input to the input layer.

[0047] Multiple convolutional and pooling layers may be used, and convolutional layers can transform an image by applying a filter to the input data and then applying an activation function. Here, the pooling layer may be a subsampling layer that reduces the dimensionality of the image, or it may be omitted.

[0048] One or more convolutional layers may include a convolutional neural network layer with 64 filters of size 3x3, a convolutional neural network layer with 128 filters of size 3x3, a convolutional neural network layer with 256 filters of size 3x3, and a convolutional neural network layer with 512 filters of size 3x3.

[0049] The final layer fine-tuning step (S15) of the learning model is a step of fine-tuning the final layer by replacing it with a sigmoid activation function.

[0050] The final layer can determine the output of the discriminative model based on the input data set. The final layer can be fine-tuned through a sigmoid activation function, and after fine-tuning, the entire neural network model is unlocked again to obtain the output of the learning, thereby improving the processing speed and diagnostic accuracy of the neural network model learning.

[0051] The primary reason for using the sigmoid function is to perform binary discrimination. Therefore, it is particularly used in models that output a probability (0 or 1). The sigmoid function is suitable for the artificial intelligence model-based breast cancer histopathology image discrimination method of the present invention because there are only two classes: normal and invasive breast cancer.

[0052] The neural network model used in this invention is Resnet-34, which, unlike conventional models such as VGG16 and patch-based classifiers (PBC), is structured such that each layer is connected by residual connections.

[0053] Resnet-34 was trained for 10 epochs using the Adam Optimizer with a maximum learning rate of 0.01. Gradient clipping was set to 0.1 and weight decay was set to 0.0001.

[0054] Adam Optimizer is an alternative optimization algorithm for stochastic gradient descent used to train deep learning models. Adam Optimizer combines the best properties of AdaGrad and RMSProp algorithms to provide an optimization algorithm capable of handling sparse gradients in noisy problems.

[0055] Gradient clipping is a technique for handling exploding gradients. The concept of gradient clipping is very simple. If the gradient becomes too large, it scales to keep it small.

[0056] Weight decay or normalization is a normalization technique applied to the weights of a neural network. It minimizes the loss function that impairs both the first-order loss function and the penalty to the weight's norm.

[0057] Figure 3 shows the approximate structure of the neural network based on the artificial intelligence model of the present invention and the conventional VGG19 artificial intelligence model-based neural network, where the 34-layer residual is the neural network model used in the present invention.

[0058] Through residual connections as described above, a plurality of residual blocks may be formed within the neural network model of the present invention, and at least one residual block may receive the output value of another residual block as an input value.

[0059] For example, if a neural network model includes two residual blocks and the respective blocks existing sequentially from the input layer are called the first residual block and the second residual block, the second residual block can receive the output value of the first residual block as an input value.

[0060] In a deep structured convolutional neural network-based model, the residual block may have a structure that adds a type of skip connection (or shortcut connection) to the existing stack structure.

[0061] Figure 5 shows the form of a residual block in which the input and output of a weight layer are connected through an ID connection.

[0062] In the residual block, if the input is x, the data to be finally learned is H(x), and H(x) is F(x) + x, which can be expressed as F(x) = H(x) - x.

[0063] Therefore, the residual block can obtain the result F(x) originally intended for learning by learning the residuals between the input and the output H(x) stacked through the weight layer, and the training of a convolutional neural network-based model having a residual learning block structure that only requires learning residuals can be referred to as residual learning.

[0064] In the present invention, each of the plurality of residual blocks in the neural network learning model may each include a plurality of layer arrays, and the layer arrays may include a final layer.

[0065] By applying residual learning as described above, when a convolutional neural network-based model learns breast cancer histopathology images input to a convolutional layer, it can learn solely from the difference values ​​between the existing parameters of the convolutional layer and the parameters newly generated from the input breast cancer histopathology images.

[0066] By applying residual learning as described above, the neural network learning model of the present invention can maintain constant gradient values ​​through connections between convolutional layers, and when gradients are reversed to learn weights during model training, gradients are not lost due to loss between layers, thereby improving the performance and discrimination accuracy of the breast cancer histopathology image discrimination system.

[0067] In the present invention, the method for training a neural network model is gradient descent, which calculates the gradient of a function and continuously moves toward the side with the lower gradient, repeating this process until an extremum (optimal value) is reached.

[0068] Gradient Descent is a weight learning method that measures the change in weights of a convolutional layer in response to a change in error and backpropagates the gradient from the final layer to the final layer in multiple convolutional layers.

[0069] Although neural network models have multiple layers, the gradient values ​​can continue to decrease while measuring weights and can become zero when reaching the input layer; this is called the vanishing gradient.

[0070] Since the ResNet-34 model has ID connections between weight layers, residuals from multiple layers are retrieved during gradient backpropagation and used in subsequent layers, allowing gradient values ​​to be maintained consistently across layers and improving the overall performance of the neural network model.

[0071] On the other hand, other neural network models such as VGG16 and Patch-Based Classifier (PBC) lack ID connections between weight layers, so gradient values ​​decrease during model training, which can degrade the overall performance of the neural network model.

[0072] In the step of applying the learned model to the dataset (S16), the source code for training the neural network model is code written in the Python language of Google Colab.

[0073] Figure 6 is part of an algorithm that fine-tunes the final layer with a discriminator (ImageClassifiactionBase) and then feeds it into a pre-trained model (BreastCancerResnet34).

[0074] The detailed process for performing the step (S16) of applying a trained model to a dataset is described as follows: a step of importing a Python library; a step of linking Google Colab with a cloud repository; a step of importing a dataset from the cloud repository; a step of splitting and storing the dataset into a training set, a test set, and a validation set; a step of adjusting the batch size; a step of defining an F-score algorithm; a step of importing an image discrimination algorithm including SMOTE oversampling; a step of importing a Resnet-34 neural network model for breast cancer diagnosis; a step of replacing the final layer and fixing the remaining layers including the convolutional layer; a step of evaluating the neural network model; a step of setting the number of epochs and the training time; a step of training the final layer while fixing the parameters stored in the remaining layers including the convolutional layer; and a step of unlocking the fixed parameters and training the model again.

[0075] Instead of training the entire neural network model from the beginning, time can be saved and model performance improved by fixing the initial layers and training only the last few layers with the dataset. Specifically, the transfer learning is performed by fixing the convolutional layers in the trained neural network model and training only the final layers.

[0076] Figure 7 is a flowchart illustrating the transfer learning process of the learning model. The final layer of ResNet-34 determines the output of the discriminant model based on an image dataset. As shown in Figure 7, only the final layer is fine-tuned by replacing it with a sigmoid activation function, while the remaining four convolutional layers are fixed. Using transfer learning, pre-trained features are extracted from all convolutional layers of ResNet-34, the layer where extraction is complete is fixed, fine-tuning is applied to the final layer, and then the entire model is unfixed to determine the output.

[0077] Transfer learning is a learning method that applies variables (e.g., weight values) of another artificial neural network that has already been trained to the corresponding artificial neural network. That is, during the initial installation process of the system, transfer learning using an artificial neural network of another system according to an embodiment of the present invention may be performed depending on a user's command or the satisfaction of given conditions.

[0078] Python libraries can be software such as TensorFlow, PyTorch, Keras, or Theano.

[0079] TensorFlow refers to open-source software for deep learning and machine learning technologies released by Google. PyTorch is an open-source machine learning library for Python based on Torch, used for applications such as natural language processing. It is significantly fast because it supports GPU usage.

[0080] Keras is an open-source neural network library written in Python. It can be run on MXNet, Deeplearning4j, TensorFlow, Microsoft Cognitive Toolkit, or Theano. It is designed to enable rapid experimentation with deep neural networks and focuses on minimal modular extensibility. By including deep learning libraries such as TensorFlow, Theano, and CNTK, it allows for the easy construction of multi-layer perceptron neural network models, convolutional neural network models, recurrent neural network models, combinatorial models, and more.

[0081] Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions related to multidimensional arrays.

[0082] The dataset is divided into a training set, a test set, and a validation set and stored. The training dataset is used to train the neural network, the validation dataset is used to tune hyperparameters such as the number of hidden layers, and the test dataset is used to finally evaluate the performance of the neural network.

[0083] The breast cancer histopathology image discrimination and output step (S17) is performed in an image discrimination module, and is a step of discriminating a breast cancer histopathology image dataset to which a neural network model is applied through transfer learning into two classes of normal or invasive breast cancer, and applying it to a software application to output the discrimination result.

[0084] FIG. 9 is a diagram showing a software application that is actually being used by implementing the breast cancer histopathology image algorithm of the present invention.

[0085] Specifically, the software application can be designed with individual modules for blot normalization, oversampling, and cancer identification, and the user can import breast cancer histopathology images into the application using the “Import” tab to perform image identification, and obtain the output of the result in the format of the identified image with an image ID from the “Result” tab.

[0086] Figure 10 is an output image obtained through the artificial intelligence model-based breast cancer histopathology image identification method of the present invention. The image on the left is an image of normal tissue, and the image on the right is an image of invasive breast cancer tissue.

[0087] After the step of identifying and outputting breast cancer histopathology images, a verification and evaluation step may be additionally included.

[0088] Figure 8 is a diagram showing the confusion matrix of an embodiment of the present invention using SMOTE and a Resnet-34 model and a comparative example using a Resnet-34 model without SMOTE application, respectively, after evaluating the discrimination results.

[0089] The confusion matrix is ​​a matrix used to evaluate the prediction results of a discriminant model and consists of True Positives (TP), False Positives (FP), False Negatives (FN), and True Negatives (TN) as components.

[0090] A true positive is when a breast cancer image is identified as positive, a false positive is when a normal image is identified as positive, a false negative is when a breast cancer image is identified as negative, and a true negative is when a normal image is identified as negative.

[0091] Confusion matrices can be used to calculate discriminant model performance metrics such as accuracy, precision, recall, and F-score. Confusion matrices are a very useful tool for visualizing and evaluating the performance of discriminant models.

[0092] Accuracy is the most intuitive performance metric, representing the ratio of the sum of true positives and true negatives to the total number of discriminants. If the dataset is unbalanced, accuracy may be poor.

[0093]

[0094] Precision is the ratio of true positives to the total number of discriminants classified as positive. The closer the precision is to 1, the better the discriminator. Precision becomes 1 only when the numerator and denominator are identical, which means that the number of false positives is 0.

[0095]

[0096] Recall, also known as sensitivity or the ratio of true positives, can be defined as the number of true positives divided by the sum of true positives and false negatives. A recall closer to 1 indicates a better discriminator. Recall becomes 1 only when the numerator and denominator are identical, which means that the number of false negatives is 0.

[0097]

[0098] The F-score is a measurement criterion that considers both precision and recall as described above, and is defined as the harmonic mean of precision and recall. When both false positives and false negatives are 0, the F-score is 1.

[0099]

[0100] The accuracy, loss, and learning rate of the model implementing the breast cancer histopathology image algorithm of the present invention allow one to observe trends in model performance throughout the training based on epochs. Here, an epoch is a hyperparameter that defines the number of times a learning algorithm is executed through the entire training dataset, and the neural network model of the present invention can have up to 10 epochs.

[0101] In the present invention, the F-score may appear consistently in the range of 0.5 to 0.6 in 8 to 10 epochs. If the F-score appears consistently when the number of epochs is less than 8, the recall of the neural network model's data processing may decrease, and if the F-score appears consistently when the number of epochs exceeds 10, it means that the neural network model's data processing speed is lowered and the model performance has deteriorated.

[0102] In addition, in the present invention, the F-score may appear consistently within the range of 0.5 to 0.6 in the range of 8 to 10 epochs. If the F-score is less than 0.5, there may be a problem with the accuracy of the neural network model's data processing, and if it exceeds 0.6, the recall of the neural network model's data processing may decrease.

[0103] In the present invention, the loss may converge to 0 in 7 to 10 epochs. If the loss converges to 0 in less than 7 epochs, the recall of data processing of the neural network model may decrease, and if the loss converges to 0 in more than 10 epochs, the speed of data processing of the neural network model may decrease, and a phenomenon may occur in which the model performance deteriorates.

[0104] Before 8 epochs, the loss was high because the validation dataset attempted to adapt to the neural network model, but after 8 epochs, the over-fitting factor was removed and the loss decreased to 0.

[0105] In the present invention, the maximum value of the learning rate may appear at a batch size of 14 to 20. When the batch size is within the above range, the data processing speed of the neural network model is optimized, and the time required to reach the maximum value can be shortened as much as possible.

[0106] Specific embodiments are provided below to aid in understanding the present invention. However, the following embodiments are provided merely to facilitate a better understanding of the invention, and the scope of the invention is not limited by these embodiments.

[0107] <Measures of accuracy, precision, recall, F-score, false positive, and false negative for neural network models>

[0108] [Table 1] below shows the datasets and neural network models used for the Resnet-34 training model with SMOTE oversampling and the existing neural network model without oversampling.

[0109] The existing neural network model is VGG16, Image Wise Classification.

[0110] Here, Resnet-34 with SMOTE oversampling performed together was designated as Example 1, and Resnet-34 without SMOTE oversampling, VGG16, and Image Wise Classification were designated as Comparative Examples 1 to 3, respectively. All images input to the training models were in TIFF format with a high resolution of 2048X1536, and the discrimination performance of each deep learning framework was evaluated using 200 images with 3.61GB of RAM.

[0111] division data set Neural network model Example 1 ICIAR Breast Cancer Dataset Resnet-34 - with SMOTE Sampling Technique Comparative Example 1 ICIAR Breast Cancer Dataset Resnet-34 Comparative Example 2 Breakhis Dataset VGG16 Comparative Example 3 ICIAR Breast Cancer Dataset Image Wise Classification

[0112] Accuracy, precision, recall, and F-score were measured based on the neural network models and datasets of Example 1 and Comparative Examples 1 to 3 above, and the results are as shown in [Table 2] below.

[0113] division Accuracy Precision Recall rate F-score Example 1 96.5% 92% 100% 96% Comparative Example 1 94% 94% 94% 94% Comparative Example 2 92.4% 61.56% 62.40% 61.55% Comparative Example 3 92.5% 92.59 92.39% 92.49

[0114] In the example of the breast cancer histopathology image identification method of the present invention, a 100% reproducibility rate was obtained, which means that the false negative is 0.

[0116] Measuring the processing time of neural network models

[0117] The processing time required to identify image data under the conditions of Example 1 and Comparative Examples 1 to 3 above was measured, and the results are as shown in [Table 3] below.

[0118] Example 1 Comparative Example 1 Comparative Example 2 Comparative Example 3 Processing time 5 minutes 10 minutes 8 minutes 11 minutes

[0119] As seen in [Table 3] above, when 200 TIFF format images with a high resolution of 2048X1536 are identified using 3.61GB RAM, the time required for processing based on the neural network model of Example 1 is 5 minutes, which confirms that high-resolution TIFF format images are processed quickly compared to cases processed based on the neural network models of Comparative Examples 1 to 3.

Claims

Claim 1 A step of inputting a breast cancer histopathology image dataset in TIFF format with a pixel size of 2048x1536; a step of stain normalizing the breast cancer histopathology image dataset using the Vahadane stain normalization algorithm; A step of converting the blot-normalized image dataset into patches with a pixel size of 512x512; a step of calculating the imbalance ratio of the converted image dataset and determining whether there is imbalance based on a threshold ratio, and oversampling using the Synthetic Minority Oversampling Technique (SMOTE) to improve the imbalance of minority class data only when the imbalance ratio exceeds the threshold ratio; a step of extracting features from a pre-trained ResNet-34 model using transfer learning; a step of extracting features using transfer learning with the ResNet-34 from which all features have been extracted, wherein the convolutional layer is fixed and the final layer is replaced with a sigmoid function activation to fine-tune the final layer as a discriminator; a step of applying the fine-tune ResNet-34 model to the oversampled image dataset, wherein the dataset is divided into a training set, a validation set, and a test set to perform training and validation; and a step of determining breast cancer histopathology images and outputting them as normal or invasive breast cancer.An AI-based breast cancer histopathology image discrimination method comprising: a step of stain normalization using the Vahadane stain normalization algorithm, wherein the step of generating a stain density map to change image color while preserving tissue structure, setting stains of a target image according to stain details of the stain density map, and then generating a normalized stain image by combining the stains of the stain density map with the set stains of the target image; the fine-tuning step of training only the final layer while fixing the convolutional layers of a pre-trained ResNet-34 model, and then retraining the entire model by releasing the fixation of the fixed convolutional layers; and in the step of performing the training and verification, the epoch is set to a range of 8 to 10 and the batch size is set to a range of 14 to 20. Claim 2 delete Claim 3 A method for determining a breast cancer histopathology image according to claim 1, wherein a verification and evaluation step following the step of determining and outputting the breast cancer histopathology image is additionally included. Claim 4 In paragraph 3, the breast cancer histopathology image discrimination method comprises, in the verification and evaluation step, evaluating discriminant values ​​to obtain a confusion matrix consisting of true positives, true negatives, false positives, and false negatives, and using the confusion matrix to measure accuracy, precision, recall, and F-score.

Images