Artificial intelligence-based analysis apparatus and method for predicting lymph node metastasis by using lymphovascular invasion on basis of digital pathology

Abstract

Disclosed are an artificial intelligence-based analysis apparatus and method for predicting lymph node metastasis by using lymphovascular invasion (LVI) on the basis of digital pathology, wherein a method for training an artificial intelligence-based determination model for predicting lymph node metastasis by using lymphovascular invasion (LVI) on the basis of digital pathology, according to an embodiment of the present application, may comprise the steps of: (a) collecting primary training data including a plurality of slide images in which a state related to at least one of LVI and lymph node metastasis (LNM) is labeled; (b) using the primary training data to train an invasion determination model for analyzing the risk of LVI with respect to an input target slide image; (c) preparing secondary training data by selecting the plurality of slide images included in the primary training data, on the basis of the risk derived using the invasion determination model; and (d) using the secondary training data to train a metastasis determination model for analyzing the risk of lymph node metastasis with respect to the input target slide image.

Description

AI-based analysis device and method for predicting lymph node metastasis using lymphovascular invasion based on digital pathology

[0001] The present invention relates to an artificial intelligence-based analysis device and method for predicting lymph node metastasis using lymphovascular infiltration based on digital pathology.

[0002]

[0003] Gastric cancer is one of the leading causes of cancer-related deaths worldwide. Lymph node metastasis (LNM) is a prognostic factor that significantly influences staging, treatment, and prognosis. Evaluating the extent of cancer spread to the lymph nodes after surgery plays a crucial role in selecting adjuvant chemotherapy and improving patient prognosis.

[0004] Although imaging diagnostic methods such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), ultrasound, and Positron Emission Tomography (PET) scans are non-invasive, pathological examination of resected lymph nodes remains the gold standard because of their low sensitivity in evaluating early metastases or micrometastasis in small lymph nodes. The current standard for lymph node evaluation is pathological examination using hematoxylin-eosin (H&E) stained pathology slides, which are often supplemented by immunohistochemistry. However, the process of evaluating individual pathology slides requires significant expertise, is repetitive, and is time-consuming.

[0005] In this regard, the development of more efficient and objective methods for lymph node evaluation is crucial, and digital pathology can be considered a promising alternative to address this. Digital pathology is a technique that digitizes pathology slides and analyzes them using computer algorithms, and research such as the CAMELYON project has contributed to the advancement of algorithms for detecting lymph node metastasis. Although this project focused primarily on breast cancer, its success has inspired the application of similar approaches to other cancers, including gastric cancer.

[0006] Lymphovascular invasion (LVI) refers to the invasion of lymphatic or vascular channels by a tumor and has been identified as a risk factor for lymph node metastasis in various cancers. However, research on the impact of LVI on lymph node metastasis in gastric cancer and the integration of this information into predictive models is not yet fully complete. Extracting detailed morphological features, such as LVI, through digital pathology and integrating them into AI-based predictive models is a critical task.

[0007] The technology forming the background of the present invention is disclosed in Korean Registered Patent Publication No. 10-2208613.

[0008]

[0009] The present invention aims to solve the problems of the aforementioned conventional technology by providing an artificial intelligence-based analysis device and method that predicts lymph node metastasis using lymphovascular invasion based on digital pathology, which can predict lymph node metastasis in gastric cancer patients more efficiently and objectively.

[0010] The present invention aims to solve the problems of the aforementioned conventional technology by providing an artificial intelligence-based analysis device and method for predicting lymph node metastasis using lymphovascular invasion based on digital pathology, which can improve the accuracy of lymph node metastasis prediction by utilizing a biological indicator called lymphovascular invasion.

[0011] However, the technical problems that the embodiments of the present invention aim to solve are not limited to the technical problems described above, and other technical problems may exist.

[0012]

[0013] As a technical means for achieving the above-mentioned technical problem, a method for training an artificial intelligence-based judgment model that predicts lymph node metastasis using lymphovascular invasion based on digital pathology according to one embodiment of the present invention may include: (a) collecting primary training data including a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LMN); (b) training an invasion judgment model that analyzes the risk of lymphovascular invasion for an input target slide image using the primary training data; (c) preparing secondary training data by selecting the plurality of slide images included in the primary training data based on the risk derived using the invasion judgment model; and (d) training a metastasis judgment model that analyzes the risk of lymph node metastasis for an input target slide image using the secondary training data.

[0014] In addition, step (c) above may select slide images with a risk level greater than or equal to a preset threshold value as the secondary training data.

[0015] In addition, the above-mentioned transition decision model may be a CLAM (Clustering-constrained Attention Multiple Instance Learning) model built using a multiple instance learning method.

[0016] Additionally, the above step (a) may include a step of performing preprocessing to divide at least some of the plurality of slide images into a plurality of patch images of a preset size.

[0017] In addition, the above infiltration judgment model may be a ResNet-18-based deep neural network model.

[0018] Additionally, the above step (a) can generate the plurality of slide images by digitizing the H&E (Hematoxylin and Eosin) stained tissue slides.

[0019] In addition, step (b) above can train the infiltration judgment model to determine whether the lymphovascular infiltration has occurred through binary classification.

[0020] In addition, step (c) above can select slide images in which lymphovascular infiltration is determined to be positive using the infiltration determination model as secondary training data.

[0021] Meanwhile, an artificial intelligence-based judgment method for predicting lymph node metastasis using lymphovascular invasion based on digital pathology according to one embodiment of the present invention may include: (a) collecting primary training data including a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LNM); (b) training an invasion judgment model that analyzes the risk of lymphovascular invasion for an input target slide image using the primary training data; (c) preparing secondary training data by selecting the plurality of slide images included in the primary training data based on the risk derived using the invasion judgment model; (d) training a metastasis judgment model that analyzes the risk of lymph node metastasis for an input target slide image using the secondary training data; and (e) acquiring the target slide image and inputting the target slide image into the metastasis judgment model to evaluate the risk of lymph node metastasis.

[0022] Meanwhile, a learning device for an artificial intelligence-based judgment model that predicts lymph node metastasis using lymphovascular invasion based on digital pathology according to one embodiment of the present invention may include: a data collection unit that collects primary learning data including a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LNM); a first learning unit that learns an invasion judgment model that analyzes the risk of lymphovascular invasion for an input target slide image using the primary learning data; a data selection unit that prepares secondary learning data by selecting the plurality of slide images included in the primary learning data based on the risk derived using the invasion judgment model; and a second learning unit that learns a metastasis judgment model that analyzes the risk of lymph node metastasis for an input target slide image using the secondary learning data.

[0023] In addition, the data selection unit can select slide images in which the risk level is greater than or equal to a preset threshold value as the secondary training data.

[0024] In addition, a learning device for an artificial intelligence-based judgment model that predicts lymph node metastasis using lymphovascular infiltration based on digital pathology according to one embodiment of the present invention may include a preprocessing unit that performs preprocessing by dividing at least a portion of the plurality of slide images into a plurality of patch images of a preset size.

[0025] In addition, the data collection unit can digitize H&E (Hematoxylin and Eosin) stained tissue slides to generate the plurality of slide images.

[0026] In addition, the first learning unit can train the infiltration judgment model to determine whether the lymphovascular infiltration has occurred through binary classification.

[0027] In addition, the data selection unit can select slide images in which lymphovascular infiltration is determined to be positive using the infiltration judgment model as secondary training data.

[0028] Meanwhile, an artificial intelligence-based judgment device for predicting lymph node metastasis using lymphovascular invasion based on digital pathology according to one embodiment of the present invention may include: a data collection unit that collects primary training data including a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LNM); a first learning unit that trains an invasion judgment model analyzing the risk of lymphovascular invasion for an input target slide image using the primary training data; a data selection unit that prepares secondary training data by selecting the plurality of slide images included in the primary training data based on the risk derived using the invasion judgment model; a second learning unit that trains a metastasis judgment model analyzing the risk of lymph node metastasis for an input target slide image using the secondary training data; and an analysis execution unit that acquires the target slide image and inputs the target slide image into the metastasis judgment model to evaluate the risk of lymph node metastasis.

[0029] The means for solving the problem described above are merely exemplary and should not be interpreted as intended to limit the present invention. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and the detailed description of the invention.

[0030] According to the solution to the problem of the present invention described above, it is possible to provide an artificial intelligence-based analysis device and method that predicts lymph node metastasis using lymphovascular invasion based on digital pathology, which can predict lymph node metastasis in gastric cancer patients more efficiently and objectively.

[0031] According to the solution to the problem of the present invention described above, it is possible to provide an artificial intelligence-based analysis device and method for predicting lymph node metastasis using lymphovascular infiltration based on digital pathology, which can improve the accuracy of lymph node metastasis prediction by utilizing a biological indicator called lymphovascular invasion.

[0032] According to the solution to the problem of the present invention described above, by first analyzing the risk of lymphovascular invasion and then predicting lymph node metastasis based on this, a more accurate and efficient prediction of lymph node metastasis can be achieved.

[0033] According to the solution to the problem of the present invention described above, biological knowledge can be effectively reflected in an artificial intelligence model through a two-stage learning structure, so excellent predictive performance can be achieved even with a small amount of training data.

[0034] According to the aforementioned means for solving the problem of the present institution, the workload of pathologists can be reduced, and the objectivity and consistency of lymph node metastasis evaluation can be improved.

[0035] However, the effects obtainable from this invention are not limited to those described above, and other effects may exist.

[0036] FIG. 1 is a schematic diagram of an artificial intelligence-based lymph node metastasis prediction system according to one embodiment of the present invention.

[0037] FIG. 2 is a conceptual diagram showing a two-stage learning framework for predicting lymph node metastasis according to one embodiment of the present invention.

[0038] FIG. 3 is a conceptual diagram illustrating a feature selection process using a lymphovascular infiltration risk score according to one embodiment of the present invention.

[0039] FIG. 4 is a diagram exemplifying the performance of a lymphovascular infiltration classification model according to one embodiment of the present invention.

[0040] Figure 5 is a graph showing a comparison of performance according to a change in the number of features according to one embodiment of the present invention.

[0041] FIG. 6 is a diagram exemplarily showing the results of a cross-analysis between the risk of lymphovascular infiltration and the lymph node metastasis caution score according to one embodiment of the present invention.

[0042] FIG. 7 is a diagram exemplarily showing the correlation between the probability value of a lymphovascular infiltration classification model and the attention score of a lymph node metastasis classification model according to one embodiment of the present invention.

[0043] FIG. 8 is a diagram illustrating exemplary patch images to which a lymph node metastasis classification model according to one embodiment of the present invention has assigned a high attention score.

[0044] FIG. 9 is a schematic diagram of an artificial intelligence-based judgment device that predicts lymph node metastasis using lymphovascular infiltration based on digital pathology according to one embodiment of the present invention.

[0045] FIG. 10 is a flowchart of an artificial intelligence-based judgment method for predicting lymph node metastasis using lymphovascular infiltration based on digital pathology according to one embodiment of the present invention.

[0046] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0047] Throughout this specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" or "indirectly connected" with other elements interposed between them.

[0048] Throughout the entire specification, when a component is described as being located "on," "on top," "on top," "under," "on bottom," or "on bottom" of another component, this includes not only cases where the component is in contact with the other component but also cases where another component exists between the two components.

[0049] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0050] The present invention relates to an artificial intelligence-based analysis device and method for predicting lymph node metastasis using lymphovascular infiltration based on digital pathology.

[0051] FIG. 1 is a schematic diagram of an artificial intelligence-based lymph node metastasis prediction system according to one embodiment of the present invention.

[0052] Referring to FIG. 1, an artificial intelligence-based lymph node metastasis prediction system (10) according to one embodiment of the present invention may include an artificial intelligence-based judgment device (100) (hereinafter referred to as 'judgment device (100)') that predicts lymph node metastasis using lymphovascular infiltration based on digital pathology according to one embodiment of the present invention, a scanning device (200), a user terminal (300), and a database (400).

[0053] The judgment device (100), scanning device (200), user terminal (300), and database (400) can communicate with each other through a network (20). The network (20) refers to a connection structure that enables information exchange between each node, such as terminals and servers. Examples of such a network (20) include, but are not limited to, a 3GPP (3rd Generation Partnership Project) network, an LTE (Long Term Evolution) network, a 5G network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Wi-Fi network, a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, and a DMB (Digital Multimedia Broadcasting) network.

[0054] The user terminal (300) can be any type of wireless communication device, such as a smartphone, smartpad, tablet PC, PCS (Personal Communication System), GSM (Global System for Mobile communication), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal.

[0055] In the description of the embodiments of the present invention, the scanning device (200) may be a device for converting H&E stained tissue slides into digital images. For example, the scanning device (200) may be a high-resolution digital pathology scanner such as a Leica-Aperio GT450 scanner, and may generate a digitized whole slide image by scanning a tissue slide at 40x magnification. The generated digital slide image may be transmitted to a judgment device (100) and utilized for further processing and analysis.

[0056] Additionally, in the description of the embodiments of the present invention, the database (400) may be a device or server for storing and managing various data related to the operation of the judgment device (100). Specifically, the database (400) may be a storage facility for storing and managing digital slide images obtained from the scanning device (200), learning parameters of the infiltration judgment model and metastasis judgment model, risk analysis results for lymphovascular infiltration (LVI) and lymph node metastasis (LNM), etc.

[0057] FIG. 2 is a conceptual diagram showing a two-stage learning framework for predicting lymph node metastasis according to one embodiment of the present invention.

[0058] Referring to FIG. 2, the judgment device (100) disclosed herein can predict lymph node metastasis through four main steps. First, in the preprocessing step, H&E stained slides can be digitized and divided into patch images of a predetermined size (e.g., 512x512 pixels). Second, in the LVI classification step, the risk of lymphovascular invasion of each patch image can be evaluated through a ResNet-18-based invasion judgment model. Third, in the LNM classification step, patches classified as high-risk LVI groups using the invasion judgment model can be selected as secondary training data to train a metastasis judgment model based on a CLAM model. Finally, in the interpretation step, the correlation between the two models can be analyzed and the prediction results can be interpreted.

[0059] Below, the specific functions and operations of the judgment device (100) will be described in detail.

[0060] First, the judgment device (100) can collect primary learning data including a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LMM).

[0061] For example, the judgment device (100) can digitize H&E (Hematoxylin and Eosin) stained tissue slides to generate multiple slide images.

[0062] Additionally, the judgment device (100) can perform preprocessing to divide at least some of the collected slide images into a plurality of patch images of a preset size.

[0063] Meanwhile, according to one embodiment of the present invention, the judgment device (100) can independently collect a dataset labeled with LVI-related states (LVI Dataset) and a dataset labeled with LNM-related states (LNM Dataset).

[0064] For example, the LVI dataset may be collected from slides of patients diagnosed as LVI positive, and may include all slides except for one slide per patient. In this case, the LVI dataset may be divided into training, validation, and test sets according to a first, second, and third ratio, and patient-unit partitioning may be applied to avoid slide duplication. In the embodiments of the present invention, the LVI dataset may consist of positive slides with a first threshold value and negative slides with a second threshold value.

[0065] Meanwhile, the LNM dataset is collected independently of the LVI, and a preset number of cross-validations may be applied. For example, the LNM dataset may be configured to include positive slides of a third threshold value and negative slides of a fourth threshold value.

[0066] Additionally, the judgment device (100) can use primary training data to which collection and preprocessing have been applied to train an infiltration judgment model that analyzes the risk of lymphovascular infiltration (LVI) for a corresponding slide image (target slide image) when a slide image is input.

[0067] For example, the judgment device (100) can train an infiltration judgment model to determine whether lymphovascular infiltration (LVI) has occurred (e.g., determined as 'positive' or 'negative') by performing binary classification on an input target slide image.

[0068] In addition, according to one embodiment of the present invention, the judgment device (100) can train an infiltration judgment model corresponding to a ResNet-18-based deep neural network model type.

[0069] The following describes in detail the process of training the infiltration judgment model.

[0070] In other words, the invasion determination model may be a neural network model for determining lymphovascular invasion, etc., from digital pathology images such as whole slide images (WSI) related to biological tissues, and according to one embodiment of the present invention, the invasion determination model may be a neural network model for determining lymphovascular invasion to observe the prognosis of cancer patients, but is not limited thereto. As another example, the invasion determination model is a neural network model for determining invasion of various pathologies, and can be widely applied to neural network models for determining various tissue invasions and various tissue invasions.

[0071] For example, an artificial intelligence algorithm for determining infiltration can be formed by dividing it into multiple neural network models. For instance, an artificial intelligence algorithm for an infiltration determination program can be formed by dividing it into a classification model that classifies acquired images and a detection (or object detection) model that determines and / or detects infiltration from the classified images.

[0072] To explain in more detail, the classification model may be composed of at least some of various models based on Convolutional Neural Networks (CNNs), such as the ResNet model, the EfficientNet model, and the ConViT model. Additionally, the detection model may be composed of at least some of the YOLO model and the YOLOX model.

[0073] In order to train such an infiltration judgment model, the judgment device (100) can first acquire a plurality of full slide images in which at least one state is labeled in relation to lymphovascular infiltration.

[0074] For example, the judgment device (100) can acquire at least one whole slide image as a digital pathology image related to lymphovascular infiltration from the scanning device (200). The judgment device (100) can organize and store the received whole slide image as a dataset.

[0075] According to one embodiment of the present invention, the entire slide image acquired by the judgment device (100) may be configured to include an image labeled based on at least some of three states related to infiltration.

[0076] For example, the image acquired by the judgment device (100) may be configured to include an image in which the state of the tissue is classified and labeled as positive (+) (hereinafter positive), negative (-) (hereinafter negative), and normal.

[0077] Here, a positive image (e.g., a whole slide image labeled as having a positive condition, or a whole slide image positive for lymphovascular infiltration) may be an image labeled as having lymphovascular infiltration in a tumor (e.g., tumor cells and / or tumor tissue, hereinafter tumor).

[0078] Additionally, a negative image (e.g., a full slide image labeled as negative, or a full slide image negative for lymphovascular invasion) may be an image labeled as not having lymphovascular invasion in the tumor.

[0079] Additionally, the normal image (e.g., a full slide image labeled with normal status) is an image of normal tissue where no tumor is identified, and may be an image labeled as normal tissue including an epithelial layer, a muscle layer, and / or a fat layer.

[0080] Meanwhile, the positive image may be an image containing a benign tumor and additionally containing at least one negative tumor. The positive image may be labeled for at least some of the location, shape, boundaries, and size of at least one tumor and / or tissue, and may be labeled as a positive state (and / or area of ​​the positive state), a negative state (and / or area of ​​the negative state), or a normal state (and / or area of ​​the normal state) for at least some of the labeled tumor and / or tissue.

[0081] Additionally, the negative image may include at least one negative tumor, and may be labeled for at least some of the location, shape, boundaries, and size of at least one tumor and / or tissue, and may be labeled as negative or normal for at least some of the labeled tumor and / or tissue.

[0082] Meanwhile, according to one embodiment of the present invention, a normal state in a positive image and / or a negative image may be provided as an unlabeled state. In this case, the judgment device (100) may identify the remaining state, excluding the positive state and the negative state in the acquired image, as a normal state.

[0083] Here, the labeled image may be an image containing information based on the actions of at least some of labeling, annotation, and commenting.

[0084] According to one embodiment of the present invention, the entire slide image received by the judgment device (100) may be provided as an image in a stained state, generated using the excised tissue of the patient after surgery.

[0085] For example, the entire slide image received by the judgment device (100) may include an entire slide image stained by at least one of various tissue staining methods, such as hematoxylin and eosin staining (H&E) and immunohistochemical staining (immunohistochemistry, IHC).

[0086] According to one embodiment of the present invention, the entire slide image obtained by the judgment device (100) may be an image in which at least some information among the tumor location, tumor boundary, tumor size, invasion state, invasion location, invasion boundary, and invasion size is labeled, in addition to the tumor state as described above.

[0087] Additionally, the judgment device (100) may generate a plurality of patch images having a first volume for each of the plurality of total slide images. For example, the judgment device (100) may generate patch images having a designated volume (e.g., a first volume) based on each of the acquired total slide images, and in the description of the embodiments of the present invention, the volume unit of the image may be indicated as a pixel, for example.

[0088] For example, one of the slide images can be divided (e.g., image segmentation) by the entire slide image to generate a specified number of patch images having a first volume. At this time, the judgment device (100) can perform image segmentation based on a convolutional neural network (CNN).

[0089] At this time, the judgment device (100) may set the first volume to be smaller than the volume of the entire slide image. However, not limited thereto, the judgment device (100) may also set the first volume to be larger than the volume of the entire slide image.

[0090] When the first volume is set to be larger than the volume of the entire slide image, the determining device (100) can generate a patch image according to the first volume after processing the volume of the entire slide image to be greater than or equal to the first volume when the volume of the entire slide image is smaller than the first volume.

[0091] At this time, the judgment device (100) can improve the resolution of the entire slide image based on at least some of the upscaling methods, such as bilinear interpolation, trilinear interpolation, and super resolution.

[0092] The judgment device (100) can set a label for each patch image based on the label of the original image (e.g., the entire slide image) when generating patch images.

[0093] For example, when the judgment device (100) generates patch images using a whole slide image that is positive for lymphovascular infiltration, it may set a positive label for patch images that contain at least some tumor and set a normal label for patch images that do not contain any tumor.

[0094] To explain in more detail, the entire slide image positive for lymphovascular infiltration may be in a state where tumors with confirmed infiltration and tumors without confirmed infiltration are mixed. In this case, the judgment device (100) may set a positive label for a patch image containing at least a portion of tumors with confirmed infiltration and set a negative label for a patch image containing at least a portion of tumors without confirmed infiltration.

[0095] At this time, the judgment device (100) distinguishes the boundaries of tumors with confirmed invasion, tumors with confirmed invasion, and / or tumors based on the labels set on the entire slide image, and can set the labels of the patch image based thereon.

[0096] Here, the judgment device (100) can set a label as the state of the patch image as the representative state of the entire slide image based on the label set on the entire slide image, in cases where the boundaries of the tumors are not distinguished as tumors with confirmed invasion, tumors without confirmed invasion, and / or tumors.

[0097] For example, the representative state of the entire slide image containing the tumor with invasion can be determined to be a benign state, the representative state of the entire slide image containing the tumor without invasion can be determined to be a negative state, and the representative state of the entire slide image not containing the tumor can be determined to be a normal state.

[0098] According to the above description, when the judgment device (100) generates patch images using a whole slide image that is negative for lymphovascular infiltration, it can set a negative label for patch images that contain at least some tumor and set a normal label for patch images that do not contain any tumor.

[0099] On the other hand, according to another embodiment, the judgment device (100) may not label the state of the patch image based on the setting information when the boundaries of the tumors are not distinguished, the tumor with confirmed invasion, the tumor with confirmed invasion, and / or the tumors are not distinguished based on the label set on the entire slide image.

[0100] When setting labels for the generated patch images, the judgment device (100) may set an LOI for each patch image based on the level of interest (LOI) of the original entire slide image.

[0101] The judgment device (100) can generate a plurality of training images having a second volume based on each of the plurality of patch images.

[0102] First, the judgment device (100) can distinguish between a training patch image set, a verification patch image set, and a test patch image set for the generated patch images.

[0103] At this time, the judgment device (100) can acquire patch images included in each patch image set at a ratio set for the images of the dataset. For example, the judgment device (100) can distinguish positive patch images, negative patch images, and normal patch images based on the labels of the patch images, and acquire a patch image set for each distinguished label.

[0104] According to the above description, the judgment device (100) can distinguish patch images according to labels and, based on the distinguished patch images, can distinguish them into a training patch image set, a verification patch image set, and a test patch image set.

[0105] However, it is not limited to this, and according to an embodiment of the present invention, the judgment device (100) may perform an operation of distinguishing between a patch image set for training, a patch image set for verification, and a patch image set for testing before performing an operation to generate a patch image.

[0106] Afterwards, the judgment device (100) can divide each of the generated multiple patch images to generate training images having a designated volume (e.g., a second volume).

[0107] For example, the judgment device (100) can divide each of the patch images of the training patch image set to generate training images having a second volume.

[0108] For example, one of the patch images for training, the decision device (100) can generate a specified number of training images by dividing (e.g., cropping) the training patch image to have a second volume. In other words, the decision device (100) can obtain a training dataset including the training images generated for the training patch images.

[0109] Subsequently, the decision device (100) may perform data augmentation when generating a training dataset. For example, the decision device (100) may generate training images that have undergone data augmentation processing based on at least some of a geometric transform method, a blurring method, a color transform method, and a normalization method.

[0110] More specifically, the judgment device (100) may apply at least some of various geometric transformation methods, such as random cropping, transpose, and horizontal / vertical flip, when applying a data augmentation method of geometric transformation to a training image.

[0111] In addition, when applying a data augmentation method for blurring to a training image, the judgment device (100) may apply at least some of various blurring methods, such as blur, Gaussian blur, Gaussian noise, and median blur.

[0112] In addition, when applying a data augmentation method for color conversion to a training image, the judgment device (100) may apply at least some of various color conversion methods, such as channel shuffle, color jitter, hue saturation value (HSV), and random brightness.

[0113] Additionally, according to one embodiment of the present invention, the judgment device (100) can perform data augmentation by applying an elastic transform method to a training image. However, if the data processing amount of the processing unit exceeds a preset value (reference value), the judgment device (100) may process by excluding the elastic transform method and applying at least some of the remaining data augmentation / decrease methods.

[0114] Additionally, the judgment device (100) may apply at least some data augmentation methods to a previously generated training image and / or generate another training image to which at least some data augmentation methods have been applied to the previously generated training image. Furthermore, the judgment device (100) may generate another training image by applying at least some data augmentation methods to a training image to which at least some data augmentation methods have been applied.

[0115] In summary, the judgment device (100) can construct an infiltration judgment model based on a plurality of collected training images (first training data).

[0116] At this time, the judgment device (100) may limit the types of labels used for training the infiltration judgment model. For example, the judgment device (100) may partially integrate three labels of positive, negative, and normal into two labels to distinguish between positive and background (e.g., integrating negative and normal labels).

[0117] In addition, the judgment device (100) can integrate and distinguish between labels for benign and other (e.g., background) conditions when there are four or more labels related to the condition of the tumor.

[0118] That is, the judgment device (100) can classify the labels of the training images into two types of labels, positive and other (e.g., background), and perform training of the infiltration judgment model using the positive label and the background label.

[0119] Meanwhile, regarding the hyperparameters applied during the learning process of the infiltration judgment model, the judgment device (100) may set the learning rate to, for example, 1e-4 and may use Focal Loss as the loss function. Additionally, the judgment device (100) may determine the final model based on the minimum validation loss during 10 epochs.

[0120] The judgment device (100) can train an infiltration judgment model by processing training images of a generated training dataset as input data. The judgment device (100) can train an infiltration judgment model by processing training images as input data, comparing the predicted output data with a pre-set label for each of the training images, and modifying at least a part of the algorithm of the infiltration judgment model.

[0121] Here, the judgment device (100) can perform the learning of the infiltration judgment model by distinguishing between a classification model and a detection model.

[0122] In relation to the learning of a classification model, the judgment device (100) can perform learning on the operation of determining whether there is infiltration in the training image based on training images and labels pre-set on the training images.

[0123] At this time, the judgment device (100) may use a backpropagation algorithm when performing training of the classification model.

[0124] In relation to the learning of the detection model, the judgment device (100) can perform learning on the operation of detecting the location where infiltration exists based on training images and labels pre-set on the training images. In addition, the judgment device (100) can perform learning on the operation of detecting not only the location where infiltration exists, but also the shape of the infiltration and / or the boundary of the infiltration. At this time, the judgment device (100) may use a region proposal algorithm when performing the learning of the detection model.

[0125] To this end, the judgment device (100) can resize each of the patch images of the generated verification patch image set to generate verification images having a second volume.

[0126] The judgment device (100) can perform data augmentation on resized verification images. According to one embodiment of the present invention, the judgment device (100) can generate resized verification images through a method identical or similar to the operation of performing data augmentation on training data. The judgment device (100) can generate a verification dataset including resized verification images.

[0127] Here, the validation dataset and / or test dataset are described as being generated in the operation validating the infiltration judgment model or the operation testing the infiltration judgment model.

[0128] However, not limited to this, the verification dataset and / or test dataset may be generated by the judgment device (100) before or after the operation of generating the training test set.

[0129] The judgment device (100) can verify an infiltration judgment model by processing verification images of a verification dataset as input data. In this process, the judgment device (100) can tune the infiltration judgment model through methods such as hyperparameter tuning, data acquisition, model architecture change, transfer learning, and ensemble.

[0130] At this time, the judgment device (100) can distinguish between a classification model and a detection model and perform tuning operations for each. Based on the tuning results, the judgment device (100) can additionally perform learning and / or detection operations for the infiltration judgment model.

[0131] When training and / or verification of the infiltration judgment model is performed, the judgment device (100) can resize each of the patch images of the generated test patch image set to generate test images having a second volume.

[0132] The judgment device (100) can perform data augmentation on resized test images. According to one embodiment of the present invention, the judgment device (100) can generate resized test images through a method identical or similar to the operation of performing data augmentation on training data. The judgment device (100) can generate a test dataset including resized test images.

[0133] The judgment device (100) can test an infiltration judgment model by processing test images of a test dataset as input data. In this process, the judgment device (100) can obtain test results for at least some of the accuracy, precision, recall, receiver judgment curve area, and precision-recall curve area of ​​the infiltration judgment model based on the test execution results and the labels set on the test images of the test dataset.

[0134] It can be confirmed that the test results of the infiltration detection models constructed based on Yoon v3, ConViT, and YoloX models were highly evaluated. In particular, regarding the test results of the infiltration detection models based on ConViT and YoloX models, it can be seen that the test results for items other than precision—such as accuracy, precision, recall, receiver judgment curve area, and precision-recall curve area—were evaluated higher than the test results of infiltration detection models constructed with other neural networks.

[0135] The judgment device (100) may additionally perform learning, verification, and / or testing operations of an infiltration judgment model based on test results. If the judgment device (100) satisfies (e.g., exceeds) a preset value for at least some of the test results, accuracy, precision, recall, receiver judgment curve area, and precision-recall curve area, the judgment device (100) may determine the infiltration judgment model as a learned infiltration judgment model.

[0136] Accordingly, the judgment device (100) can determine lymphovascular infiltration from the entire slide image based on the completed infiltration judgment model.

[0137] FIG. 3 is a conceptual diagram illustrating a feature selection process using a lymphovascular infiltration risk score according to one embodiment of the present invention.

[0138] Referring to FIG. 3, the judgment device (100) disclosed herein can prepare secondary training data by selecting from a plurality of slide images included in primary training data collected earlier, based on the risk level derived using a constructed infiltration judgment model.

[0139] For example, the judgment device (100) can select slide images from the primary training data that have a risk level greater than or equal to a preset threshold value evaluated using an infiltration judgment model and collect them as secondary training data.

[0140] As another example, if the judgment device (100) is a type of model that performs binary classification of the infiltration judgment model, it can select slide images in which lymphovascular infiltration (LVI) is determined to be positive using the infiltration judgment model as secondary training data.

[0141] As another example, the judgment device (100) can select secondary training data based on the ranking of output values ​​(probability values) derived from the infiltration judgment model. For example, the judgment device (100) can select the top N% or top K patches of LVI risk for each patch. Additionally, the judgment device (100) may set multiple threshold values ​​and apply selection criteria differentially according to the ratio of patches exceeding each threshold value. For example, all patches exceeding the first threshold value (e.g., 0.8, etc.) may be selected, and some of the patches between the second threshold value (e.g., 0.6, etc.) and the first threshold value may be randomly selected.

[0142] In particular, FIG. 3 illustrates in detail the feature selection process disclosed herein, illustrating a Bag-of-features selection process based on LVI risk scores and / or LVI classification results performed by a judgment device (100). According to one embodiment of the present invention, the judgment device (100) may select only the top K patches with high LVI risk from the entire collection of patch images and use them for learning. This selective learning method can enable more efficient and accurate prediction of lymph node metastasis than using all patches. In particular, the performance of the metastasis judgment model may vary depending on the K value, and the performance of the metastasis judgment model can be optimized by selecting an appropriate K value in consideration of this.

[0143] Next, the judgment device (100) can train a metastasis judgment model that analyzes the risk of lymph node metastasis (LNM) for an input target slide image using selected secondary training data.

[0144] In addition, according to one embodiment of the present invention, the decision device (100) can train a transfer decision model corresponding to the type of CLAM (Clustering-constrained Attention Multiple Instance Learning) model built in a multiple instance learning method.

[0145] Here, the CLAM model is a model with a specialized structure for multi-instance learning that may combine an attention mechanism and clustering constraints. The CLAM model calculates an attention score for each input patch image and, based on this, can comprehensively evaluate the lymph node metastasis risk of the entire slide image. In particular, in this institution, the learning efficiency and prediction accuracy of the model can be improved by selectively inputting patches with a high LVI risk.

[0146] Additionally, the judgment device (100) can acquire a target slide image to be analyzed. For example, the judgment device (100) can receive a newly captured slide image from the scanning device (200) as a target slide image.

[0147] Additionally, the judgment device (100) can input the acquired target slide image into a learned metastasis judgment model to evaluate the risk of lymph node metastasis (LNM) reflected in the target slide image.

[0148] Hereinafter, an experimental example linked to the judgment device (100) disclosed herein will be described with reference to FIGS. 4 to 8.

[0149] FIG. 4 is a diagram exemplifying the performance of a lymphovascular infiltration classification model according to one embodiment of the present invention.

[0150] Referring to the graph in Figure 4 showing the performance of the infiltration judgment model, the model achieved an AUROC of 0.745 and an accuracy of 0.654 (sensitivity: 0.667, specificity: 0.647) despite a limited number of positive images. In particular, when the decision threshold was set to 0.4, false results were minimized in both positive and negative cases, and reliable performance was demonstrated through a high true negative rate.

[0151] Figure 5 is a graph showing a comparison of performance according to a change in the number of features according to one embodiment of the present invention.

[0152] Referring to Figure 5, when comparing the performance of the model according to changes in the number of features (K), it was found that when only 10 instances with high LVI risk were selected for training, superior performance (AUROC 0.930) was achieved compared to when all instances were used (AUROC 0.890). This can be interpreted as a result proving the effectiveness of the selective learning approach.

[0153] FIG. 6 is a diagram exemplarily showing the results of a cross-analysis between the risk of lymphovascular infiltration and the lymph node metastasis caution score according to one embodiment of the present invention.

[0154] Referring to Figure 6, a significant positive correlation (R² = 0.246) was confirmed between the LVI risk score and the LNM attention score in positive samples. In contrast, the correlation was negligible (R² = 0.046) in negative samples. These results suggest that the LNM classifier effectively utilizes LVI information to predict lymph node metastasis.

[0155] FIG. 7 is a diagram exemplarily showing the correlation between the probability value of a lymphovascular infiltration classification model and the attention score of a lymph node metastasis classification model according to one embodiment of the present invention.

[0156] Referring to Figure 7, it can be observed that there is a monotonically increasing trend between the probability value of the infiltration classification model and the attention score of the lymph node metastasis classification model. This correlation may imply that the metastasis judgment model utilizes LVI information for learning in a direction consistent with biological knowledge, even without separate explicit instructions.

[0157] FIG. 8 is a diagram illustrating exemplary patch images to which a lymph node metastasis classification model according to one embodiment of the present invention has assigned a high attention score.

[0158] Referring to Figure 8, analysis of patch images to which the lymph node metastasis classification model assigned high attention scores revealed that typical morphological features of LVI were observed. These results suggest that the metastasis judgment model has actually learned important pathological patterns related to lymphovascular invasion, and can provide interpretability for the prediction results of the metastasis judgment model.

[0159] FIG. 9 is a schematic diagram of an artificial intelligence-based judgment device that predicts lymph node metastasis using lymphovascular infiltration based on digital pathology according to one embodiment of the present invention.

[0160] Referring to FIG. 9, the judgment device (100) may include a data collection unit (110), a preprocessing unit (120), a first learning unit (130), a data selection unit (140), a second learning unit (150), and an analysis unit (160).

[0161] The data collection unit (110) can collect primary training data including multiple slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LMN).

[0162] For example, the data collection unit (110) can digitize H&E (Hematoxylin and Eosin) stained tissue slides to generate multiple slide images.

[0163] The preprocessing unit (120) can perform preprocessing by dividing at least some of the collected slide images into multiple patch images of a preset size.

[0164] The first learning unit (130) can use primary learning data to which collection and preprocessing have been applied to train an infiltration judgment model that analyzes the risk of lymphovascular infiltration (LVI) for a corresponding slide image (target slide image) when a slide image is input.

[0165] For example, the first learning unit (130) can train an infiltration judgment model to determine whether lymphovascular infiltration (LVI) occurs (e.g., determined as 'positive' or 'negative') by performing binary classification on the input target slide image.

[0166] In addition, according to one embodiment of the present invention, the first learning unit (130) can train an infiltration judgment model corresponding to a ResNet-18-based deep neural network model type.

[0167] The data selection unit (140) can select and prepare secondary training data from among multiple slide images included in the primary training data collected earlier, based on the risk level derived using the learned infiltration judgment model.

[0168] For example, the data selection unit (140) can select slide images from the primary training data that have a risk level greater than or equal to a preset threshold value evaluated using an infiltration judgment model and collect them as secondary training data.

[0169] As another example, the data selection unit (140) can select slide images in which lymphovascular infiltration (LVI) is determined to be positive using the infiltration determination model as secondary training data when the infiltration determination model is a type of model that performs binary classification.

[0170] The second learning unit (150) can train a metastasis judgment model that analyzes the risk of lymph node metastasis (LNM) for an input target slide image using secondary learning data selected by the data selection unit (140).

[0171] In addition, according to one embodiment of the present invention, the second learning unit (150) can train a transfer decision model corresponding to the type of CLAM (Clustering-constrained Attention Multiple Instance Learning) model built in a multiple instance learning method.

[0172] The analysis execution unit (160) can acquire a target slide image to be analyzed. For example, the analysis execution unit (160) can receive a newly captured slide image from the scanning device (200) as a target slide image.

[0173] Additionally, the analysis execution unit (160) can input the acquired target slide image into a learned metastasis judgment model to evaluate the risk of lymph node metastasis (LNM) reflected in the target slide image.

[0174] Below, based on the details described above, we will briefly examine the operation flow of the present invention.

[0175] FIG. 10 is a flowchart of an artificial intelligence-based judgment method for predicting lymph node metastasis using lymphovascular infiltration based on digital pathology according to one embodiment of the present invention.

[0176] An artificial intelligence-based judgment method for predicting lymph node metastasis using lymphovascular infiltration based on digital pathology illustrated in FIG. 10 can be performed by the judgment device (100) described above. Therefore, even if the content is omitted below, the description of the judgment device (100) can be equally applied to the description of the artificial intelligence-based judgment method for predicting lymph node metastasis using lymphovascular infiltration based on digital pathology.

[0177] Referring to FIG. 10, in step S11, the data collection unit (110) can collect primary training data including a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LMN).

[0178] For example, in step S11, the data collection unit (110) can digitize H&E (Hematoxylin and Eosin) stained tissue slides to generate multiple slide images.

[0179] Additionally, according to one embodiment of the present invention, in step S11, the preprocessing unit (120) may perform preprocessing by dividing at least some of the collected slide images into a plurality of patch images of a preset size.

[0180] Next, in step S12, the first learning unit (130) can (b) train an infiltration judgment model that analyzes the risk of lymphovascular infiltration (LVI) for an input target slide image using the collected first learning data.

[0181] For example, in step S12, the first learning unit (130) can train an infiltration judgment model to determine whether lymphovascular infiltration (LVI) has occurred (e.g., determined as 'positive' or 'negative') by performing binary classification on the input target slide image.

[0182] According to one embodiment of the present invention, in step S12, the first learning unit (130) can train an infiltration judgment model corresponding to a ResNet-18-based deep neural network model type.

[0183] Next, in step S13, the data selection unit (140) can select and prepare secondary training data from among multiple slide images included in the primary training data collected earlier, based on the risk level derived using (c) the learned infiltration judgment model.

[0184] For example, in step S13, the data selection unit (140) can select slide images from the primary training data that have a risk level greater than or equal to a preset threshold value evaluated using an infiltration judgment model and collect them as secondary training data.

[0185] As another example, in step S13, the data selection unit (140) can select slide images in which lymphovascular infiltration (LVI) is determined to be positive using the infiltration determination model as secondary training data if the infiltration determination model is a type of model that performs binary classification.

[0186] Next, in step S14, the second learning unit (150) can train a metastasis judgment model that analyzes the risk of lymph node metastasis (LNM) for an input target slide image using (d) selected secondary learning data.

[0187] According to one embodiment of the present invention, in step S14, the second learning unit (150) can train a transfer decision model corresponding to the type of CLAM (Clustering-constrained Attention Multiple Instance Learning) model built in a multiple instance learning method.

[0188] Next, in step S15, the analysis execution unit (160) can (e) acquire a target slide image to be analyzed and input the target slide image into a learned metastasis judgment model to evaluate the risk of lymph node metastasis (LNM).

[0189] In the description above, steps S11 through S15 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order between steps may be changed.

[0190] An artificial intelligence-based judgment method for predicting lymph node metastasis using lymphovascular infiltration based on digital pathology according to one embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The above-described hardware device may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

[0191] In addition, the artificial intelligence-based judgment method for predicting lymph node metastasis using lymphovascular infiltration based on the aforementioned digital pathology can also be implemented in the form of a computer program or application executed by a computer stored on a recording medium.

[0192] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical concept or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0193] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents thereof should be interpreted as being included within the scope of the present invention.

[0194] [Explanation of the symbol]

[0195] 10: AI-based lymph node metastasis prediction system

[0196] 100: An AI-based judgment device that predicts lymph node metastasis using lymphovascular invasion based on digital pathology

[0197] 110: Data Collection Unit

[0198] 120: Preprocessing unit

[0199] 130: 1st Learning Department

[0200] 140: Data Selection Unit

[0201] 150: 2nd Learning Department

[0202] 160: Analysis Execution Unit

[0203] 200: Scanning device

[0204] 300: User terminal

[0205] 400: Database

[0206] 20: Network

Claims

1. In a training method for an artificial intelligence-based judgment model that predicts lymph node metastasis using lymphovascular invasion based on digital pathology, (a) collecting primary training data comprising a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LMN); (b) a step of training an infiltration judgment model that analyzes the risk of lymphovascular infiltration for an input target slide image using the above-mentioned first training data; (c) a step of preparing secondary training data by selecting the plurality of slide images included in the primary training data based on the risk level derived using the infiltration judgment model; and (d) A step of training a metastasis determination model that analyzes the risk of lymph node metastasis for an input target slide image using the above secondary training data, A learning method including 2. In Paragraph 1, The above step (c) is, A learning method comprising selecting slide images whose risk level is greater than or equal to a preset threshold value as the secondary learning data.

3. In Paragraph 1, A learning method characterized in that the above-mentioned transfer decision model is a CLAM (Clustering-constrained Attention Multiple Instance Learning) model constructed using a multiple instance learning method.

4. In Paragraph 1, The above step (a) is, A step of performing preprocessing to divide at least a portion of the plurality of slide images into a plurality of patch images of a preset size, A learning method that includes 5. In Paragraph 1, A learning method characterized in that the above-described infiltration judgment model is a ResNet-18-based deep neural network model.

6. In Paragraph 1, The above step (a) is, A learning method comprising digitizing H&E (Hematoxylin and Eosin) stained tissue slides to generate the plurality of slide images.

7. In Paragraph 1, The above step (b) is, A learning method for training an infiltration judgment model to determine whether the above lymphovascular infiltration occurs through binary classification.

8. In Paragraph 7, The above step (c) is, A learning method comprising selecting slide images in which lymphovascular infiltration is determined to be positive using the above-mentioned infiltration judgment model as the above-mentioned secondary training data.

9. In an artificial intelligence-based judgment method for predicting lymph node metastasis using lymphovascular invasion based on digital pathology, (a) collecting primary training data comprising a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LMN); (b) a step of training an infiltration judgment model that analyzes the risk of lymphovascular infiltration for an input target slide image using the above-mentioned first training data; (c) A step of preparing secondary training data by selecting the plurality of slide images included in the primary training data based on the risk level derived using the infiltration judgment model; (d) a step of training a metastasis determination model that analyzes the risk of lymph node metastasis for an input target slide image using the above secondary training data; and (e) acquiring the target slide image and inputting the target slide image into the metastasis determination model to evaluate the risk of lymph node metastasis, A judgment method including 10. In a learning device for an artificial intelligence-based judgment model that predicts lymph node metastasis using lymphovascular invasion based on digital pathology, A data collection unit that collects primary training data comprising a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LMN); A first learning unit that trains an infiltration judgment model analyzing the risk of lymphovascular infiltration for an input target slide image using the above first learning data; A data selection unit that prepares secondary training data by selecting the plurality of slide images included in the primary training data based on the risk level derived using the infiltration judgment model; and A second learning unit that trains a metastasis judgment model analyzing the risk of lymph node metastasis for an input target slide image using the above second learning data, A learning device including 11. In Paragraph 10, The above data selection unit is, A learning device that selects slide images having a risk level greater than or equal to a preset threshold value as secondary learning data.

12. In Paragraph 10, A learning device characterized in that the above-mentioned transition decision model is a CLAM (Clustering-constrained Attention Multiple Instance Learning) model constructed using a multiple instance learning method.

13. In Paragraph 10, A preprocessing unit that performs preprocessing by dividing at least a portion of the above plurality of slide images into a plurality of patch images of a preset size, A learning device that further includes 14. In Paragraph 10, A learning device characterized in that the above-mentioned infiltration judgment model is a ResNet-18-based deep neural network model.

15. In Paragraph 10, The above data collection unit is, A learning device that digitizes H&E (Hematoxylin and Eosin) stained tissue slides to generate the plurality of slide images.

16. In Paragraph 10, The above-mentioned first learning unit is, A learning device that trains the infiltration judgment model to determine whether the above lymphovascular infiltration occurs through binary classification.

17. In Paragraph 16, The above data selection unit is, A learning device that selects slide images in which lymphovascular infiltration is determined to be positive using the above-described infiltration judgment model as the above-described secondary learning data.

18. An artificial intelligence-based judgment device for predicting lymph node metastasis using lymphovascular invasion based on digital pathology, A data collection unit that collects primary training data comprising a plurality of slide images labeled with a state related to at least one of lymphovascular invasion (LVI) and lymph node metastasis (LMN); A first learning unit that trains an infiltration judgment model analyzing the risk of lymphovascular infiltration for an input target slide image using the above first learning data; A data selection unit that prepares secondary training data by selecting the plurality of slide images included in the primary training data based on the risk level derived using the infiltration judgment model; A second learning unit that trains a metastasis determination model analyzing the risk of lymph node metastasis for an input target slide image using the above second learning data; and An analysis performing unit that acquires the target slide image and inputs the target slide image into the metastasis determination model to evaluate the risk of lymph node metastasis, A judgment device including

Images