Method and device for analyzing pathology slide image
The method identifies and analyzes tertiary lymphocyte structures in pathology slide images to enhance the prediction of treatment responsiveness and immunotherapy prognosis, addressing the limitations of current AI-based analysis techniques by providing a more accurate and efficient analysis of pathology slide images.
Patent Information
- Application Number
- PCT/KR2024/006298
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-06-09
- Filing Date
- 2024-05-09
- Publication Date
- 2024-12-12
AI Technical Summary
Current methods for analyzing pathology slide images using artificial intelligence models are limited in accuracy and speed for predicting medical information, particularly in predicting treatment responsiveness and prognosis for immunotherapy.
A method and device that identify tertiary lymphocyte structures (TLS) in pathology slide images, generate information related to these structures, and predict treatment responsiveness based on their characteristics, using a computing device with a processor that classifies and quantifies TLS features such as location, area, and maturity.
This approach provides a fast and accurate method for predicting medical information and predicting immunotherapy prognosis by utilizing TLS as a new biomarker, improving analysis speed and accuracy compared to existing methods.
Smart Images

Figure KR2024006298_12122024_PF_FP_ABST
Abstract
Description
Method and device for analyzing pathology slide images
[0001] The present disclosure relates to a method and device for analyzing a pathology slide image.
[0002] The field of digital pathology is a field that obtains histological information or predicts prognosis of a patient by using a whole slide image created by scanning a pathological slide image.
[0003] Recently, technologies have been developed to predict medical information by analyzing pathology slide images using artificial intelligence models. However, there is a need for methods to analyze pathology slide images to improve accuracy and speed in predicting medical information.
[0004] The present invention provides a method and device for analyzing pathology slide images. Furthermore, the present invention provides a computer-readable recording medium containing a program for executing the method on a computer. The technical problems to be solved are not limited to the technical problems described above, and other technical problems may exist.
[0005] A computing device according to one aspect comprises at least one memory; and at least one processor; wherein the at least one processor identifies at least one tertiary lymphoid structure (TLS) from a pathology slide image, generates information related to the at least one tertiary lymphoid structure, and predicts a treatment responsiveness of a lesion associated with the pathology slide image based on the information.
[0006] A method for analyzing a pathology slide image according to another aspect comprises the steps of: identifying at least one tertiary lymphocyte structure from the pathology slide image; generating information related to the at least one tertiary lymphocyte structure; and predicting a treatment responsiveness of a lesion associated with the pathology slide image based on the information.
[0007] Another aspect of a computer-readable recording medium includes a recording medium having recorded thereon a program for executing the above-described method on a computer.
[0008] According to the problem solving means of the present disclosure described above, a method for predicting medical information at a high speed and with high accuracy can be provided.
[0009] In addition, according to the problem solving means of the present disclosure described above, the prognosis for immunotherapy can be predicted using a new biomarker related to the tertiary lymphocyte structure.
[0010] FIG. 1 is a diagram illustrating an example of a system for analyzing a pathology slide image according to one embodiment.
[0011] FIG. 2A is a configuration diagram illustrating an example of a user terminal according to one embodiment.
[0012] FIG. 2b is a configuration diagram illustrating an example of a server according to one embodiment.
[0013] FIG. 3 is a flowchart illustrating an example of a method for analyzing a pathology slide image according to one embodiment.
[0014] FIG. 4 is a drawing for explaining an example of obtaining a pathology slide image according to one embodiment.
[0015] FIG. 5 is a drawing illustrating an example of a pathology slide image according to one embodiment.
[0016] FIG. 6 is a diagram for explaining a method for calculating a quantified value based on a parameter according to one embodiment of the present invention.
[0017] FIGS. 7A to 7C are drawings showing classification according to the location of a tertiary lymphocyte structure (710) according to one embodiment of the present invention.
[0018] Figures 8a to 8c are drawings showing classification according to maturity of tertiary lymphocyte structures according to one embodiment of the present invention.
[0019] FIG. 9 is a drawing for explaining a region of interest according to one embodiment of the present invention.
[0020] FIG. 10 is a drawing showing an example of an image displaying TLS information according to one embodiment of the present invention.
[0021] FIG. 11 is a diagram illustrating another example of a system for analyzing pathology slide images.
[0022] A computing device according to one aspect comprises at least one memory; and at least one processor; wherein the at least one processor identifies at least one tertiary lymphoid structure (TLS) from a pathology slide image, generates information related to the at least one tertiary lymphoid structure, and predicts a treatment responsiveness of a lesion associated with the pathology slide image based on the information.
[0023] The terms used in the examples are selected from widely used, current terms, as much as possible. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, the applicant may arbitrarily select terms, in which case their meanings will be described in detail in the relevant description. Therefore, the terms used in the specification should be defined based on their intended meaning and the overall content of the specification, rather than simply their names.
[0024] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "unit" and "module" used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.
[0025] Additionally, terms including ordinal numbers, such as "first" or "second," used in the specification may be used to describe various components, but the components should not be limited by the terms. The terms may be used to distinguish one component from another.
[0026] In one embodiment, a "pathology slide image" may refer to an image of a pathology slide that has been fixed and stained through a series of chemical treatments, such as for tissues removed from a human body. In addition, the pathology slide image may refer to a whole slide image (Whole Slide Image, WSI) that includes a high-resolution image of the entire slide, or may refer to a portion of the whole slide image, for example, one or more patches. For example, the pathology slide image may refer to a digital image captured or scanned by a scanning device (e.g., a digital scanner, etc.), and may include information about specific proteins, cells, tissues, and / or structures within the human body. In addition, the pathology slide image may include one or more patches, and histological information may be applied (e.g., tagged) to one or more patches through annotation.
[0027] "Medical information" may refer to any medically meaningful information that can be extracted from a medical image. For example, the medical information may include at least one of immune phenotype, genotype, biomarker score, tumor purity, RNA information, tumor microenvironment information, and treatment methods for cancer depicted in a pathology slide image.
[0028] Additionally, medical information may include, but is not limited to, the area, location, size of specific tissues (e.g., cancer tissue, cancer stromal tissue, etc.) and / or specific cells (e.g., tumor cells, lymphocytes, macrophages, endothelial cells, fibroblasts, etc.) within a medical image, diagnostic information of cancer, information related to the likelihood of a subject developing cancer, and / or medical conclusions related to cancer treatment.
[0029] Additionally, medical information may include not only quantifiable figures obtained from medical images, but also visualized figures, predicted figures, image information, and statistical information.
[0030] For example, medical information may be provided to a user terminal or output through a display device.
[0031] Below, embodiments are described in detail with reference to the attached drawings. However, the embodiments may be implemented in various different forms and are not limited to the examples described herein.
[0032] FIG. 1 is a diagram illustrating an example of a system for analyzing a pathology slide image according to one embodiment.
[0033] Referring to FIG. 1, the system includes a user terminal (10) and a server (20). For example, the user terminal (10) and the server (20) may be connected via wired or wireless communication to transmit and receive various data between each other.
[0034] For convenience of explanation, FIG. 1 illustrates the system as including a user terminal (10) and a server (20), but the present invention is not limited thereto. For example, the system may include other external devices (not shown). Furthermore, the operations of the user terminal (10) and server (20), which will be described below, may be implemented by a single device (e.g., the user terminal (10) or the server (20)) or by multiple devices.
[0035] The user terminal (10) may be a computing device equipped with a display device and a device for receiving user input (e.g., a keyboard, a mouse, etc.), and including memory and a processor. Furthermore, the display device may be implemented as a touch screen and perform a function for receiving user input. For example, the user terminal (10) may be, but is not limited to, a notebook PC, a desktop PC, a laptop, a tablet computer, a smartphone, etc.
[0036] The server (20) may be a device that communicates with an external device (not shown) including a user terminal (10). For example, the server (20) may be a device that stores various data, including a pathology slide image, a bitmap image corresponding to the pathology slide image, information generated by analysis of the pathology slide image (e.g., information on an area of at least one cell, tissue, or structure expressed in the pathology slide image, information on the expression of at least one biomarker, medical information related to the pathology slide image, etc.), and information on an artificial intelligence model used for analysis of the pathology slide image. Alternatively, the server (20) may be a computing device that includes a memory and a processor and has its own computing capability. When the server (20) is a computing device, the server (20) may perform at least some of the operations of the user terminal (10) that will be described later with reference to FIGS. 1 to 11. For example, the server (20) may be a cloud server, but is not limited thereto.
[0037] The user terminal (10) outputs information generated by analysis of a pathology slide image and / or pathology slide. For example, the user terminal (10) may output various information regarding the region of at least one cell, tissue, or structure represented in the pathology slide image. Additionally, the user terminal (10) may output biomarker expression information.
[0038] A pathology slide image can refer to an image of a pathology slide that has undergone a series of chemical treatments, fixed, and stained for microscopic observation of tissues removed from the human body. As an example, a pathology slide image can refer to a whole slide image, which includes a high-resolution image of the entire slide. Alternatively, a pathology slide image can refer to a portion of such a high-resolution whole slide image.
[0039] Meanwhile, a pathology slide image may refer to an area divided into patches (or tiles) from the entire slide image. For example, a patch (or tile) may have a certain area size.
[0040] Additionally, a pathology slide image may refer to a digital image captured using a microscope, and may include information about cells, tissues, and / or structures within the human body.
[0041] Analysis of pathology slide images can identify biological elements (e.g., cancer cells, immune cells, tumor regions, stromal regions, specific structures, etc.) depicted in the images. These biological elements can be used for histological diagnosis of disease, prediction of disease prognosis, and determination of treatment strategies.
[0042] The user terminal (10) can analyze a pathology slide image using an artificial intelligence model. For example, the user terminal (10) can individually detect (detection) numerous cells distributed on a pathology slide image and segment (segment) each region in the tissue using the artificial intelligence model. Furthermore, the user terminal (10) can analyze the detected cells and segmented regions using the artificial intelligence model and derive analysis results.
[0043] Hereinafter, with reference to FIGS. 2 to 11, an example of a user terminal (10) analyzing a pathology slide image will be described.
[0044] Meanwhile, for convenience of explanation, the user terminal (10) is described as performing all operations throughout the specification, but this is not limited to this. For example, at least some of the operations performed by the user terminal (10) may also be performed by the server (20).
[0045] FIG. 2A is a configuration diagram illustrating an example of a user terminal according to one embodiment.
[0046] Referring to FIG. 2A, the user terminal (100) includes a processor (110), a memory (120), an input / output interface (130), and a communication module (140). For convenience of explanation, only components related to the present invention are illustrated in FIG. 2A. Therefore, in addition to the components illustrated in FIG. 2A, other general-purpose components may be further included in the user terminal (100). In addition, it will be apparent to those skilled in the art that the processor (110), memory (120), input / output interface (130), and communication module (140) illustrated in FIG. 2A may be implemented as independent devices.
[0047] The processor (110) can process computer program commands by performing basic arithmetic, logic, and input / output operations. Here, the commands can be provided from memory (120) or an external device (e.g., a server (20), etc.). In addition, the processor (110) can generally control the operations of other components included in the user terminal (100).
[0048] The processor (110) can identify a tertiary lymphoid structure from a pathology slide image. In addition, the processor (110) can identify a component including at least one of a tumor region, a stroma region, and an immune cell from the pathology slide image.
[0049] The processor (110) can identify at least one tertiary lymphocyte structure using an artificial intelligence model learned based on a plurality of training images. At this time, the plurality of training images may be pathology slide images in which tertiary lymphocyte structures are labeled.
[0050] The processor (110) can generate information related to the identified tertiary lymphocyte structure. As an example, the processor (110) can calculate a quantified value based on at least one of a first parameter related to the tertiary lymphocyte structure and a second parameter related to a component other than the tertiary lymphocyte structure. Specifically, the first parameter may include at least one of a location of the tertiary lymphocyte structure, an area of the tertiary lymphocyte structure, and a maturity of the tertiary lymphocyte structure, and the second parameter may include at least one of a location of a tumor region, an area of the tumor region, a location of a stromal region, an area of the stromal region, and a location of an immune cell and a number of immune cells.
[0051] Meanwhile, the quantified value may be a value reflecting structural information related to the treatment responsiveness of the tertiary lymphocyte structure within the pathology slide image. As an example, the quantified value may include at least one of the number of tertiary lymphocyte structures per unit area of the tumor region, the ratio of the area of the tumor region to the area of the tertiary lymphocyte structure, the number of tertiary lymphocyte structures within the tumor region, the number of tertiary lymphocyte structures around the tumor region, the distance between the tumor region and the tertiary lymphocyte structure, the distance between a plurality of tertiary lymphocyte structures, the size of the tertiary lymphocyte structure, the number of tertiary lymphocyte structures forming a germinal center, the number of tumor-infiltrating lymphocytes located within a predetermined radius from the tertiary lymphocyte structure, and the density of immune cells within the tertiary lymphocyte structure.
[0052] The processor (110) can generate information related to tertiary lymphocyte structures within a region of interest determined based on the distance from the tumor region.
[0053] The processor (110) can predict the treatment responsiveness of a lesion associated with a pathology slide image based on information related to the tertiary lymphocyte structure. For example, the processor (110) can classify the class of the pathology slide image based on a threshold value set for the information.
[0054] Meanwhile, the processor (110) can output a plurality of detection targets (e.g., components including at least one of a tumor region, a stromal region, and an immune cell) including a tertiary lymphocyte structure identified from a pathology slide image. In addition, the processor (110) can output a report including information related to the tertiary lymphocyte structure. In addition, the processor (110) can output a report including treatment responsiveness of a lesion associated with the pathology slide image. For example, the processor (110) can control the display device so that at least one of the information related to the tertiary lymphocyte structure described above, the predicted treatment responsiveness based on the information related to the three-dimensional lymphocyte structure, and the report is output.
[0055] The processor (110) may be implemented as an array of a plurality of logic gates, or may be implemented as a combination of a general-purpose microprocessor and a memory storing a program that can be executed on the microprocessor. For example, the processor (110) may include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the processor (110) may include an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. For example, the processor (110) may refer to a combination of processing devices, such as a combination of a digital signal processor (DSP) and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors coupled with a digital signal processor (DSP) core, or any other such combination of configurations.
[0056] The memory (120) may include any non-transitory computer-readable recording medium. As an example, the memory (120) may include a non-permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, the non-permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be a separate permanent storage device distinct from the memory. In addition, the memory (120) may store an operating system (OS) and at least one program code (e.g., a code for the processor (110) to perform an operation to be described later with reference to FIGS. 3 to 11).
[0057] These software components may be loaded from a computer-readable recording medium separate from the memory (120). This separate computer-readable recording medium may be a recording medium that can be directly connected to the user terminal (100), and may include, for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. Alternatively, the software components may be loaded into the memory (120) through a communication module (140) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (120) based on a computer program (e.g., a computer program for the processor (110) to perform the operations described below with reference to FIGS. 3 to 11) that is installed by files provided by developers or a file distribution system that distributes installation files of applications through the communication module (140).
[0058] The input / output interface (130) may be a means for interfacing with a device (e.g., a keyboard, a mouse, etc.) for input or output that may be connected to or included in the user terminal (100). In FIG. 2A, the input / output interface (130) is illustrated as an element configured separately from the processor (110), but is not limited thereto, and the input / output interface (130) may be configured to be included in the processor (110).
[0059] The communication module (140) may provide a configuration or function for the server (200) and the user terminal (100) to communicate with each other via a network. In addition, the communication module (140) may provide a configuration or function for the user terminal (100) to communicate with other external devices. For example, control signals, commands, data, etc. provided under the control of the processor (110) may be transmitted to the server (200) and / or external devices via the communication module (140) and the network.
[0060] Meanwhile, although not illustrated in FIG. 2A, the user terminal (100) may further include a display device. Alternatively, the user terminal (100) may be connected to an independent display device via wired or wireless communication to transmit and receive data between the two devices. For example, pathology slide images, analysis information on pathology slide images, medical information, and additional information based on medical information may be provided to the user (30) via the display device.
[0061] FIG. 2b is a configuration diagram illustrating an example of a server according to one embodiment.
[0062] Referring to FIG. 2B, the server (200) includes a processor (210), a memory (220), and a communication module (230). For convenience of explanation, only components related to the present invention are illustrated in FIG. 2B. Therefore, in addition to the components illustrated in FIG. 2B, other general-purpose components may be further included in the server (200). Furthermore, it will be apparent to those skilled in the art that the processor (210), memory (220), and communication module (230) illustrated in FIG. 2B may be implemented as independent devices.
[0063] The processor (210) can obtain a pathology slide image from at least one of the internal memory (220), the user terminal (100), and another external device. The processor (210) can identify at least one tertiary lymphocyte structure from the pathology slide image, generate information related to the at least one tertiary lymphocyte structure, predict a treatment responsiveness of a lesion associated with the pathology slide image based on the generated information, or transmit at least one of the identified at least one tertiary lymphocyte structure, the generated information, and the predicted treatment responsiveness to the user terminal (100). In addition, the processor (210) can transmit a report (e.g., including a classified class or treatment responsiveness of the pathology slide image) to the user terminal (100).
[0064] In other words, at least one of the operations of the processor (110) described above with reference to FIG. 2A may be performed by the processor (210). In this case, the user terminal (100) may output information transmitted from the server (200) through a display device.
[0065] Meanwhile, since the implementation example of the processor (210) is the same as the implementation example of the processor (110) described above with reference to FIG. 2a, a detailed description thereof is omitted.
[0066] The memory (220) may store various data, such as pathology slide images and data generated according to the operation of the processor (210). In addition, the memory (220) may store an operating system (OS) and at least one program (e.g., a program required for the processor (210) to operate).
[0067] Meanwhile, the implementation example of the memory (220) is the same as the implementation example of the memory (120) described above with reference to FIG. 2a, so a detailed description is omitted.
[0068] The communication module (230) may provide a configuration or function for the server (200) and the user terminal (100) to communicate with each other via a network. In addition, the communication module (230) may provide a configuration or function for the server (200) to communicate with other external devices. For example, control signals, commands, data, etc. provided under the control of the processor (210) may be transmitted to the user terminal (100) and / or external devices via the communication module (230) and the network.
[0069] FIG. 3 is a flowchart illustrating an example of a method for analyzing a pathology slide image according to one embodiment.
[0070] The method illustrated in FIG. 3 is comprised of steps that are processed in time series in the user terminal (10, 100) or processor (110) illustrated in FIGS. 1 and 2A. Therefore, even if omitted below, the content described above regarding the user terminal (10, 100) or processor (110) illustrated in FIGS. 1 to 2A can also be applied to the method illustrated in FIG. 3.
[0071] Additionally, as described above with reference to FIGS. 1 to 2b, at least one of the steps of the method illustrated in FIG. 3 may be processed in the server (20, 200) or the processor (210).
[0072] At step 310, the processor identifies at least one tertiary lymphocyte structure from the pathology slide image.
[0073] In the present disclosure, a pathology slide image may be a whole slide image or a portion of a whole slide image. Here, the portion may be referred to as a patch or a tile.
[0074] In one embodiment, the processor can analyze a pathology slide image to identify and classify a tumor region, a stroma region, a necrosis region, and a background region from the pathology slide image. Furthermore, the processor can classify a plurality of cells represented in the pathology slide image into at least one of tumor cells, immune cells including lymphocytes, and other cells. In this case, the lymphocytes may include tumor-infiltrating lymphocytes. The processor can determine the location and area of the identified tumor region, stroma region, necrosis region, and background region, and / or the location and area of a tertiary lymphocyte structure. Furthermore, the processor can determine the number of each of the plurality of cells. Furthermore, the processor can determine structural and biological information of the components identified from the pathology slide image as well as the tertiary lymphocyte structure.
[0075] In addition, biomarker expression information may include the number of positive (+) / negative (-) tumor cells for PD-L1 (programmed death-ligand 1), the number of positive (+) / negative (-) immune cells, the tumor proportion score (TPS), the combined positive score (CPS), the immune phenotype, etc. However, the biomarker expression information is not limited to what is described above.
[0076] FIG. 4 is a drawing for explaining an example of obtaining a pathology slide image according to one embodiment.
[0077] Referring to FIG. 4, in one embodiment, the processor can identify at least one tertiary lymphocyte structure from a pathology slide image (430). Specifically, the processor can identify at least one tertiary lymphocyte structure included in the pathology slide image (430), which is an image of the pathology slide (410). The pathology slide image (430) may be an image of a pathology slide (410) that has been fixed and stained through a series of chemical treatments, such as tissue removed from a human body.
[0078] In one embodiment, the processor can identify at least one tertiary lymphocyte structure using an artificial intelligence model (420). In this case, the artificial intelligence model (420) may be a machine learning model. As an example, the artificial intelligence model (420) may be trained to infer histological information for all or a portion of a pathology slide image (430) (e.g., at least one patch included in the pathology slide image (430). In this case, the histological information generated through an annotation task may be used to train the artificial intelligence model (420). For example, the histological information may include, but is not limited to, information about cells (e.g., tumor cells, lymphocytes, macrophages, dendritic cells, fibroblasts, endothelial cells, etc.) within the patch (e.g., the number of specific cells, information about the tissue in which specific cells are located). As another example, the artificial intelligence model (420) can be trained to infer characteristics of at least one cell, tissue, or structure within a pathology slide image (430).
[0079] In one embodiment, the artificial intelligence model (420) can identify multiple detection targets, such as tertiary lymphocyte structures, tumor areas, stromal areas, and immune cells, by using histological information to determine information about the areas where cells are located in a pathology slide image (430).
[0080] FIG. 5 is a drawing illustrating an example of a pathology slide image according to one embodiment.
[0081] Referring to FIG. 5, a plurality of detection targets included in a pathology slide image (500) are illustrated. For example, the detection targets may include a tertiary lymphocyte structure (510) and components other than a tertiary lymphocyte structure.
[0082] Tertiary lymphocyte structures (510) are organized aggregates of immune cells formed in non-lymphoid tissues. Tertiary lymphocyte structures (510) may influence the prognosis of immunotherapy in relation to tumor areas (520), and thus require research and development as a biomarker.
[0083] Meanwhile, components other than the tertiary lymphocyte structure (510) may include at least one of a tissue region and immune cells. A tissue is a collection of cells specialized to perform a common function, and a predetermined region where such a collection of cells is clustered is called a tissue region. In one embodiment, the tissue region may include a tumor region (520) and a stromal region (530). The tumor region (520) refers to a region where tumor cells exhibiting invasion are clustered. The stromal region (530) refers to a region surrounding the tumor region where tumor-related stromal changes, such as desmoplasia or aggregation of lymphoid cells, are observed.
[0084] Immune cells, part of the immune system, detect and destroy foreign pathogens or abnormal cells in response to infections, allergies, tumors, and other conditions. Immune cells can include lymphocytes, including T cells and B cells, natural killer (NK) cells, dendritic cells, macrophages, neutrophils, eosinophils, and basophils. The tertiary lymphocyte structure (510) is an organized aggregate of these immune cells.
[0085] In one embodiment, the histological information generated through the annotation process described above with reference to FIG. 4 may be a plurality of training images in which all tertiary lymphocyte structures included in an arbitrary pathology slide image are labeled. Accordingly, the artificial intelligence model may be a model trained based on the plurality of training images. As an example, the artificial intelligence model may be an artificial neural network-based detection model. Accordingly, when a pathology slide image (500) is input, the artificial intelligence model can identify tertiary lymphocyte structures (510) based on what has been trained.
[0086] Referring again to FIG. 3, at step 320, the processor generates information associated with at least one tertiary lymphocyte structure.
[0087] Tumor regions containing tertiary lymphocyte structures within or adjacent to the tumor are associated with improved prognosis and clinical outcomes following immunotherapy. Therefore, a method is needed to quantify tertiary lymphocyte structures, either in relation to the tumor region or on their own, for use as biomarkers. Accordingly, a processor can generate information related to tertiary lymphocyte structures identified from pathology slide images.
[0088] In one embodiment, the processor can produce a quantified value based on at least one of a first parameter associated with a tertiary lymphocyte structure and a second parameter associated with a component other than the tertiary lymphocyte structure.
[0089] FIG. 6 is a diagram for explaining a method for calculating a quantified value based on a parameter according to one embodiment of the present invention.
[0090] Referring to FIG. 6, a tertiary lymphoid structure (610), a tumor region (620), and a stromal region (630) including B cells (611), T cells (612), dendritic cells (DCs) (613), follicular dendritic cells (FDCs) (614), fibroblastic reticular cells (FRCs) (615), and high endothelial venules (HEVs) (616) are depicted.
[0091] At this time, the immune cells (B cells (611), T cells (612), dendritic cells (613), endothelium-forming cells (614), fibroblasts (615), and stromal cells (616)) included in the tertiary lymphocyte structure (610) are only examples, and some of them may be omitted, and the tertiary lymphocyte structure (610) may further include immune cells not shown in FIG. 6. In addition, the immune cells (611 to 616) may be located inside the tertiary lymphocyte structure (610) to form the tertiary lymphocyte structure (610), but may also be located outside the tertiary lymphocyte structure (610).
[0092] The tertiary lymphocyte structure (610) has not only its own structural and biological characteristics, but also the area of the tumor region (620) and the stromal region (630) and the relative position with respect to the tumor region (620) and the stromal region (630) influence the immune treatment responsiveness to the tumor. Therefore, the processor can generate information (hereinafter referred to as “TLS information”) related to the tertiary lymphocyte structure (610) by considering both a first parameter related to the tertiary lymphocyte structure (610) and a second parameter related to an area including at least one of the tumor region (620), the stromal region (630), and the immune cells (611 to 616).
[0093] In one embodiment, the first parameter may include at least one of the number of tertiary lymphocyte structures (610), the location of the tertiary lymphocyte structures (610), the area of the tertiary lymphocyte structures (610), and the maturity of the tertiary lymphocyte structures (610). For example, the location of the tertiary lymphocyte structures (610) may be defined by (x, y) coordinate values within the pathology slide image, and the area of the tertiary lymphocyte structures (610) may be defined by It can be expressed in units. The maturity of the tertiary lymphocyte structure (610) will be described later with reference to FIGS. 8a to 8c.
[0094] In one embodiment, the second parameter may include at least one of the location of the tumor region (620), the area of the tumor region (620), the location of the stroma region (630), the area of the stroma region (630), the location of the immune cells (611 to 616), and the number of the immune cells (611 to 616). The locations of the tumor region (620), the stroma region (630), and the immune cells (611 to 616) may be defined by (x, y) coordinate values within the pathology slide image, and the areas of the tumor region (620) and the stroma region (630) may be defined by (x, y) coordinate values within the pathology slide image. It can be expressed in units. The number of immune cells (611 to 616) can mean the number of cells per unit.
[0095] In one embodiment, the processor may generate a quantified value based on at least one of the first parameter and the second parameter. Specifically, the quantified value may be a value reflecting structural information associated with the treatment responsiveness of a tertiary lymphocyte structure (610) within a pathology slide image.
[0096] As an example, the TLS information may include at least one of the number of tertiary lymphocyte structures (610) per unit area of the tumor region (620), the ratio of the area of the tumor region (620) to the area of the tertiary lymphocyte structures (610), the number of tertiary lymphocyte structures (610) inside the tumor region (620), the number of tertiary lymphocyte structures (610) around the tumor region (620), the distance between the tumor region (620) and the tertiary lymphocyte structures (610), the distance between a plurality of tertiary lymphocyte structures (610), the size of the tertiary lymphocyte structures (610), the number of tertiary lymphocyte structures (610) forming a germinal center, the number of tumor-infiltrating lymphocytes located within a predetermined radius from the tertiary lymphocyte structures (610), and the density of immune cells (611 to 616) within the tertiary lymphocyte structures (610).
[0097] Specifically, the number of tertiary lymphocyte structures (610) per unit area of the tumor area (620) can be calculated by the processor from the number of tertiary lymphocyte structures (610) as the first parameter and the area of the tumor area (620) as the second parameter, (number) / It can be calculated in units. In addition, the ratio of the area of the tumor region (620) and the area of the tertiary lymphocyte structure (610) can be calculated by the processor from the area of the tertiary lymphocyte structure (610) as the first parameter and the area of the tumor region (620) as the second parameter. The distance between the tumor region (620) and the tertiary lymphocyte structure (610) can be calculated by the processor from the position of the tertiary lymphocyte structure (610) as the first parameter and the position of the tumor region (620) as the second parameter.
[0098] The distance between multiple tertiary lymphocyte structures (610) can be calculated by the processor from the position of the tertiary lymphocyte structure (610) as the first parameter. In addition, the size of the tertiary lymphocyte structure (610) can be generated by the processor with reference to the first parameter.
[0099] The number of tumor-infiltrating lymphocytes located within a predetermined radius from a tertiary lymphocyte structure (610) can be calculated by a processor from the first parameter, which is the location of the tertiary lymphocyte structure (610), and the second parameter, which is the location of the immune cells (611 to 616). As an example, 'located within a predetermined radius' may mean located in a substrate region (630), and the 'predetermined radius' may be preset as a radius between 2.5 mm and 5 mm, but is not limited thereto. Meanwhile, the tumor-infiltrating lymphocytes may include B cells (611), T cells (612), natural killer (NK) cells (not shown), etc.
[0100] The density of immune cells (611 to 616) within the tertiary lymphocyte structure (610) can be calculated by the processor from the first parameter, the location and area of the tertiary lymphocyte structure (610), and the second parameter, the location and number of immune cells (611 to 616).
[0101] FIGS. 7A to 7C are drawings showing classification according to the location of a tertiary lymphocyte structure (710) according to one embodiment of the present invention.
[0102] In one embodiment, the processor can generate at least one of the number of tertiary lymphocyte structures (710) within the tumor region (720) and the number of tertiary lymphocyte structures (710) around the tumor region (720) as TLS information. Specifically, the processor can calculate the number of tertiary lymphocyte structures (710) within the tumor region (720) and the number of tertiary lymphocyte structures (710) around the tumor region (720) from the second parameter, the location of the tumor region (720) (and / or the location of the stromal region) and the first parameter, the location of the tertiary lymphocyte structures (710).
[0103] In one embodiment, tertiary lymphocyte structures can be classified as stromal, peritumoral, or intratumoral based on their relative location to the tumor area.
[0104] Figure 7a illustrates a stromal tertiary lymphocyte structure (710). The stromal tertiary lymphocyte structure (710) refers to a structure in which all or part of the tertiary lymphocyte structure (710) is not located inside the tumor area (720), but is located in the stromal area (730) that is in contact with the tumor area (720). As an example, the stromal tertiary lymphocyte structure (710) may be located in which all or part of the tertiary lymphocyte structure (710) is located within a 'predetermined radius' from the stromal area (630), and the 'predetermined radius' may be preset as a radius between 2.5 mm and 5 mm, but is not limited thereto.
[0105] Figure 7b illustrates a tumor-peripheral tertiary lymphocyte structure (710). A tumor-peripheral tertiary lymphocyte structure (710) refers to a structure in which a portion of the tertiary lymphocyte structure (710) is located within the tumor area (720), but not all of the tertiary lymphocyte structure (710) is located within the tumor area (720).
[0106] Figure 7c illustrates a tumor-resistant tertiary lymphocyte structure (710). The tumor-resistant tertiary lymphocyte structure (710) refers to a structure in which the entirety of the tertiary lymphocyte structure (710) is located within the tumor region (720).
[0107] For example, the tertiary lymphocyte structure (710) within the tumor region (720) may include the tumor-resistant tertiary lymphocyte structure (710) of FIG. 7C. The tertiary lymphocyte structure (710) around the tumor region (720) may include at least one of the tumor-peripheral tertiary lymphocyte structure (710) of FIG. 7B or the stromal tertiary lymphocyte structure (710) of FIG.
[0108] The closer the tertiary lymphocyte structures (710) are to the tumor area (720) and the greater their number, the more positive the responsiveness or prognosis of the tumor area (720) to immunotherapy. Therefore, by generating TLS information, which is a quantified value of the number of tertiary lymphocyte structures (710) within the tumor area (720) and the number of tertiary lymphocyte structures (710) around the tumor area (720), the accuracy of predicting treatment responsiveness can be improved.
[0109] In addition, only one tertiary lymphocyte structure (710) is illustrated in FIGS. 7A to 7C for convenience of explanation, but one or more tertiary lymphocyte structures (710) may exist in one tumor region (720) or stromal region (730), and accordingly, the processor may generate the number of tertiary lymphocyte structures (710) around the tumor region (720) as TLS information.
[0110] Figures 8a to 8c are drawings showing classification according to maturity of tertiary lymphocyte structures according to one embodiment of the present invention.
[0111] In one embodiment, the processor can generate the number of tertiary lymphocyte structures forming a germinal center using TLS information. Specifically, the processor can calculate the number of tertiary lymphocyte structures forming a germinal center based on the maturity of the tertiary lymphocyte structures, which is a first parameter.
[0112] In one embodiment, tertiary lymphoid structures can be classified into lymphoid aggregates, primary follicles, and secondary follicles based on their maturity.
[0113] Figures 8a to 8c illustrate tertiary lymphocyte structures in the lymphoid follicle, primary follicle, and secondary follicle states, respectively.
[0114] When B cells in a tertiary lymphocyte structure receive a proliferation command from T cells, they enter a primary follicle state and begin to proliferate actively. As a result of the repeated proliferation of B cells, a germinal center is formed at the center of the tertiary lymphocyte structure, and the area where the primary follicle is pushed out becomes the mantle zone. Referring to Figure 8c, it can be confirmed that a germinal center has formed in a tertiary lymphocyte structure in a secondary follicle state.
[0115] Among tertiary lymphocyte structures, the more secondary follicular tertiary lymphocyte structures there are, the more positive the responsiveness or prognosis to immunotherapy in the tumor region. Therefore, the processor can generate TLS information for the number of secondary follicular tertiary lymphocyte structures forming germinal centers among the tertiary lymphocyte structures identified in the pathology slide image. Furthermore, the processor can generate TLS information for the ratio of the number of secondary follicular tertiary lymphocyte structures to the number of tertiary lymphocyte structures identified in the pathology slide image.
[0116] FIG. 9 is a drawing for explaining a region of interest according to one embodiment of the present invention.
[0117] Referring to FIG. 9, a region of interest (900), a tumor region (910), and multiple tertiary lymphocyte structures (911 to 914) are depicted.
[0118] In one embodiment, the processor may determine a region of interest (ROI) (900) based on the distance from the tumor region (910). If the purpose is to predict treatment responsiveness by analyzing a pathology slide image to detect pathologically significant biomarkers, it may be necessary to exclude areas where the tumor region (910) does not exist from the analysis target in terms of analysis efficiency. Therefore, as an example, the region of interest (900) may be preset, automatically set, or set based on user input, depending on the type of tumor cells forming the tumor region (910), the presence or absence of the tumor region (910), and the distribution of the tumor region (910). For example, the processor may determine a region within 2 mm of the tumor region (910) as the region of interest (900).
[0119] In one embodiment, the processor may generate TLS information associated with tertiary lymphocyte structures (911, 912, 913) within the region of interest (900). That is, the processor may only generate TLS information associated with tertiary lymphocyte structures (911, 912, 913) within the region of interest (900), and may not generate TLS information associated with tertiary lymphocyte structures (914) outside the region of interest (900).
[0120] FIG. 10 is a drawing showing an example of an image displaying TLS information according to one embodiment of the present invention.
[0121] In one embodiment, the processor may output an image including a visual image representing TLS information in a pathology slide image. As an example, the processor may output an image representing a detected tertiary lymphocyte structure and TLS information corresponding to the detected tertiary lymphocyte structure.
[0122] For example, the processor may output an image in which a visual image representing a region of interest and TLS information is overlaid on a pathology slide image. That is, the processor may output an image in which one or more regions of interest are displayed on a display device in the pathology slide image in which the visual image representing the TLS information is overlaid.
[0123] Referring again to FIG. 3, at step 330, the processor predicts the treatment responsiveness of a lesion associated with the pathology slide image based on the TLS information.
[0124] Specifically, the processor can classify the pathology slide image into a class based on a preset threshold value (cut-off value) for the TLS information. For example, the threshold value can be determined as a value that improves the prediction accuracy of treatment responsiveness for each piece of TLS information.
[0125] In one embodiment, the processor can classify the pathology slide image into at least two classes based on the number of tertiary lymphocyte structures per unit area of the tumor region and a first threshold value. Specifically, the processor can compare the number of tertiary lymphocyte structures per unit area of the tumor region with the first threshold value, and determine the class of the pathology slide image as "High" if the number is greater than or equal to the first threshold value, and determine the class of the pathology slide image as "Low" if the number is less than the first threshold value. The first threshold value can be determined as a value that improves the prediction accuracy of treatment responsiveness based on clinical trial values for the number of tertiary lymphocyte structures per unit area of the tumor region.
[0126] In one embodiment, the processor can classify the pathology slide image into at least two classes based on a ratio of the area of the tumor region to the area of the tertiary lymphocyte structures and a second threshold value. Specifically, the processor can compare the ratio of the area of the tumor region to the area of the tertiary lymphocyte structures with the second threshold value, and determine the class of the pathology slide image as "High" if the ratio is greater than or equal to the second threshold value, and determine the class of the pathology slide image as "Low" if the ratio is less than the second threshold value. The second threshold value can be determined as a value that improves the prediction accuracy of treatment responsiveness based on clinical trial values for the number of tertiary lymphocyte structures per unit area of the tumor region.
[0127] In one embodiment, the processor may classify the pathology slide image into at least two classes based on the number of tertiary lymphocyte structures within / around the tumor region and a third threshold value. As an example, the processor may classify the pathology slide image into three classes by comparing the number of tertiary lymphocyte structures within / around the tumor region with the third threshold value and the fourth threshold value. Specifically, the processor may determine a class of the pathology slide image as "TLS structured" or "TLS included" indicating that there are more tertiary lymphocyte structures inside the tumor region than in the region surrounding the tumor region if the number of tertiary lymphocyte structures inside the tumor region is greater than or equal to a third threshold and the number of tertiary lymphocyte structures around the tumor region is less than a fourth threshold, "TLS excluded" indicating that there are more tertiary lymphocyte structures around the tumor region than in the region surrounding the tumor region if the number of tertiary lymphocyte structures around the tumor region is greater than or equal to the fourth threshold, and "TLS desertic" or "TLS desert" indicating that there are more tertiary lymphocyte structures around the tumor region if the number of tertiary lymphocyte structures inside the tumor region is less than or equal to the third threshold and the number of tertiary lymphocyte structures around the tumor region is less than or equal to the fourth threshold. The third threshold value and the fourth threshold value may be determined as values that improve the prediction accuracy of treatment responsiveness based on clinical trial values for the number of tertiary lymphocyte structures inside / around the tumor area.
[0128] In one embodiment, the processor can classify the pathology slide image into at least two classes based on the distance between the tumor region and the tertiary lymphocyte structure and the fifth threshold value and the sixth threshold value. As an example, the processor can determine the type of the tertiary lymphocyte structure based on the distance between the tumor region and the tertiary lymphocyte structure, and classify the pathology slide image into at least two classes based on the type of the tertiary lymphocyte structure. Specifically, the processor can compare an average (as another example, a median, a minimum distance, and a maximum distance, etc.) of the distance between the tumor region and the tertiary lymphocyte structure with the fifth threshold value, and determine the type of the tertiary lymphocyte structure that is less than the fifth threshold value as "in," the type of the tertiary lymphocyte structure that is greater than or equal to the fifth threshold value but less than the sixth threshold value as "near-out," and the type of the tertiary lymphocyte structure that is greater than or equal to the sixth threshold value as "far-out." Additionally, the processor may classify the pathology slide image into a "High" or "Low" class based on the type of tertiary lymphocyte structure and the number or ratio of tertiary lymphocyte structures corresponding to each type. In this case, for example, the average (as another example, the median, minimum distance, maximum distance, etc.) of the distance between the tumor region and the tertiary lymphocyte structure may be determined based on the distance measured relative to the border of the tumor region. The fifth threshold value and the sixth threshold value may be determined as values that improve the prediction accuracy of treatment responsiveness based on clinical trial values for the distance between the tumor region and the tertiary lymphocyte structure.
[0129] In one embodiment, the processor may classify the pathology slide image into at least two classes based on the distances between the plurality of tertiary lymphocyte structures and a seventh threshold value. Specifically, the processor may compare an average of the center-to-center distances of each of the plurality of tertiary lymphocyte structures (which may be, for example, a median, a minimum distance, and a maximum distance) with the seventh threshold value, and determine the class of the pathology slide image as "Dispersed" if the average is greater than or equal to the seventh threshold value, and determine the class of the pathology slide image as "Crowded" if the average is less than or equal to the seventh threshold value. The seventh threshold value may be determined to be a value that improves the prediction accuracy of treatment responsiveness based on clinical trial values for the number of tertiary lymphocyte structures per unit area of a tumor region.
[0130] In one embodiment, the processor can classify the pathology slide image into at least two classes based on the size of the tertiary lymphocyte structures or the number of tertiary lymphocyte structures forming germinal centers and an eighth threshold value. Specifically, the processor can compare the size of the tertiary lymphocyte structures with the eighth threshold value and determine the class of the pathology slide image as "Mature" if the size is equal to or greater than the eighth threshold value, and determine the class of the pathology slide image as "Immature" if the size is equal to or less than the eighth threshold value. Alternatively, the processor can compare the number of tertiary lymphocyte structures forming germinal centers with the eighth threshold value and determine the class of the pathology slide image as "Mature" if the size is equal to or greater than the eighth threshold value, and determine the class of the pathology slide image as "Immature" if the size is equal to or less than the eighth threshold value. The eighth threshold value can be determined as a value that improves the prediction accuracy of treatment responsiveness based on clinical trial values for the size of the tertiary lymphocyte structures or the number of tertiary lymphocyte structures forming germinal centers.
[0131] In one embodiment, the processor can classify the pathology slide image into at least two classes based on the number of tumor-infiltrating lymphocytes located within a predetermined radius from a tertiary lymphoid tissue and a ninth threshold value. Specifically, the processor can compare the number of tumor-infiltrating lymphocytes located within a predetermined radius from a tertiary lymphoid tissue with the ninth threshold value, and determine the class of the pathology slide image as "TIL rich" if the number is greater than or equal to the ninth threshold value, and determine the class of the pathology slide image as "TIL poor" if the number is less than the ninth threshold value. The ninth threshold value can be determined as a value that improves the prediction accuracy of treatment responsiveness based on clinical trial values for the number of tumor-infiltrating lymphocytes located within a predetermined radius from a tertiary lymphoid tissue.
[0132] In one embodiment, the processor may classify the pathology slide image into at least two classes based on the density of immune cells within the tertiary lymphocyte structure and a tenth threshold value. Specifically, the processor may compare the density of immune cells within the tertiary lymphocyte structure with the tenth threshold value, and determine the class of the pathology slide image as "Dense TLS" if the density is greater than or equal to the tenth threshold value, and determine the class of the pathology slide image as "Sparse TLS" if the density is less than the tenth threshold value. The tenth threshold value may be determined to be a value that improves the prediction accuracy of treatment responsiveness based on clinical trial values for the density of immune cells within the tertiary lymphocyte structure.
[0133] In one embodiment, the processor can output a pathology slide image together with information associated with a class (or TLS information) of the pathology slide image and an immunophenotype. The processor can determine at least one of an immunophenotype of the pathology slide image (or at least one region of interest within the pathology slide image) or information associated with the immunophenotype. The information associated with the immunophenotype can include at least one of a plurality of classes representing an immune environment of the pathology slide image (e.g., Immune inflamed, Immune excluded, Immune desert), at least one of a score for each of the plurality of classes representing an immune environment (e.g., a probability that an immunophenotype of at least one region of interest corresponds to a corresponding class), or at least one of features representing an immune environment of the pathology slide image.
[0134] Below, an embodiment is described in which a processor predicts therapeutic responsiveness to an immunotherapy based on the density of immune cells. In this case, the processor can determine an immunophenotype for each of at least one region within a pathology slide image.
[0135] Specifically, the processor may determine the immunophenotype of the corresponding region of the pathology slide image as immune activity if the density of immune cells within the tumor region is equal to or greater than a first threshold density. Furthermore, the processor may determine the immunophenotype of the corresponding region of the pathology slide image as immune exclusion if the density of immune cells within the tumor region is less than the first threshold density and the density of immune cells within the stroma region is equal to or greater than a second threshold density. Furthermore, the processor may determine the immunophenotype of the corresponding region of the pathology slide image as immune deficiency if the density of immune cells within the tumor region is less than the first threshold density and the density of immune cells within the stroma region is less than the second threshold density.
[0136] Through this, if the most abundant immunophenotype in a pathology slide image is immune activation, the processor can predict that the patient associated with the pathology slide image will respond to the immunotherapy (i.e., the patient is a responder). Conversely, if the most abundant immunophenotype in the pathology slide image is immune exclusion or immune deficiency, the processor can predict that the patient associated with the pathology slide image will not respond to the immunotherapy (i.e., the patient is a non-responder).
[0137] In one embodiment, the processor may determine a class of a pathology slide image based on preset threshold values (e.g., the first to tenth threshold values described above) for TLS information, and may determine a class representing an immunophenotype of the pathology slide image. In addition, the processor may output the class of the pathology slide image and the class representing the immunophenotype of the pathology slide image. For example, the processor may determine the class of the pathology slide image as "Mature," determine the immunophenotype of the pathology slide image as immune activity, and then output the determination result for the pathology slide image as "Inflamed-Mature."
[0138] In one embodiment, the processor can determine TLS information by analyzing at least two pathology slide images stained with different reagents (e.g., Hematoxylin and Eosin (H&E) stain and Immunohistochemistry (IHC) stain).
[0139] The method for classifying a pathology slide image into a class based on preset threshold values (e.g., the first to tenth threshold values described above) for TLS information by the processor is not limited to the examples described above.
[0140] For example, the processor may determine a class corresponding to each of at least one region of interest included in a pathology slide image based on a preset threshold value for TLS information (e.g., the first to tenth threshold values described above), and classify the class corresponding to the at least one region of interest.
[0141] In one embodiment, the processor can assign weights to the generated TLS information and classify the class of the pathology slide image based on the weights. For example, if the processor assigns a greater weight to the number of tertiary lymphocyte structures per unit area of the tumor region than to the ratio of the area of the tumor region to the area of the tertiary lymphocyte structures, the processor can determine the class of the pathology slide image in which the number of tertiary lymphocyte structures per unit area of the tumor region is greater than or equal to the first threshold value based on the assigned weights as "High" even if the ratio of the area of the tumor region to the area of the tertiary lymphocyte structures is less than the second threshold value. Meanwhile, the weight assigned to the TLS information can be determined based on the clinical significance of each piece of TLS information.
[0142] FIG. 11 is a diagram illustrating another example of a system for analyzing pathology slide images.
[0143] Referring to FIG. 11, a system (1100) is an example of a system and network for preparing, processing, and reviewing slide images of tissue samples using an artificial intelligence model.
[0144] According to various embodiments of the present disclosure, the method described above with reference to FIGS. 2A to 10 may be performed by at least one or a combination of a user terminal (1122, 1123), an image management system (1130), an AI-based biomarker analysis system (1140), a laboratory information management system (1150), and a hospital or laboratory server (1160).
[0145] The scanner (1121) can acquire a digitized image from a tissue sample slide generated using a tissue sample of a subject (1111). For example, the scanner (1121), the user terminal (1122, 1123), the image management system (1130), the AI-based biomarker analysis system (1140), the laboratory information management system (1150), and / or the hospital or laboratory server (1160) can each be connected to a network (1170) such as the Internet via one or more computers, servers, and / or mobile devices, or can communicate with the user (1112) via one or more computers and / or mobile devices.
[0146] The user terminals (1122, 1123), the image management system (1130), the AI-based biomarker analysis system (1140), the laboratory information management system (1150), and / or the hospital or laboratory server (1160) may generate, or otherwise acquire from another device, one or more tissue samples, tissue sample slides (pathology slides), digitized images of tissue sample slides (pathology slides), or any combination thereof of the subject (1111). In addition, the user terminals (1122, 1123), the image management system (1130), the AI-based biomarker analysis system (1140), the laboratory information management system (1150), and / or the hospital or laboratory server (1160) may acquire any combination of subject-specific information, such as the age, medical history, cancer treatment history, family history, past biopsy records, or disease information of the subject (1111).
[0147] A scanner (1121), a user terminal (1122, 1123), an AI-based biomarker analysis system (1140), a laboratory information management system (1150), and / or a hospital or laboratory server (1160) may transmit digitized pathology slide images, subject-specific information, and / or analysis results of digitized pathology slide images to an image management system (1130) via a network (1170). The image management system (1130) may include a storage for storing received images and a storage for storing analysis results.
[0148] In addition, according to various embodiments of the present disclosure, an artificial intelligence model learned and trained to predict at least one of information about at least one cell, information about at least one region, information related to a biomarker, medical diagnosis information, and / or medical treatment information from a pathology slide image of a subject (1111) may be stored and operated in a user terminal (1122, 1123), an image management system (1130), etc.
[0149] As described above, a computer device according to one embodiment of the present disclosure can generate information related to tertiary lymphocyte structures from pathology slide images and automatically predict a patient's treatment responsiveness based on the information related to the tertiary lymphocyte structures. Therefore, this can replace the analysis method in which a pathologist directly examines only a partial tissue distribution across the entire pathology slide image. Furthermore, the prediction accuracy and analysis speed of a patient's treatment responsiveness to an immuno-oncology agent can be improved.
[0150] Meanwhile, the above-described method can be written as a program that can be executed on a computer, and can be implemented on a general-purpose digital computer that runs the program using a computer-readable recording medium. In addition, the structure of the data used in the above-described method can be recorded on a computer-readable recording medium through various means. The computer-readable recording medium includes storage media such as magnetic storage media (e.g., ROM, RAM, USB, floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM, DVD, etc.).
[0151] Those skilled in the art will appreciate that the present invention can be implemented in modified forms without departing from the essential characteristics of the above-described invention. Therefore, the disclosed methods should be considered illustrative rather than restrictive. The scope of the claims, not the foregoing description, is defined by the scope of the patent, and should be interpreted to encompass all differences within the scope equivalent thereto.
Claims
1. At least one memory; and comprising at least one processor; At least one processor, Identify at least one tertiary lymphoid structure (TLS) from a pathology slide image, Generating information related to at least one tertiary lymphocyte structure, A computing device that predicts the treatment responsiveness of a lesion associated with the pathology slide image based on the above information.
2. In paragraph 1, At least one processor, A computing device that identifies a component including at least one of a tumor region, a stroma region, and an immune cell from the above pathology slide image.
3. In paragraph 2, At least one processor, A computing device that produces a quantified value based on at least one of a first parameter associated with the tertiary lymphocyte structure and a second parameter associated with the component.
4. In paragraph 3, The above first parameter is, At least one of the number of the tertiary lymphocyte structures, the location of the tertiary lymphocyte structures, the area of the tertiary lymphocyte structures, and the maturity of the tertiary lymphocyte structures, The second parameter above is, A computing device comprising at least one of the location of the tumor region, the area of the tumor region, the location of the stromal region, the area of the stromal region, the location of the immune cells, and the number of the immune cells.
5. In paragraph 3, The above quantified values are, A computer device that reflects structural information related to the therapeutic responsiveness of the tertiary lymphocyte structure in the pathology slide image.
6. In paragraph 5, The above quantified values are, A computer device comprising at least one of the number of the tertiary lymphocyte structures per unit area of the tumor region, the ratio of the area of the tumor region to the area of the tertiary lymphocyte structures, the number of the tertiary lymphocyte structures inside the tumor region, the number of the tertiary lymphocyte structures around the tumor region, the distance between the tumor region and the tertiary lymphocyte structures, the distance between a plurality of the tertiary lymphocyte structures, the size of the tertiary lymphocyte structures, the number of the tertiary lymphocyte structures forming a germinal center, the number of tumor infiltrating lymphocytes (TILs) located within a predetermined radius from the tertiary lymphocyte structures, and the density of the immune cells within the tertiary lymphocyte structures.
7. In paragraph 1, The above processor, A computer device that generates the information related to the tertiary lymphocyte structure within a region of interest determined based on the distance from the tumor area.
8. In paragraph 1, The above processor, A computer device that classifies the class of the pathology slide image based on a preset cut-off value for the above information.
9. In paragraph 1, The above processor, Identifying at least one tertiary lymphocyte structure using an artificial intelligence model learned based on multiple training images, The above multiple learning images are, A computer device in which the tertiary lymphocyte structure is labeled on the above pathology slide image.
10. A step of identifying at least one tertiary lymphocyte structure from a pathology slide image; generating information related to at least one tertiary lymphocyte structure; and A method for analyzing a pathology slide image, comprising: a step of predicting the treatment responsiveness of a lesion associated with the pathology slide image based on the above information.
11. In paragraph 10, A method further comprising: identifying a component including at least one of a tumor region, a stromal region, and an immune cell from the pathology slide image; 12. In paragraph 11, The above generating steps are: A method comprising: calculating a quantified value based on at least one of a first parameter associated with the tertiary lymphocyte structure and a second parameter associated with the component.
13. In paragraph 12, The above first parameter is, At least one of the number of the tertiary lymphocyte structures, the location of the tertiary lymphocyte structures, the area of the tertiary lymphocyte structures, and the maturity of the tertiary lymphocyte structures, The second parameter above is, A method comprising at least one of the location of the tumor region, the area of the tumor region, the location of the stromal region, the area of the stromal region, the location of the immune cells, and the number of the immune cells.
14. In paragraph 12, The above quantified values are, A method in which structural information related to the treatment responsiveness of the tertiary lymphocyte structure is reflected in the above pathology slide image.
15. In paragraph 14, The above quantified values are, A method comprising at least one of the number of the tertiary lymphocyte structures per unit area of the tumor region, the ratio of the area of the tumor region to the area of the tertiary lymphocyte structures, the number of the tertiary lymphocyte structures inside the tumor region, the number of the tertiary lymphocyte structures around the tumor region, the distance between the tumor region and the tertiary lymphocyte structures, the distance between a plurality of the tertiary lymphocyte structures, the size of the tertiary lymphocyte structures, the number of the tertiary lymphocyte structures forming a germinal center, the number of tumor-infiltrating lymphocytes located within a predetermined radius from the tertiary lymphocyte structures, and the density of the immune cells within the tertiary lymphocyte structures.
16. In paragraph 10, The above generating steps are: A method for generating information related to said tertiary lymphocyte structures within a region of interest determined based on a distance from said tumor region.
17. In paragraph 10, The above predicting steps are: A method comprising: a step of classifying a class of the pathology slide image based on a preset threshold value for the above information; 18. In paragraph 10, The above identifying step is, A step of identifying at least one tertiary lymphocyte structure using an artificial intelligence model learned based on a plurality of learning images; The above multiple learning images are, A method wherein the tertiary lymphocyte structure is labeled on the above pathology slide image.
19. A computer-readable recording medium recording a program for executing the method of Article 10 on a computer.
Citation Information
Patent Citations
Methods and systems for characterizing tissue of subject by utilizing machine learning
JP2021090763A
Nutrient carriers for pets
KR1020220138518A
Semiconductor memory device and memory system including the same
KR1020230055614A
Electrostatic dust collection and plasma combustion conversion type fine dust removal system and method
KR1020240120780A
KR20220112121A
