Method, device, and program for classifying and detecting cells in pathological image on basis of ai

Abstract

Disclosed is a method for classifying and detecting cells in a pathological image on the basis of AI according to various embodiments of the present invention for achieving the objectives described above. The method comprises the steps of: acquiring a pathological image; performing preprocessing on the pathological image; and generating analysis information corresponding to the pathological image by using a pathological image analysis model. The step of generating the analysis information comprises the steps of: detecting one or more cells from the pathological image; determining whether the cells are tumor cells; and determining the staining level of the cells.

Description

AI-based cell classification and detection method, device, and program for pathology images

[0001] The present invention relates to a technology for classifying and detecting cells in pathology images based on artificial intelligence (AI). Specifically, it relates to a method, apparatus, and program that analyzes digital scan images of tissue or cell slides to classify and detect the location, morphology, staining intensity, etc., of cells, and utilizes this to support pathological diagnosis and research.

[0002] Pathology is a field that diagnoses the presence or progression of disease by observing slide images of tissues or cells under a microscope to analyze factors such as cell morphology and staining intensity. Traditionally, pathology images have been analyzed manually by experts; however, this method presents problems such as requiring significant time and effort, as well as a high potential for subjective judgment to influence the analysis results. Furthermore, efficiently processing and accurately analyzing vast volumes of slide images requires skilled personnel and experience, highlighting the need for the development of technologies in clinical diagnosis and research that can automate pathology image analysis and improve accuracy.

[0003] Recently, the automatic analysis and detection of pathology images are being actively researched through the advancement of digital pathology technology and the introduction of artificial intelligence (AI) and deep learning technologies. In particular, image analysis technology utilizing deep learning demonstrates excellent performance in semantic segmentation and object detection, and is being applied to precisely analyze the morphology, distribution, and staining intensity of cells within tissue images. These AI-based analysis methods have the advantage of not only being able to automatically process a vast amount of pathology images but also reducing deviations caused by conventional manual work by providing consistent analysis results.

[0004] In particular, cell classification and detection are critical steps in pathology image analysis, where it is important to accurately identify the location and morphology of cells and analyze the staining intensity of cells according to the expression status of specific biomarkers (e.g., HER2, ER, PR, etc.). This allows for precise diagnosis of tissue conditions or disease characteristics and can be utilized to establish treatment strategies. At this time, more accurate and rapid analysis is possible by automating cell counting, staining intensity classification, and the distinction between positive and negative cells using deep learning models and AI algorithms specialized for image processing.

[0005] Existing pathology image analysis techniques have been limited to the classification of specific biomarker staining intensities and cell morphologies, and have faced the limitation of being unable to perform comprehensive analysis reflecting diverse cellular characteristics. Furthermore, there have been cases where the processing of large volumes of pathology images was inefficient or the accuracy of staining intensity classification was low. Therefore, to overcome these limitations, a method capable of performing cell classification and detection in pathology images more efficiently and accurately may be required.

[0006] Various embodiments of the present invention are intended to solve the above-mentioned problems and to provide an AI-based pathology image analysis method that effectively classifies and detects the location, morphology, and staining intensity of cells within a pathology image to perform accurate analysis.

[0007] The problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below.

[0008] A method for classifying and detecting cells in an AI-based pathology image according to various embodiments of the present invention for solving the aforementioned problem is disclosed. The method comprises the steps of acquiring a pathology image, performing preprocessing on the pathology image, and generating analysis information corresponding to the pathology image using a pathology image analysis model. The pathology image analysis model may include a cell counting model that detects the location of cells from the pathology image and classifies the cells into positive and negative cells, and a staining intensity classification model that generates a staining intensity score of a region of interest (ROI) based on a specific staining intensity in the pathology image.

[0009] In an alternative embodiment, the step of performing preprocessing on the pathology image may include identifying the region of interest (ROI) in the pathology image, dividing the identified region of interest into multiple patches of a predetermined size, and performing pixel value normalization, color scaling, and contrast adjustment on the region of interest divided into multiple patches.

[0010] In an alternative embodiment, the cell counting model comprises a dimensionality reduction submodel and a dimensionality restoration submodel, wherein the dimensionality reduction submodel extracts features for each of the multiple patch units, and the dimensionality restoration submodel generates multiple maps through analysis of the extracted features, and the step of generating analysis information includes the step of determining the spatial distribution, density, and classification information of cells based on the generated multiple maps, and the multiple maps may include a Binary Nuclei Map, Horizontal and Vertical Distance Maps, and a Nuclei Type Map.

[0011] In an alternative embodiment, the cell counting model may be characterized by applying an attention mechanism during the analysis process in the dimension restoration submodel to reflect spatial correlations between cells within a plurality of generated maps, predicting spatial distribution, density, and classification information of the cells, and calculating the importance of the predicted information as a weight to improve the accuracy of the analysis results.

[0012] In an alternative embodiment, the method further comprises the step of performing post-processing on the plurality of maps, wherein the step of performing post-processing may include: identifying important instances within the region of interest and determining them as instances to be preserved; generating spatial indexes corresponding to the instances to be preserved and generating relationship information between the spatial indices; utilizing the relationship information between the spatial indices to search for other instances that spatially overlap with each instance and calculating the degree of overlap with the other instances found; grouping instances having an overlap degree greater than or equal to a threshold threshold; and preserving the instance with the largest area within the instance group and removing the remaining instances.

[0013] In an alternative embodiment, the staining intensity classification model is configured to include a plurality of staining intensity classification sub-models that analyze the characteristics of different biomarkers, and the plurality of staining intensity classification sub-models may include a first staining intensity classification sub-model that generates analysis information for human epidermal growth factor receptor 2 (HER2) and a second staining intensity classification sub-model that generates analysis information for estrogen receptor (ER) and progesterone receptor (PR).

[0014] In an alternative embodiment, the first dyeing intensity classification submodel is provided with a Global Average Pooling layer and a Fully Connected layer, wherein the Global Average Pooling layer generates a feature vector corresponding to each patch, the Fully Connected layer classifies the dyeing intensity into at least one of a plurality of stages, and generates analysis information by integrating the patch-specific classification results at the region of interest unit to predict the dyeing intensity distribution.

[0015] In an alternative embodiment, the step of generating the analysis information includes the step of generating analysis information regarding the state of the estrogen receptor and the progesterone receptor by utilizing the second staining intensity classification submodel, and the step of generating analysis information regarding the state of the estrogen receptor and the progesterone receptor by utilizing the second staining intensity classification submodel may include the step of separating the preprocessed pathology image into a plurality of channels; the step of analyzing the pixel value distribution of the DAB channel among the plurality of channels to set the range of staining intensity levels and deriving a threshold value for each level; the step of calculating the staining intensity based on the average pixel value of the DAB channel in each detected cell region and classifying the calculated staining intensity into one of the plurality of levels; the step of calculating the Proportion Score and Intensity Score related to the staining intensity and calculating the Allred Score based on the Proportion Score and the Intensity Score; and the step of generating analysis information regarding the state of the estrogen receptor and the progesterone receptor at the level of the region of interest by weighting the Allred Score of each positive cell within the region of interest.

[0016] In an alternative embodiment, the output of the cell counting model is used as reference information for assigning weights to the analysis results of the first staining intensity classification sub-model and the second staining intensity classification sub-model, and the weights are calculated based on cell density, cell distribution, and cell type information within the region of interest to correct the region of interest unit results of the staining intensity classification model.

[0017] In an alternative embodiment, the method further comprises the step of evaluating the reliability of the analysis results by cross-verifying the output information of the cell counting model, the first staining intensity classification sub-model, and the second staining intensity classification sub-model, wherein the step of evaluating the reliability may include: generating an expected staining intensity distribution of the region of interest based on cell density, cell distribution, and cell ratio information within the region of interest calculated through the cell counting model; generating an actual staining intensity distribution of the region of interest unit based on the outputs of the first staining intensity classification sub-model and the second staining intensity classification sub-model; evaluating the degree of agreement between the expected staining intensity distribution and the actual staining intensity distribution to generate a reliability score corresponding to the region of interest; and calculating and providing the reliability score of the region of interest unit.

[0018] In another embodiment of the present invention, an apparatus for performing a method for classifying and detecting cells in an AI-based pathology image is disclosed. The apparatus comprises a memory for storing one or more instructions and a processor for executing the one or more instructions stored in the memory, and the processor can perform the method for classifying and detecting cells in an AI-based pathology image by executing the one or more instructions.

[0019] In another embodiment of the present invention, a computer program stored on a computer-readable recording medium is disclosed to enable the execution of an AI-based cell classification and detection method of a pathology image. The computer program can be combined with a computer, which is hardware, to execute the AI-based cell classification and detection method of a pathology image.

[0020] Other specific details of the present invention are included in the detailed description and drawings.

[0021] According to an embodiment of the present invention, the process of cell classification and detection in pathology images can be automated and accuracy improved. This enables the efficient analysis of a vast amount of pathology images, allowing medical professionals to quickly identify information on cell location, morphology, and staining intensity for use in disease diagnosis and prognosis evaluation.

[0022] In addition, the present invention can accurately classify positive and negative cells and quantitatively evaluate the expression level of each biomarker by analyzing pathology images based on AI, thereby reducing deviations caused by conventional manual methods and providing consistent results.

[0023] Additionally, the present invention ensures scalability and flexibility through a microservices architecture and an event-driven structure, and provides a secure pathology image analysis environment by maintaining data confidentiality and enhancing security through encryption. These effects can contribute to the advancement of the medical and pathology fields by providing reliable information for clinical diagnosis, research, and treatment planning.

[0024] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0025] Various aspects are described with reference to the drawings, wherein similar reference numbers are used to collectively refer to similar components. In the following embodiments, for illustrative purposes, a number of specific details are presented to provide a comprehensive understanding of one or more aspects. However, it will be apparent that such aspect(s) may be practiced without these specific details.

[0026] FIG. 1 is an exemplary diagram schematically illustrating a system for implementing an AI-based method for cell classification and detection of pathology images related to one embodiment of the present invention.

[0027] FIG. 2 is a hardware configuration diagram of a computing device that performs a method for classifying and detecting cells in an AI-based pathology image related to one embodiment of the present invention.

[0028] FIG. 3 illustrates an exemplary flowchart of an AI-based method for cell classification and detection of pathology images related to one embodiment of the present invention.

[0029] FIG. 4 illustrates a flowchart exemplarily showing the process of performing post-processing correction on the output of a cell counting model related to one embodiment of the present invention.

[0030] FIG. 5 illustrates a flowchart exemplarily showing the process of generating analysis information related to estrogen receptors and progesterone receptors related to one embodiment of the present invention.

[0031] FIG. 6 illustrates a flowchart that exemplarily shows a process of evaluating the reliability of an analysis result by cross-verifying multiple outputs of a pathology image analysis model related to an embodiment of the present invention.

[0032] FIG. 7 is a diagram illustrating the AI-based cell classification and detection process of a pathology image according to one embodiment.

[0033] FIG. 8 is a diagram illustrating the process of cell classification and detection of an AI-based pathology image according to another embodiment.

[0034] Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to facilitate an understanding of the invention. However, it is evident that these embodiments can be practiced without such specific descriptions.

[0035] As used herein, terms such as “component,” “module,” “system,” etc. refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or executions of software. For example, a component may be, but is not limited to, a procedure executed on a processor, a processor, an object, an execution thread, a program, and / or a computer. For example, both an application executed on a computing device and the computing device itself may be a component. One or more components may reside within a processor and / or an execution thread. A component may be localized within a single computer. A component may be distributed among two or more computers. Additionally, these components may be executed from various computer-readable media having various data structures stored therein. Components may communicate through local and / or remote processes, for example, according to signals having one or more data packets (e.g., data from a component interacting with another component in a local system or distributed system, and / or data transmitted through signals to other systems and networks such as the Internet).

[0036] Furthermore, the term "or" is intended to mean an implicit "or" rather than an exclusive "or." That is, unless otherwise specified or evident from the context, "X uses A or B" is intended to mean one of the natural implicit substitutions. In other words, if X uses A; if X uses B; or if X uses both A and B, "X uses A or B" may apply to any of these cases. Additionally, the term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the enumerated related items.

[0037] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that such features and / or components are present. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, components and / or groups thereof. Furthermore, unless otherwise specified or clearly evident from the context to indicate a singular form, the singular in this specification and claims should generally be interpreted to mean “one or more.”

[0038] Those skilled in the art should recognize that the various exemplary logical blocks, configurations, modules, circuits, means, logics, and algorithmic steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly exemplify the interchangeability of hardware and software, various exemplary components, blocks, configurations, means, logics, modules, circuits, and steps have been generally described above in terms of their functionality. Whether such functionality is implemented in hardware or software depends on the specific application and design constraints imposed on the overall system. Skilled technicians may implement the described functionality in various ways for each specific application. However, such decisions regarding implementation should not be interpreted as moving out of the scope of the invention.

[0039] The description of the presented embodiments is provided to enable those skilled in the art to use or practice the present invention. Various modifications to these embodiments will be apparent to those skilled in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present invention. Thus, the present invention is not limited to the embodiments presented herein. The present invention should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

[0040] In this specification, the term "computer" refers to any type of hardware device comprising at least one processor, and may be understood to include software configurations operating on said hardware device according to the embodiments. For example, the term "computer" may be understood to include smartphones, tablet PCs, desktops, laptops, and user clients and applications running on each of these devices, but is not limited thereto.

[0041] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0042] Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and depending on the embodiment, at least some of each step may be performed on different devices.

[0043]

[0044] FIG. 1 is an exemplary diagram schematically illustrating a system for implementing an AI-based method for cell classification and detection of pathology images related to one embodiment of the present invention.

[0045] As illustrated in FIG. 1, a system according to embodiments of the present invention may include a computing device (100), a user terminal (200), an external server (300), and a network (400). The components illustrated in FIG. 1 are exemplary, and additional components may exist or some of the components illustrated in FIG. 1 may be omitted. The computing device (100), the external server (300), and the user terminal (200) according to embodiments of the present invention may mutually transmit and receive data for a system according to embodiments of the present invention through a network (400).

[0046] A network (400) according to embodiments of the present invention can use various wired communication systems such as a Public Switched Telephone Network (PSTN), xDSL (x Digital Subscriber Line), RADSL (Rate Adaptive DSL), MDSL (Multi Rate DSL), VDSL (Very High Speed ​​DSL), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and a Local Area Network (LAN).

[0047] In addition, the network (400) presented here may use various wireless communication systems such as CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA (Single Carrier-FDMA), and other systems.

[0048] A network (400) according to embodiments of the present invention may be configured regardless of the mode of communication, such as wired or wireless, and may be configured as various communication networks, such as a Personal Area Network (PAN) or a Wide Area Network (WAN). Additionally, the network (400) may be a known World Wide Web (WWW) and may utilize wireless transmission technologies used for short-range communication, such as Infrared Data Association (IrDA) or Bluetooth. The technologies described in this specification may be used in other networks as well as the networks mentioned above.

[0049] According to an embodiment of the present invention, a computing device (100) (hereinafter referred to as the 'computing device (100)') that performs a method for classifying and detecting cells in AI-based pathology images can efficiently process a vast amount of digital pathology images and precisely analyze the location, shape, staining intensity, and distribution of cells within each image to provide rapid and accurate diagnostic information to medical staff and researchers.

[0050] More specifically, the computing device (100) can acquire multiple pathology images and perform a preprocessing process to process them into a state optimized for analysis. Through the preprocessing process, noise in the pathology images can be removed, and the brightness, contrast ratio, color balance, etc. of the images can be adjusted to convert the images into a form optimized for training data. A training data set for training a neural network can be constructed based on the noise-removed pathology images, and the computing device (100) can generate a pathology image analysis model based on this to detect cell distribution within the pathology images and analyze the staining intensity of each cell.

[0051] In one embodiment, the pathology image analysis model may include a cell counting model and a staining intensity classification model. The cell counting model can accurately detect the location and morphology of cells within the pathology image and identify the characteristics and distribution of individual cells by segmenting the boundaries between cells. Through this, the number of cells within the image can be counted, and the location and shape of each cell can be analyzed to classify them as positive or negative cells. The cell counting model is composed of a dimensionality reduction network function (e.g., an encoder) and a dimensionality restoration network function (e.g., a decoder). By utilizing the dimensionality reduction network function and the dimensionality restoration network function, features of image patches can be extracted and the extracted features restored to generate a Binary Nuclei Map representing the segmented regions of cells, Horizontal and Vertical Distance Maps for identifying distance information between cells, and Nuclei Type Maps classified according to cell characteristics. The generated maps are used to clearly identify and visualize the location, boundaries, and types of cells within the pathology image, thereby enabling the identification and analysis of individual cells by accurately segmenting the overlaps or boundaries between cells. By precisely distinguishing the characteristics of each cell through these maps, it is possible to quantitatively analyze not only the exact location and shape of cells but also the distances and distribution between them. This enables accurate classification of cells within an image as positive or negative, and by comprehensively identifying the interactions and distribution patterns between various cell types, it can provide highly reliable information necessary for medical professionals to diagnose disease progression or evaluate prognosis.

[0052] Meanwhile, the staining intensity classification model is configured to analyze the expression level of specific biomarkers and may include multiple staining intensity classification sub-models that classify the staining intensities of different biomarkers. For example, the first staining intensity classification sub-model can identify the staining intensity for human epidermal growth factor receptor 2 (HER2), and the second staining intensity classification sub-model can identify the cell type and characteristics by analyzing the staining intensity for estrogen and progesterone receptors (ER / PR). Through this staining intensity classification model, detailed classification of the staining intensity of each cell is performed, and the expression status of intracellular biomarkers can be quantified to provide information for diagnosis and prognosis evaluation.

[0053] The computing device (100) of the present invention enables accurate and efficient analysis of pathology images by comprehensively performing cell detection and classification and biomarker staining intensity analysis using a pre-trained pathology image analysis model.

[0054] The computing device (100) of the present invention provides effects such as accurate classification and detection of cells, analysis of the expression levels of various biomarkers, and enhancement of data scalability and flexibility through automatic analysis of pathology images, and can provide innovative pathological information in the field of diagnosis and research.

[0055] That is, the computing device (100) of the present invention can provide efficient processing of a vast amount of pathology images and accurate diagnostic information by utilizing a pre-trained pathology image analysis model to rapidly detect the location and shape of cells and precisely analyze the staining intensity of each cell. In particular, by classifying the characteristics of individual cells within the pathology image through a cell counting model and quantifying the expression status of intracellular biomarkers in detail through a staining intensity classification model, it enables medical staff to consistently grasp the information necessary for determining the disease progression, prognosis, and treatment strategy. This analysis process improves speed and accuracy compared to conventional manual methods, not only increasing the reliability of diagnostic results but also helping researchers discover new insights from pathological data.

[0056] In addition, in the embodiment, the computing device (100) develops various functions as independent services to enhance scalability, flexibility, and maintainability, and enables stable pathology image analysis. The computing device (100) is designed based on a microservices architecture, so that each function, such as cell detection, staining intensity analysis, data preprocessing, and postprocessing, is implemented individually. For example, the cell detection service is responsible for extracting the location and shape of cells from an image, and the staining intensity analysis service classifies each cell by identifying the biomarker expression intensity. Since these services can be developed and deployed independently, when adding a new analysis algorithm or updating a model, only the relevant service needs to be updated, and maintenance and expansion are possible without interrupting the entire system. According to the embodiment, the computing device (100) reduces coupling between services through a microservices architecture and allows each service to be independently expanded and coordinated. For example, if the input volume of pathology image data suddenly increases, additional containers for the data preprocessing service can be deployed to increase throughput. This structure minimizes dependencies between services, allowing the training, prediction, and analysis of neural network models to be executed independently, and enables flexible response to the addition of new features or changes in data volume.

[0057] In addition, in an embodiment, the computing device (100) adopts an event-driven architecture to manage message transmission and reception between the image processing pipeline and internal services through event-based asynchronous processing. This allows for efficient handling of interactions between each service and improves service response speed and processing efficiency by effectively managing asynchronous operations. For example, when pathology image preprocessing is completed, the corresponding event is automatically transmitted to the neural network learning service to start the work, and through this asynchronous processing, a large volume of pathology image analysis work can be performed quickly without bottlenecks.

[0058] In one embodiment, the computing device (100) may be characterized by virtualizing each service into a Docker container to improve availability and resilience, and enabling an automatic restart option for the container to respond to unexpected service interruptions or failures. That is, by containerizing all services and components, the convenience of management and deployment is improved, and rapid recovery is possible in the event of a service interruption. For example, if an error occurs in a specific service during a pathology image analysis task, the corresponding service container can be immediately restarted through the automatic restart function to resume the task. Through this configuration, the stability and availability of the entire system are enhanced, and security can be maintained by restricting access from the outside.

[0059] In addition, in the embodiment, the computing device (100) performs user authentication and authorization management and can protect all communications through encryption technology such as TLS (Transport Layer Security). This maintains the confidentiality and integrity of data during the transmission and storage of sensitive data, such as pathology images, and ensures the safety of the system in response to external threats. For example, when medical personnel remotely view pathology image analysis results, the communication is encrypted to minimize the risk of data leakage or tampering.

[0060] As such, the computing device (100) of the present invention can perform AI-based cell classification and detection of pathology images in an environment with enhanced scalability, flexibility, availability, and security, which enables the processing of a large volume of images and the derivation of rapid analysis results. Through this, medical personnel can efficiently perform disease diagnosis and prognosis evaluation through accurate and reliable analysis results, and researchers can conduct research by discovering new insights from pathological data. A detailed description of the AI-based cell classification and detection method of pathology images according to the present invention will be provided later with reference to FIGS. 3 to 8.

[0061] In the embodiment, only one computing device (100) in FIG. 1 is illustrated, but it will be obvious to those skilled in the art that more servers may also be included within the scope of the invention and that the computing device (100) may include additional components. That is, the computing device (100) may be composed of multiple computing devices. In other words, a set of multiple nodes may constitute the computing device (100).

[0062] According to one embodiment of the present invention, the computing device (100) may be a server that provides cloud computing services. More specifically, the computing device (100) may be a server that provides cloud computing services, which are a type of internet-based computing, where information is processed by another computer connected to the internet rather than the user's computer. The cloud computing service may be a service that stores data on the internet and allows users to access necessary data or programs anytime and anywhere via internet access without installing them on their own computers, and allows data stored on the internet to be easily shared and transmitted through simple operations and clicks. Furthermore, the cloud computing service may not only simply store data on a server on the internet but also allow users to perform desired tasks using the functions of applications provided on the web without installing separate programs, and may be a service that allows multiple people to work while sharing documents simultaneously. Additionally, the cloud computing service may be implemented in at least one form among IaaS (Infrastructure as a Service), PaaS (Platform as a Service), SaaS (Software as a Service), a virtual machine-based cloud server, and a container-based cloud server. That is, the computing device (100) of the present invention may be implemented in at least one form among the cloud computing services described above. The specific description of the aforementioned cloud computing service is merely an example and may include any platform for establishing the cloud computing environment of the present invention.

[0063] A user terminal (200) according to an embodiment of the present invention may refer to any type of node(s) in a system having a mechanism for communication with a computing device (100). The user terminal (200) is a terminal capable of receiving pathology image analysis results and cell classification-related information through information exchange with the computing device (100), and may refer to a terminal such as a smartphone, tablet, or PC possessed by a user. For example, the user terminal (200) may be a terminal related to a researcher, clinician, or medical professional who intends to analyze a patient's pathology image. Additionally, the user terminal may be a terminal related to a specialist who intends to determine the direction of treatment based on the analysis results. Here, the specialist is a medical professional and may include, but is not limited to, pathologists, oncologists, surgeons, or internists.

[0064] When the user terminal (200) is a terminal related to a specialist, the pathology image analysis results and cell classification information received from the computing device (100) can be utilized as a medical assistance terminal for diagnosis determination, treatment planning, and prognosis evaluation. Such a user terminal is equipped with a display to receive user input and provide the user with output of any form, such as a cell distribution graph, staining intensity classification results, a comprehensive diagnostic report, etc. Through this interface, the specialist can intuitively check the pathology image analysis data provided from the computing device (100) and effectively utilize the detailed information necessary to determine the treatment plan.

[0065] A user terminal (200) may refer to any form of entity(s) in a system having a mechanism for communicating with a computing device (100). For example, such a user terminal (200) may include a PC (personal computer), a notebook, a mobile terminal, a smartphone, a tablet PC, and a wearable device, and may include any type of terminal capable of connecting to a wired or wireless network. Additionally, the user terminal (200) may include any server implemented by at least one of an agent, an API (Application Programming Interface), and a plug-in. Additionally, the user terminal (200) may include an application source and / or a client application.

[0066] In one embodiment, an external server (300) may be connected to a computing device (100) via a network (400) and may provide various information or data necessary for the computing device (100) to perform an AI-based method for cell classification and detection of pathology images, or receive, store, and manage result data derived from performing said method. For example, the external server (300) may be a storage server separately provided outside the computing device (100), but is not limited thereto.

[0067] In various embodiments, the external server (300) may be a server that stores pathology images for multiple users. For example, the external server (300) may be at least one of a hospital server, a medical institution server, or a government database server, and may be a server that stores information regarding pathology image data, patient clinical information, medical records, etc. related to multiple users. Such an external server can reliably store and manage large volumes of image data, thereby helping the computing device (100) to quickly search for and process data required for analysis.

[0068] For example, the external server (300) may include information such as pathology image data corresponding to each of the multiple users, patient diagnosis and treatment information, expression patterns of biomarkers, and cell classification results. The information stored in the external server (300) can be used as training data, verification data, and test data for training the neural network of the present invention. That is, the external server (300) may be a server that stores a data set for training the deep learning model of the present invention (e.g., a cell counting model or a staining intensity classification model). The computing device (100) can use the data stored in the external server (300) to improve the accuracy of the neural network model and increase the reliability of image analysis.

[0069] The external server (300) is a digital device and may be a digital device equipped with a processor and memory, such as a laptop computer, notebook computer, desktop computer, web pad, or mobile phone. Additionally, the external server (300) may be a web server processing services, a cloud-based database server, or a medical information system server. The types of external servers described above are merely examples and the present invention is not limited thereto.

[0070]

[0071] FIG. 2 is a hardware configuration diagram of a computing device that performs a method for classifying and detecting cells in an AI-based pathology image related to one embodiment of the present invention.

[0072] Referring to FIG. 2, a computing device (100) for performing a method for classifying and detecting cells in an AI-based pathology image related to one embodiment of the present invention may include one or more processors (110), a memory (120) for loading a computer program (151) executed by the processor (110), a bus (130), a communication interface (140), and a storage (150) for storing the computer program (151). Here, FIG. 2 only illustrates components related to the embodiment of the present invention. Therefore, a person skilled in the art to which the present invention pertains will understand that other general-purpose components may be included in addition to the components illustrated in FIG. 2.

[0073] According to one embodiment of the present invention, the processor (110) can typically handle the overall operation of the computing device (100). The processor (110) can provide or process appropriate information or functions to a user or user terminal by processing signals, data, information, etc. that are input or output through the components described above, or by running an application program stored in memory (120).

[0074] Additionally, the processor (110) can perform operations for at least one application or program for executing the method according to embodiments of the present invention, and the computing device (100) may have one or more processors.

[0075] According to one embodiment of the present invention, the processor (110) may be composed of one or more cores and may include a processor for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device.

[0076] The processor (110) can read a computer program stored in memory (120) to perform a method for classifying and detecting cells in an AI-based pathology image according to one embodiment of the present invention.

[0077] In various embodiments, the processor (110) may further include RAM (Random Access Memory, not shown) and ROM (Read-Only Memory, not shown) for temporarily and / or permanently storing signals (or data) processed within the processor (110). Additionally, the processor (110) may be implemented in the form of a system-on-chip (SoC) comprising at least one of a graphics processing unit, RAM, and ROM.

[0078] Memory (120) stores various data, instructions and / or information. Memory (120) may load a computer program (151) from storage (150) to execute a method / operation according to various embodiments of the present invention. When the computer program (151) is loaded into memory (120), the processor (110) may perform the method / operation by executing one or more instructions constituting the computer program (151). Memory (120) may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0079] The bus (130) provides communication functions between components of the computing device (100). The bus (130) can be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0080] The communication interface (140) supports wired and wireless internet communication of the computing device (100). Additionally, the communication interface (140) may support various communication methods other than internet communication. To this end, the communication interface (140) may be configured to include a communication module well known in the art of the present invention. In some embodiments, the communication interface (140) may be omitted.

[0081] Storage (150) can store computer programs (151) non-temporarily. When performing a cell classification and detection process of AI-based pathology images through a computing device (100), storage (150) can store various information necessary to provide a process of classifying and detecting cells of AI-based pathology images.

[0082] The storage (150) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0083] A computer program (151) may include one or more instructions that cause a processor (110) to perform a method / operation according to various embodiments of the present invention when loaded into memory (120). That is, the processor (110) may perform the method / operation according to various embodiments of the present invention by executing the one or more instructions.

[0084] In one embodiment, the computer program (151) may include one or more instructions for performing an AI-based method for cell classification and detection of pathology images, comprising the steps of acquiring a plurality of pathology images, performing preprocessing on a plurality of pathology images, constructing a training data set based on the preprocessed plurality of pathology images, and generating a pathology image analysis model using the training data set.

[0085] The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0086] The components of the present invention may be implemented as a program (or application) and stored on a medium to be executed in combination with a computer, which is hardware. The components of the present invention may be executed as software programming or software elements, and similarly, embodiments may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, etc., including various algorithms implemented as a combination of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors. Hereinafter, with reference to FIGS. 3 to 8, a method for cell classification and detection of an AI-based pathology image performed by a computing device (100) will be described in detail.

[0087]

[0088] FIG. 3 illustrates an exemplary flowchart of an AI-based method for cell classification and detection of pathology images related to an embodiment of the present invention. According to an embodiment of the present invention, the AI-based method for cell classification and detection of pathology images may include the steps illustrated in FIG. 3. The order of the steps illustrated in FIG. 3 may be changed as necessary, and at least one step may be omitted or added. That is, the following steps are merely an embodiment of the present invention, and the scope of the present invention is not limited thereto.

[0089] According to one embodiment of the present invention, the method for cell classification and detection of an AI-based pathology image may include the step (S110) of acquiring a pathology image. According to one embodiment, a computing device (100) may generate a digital image from a tissue or cell sample slide to generate key data necessary for pathological analysis.

[0090] In the embodiments, the pathology image can be acquired by staining a specimen prepared primarily through biopsy or cytology onto a pathology slide and then converting it into a high-resolution digital image using a digital pathology scanner. The acquired pathology image takes the form of a Whole Slide Image (WSI) and can be scanned at a high magnification to include all areas of the pathology slide. For example, the WSI can be captured at a magnification of 40x or more to allow for the analysis of cellular-level details.

[0091] According to one embodiment, in the image acquisition process, a slide with immunohistochemistry (IHC) staining is used, and such staining can visually highlight the expression status of specific proteins (e.g., HER2, ER, PR). In particular, by using a diaminobenzidine (DAB) stain, the location and staining intensity of specific biomarkers can be clearly expressed, and visual information necessary for pathological analysis can be provided.

[0092] According to the embodiments, a digital pathology scanner can divide a pathology slide into tile units by dividing it into regular intervals, capture data, and then combine them to generate a single high-resolution digital image, i.e., a pathology image. During this process, the slide focus is automatically adjusted, and corrections can be made to maintain color uniformity. Some scanners can precisely capture the fine structure of the slide through multi-focus scanning or spectral analysis-based scanning.

[0093] Pathology images acquired in this manner contain pathological information such as cell density, distribution, and staining intensity, and can be utilized as foundational data for evaluating pathological conditions. These images serve as input data for cell classification, density assessment, and biomarker status analysis during subsequent analysis processes, assisting users (e.g., medical professionals) in accurately assessing the pathological status of tissues.

[0094] According to one embodiment of the present invention, the AI-based method for cell classification and detection of a pathology image may include a step (S120) of performing preprocessing on the pathology image. The preprocessing process for the pathology image is a process of processing the pathology image data into an analyzable form, and is performed to further clarify pathological features and improve analysis accuracy.

[0095] According to one embodiment, the step of performing preprocessing on a pathology image may include identifying a region of interest (ROI) in the pathology image, dividing the identified region of interest into multiple patches of a predetermined size, and performing pixel value normalization, color scaling, and contrast adjustment on the region of interest divided into multiple patches.

[0096] More specifically, pathology images consist of whole slide images (WSIs) obtained by scanning samples from tissue or cytological examinations with a digital pathology scanner. While these WSIs cover the entire area of ​​the slide, the data required for pathological analysis is often limited to only a portion of the slide.

[0097] In one embodiment, the computing device (100) identifies an ROI requiring pathological analysis in the WSI through an automated algorithm. For example, the computing device may automatically detect areas of high cell density or areas with prominent specific staining intensities on the slide and set them as ROIs. The ROIs may be specified by a predefined size or shape, and surrounding areas without pathological significance may be excluded from the analysis.

[0098] In another embodiment, the computing device (100) may manually designate an ROI based on user input or preset criteria. For example, medical personnel may directly select a specific pathological area of ​​interest through a digital pathology image viewer or define an area where a specific staining intensity is observed to set it as an ROI. This method may be useful when analyzing images containing complex pathological features where it is difficult for automated algorithms to accurately identify the ROI.

[0099] In another embodiment, the computing device (100) can dynamically adjust the ROI based on the staining intensity distribution of the pathology image. For example, if the ROI set in the initial automatic detection step has non-uniform cell density or staining intensity distribution, the computing device (100) analyzes these characteristics to expand or reduce the size of the ROI to obtain optimal analysis results. In this process, areas within the ROI where the cell density is below a reference value or the staining intensity is lower than a specific threshold value may be excluded from the analysis.

[0100] In addition, when processing multiple ROIs simultaneously, the computing device (100) may analyze each ROI independently or perform an integrated analysis by considering the pathological association between adjacent ROIs. For example, if a pattern in which the same biomarker appears in adjacent ROIs is observed, the computing device may merge the corresponding areas into a single ROI to increase analysis efficiency.

[0101] In this way, the computing device (100) can identify an ROI with concentrated pathological characteristics through an automated algorithm, user input, or dynamic adjustment, and support efficient and precise analysis based thereon. These various embodiments can be flexibly utilized depending on the characteristics of the pathological data and analysis requirements.

[0102] Additionally, in the embodiment, when an ROI (or region of interest) is identified, the computing device (100) may divide the identified ROI area into multiple patches of a preset size. Multiple patches generally refer to image blocks divided into a fixed size for analysis efficiency and accuracy. For example, if the ROI has a size of 1024x1024 pixels, it may be divided into patches of 256x256 pixels to generate a total of 16 patches. The specific description of the aforementioned figures is merely an example, and the present invention is not limited thereto. Such division allows each patch to be analyzed independently and further helps to evaluate micropathological features within the ROI more finely.

[0103] In addition, in the embodiments, preprocessing operations such as pixel value normalization, color scaling, and contrast adjustment may be performed on regions of interest divided into multiple patches. For example, normalization can be used to increase the consistency of model training and analysis by scaling the value of each pixel to a range of 0 to 1. Color scaling can correct staining deviations in pathological images to minimize differences in staining intensity that may occur from slide to slide. This process may be essential to address the problem that the concentration of dyes used for staining, such as DAB, Hematoxylin, or Eosin, may vary from slide to slide. Contrast adjustment can enhance brightness contrast between pixels to more clearly reveal pathological features (e.g., cell nuclei, cytoplasm, staining intensity, etc.).

[0104] For example, if there are cells with weak staining intensity within a specific ROI, they can be highlighted through contrast adjustment so that they can be clearly distinguished. Such preprocessing supports the cell counting model and staining intensity classification submodel performed thereafter to effectively analyze pathological features. Additionally, patches containing ROIs can be processed sequentially or in parallel by the computing device (100), which can contribute to the efficient analysis of a large amount of pathological data.

[0105] Therefore, the preprocessing step of pathology images can improve the accuracy and reliability of pathological analysis by converting raw pathology data into an analyzable and refined form.

[0106] According to one embodiment of the present invention, a method for classifying and detecting cells in an AI-based pathology image may include the step (S130) of generating analysis information corresponding to a pathology image by utilizing a pathology image analysis model.

[0107] According to one embodiment, the computing device (100) can generate a pathology image analysis model through learning one or more network functions.

[0108] More specifically, the computing device (100) can build a training data set to generate a pathology image analysis model and perform training on an artificial intelligence model based thereon. The training data set includes pathology images containing various pathological conditions and reference standard data for said images, and may include pathological information such as cell location, type, and staining intensity. The computing device (100) can utilize the training data set to train one or more network functions specialized for pathology image analysis (e.g., cell counting model, staining intensity classification model) to enable precise analysis of the cell distribution and biomarker status of the pathology images.

[0109] Below, we will explain in detail the process of constructing the training dataset and the process of performing pre-training on the pathology image analysis model using the training dataset.

[0110] According to one embodiment, a method for generating a pathology image analysis model may include the step of acquiring a plurality of pathology images. The plurality of pathology images relates to image data for diagnosing or analyzing the pathological condition of patient tissue, and may relate to digital scan images of tissue slides, images of cell specimens, or microscopic images of stained tissue. For example, each of the plurality of pathology images may be a stained cell tissue image showing the expression status of biomarkers such as human epidermal growth factor receptor 2 (HER2), estrogen receptor (ER), and progesterone receptor (PR), and may be an image collected according to various staining protocols to distinguish between cancer cells and normal cells. The plurality of pathology images may be utilized for training a neural network and may be used to train a model to analyze the location, morphology, staining intensity, etc. of cells.

[0111] According to one embodiment of the present invention, the acquisition of a plurality of pathology images may involve receiving or loading a plurality of pathology image data stored in a memory (120). Additionally, the acquisition of a plurality of pathology images may involve receiving data from another storage medium, such as a cloud server, an external storage device, or an external database, based on wired or wireless communication means, or loading data from another computing device or a separate processing module (e.g., a data collection module) within the same computing device. Through this, the computing device (100) can collect pathology images from various sources and acquire data necessary for analysis quickly and flexibly.

[0112] According to one embodiment of the present invention, a method for generating a pathology image analysis model may include the step of performing preprocessing on a plurality of pathology images.

[0113] According to an embodiment, the computing device (100) can perform size conversion on a plurality of pathology images. For example, an ROI region can be found in WSI, cropped into an image, and the image can be subdivided and classified into patches of size 1024x1024. At this time, the original image of size 1024x1024 can be converted into an image of size 512x512 by applying a stride of 128 pixels. Through this, a smaller size image can be obtained, and a neural network model optimized for that size can be utilized.

[0114] Additionally, the computing device (100) can enhance the diversity of training data by applying data augmentation techniques, such as rotation, symmetry transformation, and inversion, to each of the multiple pathology images. Generally, medical data, particularly pathology images, may have limitations such as difficulty in data acquisition and variability between images. This is because the collection and acquisition of data are limited due to various factors such as the specificity of each patient's tissue, differences in staining methods, and scanning conditions, and consequently, it may often be difficult to obtain a sufficient amount of training data.

[0115] To overcome this, the computing device (100) can increase the diversity and volume of data by using data augmentation techniques to transform the existing data set in various ways. For example, by applying augmentation techniques such as random rotation at 90-degree intervals and inversion transformations in the up, down, left, and right directions to the training data, the volume of the existing data can be increased by 16 times. Through this augmentation process, the model is exposed to various shape transformations and visual variations, allowing the neural network to acquire consistent generalization capabilities even with a limited amount of data. This enables the neural network model to have robust recognition capabilities for pathology images under various conditions and to effectively analyze tissue images and cell morphologies from various angles and directions that may be encountered in actual clinical settings. Through data augmentation, the neural network model can perform more accurate cell detection and classification, thereby improving the performance and reliability of pathology image analysis.

[0116] Additionally, the computing device (100) can perform normalization on a plurality of pathology images and augmented pathology images. Specifically, normalization is a process to improve the stability and convergence speed of model learning by adjusting the distribution of the data.

[0117] First, the computing device (100) can perform scaling of each pixel value to a value between 0 and 1. This scaling is intended to allow the neural network model to learn all pixel values ​​evenly by adjusting the brightness and color information of the image to be distributed within a certain range. For example, if the original pixel value has a range of [0, 255], scale normalization of the image data is achieved by adjusting it to a value between [0, 1].

[0118] Afterward, the computing device (100) can perform the process of subtracting 0.5 from the average value of each channel (e.g., R, G, B channels) and dividing by the standard deviation of 0.5. This process may be a process of adjusting the data distribution of each channel to a normal distribution with a mean close to 0 and a standard deviation of 1. This normalization can minimize the difference in distribution between channels and prevent the model from being biased or overfitted by a specific channel. Through this, all channels of the data can participate in model training with equal importance. The range of the input data is converted to [-1, 1], which allows the model's activation function or weight updates to be performed consistently and in a balanced manner.

[0119] Through the aforementioned normalization process, the model becomes less sensitive to the distribution variability of image data and can effectively learn the features of each image. Therefore, it is possible to reduce overfitting during the training process and ensure consistent performance and training stability of the model in handling data diversity.

[0120] In addition, according to one embodiment, a method for generating a pathology image analysis model may include the step of constructing a training data set based on a plurality of preprocessed pathology images.

[0121] More specifically, the computing device (100) can classify and organize data sets necessary for learning by considering features such as image type, staining intensity, and cell shape based on preprocessed pathology images.

[0122] In an embodiment, the computing device (100) can enhance the learning, performance evaluation, and generalization capabilities of a neural network model by separating the constructed learning data set into training data, validation data, and test data. For example, training data is used to train the model, and during the training process, the model becomes able to learn various patterns such as the location and shape of cells within pathology images and staining intensity. Validation data is used to evaluate the performance of the model during training and to adjust hyperparameters, helping the model to have generalization capabilities without overfitting to specific data. Additionally, test data can be used to measure the classification and detection performance of the model in practice after the model training is completed.

[0123] In one embodiment, the computing device (100) may combine label data corresponding to each pathology image (e.g., positive / negative cell information, staining intensity classification information) to form input-output pairs necessary for model training, and through this data, enable the neural network model to acquire accurate prediction and analysis capabilities for the input images. The constructed training data set is configured considering the quantity and quality of the data and various variability, and the computing device (100) utilizes this to enable the neural network model to consistently learn various features of the pathology images.

[0124] Consequently, the computing device (100) performs learning for accurate cell detection and staining intensity classification of a neural network model based on a learning data set built from multiple pathology images, thereby optimizing the performance and reliability of the model.

[0125] Meanwhile, according to one embodiment of the present invention, the accuracy of cell detection and classification can be further improved through a post-processing step following learning and analysis. In the post-processing step, duplicate removal, boundary refinement, and exclusion of unnecessary instances are performed on the detected cell instances. This process is intended to identify the precise location and shape of the cells and to consider the relationships between cells in detail, thereby contributing to increasing the accuracy of the final cell classification result.

[0126] More specifically, the computing device (100) can select only instances where the center point is located within a given ROI during the post-processing process, thereby excluding cell instances outside the ROI. This ensures the accuracy of cell detection and minimizes calculations for unnecessary areas outside the ROI. Additionally, by removing instances that are touching the edges of the image and overlapping with other instances, partial segmentation errors that may occur at the boundaries can be prevented.

[0127] Subsequently, all non-overlapping instances are marked as preservation targets, and a spatial index of the instances is created using a STRtree. A STRtree is a data structure for the efficient search and management of spatial data, which is used to index the location of each instance and identify overlapping instances. The computing device (100) can search for other instances that spatially overlap with each instance and calculate the Intersection over Union (IoU) with the overlapping instances. The IoU represents the ratio of the overlapping area and can group instances that exceed a specified threshold (e.g., 0.8).

[0128] The computing device (100) can effectively process duplicate cell detection and improve the reliability of the final cell classification result by selecting and preserving the instance with the largest area among the grouped instances. Through this post-processing process, instances outside the ROI can be removed, and duplicate predictions at the boundary and overlaps between adjacent cells can be effectively adjusted to increase the accuracy and reliability of the segmentation result.

[0129] The computing device (100) can automatically perform such post-processing, thereby providing more sophisticated and consistent cell classification results based on what the neural network model has learned. Accordingly, the present invention enables accurate pathology image analysis by organically combining each step of pre-processing, learning, and post-processing, and thereby supports medical professionals in making rapid and accurate diagnoses.

[0130] In addition, according to one embodiment, a method for generating a pathology image analysis model may include the step of generating a pathology image analysis model by utilizing a training data set.

[0131] According to one embodiment, the pathology image analysis model may include a cell counting model and a staining intensity classification model, which are pre-trained neural network models for detecting cell distribution within a pathology image and analyzing the staining intensity of each cell.

[0132] In the example, the cell counting model may be characterized by being pre-trained to detect the location and shape of cells in a plurality of pathology images and to classify them as positive or negative according to the characteristics of each cell.

[0133] The cell counting model may be characterized by comprising a dimensionality reduction submodel and a dimensionality restoration submodel, extracting features of each patch within the pathology image through the dimensionality reduction submodel, analyzing the extracted features through the dimensionality restoration submodel to generate multiple maps, and predicting the arrangement and characteristics of cells based on the generated multiple maps. Here, the multiple maps may include a binary nuclear map, horizontal and vertical distance maps, and a nuclear type map.

[0134] As a specific example, cell counting models are designed based on Vision Transformers (ViTs) to perform accurate detection and classification of cells within pathology images. Vision Transformers are an advanced form of Convolutional Neural Networks (CNNs) specialized for image processing; they utilize attention mechanisms to understand the interrelationships between cells in an image and can possess a structure that improves the performance of semantic segmentation tasks.

[0135] Specifically, the dimensionality reduction network function (i.e., the Encoder) borrows the structure of ViTs to extract visual features from image patches through linear projection and position embedding. In this process, each image patch can be processed within the encoder using a Multi-Head Attention mechanism. The Multi-Head Attention mechanism effectively learns the correlations between cells appearing in each patch of the image and identifies the spatial features of cell distribution and structure by considering the characteristics of each patch and their interactions with other patches. Through this process, the encoder can generate high-dimensional feature vectors that reflect the location, shape, and interrelationships of the cells within the image.

[0136] Subsequently, the dimensionality restoration network function (i.e., the Decoder) can perform dimensionality restoration and reconstruction based on features extracted from the encoder. The Decoder can analyze and integrate the feature vectors of the encoder through various paths and layers to ultimately generate multiple maps. In the embodiment, the multiple maps may include a binary nuclear map indicating the presence of cells, horizontal and vertical distance maps expressing the distance between cells, and nuclear type maps classified according to cell characteristics. The multiple maps clearly indicate the arrangement, distance, shape, and staining intensity of the cells, enabling the model to predict cell distribution and characteristics.

[0137] In the examples, the binary nuclear map is a binarized map used to identify the location of cell nuclei within each image, allowing for clear distinction between the background and the cells. Horizontal and vertical distance maps represent the distances and boundaries between cells, and are used to identify the distribution patterns and shapes of the cells; these maps can accurately represent the adjacency and boundary shapes between cells. The nuclear type map can identify and classify the type of each cell based on its staining intensity and biomarkers. Through these multiple maps, it is possible to predict whether each cell is positive or negative, and which biomarker it is expressing.

[0138] In the embodiments, the cell counting model can effectively learn and generate multiple maps by utilizing the constructed training dataset. During the training process, the encoder learns the fine structures of cells and variations in staining intensity from pathological images of various shapes and colors, and the decoder accurately reconstructs each map based on this information, ultimately optimizing the model's performance. This enables comprehensive cell detection and classification considering the correlations and spatial arrangement between cells, and allows the model to rapidly and accurately analyze pathological images in various situations.

[0139] In the example, the training dataset is divided into training data, validation data, and test data and used for training and evaluating the model. Through the training data, the model learns the characteristics and distribution of cells, and using the validation data, the model's performance is continuously evaluated and parameters are optimized. After training is complete, the final performance of the model is verified and generalization ability is evaluated using the test data.

[0140] In other words, the cell counting model comprehensively learns features such as the location, shape, and staining intensity of cells from various pathology images in the training dataset, thereby enabling effective identification of the complex structures of pathology images and minute differences between cells, which in turn enables accurate cell detection and classification.

[0141] According to one embodiment of the present invention, a staining intensity classification model may be configured to include a plurality of staining intensity classification sub-models that analyze the characteristics of different biomarkers.

[0142] A plurality of staining intensity classification submodels may include a first staining intensity classification submodel that analyzes staining intensity for human epidermal growth factor receptor 2 (HER2) and a second staining intensity classification submodel that analyzes staining intensity for estrogen and progesterone receptors.

[0143] More specifically, the first staining intensity classification submodel is optimized to determine the staining intensity of the HER2 biomarker and can perform intensity classifications such as negative, 1, 2, and 3 depending on the degree of HER2 expression in the tissue. In the example, since HER2 staining intensity acts as an important factor in determining the diagnosis and treatment direction for the growth and metastasis of cancer cells, the model can be trained to classify it accurately.

[0144] To explain in more detail, the first staining intensity classification submodel can be constructed based on the DenseNet structure. DenseNet is a Convolutional Neural Network (CNN)-based architecture featuring a structure where each layer is directly connected to all previous layers, offering the advantage of efficient learning through feature reuse. By utilizing this DenseNet structure as a backbone, the model can rapidly and accurately extract fine features of cells within an image.

[0145] Data for training may be provided from a training dataset consisting of preprocessed 512x512 image patches (i.e., image patches segmented from the ROI of a pathology image). A computing device (100) may locate the ROI in the WSI, cut out the corresponding region, and subdivide it into 512x512 patches. This process is intended to obtain data patches optimized for classifying the staining intensity of HER2, enabling detailed analysis of cell structures and staining status within each patch.

[0146] In the embodiment, the preprocessed images can be used for training a model after undergoing a normalization process. Normalization can be an important step to increase the learning stability of the data and to ensure that the model processes each pixel information evenly. Specifically, the computing device (100) can scale the value of each pixel from [0, 255] to [0, 1], and then subtract 0.5 for each channel (e.g., R, G, B) and divide by 0.5 to finally convert the range of the data to [-1, 1]. Data normalized in this way enables the model to stably learn various cell structures and changes in staining intensity.

[0147] The first staining intensity classification submodel based on DenseNet can extract image features through Global Average Pooling and Fully Connected layers, and based on this, classify HER2 staining intensity into four levels: 0 (negative), 1, 2, and 3. Global Average Pooling aggregates cell features within the image, and the Fully Connected layer performs the role of classifying into each class (intensity level) based on these features.

[0148] The architecture of DenseNet enables the model to effectively recognize and classify differences in cell structure, boundaries, and staining intensity within images through close connections between layers. The trained neural network model can be configured to learn the HER2 staining intensity patterns of each patch and accurately predict the HER2 expression status within actual tissues. This provides the information necessary to determine patient diagnosis and treatment strategies based on the status of HER2 biomarkers.

[0149] Consequently, the first staining intensity classification submodel can classify HER2 staining intensities within pathology images by utilizing a structure that maximizes feature reuse and learning efficiency of DenseNet, along with a training dataset that has undergone preprocessing and normalization. This classification serves as a basis for medical professionals to assess the condition of tissues and formulate appropriate treatment plans, and can provide clinically important information such as the degree of HER2 positivity or tissue response to treatment.

[0150] In an embodiment, the computing device (100) can perform a post-processing process to refine the cell distribution and staining intensity within the pathology image analyzed by the model and improve accuracy. Specifically, the computing device (100) can generate a mask based on staining intensity classification values ​​for each ROI patch classified to a size of 512x512. The mask reflects information regarding the location and staining intensity of cells within the ROI patch and can visually represent the cell state of the corresponding patch.

[0151] The computing device (100) merges the generated individual masks into a single integrated ROI mask. This enables comprehensive evaluation of the cell distribution and staining intensity of the entire ROI through a single mask, and allows for consistent analysis of all cells within the ROI.

[0152] Additionally, based on the integrated ROI mask, the computing device (100) can detect all cells within the ROI by utilizing a cell counting model. For each cell detected, the staining intensity classification value of the mask area where the cell is located can be checked to assign a suitable staining intensity score to each cell. Through this, the staining status of each cell within the ROI is quantitatively evaluated, and a positive or negative status is identified.

[0153] The computing device (100) can calculate the final staining intensity score of the ROI by utilizing the staining intensity scores of positive cells, excluding cells with a score of 0 (negative) among all cells present in the ROI. At this time, a weighted average of the staining intensity scores of positive cells can be calculated, and the calculated average value can be rounded to determine the final staining intensity score of the ROI.

[0154] Through this post-processing process, the computing device (100) combines precise analysis at the individual cell level with comprehensive evaluation at the ROI level to improve the accuracy and reliability of cell detection and staining intensity classification. This enables medical personnel to perform pathological analysis and diagnosis more reliably based on the accurate cell condition.

[0155] In addition, in the embodiments, the second staining intensity classification submodel may be a neural network model for learning and analyzing the DAB staining intensity of estrogen receptors (ER) and progesterone receptors (PR). Specifically, the second staining intensity classification submodel may classify the DAB staining intensity into four levels, assign a score to each cell, and calculate a comprehensive staining intensity score for all cells within the ROI. This learning process supports the model in accurately classifying the staining intensity and biomarker status of each cell within the pathology image.

[0156] According to an embodiment, a computing device (100) can perform image preprocessing. ER and PR images are separated into Hematoxylin, Eosin, and DAB channels, and the DAB channel can be utilized as a key channel for determining the degree of staining of positive cells. The characteristic staining pattern and intensity of the DAB channel are key information that a model needs to learn, and the data can be transformed so that this can be effectively extracted and analyzed during the image preprocessing stage.

[0157] Next, the computing device (100) performs the task of setting thresholds for each staining intensity based on the distribution of pixel values ​​within the DAB channel. At this time, threshold intervals according to staining intensity can be set through statistical analysis of various DAB values ​​within the training data set. Since each DAB staining intensity has a specific distribution, this is a step of establishing criteria to accurately distinguish them, and the model becomes able to classify the staining intensity of cells into stages based on these thresholds.

[0158] During the training process, the second staining intensity classification submodel utilizes a neural network structure to learn how to assign scores to individual cells. First, it calculates the average of the DAB channel values ​​for each cell region and learns which threshold range that average falls within. Based on this, it learns how to assign an appropriate staining intensity score to each cell. This enhances the model's ability to quantify the staining intensity of each cell and enables the neural network to respond to various staining patterns.

[0159] Through this process, the trained model becomes capable of calculating a comprehensive staining intensity score at the ROI level. Specifically, a weighted average is calculated using the scores of positive cells within the ROI, excluding those with a score of 0 (negative), and this is rounded to determine the final ROI score. During the training process, the model acquires the ability to accurately evaluate the DAB intensity of each cell and to comprehensively analyze the scores of cells within the ROI.

[0160] To this end, the model utilizes datasets classified into training, validation, and test data. The distribution and characteristics of DAB staining intensities are learned through the training data, while the model's classification performance is evaluated and parameters are adjusted using the validation data. Finally, the model's final performance and generalization ability can be evaluated using the test data.

[0161] Consequently, the second staining intensity classification submodel learns various DAB staining patterns and cell morphologies during the learning process, thereby accurately classifying the staining intensity of each cell and effectively evaluating the comprehensive biomarker status of the entire ROI. In other words, it has the advantage of precisely quantifying the staining intensity of ER and PR-positive cells to provide key information for establishing diagnostic and treatment strategies, and contributing to increasing the overall accuracy and efficiency of the analysis.

[0162] In summary, the computing device (100) of the present invention can efficiently perform a series of processes including preprocessing of a plurality of pathology images, construction of a training data set, training of a neural network model, cell detection and staining intensity classification, and postprocessing, thereby automating cell analysis of pathology images and improving accuracy. Through the preprocessing process, noise removal and conversion to an appropriate size of the pathology images are achieved, and through normalization, data uniformity and learning stability can be ensured.

[0163] The computing device (100) generates a neural network model (i.e., a pathology image analysis model) that detects the location and shape of cells and classifies staining intensity using a training data set, and can accurately identify detailed characteristics of cells within an ROI by utilizing the generated pathology image analysis model. In particular, the combination of a cell counting model and a staining intensity classification model enables accurate analysis of the positive / negative status of cells and the degree of expression of specific biomarkers. Additionally, through a post-processing process, unnecessary instances and redundant predictions are removed, thereby increasing the reliability of the results and improving the accuracy of cell detection and classification.

[0164] As described above, the computing device (100) of the present invention provides a pathology image analysis model to accurately analyze the location, shape, and staining intensity of cells within a pathology image, thereby providing rapid and consistent diagnostic information to medical staff and researchers. Through this, key information necessary for diagnosis and treatment can be effectively derived, and the efficiency and reliability of pathological data analysis can be improved. Furthermore, the computing device (100) can efficiently process and analyze a vast amount of pathology images by automating the AI-based cell classification and detection process, thereby enabling medical staff to quickly grasp accurate information regarding cell characteristics and utilize it for disease diagnosis and prognosis evaluation. This can contribute to the advancement of the medical and pathology fields by providing reliable information in clinical diagnosis, research, and treatment planning.

[0165] According to one embodiment, the trained pathology image analysis model may include a cell counting model and a staining intensity classification model, and can generate analysis information regarding the distribution, density, and staining intensity of cells within the pathology image by utilizing each model.

[0166] In one embodiment, a cell counting model can detect the location and boundaries of cells in a pathology image and classify the characteristics of each cell as positive or negative. This allows for the identification of cell density, spatial distribution, and arrangement within an ROI (Region of Interest), and the analysis results can be used to evaluate pathological abnormalities. For example, if cell density is abnormally high or low in a specific area, this can be used as a pathological indicator.

[0167] According to an embodiment, the cell counting model is configured to include a dimensionality reduction sub-model and a dimensionality restoration sub-model, wherein the dimensionality reduction sub-model extracts features for each of the multiple patch units, and the dimensionality restoration sub-model can generate multiple maps through analysis of the extracted features.

[0168] In an embodiment, the step of generating analysis information may include the step of determining spatial distribution, density, and classification information of cells based on a plurality of generated maps. Here, the plurality of maps may include a binary nuclear map, horizontal and vertical distance maps, and a nuclear type map.

[0169] More specifically, the cell counting model can detect the location of cells in preprocessed pathology images (more specifically, each patch classified from the ROI of the pathology image) and analyze the spatial relationships between cells to generate information on the distribution and density of cells within the ROI. In this process, the dimensionality reduction submodel can extract feature vectors representing the location, size, and boundaries of cells in multiple patches of the input image. Subsequently, the dimensionality restoration submodel can generate multiple maps, such as binary nuclear maps, horizontal and vertical distance maps, and nuclear type maps, based on the extracted features.

[0170] The multiple generated maps comprehensively represent the spatial relationships between cells and the individual characteristics of cells; through this, the cell counting model can analyze cell density and distribution within the ROI, as well as classification information for each cell, to generate analytical information related to the pathological condition.

[0171] For example, if cell density appears abnormally high in a specific ROI, it may indicate the potential for cancer cell proliferation, and if the intercellular distances show a dense pattern below a certain threshold, it may suggest a pathological condition such as an inflammatory disease. Furthermore, pathological characteristics within an ROI can be quantitatively evaluated based on the distribution of benign and negative cells.

[0172] In other words, cell counting models analyze cell density, distribution, and characteristics within pathology images to help medical professionals accurately diagnose pathological abnormalities and establish treatment plans.

[0173] According to one embodiment, the cell counting model may be characterized by applying an attention mechanism during the analysis process in the dimension restoration submodel to reflect spatial correlations between cells within a plurality of generated maps, predicting spatial distribution, density, and classification information of cells, and calculating the importance of the predicted information as a weight to improve the accuracy of the analysis results.

[0174] Specifically, the dimensionality reconstruction submodel utilizes an attention mechanism to learn spatial correlations between cells within pathology images and can analyze multiple maps generated based on this. The attention mechanism dynamically evaluates relationships between cells to calculate the importance of a specific cell's location and the distance between it and surrounding cells in pathological analysis. For example, in areas with high cell density or specific directional relationships between cells, the attention mechanism can calculate the importance of the corresponding area as a weight, enabling the model to incorporate this information into the analysis results.

[0175] In this process, the attention mechanism compares the feature vectors of all cells within the ROI and mathematically models the relationships between them. Specifically, it utilizes Multi-Head Attention techniques to analyze inter-cell relationships from various perspectives, enabling a comprehensive understanding of the spatial patterns of the entire ROI. This attention calculation reflects inter-cell interactions and is particularly useful for highlighting pathological features, such as the irregular cell distribution observed in tumor tissues or collective cellular responses in inflamed areas.

[0176] Weights generated through the attention mechanism are applied to multiple maps (e.g., binary nuclear map, horizontal and vertical distance map, nuclear type map) to accurately predict the spatial distribution and density information of cells. For instance, if the distance and directionality between cells appear abnormal in a specific region, the attention mechanism incorporates these specific patterns into the analysis results, enabling more accurate detection of pathological abnormalities.

[0177] In addition, the importance of predicted information can be dynamically weighted to determine the priority of pathological analysis within the ROI. For example, areas with high cell density and a high proportion of benign cells are considered more important indicators in the analysis, and the analysis results for these areas can be emphasized and presented to medical staff.

[0178] In other words, a cell counting model applying an attention mechanism can finely analyze the spatial correlations between cells within pathology images and generate highly reliable predictive information regarding cell distribution and characteristics, thereby improving the accuracy and efficiency of pathological analysis results.

[0179] In addition, in the embodiments, the step of generating analysis information may further include the step of performing post-processing on a plurality of maps. The post-processing on the plurality of maps may be intended to improve the accuracy of cell detection and classification within pathology images and to perform fine-tuning, such as removing duplicate instances.

[0180] The post-processing step for multiple maps may include selecting only cell instances where the center point is located within the ROI to exclude unnecessary instances outside the ROI, adjusting for partial segmentation errors that may occur at image boundaries, and removing duplicate predicted cell instances. This process is intended to increase the reliability of analysis results and provide clearer pathological information to medical professionals, and can contribute to ensuring the consistency and accuracy of the generated multiple maps. A detailed description of the post-processing step for multiple maps will be provided below with reference to Fig. 4.

[0181] According to one embodiment, the post-processing step for a plurality of maps may include the step (S210) of identifying important instances within a region of interest and determining them as instances to be preserved.

[0182] In the embodiments, an instance may refer to a cell or cell structure that is individually identifiable within a pathology image. This refers to a cell or cell aggregate represented in a binary nuclear map, a horizontal and vertical distance map, a nuclear type map, etc., where each instance possesses attributes such as spatial location, size, shape, and staining intensity. For example, in a binary nuclear map, an instance is defined by the boundary of a specific cell nucleus and may represent the shape and location of the corresponding cell.

[0183] According to an embodiment, the computing device (100) can determine important instances based on at least one of the cell shape, staining intensity, location and spatial relationship between cells.

[0184] More specifically, the computing device (100) can analyze the size and shape, i.e., form, of the cells to identify abnormally large cells or cells of a distorted shape as important instances. For example, cells that are more than twice the size of normal cells or have a distorted nucleus may indicate the presence of a tumor, so these cells may be preferentially preserved.

[0185] Additionally, the computing device (100) can evaluate the staining intensity of specific biomarkers and determine strongly stained cells as important instances. For example, cells classified as having a HER2 biomarker intensity of “3” can be an indicator of HER2-positive breast cancer, and thus play an important role in pathological analysis. Similarly, cells with high staining intensity of estrogen receptor (ER) and progesterone receptor (PR) can also be used as important indicators to evaluate responsiveness to hormone therapy.

[0186] Additionally, the computing device (100) can analyze the location and density of cells and consider patterns having abnormally high cell density in specific areas as important. For example, in inflamed tissue, cell density may appear to be more than twice as high as in surrounding areas. In this case, cells within that area can be identified as important instances and set as targets for analysis.

[0187] Additionally, the computing device (100) can evaluate the spatial relationships between cells and determine cells exhibiting abnormal arrangement or distribution as important instances. For example, in invasive cancer tissue, cells are often arranged randomly or form patterns with indistinctly connected boundaries. Cells constituting such arrangements have significant meaning in pathological analysis, and the computing device can automatically detect them and set them as instances to be preserved.

[0188] In this way, the computing device (100) can determine important instances by comprehensively considering various attributes and pathological patterns. For example, by simultaneously evaluating the morphology, staining intensity, density, and spatial relationships of cells within an ROI, it can provide pathological indicators such as cancer diagnosis or inflammatory response evaluation.

[0189] Additionally, according to an embodiment, the post-processing step for a plurality of maps may include the step (S220) of generating a spatial index corresponding to a preservation target instance and generating relationship information between the spatial indices.

[0190] Specifically, the computing device (100) can generate a spatial index based on structural information of the instance to be preserved (e.g., information such as location, size, shape, etc.). The spatial index is a data structure that efficiently represents the coordinates and boundaries of the instance to be preserved and can be used to systematically organize and manage the locations of instances across multiple maps. For example, by generating a spatial index using a data structure such as a STRtree (Spatial-Temporal R-tree), the relationships between instances can be explored quickly and efficiently.

[0191] Based on the generated spatial index, the computing device (100) can generate relationship information by analyzing the spatial relationships between each instance. For example, it can evaluate how close a specific instance is to another instance, whether there are overlapping areas between them, or whether they form the same pattern. This relationship information can serve as a basis for understanding the interactions between instances and deriving pathological meanings.

[0192] Additionally, relationship information can be utilized to identify pathological patterns based on the spatial distribution of instances. For example, if cells form abnormally dense clusters within a specific ROI or show a distribution spreading in a specific direction, this may indicate the possibility of invasive cancer. Based on this relationship information, the computing device (100) can further evaluate the pathological significance of instances and determine the priority of subsequent analysis.

[0193] In other words, the step of generating spatial indexes corresponding to instances to be preserved and generating relationship information between spatial indexes can serve as a basis for systematically organizing data for multiple maps and deriving pathological meanings.

[0194] Additionally, according to an embodiment, the post-processing step for a plurality of maps may include a step (S230) of utilizing relationship information between spatial indices to search for other instances that spatially overlap with each instance, and calculating the degree of overlap with the other instances found.

[0195] Specifically, the computing device (100) can search for other instances located within the same region of interest (ROI) as the instance to be preserved by referring to the spatial index of the instance to be preserved. The search process can be performed using a spatial data structure such as a STRtree (Spatial-Temporal R-tree), and can identify adjacent or overlapping instances based on the coordinates, boundaries, and size information of each instance.

[0196] The degree of overlap with the discovered instances can be calculated using IoU (Intersection over Union) or similar metrics. IoU is defined as the value obtained by dividing the overlapping area of ​​two instances by the total sum area, and it quantitatively represents the strength of the overlap. For example, if two instances overlap by more than 50% in the same area, the IoU value is 0.5 or higher, which indicates that the two instances have a strong spatial association.

[0197] The computing device (100) can contribute to quantifying spatial relationships between instances and increasing the reliability of pathological analysis by calculating the degree of overlap. For example, if a specific instance has a high degree of overlap with a number of adjacent instances, it may suggest that the instance is part of a cell cluster or group. Conversely, instances with a low degree of overlap may exist alone or be located in boundary areas, requiring separate processing criteria during the analysis process.

[0198] In other words, the step of calculating the degree of overlap by utilizing relationship information between spatial indices contributes to precisely analyzing the interactions between each instance and gaining a deeper understanding of pathological abnormalities. Through this process, the computing device can identify pathological patterns based on spatial associations between cells and provide highly reliable analytical information.

[0199] Additionally, according to an embodiment, the post-processing step for a plurality of maps may include a step (S240) of grouping instances having an overlap degree greater than or equal to a threshold threshold.

[0200] Specifically, the computing device (100) can group instances that overlap by more than a specified threshold based on the degree of overlap between each instance calculated earlier. Here, the threshold can be predefined according to the accuracy required in pathological analysis and the purpose of the analysis. For example, instances with an Intersection over Union (IoU) of 0.8 or higher may be considered to have the same spatial association and can be grouped into one.

[0201] The grouping process is intended to efficiently analyze complex cell clusters or pathological patterns by considering the continuous overlap between instances. For example, when an invasive pattern is observed in cancer tissue, the cells constituting that pattern are likely to appear adjacent or overlapping. Through such grouping, computing devices can clearly define the boundaries of specific cell clusters and enhance the reliability of pathological analysis.

[0202] Grouped instances can be utilized as important units of analysis in subsequent analyses. For example, they can be used to comprehensively evaluate the distribution of staining intensity among cells within a group or to derive pathological characteristics based on the spatial density and arrangement of cell clusters. Furthermore, grouping can contribute to improving data processing efficiency during the analysis process. In other words, instead of analyzing individual instances separately, evaluating pathological conditions using grouped instances as the unit can simultaneously enhance processing speed and accuracy.

[0203] According to one embodiment, the post-processing step for a plurality of maps may include a step (S250) of preserving the instance with the largest area within the instance group and removing the remaining instances.

[0204] Specifically, the computing device (100) can analyze area information for each grouped instance and determine the instance with the largest area within the group as a preservation target. Here, the largest area refers to the size of the region defined by the boundaries of each instance and can be used as a criterion for identifying pathologically important instances. For example, large cells with high staining intensity or major cells located at the center of the cluster are likely to be selected.

[0205] The computing device (100) can select instances to be preserved and then remove other instances within the same group. This process is intended to prevent duplicate cell detection and improve the accuracy of the analysis results. For example, if the same cell is detected redundantly in different maps within an ROI, the results can be refined by preserving only the representation with the maximum area of ​​the cell and removing the remaining duplicate representations.

[0206] The process of selecting instances based on area is useful for maximizing pathological significance. For example, in invasive cancer tissue, large cells in the center of a cell cluster are likely to strongly indicate pathological abnormalities. Conversely, small cells in the periphery or instances with indistinct boundaries can be excluded from the analysis. Through this, computing devices can improve the reliability and effectiveness of pathology image analysis results by preferentially preserving the most pathologically significant instances.

[0207] In other words, the step of preserving the instance with the largest area within an instance group and removing the rest provides a precise post-processing process that takes pathological significance into account. This minimizes errors caused by redundant data and maximizes analysis efficiency, thereby enhancing the accuracy and reliability of pathology image analysis.

[0208] According to one embodiment of the present invention, a staining intensity classification model may be configured to include a plurality of staining intensity classification sub-models that analyze the characteristics of different biomarkers.

[0209] A plurality of staining intensity classification submodels may include a first staining intensity classification submodel that generates analysis information for human epidermal growth factor receptor 2 (HER2) and a second staining intensity classification submodel that generates analysis information for estrogen receptor (ER) and progesterone receptor (PR).

[0210] For example, the first staining intensity classification submodel can utilize a DenseNet-based neural network structure to evaluate the staining intensity of the HER2 biomarker. The first staining intensity classification submodel analyzes the HER2 staining intensity of each cell in the input pathology image and classifies it into one of four levels: 0 (negative), 1, 2, or 3. The first staining intensity classification submodel processes each patch in the input image to evaluate the HER2 expression status and, based on this, can quantify the cell staining intensity value of each patch.

[0211] HER2 staining intensity is classified into four levels: 0 (negative), 1, 2, and 3, with each level serving as an important indicator for evaluating the growth and metastatic potential of cancer cells. The first staining intensity classification submodel analyzes the degree of HER2 expression within an ROI (region of interest) and can quantify the staining intensity of each cell based on this analysis. For example, cells with HER2 overexpression are classified as level 3, which can provide important data for determining treatment.

[0212] Meanwhile, the second staining intensity classification submodel can generate staining intensity information by analyzing the pixel value distribution of the DAB channel to evaluate the status of ER and PR biomarkers. The second staining intensity classification submodel classifies the staining intensity of positive cells into four stages, calculates the Proportion Score and Intensity Score for each cell, and then sums them to calculate the Allred Score. The calculated Allred Score comprehensively represents the ER and PR status at the cellular level within a specific ROI, playing an important role in breast cancer diagnosis and the prediction of treatment response. For example, staining intensity information at the ROI level can be generated by calculating the weighted average of the Allred Scores of positive cells within the ROI.

[0213] More specifically, the first dyeing intensity classification submodel may be provided to include a Global Average Pooling layer and a Fully Connected layer.

[0214] In an embodiment, the global average pooling layer generates a feature vector corresponding to each patch, and the fully connected layer classifies the dyeing intensity into at least one of a plurality of stages, and integrates the patch-specific classification results at the region of interest unit to predict the dyeing intensity distribution and generate analysis information.

[0215] To explain in detail, the first staining intensity classification submodel can utilize a DenseNet-based neural network structure and a global average pooling layer to evaluate the staining intensity of the HER2 biomarker. The global average pooling layer is responsible for generating feature vectors from each patch of the pathology image, and the generated vectors contain key information indicating the HER2 expression status within the patch. The generated feature vectors are passed to a fully connected layer, where the HER2 staining intensity can be classified into one of 0 (negative), 1, 2, or 3.

[0216] In the analysis process, the first staining intensity classification submodel calculates HER2 staining intensity values ​​for all patches within the ROI (Region of Interest) and integrates the calculated patch-specific classification results to predict the distribution of HER2 staining intensity across the entire ROI. Specifically, the computing device (100) can generate HER2 expression information at the ROI level by averaging the HER2 intensity values ​​derived from each patch or by applying a weighted average calculation method. For example, if more than 80% of the patches within the ROI are classified as 3, the ROI may be considered an area where HER2 is overexpressed. These analysis results provide key information for determining whether to perform HER2-targeted therapy.

[0217] Additionally, according to the embodiment, the step of generating analysis information may include the step of generating analysis information regarding the status of estrogen receptors and progesterone receptors by utilizing a second staining intensity classification sub-model. The second staining intensity classification sub-model evaluates the DAB staining intensity of each cell in an input pathology image and can quantitatively analyze the expression status of ER and PR through this.

[0218] The process of generating analysis information on the status of estrogen receptors and progesterone receptors using the second staining intensity classification sub-model will be described later with reference to Fig. 5.

[0219] Referring to FIG. 5, the step of generating analysis information on the state of estrogen receptors and progesterone receptors using a second staining intensity classification sub-model may include the step (S310) of separating a preprocessed pathology image into multiple channels.

[0220] Specifically, the computing device (100) can analyze the input preprocessed pathology image and separate it into Hematoxylin channels, Eosin channels, and DAB channels. At this time, each channel can be utilized to highlight different pathological features within the pathology image. For example, the Hematoxylin channel can be used to highlight the nuclear structure and location by staining the cell nucleus in blue, the Eosin channel can be used to represent tissue structure by staining the cytoplasm and extracellular matrix in pink, and the DAB channel can be used to distinguish between positive and negative cells by showing the expression intensity of specific biomarkers (e.g., estrogen receptors and progesterone receptors) in brown.

[0221] The computing device (100) can extract Hematoxylin, Eosin, and DAB channels, respectively, by analyzing the color components of each pixel based on the RGB color information of the digital pathology image. Specifically, the RGB channel data can be converted, and channel-specific information can be separated by applying a specific threshold value according to the color components of each channel. For example, the DAB channel can be extracted based on pixel values ​​corresponding to the brown color range. In this process, the information of each channel can be stored as individual images so that independent analysis is possible.

[0222] The step of separating into multiple channels is a process for evaluating the status of each biomarker in pathology images. By independently analyzing the information provided by each channel, it enables the quantitative assessment of tissue structure, cell morphology, and the expression intensity of specific biomarkers. In particular, the DAB channel is utilized to analyze the expression intensity of estrogen receptors and progesterone receptors, thereby providing the data necessary to classify staining intensities stepwise and generate analysis information in subsequent steps. Through this channel separation process, the computing device can accurately separate and preserve various pathological features within pathology images, thereby improving the reliability and accuracy of subsequent analyses.

[0223] Additionally, the step of generating analysis information on the state of estrogen receptors and progesterone receptors using a second staining intensity classification sub-model may include a step (S320) of analyzing the pixel value distribution of the DAB channel among multiple channels to set the range of staining intensity levels and deriving a threshold value for each level.

[0224] Specifically, the computing device can analyze the statistical distribution of pixel values ​​in the DAB channel to determine the range of pixel values ​​for which the staining intensity corresponds to each step (e.g., 0, 1, 2, 3). To do this, the pixel data of the DAB channel can be represented as a histogram, and intervals corresponding to each step can be set in the histogram. For example, when the range of pixel values ​​is [0, 255], the staining intensity can be divided into 4 steps based on specific intervals (e.g., 0-50, 51-100, 101-150, 151-255).

[0225] In one embodiment, the computing device (100) can automatically derive a threshold based on the distribution of pixel values ​​of DAB channels collected from a training data set. For example, by comparing the distribution of pixel values ​​of cells in which a specific biomarker is expressed and cells in which it is not expressed, a threshold forming a boundary between the two groups can be set. Such a threshold can be calculated using a machine learning algorithm (e.g., K-Means Clustering or Otsu Thresholding).

[0226] In addition, the derived threshold is used as a reference value for each staining intensity level, and in subsequent steps, the staining intensity of the cell can be classified by determining whether the pixel value of the DAB channel falls within that range. For example, if the pixel value falls within the range of 0-50, the staining intensity can be classified as 0, and if it falls within the range of 51-100, it can be classified as 1.

[0227] This process enables the quantification of staining intensity and allows for the stepwise differentiation of the expression levels of estrogen receptors and progesterone receptors. This allows for the differentiation of the staining status of cells within the ROI and provides data necessary for pathological diagnosis. Consequently, the computing device (100) can provide reliable and quantified staining intensity information in subsequent analysis steps by setting a range of staining intensity levels and deriving threshold values ​​based on the pixel value distribution of the DAB channel.

[0228] Additionally, the step of generating analysis information on the state of estrogen receptors and progesterone receptors using a second staining intensity classification sub-model may include a step (S330) of calculating staining intensity based on the average pixel value of the DAB channel in each detected cell region and classifying the calculated staining intensity into one of a plurality of steps.

[0229] Specifically, the computing device can collect pixel value data of the DAB channel for each detected cell region and quantify the staining intensity by calculating the average of all pixel values ​​within the cell region. For example, if the pixel values ​​of the DAB channel containing a specific cell region are 80, 85, and 90, the average pixel value of this cell is calculated as 85. The average pixel value calculated in this way is used as a reference value to indicate the staining intensity of the cell.

[0230] The computing device can map the average pixel value calculated based on the previously derived threshold values ​​of the staining intensity levels (e.g., 0-50, 51-100, 101-150, 151-255) to one of multiple levels (e.g., 0, 1, 2, 3). For instance, if the average pixel value is 85, it falls within the 51-100 range and is therefore classified as staining intensity level 1. Through this process, the staining intensity of each cell is classified into levels and organized into a form that can be interpreted in pathological diagnosis.

[0231] In addition, the size, shape, and uniformity of pixel values ​​of the cell region can also be taken into account when calculating staining intensity. For example, if the cell region is abnormally large or the variability of pixel values ​​is high, additional corrections or filtering may be applied in addition to the average pixel value. This approach prevents distortion that may occur during the process of calculating the staining intensity of the cell region and improves the reliability of the analysis results.

[0232] That is, the computing device (100) can generate clear and consistent staining intensity analysis information at the cell level by quantifying the staining intensity of cells based on the average pixel value of the DAB channel and classifying it into one of a plurality of steps. This process provides important data in pathological diagnosis and can serve as a basis for evaluating the expression status of estrogen receptors and progesterone receptors.

[0233] Additionally, the step of generating analysis information on the state of estrogen receptors and progesterone receptors using a second staining intensity classification sub-model may include the step (S340) of calculating a Proportion Score and an Intensity Score related to staining intensity, and calculating an Allred Score based on the Proportion Score and the Intensity Score.

[0234] Specifically, the computing device (100) can independently calculate a Proportion Score and an Intensity Score, respectively, for a detected cell region. In one embodiment, the Proportion Score may be a value converted into a score from 0 to 5 representing the proportion of cells identified as positive within a specific region of interest (ROI) relative to the total number of cells. For example, if there are 100 total cells within the ROI and 70 of them are identified as positive, the Proportion Score is calculated as 4 based on the proportion of positive cells (70%). The criteria for the Proportion Score may be based on pathological data or a value defined by a pre-trained neural network.

[0235] In addition, in the examples, the Intensity Score may be a value converted to a score from 0 to 3 by evaluating the staining intensity of individual cells. The Intensity Score may be determined based on whether the average DAB channel value of each cell falls within a specific intensity range (e.g., 0-50: 0, 51-100: 1, 101-150: 2, 151-255: 3). For example, a cell with an average DAB channel value of 120 corresponds to the intensity range 101-150, so the Intensity Score may be calculated as 2.

[0236] The computing device (100) can calculate the final Allred Score by summing the calculated Proportion Score and Intensity Score. For example, if the Proportion Score is 4 and the Intensity Score is 2, the Allred Score can be calculated as 4 + 2 = 6. The Allred Score is an indicator that comprehensively represents the degree of ER and PR expression of each cell within the ROI and can be used as a meaningful criterion in breast cancer diagnosis and treatment decision-making.

[0237] The calculation of the Proportion Score and Intensity Score independently provides pathological information regarding cell density and staining intensity, respectively, and the Allred Score integrates these to derive results that allow for the quantitative evaluation of estrogen and progesterone receptor status at the ROI level. The Allred Score serves as the basis for the diagnosis of pathological abnormalities and prognosis assessment, and in particular, supports medical professionals in making reliable treatment decisions. For example,

[0238] Additionally, the step of generating analysis information on the state of estrogen receptors and progesterone receptors using a second staining intensity classification sub-model may include the step (S350) of generating analysis information on the state of estrogen receptors and progesterone receptors at the region of interest unit by weighting and averaging the Allred Score of each positive cell within the region of interest.

[0239] Specifically, the computing device (100) can collect the Allred Score of all cells classified as positive within the region of interest and calculate a weighted average by applying the proportion of the cell within the ROI as a weight to each Allred Score. For example, it can be assumed that there are a total of 100 positive cells within the ROI, and each cell has the following Allred Score. For example, the Allred Score of Cell 1 may be 6, the Allred Score of Cell 2 may be 7, and the Allred Score of Cell 3 may be 8. In this case, the weighted average is calculated based on the proportion of each cell among all cells within the ROI (e.g., 10%, 20%, 70%). Expressed as a formula, the Allred Score per ROI unit can be calculated as follows.

[0240]

[0241] Weight here i is the proportion of cell i within the ROI. For example, in the example above, the ROI Allred Score can be calculated as follows.

[0242] ROI Allred Score=0.1 + 0.2 + 0.7(6 Х 0.1) + (7 Х 0.2) + (8 Х 0.7) = 7.7

[0243] The ROI Allred Score calculated in this manner can be used as a value that quantitatively indicates the ER and PR status of the ROI.

[0244] The Allred Score at the ROI level can function as an indicator that can comprehensively evaluate the status of estrogen and progesterone receptors across the entire ROI, beyond the staining intensity at the single-cell level. Based on the Allred Score, the computing device (100) can quantify the average biomarker expression status of the cell population within the ROI and generate pathological analysis information based on this.

[0245] For example, a high ROI Allred Score indicates that the cells within that ROI are exhibiting a strong positive response, and this information can be used by medical professionals as an important criterion to determine the suitability of a specific anti-hormone therapy in breast cancer treatment. Based on the Allred Score, the computing device can visually represent the comprehensive biomarker expression status at the ROI level or generate quantitative data to evaluate the suitability of specific treatment strategies.

[0246] The analysis information generated in this manner contributes to specifically identifying pathological abnormalities and supports medical professionals in clearly understanding local changes in biomarker expression within pathology images by comparing Allred Score differences between ROIs. Consequently, the analysis information based on Allred Score enhances diagnostic accuracy and can serve as important data for establishing personalized treatment plans for patients.

[0247] This approach, in particular through comparisons between ROIs, can more clearly distinguish pathological abnormalities and be utilized as important data in the clinical decision-making process.

[0248] In other words, the ROI-level Allred Score calculated using the second staining intensity classification submodel functions as an indicator capable of comprehensively evaluating the biomarker expression status of the entire region of interest, going beyond the analysis of staining intensity at the single-cell level. Through this, it quantitatively represents the average estrogen receptor and progesterone receptor status of cell populations within a specific ROI and supports the comprehensive assessment of the pathological characteristics of that region.

[0249] For example, a high ROI Allred Score indicates that cells within the region of interest are exhibiting a strong positive reaction; this can be utilized as important data for assessing the suitability of specific anti-hormone therapies in breast cancer treatment or for monitoring treatment responses. Consequently, the secondary staining intensity classification submodel assists medical professionals in making rapid and accurate diagnostic and treatment decisions by comprehensively evaluating the pathological characteristics of cell populations at the ROI level.

[0250] In various embodiments, the output of the cell counting model may be used as reference information for assigning weights to the analysis results of the first staining intensity classification sub-model and the second staining intensity classification sub-model. In this case, the weights may be calculated based on cell density, cell distribution, and cell type information within the region of interest, and may be characterized by correcting the region-of-interest unit results of the staining intensity classification model.

[0251] In a specific embodiment, the first staining intensity classification sub-model and the second staining intensity classification sub-model utilize the output information of the cell counting model to apply interrelated weights to the staining intensity results of each cell within the ROI, wherein the weights are dynamically adjusted by considering the location and distribution of each cell and the average staining intensity value of the entire ROI, and the pathological abnormality state at the ROI unit is integrally evaluated based on the adjusted results.

[0252] More specifically, the output of the cell counting model includes information on cell density, distribution, and type within the ROI, and can be used to dynamically calculate weights for the staining intensity results derived from the first staining intensity classification sub-model and the second staining intensity classification sub-model. The cell counting model analyzes inter-cell correlations based on the spatial location and density of each cell within the ROI, and these analysis results can be utilized to correct the staining intensity classification results.

[0253] For example, if cell density is abnormally high in a specific ROI, weights can be increased to give high reliability to the staining intensity results for that region. Conversely, if cell distribution is irregular or density is low, weights can be decreased to adjust the reliability of the staining intensity results. In this way, the output of the cell counting model can contribute to improving the accuracy and reliability of ROI-level results.

[0254] Furthermore, the output of the cell counting model can assign spatial weights to the staining intensity classification results based on the positional information of each cell. For example, since cells located in the center of an ROI may play a more significant role in the evaluation of staining intensity, relatively higher weights can be assigned to the staining intensity results of those cells. In this way, the output of the cell counting model helps the staining intensity classification model evaluate the pathological condition at the ROI level more precisely.

[0255] In other words, weight adjustments based on the output information of the cell counting model can integrally correct staining intensity results in ROI-based pathological analysis, contributing to increased reliability and accuracy of the analysis. This approach can more clearly identify various pathological abnormalities and support medical professionals in effectively assessing the pathological characteristics of the ROI.

[0256] According to various embodiments, the method may further include the step of evaluating the reliability of the analysis results by cross-verifying the output information of the cell counting model, the first staining intensity classification sub-model, and the second staining intensity classification sub-model.

[0257] Referring to FIG. 6, the step of evaluating reliability may include: generating an expected staining intensity distribution of the region of interest based on cell density, cell distribution, and cell ratio information within the region of interest calculated through a cell counting model (S410); generating an actual staining intensity distribution of the region of interest unit based on the outputs of a first staining intensity classification sub-model and a second staining intensity classification sub-model (S420); generating a reliability score corresponding to the region of interest by evaluating the degree of agreement between the expected staining intensity distribution and the actual staining intensity distribution (S430); and calculating and providing a reliability score of the region of interest unit (S440).

[0258] To explain in more detail, the computing device (100) can predict the expected staining intensity distribution within the region of interest based on the output information of the cell counting model. Specifically, the computing device can predict the spatial distribution of the expected staining intensity within the region of interest by analyzing information such as cell density, cell distribution, and positive cell ratio within the region of interest calculated from the cell counting model. For example, in a region with high cell density, there is a high probability that inter-cell signal interactions will be activated, which may manifest as strong staining intensity in that region. Conversely, in a region with wide inter-cell spacing and a dispersed pattern, there is a possibility that the staining intensity will be weak or non-uniform. Based on these spatial patterns, the computing device analyzes the region of interest from various angles to reflect the influence of interactions between each cell and neighboring cells on the staining intensity.

[0259] Additionally, the computing device (100) can assign local weights to staining intensity based on cell location and nuclear type information. For example, cells that strongly express specific biomarkers (e.g., HER2, ER, PR) are assigned high weights and are set to have a greater influence on the expected staining intensity. This method quantitatively reflects the differences between cells regarding pathological conditions and contributes to generating a more accurate expected staining intensity distribution.

[0260] Additionally, the computing device (100) can correct the expected distribution of the region of interest currently being analyzed by referring to the staining intensity distribution of pathology images with the same biomarker in past training data. For example, by utilizing a general staining intensity pattern observed in the training data (e.g., a tendency to be strong in the center and weak towards the periphery), the computing device can model the staining intensity pattern of the region of interest currently being analyzed. This contributes to increasing the reliability of the results by identifying early the possibility of abnormally high or low intensity occurring within the ROI.

[0261] In this way, the computing device (100) can perform precise predictions regarding spatial distribution and intensity based on various output information of the cell counting model. Thus, the predicted staining intensity distribution is generated as data that can reflect minute cellular differences within the pathology image and can be used as important basic data for comprehensively evaluating pathological abnormal conditions according to staining intensity.

[0262] In an embodiment, the computing device (100) can generate an actual staining intensity distribution within a region of interest based on the outputs of a first staining intensity classification sub-model and a second staining intensity classification sub-model. For example, after calculating the staining intensity results for each cell within the ROI in each sub-model, the results can be integrated to calculate an overall staining intensity distribution at the ROI level. This actual distribution reflects the status of specific biomarkers (e.g., HER2, ER, PR) and can quantitatively represent the staining intensity of each cell and ROI.

[0263] In addition, in an embodiment, the computing device (100) can generate a reliability score by evaluating the degree of agreement between the expected dyeing intensity distribution and the actual dyeing intensity distribution. Specifically, the correlation between the two distributions can be evaluated using a measure that quantifies the difference between the expected distribution and the actual distribution. For example, the computing device can quantify the degree of agreement by utilizing metrics such as the correlation coefficient between the two distributions, the mean squared error (MSE), or cross-entropy. The higher the degree of agreement, the higher the reliability of the analysis result is evaluated, and conversely, if the degree of agreement is low, it may indicate that further review of the analysis result may be necessary.

[0264] Additionally, the computing device (100) can finally calculate and provide a reliability score for each region of interest to the user. In this process, the reliability score is used as data to comprehensively evaluate the pathological condition of each ROI. For example, the computing device can include the reliability score in an analysis report so that medical staff can refer to it when interpreting the analysis results of a specific ROI, and can indicate ROIs with low reliability so that additional verification procedures can be performed.

[0265] The present invention can improve the accuracy and reliability of pathological analysis results by cross-validating the output information of a cell counting model and a staining intensity classification sub-model, and by evaluating the reliability of the analysis results based on the degree of agreement between the predicted distribution and the actual distribution. This reliability evaluation can support medical professionals in placing greater trust in the analysis results and making diagnostic and treatment decisions based on them.

[0266] According to various embodiments of the present invention, a computing device (100) is characterized by a configuration that integrates a microservices architecture and an event-driven architecture to dramatically enhance scalability, flexibility, security, and stability in pathology image analysis tasks. This configuration overcomes the limitations of scalability and maintenance of existing monolithic structures and can improve the accuracy and reliability of pathology image analysis.

[0267] Specifically, the computing device (100) can implement core functions related to pathology image analysis, such as cell detection, staining intensity analysis, data preprocessing, and postprocessing, by separating them into independent microservices. For example, the cell detection service performs the task of extracting the location and shape of cells from an image, and the staining intensity analysis service can evaluate the expression intensity of biomarkers and classify them into stages. These individual services are designed and deployed independently, and maintenance and expansion can be easily performed without interrupting the entire system, even when applying new algorithms or updating specific services. This allows for breaking away from the high coupling of existing integrated systems, minimizing dependencies between individual services, and enabling the flexible configuration of the analysis environment.

[0268] In particular, the microservices architecture in this invention can provide dynamic scalability in response to changes in the amount of data input. For example, if the amount of pathology image data input increases rapidly, throughput can be expanded by automatically deploying additional containers for the data preprocessing service. This structure effectively manages the load of each service and enables stable response even when processing large-scale pathology image data. This allows neural network training, analysis, and prediction tasks to be executed independently, offering the advantage of maintaining high processing performance even in large-scale data environments.

[0269] Additionally, the computing device (100) can manage communication between the image processing pipeline and each service through an event-driven architecture using asynchronous event-based processing. For example, when pathology image preprocessing is completed, the corresponding event is automatically generated and transmitted to the staining intensity analysis service, and the analysis task can be performed asynchronously. This prevents bottlenecks that occur in synchronous processing methods and can improve the processing speed and efficiency of the task through parallel processing between services. In particular, since parallel execution of analysis tasks is possible even when processing large-scale data, it can support rapid diagnosis and treatment decisions in the medical field.

[0270] In addition, the computing device (100) utilizes containerization technology to virtualize each service into an independent Docker container, and by applying an automatic restart option to the container, it can quickly recover even in the event of an unexpected service interruption. For example, even if an error occurs in a specific service during pathology image analysis, the corresponding service container is automatically restarted so that the operation can continue without interruption. This strengthens the stability of the entire system and minimizes service restoration time, thereby enabling medical staff to obtain analysis results reliably.

[0271] In addition, the computing device (100) of the present invention can securely protect sensitive pathology image data through TLS (Transport Layer Security)-based encrypted communication and a user authentication and authorization management system. For example, even when remotely viewing pathology image analysis results, all communication is encrypted to effectively block the risk of data leakage and tampering. This security feature has the advantage of providing a safe medical data analysis environment by maintaining the confidentiality and integrity of medical data.

[0272] That is, the computing device (100) of the present invention improves the scalability, stability, and security of pathology image analysis by integrating the adoption of a microservices architecture and an event-driven architecture, and by combining containerization and security technologies, and can overcome problems that were difficult to solve in existing analysis systems. Through this configuration, medical professionals can make accurate diagnostic and treatment decisions based on highly reliable analysis results and maximize the efficiency of pathological data processing.

[0273]

[0274] In one embodiment, a user interface screen provided during the pathology image analysis process related to the present invention may be provided.

[0275] In one embodiment, a user interface (UI) screen visually representing a single ROI analysis result according to one embodiment of the present invention may be provided. The pathology image is displayed with the ROI designated while magnified to a high magnification, and the boundary of the ROI may be highlighted in red. Individual cells located within the ROI may be visualized as positive (red dots) and negative (blue dots). The area information of the ROI (17941.26 μm²) is provided at the center of the ROI boundary, and this can be used as the unit of analysis for the corresponding area.

[0276] The "Assessment" panel on the screen can display a summary of the analysis results. For example, within ROI 1, 124 positive cells and 36 negative cells are counted, and the positivity rate can be calculated as 78%. This information is useful for medical professionals to quickly grasp the pathological status at the ROI level and evaluate cell density or the level of positive reactions. Such a visual representation clearly indicates the condition of each cell, and summarizing data at the ROI level can improve the readability of the analysis results.

[0277] In one embodiment, a user interface (UI) that integrates a slide map and a single ROI analysis may be provided. A high-magnification image of a specific ROI is displayed in the center of the screen, and may include the boundaries of the ROI and analysis data. For example, ROI 1 may be indicated by a red boundary line to clearly show that it is the area of ​​focus for analysis. For example, positive cells (124) and negative cells (36) within the ROI may be visualized as red and blue dots, respectively, and the positive rate may be shown as 78%.

[0278] The "Navigator" panel on the screen can provide a minimap that displays the entire slide in a scaled-down form. For example, the location of ROI 1 is highlighted on the minimap, and the user can use the panel to quickly navigate to a specific area of ​​interest across the entire slide. This feature can facilitate the process of exploring and zooming in on specific analysis areas in large-scale pathology images.

[0279] In one embodiment, information regarding changes in the zoom ratio of the navigator panel may be displayed along with multiple ROI analysis results. For example, the "Navigator" panel on the screen is provided in the form of a minimap that reduces the entire slide, and in the panel, the user can adjust the ratio of the slide image from 1x to 40x.

[0280] When set to a low magnification (e.g., 5x), the structural context of the entire slide can be grasped at a glance, and the location and distribution of multiple ROIs can be checked overall. In this state, it is suitable for confirming the relative location and size of regions of interest within the overall pathology image rather than details at the individual cell level. For example, one can visually determine how close or far apart ROI 1 and ROI 2 are from each other within the slide, as well as the proportion of the slide occupied by the ROIs.

[0281] On the other hand, setting the navigator ratio to 40x allows for high-magnification of a specific ROI, enabling detailed analysis at the cellular level. At this ratio, individual cells within the ROI are clearly marked as positive (red dots) or negative (blue dots), allowing for a detailed evaluation of cell distribution, arrangement, and density. For example, it is possible to determine whether cells within ROI 1 are arranged in a specific pattern or if staining intensity is concentrated in a particular area.

[0282] This navigator's scale adjustment provides various levels of analysis between the entire slide and a specific ROI, enabling users to simultaneously view the overall context and detailed information of the pathology image. For example, at a 5x scale, the macroscopic context can be viewed, while at a 40x scale, microscopic details can be viewed, allowing for a multifaceted analysis of the pathological condition.

[0283] In one embodiment, a user interface (UI) screen may be provided that displays the results of a comprehensive analysis of the pathological status of multiple ROIs. For example, a pathological image may be displayed in the center of the screen, and ROI 1, ROI 2, ROI 3, and ROI 4 may be visually highlighted by being separated by boundary lines of different colors. The cells inside each ROI are indicated as positive (red dot) and negative (blue dot), allowing for an intuitive check of the cell distribution and status within the ROI.

[0284] For example, the "Assessment" panel on the screen can provide a summary of the analysis results by group. For instance, the positive rates for ROI 1 and ROI 2 in Group 1 may be displayed as 78% and 50%, respectively, while ROI 3 and ROI 4 in Group 2 may be displayed as being awaiting analysis. Additionally, area information for each ROI (ROI 1: 17941.26 μm², ROI 2: 10150.17 μm², etc.) may be provided, offering additional data for pathological analysis.

[0285] In one embodiment, by enabling comparison between multiple ROIs and integrated evaluation of the entire ROI, it is possible to support medical personnel in analyzing pathological abnormalities more reliably and efficiently. The user interface can provide a function that allows for a quick overview of the pathological condition across the entire slide, as well as the cellular condition in a specific region of interest.

[0286]

[0287] FIG. 7 is a diagram illustrating the AI-based cell classification and detection process of a pathology image according to one embodiment.

[0288] In one embodiment, the AI-based cell classification and detection process of a pathology image may be performed by a cell nucleus segmentation model and a model that determines the staining intensity of a cell membrane staining image. For example, the model that determines the staining intensity of a cell membrane staining image may be a model that classifies the staining intensity of a cell membrane staining image (e.g., HER2) into 0 / 1+ / 2+ / 3+ at the patch level, but is not limited thereto.

[0289] In one embodiment, the computing device (100) can obtain information about an area of ​​interest (ROI) to be analyzed within an image. Information about the area of ​​interest to be analyzed can be obtained based on user input, and the types of user input may include, but are not limited to, free drawing or setting an area of ​​a pre-set shape.

[0290] In various embodiments, the computing device (100) may automatically set an area to be analyzed, or may perform an analysis on the entire image without setting a specific area.

[0291] The computing device (100) can detect one or more cells from an image of the area to be analyzed. For example, the computing device (100) can generate outline polygon coordinates of the cell nucleus, but is not limited thereto.

[0292] The computing device (100) can determine whether a cell nucleus is negative or positive, or determine the staining intensity, through analysis of each cell nucleus. Based on the determination result, the computing device (100) can assist the user in reading by displaying the number and ratio of negative / positive cells on a user interface, or by displaying the number of cells by staining intensity or the weighted average value of the staining intensity of all cells.

[0293] In various embodiments, the computing device (100) may generate and display a final reading result value based on the reading result, and the specific method thereof is not limited.

[0294]

[0295] FIG. 8 is a diagram illustrating the process of cell classification and detection of an AI-based pathology image according to another embodiment.

[0296] In one embodiment, the AI-based cell classification and detection process of a pathology image can be performed by a single model. The single model can generate one or more maps among a Binary Nuclei Map, Horizontal and Vertical Distance Maps, and a Nuclei Type Map.

[0297] For example, the model can be designed with a backbone based on Large Convolution Kernels instead of a Vision Transformer-based structure, and can adopt a structure that significantly reduces the number of parameters and computational load by applying Structural Reparameterization techniques during the inference phase to replace large convolution kernels with multiple small convolution kernels, but is not limited to this.

[0298] In one embodiment, a single model may be trained using training data containing pathology images and labeling information therefor. For example, the training data may include pathology images (e.g., IHC-stained RGB patch images) as input data.

[0299] In addition, the training data may include a Binary Nuclei Map, Horizontal and Vertical Distance Maps, and a Nuclei Type Map as output data.

[0300] For example, the binary nucleus map may be a binary image where the nucleus portion is 1 and the background portion is 0. Additionally, the horizontal and vertical distance map is a gradient map representing the horizontal and vertical boundaries of the nucleus and may have values ​​between -1 and 1. The horizontal and vertical distance map can be used to separate cell regions during the post-processing step. Furthermore, the nucleus type map may be an image containing type information regarding whether the pixels (nuclei) in the corresponding region are tumor or non-tumor.

[0301] In one embodiment, the training data may include a membrane stain completeness map as output data. The membrane stain completeness map may be an image having a value between 0 and 1 representing the stain completeness of each cell.

[0302] A single model can be trained using the aforementioned training data, and the trained single model can receive a pathology image as input and output at least one of a binary nuclear map, a horizontal and vertical distance map, a nuclear type map, and a cell membrane staining integrity map, or output information associated therewith.

[0303] In one embodiment, the computing device (100) can detect one or more cells from an image of an area to be analyzed by post-processing the output of a single model. For example, the computing device (100) can generate outline polygon coordinates of a cell nucleus, but is not limited thereto.

[0304] In one embodiment, the computing device (100) can determine whether each cell is a tumor cell (tumor cell / non-tumor cell) using a nuclear type map. Through this, the computing device (100) can automatically select only tumor cells as analysis targets and perform analysis without separate user input, and even if the user inputs a specific region (ROI), it can select only tumor cells within that region and perform analysis.

[0305] In one embodiment, the computing device (100) can determine the staining level of each cell using a cell membrane staining integrity map. For example, the computing device (100) can determine the staining integrity of the cell membrane based on the output of a model. For example, the staining integrity can be output as a value from 0 to 1, and can be output as 0 if there is no staining, and as 1 if it is stained without defects.

[0306] In various embodiments, the computing device (100) may determine the staining intensity of a cell nucleus or cell membrane based on the output of the model. The staining intensity may be determined as a real value from 0 to 1, but is not limited thereto. In one embodiment, the computing device (100) may calculate the staining intensity by utilizing a rule-based image processing algorithm in the step of post-processing the output of the model, but is not limited thereto. Specifically, the computing device (100) may obtain information about the location and region of each cell (e.g., cell outline information) from the output of the model and determine the staining intensity of each cell using the RGB pixel values ​​included in the corresponding region.

[0307] In various embodiments, the computing device (100) can analyze cells at the individual cell level rather than at the patch level to generate and provide information necessary for reading, such as staining intensity.

[0308] In one embodiment, a report including analysis results may be automatically generated and provided to the user. The report may be provided through a dashboard of the user interface or generated as a separate file and provided in a downloadable format, but is not limited thereto.

[0309] In various embodiments, the computing device (100) may set the display method of the cell differently depending on the magnification of the screen. For example, at a magnification of less than 40, only the center coordinates of the cell nucleus may be displayed as points, and at a magnification of 40 or more, the outline of the cell nucleus may be displayed, but is not limited thereto. In addition, when the magnification is below a preset standard, the number of center coordinate points displayed may be reduced so that the zoom in / out operation is performed quickly.

[0310]

[0311] The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0312] The components of the present invention may be implemented as a program (or application) and stored on a medium to be executed in combination with a computer, which is hardware. The components of the present invention may be implemented as software programming or software elements, and similarly, embodiments may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, etc., including various algorithms implemented as combinations of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors.

[0313] Those skilled in the art will understand that the various exemplary logic blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented by electronic hardware, various forms of programs or design code (referred to herein as “software”), or a combination of all such. To clearly illustrate this interoperability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in relation to their functions. Whether such functions are implemented as hardware or software depends on the design constraints imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementation decisions should not be interpreted as being outside the scope of the invention.

[0314] The various embodiments presented herein may be implemented as methods, devices, or articles of manufacture using standard programming and / or engineering techniques. The term “article of manufacture” includes a computer program, carrier, or medium accessible from any computer-readable device. For example, computer-readable media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical discs (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Additionally, the various storage media presented herein include one or more devices and / or other machine-readable media for storing information. The term “machine-readable media” includes, but is not limited to, wireless channels and various other media capable of storing, holding, and / or transmitting command(s) and / or data.

[0315] It should be understood that the specific order or hierarchy of steps in the presented processes is merely an example of exemplary approaches. It should be understood that, based on design priorities, the specific order or hierarchy of steps in the processes may be rearranged within the scope of the invention. The appended method claims provide various step elements in a sample order, but do not imply limitation to the specific order or hierarchy presented.

[0316] The description of the presented embodiments is provided so that any person skilled in the art may use or practice the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present invention. Thus, the present invention is not limited to the embodiments presented herein, but should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

Claims

1. A method performed on one or more processors of a computing device, Step of acquiring pathological images; A step of performing preprocessing on the above pathology image; and A step of generating analysis information corresponding to the pathology image using a pathology image analysis model; Includes, The step of generating the above analysis information is, A step of detecting one or more cells from the above pathology image; A step of determining whether the above cell is a tumor cell; and A step of determining the staining level of the above cell; comprising, AI-based cell classification and detection method for pathology images.

2. In Paragraph 1, The step of determining the dyeing level above is, A step of determining the integrity of the cell membrane staining of the above cell; comprising AI-based cell classification and detection method for pathology images.

3. In Paragraph 1, The step of determining the dyeing level above is, A step of determining the staining intensity of the above cell; comprising, AI-based cell classification and detection method for pathology images.

4. In Paragraph 1, The step of generating the above analysis information is, A step comprising generating one or more maps among a Binary Nuclei Map, Horizontal and Vertical Distance Maps, and Nuclei Type Map using the above pathology image analysis model; AI-based cell classification and detection method for pathology images.

5. In Paragraph 4, The step of generating the above analysis information is, A step of determining whether the cell is a tumor cell using the above nuclear type map; comprising AI-based cell classification and detection method for pathology images.

6. In Paragraph 4, The step of generating the above analysis information is, The method further comprises the step of generating a Membrane Stain Completeness Map using the above pathology image analysis model. AI-based cell classification and detection method for pathology images.

7. In Paragraph 6, The step of generating the above analysis information is, The step of determining the cell membrane staining integrity of the cell using the cell membrane staining integrity map; further comprising AI-based cell classification and detection method for pathology images.

8. Memory for storing one or more instructions; and It includes a processor that executes one or more instructions stored in the memory, The above processor executes the above one or more instructions, A device that performs the method of paragraph 1.

9. A computer program stored on a computer-readable recording medium that is combined with a computer, which is hardware, to perform the method of claim 1.

Images