Method for generating representative lesion image of biological tissue image, and computing system for performing same

Abstract

The present invention provides a method comprising the steps of: acquiring a biological tissue image and a pixel-level diagnosis result of the biological tissue image, wherein a primary lesion pattern and a secondary lesion pattern of the biological tissue image is acquired; searching for a first partial region that includes a large part of the primary lesion pattern in the pixel-level diagnosis result corresponding to a corresponding region from among partial regions of the biological tissue image, and generating the corrected pixel-level diagnosis result; and searching for a second partial region that includes a large part of the secondary lesion pattern in the corrected pixel-level diagnosis result corresponding to the corresponding region from among the partial regions of the biological tissue image, and outputting a representative lesion image of the biological tissue image.

Description

Method for generating representative lesion images of biological tissue images and computing system for performing the same

[0001] The present invention relates to a method for generating a representative lesion image representing a biological tissue image among partial images of the biological tissue image, and a computing system for performing the same.

[0002]

[0003] In medicine, pathology, which involves interpreting actual specimens, is crucial for accurately diagnosing and predicting the prognosis of serious diseases. Conventional pathology diagnostic methods involve visual observation and interpretation of diagnostic pathology slides produced from specimens through an optical microscope by a pathologist. The dawn of digital pathology began with the use of a computer-connected microscope camera to convert pathology slides into digital images, which are then viewed and interpreted on a monitor. Recently, the advent of digital slide scanners has led to the widespread adoption of this method, which converts entire pathology slides into a single digital image, which is then viewed and interpreted on a computer monitor. Digital pathology-based pathology, which interprets scanned images on a monitor, is replacing the traditional method of visually examining specimen slides through an optical microscope.

[0004] Meanwhile, to diagnose serious diseases such as cancer, part or all of the tissue is collected, an image of the biological tissue is created, and histopathological examination is performed to determine the severity of the disease. There is an agreed-upon system for reporting the histological severity for each disease, and these systems grade and report the severity based on the morphological characteristics of the tissue. For example, in the case of prostate cancer, there is a grading system called the Gleason grading system, which classifies the tissue pattern into grades from 1 to 5 based on the morphological characteristics of the gland tissue, and grades the overall severity based on the proportion of each grade area in the entire tissue. In addition, breast cancer can be graded based on the histological grade.

[0005] Meanwhile, histopathological examination of tissue images can also be used to histologically classify lesions. For example, it can be used to distinguish between invasive or intraepithelial breast lesions, or between invasive and precancerous lung lesions (atypical adenomatous (glandular) hyperplasia).

[0006] When explaining these diagnostic results to patients or sharing them with other pathologists, annotated biopsy images with severity grades, such as Gleason patterns, can be utilized. For example, inserting biopsy images annotated with severity grades into reports to patients or specialists is a common example.

[0007] However, the whole slide image generated by the slide scanner needs to be scanned at a magnification of 200x or more for an accurate diagnosis, and the number of pixels in the image generated in this way reaches 2.4 billion when scanning a 2cm x 3cm area at 200x, and 9.6 billion when scanning at 400x. Therefore, it is difficult to use the biological tissue image as is due to such a large size, and a technology is needed to accurately extract a certain portion of the lesion area that can represent the biological tissue image among the biological tissue image.

[0008]

[0009] The technical problem to be achieved by the present invention is to provide a method for extracting a lesion area representing a biological tissue image from a biological tissue image and a computing system for performing the method.

[0010]

[0011] According to one aspect of the present invention, there is provided a computing system, comprising: a step of obtaining a biological tissue image and a pixel-wise diagnosis result of the biological tissue image, wherein the pixel-wise diagnosis result of the biological tissue image includes a lesion pattern for each pixel constituting the biological tissue image, and the lesion pattern means a grade according to a predetermined severity grading system or a histological classification of a predetermined lesion; a step of obtaining a primary lesion pattern and a secondary lesion pattern of the biological tissue image by the computing system; a step of searching, by the computing system, a first partial region among partial regions of the biological tissue image, in which the primary lesion pattern is most frequently included in a pixel-wise diagnosis result corresponding to the corresponding region; wherein the partial region of the biological tissue image is a part of the biological tissue image and has a predefined shape and area; a step of generating a modified pixel-wise diagnosis result by the computing system by removing the pixel-wise diagnosis result corresponding to the first partial region from the pixel-wise diagnosis result of the biological tissue image; A method is provided, comprising: a step in which the computing system searches for a second partial region among the partial regions of the biological tissue image, in which the second lesion pattern is most included in the modified pixel-unit diagnosis result corresponding to the corresponding region; and a step in which the computing system outputs the first partial region of the biological tissue image and the second partial region of the biological tissue image as representative lesion images of the biological tissue image.

[0012] In one embodiment, the step of searching the first partial region may include a step of searching the first partial region in a sliding window manner in which the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image, and the step of searching the second partial region may include a step of searching the second partial region in a sliding window manner in which the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image.

[0013] In one embodiment, the step of outputting the first partial region of the biological tissue image and the second partial region of the biological tissue image as a representative image of a lesion of the biological tissue image may include the step of overlapping and outputting a pixel-wise diagnosis result corresponding to the first partial region of the biological tissue image; and the step of overlapping and outputting a pixel-wise diagnosis result corresponding to the second partial region of the biological tissue image.

[0014] In one embodiment, the lesion pattern may be any one of a Gleason pattern according to the Gleason grading system for prostate cancer, a grade according to the histologic grading system for breast cancer, a classification of invasive carcinoma lesions and intraepithelial carcinoma lesions of the breast, and a classification of invasive carcinoma lesions and precancerous lesions of the lung.

[0015] According to another aspect of the present invention, a computer program is provided, which is installed in a data processing device and recorded on a recording medium for performing the above-described method.

[0016] According to another aspect of the present invention, a computer-readable recording medium having recorded thereon a computer program for performing the above-described method is provided.

[0017] According to another aspect of the present invention, a computing system is provided, comprising: a processor; and a memory storing a computer program, wherein the computer program, when executed by the processor, causes the computing system to perform the above-described method.

[0018] According to another aspect of the present invention, there is provided a first acquisition module for acquiring a biological tissue image and a pixel-wise diagnosis result of the biological tissue image, wherein the pixel-wise diagnosis result of the biological tissue image includes a lesion pattern for each pixel constituting the biological tissue image, and the lesion pattern means a grade according to a predetermined severity grading system or a histological classification of a predetermined lesion; a second acquisition module for acquiring a primary lesion pattern and a secondary lesion pattern of the biological tissue image; a first search module for searching a first partial region among partial regions of the biological tissue image, in which the first lesion pattern is most included in the pixel-wise diagnosis result corresponding to the corresponding region, wherein the partial region of the biological tissue image is a part of the biological tissue image and has a predefined shape and area; a removal module for generating a modified pixel-wise diagnosis result by removing the pixel-wise diagnosis result corresponding to the first partial region from the pixel-wise diagnosis result of the biological tissue image; A computing system is provided, including a second search module that searches for a second partial region among the partial regions of the biological tissue image, in which the second lesion pattern is most included in the modified pixel unit diagnosis result corresponding to the corresponding region; and an output module that outputs the first partial region of the biological tissue image and the second partial region of the biological tissue image as representative lesion images of the biological tissue image.

[0019] In one embodiment, the first search module may search the first partial region in a sliding window manner in which the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image, and the second search module may search the second partial region in a sliding window manner in which the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image.

[0020] In one embodiment, the output module may output a pixel-by-pixel diagnostic result corresponding to the first partial region by overlapping the first partial region of the biological tissue image, and may output a pixel-by-pixel diagnostic result corresponding to the second partial region by overlapping the second partial region of the biological tissue image.

[0021]

[0022] According to the technical idea of ​​the present invention, a method for extracting a lesion region representing a biological tissue image from a biological tissue image and a computing system for performing the same can be provided. In particular, a method for detecting a portion of a biological tissue image in which a specific grade in a predetermined grading system for grading the severity of a disease such as cancer is most clearly expressed, or a portion of a lesion in which a histological classification is most clearly expressed, and a computing system for performing the same can be provided.

[0023]

[0024] In order to more fully understand the drawings cited in the detailed description of the present invention, a brief description of each drawing is provided.

[0025] FIG. 1 is a drawing for explaining a schematic system configuration for implementing a method for generating a representative lesion image of a biological tissue image according to an embodiment of the present invention.

[0026] FIG. 2 is a flowchart illustrating a method for generating a representative lesion image according to one embodiment of the present invention.

[0027] Figure 3 is a drawing illustrating an example of a biological tissue image and a partial region of the biological tissue image.

[0028] Figure 4 is a diagram illustrating an initial part of the process of searching the first sub-area using a sliding window method.

[0029] Figure 5(a) illustrates an example of a first partial region searched by the computing system.

[0030] Figure 5(b) shows an example of the result of removing the pixel unit diagnosis result corresponding to the first partial area.

[0031] Figure 5(c) illustrates an example of a second partial region searched by the computing system.

[0032] Fig. 6a illustrates an example of a biological tissue image in which pixel-wise diagnostic results overlap, and Fig. 6b illustrates an enlarged view of the result in which pixel-wise diagnostic results overlap the first partial image searched in 6a.

[0033] FIG. 7 is a block diagram illustrating a logical configuration of a computing system according to one embodiment of the present invention.

[0034] FIG. 7 is a block diagram illustrating the physical configuration of a computing system according to one embodiment of the present invention.

[0035]

[0036] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. In describing the present invention, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the gist of the present invention.

[0037] Terms such as "first," "second," etc. may be used to describe various components, but these components should not be limited by these terms. Terms such as "first," "second," etc. do not denote a particular order and are used solely to distinguish one component from another.

[0038] The terminology used in this application is solely for the purpose of describing specific embodiments and is not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0039] In this specification, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0040] Additionally, in the present specification, when a component "transmits" data to another component, it means that the component may transmit the data directly to the other component, or may transmit the data to the other component via at least one other component. Conversely, when a component "directly transmits" data to another component, it means that the data is transmitted from the component to the other component without going through the other component.

[0041] Hereinafter, the present invention will be described in detail, focusing on embodiments thereof, with reference to the attached drawings. The same reference numerals in each drawing represent the same components.

[0042] FIG. 1 is a drawing for explaining a schematic system configuration for implementing a method for generating a representative lesion image of a biological tissue image (hereinafter referred to as “representative lesion image generation method”) according to an embodiment of the present invention.

[0043] Referring to FIG. 1, a method for generating a representative lesion image according to the technical concept of the present invention can be performed by a computing system (100). The computing system (100) can select a lesion area representing a corresponding slide image from a biological tissue image and generate a representative lesion image of the biological tissue image.

[0044] A biological tissue image may be an image generated from a slide of a pathological specimen tissue stained using a predetermined staining method. The pathological specimen tissue may be a biopsy taken from various organs of the human body or a surgically excised biological tissue. In one embodiment, the pathological specimen tissue may be a biological tissue collected through a core needle biopsy. Meanwhile, the biological tissue image may be a digital slide image of a stained pathological specimen or a portion of a digital slide image. The pathological specimen slide may be a portion of a sliced ​​pathological specimen. The digital slide image of a pathological specimen may be generated by slicing a pathological specimen to produce a glass slide, staining the glass slide with a predetermined staining agent, and digitizing the glass slide.

[0045] In one embodiment, a computing system (100) may be installed on a predetermined server (10) to implement the technical idea of ​​the present invention. The server (10) refers to a data processing device having a computing capability for implementing the technical idea of ​​the present invention. Generally, a person skilled in the art will be able to easily infer that not only a data processing device that can be accessed by a client through a network, but also any device capable of performing a specific service, such as a personal computer or a mobile terminal, can be defined as a computing device.

[0046] According to one embodiment, the computing system (100) may need to directly determine the pixel-wise diagnosis result of a biological tissue image (e.g., Gleason pattern for each pixel) or the severity grade of a biological tissue image (e.g., Gleason score), and in this case, the computing system (100) may use a pre-trained deep learning model (200-1, 200-2). The deep learning model (200-1, 200-2) may be an artificial neural network that receives a biological tissue image or a patch obtained by dividing a biological tissue image into a certain size as input, and outputs a pixel-wise diagnosis result of the input image or a result value for determining the severity grade of the input image itself (e.g., the probability corresponding to each Gleason pattern, the probability corresponding to each severity grade, etc.).

[0047] The above deep learning model (200-1, 200-2) is a neural network artificially constructed based on the operating principles of human neurons, includes a multilayer perceptron model, and may refer to a set of information expressing a series of design specifications defining an artificial neural network. The above deep learning model (200-1, 200-2) may include, but is not limited to, a convolutional neural network (CNN) widely known in the fields of artificial intelligence, machine learning, or deep learning.

[0048] The server (10) including the computing system (100) may be a system that stores or applies the results output by the computing system (100) (i.e., the generated representative lesion image). For example, the server (10) may be a system for generating a digital document including a representative lesion image of a biological tissue image generated by the computing system (100), or a diagnostic assistance system for providing information necessary for a doctor to diagnose a disease, but is not limited thereto. The deep learning model (200-1, 200-2) and the computing system (100) may be provided in one physical device, i.e., the server (10), but may be provided in different physical devices depending on the embodiment, and various modifications may be possible as needed, as an average expert in the technical field of the present invention will be able to easily infer.

[0049] In one embodiment, the server (10) and / or the computing system (100) may communicate with a user terminal (20). For example, the user terminal (20) may transmit a biological tissue image to the computing system (100) and may also receive a result generated from the computing system (100).

[0050] Meanwhile, in FIG. 1, an example is shown in which the computing system (100) is implemented in the form of a subsystem installed in a server (10), but it is obvious that, depending on the embodiment, the computing system (100) may be implemented as an independent single system. In this case, the computing system (100) may be a computing system that is a data processing device having a computing capability for implementing the technical idea of ​​the present invention, and may generally include not only a server that is a data processing device that can be accessed by a client through a network, but also a computing device such as a personal computer or a mobile terminal.

[0051] FIG. 2 is a flowchart illustrating a method for generating a representative lesion image according to one embodiment of the present invention.

[0052] The computing system (100) can acquire a biological tissue image (S110). The biological tissue image may be a full slide image or a portion of a full slide image created by digitally scanning a slide of biological tissue collected through a biopsy or surgery. In one embodiment, the computing system (100) may receive a biological tissue image from the user terminal (20), or acquire a biological tissue image from a database or storage device provided by the server (10) or the computing system (100).

[0053] Meanwhile, the computing system (100) can obtain a pixel-by-pixel diagnosis result of the biological tissue image (S120). At this time, the pixel-by-pixel diagnosis result of the biological tissue image can include a lesion pattern for each pixel constituting the biological tissue image.

[0054] In this specification, a lesion pattern may mean a grade according to a severity grading system of a given disease, such as cancer, or may mean a histological classification of a given lesion.

[0055] For example, the lesion pattern may be a Gleason pattern according to the Gleason grading system for prostate cancer. The Gleason pattern is a representative grading system indicating the severity of prostate cancer tissue, and is classified into a benign or a pattern of 1 to 5 depending on the histological differentiation. A classification of 1 indicates differentiation close to normal and low malignancy, whereas a score of 5 indicates cancer with the most severe malignancy. The Gleason pattern used in the present invention can be divided into a severity grade of 1 to 5 indicating prostate cancer and a benign grade for each pixel, but in some cases, it can be divided into a grade of 3 to 5 and a benign grade.

[0056] Another example of a lesion pattern is the tumor grade according to the histologic grading system for breast cancer. In addition to grades indicating severity, such as the Gleason pattern or histologic grade of breast cancer, as mentioned above, a lesion pattern can also refer to a histologic classification of a given lesion, such as the classification of invasive versus non-invasive lesions in the breast, or invasive versus precancerous lesions (atypical adenomatous hyperplasia) in the lung.

[0057] The pixel-wise diagnosis result of the biological tissue image may be in the form of a two-dimensional array of the same size as the biological tissue image. For example, if the biological tissue image has a size of W pixels × H pixels, the pixel-wise diagnosis result of the biological tissue image may be a two-dimensional array of W × H, and each element of the array may be assigned a pixel-wise diagnosis result of the pixel corresponding to that element.

[0058] For example, if the lesion pattern is a Gleason pattern, each element of the array may be assigned a value indicating the corresponding Gleason pattern if the pixel corresponding to the element corresponds to Gleason pattern 1 to 5, and a value indicating positivity (e.g., 0) may be assigned otherwise.

[0059] In an embodiment where the lesion pattern is a histological classification of a breast lesion, i.e., a classification of an invasive lesion and an intraepithelial carcinoma lesion, each element of the array may be assigned a value corresponding to the classification (e.g., 1) if the pixel corresponding to the element is classified as an invasive lesion, a value corresponding to the classification (e.g., 2) if the pixel is classified as an intraepithelial carcinoma, and a value indicating positivity (e.g., 0) otherwise.

[0060] However, depending on the embodiment, each element of the array may be implemented such that if the pixel corresponding to the element does not belong to the tissue region, a value indicating that it is not a tissue region is assigned, and if it belongs to the tissue region, a pixel-by-pixel diagnostic result of the pixel corresponding to the element is assigned. For example, each element of the array may be assigned -1 if the pixel corresponding to the element belongs to the non-tissue region, 0 if it is benign, and otherwise a value corresponding to the lesion pattern.

[0061] For convenience of explanation, below, an embodiment is described in which the same positive value (e.g., 0) is assigned to both cases in which a pixel belongs to a non-organized area and cases in which it is positive.

[0062] In one embodiment, the pixel-by-pixel diagnostic results of the biological tissue image may be input into the computing system together with the biological tissue image as a result of manual annotation by a pathologist.

[0063] According to an embodiment, the computing system (100) may analyze the biological tissue image to directly obtain a pixel-wise diagnosis result of the biological tissue image. In this case, the computing system (100) may obtain a pixel-wise diagnosis result of the biological tissue image using a pre-trained deep learning model (200-1). More specifically, the computing system (100) may input each of the biological tissue images or patches divided into a grid shape into an input layer of the deep learning model (200-1) and determine a lesion pattern for each pixel of the biological tissue image based on an output value output from an output layer of the deep learning model (200-1). In this case, the deep learning model (200-1) may be an artificial neural network that has been trained in advance to determine a pixel-wise lesion pattern of an input image, and may be an artificial neural network that has been trained in advance to receive an image through an input layer and output a probability that each pixel included in the input image corresponds to each lesion pattern.

[0064] The computing system (100) can acquire the primary lesion pattern and the secondary lesion pattern of the biological tissue image (S130). The primary lesion pattern and the secondary lesion pattern of the biological tissue image are lesion patterns of the same type as the lesion pattern for each pixel included in the pixel-by-pixel diagnosis result of the biological tissue image.

[0065] For example, if the pixel-by-pixel diagnosis result of the biological tissue image is composed of a Gleason pattern for each pixel, the computing system (100) can obtain a Gleason score of the biological tissue image, and the Gleason score is composed of a primary Gleason pattern and a secondary Gleason pattern. The primary Gleason pattern and the secondary Gleason pattern may be classified into grades 1 to 5 and benign, or grades 3 to 5 and benign, depending on the embodiment. A Gleason score in which the primary Gleason pattern is a and the secondary Gleason pattern is b is typically expressed as 'a+b'. For example, if the Gleason score is expressed as 3+4, this indicates that the primary Gleason pattern is 3 and the secondary Gleason pattern is 4. In some cases, a and b may be the same value.

[0066] In one embodiment, the computing system (10) may obtain the primary lesion pattern and the secondary lesion pattern of the biological tissue image that are stored in advance in a storage device provided by the computing system (100) or the server, or may obtain the primary lesion pattern and the secondary lesion pattern of the biological tissue image from the user terminal (20).

[0067] Or, according to an embodiment, the computing system (10) may analyze the biological tissue image or the pixel-unit diagnosis result of the biological tissue image to obtain the primary lesion pattern and the secondary lesion pattern of the biological tissue image. At this time, the computing system (100) may use a pre-trained deep learning model (200-2). The deep learning model (200-2) may be an artificial neural network that has been pre-trained to receive an input image or a pixel-unit diagnosis result of an image and determine the primary lesion pattern and the secondary lesion pattern of the corresponding image, and may be an artificial neural network that has been pre-trained to receive an image or a pixel-unit diagnosis result of an image through an input layer and output a probability that the corresponding image corresponds to a predetermined lesion pattern.

[0068] Steps S110 to S130 may be performed sequentially, but this is not a limitation. These steps may be performed in parallel or out of order. For example, step S130 may be performed before step S120.

[0069] Thereafter, the computing system (100) can search for a first partial region among the partial regions of the biological tissue image, in which the primary lesion pattern is most frequently included in the pixel-unit diagnostic results corresponding to the corresponding region (S140). At this time, the partial region of the biological tissue image may be a region having a predefined shape and area as a part of the biological tissue image. That is, the shape and area of ​​the partial region may be preset within the computing system (100).

[0070] Fig. 3 is a diagram illustrating an example of a biological tissue image and a partial region of the biological tissue image. The partial region (2) of the biological tissue image (1) may be any part of the biological tissue image (1). In the example of Fig. 3, the partial region (2) is a rectangular region having a predetermined size of w pixels × h pixels (w is a natural number such that w < W, and h is a natural number such that h < H).

[0071] Referring back to FIG. 2, the computing system (100) can search the first partial region in a sliding window manner in which the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image. For example, if the size of the partial region is set as in FIG. 3, the window can also be a rectangular shape having a size of w pixels × h pixels. In this case, the computing system (100) can calculate the number of pixels in the corresponding window having the primary lesion pattern while moving the rectangular window having the size of w × h by a predetermined sliding step (one or more pixels), and can determine the window that contains the largest number of the primary lesion patterns as the first partial region.

[0072] Thereafter, the computing system (100) can generate a modified pixel unit diagnosis result by removing the pixel unit diagnosis result corresponding to the first partial region from the pixel unit diagnosis result of the biological tissue image (S150).

[0073] Thereafter, the computing system (100) can search for a second partial region among the partial regions of the biological tissue image, in which the second lesion pattern is most frequently included in the modified pixel-unit diagnostic results corresponding to the corresponding region (S160). It goes without saying that the sliding window method can also be used in the process in which the computing system (100) searches for the second partial region.

[0074] FIG. 4 is a diagram illustrating an initial part of a process for searching a first partial region using a sliding window method. FIG. 4 illustrates a pixel-by-pixel diagnosis result and a window (bold box) for each pixel of a biological tissue image. Meanwhile, FIG. 4 exemplifies a case where the lesion pattern is a Gleason pattern. In the example of FIG. 4, the computing system (100) scans the pixel-by-pixel diagnosis result for each pixel, which is classified into Gleason patterns 3 to 5 and benign (indicated as 0), in a 6×4 window. At this time, the computing system can calculate the number of primary Gleason patterns within the window by moving the window one pixel at a time. FIG. 4 illustrates only an initial part of this process. In FIGS. 4 (a), (b), and (c), a 6×4 window moved one pixel at a time is indicated by a bold box. If the scan of one line is completed, the computing system (100) will calculate the number of primary Gleason patterns within the window by moving one pixel at a time from the beginning of the next line.

[0075] Fig. 5(a) illustrates an example of a first partial region searched by the computing system (100). If the first Gleason pattern is 3, and the computing system searches the first partial region as described with reference to Fig. 4, the computing system (100) will determine that the region on the entire slide corresponding to the region (3-1) of Fig. 5(a) containing the largest number of the first Gleason pattern (i.e., pattern 3) is the first partial region (step S140 of Fig. 2).

[0076] Thereafter, the computing system (100) can generate a modified pixel-by-pixel diagnosis result as shown in FIG. 5(b) by removing the pixel-by-pixel diagnosis result corresponding to the first partial region from the pixel-by-pixel diagnosis result of the biological tissue image (S150 of FIG. 2). FIG. 5 illustrates an example of assigning 0, indicating positive, to the corresponding region in order to remove the pixel-by-pixel diagnosis result corresponding to the first partial region. However, depending on the embodiment, a value indicating non-tissue or a third value indicating that the corresponding region is a deleted region may be assigned to the corresponding region.

[0077] Thereafter, the computing system (100) can sequentially scan the modified pixel-by-pixel diagnostic results as in Fig. 5(b) in a 6×4 sized window to search for the second partial region containing the largest number of secondary Gleason patterns. If the secondary Gleason pattern is 5, the computing system (100) will determine that the region on the entire slide corresponding to the region (3-2) of Fig. 5(c) is the second partial region (step S150 of Fig. 2).

[0078] Those skilled in the art will readily understand that the above description with reference to FIGS. 4 and 5 can be applied almost similarly to cases where the lesion pattern is not a Gleason pattern.

[0079] Referring back to FIG. 2, the computing system (100) can output the first partial region and / or the second partial region searched through the preceding process (S160). The computing system (100) can output the searched first partial region and / or the second partial region in the form of a separate image file and store it in a storage device provided by the computing system (100) or the server (10) or transmit it to an external device (e.g., the user terminal (20)). Alternatively, the computing system can output information that can identify the first partial region and / or the second partial region (e.g., coordinates and sizes of reference points on a biological tissue image, etc.).

[0080] In one embodiment, the computing system (100) can output a pixel-by-pixel diagnosis result corresponding to the first partial region of the biological tissue image by overlapping the first partial region. For example, the computing system (100) can display the pixel-by-pixel diagnosis result of each pixel in a specific color and overlap it with the first partial region of the biological tissue image. For example, the computing system (100) can set a certain level of transparency to an image in which the pixel-by-pixel diagnosis result is yellow when it is Gleason pattern 3, orange when it is Gleason pattern 4, and red when it is Gleason pattern 5, and overlap it with the first partial region of the biological tissue image. FIG. 6A illustrates an example of a biological tissue image in which pixel-by-pixel diagnosis results are overlapped, and FIG. 6B illustrates an enlarged result (4) in which the pixel-by-pixel diagnosis result is overlapped with the first partial image searched in 6A.

[0081] Meanwhile, the computing system (100) can, of course, output pixel-by-pixel diagnostic results corresponding to the second partial region by overlapping them with the second partial region of the biological tissue image.

[0082] FIG. 7 is a block diagram illustrating a logical configuration of a computing system (100) according to one embodiment of the present invention.

[0083] The computing system (100) may refer to a logical configuration equipped with hardware resources and / or software necessary to implement the technical idea of ​​the present invention, and does not necessarily refer to a single physical component or device. That is, the computing system (100) may refer to a logical combination of hardware and / or software provided to implement the technical idea of ​​the present invention, and, if necessary, may be implemented as a set of logical configurations for implementing the technical idea of ​​the present invention by being installed in devices spaced apart from each other and performing their respective functions. In addition, the computing system (100) may refer to a set of configurations that are separately implemented for each function or role for implementing the technical idea of ​​the present invention. Each configuration of the computing system (100) may be located on different physical devices or may be located on the same physical device. In addition, depending on the implementation example, the combination of software and / or hardware constituting each component of the computing system (100) may also be located on different physical devices, and the configurations located on different physical devices may be organically combined with each other to implement each of the modules.

[0084] Furthermore, the term "module" in this specification may refer to a functional and structural combination of hardware for implementing the technical concepts of the present invention and software for operating the hardware. For example, the module may refer to a logical unit of a given code and hardware resources for executing the given code, and it is readily apparent to an average expert in the technical field of the present invention that it does not necessarily refer to physically connected code or a single type of hardware.

[0085] Referring to FIG. 7, the computing system (100) may include a first acquisition module (110), a second acquisition module (120), a first search module (130), a removal module (140), a second search module (150), and an output module (160). Depending on the embodiment of the present invention, some of the above-described components may not necessarily correspond to components essential to the implementation of the present invention, and further, depending on the embodiment, the computing system (100) may include more components than this. For example, the computing system (100) may further include a communication module (not shown) for communicating with an external device, a storage module (not shown) for storing data, a control module (not shown) for controlling components and resources of the computing system (100), etc.

[0086] The first acquisition module (110) can acquire a biological tissue image of a biological tissue collected through a biopsy and a pixel-by-pixel diagnostic result of the biological tissue image. At this time, the pixel-by-pixel diagnostic result of the biological tissue image can include a lesion pattern for each pixel constituting the biological tissue image.

[0087] The second acquisition module (120) can acquire the primary lesion pattern and the secondary lesion pattern of the biological tissue image. The first search module (130) can search for a first partial region among the partial regions of the biological tissue image, in which the primary lesion pattern is most included in the pixel-unit diagnosis result corresponding to the corresponding region. At this time, the partial region of the biological tissue image is a part of the biological tissue image and is a region having a predefined shape and area.

[0088] In one embodiment, the first search module (130) can search the first partial region in a sliding window manner in which the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image.

[0089] The above removal module (140) can generate a modified pixel unit diagnosis result by removing the pixel unit diagnosis result corresponding to the first partial region from the pixel unit diagnosis result of the biological tissue image.

[0090] The second search module (150) can search for a second partial region among the partial regions of the biological tissue image, in which the second lesion pattern is most frequently included in the modified pixel unit diagnosis result corresponding to the corresponding region.

[0091] In one embodiment, the second search module (150) can search the second partial region in a sliding window manner in which the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image.

[0092] The output module (160) may output the first partial region of the biological tissue image and the second partial region of the biological tissue image as representative lesion images of the biological tissue image. For example, the output module (160) may output the first partial region of the biological tissue image and the second partial region of the biological tissue image to a predetermined storage device or output device (e.g., a display device) connected to the computing system (100) or to a predetermined external device (e.g., the user terminal (20)) connected via a network.

[0093] In one embodiment, the output module (160) may output a pixel-by-pixel diagnostic result corresponding to the first partial region by overlapping the first partial region of the biological tissue image, and may output a pixel-by-pixel diagnostic result corresponding to the second partial region by overlapping the second partial region of the biological tissue image.

[0094] Meanwhile, the above-described content describes the process of extracting a representative lesion image corresponding to the primary lesion pattern (i.e., an image of the first partial region) and a representative lesion image corresponding to the secondary lesion pattern (i.e., an image of the second partial region), but the lesion image extraction method according to the technical idea of ​​the present invention can of course also be applied to a method of extracting three or more representative lesion images.

[0095] Meanwhile, depending on the implementation example, the computing system (100) may include a processor and a memory that stores a program executed by the processor. In this case, the computing system (100) may have a configuration as illustrated in FIG. 8. FIG. 8 is a diagram illustrating a schematic configuration of a computing system (100) according to an embodiment of the present invention.

[0096] The above computing system (100) may be equipped with a memory (102) in which a program for implementing the technical idea of ​​the present invention is stored, and a processor (101) for executing the program stored in the memory (102).

[0097] An average expert in the technical field of the present invention will be able to easily infer that the processor (101) may be named by various names such as CPU, APU, mobile processor, etc., depending on the implementation example of the computing system (100). The processor may include a single-core CPU or a multi-core CPU. In addition, the computing system (100) may be implemented by organically combining a plurality of physical devices, and in this case, an average expert in the technical field of the present invention will be able to easily infer that the computing system (100) of the present invention may be implemented by providing at least one processor (101) for each physical device.

[0098] The above memory (102) stores the program and may be implemented as any type of storage device that the processor can access to run the program. The memory (102) may include a high-speed random access memory and may also include non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to the memory by the processor and other components may be controlled by a memory controller. In addition, depending on the hardware implementation example, the memory (102) may be implemented as a plurality of storage devices rather than a single storage device. In addition, the memory (102) may include a temporary storage device as well as a main memory device. In addition, it may be implemented as a volatile memory or a non-volatile memory, and may be defined to mean all types of information storage means that are implemented so that the program can be stored and run by the processor.

[0099] The above program, when executed by the processor (101), can cause the computing system (100) to perform the representative lesion image generation method described above.

[0100] Meanwhile, the method according to an embodiment of the present invention may be implemented in the form of computer-readable program commands and stored on a computer-readable recording medium, and the control program and target program according to an embodiment of the present invention may also be stored on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system.

[0101] The program commands recorded on the recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the software field.

[0102] Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Additionally, the computer-readable storage media can be distributed over network-connected computer systems so that the computer-readable code can be stored and executed in a distributed manner.

[0103] Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a device that processes information electronically, such as a computer, using an interpreter.

[0104] The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

[0105] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0106] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0107]

[0108] The present invention can be used in a method for generating a representative lesion image of a biological tissue image and a computing system for performing the same.

[0109]

Claims

1. A step of a computing system obtaining a biological tissue image and a pixel-by-pixel diagnosis result of the biological tissue image, wherein the pixel-by-pixel diagnosis result of the biological tissue image includes a lesion pattern for each pixel constituting the biological tissue image, and the lesion pattern means a grade according to a predetermined severity grade system or a histological classification of a predetermined lesion; A step in which the computing system acquires a primary lesion pattern and a secondary lesion pattern of the biological tissue image; A step in which the computing system searches for a first partial region among the partial regions of the biological tissue image, wherein the first lesion pattern is most frequently included in the pixel-unit diagnosis results corresponding to the corresponding region, wherein the partial region of the biological tissue image is a region that is a part of the biological tissue image and has a predefined shape and area; A step in which the computing system generates a modified pixel-unit diagnosis result by removing a pixel-unit diagnosis result corresponding to the first partial region from the pixel-unit diagnosis result of the biological tissue image; The computing system searches for a second partial region among the partial regions of the biological tissue image, in which the second lesion pattern is most included in the modified pixel unit diagnosis result corresponding to the corresponding region; and A method comprising the step of the computing system outputting a first partial region of the biological tissue image and a second partial region of the biological tissue image as representative lesion images of the biological tissue image.

2. In paragraph 1, The step of exploring the above first part area is: A step of searching the first partial region in a sliding window manner in which the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image, The step of exploring the above second part area is: A method comprising the step of searching the second partial region in a sliding window manner, wherein the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image.

3. In paragraph 1, The step of outputting the first partial region of the above biological tissue image and the second partial region of the above biological tissue image as a representative image of the lesion of the above biological tissue image is as follows. A step of overlapping and outputting a pixel-by-pixel diagnostic result corresponding to the first partial region of the above biological tissue image; and A method comprising the step of overlapping and outputting a pixel unit diagnostic result corresponding to the second partial region with a second partial region of the above biological tissue image.

4. In paragraph 1, The above lesion pattern is, Gleason pattern according to the Gleason grading system for prostate cancer. Tumor grade according to the histologic grading system for breast cancer, Classification of invasive breast carcinoma and intraepithelial carcinoma lesions, and One of the methods for classifying invasive lesions and precancerous lesions of the lung.

5. A computer program recorded on a recording medium installed in a data processing device and for performing the method described in any one of claims 1 to 4.

6. A computer-readable recording medium having recorded thereon a computer program for performing the method described in any one of paragraphs 1 to 4.

7. As a computing system, a processor; and a memory storing a computer program, The above computer program, A computing system which, when executed by said processor, causes said computing system to perform the method described in any one of claims 1 to 4.

8. A first acquisition module for acquiring a biological tissue image and a pixel-by-pixel diagnosis result of the biological tissue image, wherein the pixel-by-pixel diagnosis result of the biological tissue image includes a lesion pattern for each pixel constituting the biological tissue image, and the lesion pattern means a grade according to a predetermined severity grade system or a histological classification of a predetermined lesion; A second acquisition module for acquiring a primary lesion pattern and a secondary lesion pattern of the above biological tissue image; A first search module for searching for a first partial region among the partial regions of the biological tissue image, wherein the first lesion pattern is most frequently included in the pixel-unit diagnosis results corresponding to the corresponding region, wherein the partial region of the biological tissue image is a region that is a part of the biological tissue image and has a predefined shape and area; A removal module for removing a pixel-by-pixel diagnosis result corresponding to the first partial region from the pixel-by-pixel diagnosis result of the above biological tissue image to generate a modified pixel-by-pixel diagnosis result; A second search module that searches for a second partial region among the partial regions of the above biological tissue image, in which the second lesion pattern is most included in the modified pixel unit diagnosis result corresponding to the corresponding region; and A computing system including an output module that outputs a first partial region of the biological tissue image and a second partial region of the biological tissue image as representative lesion images of the biological tissue image.

9. In paragraph 8, The above first search module, The first partial area is searched using a sliding window method in which the shape and area of ​​the window are set to be the same as the partial area of ​​the biological tissue image, The above second search module, A computing system for searching the second partial region using a sliding window method in which the shape and area of ​​the window are set to be the same as the partial region of the biological tissue image.

10. In paragraph 8, The above output module is, Overlapping and outputting the pixel unit diagnostic results corresponding to the first partial region of the above biological tissue image, A computing system that overlaps and outputs pixel-by-pixel diagnostic results corresponding to the second partial region with the second partial region of the above biological tissue image.

Images