Tumor volume estimation method and computing system performing same
The method and system analyze slide images to accurately estimate tumor volume by segmenting tumor areas and processing pixel data, addressing the inaccuracy of traditional visual methods and accounting for section arrangement, resulting in improved diagnostic precision.
Patent Information
- Application Number
- PCT/KR2024/006462
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-25
- Filing Date
- 2024-05-13
- Publication Date
- 2024-11-28
AI Technical Summary
Current methods for estimating tumor volume in pathological tissue are inaccurate and rely on visual measurements, which fail to precisely calculate the volume or weight of tumors, especially when multiple adjacent sections are placed on a single slide, leading to potential overestimation of tumor volume.
A method and computing system that analyze entire slide images using algorithms to identify and segment tumor areas, calculating tumor volume by acquiring and processing pixel-level data, contour classification, and employing artificial neural networks for precise area determination, accounting for the thickness and arrangement of tissue sections.
This approach provides an accurate estimation of tumor volume, even when multiple adjacent sections are present, offering a more precise and automated method compared to traditional visual techniques, enhancing diagnostic accuracy in pathology.
Smart Images

Figure KR2024006462_28112024_PF_FP_ABST
Abstract
Description
Tumor volume estimation method and computing system performing the same
[0001] The present invention relates to a method for estimating tumor volume by analyzing a whole slide image, and a computing system for performing the same. More specifically, the present invention relates to a method for estimating tumor volume by analyzing a whole slide image of pathological tissue and utilizing an algorithm for identifying or segmenting a tumor region therefrom, and a computing system for performing the same.
[0002]
[0003] One of the primary tasks of pathology, or the department of pathology, is to diagnose a patient's biological tissue by interpreting slide images to determine the condition or signs of a specific disease. Conventional pathology diagnostic methods involve pathologists visually observing and interpreting diagnostic pathology slide images created from specimens through an optical microscope. The dawn of digital pathology began with the use of a microscope camera connected to a computer to convert pathology slides into digital images, which are then viewed and interpreted on a monitor. Recently, the advent of digital slide scanners has led to the widespread adoption of this method, which converts entire pathology slides into a single digital image, creating a pathology slide image that is then viewed and interpreted on a computer monitor.
[0004] Meanwhile, if cancer is confirmed in an organ, the standard first-line treatment is partial or complete resection of the organ. The size of the cancer (tumor) within the resected tissue is a key factor in predicting the patient's prognosis. For example, in cases of slow-growing tumors, such as breast or prostate cancer, tumor size is proportional to the length of time the tumor has been in the body since its onset, and is also proportional to the likelihood of cancer cells breaking away from the tumor and metastasizing to other tissues.
[0005] Currently, there is no method to accurately measure the volume or weight of a tumor, so the following alternative methods are used:
[0006] i) After dividing the tissue into equal intervals, the tumor location is visually confirmed for each divided tissue piece and the longest width (longest diameter) of the tumor is measured using a ruler, etc.
[0007] ii) A method of counting the number of adjacent tumors in a number of consecutive slices and then estimating the length by considering the slice thickness.
[0008] iii) After visually estimating the tumor proportion in each segmented tissue piece, the tumor proportion in the entire tissue is estimated by considering the size of each piece; a method of estimating the tumor weight from the weight of the resected tissue and the tumor proportion
[0009] However, none of these methods can be considered as methods that can accurately measure or estimate the volume or weight of a tumor using the naked eye.
[0010] An automated method for estimating the volume or weight of a tumor can be considered by analyzing pathological tissue slide images produced from tissue slices using an algorithm to identify the tumor area and then estimating the volume or weight of the tumor based on this.
[0011] Furthermore, when creating pathology slides from tissue fragments, in some cases, multiple adjacent sections (serial sections) of the same tissue are cut simultaneously and placed on a single pathology slide. In such cases, if the number of adjacent sections is not considered, the tumor volume can be estimated to be larger than the actual volume. Therefore, a tumor volume estimation method that takes this into account is needed.
[0012]
[0013] The technical task of the present invention is to provide a method for estimating tumor volume by analyzing a full slide image of pathological tissue and utilizing an algorithm to identify or segment tumor regions from the analysis. In particular, the present invention provides a method capable of estimating tumor volume as accurately as possible even when a pathological tissue slide image is created by placing multiple adjacent sections on a single pathological tissue slide.
[0014]
[0015] According to one aspect of the present invention, a tumor volume estimation system comprises: a step of acquiring a plurality of whole slide images to be used for tumor volume estimation; a step of calculating, by the tumor volume estimation system, a volume of a tumor corresponding to an ith whole slide image for all natural numbers i where 1<=i<=M (wherein, M is the number of the plurality of whole slide images); and a step of estimating, by the tumor volume estimation system, a total volume of a tumor in a tissue based on the volume of the tumor corresponding to each of the plurality of whole slide images, wherein the step of calculating the volume of the tumor corresponding to the ith whole slide image comprises: a thickness value (D) of an ith tissue slice corresponding to the ith whole slide image i ) is obtained, the area of the tumor (A) included in the i-th tissue piece i ) and a step of obtaining a thickness value (D) of the i-th tissue piece i ) and the area of the tumor included in the above tissue fragment (A i ) is provided, a tumor volume estimation method including a step of calculating a tumor volume (Vi) corresponding to the i-th entire slide image.
[0016] In one embodiment, the step of obtaining the area of the tumor included in the i-th tissue piece comprises: a step of obtaining the actual area of each pixel constituting the i-th full slide image; a step of identifying a tumor pixel which is a pixel belonging to a tumor area in the i-th full slide image; a step of identifying each tissue area included in the i-th full slide image; a step of extracting a contour of each tissue area included in the i-th full slide image; a step of determining whether contours of each tissue area included in the i-th full slide image are similar; a step of classifying similar contours among the contours of each tissue area included in the i-th full slide image into the same contour group, 1<=k<=N i For all natural numbers k (where N i The method may include a step of calculating the number of tumor pixels belonging to the kth contour group of the i-th entire slide image, wherein the number of contour groups of the i-th entire slide image is a step of calculating the number of tumor pixels belonging to the kth contour group of the i-th entire slide image, wherein the tumor pixels belonging to the kth contour group are tumor pixels located within any one contour belonging to the kth contour group among the tumor pixels of the i-th entire slide image, and a step of calculating the area of the tumor included in the i-th tissue piece by the following [Formula 1].
[0017] [Formula 1]
[0018]
[0019] (Here, A i is the area of the tumor contained in the above tissue fragment, P i,k is the number of tumor pixels belonging to the kth contour group of the ith full slide image, C i,k is the number of contours belonging to the kth contour group of the ith full slide image, X iis the actual area of the individual pixels that constitute the above i-th entire slide image)
[0020] In one embodiment, the step of obtaining the area of the tumor included in the i-th tissue piece may include the steps of: obtaining the actual area of individual pixels constituting the i-th entire slide image; identifying a tumor pixel, which is a pixel belonging to a tumor area in the i-th entire slide image; identifying each tissue area included in the i-th entire slide image; extracting a contour of each tissue area included in the i-th entire slide image; determining whether contours of each tissue area included in the i-th entire slide image are similar; classifying similar contours among contours of each tissue area included in the i-th entire slide image into the same contour group; and calculating the area of the tumor included in the i-th tissue piece by the following [Formula 2].
[0021] [Formula 2]
[0022]
[0023] (Here, A i is the area of the tumor contained in the above tissue fragment, P i is the number of tumor pixels in the above-mentioned i-th full slide image, C i is the number of contours of the i-th full slide image, N i is the number of contour groups of the above-mentioned i-th full slide image, X i is the actual area of the individual pixels that constitute the above i-th entire slide image)
[0024] In one embodiment, the step of identifying pixels belonging to a tumor region in the i-th full slide image includes the steps of dividing the i-th full slide image into a plurality of patches, and inputting each of the plurality of patches into a segmentation artificial neural network and identifying pixels belonging to a tumor region from each of the plurality of patches based on an output value output from the segmentation artificial neural network, wherein the segmentation artificial neural network may be an artificial neural network trained in advance to receive a patch through an input layer and output an output value for identifying a tumor region included in the input patch on a pixel-by-pixel basis to an output layer.
[0025] In one embodiment, the step of determining whether the contours of each tissue region included in the i-th full slide image are similar may include a step of determining that the two contours are similar if the difference or ratio of the areas of two contours among the contours of each tissue region included in the i-th full slide image satisfies a predetermined standard.
[0026] In one embodiment, the step of determining whether contours of each tissue region included in the i-th full slide image are similar may include the step of matching two contours among the contours of each tissue region included in the i-th full slide image, and the step of determining that the two contours are similar if the overlapping area of the two contours satisfies a predetermined criterion.
[0027] According to another aspect of the present invention, a computer program is provided, which is installed in a data processing device and recorded on a medium for performing the above-described method.
[0028] According to another aspect of the present invention, a computer-readable recording medium having recorded thereon a computer program for performing the above-described method is provided.
[0029] According to another aspect of the present invention, there is provided an acquisition module for acquiring a plurality of whole slide images to be used for estimating a tumor volume, a calculation module for calculating a tumor volume corresponding to an ith whole slide image for all natural numbers i where 1<=i<=M (wherein, M is the number of the plurality of whole slide images), and an estimation module for estimating the total volume of a tumor in a tissue based on the tumor volume corresponding to each of the plurality of whole slide images, wherein the calculation module calculates, in order to calculate the tumor volume corresponding to the ith whole slide image, a thickness value (D) of an ith tissue slice corresponding to the ith whole slide image. i ) is obtained, and the area of the tumor (A) included in the above tissue fragment is i ) and obtain the thickness value (D) of the i-th tissue piece. i ) and the area of the tumor included in the above tissue fragment (A i ), a tumor volume estimation system is provided that calculates the tumor volume (Vi) corresponding to the i-th full slide image.
[0030] In one embodiment, the output module obtains the actual area of individual pixels constituting the i-th full slide image, in order to obtain the area of the tumor included in the i-th tissue piece, identifies a tumor pixel that is a pixel belonging to a tumor area in the i-th full slide image, identifies each tissue area included in the i-th full slide image, extracts a contour of each tissue area included in the i-th full slide image, determines whether the contours of each tissue area included in the i-th full slide image are similar, and classifies similar contours among the contours of each tissue area included in the i-th full slide image into the same contour group, and 1<=k<=N i For all natural numbers k (where N iis the number of contour groups of the i-th full slide image), the number of tumor pixels belonging to the k-th contour group of the i-th full slide image is calculated - wherein the tumor pixels belonging to the k-th contour group are tumor pixels located within any one contour belonging to the k-th contour group among the tumor pixels of the i-th full slide image - and the area of the tumor included in the i-th tissue piece can be calculated by the following [Formula 3].
[0031] [Formula 3]
[0032]
[0033] (Here, A i is the area of the tumor contained in the above tissue fragment, P i,k is the number of tumor pixels belonging to the kth contour group of the ith full slide image, C i,k is the number of contours belonging to the kth contour group of the ith full slide image, X i is the actual area of the individual pixels that constitute the above i-th entire slide image)
[0034] In one embodiment, the output module may obtain the actual area of individual pixels constituting the i-th full slide image, identify a tumor pixel that is a pixel belonging to a tumor region in the i-th full slide image, identify each tissue region included in the i-th full slide image, extract a contour of each tissue region included in the i-th full slide image, determine whether contours of each tissue region included in the i-th full slide image are similar, classify similar contours among the contours of each tissue region included in the i-th full slide image into the same contour group, and calculate the area of the tumor included in the i-th tissue region by the following [Formula 4].
[0035] [Formula 4]
[0036]
[0037] (Here, A i is the area of the tumor contained in the above tissue fragment, P i is the number of tumor pixels in the above-mentioned i-th full slide image, C i is the number of contours of the i-th full slide image, N i is the number of contour groups of the above-mentioned i-th full slide image, X i is the actual area of the individual pixels that constitute the above i-th entire slide image)
[0038] In one embodiment, the output module divides the i-th entire slide image into a plurality of patches to identify pixels belonging to a tumor region in the i-th entire slide image, inputs each of the plurality of patches to a segmentation artificial neural network, and identifies pixels belonging to a tumor region from each of the plurality of patches based on an output value output from the segmentation artificial neural network, wherein the segmentation artificial neural network may be an artificial neural network trained in advance to receive a patch through an input layer and output an output value for identifying a tumor region included in the input patch on a pixel-by-pixel basis to an output layer.
[0039] In one embodiment, the output module (120) can determine that two contours are similar if the difference or ratio of the area of two contours among the contours of each tissue region included in the i-th entire slide image satisfies a predetermined standard.
[0040] In another embodiment, the output module (120) may determine that two contours are similar when the overlapping area of two contours among the contours of each tissue area included in the i-th entire slide image satisfies a predetermined standard.
[0041]
[0042] According to the technical concept of the present invention, a method for estimating tumor volume can be provided by analyzing a full slide image of a pathological tissue and utilizing an algorithm to identify or segment the tumor region from the analysis. In particular, a method for accurately estimating tumor volume can be provided even when a pathological tissue slide image is created by placing multiple adjacent sections on a single pathological tissue slide.
[0043]
[0044] In order to more fully understand the drawings cited in the detailed description of the present invention, a brief description of each drawing is provided.
[0045] FIG. 1 is a diagram for explaining a schematic system configuration for implementing a tumor volume estimation method according to an embodiment of the present invention.
[0046] Figure 2 is a flowchart illustrating a tumor volume estimation method according to one embodiment of the present invention.
[0047] Figure 3 is a flowchart illustrating step S200 of Figure 2 in more detail.
[0048] Figures 4 and 5 are flowcharts illustrating step S220 of Figure 3 in more detail, respectively.
[0049] FIG. 6 is a flowchart illustrating an example of step S222 of FIGS. 4 and 5.
[0050] FIG. 7 is a block diagram illustrating a logical configuration of a tumor volume estimation system according to one embodiment of the present invention.
[0051] FIG. 8 is a block diagram illustrating the physical configuration of a tumor volume estimation system according to one embodiment of the present invention.
[0052]
[0053] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. In describing the present invention, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the gist of the present invention.
[0054] Terms such as "first," "second," etc. may be used to describe various components, but these components should not be limited by these terms. Terms such as "first," "second," etc. do not denote a particular order and are used solely to distinguish one component from another.
[0055] The terminology used in this application is solely for the purpose of describing specific embodiments and is not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.
[0056] In this specification, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.
[0057] Additionally, in the present specification, when a component "transmits" data to another component, it means that the component may transmit the data directly to the other component, or may transmit the data to the other component via at least one other component. Conversely, when a component "directly transmits" data to another component, it means that the data is transmitted from the component to the other component without going through the other component.
[0058] Hereinafter, the present invention will be described in detail, focusing on embodiments thereof, with reference to the attached drawings. The same reference numerals in each drawing represent the same components.
[0059] FIG. 1 is a diagram for explaining a schematic system configuration for implementing a tumor volume estimation method according to an embodiment of the present invention.
[0060] Referring to FIG. 1, a tumor volume estimation method according to the technical idea of the present invention can be performed by a tumor volume estimation system (100).
[0061] The above tumor volume estimation system (100) can estimate the volume of a tumor within a biological tissue using multiple whole slide images produced from the resected biological tissue. The biological tissue may be biopsy samples taken from various organs of the human body or surgically resected biological tissue. The whole slide image may be a slide image obtained by scanning the biological tissue in the form of a glass slide and converting it into a digital image.
[0062] In one embodiment, the tumor volume estimation system (100) may be installed on a predetermined server (10) to implement the technical idea of the present invention. The server (10) refers to a data processing device having a computational capability for implementing the technical idea of the present invention. Generally, a person of average skill in the art will be able to easily infer that not only a data processing device that can be accessed by a client through a network, but also any device capable of performing a specific service, such as a personal computer or a mobile terminal, can be defined as a computing device.
[0063] The above tumor volume estimation system (100) can estimate the tumor volume by utilizing an algorithm that analyzes the entire slide image to identify and segment the tumor area.
[0064] In particular, in one embodiment, the tumor volume estimation system (100) may estimate the tumor volume by utilizing an artificial neural network (200) that receives an entire slide image or a patch obtained by dividing the entire slide image into a certain size as input and outputs a result value for identifying an area corresponding to a tumor in the input image. The neural network (200) is a neural network artificially constructed based on the operating principles of human neurons, includes a multilayer perceptron model, and may mean a set of information expressing a series of design specifications defining the artificial neural network. The neural network (200) may include, but is not limited to, a convolutional neural network (CNN) widely known in the fields of artificial intelligence, machine learning, or deep learning.
[0065] The server (10) including the neural network (200) and the tumor volume estimation system (100) may be a diagnostic system for diagnosing a disease or a diagnostic assistance system for providing information necessary for a doctor to diagnose a disease. The neural network (200) and the tumor volume estimation system (100) may be provided in a single physical device, i.e., the server (10), but may also be provided in different physical devices depending on the embodiment, and various modifications may be possible as needed, as will be readily apparent to an average expert in the technical field of the present invention.
[0066] In one embodiment, the server (10) and / or the tumor volume estimation system (100) may communicate with a user terminal (20). For example, the user terminal (20) may transmit full slide images to be used for tumor volume estimation to the tumor volume estimation system (100), and may also receive an estimated tumor volume value from the tumor volume estimation system (100).
[0067] Meanwhile, in Fig. 1, an example in which the tumor volume estimation system (100) is implemented in the form of a subsystem installed in a server (10) is illustrated, but it is obvious that, depending on the embodiment, the tumor volume estimation system (100) may be implemented as a single, independent system. In this case, the tumor volume estimation system (100) may be a computing system, which is a data processing device having a computational ability to implement the technical idea of the present invention, and may generally include a server, which is a data processing device that a client can access through a network, as well as a computing device such as a personal computer or a mobile terminal.
[0068] Figure 2 is a flowchart illustrating a tumor volume estimation method according to one embodiment of the present invention.
[0069] In one embodiment, the tumor volume estimation system (100) can acquire multiple full slide images to be used for tumor volume estimation (S100).
[0070] The above tumor volume estimation system (100) may obtain a plurality of full slide images from the user terminal (20), and in some cases, may obtain a plurality of full slide images previously stored in a database within the server (10).
[0071] The above multiple full slide images are produced by scanning pathological tissue slides produced from resected tissue.
[0072] Meanwhile, the tumor volume estimation system (100) can perform step S200 for all natural numbers i where 1<=i<=M (S110). Here, M is the number of the plurality of total slide images.
[0073] In step S200, the tumor volume estimation system (100) estimates the tumor volume (V) corresponding to the i-th full slide image. i ) can be produced.
[0074] Thereafter, the tumor volume estimation system (100) can estimate the total volume of the tumor within the biological tissue based on the tumor volume corresponding to each of the plurality of full slide images (S120). More specifically, the tumor volume estimation system (100) can estimate the total volume of the tumor within the biological tissue by adding up the tumor volumes corresponding to each of the plurality of full slide images.
[0075] Figure 3 is a flowchart illustrating step S200 of Figure 2 in more detail.
[0076] Referring to FIG. 3, the tumor volume estimation system (100) calculates the thickness value (D) of the i-th tissue piece corresponding to the i-th full slide image. i ) can be obtained (S210). The thickness value of the tissue piece can be input from the user terminal (20) or a preset value can be used.
[0077] In addition, the above tumor volume estimation system (100) estimates the area (A) of the tumor contained in the i-th tissue piece. i ) can be obtained (S220). The area of the tumor included in the i-th tissue piece (A i ) will be described later with reference to FIGS. 4 and 5.
[0078] Afterwards, the tumor volume estimation system (100) calculates the thickness value (D) of the i-th tissue piece i ) and the area of the tumor included in the i-tissue fragment (A i ) based on the tumor volume (V) corresponding to the i-th full slide image. i ) can be calculated (S230). More specifically, the tumor volume estimation system (100) calculates the thickness value (D) of the i-th tissue piece. i ) and the area of the tumor included in the i-tissue fragment (A i ) is multiplied to obtain the tumor volume (V) corresponding to the i-th full slide image.i ) can be produced.
[0079] Figures 4 and 5 are flowcharts illustrating step S220 of Figure 3 in more detail, respectively.
[0080] In the example of FIGS. 4 and 5, in the process of obtaining the area (Ai) of the tumor contained in the i-th tissue piece, the tumor volume estimation system (100) may consider adjacent sections. When creating a pathological tissue slide from a tissue piece, in some cases, adjacent sections created by cutting the same tissue multiple times are placed on a single slide at once. In this case, when multiple adjacent sections are placed on a single entire slide, the number of adjacent sections must be taken into account when calculating the area of the tumor region.
[0081] For example, in the case of surgical specimens, since it is impossible to place the entire specimen on a single slide, the surgical specimen is divided into as many as dozens of pieces and each piece is made into a slide. At this time, the divided specimen is formalin-fixed and paraffin-embedded to create a tissue block. The created tissue block is then sectioned and placed on a slide, and the paraffin is melted and stained, etc. During this process, multiple sections are sometimes cut and placed on the slide, and these are called adjacent sections. In addition to sections cut from the same block, sections cut from multiple blocks are sometimes placed on a single slide. For example, one section from each of three blocks can be placed on a single slide, or two sections from one block (i.e., two adjacent sections) can be placed on a single slide. In this case, a total of 5 sections are placed on one slide, but when actually calculating the area of the tissue, two adjacent sections must be treated as one section, and the area must be calculated for a total of 4 sections.
[0082] First, referring to FIG. 4, the tumor volume estimation system (100) calculates the actual area (X) of each individual pixel constituting the i-th entire slide image. i ) can be obtained (S221). Here, the actual area of each pixel constituting the entire slide image may mean the area of the area corresponding to each pixel among the areas of the tissue slide used to create the entire slide image. In other words, the actual area of each pixel may be the area occupied by one pixel, which is the smallest unit constituting the adjusted entire slide image, when the entire slide image is enlarged or reduced to adjust the size of the entire slide to be the same as the size of the tissue slide used to create the entire slide image. For example, if the entire slide image is composed of 2000×1000 pixels and the entire slide image is created by scanning a tissue slide with a size of 2 cm×1 cm, the area of each pixel of the entire slide image is 2 cm / 2000×1 cm / 1000 = 10 -6 cm 2 am.
[0083] The above tumor volume estimation system (100) can calculate the actual area of each pixel constituting the ith full slide image using the above method. Alternatively, the above tumor volume estimation system (100) can receive the actual area of each pixel from the user terminal (20) or extract and use the value stored in the metadata of the ith full slide image file by the scanner.
[0084] In addition, the tumor volume estimation system (100) can identify pixels belonging to the tumor area in the i-th full slide image (S222). The tumor volume estimation system (100) applies a predetermined algorithm to identify and segment the tumor area in the i-th full slide image, and calculates the area (A) of the identified tumor area. i) can be obtained, and in particular, pixels belonging to the tumor area in the i-th full slide image can be identified using an artificial neural network (200) learned through deep learning, and the process is illustrated in FIG. 6.
[0085] FIG. 6 is a flowchart illustrating an example of step S222 of FIGS. 4 and 5.
[0086] Referring to FIG. 6, the tumor volume estimation system (100) can divide the i-th entire slide image into multiple patches (S222-1). A patch refers to a unit grid of an image in which the entire slide is divided into equal sizes.
[0087] Thereafter, the tumor volume estimation system (100) can perform step S222-3 for each of the plurality of patches constituting the i-th full slide image (S222-2).
[0088] In step S222-3, the tumor volume estimation system (100) inputs a patch into a segmentation artificial neural network and can identify pixels belonging to a tumor area in the input patch based on output values output from the segmentation artificial neural network (200) (S222-3). At this time, the segmentation artificial neural network (200) is an artificial neural network for segmenting an area corresponding to a tumor from an input image, and may be an artificial neural network trained in advance to receive a patch through an input layer and output an output value for identifying a tumor area included in the input patch on a pixel-by-pixel basis to an output layer.
[0089] Thereafter, the tumor volume estimation system (100) can identify pixels belonging to the tumor region in the i-th full slide image by combining pixels belonging to the tumor region of each of the plurality of patches (S222-4).
[0090] Referring again to FIG. 4, the tumor volume estimation system (100) can identify each tissue region included in the i-th full slide image (S223).
[0091] In one embodiment, the tumor volume estimation system (100) can identify each tissue region included in the i-th full slide image by using another artificial neural network that has been pre-trained to identify tissue regions, similar to the method of identifying tumor regions using the segmentation artificial neural network (200) described above.
[0092] Thereafter, the tumor volume estimation system (100) can extract the contour of each tissue area included in the i-th entire slide image (S224).
[0093] More specifically, since the tumor volume estimation system (100) has previously distinguished between tissue regions and non-tissue regions, it can generate a binary mask image in which the tissue regions and non-tissue regions are distinguished by binary values for the i-th entire slide image. For example, the binary mask image may be an image in which the tissue region of the i-th entire slide image is composed of 1s and the non-tissue region is composed of 0s. Then, the tumor volume estimation system (100) can extract the contour of each tissue region based on the binary mask. The tumor volume estimation system (100) can extract the plurality of contours based on the binary mask through various known contour extraction methods, such as the contour extraction method proposed by Suzuki, S. and Abe. K (Topological Structural Analysis of Digitized Binary Images by Border Following. CVGIP 30 1, pp 32-46 (1985)), and in addition, any algorithm that can extract a contour composed of a set of coordinate series can be used.
[0094] When the contours of each tissue region included in the i-th full slide image are extracted, it is possible to determine whether the contours of each tissue region included in the i-th full slide image are similar (S225).
[0095] There may be various specific methods for determining whether two contours are similar.
[0096] In one embodiment, the tumor volume estimation system (100) can determine whether two contours are similar by comparing the areas of the two contours. More specifically, the tumor volume estimation system (100) can determine that the two contours are similar if the difference or ratio of the areas of two contours among the contours of each tissue region included in the i-th full slide image satisfies a predetermined standard. For example, the tumor volume estimation system (100) can determine that the two contours are similar if the absolute value of the difference in the areas of the two contours is less than or equal to a predetermined standard, or if the ratio of the areas of the two contours (area of the small contour / area of the large contour) is greater than or equal to a predetermined standard.
[0097] This method utilizes the fact that adjacent sections are sections cut from the same tissue block at different locations, resulting in very similar tissue shapes. Furthermore, sections cut from different tissues typically differ in size. This method has the advantage of being very fast, as it simply compares the area of the two contours.
[0098] In another embodiment, the tumor volume estimation system (100) can first align two contours using various image matching algorithms known in the art, and if the overlapping area of the two contours satisfies a predetermined standard, the two contours can be determined to be similar. For example, if the ratio of the area of the intersection of the two aligned contours to the area of the union is greater than a predetermined standard, the two contours can be determined to be similar. This method requires more calculation time than a method that determines the similarity of the contours only based on the similarity of the areas, but has the advantage of being able to determine the similarity of the two contours more accurately.
[0099] Meanwhile, the tumor volume estimation system (100) can classify similar contours among the contours of each tissue area included in the i-th full slide image into the same contour group (S226).
[0100] For example, if tissue regions 1 to 6 are identified in the i-th full slide image, and tissue regions 1, 2, and 6 are similar to each other, and tissue regions 3 and 4 are determined to be similar to each other, the tissue regions included in the i-th full slide image can be classified into a first group consisting of <tissue region 1, tissue region 2, and tissue region 6>, a second group consisting of <tissue region 3, and tissue region 4>, and a third group consisting of <tissue region 5>.
[0101] If N in the above i-th full slide image i If the contour group of the dog is classified, the tumor volume estimation system (100) is 1<=k<=N i Steps S227-1 and S227-2 can be performed for all natural numbers k (S227). Here, N i is the number of contour groups of the above-mentioned i-th entire slide image.
[0102] In step S227-1, the tumor volume estimation system (100) can identify a tumor pixel belonging to the kth contour group, and at this time, the tumor pixel belonging to the kth contour group means a tumor pixel located within any one contour belonging to the kth contour group among the tumor pixels of the ith entire slide image. In one embodiment, the tumor volume estimation system (100) can identify a tumor pixel belonging to the kth contour group by performing a logical AND operation on a first binary mask in which a tumor area and a non-tumor area are distinguished as binary values, and a second binary mask in which an area belonging to a tissue area within the kth contour group and an area not belonging to the tissue area are distinguished as binary values.
[0103] In step S227-1, the tumor volume estimation system (100) calculates the number of tumor pixels (P) belonging to the kth contour group. i,k ) can be counted.
[0104] Thereafter, the tumor volume estimation system (100) can calculate the area of the tumor included in the i-th tissue piece by the following [Formula 1].
[0105] [Formula 1]
[0106]
[0107] Here, A i is the area of the tumor contained in the above tissue fragment, P i,k is the number of tumor pixels belonging to the kth contour group of the ith full slide image, C i,k is the number of contours belonging to the kth contour group of the ith full slide image, X i is the actual area of the individual pixels that constitute the above-mentioned i-th entire slide image.
[0108] Meanwhile, in the embodiment of FIG. 4 discussed above, a process of identifying tumor pixels for each contour group must be performed, and this process may have a somewhat large overhead. Therefore, even if the area of the tumor included in the i-th tissue piece is calculated somewhat inaccurately, in order to reduce the computational overhead, the tumor volume estimation system (100) may calculate the area of the tumor included in the i-th tissue piece through a process such as FIG. 5.
[0109] In the embodiment of Fig. 5, steps S221 to S226 of Fig. 4 are performed in the same manner.
[0110] Thereafter, the tumor volume estimation system (100) can calculate the area of the tumor included in the i-th tissue piece by the following [Formula 2] (S229).
[0111] [Formula 2]
[0112]
[0113] Here, A i is the area of the tumor contained in the above tissue fragment, P i is the number of tumor pixels in the above-mentioned i-th full slide image, C i is the number of contours of the i-th full slide image, N i is the number of contour groups of the above-mentioned i-th full slide image, X i is the actual area of the individual pixels that constitute the above-mentioned i-th entire slide image.
[0114] FIG. 7 is a block diagram illustrating the logical configuration of a tumor volume estimation system (100) according to one embodiment of the present invention.
[0115] The above tumor volume estimation system (100) may refer to a logical configuration equipped with hardware resources and / or software necessary to implement the technical idea of the present invention, and does not necessarily refer to a single physical component or a single device. That is, the tumor volume estimation system (100) may refer to a logical combination of hardware and / or software provided to implement the technical idea of the present invention, and, if necessary, may be implemented as a set of logical configurations for implementing the technical idea of the present invention by being installed in devices spaced apart from each other and performing each function. In addition, the tumor volume estimation system (100) may refer to a set of configurations that are separately implemented for each function or role for implementing the technical idea of the present invention. Each configuration of the tumor volume estimation system (100) may be located in a different physical device, or may be located in the same physical device. In addition, depending on the implementation example, the combination of software and / or hardware constituting each component of the tumor volume estimation system (100) may also be located in different physical devices, and the components located in different physical devices may be organically combined with each other to implement each of the modules.
[0116] Furthermore, the term "module" in this specification may refer to a functional and structural combination of hardware for implementing the technical concepts of the present invention and software for operating the hardware. For example, the module may refer to a logical unit of a given code and hardware resources for executing the given code, and it is readily apparent to an average expert in the technical field of the present invention that it does not necessarily refer to physically connected code or a single type of hardware.
[0117] Referring to FIG. 7, the tumor volume estimation system (100) may include an acquisition module (110), an output module (120), an estimation module (130), and an output module (140). Depending on the embodiment of the present invention, some of the above-described components may not necessarily correspond to components essential to the implementation of the present invention, and, of course, depending on the embodiment, the tumor volume estimation system (100) may include more components than this. For example, the tumor volume estimation system (100) may further include a communication module (not shown) for communicating with an external device, a storage module (not shown) for storing data, a control module (not shown) for controlling components and resources of the tumor volume estimation system (100), etc.
[0118] The above acquisition module (110) can acquire multiple full slide images to be used for tumor volume estimation.
[0119] The above-mentioned output module (120) can output the tumor volume corresponding to the ith full slide image for all natural numbers i where 1<=i<=M. Here, M is the number of the plurality of full slide images.
[0120] In one embodiment, the output module obtains the actual area of individual pixels constituting the i-th full slide image, in order to obtain the area of the tumor included in the i-th tissue piece, identifies a tumor pixel that is a pixel belonging to a tumor area in the i-th full slide image, identifies each tissue area included in the i-th full slide image, extracts a contour of each tissue area included in the i-th full slide image, determines whether the contours of each tissue area included in the i-th full slide image are similar, and classifies similar contours among the contours of each tissue area included in the i-th full slide image into the same contour group, and 1<=k<=Ni For all natural numbers k (where N i is the number of contour groups of the i-th full slide image), the number of tumor pixels belonging to the k-th contour group of the i-th full slide image is calculated - wherein the tumor pixels belonging to the k-th contour group are tumor pixels located within any one contour belonging to the k-th contour group among the tumor pixels of the i-th full slide image - and the area of the tumor included in the i-th tissue piece can be calculated by the following [Formula 3].
[0121] [Formula 3]
[0122]
[0123] (Here, A i is the area of the tumor contained in the above tissue fragment, P i,k is the number of tumor pixels belonging to the kth contour group of the ith full slide image, C i,k is the number of contours belonging to the kth contour group of the ith full slide image, X i is the actual area of the individual pixels that constitute the above i-th entire slide image)
[0124] In one embodiment, the output module may obtain the actual area of individual pixels constituting the i-th full slide image, identify a tumor pixel that is a pixel belonging to a tumor region in the i-th full slide image, identify each tissue region included in the i-th full slide image, extract a contour of each tissue region included in the i-th full slide image, determine whether contours of each tissue region included in the i-th full slide image are similar, classify similar contours among the contours of each tissue region included in the i-th full slide image into the same contour group, and calculate the area of the tumor included in the i-th tissue region by the following [Formula 4].
[0125] [Formula 4]
[0126]
[0127] (Here, A i is the area of the tumor contained in the above tissue fragment, P i is the number of tumor pixels in the above-mentioned i-th full slide image, C i is the number of contours of the i-th full slide image, N i is the number of contour groups of the above-mentioned i-th full slide image, X i is the actual area of the individual pixels that constitute the above i-th entire slide image)
[0128] In one embodiment, the output module (120) divides the i-th entire slide image into a plurality of patches to identify pixels belonging to a tumor region in the i-th entire slide image, inputs each of the plurality of patches to a segmentation artificial neural network, and identifies pixels belonging to a tumor region from each of the plurality of patches based on an output value output from the segmentation artificial neural network. The segmentation artificial neural network may be an artificial neural network trained in advance to receive a patch through an input layer and output an output value for identifying a tumor region included in the input patch on a pixel-by-pixel basis to an output layer.
[0129] In one embodiment, the output module (120) can determine that two contours are similar if the difference or ratio of the area of two contours among the contours of each tissue region included in the i-th entire slide image satisfies a predetermined standard.
[0130] In another embodiment, the output module (120) may determine that two contours are similar when the overlapping area of two contours among the contours of each tissue area included in the i-th entire slide image satisfies a predetermined standard.
[0131] Referring again to FIG. 7, the estimation module (130) can estimate the total volume of the tumor within the tissue by adding up the volume of the tumor corresponding to each of the plurality of full slide images.
[0132] The above output module (140) can output the total volume value of the tumor within the tissue estimated to a predetermined output device (e.g., a display device, etc.) connected to the tumor volume estimation system (100) or to a predetermined external device (e.g., the user terminal (20)) connected via a network.
[0133] Meanwhile, depending on the implementation example, the tumor volume estimation system (100) may include a processor and a memory that stores a program executed by the processor. In this case, the tumor volume estimation system (100) may have a configuration as illustrated in FIG. 8. FIG. 8 is a diagram illustrating a schematic configuration of a tumor volume estimation system (100) according to an embodiment of the present invention.
[0134] The above tumor volume estimation system (100) may be equipped with a memory (102) in which a program for implementing the technical idea of the present invention is stored, and a processor (101) for executing the program stored in the memory (102).
[0135] An average expert in the technical field of the present invention will be able to easily infer that the processor (101) may be named by various names such as CPU, APU, mobile processor, etc., depending on the implementation example of the tumor volume estimation system (100). The processor may include a single core CPU or a multi-core CPU. In addition, the tumor volume estimation system (100) may be implemented by organically combining a plurality of physical devices, and in this case, an average expert in the technical field of the present invention will be able to easily infer that the tumor volume estimation system (100) of the present invention may be implemented by providing at least one processor (101) for each physical device.
[0136] The above memory (102) stores the program and may be implemented as any type of storage device that the processor can access to run the program. The memory (102) may include a high-speed random access memory and may also include non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to the memory by the processor and other components may be controlled by a memory controller. In addition, depending on the hardware implementation example, the memory (102) may be implemented as a plurality of storage devices rather than a single storage device. In addition, the memory (102) may include a temporary storage device as well as a main memory device. In addition, it may be implemented as a volatile memory or a non-volatile memory, and may be defined to mean all types of information storage means that are implemented so that the program can be stored and run by the processor.
[0137] The above program, when executed by the processor (101), can cause the tumor volume estimation system (100) to perform the above-described tumor volume estimation method.
[0138] Meanwhile, the method according to an embodiment of the present invention may be implemented in the form of computer-readable program commands and stored on a computer-readable recording medium, and the control program and target program according to an embodiment of the present invention may also be stored on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system.
[0139] The program commands recorded on the recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the software field.
[0140] Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Additionally, the computer-readable storage media can be distributed over network-connected computer systems so that the computer-readable code can be stored and executed in a distributed manner.
[0141] Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a device that processes information electronically, such as a computer, using an interpreter.
[0142] The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.
[0143] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.
[0144] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.
[0145]
[0146] The present invention can be used in a tumor volume estimation method and a computing system performing the same.
Claims
1. A tumor volume estimation system acquires multiple full slide images to be used for tumor volume estimation; The above tumor volume estimation system comprises a step of calculating the tumor volume corresponding to the i-th full slide image for all natural numbers i where 1<=i<=M (where M is the number of the plurality of full slide images); and The above tumor volume estimation system includes a step of estimating the total volume of a tumor within a tissue based on the volume of the tumor corresponding to each of the plurality of full slide images, The step of calculating the tumor volume corresponding to the above i-th full slide image is: A step of obtaining a thickness value of an i-th tissue piece corresponding to the i-th entire slide image; A step of obtaining the area of the tumor included in the above i-th tissue piece; and A tumor volume estimation method comprising a step of calculating a tumor volume corresponding to the i-th full slide image based on a thickness value of the i-th tissue piece and an area of a tumor included in the i-th tissue piece.
2. In paragraph 1, The step of obtaining the area of the tumor included in the above i-tissue piece is: A step of obtaining the actual area of each individual pixel constituting the above i-th entire slide image; A step of identifying a tumor pixel, which is a pixel belonging to a tumor area in the above i-th full slide image; A step of identifying each tissue region included in the above i-th full slide image; A step of extracting a contour of each tissue area included in the above i-th entire slide image; A step of determining whether there is similarity between contours of each tissue area included in the above i-th full slide image; A step of classifying similar contours among the contours of each tissue area included in the above i-th entire slide image into the same contour group; 1<=k<=N i For all natural numbers k (where N i is the number of contour groups of the i-th full slide image), calculating the number of tumor pixels belonging to the k-th contour group of the i-th full slide image, wherein the tumor pixels belonging to the k-th contour group are tumor pixels located within any one contour belonging to the k-th contour group among the tumor pixels of the i-th full slide image; and A tumor volume estimation method comprising a step of calculating the area of a tumor included in the i-th tissue piece by the following [Formula 1]. [Formula 1] (Here, A i is the area of the tumor contained in the above tissue fragment, P i,k is the number of tumor pixels belonging to the kth contour group of the ith full slide image, C i,k is the number of contours belonging to the kth contour group of the ith full slide image, X i is the actual area of the individual pixels that constitute the above i-th entire slide image) 3. In paragraph 1, The step of obtaining the area of the tumor included in the above i-tissue piece is: A step of obtaining the actual area of each individual pixel constituting the above i-th entire slide image; A step of identifying a tumor pixel, which is a pixel belonging to a tumor area in the above i-th full slide image; A step of identifying each tissue region included in the above i-th full slide image; A step of extracting a contour of each tissue area included in the above i-th entire slide image; A step of determining whether there is similarity between contours of each tissue area included in the above i-th full slide image; A step of classifying similar contours among the contours of each tissue area included in the above-mentioned i-th full slide image into the same contour group; and A tumor volume estimation method comprising a step of calculating the area of a tumor included in the i-th tissue piece by the following [Formula 2]. [Formula 2] (Here, A i is the area of the tumor contained in the above tissue fragment, P i is the number of tumor pixels in the above-mentioned i-th full slide image, C i is the number of contours of the i-th full slide image, N i is the number of contour groups of the above-mentioned i-th full slide image, X i is the actual area of the individual pixels that constitute the above i-th entire slide image) 4. In paragraph 2 or 3, The step of identifying a tumor pixel, which is a pixel belonging to a tumor area in the above-mentioned i-th full slide image, is as follows: a step of dividing the above i-th entire slide image into a plurality of patches; and A step of inputting each of the plurality of patches into a segmentation artificial neural network and identifying pixels belonging to a tumor area from each of the plurality of patches based on an output value output from the segmentation artificial neural network, The above segmentation artificial neural network, A tumor volume estimation method, which is an artificial neural network trained in advance to input a patch through an input layer and output values to an output layer for identifying the tumor area included in the input patch on a pixel-by-pixel basis.
5. In paragraph 2 or 3, The step of determining whether the contours of each tissue area included in the above-mentioned i-th full slide image are similar is as follows. A tumor volume estimation method including a step of determining that two contours are similar if the difference or ratio of the areas of two contours among the contours of each tissue region included in the above-mentioned i-th full slide image satisfies a predetermined standard.
6. In paragraph 2 or 3, The step of determining whether the contours of each tissue area included in the above-mentioned i-th full slide image are similar is as follows. A step of aligning two contours among the contours of each tissue area included in the above-mentioned i-th full slide image; and A tumor volume estimation method including a step of determining that the two contours are similar if the overlapping area of the two contours satisfies a predetermined criterion.
7. A computer program recorded on a medium for performing the method described in any one of claims 1 to 3 and installed in a data processing device.
8. A computer-readable recording medium having recorded thereon a computer program for performing the method described in any one of paragraphs 1 to 3.
9. An acquisition module that acquires multiple full slide images to be used for tumor volume estimation; For all natural numbers i such that 1<=i<=M (where M is the number of the plurality of full slide images), a calculation module that calculates the tumor volume corresponding to the i-th full slide image; and An estimation module for estimating the total volume of a tumor within a tissue based on the volume of the tumor corresponding to each of the plurality of full slide images, The above output module, in order to calculate the tumor volume corresponding to the i-th full slide image, Obtain the thickness value of the i-th tissue piece corresponding to the i-th entire slide image, Obtain the area of the tumor included in the above i-th tissue fragment, A tumor volume estimation system that calculates the volume of a tumor corresponding to the i-th full slide image based on the thickness value of the i-th tissue piece and the area of the tumor included in the i-th tissue piece.
10. In paragraph 9, The above output module obtains the area of the tumor included in the i-th tissue piece, Obtain the actual area of each individual pixel constituting the above i-th entire slide image, Identifying a tumor pixel, which is a pixel belonging to a tumor area in the above i-th full slide image, Identify each tissue region included in the above i-th full slide image, Extract the contour of each tissue region included in the above i-th full slide image, Determine whether the contours of each tissue area included in the above i-th full slide image are similar, Among the contours of each tissue area included in the above i-th full slide image, similar contours are classified into the same contour group, 1<=k<=N i For all natural numbers k (where N i is the number of contour groups of the i-th full slide image), and calculates the number of tumor pixels belonging to the k-th contour group of the i-th full slide image, wherein the tumor pixels belonging to the k-th contour group are tumor pixels located within any one contour belonging to the k-th contour group among the tumor pixels of the i-th full slide image. A tumor volume estimation system that calculates the area of a tumor included in the above-mentioned i-th tissue piece by [Formula 3] below. [Formula 3] (Here, A i is the area of the tumor contained in the above tissue fragment, P i,k is the number of tumor pixels belonging to the kth contour group of the ith full slide image, C i,k is the number of contours belonging to the kth contour group of the ith full slide image, X i is the actual area of the individual pixels that constitute the above i-th entire slide image) 11. In paragraph 9, The above output module obtains the area of the tumor included in the i-th tissue piece, Obtain the actual area of each individual pixel constituting the above i-th entire slide image, Identifying a tumor pixel, which is a pixel belonging to a tumor area in the above i-th full slide image, Identify each tissue region included in the above i-th full slide image, Extract the contour of each tissue region included in the above i-th full slide image, Determine whether the contours of each tissue area included in the above i-th full slide image are similar, Among the contours of each tissue area included in the above i-th full slide image, similar contours are classified into the same contour group, A tumor volume estimation system that calculates the area of a tumor included in the above-mentioned i-th tissue piece by [Formula 4] below. [Formula 4] (Here, A i is the area of the tumor contained in the above tissue fragment, P i is the number of tumor pixels in the above-mentioned i-th full slide image, C i is the number of contours of the i-th full slide image, N i is the number of contour groups of the above-mentioned i-th full slide image, X i is the actual area of the individual pixels that constitute the above i-th entire slide image) 12. In paragraph 10 or 11, The above output module identifies pixels belonging to the tumor area in the i-th full slide image. Splitting the above i-th entire slide image into multiple patches, Inputting each of the plurality of patches into a segmentation artificial neural network and identifying pixels belonging to a tumor area from each of the plurality of patches based on the output value output from the segmentation artificial neural network. The above segmentation artificial neural network, A tumor volume estimation system, which is a pre-trained artificial neural network that receives a patch through an input layer and outputs an output value to identify the tumor area contained in the input patch on a pixel-by-pixel basis to the output layer.
13. In paragraph 10 or 11, The above output module is, A tumor volume estimation system that determines that two contours are similar if the difference or ratio of the area of two contours among the contours of each tissue region included in the above-mentioned i-th full slide image satisfies a predetermined standard.
14. In paragraph 10 or 11, The above output module is, A tumor volume estimation method in which two contours among the contours of each tissue region included in the above-mentioned i-th full slide image are aligned, and if the overlapping area of the two contours satisfies a predetermined standard, the two contours are judged to be similar.
Citation Information
Patent Citations
Intelligent and automatic delineation method for gross tumor volume and organs at risk
CN107403201A
Tumor region size measurement method, apparatus, and program
JP2009165615A
Method for computing infarct volume in a stroke model
KR1020090100688A
Flexible printed circuit board for semiconductor and display test socket
KR102259559B1
Method, device and system for estimating size of polyp using snare based on artificial intelligence
KR102428485B1
