Method for evaluating tissue images, evaluation device, and evaluation program.

Abstract

This provides an evaluation method for constructing robust quantitative indicators from tissue images. [Solution] The evaluation method provides cell nucleus contour data to each cell in the tissue image, assigns a first boundary line to one side of the tissue image and a second boundary line to the other side, and places n (n: an integer greater than or equal to 1) isometric lines in the region of the tissue image sandwiched between the first and second boundary lines to partition (n+1) layers. Here, an isometric line is a line consisting of a set of points where the ratio of the distance to the first boundary line to the distance to the second boundary line is equal, and the ratio of the distance to the first boundary line to the distance to the second boundary line at the k-th isometric line (k is an integer greater than or equal to 1 or less than or equal to n) from the first boundary line satisfies k:n+1-k. The evaluation method further determines a first index based on the contour data, sets a region of interest consisting of one or more layers, and determines a first statistical value for the first index in the region of interest.

Description

[Technical Field]

[0001] This disclosure relates to a method for evaluating tissue images, an evaluation apparatus, and an evaluation program. [Background technology]

[0002] In diagnostic imaging using pathological images, clinicians have traditionally made comprehensive judgments based on their "experienced eye" regarding the atypia and irregularity of cells and cell nuclei, as well as the depth of cancer invasion. However, the pathological specimens used in such diagnoses are not standardized. For example, the thickness of pathological specimens varies, and their shape can be bent or folded. Therefore, there has been a need for an integrated index that allows for quantitative comparison of pathological specimens.

[0003] In recent years, methodologies using artificial intelligence (AI) for image diagnosis using pathological images to classify cases are becoming mainstream. For example, Non-Patent Document 1 reports a quantification technique that uses deep neural networks to capture the characteristics of diseased tissue images. This technique enables classification and visualization of cancer pathological tissue images based on their histological characteristics, as well as the search for similar cases. Furthermore, the equivalence of evaluations performed using this technique with evaluations performed by pathologists has been confirmed. However, because this technique evaluates texture characteristics rather than the direct shape of the image, it picks up information such as the intensity of staining and color tone in individual images, making standardization difficult.

[0004] Furthermore, another fundamental problem in utilizing AI-based image diagnosis is that the process leading to the diagnosis is not clearly shown, making it difficult to compare and verify with the evaluation criteria held by pathologists, and thus making it difficult to gain the reassurance and acceptance of patients and healthcare professionals. [Prior art documents] [Patent Documents]

[0005] [Non-Patent Document 1] D. Komura, et al., Universal encoding of pan-cancer histology by deep texture representations. Cell Reports 38, 110424(2022) [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] There is a need to develop robust quantitative indicators that are not affected by the color tone of tissue images, the intensity of staining, or the experience level of the pathologist.

[0007] One of the purposes of this disclosure is to provide a method for evaluating tissue images, an evaluation device, and an evaluation program for constructing robust quantitative indicators for tissue images. [Means for solving the problem]

[0008] The method for evaluating tissue images according to this disclosure includes the steps of: providing cell nucleus contour data for each cell constituting the tissue in the tissue image; assigning a first boundary line to one side of the tissue image and a second boundary line to the other side; arranging n (where n is an integer of 1 or more) isometric lines in the region of the tissue image sandwiched between the first boundary line and the second boundary line to partition (n+1) layers, wherein the isometric lines are lines consisting of a set of points where the ratio of the distance to the first boundary line to the distance to the second boundary line is equal, and the ratio of the distance to the first boundary line to the distance to the second boundary line at the k-th (where k is an integer of 1 or more and n or less) isometric line from the first boundary line satisfies k:n+1-k; determining a first index based on contour data; setting a region of interest consisting of one or more layers from the (n+1) layers; and determining a first statistical value for the first index in the region of interest.

[0009] In an exemplary embodiment, the tissue image is an image of epithelial tissue, the first boundary line is a line indicating the epidermal surface of the epithelium, and the second boundary line is a line indicating the basement membrane of the epithelium.

[0010] In an exemplary embodiment, the first indicator is the area of the cell nucleus, the density of the nuclear contour, the roundness of the nuclear contour, the aspect ratio AR of the nuclear contour, the number density ρ of the cell nucleus, or the major axis angle formed by the major axis of the nuclear contour and the equipotential line closest to the nuclear contour.

[0011] In an exemplary embodiment, the first statistical value is the mean value, the median value, the skewness, the kurtosis, or the standard deviation.

[0012] In an exemplary embodiment, the first indicator is the number density ρ of the cell nucleus, and the first statistical value is a and b when the number density ρ(x) of the cell nucleus normalized using the quantity x (where x = h / (n + 1)) obtained by normalizing the number density of the cell nucleus in the h-th layer in the region of interest by the number of layers n + 1 is fitted with the formula f(x) = exp(ax + b).

[0013] In an exemplary embodiment, the first indicator is the major axis angle, and the first statistical value is the mean value θ of the major axis angles in the region of interest, or the variance V obtained by the following formula when the number of cell nuclei contained in the region of interest is N. Formula V = 1 - R (where R is obtained by the formula Re 2iθ =<e 2iθj >, j means an integer from 1 to N, θ j means the angle formed by the major axis of the j-th cell nucleus and its closest equipotential line, and <> means the average of the j nuclei.)

[0014] In an exemplary embodiment, the first indicator is the major axis angle, and the first statistical value is the orientation order variable S obtained by the following formula when the number of cell nuclei contained in the region of interest is N: Formula S = <cos2θ j > (where j means an integer from 1 to N, θ jmeans the angle formed by the long axis of the j-th cell nucleus and its nearest isometric line, and < > means the average of the j nuclei.) Alternatively, an orientation order variable S weighted by the aspect ratio obtained by the following formula AR : Formula S AR = <AR j ·cos(2θ j )> / <AR j > (Here, AR j is the aspect ratio of the j-th cell nucleus).

[0015] In an exemplary embodiment, it further includes the step of setting a region of interest different from the region of interest, and then the step of obtaining a second statistical value in the different region of interest.

[0016] The evaluation apparatus for tissue images according to the present disclosure includes, in a tissue image, a contour determination unit that gives contour data of cell nuclei to each cell constituting the tissue, a boundary line setting unit that gives a first boundary line on one side and a second boundary line on the other side in the tissue image, and an isometric line setting unit that arranges n isometric lines (n means an integer of 1 or more) in the region of the tissue image sandwiched between the first boundary line and the second boundary line to partition (n + 1) layers. The isometric line is a line consisting of a set of points where the ratio of the distance to the first boundary line and the distance to the second boundary line is equal, and the ratio of the distance to the first boundary line and the distance to the second boundary line in the k-th (k means an integer of 1 or more and n or less) isometric line from the first boundary line satisfies k:n + 1 - k. The isometric line setting unit, a first index calculation unit that obtains a first index based on the contour data, a region of interest setting unit that sets a region of interest consisting of one or more layers among the (n + 1) layers, and a first statistical processing unit that obtains a first statistical value in the region of interest for the first index.

[0017] The tissue image evaluation program according to this disclosure causes a computer processing circuit to execute the following steps: providing cell nucleus contour data for each cell constituting the tissue in the tissue image; assigning a first boundary line to one side of the tissue image and a second boundary line to the other side; arranging n (where n is an integer of 1 or more) isometric lines in the region of the tissue image sandwiched between the first boundary line and the second boundary line to partition (n+1) layers, wherein the isometric lines are lines consisting of a set of points where the ratio of the distance to the first boundary line to the distance to the second boundary line is equal, and the ratio of the distance to the first boundary line to the distance to the second boundary line at the k-th (where k is an integer of 1 or more and n or less) isometric line from the first boundary line satisfying k:n+1-k; determining a first index based on contour data; setting a region of interest consisting of one or more layers from the (n+1) layers; and determining a first statistical value for the first index in the region of interest. [Effects of the Invention]

[0018] This disclosure provides a method for evaluating tissue images, an evaluation apparatus, and an evaluation program for constructing robust quantitative indicators. [Brief explanation of the drawing]

[0019] [Figure 1A] Block diagram showing the configuration of the image diagnostic system 2 related to this disclosure. [Figure 1B] Flowchart showing the procedure for evaluating tissue images [Figure 1C] This figure shows an example of a processing circuit 8 having multiple hardware function blocks. [Figure 2] This figure shows the procedure (1) outlining the process of extracting cell nucleus contour data from pathological images, and an example of the extracted cell nucleus contour (2). [Figure 3] This figure illustrates borderlines 32 and 34 superimposed on a pathological image, along with multiple isometric lines. [Figure 4]Figure 1B is a diagram illustrating the boundary line extraction process in step S20. [Figure 5] This figure shows an example of a method for determining contour lines using multiple grid points. [Figure 6] A diagram showing an example of a process for smoothing contour lines. [Figure 7A] (1) to (4) are diagrams to explain indicators related to the shape of the cell nucleus. [Figure 7B] (1) and (2) are diagrams illustrating indicators related to the distribution of cell nuclei. [Figure 8] Figure 1B is a diagram illustrating the process of step S50. [Figure 9A] This diagram shows the distribution of the relative positions of cell nuclei at position x in a layer, with the basement membrane side set to 0 and the epidermal side to 1. [Figure 9B] A diagram showing the number density ρ of cell nuclei according to four stages of disease progression. [Figure 9C] A diagram showing the distribution of parameter values ​​a and b according to the progression of the disease. [Figure 10A] This diagram illustrates an example of using an index related to the shape of the cell nucleus (aspect ratio AR) and an index related to its distribution (major axis angle). [Figure 10B] (1) is a figure showing an example of calculating the mean angle θ and angular variance V as statistical values, and (2) is a figure showing an example of calculating the orientation order parameter as a statistical value. [Figure 11] A diagram to explain the significance of indicators related to angular distribution. [Figure 12A] A diagram showing the trend of the total vector when the angles of the cell nuclei are relatively well aligned. [Figure 12B] A diagram showing the trend of the total vector when the angles of the cell nuclei are uniformly distributed. [Figure 12C] A diagram showing the trend of the total vector when the angles of the cell nucleus are distributed at two extremes. [Figure 13A] A diagram showing the number density ρ of cell nuclei according to four stages of disease progression. [Figure 13B] This figure shows the distribution of the mean angle θ and angular variance V according to the progression of the disease. [Figure 14] (a) shows the angle αcell along the major axis when the cell shape is approximated as an ellipse, and (b) shows the angle αactin of the cytoskeleton. [Modes for carrying out the invention]

[0020] The embodiments will be described in detail below, with reference to the drawings as appropriate. However, unnecessary details may be omitted. For example, detailed explanations of already well-known matters or redundant explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily verbose and to facilitate understanding by those skilled in the art. The inventors provide the accompanying drawings and the following explanation so that those skilled in the art can fully understand this disclosure, and these do not limit the subject matter described in the claims.

[0021] In this specification, "pathological image" and "histological image" are considered synonymous. Pathological images are typically used to identify abnormalities or lesions in tissues or cells. Histological images, on the other hand, show the structure and condition of specific tissues. Histological images include normal tissue and tissue without lesions, and can be used for purposes other than pathological diagnosis. However, in this specification, histological images will be treated as images used for pathological diagnosis, i.e., pathological images.

[0022] [Embodiment] Figure 1A is a block diagram showing the configuration of the medical imaging system 2 according to this disclosure. The image diagnostic system 2 comprises an evaluation device 4, a storage device 12, and a data server 14. The evaluation device 4 and the data server 14 can communicate with each other via a communication network 16, such as the Internet.

[0023] Evaluation device 4 is a device that evaluates pathological images through a process that will be described in detail later. Evaluation device 4 can be implemented, for example, using a commonly available PC.

[0024] The storage device 12 is a secondary storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores multiple pre-captured pathology images. The storage device 12 may be located within the evaluation device 4, or it may be able to communicate with the evaluation device 4 via the communication network 16.

[0025] The data server 14 receives the statistical values ​​obtained by the calculation processing of the evaluation device 4 (described later) and stores them in a storage device (not shown). The data server 14 can be implemented, for example, using a commonly available server computer.

[0026] The evaluation device 4 comprises an interface device 6, a processing circuit 8, and a memory 10. The components are interconnected via a communication bus so that they can communicate with each other.

[0027] The interface device 6 receives pathology image data from the storage device 12. The interface device 6 is also used when the evaluation device 4 and the data server 14 communicate. For example, the evaluation device 4 can send and receive data to and from the data server 14 via the interface device 6. When the evaluation device 4 receives data from an external source, the interface device 6 may be called the input interface, and when the evaluation device 4 outputs data to an external source, the interface device 6 may be called the output interface.

[0028] An example of the interface device 6 is a wireless interface for wireless communication, an Ethernet terminal for wired communication, or a USB terminal. As the wireless interface, for example, a standard compliant with the Wi-Fi® standard that uses frequencies such as 2.4GHz / 5.2GHz / 5.3GHz / 5.6GHz for wireless communication, and / or a mobile communication system defined as so-called 5G, 4G, etc., may be adopted.

[0029] The processing circuit 8 may be an integrated circuit (IC) chip, such as a central processing unit (CPU) or a digital signal processing processor. The processing circuit 8 may consist of one or more integrated circuits, some of which may include an application-specific integrated circuit (ASIC). The processing circuit 8 can evaluate pathological images by performing the processing shown in Figure 1B, described later, on the pathological images.

[0030] Memory 10 is a general term for semiconductor volatile memory such as RAM and / or semiconductor non-volatile memory such as flash ROM, having multiple memory elements. At least a portion of memory 10 may be a removable recording medium. Memory 10 may store a computer program that controls the operation of the processing circuit 8.

[0031] Figure 1B is a flowchart showing the procedure for evaluating tissue images. First, we will explain the overview of the processing performed by the processing circuit 8 of the evaluation device 4, referring to Figure 1B.

[0032] This flowchart is realized by the processing circuit 8 executing a computer program stored in memory 10. In other words, this flowchart represents the execution procedure of an evaluation program for evaluating the pathological condition of tissue images. When executing the process, the evaluation device 4 reads the necessary tissue image data from the storage device 12 and stores it in the storage device 12.

[0033] In step S10, the processing circuit 8 provides cell nucleus contour data to each cell constituting the tissue in the tissue image.

[0034] In step S20, the processing circuit 8 assigns a first boundary line to one side of the tissue image and a second boundary line to the other side.

[0035] In step S30, the processing circuit 8 divides the tissue image region between the first boundary line and the second boundary line into (n+1) layers by arranging n (where n is an integer greater than or equal to 1) isometric lines. These isometric lines consist of a set of points where the ratio of the distance to the first boundary line to the distance to the second boundary line is equal, and the ratio of the distance to the first boundary line to the distance to the second boundary line at the k-th (where k is an integer greater than or equal to 1 or less than or equal to n) is k:n+1-k.

[0036] In step S40, the processing circuit 8 determines a first index based on the contour data.

[0037] In step S50, the processing circuit 8 sets a region of interest consisting of one or more layers from among the (n+1) layers.

[0038] In step S60, the processing circuit 8 obtains a first statistical value for the first indicator in the region of interest.

[0039] The processing circuit 8 can perform the processes of steps S10 to S60 described above in hardware and / or software. Figure 1C shows an example of a processing circuit 8 having multiple hardware functional blocks. The processing circuit 8 includes a contour determination unit 81, a boundary line setting unit 82, an isometric line setting unit 83, an index calculation unit 84, a region of interest setting unit 85, and a statistical processing unit 86. The contour determination unit 81 performs the process of step S10, the boundary line setting unit 82 performs the process of step S20, and the isometric line setting unit 83 performs the process of step S30. In addition, the index calculation unit 84 performs the process of step S40, the region of interest setting unit 85 performs the process of step S50, and the statistical processing unit 86 performs the process of step S60. Note that some of the processes of steps S10 to S60 may be implemented by software, and the remaining part may be implemented by hardware.

[0040] Next, referring to Figures 2 to 14, the processes described above in steps S10 to S60 will be explained in detail. In the following explanation, each component of the processing circuit 8 shown in Figure 1C will be assumed to perform its respective process.

[0041] Figure 2 shows the procedure (1) outlining the process of extracting cell nucleus contour data from pathological images, and an example of the extracted cell nucleus contour (2).

[0042] First, the contour determination unit 81 divides the loaded pathology image into multiple sub-images. In the example in Figure 2, one pathology image is divided into 16 sub-images of the same size. The reason for this division is that the inventors of the present invention have found that using smaller, divided sub-images rather than a larger image improves the accuracy of extracting cell nucleus contours. However, division is not essential. Depending on the algorithm for contour extraction, it may be possible to achieve accuracy equivalent to or better than that using sub-images even with a single pathology image.

[0043] The contour determination unit 81 sequentially inputs sub-images into a trained machine learning model (hereinafter referred to as the "trained machine learning model") and sequentially acquires sub-contour images from which contours have been extracted. By combining the obtained sub-contour images, contour image data including the contour of the cell nucleus can be obtained for each cell constituting the tissue in the original pathology image. The sub-contour image obtained from the p-th image from the left and the q-th image from the top in the pathology image is placed and combined at the p-th image from the left and the q-th image from the top in the contour image. The rightmost part of Figure 2 shows an example of a single contour image obtained by recombining the images.

[0044] In this embodiment, a pre-trained machine learning model was prepared that takes sub-images as input and sub-contour images as output, and used to extract the contours of cell nuclei within the sub-images. Such a pre-trained machine learning model can typically be constructed by performing machine learning using training data. In this embodiment, the training data consists of an image containing a cell nucleus image object as the explanatory variable, and a contour image enclosing the contour of the cell nucleus image object in that image as the target variable. For example, when one explanatory variable and one target variable are considered as one pair, at least several dozen pairs, and more often several hundred to tens of thousands, of training data are trained using a predetermined machine learning algorithm to construct a pre-trained machine learning model. An example of a machine learning algorithm is a convolutional neural network (CNN). Note that the image size of each image constituting the explanatory variable is the same as the image size of the sub-image, and the image size of each image constituting the target variable is the same as the image size of the sub-contour image. The pre-trained machine learning model can be constructed, for example, using Cellpose, a cell image analysis software that uses a deep learning algorithm. It is also possible to color-code the cell nuclei using Cellpose.

[0045] Figure 2(2) shows an example of an enlarged, extracted cell nucleus outline. It can be seen that the cell nucleus outline has been clearly extracted.

[0046] The trained machine learning model does not need to be built by the user of the evaluation device 4. For example, a third party may build a trained machine learning model using a large amount of training data and provide it commercially. The data server 14 may store such a trained machine learning model. That is, the data server 14 stores such a third-party trained machine learning model in memory or secondary storage (not shown) as a computer program, or a combination of the computer program and parameters obtained through training. The data server 14 may receive data of one or more pathological images from the evaluation device 4, perform the processing shown in Figure 2, and transmit each resulting contour image to the evaluation device 4.

[0047] It should be noted that using the aforementioned pre-trained machine learning model is not essential as long as contours can be extracted. For example, contours can also be extracted using the well-known binarization process, which is one of the image processing techniques. Specifically, noise can be reduced by averaging (smoothing) the image, and then the pixel values ​​can be binarized using a predetermined threshold to separate the cell nucleus objects from the background. By emphasizing the area around each separated cell nucleus object, the contour of each cell nucleus can be extracted.

[0048] Next, we will specifically explain the processes in steps S20 and S30 of Figure 1B.

[0049] Figure 3 illustrates border lines 32 and 34, along with multiple isometric lines, superimposed on a pathological image.

[0050] The boundary setting unit 82 sets a boundary line 32 (first boundary line) on one side of the pathological image, for example, the boundary on the epidermal side. Furthermore, the boundary setting unit 82 sets a boundary line 34 (second boundary line) on the other side of the pathological image, for example, the boundary on the basement membrane side.

[0051] Next, the isometric line setting unit 83 places n isometric lines 36 (where n is an integer of 1 or more) between boundary lines 34 to divide (n+1) layers. In this specification, an isometric line 36 refers to a line or group of lines consisting of a set of points where the ratio of the distance to boundary line 32 to the distance to boundary line 34 is equal.

[0052] The boundary line extraction process in step S20 will now be explained with reference to Figure 4. Note that the inventors of this application implemented the present invention using, for example, pathological images of carcinoma in situ of the cervix, and therefore the drawings also use images of this case for explanation. However, it should be noted that this disclosure is not an art that applies only to images of epithelial tissue. When dealing with tissue in which lesions may exist, instead of a boundary line indicating the epithelial surface and a boundary line indicating the basement membrane, two boundary lines that demarcate the extent of the tissue may be defined. As will be explained later, a person skilled in the art who has access to this disclosure will understand that the art of this disclosure can be applied to any tissue image.

[0053] Various methods can be considered for extracting boundary lines 32 and 34. For example, the boundary line setting unit 82 can extract boundary lines 32 and 34 using so-called edge detection processing. Specifically, the boundary line setting unit 82 determines the position where the change in brightness exceeds a predetermined threshold, based on the pixel values ​​of multiple adjacent pixels.

[0054] First, let's focus on the boundary on the epidermal side. Referring to (1) in Figure 4, there is a difference in brightness between the tissue and the outside of the tissue to a degree that is visible to a person. Similarly, when focusing on the basement membrane side, there is a difference in brightness within the tissue to a degree that is visible to a person. The boundary setting unit 82 can determine each position that clearly distinguishes between the tissue and the outside of the tissue, and by finding a curve connecting each position, it can extract the boundary line 32. The boundary setting unit 82 can also determine each position that clearly distinguishes within the tissue, and by finding a curve connecting each position, it can extract the boundary line 34. In the latter case, it is thought that there may be many positions where the change in brightness changes by more than a predetermined threshold. In such cases, the position of the pixels may be determined by adding the condition that the boundary line 34 is a continuous curve. For example, if the boundary setting unit 82 determines a pixel at a position where the change in brightness changes by more than a predetermined threshold, only the pixels within 5 pixels from that pixel may be evaluated to determine whether the change in brightness changes by more than a predetermined threshold. Through this process, the positions of each part of the basement membrane can be determined, and by finding the curves connecting each position, the boundary line 34 can be extracted. Figure 4(2) shows the extracted boundary lines 32 and 34.

[0055] Next, with reference to Figures 4 and 5, we will explain the process in step S30 of arranging the isometric lines 36 to partition (n+1) layers.

[0056] The contour line setting unit 83 defines the region P to be analyzed by connecting the endpoints of two curves with line segments. In Figure 4 (2), region P is enclosed by boundary line 32, boundary line 34, and two line segments 36a and 36b. Next, the contour line setting unit 83 creates grid points inside the polygon to be analyzed. Figure 4 (3) schematically shows a number of grid points within region P. Note that grid points may be set for the entire tissue image, and processing may be performed only on the grid points within region P in subsequent steps.

[0057] Figure 5 shows an example of a method for determining isometric lines using multiple grid points. Now, let's consider determining n isometric lines 36 between the basement membrane (k=0) and the epidermis (k=(n+1)). The value of n is a user-pre-set integer greater than or equal to 1.

[0058] The isoproportional line setting unit 83 determines the shortest distance (du) from each grid point to the boundary line 32 representing the epidermal surface and the shortest distance (dl) to the boundary line 34 representing the basement membrane. Hereafter, for convenience, this will be referred to as the "shortest distance to the basement membrane (dl)," etc.

[0059] Figure 5(1) schematically shows the set of shortest distances (dl) to the basement membrane. Similarly, Figure 5(2) schematically shows the set of shortest distances (du) to the epidermal surface. In both figures, the X and Y axes represent the position coordinates of each pixel in the image. Although Figures 5(1) and (2) are strictly speaking sets of points, they are shown as curved surfaces for convenience. These sets of points are referred to as "curved surfaces."

[0060] The i-th contour line is determined as follows. The contour line setting unit 83 calculates the set of intersection points between the surface (nk)du(x) and the surface kdl. Figure 5(3) schematically shows how the surface (nk)du(x) and the surface idl intersect. Calculating the set of intersection points is equivalent to finding the points where the difference between (nk)du(x) and kdl is zero.

[0061] The isometric line setting unit 83 projects the calculated intersection points onto the xy-plane. The isometric line setting unit 83 designates the curve obtained by this operation as the k-th isometric line. As is clear from the calculation method described above, the isometric line is a line consisting of a set of points where the ratio of the distance to boundary line 32 to the distance to boundary line 34 is equal.

[0062] The above process is performed sequentially from k=1 to k=n to obtain n isometric lines 36. The ratio of the distance from boundary line 32 to boundary line 34 at the k-th isometric line (where k is an integer between 1 and n) satisfies k:n+1-k. In this way, n isometric lines can be placed in the tissue image region P sandwiched between the two boundary lines 32 and 34, thereby dividing it into (n+1) layers. Note that a "layer" refers to a region defined by two adjacent isometric lines.

[0063] The advantage of partitioning these (n+1) layers is that, as will be discussed later, it allows for consistent evaluation of the position and orientation of cells even in complexly distorted tissue sections.

[0064] As mentioned above, the isometric line setting unit 83 determined the isometric lines using grid points. Since the projected images of the grid points are connected by straight lines, the isometric lines can be described as polylines composed of line segments. Therefore, the polylines of the isometric lines may be reshaped into smooth curves. However, when performing a smoothing process, it is necessary to ensure that the isometric lines do not intersect with each other due to the smoothing process.

[0065] Figure 6 shows an example of the process of smoothing contour lines. The operations described below are shown as a process in which the lines are smoothed in the order of dashed-dot lines, dashed-dot lines, single-dashed lines, and solid lines in Figure 6.

[0066] The contour line setting unit 83 moves the points projected onto each intersection (hereinafter referred to as "projection points") according to the mean curvature flow V = -H for each contour line, so that the polyline becomes smooth.

[0067] Let me explain in more detail. First, the contour line setting unit 83 approximates the curvature H and the normal vector perpendicular to the tangent line at each vertex. Then, the contour line setting unit 83 moves this projection point in the normal direction with velocity V = -H during a sufficiently short time tsize. This operation is repeated until the time reaches Tsmooth. Here, the repeated steps of the calculation are virtually referred to as "time".

[0068] To achieve smoothing, the amount of displacement of the projection point can be sloped. For example, when there are n isometric lines, the Tsmooth value is increased near k=(1 / 2)×n (the center). For example, the Tsmooth value is set to 100% of a predetermined value. On the other hand, the Tsmooth value is decreased near the basement membrane or epidermis, such as k=1 or k=n. For example, the Tsmooth value is set to 10% of a predetermined value. This allows for a slope where the smoothing is stronger the closer the isometric lines are to the center.

[0069] The smoothing process described above is just one example. Smoothing may also be achieved using other processes, such as spline processing.

[0070] Next, the process of step S40 in Figure 1B will be explained with reference to Figures 7A and 7B.

[0071] The index calculation unit 84 calculates a first index based on the contour data. The contour data is obtained by step S10 in Figure 1B.

[0072] Examples of the first indicators include the area of ​​the cell nucleus, the density of the nuclear contour, the roundness of the nuclear contour, the aspect ratio (AR) of the nuclear contour, the major axis angle formed by the major axis of the nuclear contour and the nearest isometric contour, and the number density (ρ) of the cell nuclei. Each of these will be explained below. The indicator calculation unit 84 can determine each indicator by the calculation methods shown.

[0073] The indicators can be broadly classified into two categories: indicators related to the shape of the cell nucleus and indicators related to the distribution of the cell nucleus.

[0074] (A) Indicators related to the shape of the cell nucleus Figure 7A is a diagram illustrating indicators related to the shape of the cell nucleus. (1) to (4) in Figure 7A correspond to the item numbers described below.

[0075] (1) Area of ​​the cell nucleus The area A of a cell nucleus indicates its size. Area A can be determined, for example, by the number of pixels contained within the outline of each cell nucleus.

[0076] (2) Density S of cell nucleus The density S of the cell nucleus indicates the degree of convexity of the cell nucleus's shape. The density S of the cell nucleus can be calculated by dividing the area of ​​the cell nucleus by the area of ​​the cell nucleus's convex hull.

[0077] (3) Roundness of the nuclear contour C The roundness C of the nuclear contour indicates how close the shape of the cell nucleus is to a perfect circle. Roundness C is calculated as C = 4π × (area) / (perimeter). 2 It is determined by [method].

[0078] (4) Aspect ratio of nuclear contour AR The aspect ratio (AR) of the nuclear contour indicates how the shape of the nuclear cell is elongated. The aspect ratio AR is given by AR = b / a (where a is the minor axis length and b is the major axis length).

[0079] (B) Indicators related to the distribution of cell nuclei Figure 7B is a diagram illustrating indicators related to the distribution of cell nuclei. (1) and (2) in Figure 7B correspond to the item numbers described below.

[0080] (1) Long axis angle θ of the cell nucleus To determine the major axis angle θ of the cell nucleus, the isometric lines calculated in step S30 of Figure 1B are also required. For example, suppose the cell nucleus lies on an isometric line. The major axis angle θ is the inclination of the major axis of the cell nucleus with respect to the isometric line, and its range is 0° < θ < 180°. Figure 7B (1) illustrates the major axis angle θ.

[0081] If the cell nucleus is not located on an isometric curve, the nearest isometric curve can be determined, and the slope can be calculated using its normal vector.

[0082] (2) Number density of cell nuclei ρ The number density ρ of cell nuclei indicates how cell nuclei are distributed in a tissue, or in other words, the distribution of the relative positions of cell nuclei in the tissue. The upper part of Figure 7B (2) shows the distribution of the relative positions of cell nuclei at position x in a layer, with the basement membrane side set to 0 and the epidermal side set to 1. The lower part of Figure 7B (2) shows the distribution of the number density ρ ( / pixel) of cell nuclei according to the position x in the layer. Here, the position x in the layer is the cell density of cell nuclei in the h-th layer in the region of interest, normalized by the number of layers (n+1), and is expressed as x = h / (n+1).

[0083] The inventors of this application fitted the number density ρ of cell nuclei to the formula f = exp(ax + b). Therefore, the index calculation unit 84 can calculate the best-fitted a and b based on the number density ρ of cell nuclei corresponding to the position x of the normalized layer (for example, using the least squares method).

[0084] Next, the process of step S50 in Figure 1B will be explained with reference to Figure 8. In step S50, the area of ​​interest setting unit 85 sets an area of ​​interest that includes one or more layers from the (n+1) sections (layers) obtained in step S30.

[0085] Figure 8(1) shows how a number of isometric lines placed between the boundary line 32 representing the epidermal surface and the boundary line 34 representing the basement membrane are divided into three regions of interest RoI-1, RoI-2, and RoI-3, starting from the boundary line 32. The three regions of interest RoI-1, RoI-2, and RoI-3 may be determined to have an equal number of isometric lines based, for example, on a user-pre-set number of regions "3". Alternatively, the three regions of interest may be determined based on user-pre-set positions, for example, positions corresponding to a 3:2:1 ratio from the boundary line 32 side. The user can set any number of regions and positions.

[0086] Figure 8(2) shows the distribution of aspect ratios for the entire region (ALL) and each region of interest. As can be seen, the distribution of aspect ratios differs significantly depending on the location of the region of interest. For example, compared to the overall region, the average aspect ratio in region of interest RoI-1 is approximately 1, and the distribution of aspect ratios is relatively small. In region of interest RoI-2, both the average aspect ratio and the distribution of aspect ratios are slightly larger compared to region of interest RoI-1. On the other hand, in region of interest RoI-3, the average aspect ratio increases to approximately 2, and the distribution of aspect ratios is also very large.

[0087] Based on these results, the significance of the focus area setting unit 85 setting the focus area in step S50 can be understood.

[0088] Next, the process in step S60 of Figure 1B will be explained. In step S60, the statistical processing unit 86 calculates the statistical value (first statistical value) for the first index obtained in step S40 in the region of interest set in step S50. Typical examples of statistical values ​​are the mean, the median of the index in that region of interest, skewness, kurtosis, and / or standard deviation. The following explanation is performed as a process by the statistical processing unit 86.

[0089] The inventors of this application have found that the statistical features of the distribution differ depending on the stage of lesion progression, for each region and for each indicator. In other words, they have found that by determining the statistical features of the distribution for a given region and a given indicator, it is possible to estimate the stage of lesion progression from the tissue image. The following describes the relationship between the various regional and indicator features discovered by the inventors of this application and the stage of lesion progression. The lesion is an intraepithelial neoplasia of the cervix, also known as CIN (Cervical Intraepithelial Neoplasia). CIN1 is considered mild dysplasia, CIN2 is moderate dysplasia, and CIN3 is severe dysplasia / carcinoma in situ. In other words, the disease progresses in order from CIN1, with CIN3 being the most advanced stage.

[0090] Figure 9A shows the distribution of the relative positions of cell nuclei at position x in a layer, with the basement membrane side set to 0 and the epidermal side to 1. We focus on the number density ρ of cell nuclei when they are distributed in this manner.

[0091] Figure 9B shows the number density ρ of cell nuclei according to four stages of disease progression. The four stages are Normal, CIN1, CIN2, and CIN3. Clearly, the shape of the graph of the function fitted to the number density ρ of cell nuclei according to the stage of disease progression is different. In other words, by using a distribution index to find a function that fits the number density ρ of cell nuclei according to the position x of the layer of the number density ρ of cell nuclei, it is possible to estimate the degree of disease in the tissue contained in the cell image. Finding the function here means finding the parameters a and b in f=exp(ax+b), as mentioned by referring to (2) in Figure 7B. Parameters a and b can be called statistical values.

[0092] Figure 9C shows the distribution of parameter values ​​a and b according to the progression of the disease. The left figure shows the distribution of parameter a values, and the right figure shows the distribution of parameter b values. In both figures, the horizontal axis represents the progression of the disease. The inventors of this application have found that as the disease progresses, parameter a increases and parameter b decreases. The arrows in each graph in Figure 9C represent this trend.

[0093] One reason why this trend was successfully extracted is that the position x between the epidermal surface and the basement membrane is normalized using a ratio (isoproportional curve) rather than using the position between the epidermal surface and the basement membrane directly, for example, the length measured from the epidermal surface. It is strongly presumed that this positional normalization is effective not only for cervical carcinoma in situ but also for other tumors.

[0094] By utilizing positional normalization, differences corresponding to the progression of the disease can be similarly identified for other indicators besides the number density ρ of the cell nuclei mentioned above. Below, we will explain an example of calculating an indicator to evaluate the orientational order of cell nuclei using the indicators described above.

[0095] Figure 10A shows an example of using an index related to the shape of the cell nucleus (aspect ratio AR) and an index related to its distribution (major axis angle). In Figure 10A, θ j This represents the major axis angle between the long axis of the j-th cell and the nearest isometric line (solid line at the bottom of Figure 10A), and AR j This represents the aspect ratio of the j-th cell. An example of calculating two types of statistical values ​​using the major axis angle θ and aspect ratio AR obtained in this way will be explained.

[0096] Figure 10B (1) shows an example of calculating the mean angle θ and angular variance V as statistical values. The mean angle θ and angular variance V are defined by the following formulas. Re 2iθ = <e 2iθj > (0°≦θ<180°) (Formula 1) Angular dispersion V=1-R (0≦V≦1) (Formula 2)

[0097] θ in Equation 1 j This represents the angle that the long axis of the j-th cell nucleus makes with its nearest isometric contour. The symbol <> indicates that the average is taken over N cell nuclei. Note that j is an integer between 1 and N, inclusive. 2iθ The technical significance of this will be explained later with reference to Figures 11 to 13B.

[0098] Furthermore, Figure 10B (2) shows an example of calculating the orientation order parameter as a statistical value. Here, two statistical values ​​S and S AR This explains the statistic S, which takes into account the weighting based on the aspect ratio. AR Then, weights based on aspect ratio are considered. Statistical values ​​S and S AR It is defined by the following formula. S <cos(2θ j )>...(Formula 3) S AR = <AR j · cos(2θ) j )> / <AR j >...(Formula 4)

[0099] Here too, θ j AR represents the angle between the long axis of the j-th cell nucleus and its nearest isometric contour. The symbol <> indicates that the average is taken over N cell nuclei. Note that j is an integer between 1 and N, inclusive. j This represents the aspect ratio of the j-th cell nucleus.

[0100] The above pairs of (θ,V) and S or S AR Both can be used as statistical values ​​to evaluate the orientational order of cell nuclei. The former can be used when calculating statistics related to angles. The latter can be used to evaluate the alignment of cell nuclei and the orientation of the cytoskeleton.

[0101] Figure 11 is a diagram illustrating the significance of the index related to angular distribution.

[0102] Now, let's assume there are N cell nuclei in the image. The statistical processing unit 86 calculates the angle θ of the nuclei. j N vectors of length 1 / N are generated, each pointing in the direction (j=1,2,···,N). Then the statistical processing unit 86 sums the N vectors. This generates a single vector (hereinafter referred to as the "sum vector"). Let R be the length of the sum vector, and let θ be the angle it makes with the contour lines. The length R and the mean direction θ are given by the Re in equation 1 above. 2iθ It is used in [location / area].

[0103] Figures 12A to 12C illustrate the relationship between the length R and angle θ of the total vector and the angular variance V.

[0104] Figure 12A shows the trend of the total vector when the angles of the cell nuclei are relatively well aligned. The length R of the total vector approaches 1, and as a result, the angular variance V approaches 0 according to Equation 2.

[0105] Figure 12C shows the trend of the sum vector when the angles of the cell nucleus are at both extremes, that is, when the long axis of the cell nucleus is distributed at an angle nearly parallel to the isometric lines. The length R of the sum vector approaches 0, and as a result, the angular variance V approaches 1 according to Equation 2.

[0106] Figure 12B shows the trend of the sum vector when the angles of the cell nuclei are uniformly distributed. The length R of the sum vector takes an intermediate value.

[0107] The inventors of this application have found that changes occur in the mean angle θ and angular variance V as the disease progresses. In other words, they have found that the mean angle θ and angular variance V are useful for evaluating the degree of disease progression.

[0108] Figure 13A shows the number density ρ of cell nuclei according to four stages of disease progression. Similar to the example shown in Figure 9B, the four stages are Normal, CIN1, CIN2, and CIN3.

[0109] Normal cell nuclei exhibit the tendency shown in Figure 12C. As the disease progresses, this gradually changes to the tendency shown in Figure 12B. In CIN3, the most advanced stage of the disease, the average angle θ is the steepest, as shown in Figure 12A.

[0110] Figure 13B shows the distribution of mean angle θ and angular variance V values ​​according to the progression of the disease. The left figure shows the distribution of values ​​for mean angle θ, and the right figure shows the distribution of values ​​for angular variance V. In both figures, the horizontal axis represents the progression of the disease. The mean angle θ is the value of |θ-90°|. It can be seen that the trends in the distribution of mean angle θ and angular variance V differ depending on the progression of the disease. In other words, mean angle θ and angular variance V are effective statistical values ​​for evaluating the degree of disease progression. By determining the trends in the distribution of mean angle θ and / or angular variance V, it is possible to estimate the degree of disease in the tissue contained in the cell image.

[0111] Next, we will explain an example of calculating the orientation order parameter as a statistical value.

[0112] Figure 14(a) shows the angle αcell in the direction of the major axis when the cell shape is approximated by an ellipse, and (b) shows the angle αactin of the cytoskeleton. In this example, a microscopic image of cells adhering to a substrate with unidirectional "wrinkles" on the surface corresponding to the vertical stripes in the image, and a color map of the direction of the cytoskeleton inside them are created, and the cell shape with respect to the wrinkle direction is approximated by an ellipse. Both the angle αcell in the direction of the major axis and the angle αactin of the cytoskeleton are defined as angles with respect to the wrinkle direction. These angles αcell and αactin of the cytoskeleton are used as θ in equations 3 and 4. j By substituting into this, the statistical value S and the statistical value S AR It is possible to find this.

[0113] In the above description of the series of processes, the focus area setting unit 85 was described assuming that one focus area is set. However, depending on the user input, the focus area setting unit 85 can set multiple focus areas. In that case, the statistical processing unit 86 can calculate statistical values ​​for each focus area. The statistical values ​​for each focus area do not all need to be the same; statistical values ​​may be calculated independently for each focus area.

[0114] The calculated statistical values ​​are presented to pathologists and pathology technologists as evaluation results, for example, via a display device such as a screen. These evaluation results can be used as supporting information when pathologists and pathology technologists themselves perform evaluations. This can reduce the burden on medical professionals. Alternatively, pathologists and pathology technologists can verify their own evaluation results by comparing them with the output of the statistical values ​​from the evaluation device. This allows them to improve their skills. Furthermore, the calculated statistical values ​​can be used, for example, to evaluate whether a patient's prognosis is deteriorating.

[0115] The technology described herein allows for the evaluation of tissue images using quantitative indicators and statistical values. It enables robust evaluation that is independent of the color tone, staining intensity, and the pathologist's experience of the tissue image. [Explanation of symbols]

[0116] 2. Diagnostic Imaging Systems 4. Evaluation device 6 Interface device 8 Processing Circuit 10 memory 12 Storage device 14 Data Servers 81 Contour Definition Section 82 Boundary Setting Section 83 Equal proportion line setting section 84 Indicator calculation section 85 Area of ​​Interest Setting Unit 86 Statistical Processing Section

Claims

1. A method for evaluating tissue images, The tissue image includes the step of providing cell nucleus contour data to each cell constituting the tissue, The steps include: assigning a first boundary line to one side of the tissue image and a second boundary line to the other side; A step of partitioning (n+1) layers in a region of tissue image sandwiched between the first boundary line and the second boundary line by arranging n (where n is an integer of 1 or more) isometric lines, wherein the isometric lines are lines consisting of a set of points where the ratio of the distance to the first boundary line to the distance to the second boundary line is equal, and the ratio of the distance to the first boundary line to the distance to the second boundary line at the k-th (where k is an integer of 1 or more and n or less) is k:n+1-k, A step of determining a first index based on the contour data, The steps include setting a region of interest consisting of one or more layers from the (n+1) layers, The steps include: obtaining a first statistical value for the first indicator in the area of ​​interest; A method for evaluating tissue images, including those containing tissue images.

2. The aforementioned tissue image is an image of epithelial tissue, The first boundary line is the line indicating the epidermal surface of the epithelium. The method for evaluating a tissue image according to claim 1, wherein the second boundary line is a line indicating the basement membrane of the epithelium.

3. The first indicator mentioned above is, Area of ​​the cell nucleus, Density of the nuclear contour, Roundness of the nuclear contour, Aspect ratio AR of the nuclear contour, The number density of cell nuclei ρ, or The major axis angle between the major axis of the nuclear contour and the nearest isometric line to the nuclear contour. The method for evaluating tissue images according to claim 1.

4. The first statistical value described above is Average value, Median, skewness, Kurtosis, or, Standard deviation, The method for evaluating tissue images according to claim 1.

5. The first indicator is the number density ρ of cell nuclei, The first statistical values ​​are a and b obtained by fitting the number density of cell nuclei ρ(x), which is normalized using a quantity x (where x = h / (n+1)) obtained by normalizing the cell density of cell nuclei in the h-th layer in the region of interest by the number of layers n+1, to the formula f(x) = exp(ax+b). The method for evaluating tissue images according to claim 3.

6. The evaluation method according to claim 3, wherein the first indicator is the major axis angle, and the first statistical value is the average value θ of the major axis angle in the region of interest, or the variance V obtained by the following formula, where N is the number of cell nuclei included in the region of interest. Formula V = 1-R (Here R is the formula Re ２ｉθ = < e ２ｉθｊ > is calculated, where j is an integer between 1 and N, and θ ｊ The angle between the long axis of the j-th cell nucleus and its nearest isometric curve is represented by the angle between the long axis of the j-th cell nucleus and the angle between the isometric curve and the angle between the j-th nucleus and the angle between the long axis of the j-th cell nucleus.

7. The first index is the major axis angle, and the first statistical value is the orientation order parameter S, which is obtained by the following formula, where N is the number of cell nuclei included in the region of interest: Equation S = <cos2θ ｊ > (Here, j represents an integer between 1 and N, and θ ｊ The angle between the long axis of the j-th cell nucleus and its nearest isometric curve is shown, and the angle between the two nuclei is shown as the average across the j nuclei. Alternatively, the orientation order parameter S is weighted by the aspect ratio obtained by the following formula. ＡＲ : Formula S ＡＲ = <AR ｊ ·cos(2θ ｊ )> / <AR ｊ > (Here, AR ｊ (where is the aspect ratio of the j-th cell nucleus) The evaluation method according to claim 3.

8. A method for evaluating tissue images according to claim 1, further comprising the steps of setting a region of interest other than the aforementioned region of interest, and then obtaining a second statistical value in the other region of interest.

9. A device for evaluating tissue images, In the aforementioned tissue image, a contour determination unit provides contour data of the cell nucleus to each cell constituting the tissue, A boundary setting unit that assigns a first boundary line to one side of the tissue image and a second boundary line to the other side, An isometric line setting unit that divides (n+1) layers by arranging n (where n is an integer of 1 or more) isometric lines in a region of tissue image sandwiched between the first boundary line and the second boundary line, wherein the isometric lines are lines consisting of a set of points where the ratio of the distance to the first boundary line to the distance to the second boundary line is equal, and the ratio of the distance to the first boundary line to the distance to the second boundary line at the k-th isometric line (where k is an integer of 1 or more and n or less) from the first boundary line satisfies k:n+1-k, A first indicator calculation unit that determines a first indicator based on the contour data, A focus area setting unit sets a focus area consisting of one or more layers from the (n+1) layers, A first statistical processing unit that obtains a first statistical value for the first indicator in the area of ​​interest, and A tissue image evaluation device having the following features.

10. A program for evaluating tissue images, The tissue image includes the step of providing cell nucleus contour data to each cell constituting the tissue, The steps include: assigning a first boundary line to one side of the tissue image and a second boundary line to the other side; A step of partitioning (n+1) layers in a region of tissue image sandwiched between the first boundary line and the second boundary line by arranging n (where n is an integer of 1 or more) isometric lines, wherein the isometric lines are lines consisting of a set of points where the ratio of the distance to the first boundary line to the distance to the second boundary line is equal, and the ratio of the distance to the first boundary line to the distance to the second boundary line at the k-th (where k is an integer of 1 or more and n or less) is k:n+1-k, A step of determining a first index based on the contour data, The steps include setting a region of interest consisting of one or more layers from the (n+1) layers, The steps include: obtaining a first statistical value for the first indicator in the area of ​​interest; A program for evaluating tissue images that uses a computer's processing circuit to execute the necessary steps.

Images