Image processing method, image processing device, and image processing program

Abstract

This invention relates to the image processing method that makes it possible to quantify an intracellular biological substance, to specify a cell in which the biological substance is expressed, and to specify the type of the cell using a small number of pigment types and a simple process. In this method, input are a cell form image showing, in a tissue section, the form of a specific cell stained by a pigment that stains the specific cell and a fluorescence image showing a specific biological substance as a fluorescent bright spot within the same range of the tissue section as the cell form image. Position information for the specific cell is extracted from the cell form image. Whether the specific biological substance is expressed in the specific cell is determined from a step of extracting position information for the specific biological substance from the fluorescence image, the position information for the specific cell, and the position information for the specific biological substance.

Description

[Technical Field]

[0001] The present invention relates to an image processing method, an image processing device, and an image processing program. [Background technology]

[0002] In recent years, with the expansion of treatments using molecularly targeted drugs, primarily antibody drugs, there has been an increasing need for accurate diagnostic methods to enable more effective use of molecularly targeted drugs. Specifically, there is a demand for the quantification of biological substances in tissue sections, the identification of cells expressing biological substances, and the identification of the type of said cells.

[0003] In general, in pathological diagnosis, tissue sections are stained and observed under a microscope, and a diagnosis is made based on morphological information, such as the size and shape of stained cell nuclei and changes in the tissue pattern, as well as staining information. Known staining methods include hematoxylin staining and DAB staining. Another known method for confirming the presence of biological substances within cells is to bind a fluorescently labeled substance (e.g., an antibody) that specifically binds to the biological substance to perform fluorescence observation. Images of tissue sections obtained using these methods are processed to quantify the biological substance and identify specific cell types.

[0004] For example, Patent Document 1 describes a method for evaluating the expression level of a biological substance in a cell membrane. In this method, a tissue section is stained with a fluorophore (a) that stains the cell membrane and a fluorophore (b) that binds to the biological substance and has a different emission wavelength peak from that of the fluorophore (a). The position of the stained cell membrane is then identified, and the number of bright spots and fluorescence intensity of the fluorophore (b) on the identified cell membrane are measured.

[0005] Patent Document 2 describes a cancer diagnosis method using a multi-staining method. In this method, the cell membranes of cancer cells in a pathological specimen are stained with DAB, allowing the cancer cells to be observed under visible light. A fluorescent marker is also added to the chromosomes of the cells, allowing the chromosomes to be observed under fluorescent light, and then the chromosomes are stained with New Fuchsin, allowing the chromosomes to be observed under visible light. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-57631 [Patent Document 2] Japanese Patent Application Laid-Open No. 2004-157053 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the method described in Patent Document 1 can quantify a biological substance and identify cells in which the biological substance is expressed, but cannot identify the type of cell in which the biological substance is expressed.Furthermore, the method described in Patent Document 2 can distinguish between cancer cells and other cells, but cannot quantify the biological substance expressed in cancer cells.

[0008] To quantify a biological substance expressed in a specific cell, it is necessary to quantify the biological substance, identify the cell in which the biological substance is expressed, and identify the type of cell, but the methods described in Patent Document 1 and Patent Document 2 can only perform one or two of these steps. Furthermore, when the methods described in Patent Document 1 and Patent Document 2 are combined, multiple types of dyes are used, which complicates the process and may adversely affect fluorescence observation.

[0009] The present invention has been made in consideration of the above circumstances, and aims to provide an image processing method, an image processing device, and an image processing program that can quantify biological substances within cells, identify cells in which the biological substances are expressed, and identify the type of the cells in question in a simple process using a small number of dyes. [Means for solving the problem]

[0010] An image processing method according to one embodiment of the present invention for solving the above problem includes the steps of inputting a cell morphology image showing the morphology of a specific cell stained with a dye that stains the specific cell in a tissue section, and a fluorescent image showing a specific biological substance as a fluorescent spot in the same area of ​​the tissue section as the cell morphology image; extracting positional information of the specific cell from the cell morphology image; extracting positional information of the specific biological substance from the fluorescent image; and determining whether the specific biological substance is expressed in the specific cell based on the positional information of the specific cell and the positional information of the specific biological substance.

[0011] In addition, an image processing device according to one embodiment of the present invention for solving the above-mentioned problem includes an input unit that inputs a cell morphology image showing the morphology of a specific cell stained with a dye that stains the specific cell in a tissue section, and a fluorescent image showing a specific biological substance as a fluorescent spot in the same area of ​​the tissue section as the cell morphology image, a cell extraction unit that extracts positional information of the specific cell from the cell morphology image, a biological substance extraction unit that extracts positional information of the specific biological substance from the fluorescent image, and a biological substance expression determination unit that determines whether the specific biological substance is expressed in the specific cell based on the positional information of the specific cell and the positional information of the specific biological substance.

[0012] In addition, an image processing program relating to one embodiment of the present invention for solving the above problem causes a computer to execute the steps of inputting a cell morphology image showing the morphology of a specific cell stained with a dye that stains the specific cell in a tissue section, and a fluorescent image showing a specific biological substance as a fluorescent spot in the same area of ​​the tissue section as the cell morphology image, extracting positional information of the specific cell from the cell morphology image, extracting positional information of the specific biological substance from the fluorescent image, and determining whether the specific biological substance is expressed from the specific cell based on the positional information of the specific cell and the positional information of the specific biological substance. [Effects of the Invention]

[0013] The present invention provides an image processing method, an image processing device, and an image processing program that can quantify intracellular biological substances, identify cells in which the biological substances are expressed, and identify the type of said cells in a simple process using a small number of dyes. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a flowchart of an image processing method according to the first embodiment. [Figure 2] 2A to 2E are diagrams schematically showing images obtained in each step in the image processing method according to the second embodiment. [Figure 3] FIG. 3 is a flowchart showing details of a step of extracting location information of a specific cell according to the first embodiment. [Figure 4] FIG. 4 is a flowchart showing details of a step of extracting position information of a specific biological substance according to the first embodiment. [Figure 5] FIG. 5 is a block diagram schematically showing the functional configuration of the image processing device according to the first embodiment. [Figure 6] FIG. 6 is a block diagram schematically illustrating a functional configuration of a control unit in the image processing device according to the first embodiment. [Figure 7] FIG. 7 is a flowchart of an image processing method according to the second embodiment. [Figure 8] 8A to 8F are diagrams schematically showing images obtained in each step in the image processing method according to the second embodiment. [Figure 9] FIG. 9 is a flowchart showing details of a step of extracting position information of a cell nucleus according to the second embodiment. [Figure 10] FIG. 10 is a block diagram schematically illustrating a functional configuration of an image processing device according to the second embodiment. [Figure 11] FIG. 11 is a block diagram schematically illustrating a functional configuration of a control unit of an image processing device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present invention will be described in detail, but the present invention is not limited to the following embodiments.

[0016] 1. First Embodiment 1-1. Image processing method FIG. 1 is a flowchart of an image processing method according to the first embodiment.

[0017] As shown in FIG. 1, an image processing method according to one embodiment of the present invention includes the steps of (1) inputting a cell morphology image and a fluorescence image (step S110), (2) extracting positional information of a specific cell from the cell morphology image (step S120), (3) extracting positional information of a specific biological substance from the fluorescence image (step S130), and (4) determining whether the specific biological substance is expressed in the specific cell (step S140). Step (2) may be performed at any time after the cell morphology image is input in step (1). Step (3) may also be performed at any time after the fluorescence image is input in step (1). For example, step (3) may be performed before step (2). Step (4) is performed after steps (2) and (3).

[0018] 2A to 2E are diagrams schematically illustrating images obtained in each step in the image processing method according to the present embodiment. FIG. 2A shows a cell morphology image obtained in step S110. FIG. 2B shows an image showing the position information of a specific cell extracted in step S120. FIG. 2C is a fluorescence image obtained in step S110. FIG. 2D shows an image showing the position information of a specific biological material extracted in step S130. FIG. 2E shows an image obtained in step S140 by superimposing an image showing the position information of a specific cell and an image showing the position information of a specific biological material. Each step will be described below.

[0019] 1-1-1. Step of inputting cell morphology images and fluorescence images (step S110) In this step, a cell morphology image showing the morphology of a specific cell stained with a dye that stains the specific cell in a tissue section and a fluorescent image showing a specific biological material as a fluorescent spot in the same area of ​​the tissue section as the cell morphology image are input. For example, a tissue section may be prepared, specific cells may be stained with a predetermined dye, and specific biological material may be stained with a fluorescent dye, and the cell morphology image and fluorescent image may be captured. Alternatively, the cell morphology image and fluorescent image captured in advance may be input via a storage medium, a communication line, etc.

[0020] (1) Cell morphology images In this embodiment, a cell morphology image is an image obtained by staining specific cells with a dye in a tissue section and capturing an image of the region representing the specific cells ( FIG. 2A ). It is preferable that at least one of the cytoplasm and cell membrane of the specific cells be stained. The method for staining specific cells with a dye is not particularly limited, but an example is immunohistochemical staining using an enzyme-linked immunosorbent assay. The combination of enzyme and substrate used in the enzyme-linked immunosorbent assay is not particularly limited and can be appropriately selected from known combinations of enzymes and substrates. Examples of combinations of enzymes and chromogenic substrates that label antibodies include a combination of peroxidase and diaminobenzidine (DAB) (DAB staining), a combination of peroxidase and aminoethylcarbazole (AEC) (AEC staining), a combination of alkaline phosphatase and Fast Red, a combination of alkaline phosphatase and Fast Blue, and a combination of alkaline phosphatase and bromochloroindolyl phosphate (BCIP). For example, by labeling an antibody with peroxidase that directly or indirectly binds to a marker protein expressed in a specific type of cell, and then binding this antibody to the marker protein in a tissue section, and then providing diaminobenzidine (DAB) around the bound antibody, it is possible to stain only specific types of cells brown. By changing the antibody depending on the type of cell you want to stain, you can change the type of cell you are staining.

[0021] An example of the procedure for DAB staining of paraffin sections is shown below. In this example, staining is performed after deparaffinization and activation.

[0022] (Deparaffinization) The paraffin sections are immersed in a container containing xylene to remove the paraffin. The temperature is not particularly limited, but room temperature is acceptable. The immersion time is preferably 3 minutes to 30 minutes. If necessary, the xylene may be replaced during the immersion period.

[0023] Next, the tissue sections from which the paraffin has been removed are immersed in a container containing ethanol to replace the xylene with ethanol. The temperature is not particularly limited, but room temperature is suitable. The immersion time is preferably 3 minutes to 30 minutes. If necessary, the ethanol may be replaced during the immersion.

[0024] Next, the tissue section is immersed in a container containing water to replace the ethanol with water. The temperature is not particularly limited, but room temperature is suitable. The immersion time is preferably 3 minutes to 30 minutes. If necessary, the water may be replaced during the immersion.

[0025] (Activation treatment) Activation is a process that exposes biological substances (antigens to which antibodies bind) in tissue. Depending on the type of biological substance, activation may not be necessary. Activation conditions are not particularly limited and are selected appropriately depending on the type of biological substance (antigen). For example, activation solutions such as 0.01 M citrate buffer (pH 6.0) 1 mM, EDTA solution (pH 8.0), 5% urea, and 0.1 M Tris-HCl buffer can be used. Heating equipment such as an autoclave, microwave, pressure cooker, or water bath can be used. The heating temperature should be between 50°C and 130°C, and the heating time should be between 5 and 30 minutes.

[0026] Next, the activated tissue sections are immersed in a container containing PBS (phosphate buffered saline) and washed. The temperature is not particularly limited, but room temperature is acceptable. The immersion time is preferably 3 minutes to 30 minutes. If necessary, the PBS may be replaced during the immersion period.

[0027] Prior to staining, it is preferable to drop a known blocking agent, such as PBS containing BSA (bovine serum albumin), onto the tissue section.

[0028] (staining) A liquid containing a primary antibody is added dropwise and left to stand for a predetermined period of time. The primary antibody is an antibody that specifically binds to a marker protein that is specifically expressed in the cells to be stained. The tissue section is then washed. Next, a liquid containing a secondary antibody is added dropwise to the tissue to which the primary antibody has been bound and left to stand for a predetermined period of time. The secondary antibody is an antibody that specifically binds to the primary antibody and is labeled with peroxidase. The tissue section is then washed. Next, DAB staining solution is added dropwise to the tissue to which the secondary antibody has been bound to stain it.

[0029] (Acquisition of cell morphology images) The cell morphology image can be obtained by taking a photograph using a known optical microscope equipped with a camera. The field of view of the cell morphology image obtained by taking a photograph is 3 mm. 2 It is preferable that it is 30 mm or more. 2 More preferably, it is 300 mm or more. 2 It is more preferable that the upper limit of the field of view of the cell morphology image is not particularly limited, but is, for example, the size of a tissue section. The captured image is input by transmitting it to an image analysis device such as a computer.

[0030] (2) Fluorescence image In this embodiment, a fluorescence image is an image obtained by staining (labeling) a specific biological substance in a tissue section with a fluorescent substance and capturing the fluorescent spots of the fluorescent substance ( FIG. 2C ). The fluorescent spots that appear in the fluorescence image indicate the presence of the specific biological substance. The method for staining (labeling) the specific biological substance with a fluorescent substance is not particularly limited, but an example is immunohistochemical staining using a fluorescent antibody method. For example, an antibody that directly or indirectly binds to the specific biological substance is labeled with a fluorescent substance, and the antibody is then bound to the biological substance in the tissue section. By irradiating the tissue with excitation light, fluorescence can be emitted only from the area where the specific biological substance is present. The type of biological substance to be stained can be changed by changing the antibody depending on the type of biological substance to be fluorescently stained (labeled). The specific biological substance is not particularly limited and can be selected appropriately depending on the purpose, but examples include PD-1 and Her2.

[0031] (fluorescent material) The type of fluorescent substance is not particularly limited. For example, the fluorescent substance may be a fluorescent organic dye or quantum dots (semiconductor particles). The antibody may be labeled with fluorescent substance-encapsulated nanoparticles containing multiple fluorescent organic dyes or multiple quantum dots. The fluorescent substance preferably emits fluorescence with a wavelength of 400 to 1100 nm when excited by light with a wavelength of 200 to 700 nm.

[0032] Examples of fluorescent organic dyes include fluorescein-based dye molecules, rhodamine-based dye molecules, AlexaFluor (Invitrogen)-based dye molecules, BODIPY (Invitrogen)-based dye molecules, cascade-based dye molecules, coumarin-based dye molecules, eosin-based dye molecules, NBD-based dye molecules, pyrene-based dye molecules, Texas Red-based dye molecules, and cyanine-based dye molecules. Specific examples of fluorescent organic dyes include 5-carboxy-fluorescein, 6-carboxy-fluorescein, 5,6-dicarboxy-fluorescein, 6-carboxy-2',4,4',5',7,7'-hexachlorofluorescein, 6-carboxy-2',4,7,7'-tetrachlorofluorescein, 6-carboxy-4',5'-dichloro-2',7'-dimethoxyfluorescein, naphthofluorescein, 5-carboxy-rhodamine, 6-carboxy-rhodamine, 5,6-dicarboxy-rhodamine, rhodamine 6G, tetramethylrhodamine, X-rhodamine, AlexaFluor 350, AlexaFluor 405, AlexaFluor 430, AlexaFluor 488, AlexaFluor 500, AlexaFluor 514, AlexaFluor 516, AlexaFluor 518, AlexaFluor 519, AlexaFluor 520, AlexaFluor 521, AlexaFluor 522, AlexaFluor 523, AlexaFluor 524, AlexaFluor 525, AlexaFluor 526, AlexaFluor 527, AlexaFluor 528, AlexaFluor 529, AlexaFluor 530, AlexaFluor 531, AlexaFluor 532, AlexaFluor 533, AlexaFluor 534, AlexaFluor 535, AlexaFluor 536, AlexaFluor 537, AlexaFluor 538, AlexaFluor 539, AlexaFluor 540, AlexaFluor 541, AlexaFluor 542, AlexaFluor 543, AlexaFluor 544, AlexaFluor 545, AlexaFluor 546, AlexaFluor 547, AlexaFluor 548, AlexaFluor 549, AlexaFluor 549, AlexaFluor 549, AlexaFluor Examples of fluorescent organic dyes include uor532, AlexaFluor546, AlexaFluor555, AlexaFluor568, AlexaFluor594, AlexaFluor610, AlexaFluor633, AlexaFluor635, AlexaFluor647, AlexaFluor660, AlexaFluor680, AlexaFluor700, AlexaFluor750, BODIPYFL, BODIPYTMR, BODIPY493 / 503, BODIPY530 / 550, BODIPY558 / 568, BODIPY564 / 570, BODIPY576 / 589, BODIPY581 / 591, BODIPY630 / 650, BODIPY650 / 665, methoxycoumarin, eosin, NBD, pyrene, Cy5, Cy5.5, and Cy7. These fluorescent organic dyes may be used alone or in combination.

[0033] The quantum dots may be, for example, quantum dots containing II-VI compounds, quantum dots containing III-V compounds, or quantum dots containing IV elements as components. Examples of semiconductors that constitute quantum dots include CdSe, CdS, CdTe, ZnSe, ZnS, ZnTe, InP, InN, InAs, InGaP, GaP, GaAs, Si, and Ge. These quantum dots may also be used alone or in combination.

[0034] Quantum dots having the above quantum dots as a core and a shell formed thereon can also be used. In the following, quantum dots having a shell will be referred to as CdSe / ZnS when the core is CdSe and the shell is ZnS. Examples of quantum dots with a core-shell structure include CdSe / ZnS, CdS / ZnS, InP / ZnS, InGaP / ZnS, Si / SiO2, Si / ZnS, Ge / GeO2, and Ge / ZnS.

[0035] The quantum dots may be surface-treated with an organic polymer, etc. Examples of such quantum dots include CdSe / ZnS (manufactured by Invitrogen) having a surface carboxy group, and CdSe / ZnS (manufactured by Invitrogen) having a surface amino group.

[0036] The method for binding a fluorescent substance to an antibody is not particularly limited and can be appropriately selected from known methods.

[0037] (Fluorescent substance-encapsulated nanoparticles) Fluorescent substance-encapsulated nanoparticles are nanoparticles in which multiple fluorescent substances are dispersed. The fluorescent substances may or may not be chemically bonded to the nanoparticles themselves. The material that constitutes the nanoparticles is not particularly limited, and examples include silica, polystyrene, polylactic acid, and melamine.

[0038] Fluorescent substance-encapsulated nanoparticles can be prepared by known methods. For example, silica nanoparticles encapsulating fluorescent organic dyes can be synthesized by following the synthesis of FITC-encapsulated silica particles described in Langmuir, Vol. 8, p. 2921 (1992). Various fluorescent organic dye-encapsulated silica nanoparticles can be synthesized by substituting the desired fluorescent organic dye for FITC. Quantum dot-encapsulated silica nanoparticles can be synthesized by following the synthesis of CdTe-encapsulated silica nanoparticles described in The New Journal of Chemistry, Vol. 33, p. 561 (2009). Fluorescent organic dye-encapsulated polystyrene nanoparticles can be prepared by the copolymerization method using an organic dye with a polymerizable functional group described in U.S. Patent 4,326,008 or the impregnation method of fluorescent organic dyes into polystyrene nanoparticles described in U.S. Patent 5,326,692. Quantum dot-encapsulated polymer nanoparticles can be prepared by the impregnation method of quantum dots into polystyrene nanoparticles described in Nature Biotechnology, Vol. 19, p. 631 (2001).

[0039] The average particle size of the fluorescent substance-encapsulated nanoparticles is not particularly limited, but is, for example, 30 to 800 nm. The coefficient of variation (= (standard deviation / average value) × 100%), which indicates the variation in particle size, is not particularly limited, but is preferably 20% or less. The average particle size is determined by taking electron micrographs using a scanning electron microscope (SEM) to measure the cross-sectional areas of a sufficient number of particles, and calculating the diameter of the circle when each measurement value is taken as the area of ​​the circle. In this embodiment, the average particle size is defined as the arithmetic mean of the particle sizes of 1,000 particles. The coefficient of variation is also a value calculated from the particle size distribution of 1,000 particles.

[0040] The method for binding fluorescent substance-encapsulated nanoparticles to antibodies is not particularly limited and can be appropriately selected from known methods.

[0041] An example of the procedure for fluorescent immunostaining of paraffin sections is shown below. In this example, fluorescent immunostaining is performed after deparaffinization and activation.

[0042] (Deparaffinization and activation treatment) The same procedures as for deparaffinization and activation treatment described in the section on cell morphology imaging are carried out. Before staining, it is preferable to add a drop of a known blocking agent, such as PBS containing BSA (bovine serum albumin), to the tissue section.

[0043] (staining) A liquid containing a primary antibody is dripped onto the tissue and allowed to stand for a predetermined period of time. The primary antibody is an antibody that specifically binds to the biological material to be stained. The tissue section is then washed. Next, a liquid containing a secondary antibody is dripped onto the tissue to which the primary antibody has been bound and allowed to stand for a predetermined period of time. The secondary antibody is an antibody that specifically binds to the primary antibody and is labeled with the above-mentioned fluorescent substance or fluorescent substance-encapsulated nanoparticles. The tissue section is then washed.

[0044] After the staining for cell morphology imaging and the staining for fluorescence imaging are completed, the tissue section is mounted, for example, by dropping a commercially available mounting medium onto the stained tissue section and placing a cover glass on it.

[0045] (Fluorescence image acquisition) Fluorescence images can be obtained by taking images using a known camera-equipped fluorescence microscope. When taking images, an excitation light source and optical filters corresponding to the absorption maximum wavelength and fluorescence wavelength of the fluorescent substance used are used. The field of view of the fluorescence image is the same as that of the cell morphology image.

[0046] 1-1-2. Step of extracting location information of specific cells (step S120) In this step, position information of a specific cell is extracted from the cell morphology image input in step S110 using image processing software or the like. Figure 2B is a diagram schematically showing an image showing the extracted position information of the specific cell. Figure 3 is a flowchart showing an example of this step.

[0047] 3, this process includes a step of converting the image to grayscale (step S121), a step of binarizing the image (step S122), a step of noise processing (step S124), and a step of labeling (step S125). Note that this process may further include a step of morphology processing (step S123).

[0048] In the step of converting the image to grayscale (step S121), the cell morphology image is converted to grayscale. This is because color images (RGB) have variations in brightness for each section. Known conditions for grayscale conversion can be used as appropriate.

[0049] In the step of binarizing the image (step S122), the cell morphology image converted to grayscale is binarized. Specifically, the grayscale image is subjected to threshold processing using a predetermined threshold value to binarize the value of each pixel. The image to be binarized may be a likelihood image created from the cell morphology image using a machine learning method such as deep learning.

[0050] In the noise processing step (step S124), objects other than the target object are identified as noise and removed. That is, this is a step of removing parts other than the specific cells. Specifically, the number of pixels of each object is measured, and objects with a pixel count below a specified value are considered noise and converted to the same color as the background color.

[0051] In the labeling process (step S125), each cell is labeled with a number in order to identify objects in the image.

[0052] If the image contains cells with fragmented cell membranes, it is preferable to perform morphology processing after the step of binarizing the image (step S122). In the morphology processing step (step S123), cells with fragmented cell membranes are filled in to make the cell membranes continuous.

[0053] 1-1-3. Step of extracting location information of specific biological material (step S130) In this step, position information of a specific biological substance is extracted from the fluorescence image input in step S110 using image processing software or the like. Fig. 2D is a diagram schematically showing an image showing the extracted position information of the specific biological substance. Fig. 4 is a flowchart showing an example of this step according to embodiment 1.

[0054] As shown in FIG. 4, this process includes a step of extracting color components according to the wavelength of the fluorescent spot (step S131) ​​and a step of binarizing (step S132).

[0055] In the step of extracting color components according to the wavelength of fluorescent bright spots (step S131), for example, if the emission wavelength of the fluorescent substance in the fluorescent image is 550 nm, only fluorescent bright spots having that wavelength component are extracted as an image.

[0056] In the binarization step (step S132), threshold processing is performed on the fluorescence image after color component extraction to create a binary image. Specifically, the integrated value of the fluorescence intensity of each fluorescent spot is calculated, and based on the normalized fluorescence intensity, fluorescent spots with fluorescence intensity below an arbitrarily set threshold are removed to create a binary image.

[0057] Before the threshold processing in step S132, noise removal processing for removing autofluorescence from cells and other unnecessary signal components may be performed.

[0058] 1-1-4. Step of determining biological material (S140) In this step, it is determined whether or not the specific biological substance is expressed from the specific cell, based on the location information of the specific cell and the location information of the specific biological substance extracted in step S120 and step S130, respectively.

[0059] When determining whether a specific biological substance is expressed in a specific cell, it is preferable to superimpose an image from which the positional information of the specific cell has been extracted and an image from which the positional information of the specific biological substance has been extracted. Figure 2E is a diagram schematically showing an image obtained by superimposing an image from which the positional information of the specific cell has been extracted and an image from which the positional information of the specific biological substance has been extracted.

[0060] When determining whether a specific biological substance is expressed in a specific cell, for example, it may be determined based on whether a fluorescent spot indicating the expression of the specific biological substance is located within a region representing the specific cell. The fluorescent spot may be located on the edge of the region representing the specific cell, or within the region. In this case, it is more preferable to use the stained region in the cell morphology image as a region of interest (ROI). In this determination method, if the fluorescent spot is located within the region, it is determined that the specific biological substance is expressed in the specific cell, and if the fluorescent spot is not located within the region, it is determined that the specific biological substance is not expressed in the specific cell.

[0061] When quantifying the number of specific biological substances expressed in specific cells, the number of specific biological substances determined to be expressed in specific cells in this step can be calculated; when quantifying the number of specific biological substances not expressed in specific cells, the number of specific biological substances determined to be not expressed in specific cells can be calculated.

[0062] 1-1-5. Application examples (Quantitation of PD-1, a protein expressed on killer T cells) An example of quantifying the expression level of the protein PD-1 expressed in killer T cells using the image processing method described above will be described.

[0063] First, the tissue sections are deparaffinized and activated as described above. Next, a primary reaction with an anti-PD-1 antibody (primary antibody) and a secondary reaction with a fluorescently labeled secondary antibody are performed to label PD-1 with the fluorescent substance. Furthermore, a primary reaction with an anti-CD8 antibody (primary antibody) that specifically binds to the CD8 protein expressed on killer T cells and a secondary reaction with a peroxidase-labeled secondary antibody are performed. Finally, diaminobenzidine (DAB) is added to develop color and stain the killer T cells. After these staining steps are completed, the tissue sections are mounted.

[0064] The brown-stained killer T cells are photographed using an optical microscope to obtain cell morphology images, and the fluorescence from the fluorescent substance indirectly bound to PD-1 is photographed using a fluorescence microscope to obtain fluorescence images.

[0065] The obtained cell morphology image is converted to grayscale, and then subjected to binarization, noise reduction, and labeling processes to extract the positional information of the killer T cells (obtaining an extracted image of the killer T cells).

[0066] From the obtained fluorescent image, color components corresponding to the wavelength of the fluorescent spot are extracted and binarized to extract the position information of PD-1.

[0067] The stained region (killer T cells) of the tissue section was designated as the ROI, and the cell morphology image and the fluorescence image were superimposed. The fluorescent puncta present in the stained region were identified as PD-1 expressed on killer T cells. The number of PD-1 puncta thus identified was then measured using image processing software.

[0068] By the above procedures, it is possible to quantify PD-1, identify cells expressing PD-1, and identify the type of those cells.

[0069] (Quantitation of Her2 protein expressed in tumor areas) An example of quantifying the expression level of the protein Her2 expressed in a tumor region using the image processing method described above will be described.

[0070] First, the tissue sections are deparaffinized and activated as described above. Next, a primary reaction with an anti-Her2 antibody (primary antibody) and a secondary reaction with a fluorescently labeled secondary antibody are performed to label Her2 with the fluorescent substance. Furthermore, a primary reaction with an anti-cytokeratin antibody (primary antibody) that specifically binds to the protein cytokeratin expressed in tumor areas and a secondary reaction with a peroxidase-labeled secondary antibody are performed. Finally, diaminobenzidine (DAB) is added to develop color and stain killer T cells. After these staining steps are completed, the tissue sections are mounted.

[0071] The brown-stained tumor area is photographed using an optical microscope to obtain cell morphology images, and fluorescence from the fluorescent substance indirectly bound to Her2 is photographed using a fluorescence microscope to obtain fluorescence images.

[0072] The obtained cell morphology image is converted to grayscale, and then subjected to binarization, noise reduction, and labeling processes to extract position information of the tumor region (obtaining an extracted image of the tumor region).

[0073] In the obtained fluorescent image, color components corresponding to the wavelength of the fluorescent spot are extracted and binarized to extract the position information of Her2.

[0074] The stained region (tumor region) of the tissue section was designated as the ROI, and the cell morphology image and the fluorescent image were superimposed. The fluorescent puncta present in the stained region were determined to be Her2 expressed in the tumor region. The number of Her2 puncta determined above was then measured using image processing software.

[0075] By the above procedures, it is possible to quantify Her2, identify cells expressing Her2, and identify the type of said cells.

[0076] 1-2. Image processing device An image processing device according to a first embodiment of the present invention that can be used when implementing the above image processing method will be described below. Note that the present invention is not limited to the following embodiments.

[0077] The image processing device according to the first embodiment of the present invention includes an input unit for inputting a cell morphology image showing the morphology of a specific cell stained with a dye that stains the specific cell in a tissue section, and a fluorescent image showing a specific biological substance as a fluorescent spot in the same area of ​​the tissue section as the cell morphology image; a cell extraction unit for extracting positional information of the specific cell from the cell morphology image; a biological substance extraction unit for extracting positional information of the specific biological substance from the fluorescent image; and a biological substance expression determination unit for determining whether the specific biological substance is expressed in the specific cell based on the positional information of the specific cell and the positional information of the specific biological substance.

[0078] 5 is a block diagram showing a schematic functional configuration of the image processing device 100. The image processing device 100 has an input unit 10, a control unit 20, an operation unit 30, a display unit 40, and a storage unit 50. The image processing device 100 analyzes cell morphology images and fluorescence images transmitted from or input to an external device, and determines whether a specific biological substance is expressed in a specific cell.

[0079] The input unit 10 inputs the cell morphology images and fluorescence images described above. The input unit 10 is, for example, a fluorescence microscope (which can also function as an optical microscope) equipped with a camera. The input unit 10 may also input image information sent from an external device (for example, the aforementioned fluorescence microscope) or image information stored in a storage medium.

[0080] Fig. 6 is a block diagram showing a schematic functional configuration of the control unit 20. The control unit 20 has a CPU (Central Processing Unit), RAM (Random Access Memory), etc., and executes various processes in cooperation with various programs stored in the storage unit 50, thereby providing overall control over the operation of the image processing device 100. As shown in Fig. 5, the control unit 20 also has a cell extraction unit 21, a biological substance extraction unit 22, and a biological substance expression determination unit 23. The control unit 20 operates the cell extraction unit 21, the biological substance extraction unit 22, and the biological substance expression determination unit 23 in cooperation with the image processing program stored in the storage unit 50, thereby causing the cell extraction unit 21, the biological substance extraction unit 22, and the biological substance expression determination unit 23 to function and perform image analysis processing.

[0081] The cell extraction unit 21 has a function of extracting position information of a specific cell from the cell morphology image input by the input unit 10. The cell extraction unit 21 converts the cell morphology image to grayscale, and performs binarization processing, noise processing, and labeling processing to extract position information of the specific cell. Note that the cell extraction unit 21 may also perform morphology processing on the cell morphology image after binarizing it to extract position information of the specific cell.

[0082] The biological substance extraction unit 22 has a function of extracting position information of specific biological substances from the fluorescent image input by the input unit 10. The biological substance extraction unit extracts color components corresponding to the wavelengths of fluorescent spots from the fluorescent image, performs binarization processing, and extracts position information of the biological substances. Note that before performing binarization processing, the biological substance extraction unit 22 may perform noise removal processing such as cellular autofluorescence and other unnecessary signal components, and then extract the position information of the biological substances.

[0083] The biological substance expression determination unit 23 has the function of determining whether a specific biological substance is expressed from a specific cell based on the location information of the specific cell extracted by the cell extraction unit 21 and the location information of the specific biological substance extracted by the biological substance extraction unit 22.

[0084] The control unit 20 may further include a calculation unit 24. The calculation unit 24 has a function of calculating the number of specific biological substances determined by the biological substance expression determination unit 23 to be expressed in specific cells, or the number of specific biological substances determined not to be expressed in specific cells.

[0085] The operation unit 30 has, for example, a keyboard including character input keys, number input keys, various function keys, etc., and a pointing device such as a mouse, and outputs a press signal of a key pressed on the keyboard and an operation signal from the pointing device as input signals to the control unit 20.

[0086] The display unit 40 is configured to include a monitor such as a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display), and displays various screens according to instructions of a display signal input from the control unit 20.

[0087] The image processing device 100 may be configured to include a LAN adapter, a router, and the like, and to be connected to external devices via a communication network such as a LAN.

[0088] The storage unit 50 is configured, for example, by an HDD (Hard Disk Drive), a semiconductor nonvolatile memory, etc. The storage unit 50 stores various programs and various data as described above.

[0089] 1-3.Image processing program An image processing program according to the first embodiment of the present invention, which can be used when performing image analysis processing in the image processing device, will be described. Note that the present invention is not limited to the following embodiments.

[0090] The image processing program according to the first embodiment of the present invention causes a computer to execute the steps of inputting a cell morphology image showing the morphology of a specific cell stained with a dye that stains the specific cell in a tissue section, and a fluorescent image showing a specific biological substance as a fluorescent spot in the same area of ​​the tissue section as the cell morphology image; extracting positional information of the specific cell from the cell morphology image; extracting positional information of the specific biological substance from the fluorescent image; and determining whether the specific biological substance is expressed in the specific cell based on the positional information of the specific cell and the positional information of the specific biological substance.

[0091] The image processing program cooperates with the control unit 20 of the image processing device to cause the cell extraction unit 21, the biological substance extraction unit 22, and the biological substance expression determination unit 23 to function and execute image processing.

[0092] (effect) The image processing method, image processing device, and image processing program according to embodiment 1 can identify cells in which a specific biological substance is expressed, quantify the expression level of the specific biological substance, and further identify the type of cell in which the specific biological substance is expressed.

[0093] 2. Second Embodiment 2-1. Image processing method In addition to the steps of the image processing method of embodiment 1, the image processing method of embodiment 2 further includes the steps of extracting positional information of a cell nucleus from a cell morphology image, determining whether a cell nucleus is the cell nucleus of a specific cell based on the positional information of the cell nucleus and the positional information of the specific cell, and identifying the cell nucleus to which a specific biological substance belongs based on the positional information of the cell nucleus determined to be the cell nucleus of a specific cell and the positional information of a biological substance determined to be expressed in the specific cell.

[0094] FIG. 7 is a flowchart of an image processing method according to the second embodiment.

[0095] As shown in Figure 7, the image processing method of embodiment 2 includes: (1) a step of inputting a cell morphology image and a fluorescence image (step S110); (2) a step of extracting position information of a specific cell from the cell morphology image (step S120); (A) a step of extracting position information of a cell nucleus from the cell morphology image (step S210); (3) a step of extracting position information of a specific biological substance from the fluorescence image (step S130); (4) a step of determining whether the specific biological substance is expressed from the specific cell (step S140); (B) a step of determining whether the cell nucleus is the cell nucleus of the specific cell from the position information of the specific cell (step S220); and (C) a step of identifying the cell nucleus to which the specific biological substance belongs (step S230).

[0096] The image processing method according to the second embodiment differs from the first embodiment in that it further includes the steps of (A) extracting position information of a cell nucleus from the cell morphology image (step S210), (B) determining whether the cell nucleus is the cell nucleus of the specific cell based on the position information of the specific cell (step S220), and (C) identifying the cell nucleus to which the specific biological material belongs (step S230). The same components as those in the image processing method according to the first embodiment are denoted by the same reference numerals and will not be described.

[0097] 8A to 8E are diagrams schematically showing images obtained in each step in the image processing method according to the present embodiment. FIG. 8A shows a cell morphology image obtained in step S110. FIG. 8B shows an image showing position information of a specific cell extracted in step S120. FIG. 8C shows an image showing position information of a cell nucleus extracted in step S210. FIG. 8D shows a fluorescence image obtained in step S110. FIG. 8E shows an image showing position information of a specific biological material extracted in step S130. FIG. 8F shows an image obtained in step S140 by superimposing an image showing position information of a specific cell and an image showing position information of a specific biological material. Each step will be described below.

[0098] 2-1-1. Step of extracting position information of cell nuclei (step S210) In this step, position information of the cell nucleus is extracted from the cell morphology image input in step S110. Fig. 9 is a flowchart showing details of this step according to the second embodiment.

[0099] In the second embodiment, the cell morphology image further shows the cell nuclei of the cells stained with a dye. The dye used to stain the cell nuclei of the cells is not particularly limited, but is, for example, hematoxylin. Staining with hematoxylin is performed after deparaffinization. For example, a tissue section from which paraffin has been removed is immersed in a staining solution containing hematoxylin, and the tissue section is stained with hematoxylin. The tissue section stained with hematoxylin is then immersed in running water to release the color.

[0100] This process includes a step of binarizing the image (step S211), a step of noise processing (step S212), and a step of labeling (step S213).

[0101] In the step of binarizing the image (step S211), the cell morphology image may be color-separated according to the color used to stain the cell nuclei before being binarized, or a likelihood image of the cell nuclei may be created from the cell morphology image using a machine learning technique such as deep learning before being binarized.

[0102] Steps S212 and S213 are similar to steps S124 and S125 in the first embodiment, respectively, and therefore detailed description thereof will be omitted. Position information of the cell nucleus is extracted by the processing of steps S211 to S213.

[0103] This step (step S210) may be performed between the step of inputting cell morphology images and fluorescence images (step S110) and the step of determining cell nuclei (step S220), and may be performed before or after the step of extracting location information of specific cells (step S120) and the step of extracting location information of specific biological materials (step S130).

[0104] 2-1-2. Step of determining cell nuclei (step S220) In this step, it is determined whether or not the cell nucleus is that of a specific cell based on the position information of the specific cell extracted in step S210.

[0105] A method for determining whether a cell nucleus is the cell nucleus of a specific cell simply involves determining whether the cell nucleus is located within an area representing the specific cell. Whether a cell nucleus is located within an area representing the specific cell may be determined by whether the ratio of the contour length of the cell nucleus within the area representing the specific cell to the contour length of the cell nucleus is higher than an arbitrarily set threshold. Alternatively, whether the ratio of the area of ​​the cell nucleus within the area representing the specific cell to the area of ​​the cell nucleus is higher than an arbitrarily set threshold may be determined. If the cell nucleus is located within an area representing the specific cell by this determination method, the cell nucleus is determined to be the cell nucleus of the specific cell.

[0106] This step may be performed after the step of extracting the position information of the cell nucleus (step S210), and may be performed before the determination of the biological material (step S140).

[0107] 2-1-3. Step of identifying cell nuclei (step S230) In this step, of the cell nuclei determined to be cell nuclei of a specific cell in step S220, a cell nucleus to which a specific biological substance determined to be expressed from the specific cell in step S140 belongs is identified.

[0108] The method for identifying the cell nucleus to which a specific biological substance belongs is to identify the cell nucleus closest to a fluorescent spot indicating the specific biological substance. Specifically, within the region indicating the specific cell, the cell nucleus closest in distance to the center of a cell nucleus determined to be the cell nucleus of the specific cell and the center of a fluorescent spot indicating the specific biological substance is identified as the cell nucleus to which the specific biological substance belongs. Note that a fluorescent spot indicating the specific biological substance that exists outside the region indicating the specific cell is identified as belonging to the closest cell nucleus among the cell nuclei that were not determined to be the cell nucleus of the specific cell.

[0109] When quantifying the number of specific biological substances expressed in the cell nuclei of specific cells, it is sufficient to calculate the number of specific biological substances identified as belonging to cell nuclei determined to be cell nuclei of specific cells.When quantifying the number of specific biological substances not expressed in the cell nuclei of specific cells, it is sufficient to calculate the number of specific biological substances identified as belonging to cell nuclei not determined to be cell nuclei of specific cells.

[0110] 2-1-4. Application examples (Identifying the source of PD-1, a protein expressed on killer T cells) An example of quantifying the expression level of the protein PD-1 expressed in killer T cells using the image processing method described above will be described.

[0111] First, the tissue sections are deparaffinized and activated as described above. Next, a primary reaction with an anti-PD-1 antibody (primary antibody) and a secondary reaction with a fluorescently labeled secondary antibody are performed to label PD-1 with the fluorescent substance. A primary reaction with an anti-CD8 antibody (primary antibody) that specifically binds to the CD8 protein expressed on killer T cells and a secondary reaction with a peroxidase-labeled secondary antibody are also performed. Diaminobenzidine (DAB) is then added to develop color and stain the killer T cells. The nuclei of each cell are then stained with hematoxylin. After these staining steps, the tissue sections are mounted.

[0112] An optical microscope is used to capture images of the killer T cells stained brown and the cell nuclei stained blue-purple, and a fluorescence microscope is used to capture images of the fluorescence emitted from the fluorescent substance indirectly bound to PD-1, and a fluorescence image is obtained.

[0113] The obtained cell morphology image is converted to grayscale, and then subjected to binarization, noise reduction, and labeling processes to extract the positional information of the killer T cells (obtaining an extracted image of the killer T cells).

[0114] In addition, using deep learning, a likelihood image of the cell nucleus is created from the obtained cell morphology image, and then binarization, noise processing, and labeling processing are performed to extract the position information of the cell nucleus (obtaining an extracted image of the cell nucleus).

[0115] From the obtained fluorescent image, color components corresponding to the wavelength of the fluorescent spot are extracted and binarized to extract the position information of PD-1.

[0116] The stained area (killer T cells) of the tissue section is designated as the ROI, and the cell morphology image and the fluorescent image are superimposed. Fluorescent spots present within the stained area are determined to be PD-1 expressed on killer T cells.

[0117] The cell nuclei stained with hematoxylin located within the stained region (killer T cells) are determined to be the cell nuclei of killer T cells.

[0118] Within the stained region (killer T cell), the distance between the center of the nucleus of the determined killer T cell and the center of the fluorescent spot indicating PD-1 determined to be expressed in the killer T cell is measured, and the nucleus closest to the fluorescent spot is identified as the nucleus to which PD-1 belongs. Fluorescent spots indicating PD-1 that exist outside the stained region (killer T cell) are identified as belonging to the nucleus closest in center-to-center to a nucleus not determined to be a killer T cell nucleus.

[0119] Using image processing software, the number of PD-1 expressed on killer T cells and the number of nuclei of the determined killer T cells are measured. The number of PD-1 expressed on the determined killer T cells is divided by the number of nuclei of the determined killer T cells to calculate the number of PD-1 per killer T cell nucleus.

[0120] The above procedures make it possible to quantify PD-1, identify the cells and cell nuclei expressing PD-1, and identify the type of cells.

[0121] 2-2.Image processing device Fig. 10 is a block diagram showing a schematic functional configuration of an image processing device 200 according to embodiment 2, and Fig. 11 is a block diagram showing a schematic functional configuration of a control unit 20 of the image processing device according to embodiment 2. The image processing device 200 according to embodiment 2 differs from the image processing device 100 according to embodiment 1 in that the control unit 20 further includes a cell nucleus extraction unit 25, a cell nucleus determination unit 26, and a cell nucleus identification unit 27. The same components as those in the image processing device 100 according to embodiment 1 are denoted by the same reference numerals, and their description will be omitted.

[0122] The cell nucleus extraction unit 25 has a function of extracting position information of cell nuclei from the cell morphology image input by the input unit 10. The cell nucleus extraction unit 25 performs binarization processing, noise processing, and labeling processing on the cell morphology image to extract position information of cell nuclei. The binarization processing may be performed after color separation of the cell morphology image according to the color used to stain the cell nuclei, or after creating a likelihood image of the cell nuclei from the cell morphology image using a machine learning method such as deep learning.

[0123] The cell nucleus determination unit 26 has a function of determining whether the cell nucleus extracted by the cell nucleus extraction unit 25 is the cell nucleus of a specific cell, based on the position information of the specific cell extracted by the cell extraction unit 21.

[0124] The cell nucleus identification unit 27 has the function of identifying the cell nucleus to which a specific biological substance belongs based on the positional information of the cell nucleus determined by the cell nucleus determination unit 26 to be the cell nucleus of a specific cell and the positional information of a specific biological substance determined by the biological substance expression determination unit 23 to be expressed from a specific cell.

[0125] 2-3.Image processing program The image processing program according to the second embodiment differs from the first embodiment in that it further executes the steps of: extracting position information of a cell nucleus from a cell morphology image; determining whether the cell nucleus is the specific cell nucleus based on the position information of the cell nucleus and the position information of the specific cell; and identifying the cell nucleus to which the specific biological substance belongs based on the position information of the cell nucleus determined to be the cell nucleus of the specific cell and the position information of the biological substance determined to be expressed in the specific cell. The same components as those in the image processing program according to the first embodiment are denoted by the same reference numerals, and their description will be omitted.

[0126] The image processing program cooperates with the control unit 20 of the image processing device to cause the cell extraction unit 21, the biological substance extraction unit 22, the biological substance expression determination unit 23, the cell nucleus extraction unit 25, the cell nucleus determination unit 26, and the cell nucleus identification unit 27 to function and perform image processing.

[0127] (effect) In addition to the effects of the image processing method, image processing device, and image processing program of embodiment 2, the image processing method, image processing device, and image processing program of embodiment 2 can identify cell nuclei in which specific biological substances are expressed.

[0128] This application claims priority from Japanese Patent Application No. 2020-154817, filed September 15, 2020. The entire contents of the specification and drawings of that application are incorporated herein by reference. [Industrial Applicability]

[0129] According to the present invention, an image processing method can be provided that can quantify intracellular biological substances, identify cells in which the biological substances are expressed, and identify the type of the cells in question in a simple process using a small number of dyes. For example, the present invention is useful in pathological diagnosis. [Explanation of symbols]

[0130] 1. Area showing specific cells 2 cell nucleus 3 Cells other than specific cells 4. Fluorescent spots that indicate specific biological materials 5. Noise 10 Input section 20 Control Unit 21 Cell extraction section 22 Biological material extraction section 23 Biological substance expression determination unit 24 Calculation section 25 Cell nucleus extraction section 26 Cell nucleus determination section 27 Cell nucleus identification part 30 Control section 40 Display section 50 Storage section S110: Inputting cell morphology images and fluorescence images S120: Extracting location information of specific cells S130 Extracting specific biological materials S140 Determining biological material S210: Extracting the position information of the cell nucleus S220: Identifying cell nuclei S230: Identifying cell nuclei

Claims

1. inputting a cell morphology image showing the morphology of specific cells stained with a dye that stains only the specific cells in a tissue section, and a fluorescent image showing specific biological materials as fluorescent bright spots in the same area of ​​the tissue section as the cell morphology image; extracting position information of the specific cell from the cell morphology image; extracting position information of the specific biological material from the fluorescence image; determining whether the specific biological substance is expressed from the specific cell based on the location information of the specific cell and the location information of the specific biological substance; An image processing method comprising: the cell morphology image further shows the cell nuclei of the cells; The image processing method includes: extracting position information of the cell nuclei from the cell morphology image; determining whether the cell nucleus is the cell nucleus of the specific cell based on the position information of the cell nucleus and the position information of the specific cell; and identifying a cell nucleus to which the specific biological substance belongs based on location information of the cell nucleus determined to be the cell nucleus of the specific cell and location information of the biological substance determined to be expressed in the specific cell, In the step of identifying the cell nucleus to which the specific biological substance belongs, it is determined that the specific biological substance belongs to a cell nucleus that is closest to a fluorescent spot indicating the specific biological substance, among the cell nuclei determined to be the cell nuclei of the specific cell. Image processing methods.

2. The image processing method according to claim 1 , wherein the dye stains at least one of the cytoplasm and the cell membrane of the specific cell.

3. 3. The image processing method according to claim 1, wherein the step of determining whether the specific biological substance is expressed from the specific cell determines whether the fluorescent spot is located within an area representing the specific cell.

4. The image processing method according to claim 1 , wherein the step of determining whether the cell nucleus is the cell nucleus of the specific cell determines whether the cell nucleus is located within a region representing the specific cell.

5. The image processing method according to any one of claims 1 to 4, wherein the specific cells are stained by immunohistochemical staining targeting a marker protein expressed in the specific cells.

6. an input unit for inputting a cell morphology image showing the morphology of a specific cell stained with a dye that stains only the specific cell in a tissue section, and a fluorescent image showing a specific biological material as a fluorescent spot in the same area of ​​the tissue section as the cell morphology image; a cell extraction unit that extracts position information of the specific cell from the cell morphology image; a biological substance extraction unit that extracts position information of the specific biological substance from the fluorescent image; a biological substance expression determination unit that determines whether the specific biological substance is expressed from the specific cell based on the location information of the specific cell and the location information of the specific biological substance; An image processing device comprising: the cell morphology image further shows the cell nuclei of the cells; The image processing device includes: a cell nucleus extraction unit that extracts position information of the cell nucleus from the cell morphology image; a cell nucleus determination unit that determines whether the cell nucleus is the cell nucleus of the specific cell based on the position information of the cell nucleus and the position information of the specific cell; a cell nucleus identifying unit that identifies a cell nucleus to which the specific biological substance belongs based on positional information of the cell nucleus determined to be the cell nucleus of the specific cell and positional information of the biological substance determined to be expressed in the specific cell, the cell nucleus identifying unit determines that the specific biological substance belongs to a cell nucleus that is closest to a fluorescent spot indicating the specific biological substance, among the cell nuclei determined to be the cell nuclei of the specific cell; Image processing device.

7. On the computer, inputting a cell morphology image showing the morphology of specific cells stained with a dye that stains only the specific cells in a tissue section, and a fluorescent image showing specific biological materials as fluorescent bright spots in the same area of ​​the tissue section as the cell morphology image; extracting position information of the specific cell from the cell morphology image; extracting position information of the specific biological material from the fluorescence image; determining whether the specific biological substance is expressed from the specific cell based on the location information of the specific cell and the location information of the specific biological substance; Execute An image processing program, the cell morphology image further shows the cell nuclei of the cells; The image processing program extracting position information of the cell nuclei from the cell morphology image; determining whether the cell nucleus is the cell nucleus of the specific cell based on the position information of the cell nucleus and the position information of the specific cell; and further executing a step of identifying a cell nucleus to which the specific biological substance belongs based on positional information of the cell nucleus determined to be the cell nucleus of the specific cell and positional information of the biological substance determined to be expressed in the specific cell, In the step of identifying the cell nucleus to which the specific biological substance belongs, it is determined that the specific biological substance belongs to a cell nucleus that is closest to a fluorescent spot indicating the specific biological substance, among the cell nuclei determined to be the cell nuclei of the specific cell. Image processing program.

Images