Information processing device and information processing method

Abstract

To improve analysis accuracy in fluorescence observation.SOLUTION: An information processing device comprises a processing unit that generates an image based on a pixel value of each pixel in a cell image observed from cells labeled with one or more fluorescent dyes and a threshold value for the pixel value of each pixel.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] In recent years, with the development of cancer immunotherapy and the like, there has been an increasing need to detect and evaluate molecules that are phenotypes of immune cells classified into a number of subsets using multiple markers. A multi-color imaging analysis method using a fluorescent dye as a marker is considered to be an effective means for leading to the relationship between their localization and function.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, when fluorescently observing a stained tissue section stained with a fluorescent dye antibody, the accuracy of measuring the fluorescence intensity reflecting the amount of the antibody may be affected due to autofluorescence or the like. Even when the autofluorescence component is removed from the captured image of the stained tissue section by spectral color separation, it has been impossible to completely eliminate the autofluorescence component remaining in the stained fluorescence image obtained by this color separation.

[0005] Thus, when an autofluorescence component remains in the stained fluorescence image, problems such as an obstacle to quantifying the amount of the dye antibody distributed in the analysis target region in the stained fluorescence image occur, which has hindered the improvement of the analysis accuracy in fluorescence observation.

[0006] Therefore, the present disclosure has been made in view of the above circumstances, and proposes an information processing apparatus, an information processing method, a program, a microscope system, and an analysis system that enable improvement of the analysis accuracy in fluorescence observation. [Means for solving the problem]

[0007] An information processing apparatus according to an embodiment of the present disclosure comprises a processing unit that generates an image based on the pixel value of each pixel in a cell image observed from cells labeled with one or more fluorescent dyes and a threshold value for each pixel. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example configuration of an information processing system according to the first embodiment. [Figure 2] This figure shows a specific example of a fluorescence spectrum acquired by the fluorescence signal acquisition unit. [Figure 3] This diagram illustrates the overview of non-negative matrix factorization. [Figure 4] This is a diagram illustrating the basics of clustering. [Figure 5] This block diagram shows an example of the configuration of a microscope system when the information processing system according to the first embodiment is implemented as a microscope system. [Figure 6] This is a flowchart showing an example of the basic operation of the information processing system according to the first embodiment. [Figure 7] This flowchart shows an example of the operation when generating a pseudo-autofluorescence component image according to the first embodiment. [Figure 8] This figure shows an example of an autofluorescence component image generated in step S103 of Figure 6. [Figure 9] This figure shows an example of an autofluorescence reference spectrum included in the sample information obtained in step S102 of Figure 6. [Figure 10] This is a diagram illustrating step S112 in Figure 7. [Figure 11] This is a diagram illustrating step S114 in Figure 7. [Figure 12] This is a flowchart illustrating the extraction process of the analysis target area and its analysis in a comparative example of the first embodiment. [Figure 13] It is a schematic diagram for explaining the extraction process of the analysis target area and its analysis according to the comparative example shown in FIG. 12. [Figure 14] It is a flowchart for explaining the extraction process of the analysis target area and its analysis according to the first embodiment. [Figure 15] It is a schematic diagram for explaining the extraction process of the analysis target area and its analysis according to the present embodiment shown in FIG. 14. [Figure 16] It is a flowchart for explaining the extraction process of the analysis target area and its analysis according to the second embodiment. [Figure 17] It is a schematic diagram for explaining the extraction process of the analysis target area and its analysis according to the present embodiment shown in FIG. 16. [Figure 18] It is a flowchart for explaining the generation process of the spectrum intensity ratio image and its analysis according to the third embodiment. [Figure 19] It is a diagram for explaining the first generation method according to the third embodiment. [Figure 20] It is a diagram for explaining the second generation method according to the third embodiment. [Figure 21] It is a flowchart for explaining the generation process of the fluorescence component image using machine learning according to the fourth embodiment. [Figure 22] It is a schematic diagram for explaining the generation process of the fluorescence component image using machine learning according to the present embodiment shown in FIG. 21. [Figure 23] It is a diagram showing an example of the measurement system of the information processing system according to the embodiment. [Figure 24] It is a diagram for explaining a method of calculating the number of fluorescent molecules (or the number of antibodies) in one pixel according to the embodiment. [Figure 25] It is a block diagram showing an example of the hardware configuration of the information processing apparatus according to each embodiment and modification example.

Embodiments for Carrying Out the Invention

[0009] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0010] The explanation will be given in the following order. 0. Introduction 1. First Embodiment 1.1. Example Configuration 1.2. Examples of applications to microscope systems 1.3. About the least squares method 1.4. Non-negative matrix factorization (NMF) 1.5. Characteristic Configuration of the First Embodiment 1.5.1. Basic Operation Examples 1.5.2. Example of Autofluorescence Component Corrected Image Generation Flow 1.6. Extraction and Analysis of the Target Region 1.7. Action and Effects 2. Second Embodiment 2.1. Extraction and Analysis of the Target Region 2.2. Action and Effects 3. Third Embodiment 3.1 Generation and Analysis of Spectral Intensity Ratio Images 3.2. Method for generating spectral intensity images 3.2.1 First generation method 3.2.2 Second Generation Method 3.3. Action and Effects 4. Fourth Embodiment 4.1 Generation of fluorescence component images using machine learning 4.2. Action and Effects 5. Example of a measurement system configuration 6. Method for calculating the number of fluorescent molecules (or antibodies) 7. Hardware Configuration Example

[0011] <0. Introduction> In recent years, with the development of cancer immunotherapy and other therapies, there has been a growing need to detect and evaluate the molecules that represent the phenotypes of immune cells, which are classified into numerous subsets, using multiple types of markers. Multicolor imaging analysis methods using fluorescent dyes as markers are considered an effective means of deriving the relationship between their localization and function, and are employed in the following embodiments.

[0012] The following embodiments relate to image processing after performing color separation on a multichannel image (number of pixels × wavelength channels (CH)) obtained by imaging immunohistochemically stained pathological specimen sections with excitation light of multiple wavelengths and fluorescence separation using an imaging device.

[0013] In this disclosure, multichannel images may include various images with a data cube structure composed of image data from multiple wavelength channels (but not limited to a single wavelength channel), such as stained specimen images, fluorescence component images, fluorescence component corrected images, autofluorescence component images, and autofluorescence component corrected images, as described later. Therefore, fluorescence component images, fluorescence component corrected images, autofluorescence component images, and autofluorescence component corrected images are not limited to image data of a single wavelength channel of fluorescence or autofluorescence, but may be spectral images composed of fluorescence or autofluorescence components from multiple wavelength channels. Furthermore, in this description, fluorescence component and autofluorescence component refer to the fluorescence or autofluorescence wavelength components in a multichannel image obtained by imaging with an imaging device.

[0014] As described above, when observing stained tissue sections stained with fluorescent dye antibodies using fluorescence observation, the accuracy of measuring fluorescence intensity, which reflects the amount of antibody, can be affected by factors such as autofluorescence. Furthermore, even when a process is performed to remove spectral information derived from autofluorescent substances (hereinafter referred to as autofluorescence components or autofluorescence spectra) from the image of a stained tissue section (hereinafter also referred to as a stained section) (hereinafter referred to as a stained specimen image) using spectral color separation (hereinafter referred to as color separation process), it is impossible to completely eliminate the autofluorescence components contained in the stained specimen image after color separation (referred to as a fluorescence component image), which has been a factor causing problems such as hindering the quantification of the amount of dye antibody distributed in the target region of the stained specimen image.

[0015] Therefore, in the following embodiment, in order to solve these problems, we propose a method for image processing that generates a pseudo-autofluorescence component image (autofluorescence component corrected image) from a color-separated image of a stained section (spectral image of each fluorescence component and / or each autofluorescence component; i.e., fluorescence component image and / or autofluorescence component image) and uses this image, having the following characteristics.

[0016] The first feature is that, in the following embodiment, an image (also called an antibody count image) is generated from a fluorescence component image (also called an antibody count image) which consists of autofluorescence components extracted from a stained specimen image by color separation processing and spectral information derived from the fluorescent substance (fluorescence component; also called a fluorescence spectrum) obtained by color separation processing, and the spectral intensity of the autofluorescence component is calculated. By using such an autofluorescence component corrected image, it is possible to reduce the effort required to separately acquire autofluorescence component images obtained by imaging unstained sections, and to improve quantitative accuracy by using spectral information linked to positional information.

[0017] The second feature is that the luminance threshold processing of the stained specimen image is performed using an autofluorescence component correction image generated from the stained specimen image. This makes it possible to extract regions that have a specific signal of a fluorescent substance (also called a staining fluorescent dye) that is distinct from autofluorescence.

[0018] A third feature is that, similarly for fluorescent substances, an image (fluorescence component corrected image) is generated by calculating the spectral intensity of fluorescence from the fluorescent component derived from the fluorescent substance obtained by color separation processing and the fluorescent component image, and the relative ratio of the spectral intensity derived from the fluorescent substance to the spectral intensity derived from the autofluorescent substance is calculated. This makes it possible to evaluate the color separation accuracy and the designed fluorescent dye antibody panel.

[0019] Furthermore, while Patent Document 1, mentioned above, proposes a method for extracting targets by setting a threshold based on the difference in pixel brightness values, it cannot be said that there is a correlation between the difference in brightness values ​​and the presence or absence of molecular expression, so the extraction of targets using such a method may not be appropriate. In contrast, in the following embodiment, the target region is extracted using an autofluorescence component-corrected image in which a correlation is observed between the difference in brightness values ​​and the presence or absence of molecular expression, making it possible to appropriately extract the target region.

[0020] Furthermore, in silico labeling technology (Eric M. Christiansen et al., “In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images”, Cell, 173, 792-803, April 19, 2018) utilizes deep learning to predict target cell regions from transmission images (autofluorescence component images) and reproduce highly accurate fluorescently labeled images. However, this technology requires both fluorescently labeled images as input images for training, but it is not possible to prepare two input images (autofluorescence component image and fluorescently labeled image) from the same field of view; instead, images from serial sections must be used, resulting in spatial differences between the two input images. In contrast, the following embodiment generates a pseudo-autofluorescence component image from stained specimen images obtained by imaging stained sections, allowing for discussion of quantitative accuracy using data from the same population.

[0021] Furthermore, while it is conceivable to specify regions using serial sections from the same tissue block, even when using serial sections, the images used will be of regions shifted by several μm in the optical axis direction. Therefore, there is a possibility of being affected by spatial differences in the distribution morphology of cells and autofluorescent substances, as well as signal intensity, between unstained and stained sections. In contrast, in the following embodiment, an autofluorescence component-corrected image is generated based on the spectral information of autofluorescent substances extracted from the stained section itself. This makes it possible to quantify stained specimen images under conditions where the variability of autofluorescent substances is spatially identical.

[0022] <1. First Embodiment> First, the first embodiment relating to this disclosure will be described.

[0023] (1.1. Example Configuration) Referring to Figure 1, an example of the configuration of the information processing system according to this embodiment will be described. As shown in Figure 1, the information processing system according to this embodiment comprises an information processing device 100 and a database 200, and the inputs to the information processing system are a fluorescent reagent 10, a specimen 20, and a fluorescently stained specimen 30.

[0024] (Fluorescent reagent 10) The fluorescent reagent 10 is a chemical used to stain the specimen 20. The fluorescent reagent 10 may include, for example, a fluorescent antibody (including a primary antibody used for direct labeling or a secondary antibody used for indirect labeling), a fluorescent probe, or a nuclear staining reagent, but the type of fluorescent reagent 10 is not limited to these. Furthermore, the fluorescent reagent 10 is managed by attaching identification information (hereinafter referred to as "reagent identification information 11") that can identify the fluorescent reagent 10 (or the manufacturing lot of the fluorescent reagent 10). The reagent identification information 11 may be, for example, barcode information (such as one-dimensional barcode information or two-dimensional barcode information), but is not limited to this. Even if the fluorescent reagent 10 is the same product, its properties will differ from one manufacturing lot to another depending on the manufacturing method and the state of the cells from which the antibody was obtained. For example, in the fluorescent reagent 10, the fluorescence wavelength spectrum (fluorescence spectrum), quantum yield, or fluorescence labeling rate may differ from one manufacturing lot to another. Therefore, in the information processing system according to this embodiment, the fluorescent reagent 10 is managed by each manufacturing lot by attaching reagent identification information 11. This allows the information processing device 100 to perform fluorescence separation while taking into account even slight differences in properties that may appear in each manufacturing lot.

[0025] (specimen 20) Specimen 20 is prepared from a specimen or tissue sample taken from the human body for the purpose of pathological diagnosis, etc. Specimen 20 may be a tissue section, cells, or microparticles, and there are no particular limitations on the type of tissue used (e.g., organs), the type of disease targeted, the attributes of the subject (e.g., age, sex, blood type, or race), or the subject's lifestyle (e.g., diet, exercise habits, or smoking habits). Tissue sections may include, for example, a section before staining of a tissue section to be stained (hereinafter simply referred to as a section), a section adjacent to a stained section, a section different from the stained section in the same block (sampled from the same location as the stained section), a section from a different block in the same tissue (sampled from a different location than the stained section), or a section taken from a different patient. In addition, each specimen 20 is managed with identification information that can identify it (hereinafter referred to as "specimen identification information 21"). The sample identification information 21, like the reagent identification information 11, is, for example, barcode information (such as one-dimensional barcode information or two-dimensional barcode information), but is not limited to this. The properties of the sample 20 differ depending on the type of tissue used, the type of disease targeted, the attributes of the subject, or the lifestyle of the subject. For example, in the sample 20, the measurement channel or the wavelength spectrum of autofluorescence (autofluorescence spectrum), etc., differs depending on the type of tissue used. Therefore, in the information processing system according to this embodiment, each sample 20 is managed individually by being assigned sample identification information 21. This allows the information processing device 100 to perform fluorescence separation while taking into account even slight differences in properties that appear for each sample 20.

[0026] (Fluorescent stained specimen 30) The fluorescently stained specimen 30 is prepared by staining the specimen 20 with a fluorescent reagent 10. In this embodiment, it is assumed that the fluorescently stained specimen 30 is prepared by staining the specimen 20 with one or more fluorescent reagents 10, but the number of fluorescent reagents 10 used for staining is not particularly limited. Furthermore, the staining method is determined by the combination of specimen 20 and fluorescent reagents 10, and is not particularly limited.

[0027] (Information processing device 100) As shown in Figure 1, the information processing device 100 includes an acquisition unit 110, a storage unit 120, a processing unit 130, a display unit 140, a control unit 150, and an operation unit 160. The information processing device 100 may be, for example, a fluorescence microscope, but is not necessarily limited to this and may include various other devices. For example, the information processing device 100 may be a PC (Personal Computer), etc.

[0028] (Acquisition part 110) The acquisition unit 110 is configured to acquire information used for various processes of the information processing device 100. As shown in Figure 1, the acquisition unit 110 comprises an information acquisition unit 111 and a fluorescence signal acquisition unit 112.

[0029] (Information acquisition unit 111) The information acquisition unit 111 is configured to acquire information about the fluorescent reagent 10 (hereinafter referred to as "reagent information") and information about the specimen 20 (hereinafter referred to as "specimen information"). More specifically, the information acquisition unit 111 acquires reagent identification information 11 attached to the fluorescent reagent 10 used to produce the fluorescent stained specimen 30, and specimen identification information 21 attached to the specimen 20. For example, the information acquisition unit 111 acquires the reagent identification information 11 and specimen identification information 21 using a barcode reader or the like. Then, the information acquisition unit 111 acquires reagent information based on the reagent identification information 11 and specimen information based on the specimen identification information 21 from the database 200. The information acquisition unit 111 stores this acquired information in the information storage unit 121, which will be described later.

[0030] In this embodiment, the sample information includes information about the autofluorescence spectra (hereinafter also referred to as autofluorescence reference spectra) of one or more autofluorescent substances in the sample 20, and the reagent information includes information about the fluorescence spectra (hereinafter also referred to as fluorescence reference spectra) of fluorescent substances in the fluorescently stained sample 30. The autofluorescence reference spectra and fluorescence reference spectra, individually or collectively, are also referred to as "reference spectra."

[0031] (Fluorescence signal acquisition unit 112) The fluorescence signal acquisition unit 112 is configured to acquire multiple fluorescence signals corresponding to multiple excitation lights when multiple excitation lights with different wavelengths are irradiated onto a fluorescently stained specimen 30 (prepared by staining specimen 20 with fluorescent reagent 10). More specifically, the fluorescence signal acquisition unit 112 receives light and outputs a detection signal corresponding to the amount of light received, thereby acquiring a data cube (hereinafter referred to as a stained specimen image) composed of the fluorescence spectrum of the fluorescently stained specimen 30 based on the detection signal. Here, the content of the excitation light (including excitation wavelength and intensity, etc.) is determined based on reagent information, etc. (in other words, information about the fluorescent reagent 10, etc.). Note that the fluorescence signal referred to here is not particularly limited as long as it is a signal derived from fluorescence, and the fluorescence spectrum is merely one example. In this explanation, the case where the fluorescence signal is a fluorescence spectrum is given as an example.

[0032] Figures 2A to 2D show specific examples of fluorescence spectra acquired by the fluorescence signal acquisition unit 112. Figures 2A to 2D show specific examples of fluorescence spectra acquired when a fluorescently stained specimen 30 contains four types of fluorescent substances: DAPI, CK / AF488, PgR / AF594, and ER / AF647, and is irradiated with excitation light having excitation wavelengths of 392 nm (Figure 2A), 470 nm (Figure 2B), 549 nm (Figure 2C), and 628 nm (Figure 2D), respectively. Note that the fluorescence wavelength is shifted to a longer wavelength than the excitation wavelength due to the release of energy for fluorescence emission (Stokes shift). Furthermore, the fluorescent substances contained in the fluorescently stained specimen 30 and the excitation wavelength of the irradiated excitation light are not limited to those described above. The fluorescence signal acquisition unit 112 stores the stained specimen image consisting of the acquired fluorescence spectra in the fluorescence signal storage unit 122, which will be described later.

[0033] (Preservation section 120) The storage unit 120 is configured to store information used in various processes of the information processing device 100, or information output by various processes. As shown in Figure 1, the storage unit 120 comprises an information storage unit 121 and a fluorescence signal storage unit 122.

[0034] (Information storage section 121) The information storage unit 121 is configured to store reagent information and specimen information acquired by the information acquisition unit 111.

[0035] (Fluorescent signal storage unit 122) The fluorescence signal storage unit 122 is configured to store the fluorescence signal of the fluorescence-stained specimen 30 acquired by the fluorescence signal acquisition unit 112.

[0036] (Processing unit 130) The processing unit 130 is configured to perform various processes, including color separation. As shown in Figure 1, the processing unit 130 comprises a separation processing unit 132 and an image generation unit 133.

[0037] (Separation processing unit 132) The separation processing unit 132 is configured to separate the stained specimen image into fluorescence spectra for each fluorescent substance. As described later, it extracts autofluorescence spectra from the input stained specimen image and generates an autofluorescence component-corrected image using the extracted autofluorescence spectra (generation unit). The separation processing unit 132 then performs color separation processing on the stained specimen image using the generated autofluorescence component-corrected image (separation unit). This separation processing unit 132 can perform the functions of a generation unit, separation unit, correction unit, and image generation unit as described in the claims.

[0038] For color separation, methods such as least squares (LSM) or weighted least squares (WLSM) may be used. Furthermore, for the extraction of autofluorescence spectra and / or fluorescence spectra, methods such as non-negative matrix factorization (NMF), singular value decomposition (SVD), or principal component analysis (PCA) may be used.

[0039] (Operation unit 160) The operation unit 160 is configured to receive operation input from the user. More specifically, the operation unit 160 is equipped with various input means such as a keyboard, mouse, buttons, touch panel, or microphone, and the user can perform various inputs to the information processing device 100 by operating these input means. Information regarding operation input performed via the operation unit 160 is provided to the control unit 150.

[0040] (Database 200) Database 200 is a device that manages reagent information and specimen information. More specifically, database 200 manages reagent identification information 11 in association with reagent information, and specimen identification information 21 in association with specimen information. As a result, the information acquisition unit 111 can acquire reagent information from database 200 based on the reagent identification information 11 of the fluorescent reagent 10, and specimen information from database 200 based on the specimen identification information 21 of the specimen 20.

[0041] The reagent information managed by database 200 is assumed to include (but is not necessarily limited to) the measurement channels and fluorescence reference spectra specific to the fluorescent substances in the fluorescent reagent 10. "Measurement channel" is a concept that indicates the fluorescent substance contained in the fluorescent reagent 10. Since the number of fluorescent substances varies depending on the fluorescent reagent 10, the measurement channels are managed as reagent information linked to each fluorescent reagent 10. Furthermore, the fluorescence reference spectra included in the reagent information are, as described above, the fluorescence spectra of each fluorescent substance contained in the measurement channel.

[0042] Furthermore, it is assumed that the sample information managed by database 200 includes (but is not limited to) the measurement channels and autofluorescence reference spectra specific to the autofluorescent substances contained in sample 20. "Measurement channels" are a concept that indicates the autofluorescent substances contained in sample 20, and in the example in Figure 8, this refers to Hemoglobin, Archidonic Acid, Catalase, Collagen, FAD, NADPH, and ProLongDiamond. Since the number of autofluorescent substances varies depending on the sample 20, the measurement channels are managed as sample information linked to each sample 20. The autofluorescence reference spectra included in the sample information are, as described above, the autofluorescence spectra of each autofluorescent substance contained in the measurement channels. Note that the information managed in database 200 is not necessarily limited to the above.

[0043] The above describes an example of the configuration of the information processing system according to this embodiment. Note that the above configuration described with reference to Figure 1 is merely an example, and the configuration of the information processing system according to this embodiment is not limited to this example. For example, the information processing device 100 does not necessarily have to have all of the configurations shown in Figure 1, and may have configurations not shown in Figure 1.

[0044] Here, the information processing system according to this embodiment may include an imaging device (e.g., including a scanner) for acquiring fluorescence spectra and an information processing device for processing using fluorescence spectra. In this case, the fluorescence signal acquisition unit 112 shown in Figure 1 may be implemented by the imaging device, and the other components may be implemented by the information processing device. Alternatively, the information processing system according to this embodiment may include an imaging device for acquiring fluorescence spectra and software used for processing using fluorescence spectra. In other words, the information processing system does not need to have a physical configuration (e.g., memory or processor) for storing or executing the software. In this case, the fluorescence signal acquisition unit 112 shown in Figure 1 may be implemented by the imaging device, and the other components may be implemented by the information processing device on which the software is executed. The software may be provided to the information processing device via a network (e.g., from a website or cloud server) or via any storage medium (e.g., a disk). The information processing device on which the software is executed may be various types of servers (e.g., cloud servers), general-purpose computers, PCs, or tablet PCs. The method by which the software is provided to the information processing device and the type of information processing device are not limited to those described above. Furthermore, it should be noted that the configuration of the information processing system according to this embodiment is not necessarily limited to the above, and that a configuration that can be conceived by a person skilled in the art may be applied based on the state of the art at the time of use.

[0045] (1.2. Examples of applications to microscope systems) The information processing system described above may be implemented as, for example, a microscope system. Therefore, with reference to Figure 5, an example of the configuration of a microscope system when the information processing system according to this embodiment is implemented as a microscope system will be described.

[0046] As shown in Figure 5, the microscope system according to this embodiment comprises a microscope 101 and a data processing unit 107.

[0047] The microscope 101 comprises a stage 102, an optical system 103, a light source 104, a stage drive unit 105, a light source drive unit 106, and a fluorescence signal acquisition unit 112.

[0048] The stage 102 has a mounting surface on which a fluorescently stained specimen 30 can be placed, and is movable in a direction parallel to the mounting surface (xy plane direction) and perpendicular to the mounting surface (z axis direction) by the drive of the stage drive unit 105. The fluorescently stained specimen 30 has a thickness of, for example, several μm to several tens of μm in the Z direction and is fixed by a predetermined fixing method, sandwiched between a glass slide SG and a cover glass (not shown).

[0049] An optical system 103 is positioned above the stage 102. The optical system 103 comprises an objective lens 103A, an imaging lens 103B, a dichroic mirror 103C, an emission filter 103D, and an excitation filter 103E. The light source 104 is, for example, a light bulb such as a mercury lamp or an LED (Light Emitting Diode), and is driven by a light source drive unit 106 to irradiate the fluorescent label attached to the fluorescently stained specimen 30 with excitation light.

[0050] The excitation filter 103E generates excitation light by transmitting only the light of the excitation wavelength that excites the fluorescent dye from the light source 104 when obtaining a fluorescence image of the fluorescently stained specimen 30. The dichroic mirror 103C reflects the excitation light that is transmitted through the excitation filter and guides it to the objective lens 103A. The objective lens 103A focuses the excitation light onto the fluorescently stained specimen 30. The objective lens 103A and the imaging lens 103B then magnify the image of the fluorescently stained specimen 30 to a predetermined magnification and project the magnified image onto the imaging surface of the fluorescence signal acquisition unit 112.

[0051] When excitation light is shone on the fluorescently stained specimen 30, the staining agents bound to each tissue in the fluorescently stained specimen 30 emit fluorescence. This fluorescence passes through the dichroic mirror 103C via the objective lens 103A and reaches the imaging lens 103B via the emission filter 103D. The emission filter 103D absorbs the light that has been magnified by the objective lens 103A and passed through the excitation filter 103E, allowing only a portion of the colored light to pass through. The image of the colored light, from which the ambient light has been lost, is magnified by the imaging lens 103B as described above and formed on the fluorescence signal acquisition unit 112.

[0052] The data processing unit 107 drives the light source 104, acquires a fluorescence image of the fluorescently stained specimen 30 using the fluorescence signal acquisition unit 112, and performs various processing using this image. More specifically, the data processing unit 107 can function as part or all of the information acquisition unit 111, storage unit 120, processing unit 130, display unit 140, control unit 150, operation unit 160, or database 200 of the information processing device 100, as described with reference to Figure 1. For example, the data processing unit 107, by functioning as the control unit 150 of the information processing device 100, controls the driving of the stage drive unit 105 and the light source drive unit 106, and controls the acquisition of spectra by the fluorescence signal acquisition unit 112. Also, the data processing unit 107, by functioning as the processing unit 130 of the information processing device 100, generates fluorescence spectra, separates fluorescence spectra for each fluorescent substance, and generates image information based on the separation results.

[0053] The above describes an example of the configuration of a microscope system when the information processing system according to this embodiment is implemented as a microscope system. Note that the above configuration described with reference to Figure 5 is merely an example, and the configuration of the microscope system according to this embodiment is not limited to this example. For example, the microscope system does not necessarily have to include all of the configurations shown in Figure 5, and may include configurations not shown in Figure 5.

[0054] (1.3. Regarding the least squares method) Here, we will explain the least squares method used in the color separation processing by the separation processing unit 132. The least squares method calculates the color mixing ratio by fitting a reference spectrum to the fluorescence spectrum, which is the pixel value of each pixel in the input stained specimen image. The color mixing ratio is an index that indicates the degree to which each substance is mixed. Equation (1) below represents the residual obtained by subtracting the fluorescence reference spectrum and autofluorescence reference spectrum mixed at a color mixing ratio a from the fluorescence spectrum (Signal). In equation (1), "Signal (1 × number of channels)" indicates that there are as many fluorescence spectra (Signal) as there are channels of wavelength. For example, Signal is a matrix that represents one or more fluorescence spectra. Also, "St (number of substances × number of channels)" indicates that there are as many reference spectra for each substance (fluorescent substance and autofluorescent substance) as there are channels of wavelength. For example, St is a matrix that represents one or more reference spectra. Furthermore, "a(1 × number of substances)" indicates that a color mixing ratio a is assigned to each substance (fluorescent substance and autofluorescent substance). For example, a is a matrix representing the color mixing ratios of each reference spectrum in the fluorescence spectrum.

[0055]

number

[0056] The separation processing unit 132 then calculates the mixing ratio a of each substance that minimizes the sum of squares of the residual equation (1). The sum of squares of the residuals is minimized when the result of the partial derivative of equation (1) with respect to the mixing ratio a is zero. Therefore, the separation processing unit 132 calculates the mixing ratio a of each substance that minimizes the sum of squares of the residuals by solving the following equation (2). In equation (2), "St'" represents the transpose matrix of the reference spectrum St. Also, "inv(St*St')" represents the inverse matrix of St*St'.

[0057]

number

[0058] Here, specific examples of the values ​​in equation (1) above are shown in equations (3) to (5) below. In the examples of equations (3) to (5), the case is shown in which the reference spectra (St) of three substances (3 substances) are mixed at different mixing ratios a in the fluorescence spectrum (Signal).

[0059]

number

[0060]

number

[0061]

number

[0062] Then, specific examples of the calculation results of equation (2) using the values ​​of equations (3) and (5) are shown in equation (6) below. As shown in equation (6), it can be seen that the calculation result correctly yields "a = (3 2 1)" (i.e., the same value as in equation (4) above).

[0063]

number

[0064] Furthermore, as described above, the separation processing unit 132 may extract the spectra of each fluorescent substance from the fluorescence spectrum by performing calculations related to the weighted least squares method instead of the least squares method. In the weighted least squares method, weights are applied to emphasize errors at low signal levels by taking advantage of the fact that the noise of the measured fluorescence spectrum (Signal) follows a Poisson distribution. However, the upper limit for which weighting is not applied in the weighted least squares method is defined as the Offset value. The Offset value is determined by the characteristics of the sensor used for measurement, and separate optimization is required when an image sensor is used as the sensor. When the weighted least squares method is performed, the reference spectrum St in equations (1) and (2) above is replaced with St_ represented by the following equation (7). Note that the following equation (7) means that St_ is calculated by dividing each element (each component) of St, which is represented as a matrix, by the corresponding element (each component) in "Signal + Offset value", which is also represented as a matrix (in other words, elemental division).

[0065]

number

[0066] Here, when the Offset value is 1 and the values ​​of the reference spectrum St and the fluorescence spectrum Signal are expressed by equations (3) and (5) above, a specific example of St_ expressed by equation (7) above is shown in equation (8) below.

[0067]

number

[0068] A concrete example of the calculation result of the color mixing ratio a in this case is shown in equation (9) below. As can be seen in equation (9), the calculation result correctly yields "a = (3 2 1)".

[0069]

number

[0070] (1.4. Non-negative matrix factorization (NMF)) Next, the non-negative matrix factorization (NMF) used by the separation processing unit 132 for extracting the autofluorescence spectrum and / or fluorescence spectrum will be described. However, the method is not limited to non-negative matrix factorization (NMF), and singular value decomposition (SVD), principal component analysis (PCA), etc., may also be used.

[0071] Figure 3 is a diagram illustrating the overview of NMF. As shown in Figure 3, NMF decomposes a non-negative N x M (N×M) matrix A into a non-negative N x k (N×k) matrix W and a non-negative k x M (k×M) matrix H. Matrices W and H are determined such that the mean square residual D between matrix A and the product of matrices W and H (W*H) is minimized. In this embodiment, matrix A corresponds to the spectrum before the autofluorescence reference spectrum is extracted (N is the number of pixels and M is the number of wavelength channels), and matrix H corresponds to the extracted autofluorescence reference spectrum (k is the number of autofluorescence reference spectra (in other words, the number of autofluorescent substances) and M is the number of wavelength channels). Here, the mean square residual D is expressed by the following equation (10). Note that "norm(D,'fro')" refers to the Frobenius norm of the mean square residual D.

[0072]

number

[0073] Factorization in NMF employs an iterative method starting with random initial values ​​for matrices W and H. While the value of k (number of autofluorescence reference spectra) is mandatory in NMF, the initial values ​​for matrices W and H are optional and can be set. When initial values ​​for matrices W and H are set, the solution becomes constant. Conversely, if initial values ​​for matrices W and H are not set, they are randomly determined, and the solution becomes non-constant.

[0074] The specimen 20 differs in nature and autofluorescence spectrum depending on the type of tissue used, the type of disease targeted, the attributes of the subject, or the lifestyle of the subject. Therefore, as described above, the information processing apparatus 100 according to the second embodiment can realize a more accurate color separation process by actually measuring the autofluorescence reference spectrum for each specimen 20.

[0075] Note that the matrix A, which is the input of NMF, is a matrix consisting of the same number of rows as the number of pixels N (= Hpix × Vpix) of the stained specimen image and the same number of columns as the number of wavelength channels M. Therefore, when the number of pixels of the stained specimen image is large or the number of wavelength channels M is large, the matrix A becomes a very large matrix, increasing the calculation cost of NMF and lengthening the processing time.

[0076] In such a case, for example, as shown in FIG. 4, by clustering into the number of classes N (<Hpix × Vpix) with the number of pixels N (= Hpix × Vpix) of the stained image specified, it is possible to suppress the elongation of the processing time due to the enlargement of the matrix A.

[0077] In clustering, for example, among the stained images, spectra similar in the wavelength direction or intensity direction are classified into the same class. As a result, an image with a smaller number of pixels than the stained image is generated, and it becomes possible to reduce the scale of the matrix A' with this image as the input.

[0078] (Image generation unit 133) The image generation unit 133 is configured to generate image information based on the separation results of the fluorescence spectra by the separation processing unit 132. For example, the image generation unit 133 can generate image information using fluorescence spectra corresponding to one or more fluorescent substances, or using autofluorescence spectra corresponding to one or more autofluorescent substances. The number and combination of fluorescent substances (molecules) or autofluorescent substances (molecules) used by the image generation unit 133 to generate image information are not particularly limited. Furthermore, if various processing (e.g., segmentation or calculation of S / N value) is performed using the separated fluorescence spectra or autofluorescence spectra, the image generation unit 133 may generate image information showing the results of those processing.

[0079] (Display section 140) The display unit 140 is configured to present the image information generated by the image generation unit 133 to the user by displaying it on a screen. The type of screen used as the display unit 140 is not particularly limited. In addition, although not described in detail in this embodiment, the image information generated by the image generation unit 133 may also be presented to the user by projecting it with a projector or printing it with a printer (in other words, the method of outputting the image information is not particularly limited).

[0080] (Control unit 150) The control unit 150 has a functional configuration that comprehensively controls all processing performed by the information processing device 100. For example, the control unit 150 controls the start and end of various processes described above (for example, adjusting the placement position of the fluorescently stained specimen 30, irradiating the fluorescently stained specimen 30 with excitation light, acquiring the spectrum, generating an autofluorescence component corrected image, color separation, generating image information, and displaying image information) based on operation input from the user via the operation unit 160. The control contents of the control unit 150 are not particularly limited. For example, the control unit 150 may control processes that are commonly performed in general-purpose computers, PCs, tablet PCs, etc. (for example, processes related to the OS (Operating System)).

[0081] (1.5. Characteristic configuration of the first embodiment) The above describes an example configuration and application of the information processing system according to this embodiment. Next, the characteristic configuration of this embodiment will be described in detail below with reference to the drawings.

[0082] As described above, in this embodiment, in order to distinguish the fluorescence signal originating from autofluorescent substances, which is problematic when photographing tissue samples, from the fluorescence signal originating from the fluorescent substance to be analyzed, a pseudo-autofluorescent component image (autofluorescent component corrected image) is generated using spectral information (autofluorescence component (spectrum)) originating from autofluorescent substances obtained when color separation processing of stained specimen images, and this is used to enable quantitative analysis of stained specimen images.

[0083] More specifically, in this embodiment, an autofluorescence component correction image is generated using the corresponding autofluorescence reference spectrum based on the autofluorescence component extracted from the stained specimen image, and the stained specimen image is processed using the autofluorescence component correction image to generate a fluorescence component image with higher accuracy of color separation. The generated fluorescence component image may be displayed on the display unit 140, or predetermined processing (such as analysis processing) may be performed on the processing unit 130 or other components (for example, an analysis device connected via a network). The predetermined processing may be, for example, the detection of specific cells.

[0084] (1.5.1. Basic Operation Examples) Figure 6 is a flowchart showing an example of the basic operation of the information processing system according to this embodiment. The following operations are performed, for example, by the operation of each part under the control of the control unit 150.

[0085] As shown in Figure 6, in the basic operation according to this embodiment, first, the information acquisition unit 111 of the acquisition unit 110 acquires a stained specimen image by imaging the fluorescently stained specimen 30 (step S101). The acquired stained specimen image is then stored, for example, in the information storage unit 121 of the storage unit 120.

[0086] Furthermore, the information acquisition unit 111 acquires reagent information and specimen information from a database 200 connected via a network (step S102). As described above, the specimen information includes information on the autofluorescence reference spectra of one or more autofluorescent substances in the specimen 20, and the reagent information includes information on the fluorescence reference spectra of fluorescent substances in the fluorescently stained specimen 30. The acquired reagent information and specimen information are stored, for example, in the information storage unit 121 of the storage unit 120.

[0087] Next, the separation processing unit 132 acquires the stained specimen image, reagent information, and specimen information stored in the information storage unit 121, and performs color separation processing of the stained specimen image by fitting an autofluorescence reference spectrum to the acquired stained specimen image, for example, using the least squares method (step S103). This color separation processing generates a fluorescence component image and an autofluorescence component image.

[0088] Next, the separation processing unit 132 generates an autofluorescence component corrected image using the autofluorescence component image generated in step S103 and the autofluorescence reference spectrum included in the sample information acquired in step S102 (step S104). The generation of the autofluorescence component corrected image will be explained in more detail later.

[0089] Next, the separation processing unit 132 generates a fluorescence component image by processing the stained specimen image using the autofluorescence component correction image (step S105). In this way, by removing the autofluorescence component contained in the stained specimen image using the autofluorescence component correction image generated using the autofluorescence reference spectrum, it is possible to generate a fluorescence component image with improved color separation accuracy, that is, with a reduced amount of residual autofluorescence component (correction unit).

[0090] The separation processing unit 132 then transmits the generated fluorescence component image to the image generation unit 133 or an external server (step S106). After that, this operation ends.

[0091] (1.5.2. Example of Autofluorescence Component Corrected Image Generation Flow) Next, we will describe an example of the operation when generating the autofluorescence component image described in step S104 of Figure 6. Figure 7 is a flowchart of an example of the operation when generating a pseudo-autofluorescence component image according to this embodiment. Figure 8 is a diagram showing an example of the autofluorescence component image generated in step S103 of Figure 6, and Figure 9 is a diagram showing an example of the autofluorescence reference spectrum included in the sample information acquired in step S102 of Figure 6. Furthermore, Figure 10 is a diagram to explain step S112 of Figure 7, and Figure 11 is a diagram to explain step S114 of Figure 7.

[0092] As shown in Figure 7, in the autofluorescence component correction image generation flow, the separation processing unit 132 first selects one of the unselected autofluorescence component images (see Figure 8) generated in step S103 of Figure 6 (this will be the autofluorescence component image of the autofluorescence channel CHn (where n is a natural number)) (step S111). Note that the autofluorescence channel may be identification information assigned to each autofluorescence.

[0093] Next, the separation processing unit 132 generates a spectral image of the autofluorescence channel CHn from the autofluorescence component image of the autofluorescence channel CHn selected in step S111 and the autofluorescence reference spectrum of the autofluorescence channel CHn among the autofluorescence reference spectra (see Figure 9) included in the sample information acquired in step S102 in Figure 6 (step S112). The above-mentioned NMF can be used to generate this spectral image.

[0094] Next, the separation processing unit 132 determines whether all autofluorescence component images have been selected in step S111 (step S113). If they have not been selected (NO in step S113), it returns to step S111, selects the unselected autofluorescence component images, and then performs the subsequent operations.

[0095] On the other hand, if all autofluorescence component images have been selected in step S111 (YES in step S113), the separation processing unit 132 adds the spectral images of each autofluorescence channel CH generated in the repeated step S112, as shown in Figure 11 (step S114). This generates an autofluorescence component corrected image. After that, the separation processing unit 132 terminates the operation shown in Figure 7.

[0096] (1.6. Extraction and Analysis of the Target Region) Next, the extraction process of the target region from the fluorescence component image according to this embodiment and its analysis will be explained using a comparative example. Figure 12 is a flowchart illustrating the extraction process of the target region and its analysis according to the comparative example, and Figure 13 is a schematic diagram illustrating the extraction process of the target region and its analysis according to the comparative example shown in Figure 12. Furthermore, Figure 14 is a flowchart illustrating the extraction process of the target region and its analysis according to this embodiment, and Figure 15 is a schematic diagram illustrating the extraction process of the target region and its analysis according to this embodiment shown in Figure 14.

[0097] (Comparative example) As shown in Figures 12 and 13, in the comparative example, first, a stained section is imaged to obtain a stained specimen image, and an unstained tissue section adjacent to or near the stained section (hereinafter also referred to as an unstained tissue section or unstained section) is imaged to obtain an image of the unstained section (hereinafter referred to as an unstained specimen image) (step S901). Subsequently, a color separation process is performed using the acquired stained specimen image and unstained specimen image to generate a fluorescence component image and an autofluorescence component image (step S902).

[0098] Next, based on the autofluorescence component image, a threshold for the number of antibodies is set to extract candidate regions (regions to be extracted) for the region to be analyzed (analysis target region) (step S903). The threshold for the number of antibodies may be, for example, a threshold for the pixel value of each pixel in the fluorescence component image. The regions to be extracted may be the regions hatched with dots in step S903 of Figure 13.

[0099] Next, the pixel value of each pixel in the stained specimen image is compared with a threshold, and a mask image for extracting the region to be analyzed is generated based on the comparison result (step S904). This generates a binary mask image in which, for example, if the pixel value is greater than or equal to the threshold, it is assigned '1', and if it is less than the threshold, it is assigned '0'. Pixels or regions assigned '1' are candidates for analysis, while pixels or regions assigned '0' may be pixels or regions excluded from analysis.

[0100] Next, a logical AND operation is performed between each pixel of the mask image generated in step S904 and each pixel of the fluorescence component image to extract the region to be analyzed from the fluorescence component image (step S905). As a result, an extracted image is obtained in which the region labeled with the fluorescent dye antibody (morphological information of the target of analysis) is extracted, using the value extracted from the autofluorescence component image as a threshold.

[0101] Subsequently, for example, the area to be analyzed is sent to an analysis device such as an external server and evaluated quantitatively (step S906).

[0102] (Embodiment) On the other hand, as shown in Figures 14 and 15, in this embodiment, a stained specimen image is first obtained by imaging a stained section (step S1). In other words, in this embodiment, there is no need to image an unstained section to obtain an unstained specimen image.

[0103] Next, in this embodiment, a fluorescence component image is generated by performing a color separation process on the stained specimen image (step S2). Furthermore, in this embodiment, an autofluorescence component correction image is generated using the autofluorescence component image generated by the color separation process and the autofluorescence reference spectrum included in the acquired specimen information (step S3).

[0104] Next, based on the autofluorescence component-corrected image, a threshold for the number of antibodies is set to extract candidate regions for analysis (step S4). As explained using Figures 12 and 13, the threshold for the number of antibodies may be, for example, a threshold for the pixel value of each pixel in the fluorescence component image, and the regions to be extracted may be the regions hatched with dots in step S4 of Figure 15.

[0105] Subsequently, similar to steps S904-S906 in Figures 12 and 13, a mask image is generated by comparing the pixel value of each pixel in the stained specimen image with a threshold (step S5). Then, a logical AND operation is performed between the mask image generated in step S5 and the fluorescence component image to extract the region to be analyzed from the fluorescence component image (step S6). This results in an extracted image in which the region labeled with the fluorescent dye antibody (morphological information of the target of analysis) is extracted, using the value extracted from the autofluorescence component corrected image as the threshold. The extracted region to be analyzed is then sent to an analysis device, such as an external server, for quantitative evaluation (step S7).

[0106] (1.7. Action and Effects) As described above, according to this embodiment, in the process of extracting the region to be analyzed, a value defined by the user from the autofluorescence component corrected image or an objective numerical value is set as a threshold, and the region to be analyzed is extracted using a mask image (binary mask) generated based on the relationship between the pixel values ​​in the stained specimen image and the threshold.

[0107] Thus, by using autofluorescence-corrected images to set the threshold, it is possible to eliminate the need to separately image unstained sections to obtain unstained specimen images and the effort to generate autofluorescence-corrected images. Furthermore, because the threshold can be set based on an autofluorescence-corrected image that has a correlation with the stained specimen image and the fluorescence-corrected image, which retains spatial information (signal distribution such as autofluorescence components and noise origins are the same) within the stained specimen image, it becomes possible to set a threshold that is correlated with the stained specimen image and the fluorescence-corrected image.

[0108] <2. Second Embodiment> Next, a second embodiment will be described. In the first embodiment described above, a unique threshold was set for all pixels of the stained specimen image from the generated autofluorescence component corrected image. In contrast, in the second embodiment, a threshold distribution image is generated in which thresholds that are not necessarily the same are spatially distributed by setting a different threshold for each pixel. In this embodiment, the configuration and basic operation of the information processing system may be the same as the configuration and basic operation of the information processing system according to the first embodiment, so a detailed explanation will be omitted here.

[0109] (2.1. Extraction of the analysis target area and its analysis) Figure 16 is a flowchart illustrating the extraction process of the analysis target area and its analysis according to this embodiment, and Figure 17 is a schematic diagram illustrating the extraction process of the analysis target area and its analysis according to this embodiment shown in Figure 16.

[0110] As shown in Figures 16 and 17, in this embodiment, first, a stained specimen image is acquired by imaging a stained section (step S21), similar to the operation described using steps S1 to S3 in Figures 14 and 15 in the first embodiment; a fluorescence component image is generated by performing a color separation process on the stained specimen image (step S22); and an autofluorescence component corrected image is generated using the autofluorescence component image generated by the color separation process and the autofluorescence reference spectrum included in the acquired specimen information (step S23).

[0111] Next, in this embodiment, a threshold distribution image is generated based on the autofluorescence component-corrected image, in which a different antibody threshold is set for each pixel (step S24). As a method for generating a threshold distribution image in which each pixel has a different threshold, for example, a method of multiplying the pixel value of each pixel in the autofluorescence component-corrected image by a predetermined coefficient can be employed. Here, the coefficient may be determined, for example, within a range of values ​​considered reasonable. Specifically, a value based on the maximum or average value of the pixel values ​​in the autofluorescence component-corrected image may be used as the coefficient.

[0112] Next, in this embodiment, the pixel value of each pixel in the stained specimen image is compared with the threshold value of the corresponding pixel in the threshold distribution image, and a mask image for extracting the region to be analyzed is generated based on the comparison result (step S25). This generates a binary mask image in which, for example, if the pixel value is greater than or equal to the threshold, it is set to '1', and if it is less than the threshold, it is set to '0'. As in the first embodiment, pixels or regions assigned '1' may be candidates for analysis, and pixels or regions assigned '0' may be pixels or regions excluded from analysis. Also, in the pixel-by-pixel comparison result shown in step S25 of Figure 17, pixels hatched with dots may be, for example, pixels whose pixel value was greater than or equal to the corresponding threshold.

[0113] Subsequently, similar to steps S6-S7 in Figures 14 and 15, a logical AND operation is performed between the mask image generated in S25 and the fluorescence component image to extract the region to be analyzed from the fluorescence component image (step S26). This results in an extracted image in which the region labeled with the fluorescent dye antibody (morphological information of the target of analysis) is extracted, using the value extracted from the autofluorescence component corrected image as a threshold. The extracted region to be analyzed is then sent to an analysis device, such as an external server, for quantitative evaluation (step S27).

[0114] (2.2. Action and Effects) As described above, in this embodiment, when generating a binary mask for extracting only the desired region from a stained specimen image, instead of uniquely setting the threshold from the autofluorescence component correction image as in the first embodiment, a threshold distribution image is generated that allows different thresholds to be spatially distributed. This makes it possible to set the threshold using the pixel values ​​of the stained specimen image and the corresponding autofluorescence component correction image, and thus it becomes possible to extract the region to be analyzed using a threshold distribution image that retains spatial information of potential noise that may occur in the system, such as the distribution of autofluorescence components for each tissue region and the influence of hardware. As a result, it becomes possible to further improve the accuracy of the analysis in fluorescence observation.

[0115] Furthermore, since the other configurations, operations, and effects may be the same as those of the embodiments described above, a detailed explanation is omitted here.

[0116] <3. Third Embodiment> Next, a third embodiment will be described. In the embodiments described above, an example was given of generating an autofluorescence component-corrected image from a stained specimen image. In contrast, in this embodiment, in addition to the autofluorescence component-corrected image, a pseudo-fluorescence component image (hereinafter referred to as the fluorescence component-corrected image) is also generated, and the ratio of their spectral intensities is calculated to obtain information on the ratio of fluorescent dye intensity to autofluorescence. In this embodiment, the configuration and basic operation of the information processing system may be the same as the configuration and basic operation of the information processing system in the embodiments described above, so a detailed explanation will be omitted here.

[0117] (3.1. Generation and analysis of spectral intensity ratio images) In this embodiment, the spectral intensity ratio is generated, for example, as spatially distributed information, i.e., image data (spectral intensity ratio image). Figure 18 is a flowchart illustrating the generation process and analysis of the spectral intensity ratio image according to this embodiment.

[0118] As shown in Figure 18, in this embodiment, first, an autofluorescence component-corrected image is generated by performing the same operations as in steps S101 to S104 in Figure 6 in the first embodiment (step S104).

[0119] Next, in this embodiment, a fluorescence component correction image is generated by applying, for example, a similar operation to that used when generating an autofluorescence component correction image (step S301). Note that the fluorescence component correction image generation flow can be carried out by replacing the autofluorescence component image with a fluorescence component image and the autofluorescence reference spectrum with a fluorescence reference spectrum in the operation described with reference to Figure 7 in the first embodiment, so a detailed explanation is omitted here.

[0120] Next, the separation processing unit 132 generates a spectral intensity ratio image by calculating the ratio (spectral intensity ratio) of corresponding pixels in the fluorescence component corrected image and the autofluorescence component corrected image (step S302).

[0121] Subsequently, for example, the spectral intensity ratio image is sent to an analysis device such as an external server for quantitative evaluation (step S303).

[0122] (3.2. Methods for generating spectral intensity images) In this embodiment, the spectral intensity ratio image may be generated, for example, by generating a spectral intensity image showing the spectral intensity of each pixel in the fluorescence component correction image and a spectral intensity image showing the spectral intensity of each pixel in the autofluorescence component correction image, and then calculating the ratio of the spectral intensities of the corresponding pixels.

[0123] The methods for generating spectral intensity images of both the fluorescence-corrected image and the autofluorescence-corrected image are not particularly limited. Therefore, two generation methods are illustrated below.

[0124] (3.2.1 First generation method) Figure 19 is a diagram illustrating the first generation method. As shown in Figure 19, the first generation method generates a spectral intensity image by summing the pixel values ​​for each wavelength channel of the fluorescence component corrected image / autofluorescence component corrected image of the data cube structure in the wavelength direction (step S31).

[0125] (3.2.2 Second generation method) Figure 20 is a diagram illustrating the second generation method. As shown in Figure 20, the second generation method generates a spectral intensity image by extracting the pixel value with the maximum value in the wavelength direction from the pixel values ​​for each wavelength channel of the fluorescence component corrected image / autofluorescence component corrected image of the data cube structure (step S32).

[0126] (3.3. Action and Effects) As described above, in this embodiment, even for the fluorescent component, a fluorescence component correction image is generated by multiplying the fluorescence reference spectrum extracted by color separation processing for each pixel of the fluorescence component image, and spectral intensity images of the fluorescence component correction image and the autofluorescence component correction image are obtained. Then, a spectral intensity ratio image is generated from the spectral intensity ratio between each pixel of the fluorescence component correction image and each pixel of the autofluorescence component correction image, which is information that spatially shows the ratio of the fluorescence dye intensity to the autofluorescence. The spectral intensity ratio image generated in this way can be used for performance evaluation of the measurement system (corresponding to the acquisition unit 110) in the information processing system, color separation accuracy evaluation in the processing unit 130, and furthermore, for the design / evaluation of the fluorescence reagent panel.

[0127] Furthermore, according to this embodiment, for example, the spectral intensity ratio image generated as described above can be recorded in the storage unit 120 or the like, linked to image acquisition conditions such as the shooting conditions when the stained specimen image acquisition unit 110 acquired the image and the labeling conditions (staining conditions) of the fluorescent reagent 10 on the specimen 20, and used as reference information when acquiring stained specimen images using the same acquisition unit 110. In addition, by storing the spectral intensity ratio image and image acquisition conditions on a server on the network and sharing them with other information processing systems or information processing devices 100, it is possible to use them as reference information when acquiring stained specimen images using the same model acquisition unit 110 in other information processing systems. Specifically, for example, the spectral intensity ratio image and image acquisition conditions stored in the storage unit 120, server, etc., may be used for inter-instrument baseline correction and calibration curve correction when using the same model acquisition unit 110.

[0128] Furthermore, since the other configurations, operations, and effects may be the same as those of the embodiments described above, a detailed explanation is omitted here.

[0129] <4. Fourth Embodiment> Next, a fourth embodiment will be described. In the embodiments described above, for example, NMF, SVD, PCA, etc., were used to extract the autofluorescence spectrum and / or fluorescence spectrum. In contrast, in this embodiment, a case in which machine learning is used to extract the autofluorescence spectrum and / or fluorescence spectrum will be described instead. In this embodiment, the configuration and basic operation of the information processing system may be the same as the configuration and basic operation of the information processing system in the embodiments described above, so a detailed explanation will be omitted here.

[0130] (4.1. Generation of fluorescence component images using machine learning) Figure 21 is a flowchart illustrating the process of generating fluorescence component images using machine learning according to this embodiment, and Figure 22 is a schematic diagram illustrating the process of generating fluorescence component images using machine learning according to this embodiment shown in Figure 21.

[0131] As shown in Figures 21 and 22, in this embodiment, first, an autofluorescence component-corrected image is generated by performing the same operations as in steps S1 to S3 of Figure 14 in the second embodiment (steps S401 to S403).

[0132] Next, in this embodiment, the fluorescence component image generated by the color separation process in step S401 and the pseudo-autofluorescence component image generated in step S403 are input to the machine learning unit 401. The machine learning unit 401 then performs unsupervised deep learning using the fluorescence component image and the pseudo-autofluorescence component image as input images to extract features such as signal intensity and distribution derived from the autofluorescence component (e.g., autofluorescence spectrum) from the stained specimen image (step S404). Various machine learning methods such as DNN (Deep Neural Network), CNN (Convolutional Neural Network), and RNN (Recurrent Neural Network) can be applied as the deep learning method. Furthermore, the machine learning unit 401 may be implemented, for example, in the processing unit 130 of the information processing device 100, or in a cloud server connected to the information processing device 100 via a predetermined network.

[0133] Then, in this embodiment, using the features extracted in step S404, a more accurate fluorescence component image with a further reduced amount of residual autofluorescence is generated (step S405), and this operation is terminated.

[0134] (4.2. Action and Effects) As described above, in this embodiment, the fluorescent component image generated by color separation processing of a stained specimen image and the autofluorescence component correction image generated from thereto are used as input images, and unsupervised machine learning is applied to extract features such as signal intensity and distribution derived from the autofluorescence component from the stained specimen image, and then generate a more accurate fluorescent component image based on these extracted features. In this way, by performing machine learning using a stained specimen image as input, or in other words, an autofluorescence component correction image generated from a stained specimen image as input, it becomes easier to link the input image and output image of the machine learning, which makes it easier to improve the learning effect.

[0135] Furthermore, since the other configurations, operations, and effects may be the same as those of the embodiments described above, a detailed explanation is omitted here.

[0136] <5. Example of Measurement System Configuration> Next, an example of the configuration of the measurement system in the information processing device 100 according to the above-described embodiment (hereinafter simply referred to as the embodiment) will be explained. Figure 23 is a diagram showing an example of the measurement system of the information processing system according to the embodiment. Figure 23 shows an example of a measurement system when capturing a wide field of view of a fluorescently stained specimen 30 (or an unstained specimen 20), such as WSI (Whole Slide Imaging). However, the measurement system according to the embodiment is not limited to the measurement system illustrated in Figure 23, and can be modified in various ways as long as it is a measurement system capable of acquiring image data with sufficient resolution of the entire shooting area or region of interest (hereinafter referred to as wide-field image data) of the entire shooting area or region of interest, such as a measurement system that captures the entire shooting area or a necessary area thereof (also called the region of interest) at once, or a measurement system that acquires an image of the entire shooting area or region of interest by line scanning.

[0137] As shown in Figure 23, the measurement system according to this embodiment includes, for example, an information processing device 100, an XY stage 501, an excitation light source 510, a beam splitter 511, an objective lens 512, a spectrometer 513, and a photodetector 514.

[0138] The XY stage 501 is a stage on which the fluorescently stained specimen 30 (or specimen 20) to be analyzed is placed, and may be, for example, a stage that is movable in a plane (XY plane) parallel to the surface on which the fluorescently stained specimen 30 (or specimen 20) is placed.

[0139] The excitation light source 510 is a light source for exciting the fluorescently stained specimen 30 (or specimen 20), and for example, emits multiple excitation lights with different wavelengths along a predetermined optical axis.

[0140] The beam splitter 511 is composed of, for example, a dichroic mirror, and reflects the excitation light from the excitation light source 510 and transmits the fluorescence from the fluorescently stained specimen 30 (or specimen 20).

[0141] The objective lens 512 irradiates the fluorescently stained specimen 30 (or specimen 20) on the XY stage 501 with excitation light reflected by the beam splitter 511.

[0142] The spectrometer 513 is constructed using one or more prisms, lenses, etc., and spectrally separates the fluorescence emitted from the fluorescently stained specimen 30 (or specimen 20) and transmitted through the objective lens 512 and beam splitter 511 in a predetermined direction.

[0143] The photodetector 514 detects the light intensity for each wavelength of fluorescence spectrally separated by the spectrometer 513, and inputs the resulting fluorescence signal (fluorescence spectrum and / or autofluorescence spectrum) to the fluorescence signal acquisition unit 112 of the information processing device 100.

[0144] In the configuration described above, if the entire imaging area exceeds the area that can be imaged in one go (hereinafter referred to as the field of view), such as in a WSI, the XY stage 501 is moved to shift the field of view with each image, so that each field of view is imaged sequentially. Then, by tiling the image data obtained from the image of each field of view (hereinafter referred to as the field of view image data), a wide-field of view image data of the entire imaging area is generated. The generated wide-field of view image data is stored, for example, in the fluorescence signal storage unit 122. Note that the tiling of the field of view image data may be performed in the acquisition unit 110 of the information processing device 100, in the storage unit 120, or in the processing unit 130.

[0145] Then, the processing unit 130 according to the embodiment performs the above-described processing on the obtained wide-field image data to acquire a coefficient C, that is, a fluorescence separation image for each fluorescent molecule (or an autofluorescence separation image for each autofluorescent molecule).

[0146] <6. Method for calculating the number of fluorescent molecules (or antibodies)> Next, a method for calculating the number of fluorescent molecules (or antibodies) in one pixel in the embodiment described above will be explained. Figure 24 is a schematic diagram illustrating the method for calculating the number of fluorescent molecules (or antibodies) in one pixel in the embodiment. In the example shown in Figure 24, when the image sensor and sample are arranged with an objective lens in between, let's assume that the size of the bottom surface of the sample corresponding to 1 [pixel] of the image sensor is 13 / 20 (μm) × 13 / 20 (μm). And let's assume that the thickness of the sample is 10 (μm). In that case, the volume (m³) of this rectangular parallelepiped is... 3 The volume (in liters) is expressed as 13 / 20(μm) × 13 / 20(μm) × 10(μm). 3 It is represented as follows.

[0147] Furthermore, assuming that the concentration of antibodies (or fluorescent molecules) in the sample is uniform and 300 (nM), the number of antibodies per pixel can be expressed by the following equation (11).

[0148]

number

[0149] In this way, the number of fluorescent molecules or antibodies in the fluorescently stained specimen 30 is calculated as a result of the fluorescence separation process, allowing the user to compare the number of fluorescent molecules among multiple fluorescent substances or to compare data imaged under different conditions. Furthermore, since brightness (or fluorescence intensity) is a continuous value while the number of fluorescent molecules or antibodies is a discrete value, the information processing device 100 in the modified example can reduce the amount of data by outputting image information based on the number of fluorescent molecules or antibodies.

[0150] <7. Hardware Configuration Example> The embodiments and modified versions thereof of the present disclosure have been described above. Next, with reference to Figure 25, examples of the hardware configuration of the information processing device 100 according to each embodiment and modified version will be described. Figure 25 is a block diagram showing an example of the hardware configuration of the information processing device 100. Various processes performed by the information processing device 100 are realized through the cooperation of software and the hardware described below.

[0151] As shown in Figure 25, the information processing device 100 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, and a host bus 904a. The information processing device 100 also includes a bridge 904, an external bus 904b, an interface 905, an input device 906, an output device 907, a storage device 908, a drive 909, a connection port 911, a communication device 913, and a sensor 915. The information processing device 100 may have a processing circuit such as a DSP or ASIC in place of, or together with, the CPU 901.

[0152] The CPU 901 functions as both an arithmetic processing unit and a control unit, controlling the overall operation of the information processing unit 100 according to various programs. The CPU 901 may also be a microprocessor. The ROM 902 stores programs and arithmetic parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901 and parameters that change as needed during its execution. For example, the CPU 901 can embody at least the processing unit 130 and the control unit 150 of the information processing unit 100.

[0153] The CPU 901, ROM 902, and RAM 903 are interconnected by a host bus 904a, which includes the CPU bus. The host bus 904a is connected to an external bus 904b, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 904. It is not necessary to configure the host bus 904a, bridge 904, and external bus 904b separately; these functions may be implemented on a single bus.

[0154] The input device 906 can be implemented by a device into which information is input by the user, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever. The input device 906 may also be a remote control device utilizing infrared or other radio waves, or an external device such as a mobile phone or PDA that is compatible with the operation of the information processing device 100. Furthermore, the input device 906 may include, for example, an input control circuit that generates an input signal based on the information input by the user using the above-mentioned input means and outputs it to the CPU 901. By operating this input device 906, the user can input various data to the information processing device 100 or instruct it to perform processing operations. The input device 906 can, for example, embody at least the operation unit 160 of the information processing device 100.

[0155] The output device 907 is formed of a device capable of visually or audibly notifying the implementer of the acquired information. Examples of such devices include display devices such as CRT displays, liquid crystal displays, plasma displays, EL displays, and lamps, as well as audio output devices such as speakers and headphones, and printers. The output device 907 may, for example, embody at least the display unit 140 of the information processing device 100.

[0156] The storage device 908 is a device for storing data. The storage device 908 can be implemented by, for example, a magnetic storage device such as an HDD, a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 908 may also include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deletion device for deleting data recorded on the storage medium. This storage device 908 stores programs executed by the CPU 901, various data, and various data acquired from external sources. The storage device 908 can, for example, embody at least the storage unit 120 of the information processing device 100.

[0157] Drive 909 is a reader / writer for storage media and is either built into or external to the information processing unit 100. Drive 909 reads information recorded on removable storage media such as magnetic disks, optical disks, magneto-optical disks, or semiconductor memory and outputs it to RAM 903. Drive 909 can also write information to removable storage media.

[0158] Connection port 911 is an interface for connecting to external devices, and is a connection port for external devices that can transmit data via, for example, USB (Universal Serial Bus).

[0159] The communication device 913 is, for example, a communication interface formed by a communication device for connecting to the network 920. The communication device 913 is, for example, a communication card for wired or wireless LAN (Local Area Network), LTE (Long Term Evolution), Bluetooth (registered trademark), or WUSB (Wireless USB). Alternatively, the communication device 913 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication. This communication device 913 can, for example, send and receive signals to and from the Internet or other communication devices in accordance with a predetermined protocol such as TCP / IP.

[0160] In this embodiment, the sensor 915 includes a sensor capable of acquiring a spectrum (e.g., an image sensor), but it may also include other sensors (e.g., an acceleration sensor, a gyro sensor, a geomagnetic sensor, a pressure sensor, a sound sensor, or a distance measuring sensor). The sensor 915 can, for example, embody at least the fluorescence signal acquisition unit 112 of the information processing device 100.

[0161] Network 920 is a wired or wireless transmission path for information transmitted from devices connected to Network 920. For example, Network 920 may include public networks such as the Internet, telephone networks, and satellite communication networks, as well as various LANs (Local Area Networks) and WANs (Wide Area Networks), including Ethernet®. Network 920 may also include dedicated network lines such as IP-VPN (Internet Protocol-Virtual Private Network).

[0162] The above describes an example of a hardware configuration that can realize the functions of the information processing device 100. Each of the above components may be realized using general-purpose materials, or it may be realized using hardware specialized for the function of each component. Therefore, it is possible to change the hardware configuration used as appropriate, depending on the level of technology at the time of implementing this disclosure.

[0163] Furthermore, it is possible to create computer programs to realize each of the functions of the information processing device 100 as described above and implement them on a PC or the like. A computer-readable recording medium containing such computer programs can also be provided. The recording medium includes, for example, magnetic disks, optical disks, magneto-optical disks, and flash memory. Alternatively, the computer programs may be distributed without using a recording medium, for example, via a network.

[0164] While preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations may be conceived within the scope of the technical ideas described in the claims, and these will naturally also fall within the technical scope of the present disclosure.

[0165] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that are obvious to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0166] Furthermore, the following configurations also fall within the technical scope of this disclosure. (1) A separation unit separates a fluorescence image observed from a specimen labeled with one or more fluorescent dyes into a fluorescence component image containing one or more fluorescent components and an autofluorescence component image containing one or more autofluorescence components. A generation unit that generates an autofluorescence component corrected image using the reference spectra of one or more autofluorescent substances contained in the sample and the autofluorescence component image, A processing unit that processes the fluorescence component image based on the autofluorescence component corrected image, An information processing device equipped with the following features. (2) The processing unit is an information processing device according to (1) that generates a fluorescence component corrected image obtained by correcting the fluorescence component image using the autofluorescence component corrected image. (3) The information processing apparatus according to (1) or (2) above, wherein the generation unit generates the autofluorescence component corrected image for each autofluorescent substance by adding the result of multiplying the autofluorescence component image and the reference spectrum. (4) The processing unit generates region information indicating regions labeled with the fluorescent dye from the autofluorescence component corrected image, and processes the fluorescent component image using the generated region information, according to any one of (1) to (3) above, the information processing device. (5) The information processing apparatus according to (4), wherein the region information is a binary mask indicating the region labeled by the fluorescent dye. (6) The processing unit is the information processing device according to (5) above, which generates the binary mask based on the relationship between a threshold set based on the autofluorescence component corrected image and the pixel value of each pixel in the fluorescence image. (7) The processing unit sets a threshold value for each pixel based on the pixel value of each pixel in the autofluorescence component corrected image, and generates the binary mask based on the relationship between the threshold value for each pixel and the pixel value of each pixel in the fluorescence image, as described in (5). (8) The information processing apparatus according to any one of (1) to (7), wherein the generation unit further generates a fluorescence component corrected image using the reference spectra of each of the one or more fluorescent dyes and the fluorescence component images. (9) The information processing apparatus according to (8), further comprising an evaluation unit that evaluates at least one of the following based on the fluorescence component correction image and the autofluorescence component correction image: the measurement accuracy of the measurement system that acquires the fluorescence image, the separation performance of the fluorescence component image and the autofluorescence component image by the separation unit, and the staining performance of the fluorescence reagent panel composed of one or more fluorescence dyes. (10) An image generation unit that generates a spectral intensity ratio image representing the ratio between the spectral intensity of each pixel in the fluorescence component correction image and the spectral intensity of each pixel in the autofluorescence component correction image, A storage unit that stores the imaging conditions when the sample is imaged and the fluorescence image is obtained, and associates the spectral intensity ratio image with the imaging conditions. The information processing apparatus according to (8) or (9) above, further comprising the above. (11) The system further includes a correction unit that corrects the fluorescence component image based on the fluorescence image, the fluorescence component image, and the autofluorescence component correction image. The processing unit processes the corrected fluorescence component image based on the autofluorescence component corrected image. An information processing device as described in any one of (1) to (10) above. (12) The information processing apparatus according to (11), wherein the correction unit estimates the characteristic quantities of the autofluorescent substance in the fluorescence image from the fluorescence component image and the autofluorescent component correction image, and corrects the fluorescence component image based on the estimated characteristic quantities. (13) The correction unit estimates the feature quantities by using a trained model that takes the fluorescence component image and the autofluorescence component corrected image as inputs, as described in (12). (14) The information processing device described in (13) above, wherein the pre-trained model is a model trained by unsupervised learning. (15) An information processing apparatus according to any one of (1) to (14) above, further comprising an acquisition unit that acquires the fluorescence image by imaging the aforementioned specimen. (16) The processing unit detects specific cells contained in the fluorescence image based on the fluorescence component image after processing, according to any one of (1) to (15). (17) The fluorescence images observed from a specimen labeled with one or more fluorescent dyes are separated into a fluorescence component image containing one or more fluorescent components and an autofluorescence component image containing one or more autofluorescence components. An autofluorescence component-corrected image is generated using the reference spectrum of each of the one or more autofluorescent substances contained in the sample and the autofluorescence component image. The fluorescence component image is processed based on the autofluorescence component corrected image. Information processing methods that include the following. (18) A process to separate a fluorescence image observed from a specimen labeled with one or more fluorescent dyes into a fluorescence component image containing one or more fluorescent components and an autofluorescence component image containing one or more autofluorescence components, A process to generate an autofluorescence component-corrected image using the reference spectrum of each of the one or more autofluorescent substances contained in the sample and the autofluorescence component image, A process of processing the fluorescence component image based on the autofluorescence component corrected image, A program that causes a computer to execute something. (19) A microscope system comprising a light source for irradiating a specimen labeled with one or more fluorescent dyes, an imaging device for observing the fluorescence emitted from the specimen irradiated with the light, and a program for performing processing on the fluorescence image acquired by the imaging device, The aforementioned program, when executed on a computer, The process of separating the aforementioned fluorescence image into a fluorescence component image containing one or more fluorescence components and an autofluorescence component image containing one or more autofluorescence components, A process to generate an autofluorescence component-corrected image using the reference spectrum of each of the one or more autofluorescent substances contained in the sample and the autofluorescence component image, A process of processing the fluorescence component image based on the autofluorescence component corrected image, To cause the computer to execute the above, Microscope system. (20) The information processing device described in (1) above, An analysis device connected to the information processing device via a predetermined network, which analyzes the fluorescence component image processed by the information processing device, An analysis system equipped with the following features. [Explanation of Symbols]

[0167] 10 Fluorescent reagents 11 Reagent Identification Information 20 specimens 21. Specimen Identification Information 30 Fluorescently stained specimens 100 Information Processing Devices 110 Acquisition Department 111 Information Acquisition Department 112 Fluorescence signal acquisition unit 120 Preservation Department 121 Information Storage Department 122 Fluorescence signal storage section 130 Processing Unit 132 Separation Processing Unit 133 Image generation unit 140 Display section 150 Control Unit 160 Operation section 200 databases 401 Machine Learning Department

Claims

1. A processing unit that generates an image based on the pixel value of each pixel in a cell image observed from cells labeled with one or more fluorescent dyes and a threshold value for each pixel, Based on the cell image, a separation unit separates autofluorescence component images containing one or more autofluorescence components, The system comprises a generation unit that generates autofluorescence spectral information using the reference spectra of one or more autofluorescent substances contained in the cell and the autofluorescence component images, The threshold is set based on the spectral information of the autofluorescence. The spectral information of the autofluorescence generated by the generation unit is an autofluorescence component corrected image. The generation unit generates the autofluorescence component correction image by adding the result of multiplying the autofluorescence component image and the reference spectrum for each autofluorescent substance. Information processing device.

2. The image generated by the aforementioned processing unit is a binary mask image. The information processing apparatus according to claim 1.

3. The processing unit generates the binary mask image based on the relationship between the set threshold and the pixel value of each pixel in the cell image. The information processing apparatus according to claim 2.

4. The separation unit further separates a fluorescence component image containing one or more fluorescence components based on the cell image, and The processing unit processes the fluorescence component image based on the threshold. The information processing apparatus according to claim 1.

5. The separation unit performs the separation process using the least squares method or the weighted least squares method. The information processing apparatus according to claim 4.

6. A computer, An image is generated based on the pixel value of each pixel in a cell image observed from cells labeled with one or more fluorescent dyes, and a threshold value for each of the pixels. Based on the aforementioned cell image, an autofluorescence component image containing one or more autofluorescence components is separated, An information processing method comprising generating autofluorescence spectral information using the reference spectra of one or more autofluorescent substances contained in the cell and the autofluorescence component images, The threshold is set based on the spectral information of the autofluorescence. The spectral information of the autofluorescence generated is an autofluorescence component-corrected image. The generation of the aforementioned image further includes generating the autofluorescence component corrected image by adding the results obtained by multiplying the autofluorescence component image and the reference spectrum for each autofluorescent substance. Information processing methods.

Images