Information processing device and information processing method

Abstract

To improve analysis accuracy in fluorescence observation.SOLUTION: An information processing device comprises a processing unit that generates an image based on a pixel value of each pixel in a cell image observed from cells labeled with one or more fluorescent dyes and a threshold value for the pixel value of each pixel.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] In recent years, with the development of cancer immunotherapy and the like, there has been an increasing need to detect and evaluate molecules that are phenotypes of immune cells classified into a large number of subsets using multiple markers. The multi-color imaging analysis method using fluorescent dyes as markers is considered to be an effective means for leading to the relationship between their localization and function.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, when fluorescently observing a stained tissue section stained with a fluorescent dye antibody, the accuracy of measuring the fluorescence intensity reflecting the amount of the antibody may be affected due to autofluorescence or the like. Even when the autofluorescence component is removed from the captured image of the stained tissue section by spectral color separation, it has been impossible to completely eliminate the autofluorescence component remaining in the stained fluorescence image obtained by this color separation.

[0005] Thus, when the autofluorescence component remains in the stained fluorescence image, problems such as an obstacle to quantifying the amount of the dye antibody distributed in the analysis target region in the stained fluorescence image occur, and it has been an obstacle to improving the analysis accuracy in fluorescence observation.

[0006] Therefore, the present disclosure has been made in view of the above circumstances, and proposes an information processing apparatus, an information processing method, a program, a microscope system, and an analysis system that enable improvement of the analysis accuracy in fluorescence observation.

Means for Solving the Problem

[0007] An information processing apparatus according to an embodiment of the present disclosure includes a processing unit that generates an image based on pixel values of each pixel in a cell image observed from cells labeled with one or more fluorescent dyes and thresholds for the pixel values of each pixel.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

MODE FOR CARRYING OUT THE INVENTION

[0009] With reference to the accompanying drawings, preferred embodiments of the present disclosure will be described in detail. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] Note that the description will be made in the following order. 0. Introduction 1. First Embodiment 1.1. Configuration Example 1.2. Application Example to Microscope System 1.3. Least Squares Method 1.4. Non - negative Matrix Factorization (NMF) 1.5. Characteristic Configuration of the First Embodiment 1.5.1. Basic Operation Example 1.5.2. Example of Self - fluorescence Component Correction Image Generation Flow 1.6. Extraction and Analysis of Analysis Target Region 1.7. Actions and Effects 2. Second Embodiment 2.1. Extraction and Analysis of Analysis Target Region 2.2. Actions and Effects 3. Third Embodiment 3.1. Generation and Analysis of Spectrum Intensity Ratio Image 3.2. Generation Method of Spectrum Intensity Image 3.2.1 First Generation Method 3.2.2 Second Generation Method 3.3. Actions and Effects 4. Fourth Embodiment 4.1. Generation of Fluorescent Component Image Using Machine Learning 4.2. Actions and Effects 5. Configuration Example of Measurement System 6. Calculation Method of Fluorescent Molecule Number (or Antibody Number) 7. Hardware Configuration Example

[0011] <0. Introduction> In recent years, with the development of cancer immunotherapy and the like, there has been an increasing need to detect and evaluate molecules that are the phenotypes of immune cells classified into a number of subsets using multiple markers. A multi-color imaging analysis method using a fluorescent dye as a marker is considered to be an effective means for leading to the relevance between their localization and function, and is also adopted in the following embodiments.

[0012] The following embodiments relate to image processing after performing color separation processing on a multi-channel image (number of pixels × wavelength channel (CH)) obtained by imaging with an imaging device that excites a pathologically stained specimen section immunohistochemically with excitation light of multiple wavelengths and separates fluorescence.

[0013] Note that the multi-channel images in the present disclosure may include various images having a data cube structure composed of image data of a plurality of wavelength channels (however, a single wavelength channel is not excluded), such as stained specimen images, fluorescence component images, fluorescence component correction images, autofluorescence component images, and autofluorescence component correction images described later. Therefore, the fluorescence component image, the fluorescence component correction image, the autofluorescence component image, and the autofluorescence component correction image are not limited to the image data of the fluorescence component or the autofluorescence component of a single wavelength channel, and may be spectral images composed of the fluorescence components or the autofluorescence components of a plurality of wavelength channels. In addition, the fluorescence component and the autofluorescence component in the present description mean the wavelength components of fluorescence or autofluorescence in the multi-channel image obtained by imaging with an imaging device.

[0014] As described above, when fluorescently observing a stained tissue section stained with a fluorescent dye antibody, the accuracy of measuring the fluorescence intensity reflecting the amount of antibody may be affected due to autofluorescence or the like. Further, even when performing a process (hereinafter referred to as color separation process) of removing spectral information (hereinafter referred to as autofluorescence component or autofluorescence spectrum) derived from autofluorescent substances by color separation using a spectrum from an imaging image of a stained tissue section (hereinafter also referred to as a stained section) (hereinafter referred to as a stained specimen image), it is impossible to completely eliminate the autofluorescent components contained in the stained specimen image after the color separation process (referred to as a fluorescence component image), which has been a factor causing problems such as hindering the quantification of the amount of dye antibody distributed in the analysis target region in the stained specimen image.

[0015] Therefore, in the following embodiments, in order to solve these problems, regarding an image process using a pseudo autofluorescent component image (autofluorescent component correction image) generated from a color separation image of a stained section (spectral images of each fluorescent component and / or each autofluorescent component, that is, a fluorescent component image and / or an autofluorescent component image), a method having the following features is proposed.

[0016] As a first feature, in the following embodiments, an image (autofluorescent component correction image) in which the spectral intensity of autofluorescence is calculated is generated from an autofluorescent component extracted from a stained specimen image by a color separation process and a fluorescent component image (also referred to as an antibody number image) composed of spectral information (fluorescent component, also referred to as a fluorescence spectrum) derived from a fluorescent substance obtained by the color separation process. By using such an autofluorescent component correction image, it is possible to achieve effects such as reducing the trouble of separately obtaining an autofluorescent component image obtained by imaging an unstained section and improving the quantification by using spectral information linked to position information.

[0017] As a second feature, luminance threshold processing of the stained specimen image is performed using the autofluorescent component correction image generated from the stained specimen image. Thereby, it becomes possible to extract a region having a specific signal of a fluorescent substance (also referred to as a stained fluorescent dye) distinguishable from autofluorescence.

[0018] As a third feature, similarly for the fluorescent substance, an image (fluorescent component correction image) in which the spectral intensity of fluorescence is calculated is generated from the fluorescent component derived from the fluorescent substance and the fluorescent component image obtained by color separation processing, and the relative ratio between the spectral intensity derived from the fluorescent substance and the spectral intensity derived from the autofluorescent substance is calculated. This makes it possible to evaluate the color separation accuracy and the designed fluorescent dye antibody panel.

[0019] In Patent Document 1 mentioned above, a method of setting a threshold based on the difference in pixel luminance values to extract an object has been proposed. However, since there is no correlation between the difference in luminance values and the presence or absence of molecular expression, extraction of an object by such a method may not be appropriate. In contrast, in the following embodiments, since the object region is extracted using the autofluorescent component correction image in which a correlation is recognized between the difference in luminance values and the presence or absence of molecular expression, it becomes possible to appropriately extract the object region.

[0020] Also, in the in silico labeling technique (Eric M. Christiansen et al., “In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images”, Cell, 173, 792-803, April 19, 2018), it is possible to predict the target cell region from a transmission image (autofluorescent component image) by utilizing deep learning and reproduce a highly accurate fluorescent label image. However, in this technique, it is necessary to prepare an actually fluorescently labeled image as learning data together with the input image. However, since it is not possible to prepare the two input images (autofluorescent component image and fluorescent label image) of the same visual field and continuous sections are used, a spatial difference occurs between the two input images. In contrast, in the following embodiments, since a pseudo autofluorescent component image is generated from the stained specimen image obtained by imaging a stained section, quantification can be discussed using data of the same population.

[0021] Furthermore, it is also conceivable to perform region designation using consecutive sections of the same tissue block. However, even when using consecutive sections, images of regions shifted by about several micrometers in the optical axis direction will be used. Therefore, there is a possibility of being affected by spatial differences in terms of the distribution pattern and signal intensity of cells and autofluorescent substances between unstained sections and stained sections. In contrast, in the following embodiments, since an autofluorescent component correction image is generated based on the spectral information of the autofluorescent substances extracted from the stained sections themselves, it becomes possible to quantify stained specimen images under conditions where variations such as autofluorescent substances are spatially identical.

[0022] <1. First Embodiment> First, the first embodiment according to the present disclosure will be described.

[0023] (1.1. Configuration Example) With reference to FIG. 1, a configuration example of the information processing system according to this embodiment will be described. As shown in FIG. 1, the information processing system according to this embodiment includes an information processing apparatus 100 and a database 200. As inputs to the information processing system, there are a fluorescent reagent 10, a specimen 20, and a fluorescently stained specimen 30.

[0024] (Fluorescent Reagent 10) The fluorescent reagent 10 is a chemical used for staining the specimen 20. The fluorescent reagent 10 is, for example, a fluorescent antibody (including a primary antibody used for direct labeling or a secondary antibody used for indirect labeling), a fluorescent probe, or a nuclear staining reagent, etc., but the type of the fluorescent reagent 10 is not limited to these. Also, the fluorescent reagent 10 is managed with discriminable identification information (hereinafter referred to as "reagent identification information 11") attached thereto for identifying the fluorescent reagent 10 (or the production lot of the fluorescent reagent 10). The reagent identification information 11 is, for example, barcode information etc. (such as one-dimensional barcode information or two-dimensional barcode information), but is not limited thereto. Even if the fluorescent reagent 10 is the same product, its properties differ for each production lot depending on, for example, the production method or the state of the cells from which the antibody was obtained. For example, in the fluorescent reagent 10, the wavelength spectrum of fluorescence (fluorescence spectrum), the quantum yield, or the fluorescence labeling rate etc. differ for each production lot. Therefore, in the information processing system according to the present embodiment, the fluorescent reagent 10 is managed for each production lot by attaching the reagent identification information 11. Thereby, the information processing apparatus 100 can perform fluorescence separation in consideration of even the slight property differences that appear for each production lot.

[0025] (Specimen 20) The specimen 20 is prepared for pathological diagnosis or the like from a specimen or tissue sample collected from a human body. The specimen 20 may be a tissue section, a cell, or a microparticle. For the specimen 20, the type of tissue (such as an organ, etc.) used, the type of disease targeted, the attributes of the subject (such as age, gender, blood type, or race, etc.), or the lifestyle habits of the subject (such as diet, exercise habits, or smoking habits, etc.) are not particularly limited. Note that the tissue sections may include, for example, a section before staining of a tissue section to be stained (hereinafter also simply referred to as a section), a section adjacent to the stained section, a section different from the stained section in the same block (sampled from the same location as the stained section), or a section in a different block (sampled from a location different from the stained section) in the same tissue, a section collected from a different patient, etc. Further, the specimen 20 is managed with identification information (hereinafter referred to as "specimen identification information 21") that can identify each specimen 20. Similar to the reagent identification information 11, the specimen identification information 21 is, for example, barcode information or the like (one-dimensional barcode information, two-dimensional barcode information, etc.), but is not limited thereto. The nature of the specimen 20 differs depending on the type of tissue used, the type of disease targeted, the attributes of the subject, or the lifestyle habits of the subject, etc. For example, in the specimen 20, the measurement channel or the wavelength spectrum of autofluorescence (autofluorescence spectrum) or the like differs depending on the type of tissue used, etc. Therefore, in the information processing system according to the present embodiment, the specimen 20 is individually managed by attaching the specimen identification information 21. Thereby, the information processing apparatus 100 can perform fluorescence separation in consideration of even the slight differences in nature that appear for each specimen 20.

[0026] (Fluorescently stained specimen 30) The fluorescently stained specimen 30 is created by staining the specimen 20 with the fluorescent reagent 10. In the present embodiment, the fluorescently stained specimen 30 assumes that the specimen 20 is stained with one or more fluorescent reagents 10, and the number of fluorescent reagents 10 used for staining is not particularly limited. Further, the staining method is determined by the combination of the specimen 20 and the fluorescent reagent 10, etc., and is not particularly limited.

[0027] (Information Processing Apparatus 100) As shown in FIG. 1, the information processing apparatus 100 includes an acquisition unit 110, a storage unit 120, a processing unit 130, a display unit 140, a control unit 150, and an operation unit 160. The information processing apparatus 100 can be, for example, a fluorescence microscope or the like, but is not necessarily limited to this and may include various apparatuses. For example, the information processing apparatus 100 may be a PC (Personal Computer) or the like.

[0028] (Acquisition Unit 110) The acquisition unit 110 is configured to acquire information used for various processes of the information processing apparatus 100. As shown in FIG. 1, the acquisition unit 110 includes an information acquisition unit 111 and a fluorescence signal acquisition unit 112.

[0029] (Information Acquisition Unit 111) The information acquisition unit 111 is configured to acquire information about the fluorescent reagent 10 (hereinafter referred to as "reagent information") and information about the specimen 20 (hereinafter referred to as "specimen information"). More specifically, the information acquisition unit 111 acquires the reagent identification information 11 attached to the fluorescent reagent 10 used for generating the fluorescently stained specimen 30 and the specimen identification information 21 attached to the specimen 20. For example, the information acquisition unit 111 acquires the reagent identification information 11 and the specimen identification information 21 using a barcode reader or the like. Then, the information acquisition unit 111 acquires the reagent information from the database 200 based on the reagent identification information 11 and the specimen information based on the specimen identification information 21. The information acquisition unit 111 stores the acquired information in the information storage unit 121 described below.

[0030] Here, in the present embodiment, the specimen information includes information about the autofluorescence spectrum of one or more autofluorescent substances in the specimen 20 (hereinafter also referred to as the autofluorescence reference spectrum), and the reagent information includes information about the fluorescence spectrum of the fluorescent substance in the fluorescently stained specimen 30 (hereinafter also referred to as the fluorescence reference spectrum). Note that the autofluorescence reference spectrum and the fluorescence reference spectrum are also collectively referred to as the "reference spectrum".

[0031] (Fluorescence signal acquisition unit 112) The fluorescence signal acquisition unit 112 is configured to acquire a plurality of fluorescence signals corresponding to a plurality of excitation lights when the fluorescence-stained specimen 30 (created by staining the specimen 20 with the fluorescent reagent 10) is irradiated with a plurality of excitation lights having different wavelengths. More specifically, the fluorescence signal acquisition unit 112 receives light and outputs a detection signal corresponding to the received amount, thereby acquiring a data cube (hereinafter referred to as a stained specimen image) composed of the fluorescence spectrum of the fluorescence-stained specimen 30 based on the detection signal. Here, the content of the excitation light (including the excitation wavelength, intensity, etc.) is determined based on reagent information, etc. (in other words, information related to the fluorescent reagent 10). Note that the fluorescence signal here is not particularly limited as long as it is a signal derived from fluorescence, and the fluorescence spectrum is just an example. In this description, the case where the fluorescence signal is a fluorescence spectrum is exemplified.

[0032] A to D in FIG. 2 are specific examples of the fluorescence spectra acquired by the fluorescence signal acquisition unit 112. In A to D of FIG. 2, the fluorescence-stained specimen 30 contains four fluorescent substances, namely DAPI, CK / AF488, PgR / AF594, and ER / AF647, and specific examples of the fluorescence spectra acquired when the fluorescence-stained specimen 30 is irradiated with excitation lights having excitation wavelengths of 392 [nm] (A in FIG. 2), 470 [nm] (B in FIG. 2), 549 [nm] (C in FIG. 2), and 628 [nm] (D in FIG. 2) are shown. It should be noted that due to the release of energy for fluorescence emission, the fluorescence wavelength is shifted to the longer wavelength side than the excitation wavelength (Stokes shift). Also, the fluorescent substances contained in the fluorescence-stained specimen 30 and the excitation wavelength of the irradiated excitation light are not limited to the above. The fluorescence signal acquisition unit 112 stores the stained specimen image composed of the acquired fluorescence spectrum in the fluorescence signal storage unit 122 described later.

[0033] (Storage unit 120) The storage unit 120 is configured to store information used for various processes of the information processing apparatus 100 or information output by various processes. As shown in FIG. 1, the storage unit 120 includes an information storage unit 121 and a fluorescence signal storage unit 122.

[0034] (Information storage unit 121) The information storage unit 121 is configured to store the reagent information and specimen information acquired by the information acquisition unit 111.

[0035] (Fluorescent signal storage unit 122) The fluorescent signal storage unit 122 is configured to store the fluorescent signals of the fluorescently stained specimen 30 acquired by the fluorescent signal acquisition unit 112.

[0036] (Processing unit 130) The processing unit 130 is configured to perform various processes including color separation processing. As shown in FIG. 1, the processing unit 130 includes a separation processing unit 132 and an image generation unit 133.

[0037] (Separation processing unit 132) The separation processing unit 132 is configured to separate the stained specimen image into fluorescence spectra for each fluorescent substance. As will be described later, it extracts the autofluorescence spectrum from the input stained specimen image and generates an autofluorescence component correction image using the extracted autofluorescence spectrum (generation unit). Then, the separation processing unit 132 performs color separation processing of the stained specimen image using the generated autofluorescence component correction image (separation unit). This separation processing unit 132 can function as the generation unit, separation unit, correction unit, and image generation unit in the claims.

[0038] For color separation processing, for example, the least squares method (LSM), weighted least squares method (WLSM), etc. may be used. Also, for the extraction of the autofluorescence spectrum and / or fluorescence spectrum, for example, non-negative matrix factorization (NMF), singular value decomposition (SVD), principal component analysis (PCA), etc. may be used.

[0039] (Operation unit 160) The operation unit 160 is configured to receive operation inputs from the operator. More specifically, the operation unit 160 is provided with various input means such as a keyboard, a mouse, buttons, a touch panel, or a microphone, and the operator can perform various inputs to the information processing apparatus 100 by operating these input means. Information regarding the operation input performed via the operation unit 160 is provided to the control unit 150.

[0040] (Database 200) The database 200 is a device that manages reagent information, specimen information, and the like. More specifically, the database 200 manages by associating reagent identification information 11 with reagent information, and specimen identification information 21 with specimen information respectively. Thereby, the information acquisition unit 111 can acquire reagent information from the database 200 based on the reagent identification information 11 of the fluorescent reagent 10, and specimen information based on the specimen identification information 21 of the specimen 20.

[0041] It is assumed (but not necessarily limited to these) that the reagent information managed by the database 200 is information including the measurement channels and fluorescence reference spectra specific to the fluorescent substances of the fluorescent reagent 10. The "measurement channels" is a concept indicating the fluorescent substances contained in the fluorescent reagent 10. Since the number of fluorescent substances varies depending on the fluorescent reagent 10, the measurement channels are managed in association with each fluorescent reagent 10 as reagent information. Also, as described above, the fluorescence reference spectra included in the reagent information are the fluorescence spectra of each of the fluorescent substances included in the measurement channels.

[0042] Also, assuming that the specimen information managed by the database 200 is information including the measurement channels and the autofluorescence reference spectra specific to the autofluorescent substances of the specimen 20 (not necessarily limited to these). The "measurement channel" is a concept indicating the autofluorescent substances contained in the specimen 20. In the example of FIG. 8, it is a concept referring to Hemoglobin, Archidonic Acid, Catalase, Collagen, FAD, NADPH, and ProLong Diamond. Since the number of autofluorescent substances varies depending on the specimen 20, the measurement channels are managed linked to each specimen 20 as specimen information. Also, as described above, the autofluorescence reference spectra included in the specimen information are the autofluorescence spectra of each of the autofluorescent substances included in the measurement channels. Note that the information managed by the database 200 is not necessarily limited to the above.

[0043] Above, the configuration example of the information processing system according to this embodiment has been described. Note that the above configuration described with reference to FIG. 1 is merely an example, and the configuration of the information processing system according to this embodiment is not limited to such an example. For example, the information processing apparatus 100 does not necessarily have to include all of the configurations shown in FIG. 1, or may include configurations not shown in FIG. 1.

[0044] Here, the information processing system according to the present embodiment may include an imaging device that acquires a fluorescence spectrum (for example, including a scanner or the like) and an information processing device that performs processing using the fluorescence spectrum. In this case, the fluorescence signal acquisition unit 112 shown in FIG. 1 can be realized by the imaging device, and the other configurations can be realized by the information processing device. Further, the information processing system according to the present embodiment may include an imaging device that acquires a fluorescence spectrum and software used for processing using the fluorescence spectrum. In other words, a physical configuration (for example, a memory, a processor, etc.) that stores or executes the software may not be provided in the information processing system. In this case, the fluorescence signal acquisition unit 112 shown in FIG. 1 can be realized by the imaging device, and the other configurations can be realized by the information processing device on which the software is executed. Then, the software is provided to the information processing device via a network (for example, from a website, a cloud server, etc.) or via an arbitrary storage medium (for example, a disk, etc.). Further, the information processing device on which the software is executed can be various servers (for example, a cloud server, etc.), a general-purpose computer, a PC, or a tablet PC, etc. Note that the method by which the software is provided to the information processing device and the type of the information processing device are not limited to the above. Also, it should be noted that the configuration of the information processing system according to the present embodiment is not necessarily limited to the above, and configurations conceivable by those skilled in the art based on the technical level at the time of use can be applied.

[0045] (1.2. Application Example to Microscope System) The information processing system described above may be realized as, for example, a microscope system. Therefore, subsequently, with reference to FIG. 5, a configuration example of the microscope system when the information processing system according to the present embodiment is realized as a microscope system will be described.

[0046] As shown in FIG. 5, the microscope system according to the present embodiment includes a microscope 101 and a data processing unit 107.

[0047] The microscope 101 includes a stage 102, an optical system 103, a light source 104, a stage drive unit 105, a light source drive unit 106, and a fluorescence signal acquisition unit 112.

[0048] The stage 102 has a placement surface on which the fluorescently stained specimen 30 can be placed, and is movable in a parallel direction (x - y plane direction) and a perpendicular direction (z - axis direction) with respect to the placement surface by the drive of the stage drive unit 105. The fluorescently stained specimen 30 has a thickness of, for example, several μm to several tens of μm in the Z direction, and is sandwiched between a slide glass SG and a cover glass (not shown) and fixed by a predetermined fixing method.

[0049] The optical system 103 is disposed above the stage 102. The optical system 103 includes an objective lens 103A, an imaging lens 103B, a dichroic mirror 103C, an emission filter 103D, and an excitation filter 103E. The light source 104 is, for example, a light bulb such as a mercury lamp or an LED (Light Emitting Diode), and irradiates excitation light for the fluorescent label attached to the fluorescently stained specimen 30 by the drive of the light source drive unit 106.

[0050] When obtaining a fluorescence image of the fluorescently stained specimen 30, the excitation filter 103E generates excitation light by transmitting only the light of the excitation wavelength that excites the fluorescent dye among the light emitted from the light source 104. The dichroic mirror 103C reflects the excitation light transmitted through the excitation filter and guides it to the objective lens 103A. The objective lens 103A condenses the excitation light onto the fluorescently stained specimen 30. Then, the objective lens 103A and the imaging lens 103B magnify the image of the fluorescently stained specimen 30 at a predetermined magnification and form the magnified image on the imaging surface of the fluorescence signal acquisition unit 112.

[0051] When the fluorescently stained specimen 30 is irradiated with excitation light, the stain bound to each tissue of the fluorescently stained specimen 30 emits fluorescence. This fluorescence passes through the dichroic mirror 103C via the objective lens 103A and reaches the imaging lens 103B via the emission filter 103D. The emission filter 103D absorbs the light that has passed through the excitation filter 103E and has been magnified by the objective lens 103A, and transmits only a part of the colored light. The image of the colored light from which the external light has been lost is magnified by the imaging lens 103B as described above and formed on the fluorescence signal acquisition unit 112.

[0052] The data processing unit 107 is configured to drive the light source 104, acquire a fluorescence image of the fluorescently stained specimen 30 using the fluorescence signal acquisition unit 112, and perform various processes using this image. More specifically, the data processing unit 107 can function as part or all of the information acquisition unit 111, storage unit 120, processing unit 130, display unit 140, control unit 150, operation unit 160 of the information processing apparatus 100, or the database 200, which were described with reference to FIG. 1. For example, by functioning as the control unit 150 of the information processing apparatus 100, the data processing unit 107 controls the driving of the stage driving unit 105 and the light source driving unit 106, and controls the acquisition of the spectrum by the fluorescence signal acquisition unit 112. Further, by functioning as the processing unit 130 of the information processing apparatus 100, the data processing unit 107 generates a fluorescence spectrum, separates the fluorescence spectrum for each fluorescent substance, and generates image information based on the separation result.

[0053] Above, the configuration example of the microscope system in the case where the information processing system according to the present embodiment is realized as a microscope system has been described. Note that the above configuration described with reference to FIG. 5 is merely an example, and the configuration of the microscope system according to the present embodiment is not limited to such an example. For example, the microscope system does not necessarily have to include all of the configurations shown in FIG. 5, or may include a configuration not shown in FIG. 5.

[0054] (1.3. Regarding the least squares method) Here, the least squares method used in the color separation process by the separation processing unit 132 will be described. The least squares method calculates the color mixing ratio by fitting a reference spectrum to the fluorescence spectrum, which is the pixel value of each pixel in the input stained specimen image. The color mixing ratio is an index indicating the degree to which each substance is mixed. The following equation (1) represents the residual obtained by subtracting from the fluorescence spectrum (Signal) the reference spectrum (St), the fluorescence reference spectrum, and the autofluorescence reference spectrum mixed at the color mixing ratio a. Note that "Signal(1×number of channels)" in equation (1) indicates that there are as many fluorescence spectra (Signal) as the number of wavelength channels. For example, Signal is a matrix representing one or more fluorescence spectra. Also, "St(number of substances×number of channels)" indicates that there are as many reference spectra as the number of wavelength channels for each substance (fluorescent substance and autofluorescent substance). For example, St is a matrix representing one or more reference spectra. Also, "a(1×number of substances)" indicates that the color mixing ratio a is provided for each substance (fluorescent substance and autofluorescent substance). For example, a is a matrix representing the color mixing ratio of each reference spectrum in the fluorescence spectrum.

[0055]

Equation

[0056] Then, the separation processing unit 132 calculates the color mixing ratio a of each substance at which the sum of the squares of the residual equation (1) is minimized. The sum of the squares of the residuals is minimized when the result of the partial derivative of equation (1) representing the residual with respect to the color mixing ratio a is 0. Therefore, the separation processing unit 132 calculates the color mixing ratio a of each substance at which the sum of the squares of the residuals is minimized by solving the following equation (2). Note that "St'" in equation (2) indicates the transposed matrix of the reference spectrum St. Also, "inv(St*St')" indicates the inverse matrix of St*St'.

[0057]

Equation

[0058] Here, specific examples of each value in the above formula (1) are shown in the following formulas (3) to (5). In the examples of formulas (3) to (5), a case is shown where, in the fluorescence spectrum (Signal), the reference spectra (St) of three substances (the number of substances is 3) are mixed at different mixing ratios a.

[0059]

Number

[0060]

Number

[0061]

Number

[0062] And, specific examples of the calculation results of the above formula (2) based on the values of formulas (3) and (5) are shown in the following formula (6). As shown in formula (6), it can be seen that "a = (3 2 1)" (that is, the same value as the above formula (4)) is correctly calculated as the calculation result.

[0063]

Number

[0064] Note that, as described above, the separation processing unit 132 may extract the spectrum for each fluorescent substance from the fluorescence spectrum by performing calculations related to the weighted least square method instead of the least square method. In the weighted least square method, weights are assigned to emphasize errors at low signal levels by utilizing the fact that the noise of the fluorescence spectrum (Signal), which is a measured value, follows a Poisson distribution. However, an upper limit value at which no weighting is performed in the weighted least square method is defined as the Offset value. The Offset value is determined by the characteristics of the sensor used for measurement, and separate optimization is required when an image sensor is used as the sensor. When the weighted least square method is performed, the reference spectrum St in the above equations (1) and (2) is replaced with St_ represented by the following equation (7). Note that the following equation (7) means that each element (each component) of St represented by a matrix is divided (in other words, element-wise division) by each corresponding element (each component) in the "Signal + Offset value", also represented by a matrix, to calculate St_.

[0065]

Equation

[0066] Here, when the Offset value is 1 and the values of the reference spectrum St and the fluorescence spectrum Signal are represented by the above equations (3) and (5) respectively, a specific example of St_ represented by the above equation (7) is shown in the following equation (8).

[0067]

Equation

[0068] And a specific example of the calculation result of the color mixing ratio a in this case is shown in the following equation (9). As shown in equation (9), it can be seen that "a = (3 2 1)" is correctly calculated as the calculation result.

[0069]

Equation

[0070] (1.4. Non-negative Matrix Factorization (NMF)) Next, the non-negative matrix factorization (NMF) used by the separation processing unit 132 for extracting the autofluorescence spectrum and / or the fluorescence spectrum will be described. However, it is not limited to non-negative matrix factorization (NMF), and singular value decomposition (SVD), principal component analysis (PCA), etc. may be used.

[0071] FIG. 3 is a diagram for explaining the outline of NMF. As shown in FIG. 3, NMF decomposes a non-negative matrix A of N rows and M columns (N×M) into a non-negative matrix W of N rows and k columns (N×k) and a non-negative matrix H of k rows and M columns (k×M). The matrices W and H are determined such that the mean squared residual D between the matrix A and the product (W*H) of the matrices W and H is minimized. In the present embodiment, the matrix A corresponds to the spectrum before the autofluorescence reference spectrum is extracted (N is the number of pixels, and M is the number of wavelength channels), and the matrix H corresponds to the extracted autofluorescence reference spectrum (k is the number of autofluorescence reference spectra (in other words, the number of autofluorescent substances), and M is the number of wavelength channels). Here, the mean squared residual D is represented by the following equation (10). Note that "norm(D, 'fro')" refers to the Frobenius norm of the mean squared residual D.

[0072] [Equation]

[0073] The factorization in NMF uses an iterative method starting from random initial values for the matrices W and H. In NMF, the value of k (the number of autofluorescence reference spectra) is essential, but the initial values of the matrices W and H are not essential and can be set as an option. When the initial values of the matrices W and H are set, the solution becomes constant. On the other hand, when the initial values of the matrices W and H are not set, these initial values are set randomly, and the solution does not become constant.

[0074] The specimen 20 differs in nature and autofluorescence spectrum according to the type of tissue used, the type of disease targeted, the attributes of the subject, or the lifestyle of the subject, etc. Therefore, as described above, the information processing apparatus 100 according to the second embodiment can realize a more accurate color separation process by actually measuring the autofluorescence reference spectrum for each specimen 20.

[0075] Note that the matrix A, which is the input of NMF, is a matrix having the same number of rows as the number of pixels N (= Hpix × Vpix) of the stained specimen image and the same number of columns as the number of wavelength channels M as described above. Therefore, when the number of pixels of the stained specimen image is large or the number of wavelength channels M is large, the matrix A becomes a very large matrix, increasing the calculation cost of NMF and lengthening the processing time.

[0076] In such a case, for example, as shown in FIG. 4, by clustering into the number of classes N (<Hpix × Vpix) with the number of pixels N (= Hpix × Vpix) of the stained image specified, it is possible to suppress the redundancy of the processing time due to the enlargement of the matrix A.

[0077] In clustering, for example, among the stained images, spectra similar in the wavelength direction or intensity direction are classified into the same class. As a result, an image with a smaller number of pixels than the stained image is generated, and thus it is possible to reduce the scale of the matrix A' with this image as the input.

[0078] (Image generation unit 133) The image generation unit 133 is configured to generate image information based on the separation result of the fluorescence spectrum by the separation processing unit 132. For example, the image generation unit 133 can generate image information using the fluorescence spectrum corresponding to one or more fluorescent substances, or generate image information using the autofluorescence spectrum corresponding to one or more autofluorescent substances. Note that the number and combination of fluorescent substances (molecules) or autofluorescent substances (molecules) used by the image generation unit 133 for generating image information are not particularly limited. Also, when various processes (for example, segmentation or calculation of S / N value, etc.) are performed using the separated fluorescence spectrum or autofluorescence spectrum, the image generation unit 133 may generate image information indicating the results of those processes.

[0079] (Display unit 140) The display unit 140 is configured to present to the operator by displaying the image information generated by the image generation unit 133 on a display. Note that the type of display used as the display unit 140 is not particularly limited. Also, although not described in detail in this embodiment, the image information generated by the image generation unit 133 may be presented to the operator by being projected by a projector or printed by a printer (in other words, the output method of the image information is not particularly limited).

[0080] (Control unit 150) The control unit 150 is a functional configuration that comprehensively controls all the processes performed by the information processing apparatus 100. For example, based on the operation input by the operator performed via the operation unit 160, the control unit 150 controls the start and end, etc. of various processes (for example, the adjustment process of the placement position of the fluorescently stained specimen 30, the irradiation process of the excitation light on the fluorescently stained specimen 30, the spectrum acquisition process, the generation process of the autofluorescence component correction image, the color separation process, the image information generation process, and the image information display process, etc.) as described above. Note that the control content of the control unit 150 is not particularly limited. For example, the control unit 150 may control processes generally performed in a general-purpose computer, a PC, a tablet PC, etc. (for example, processes related to an OS (Operating System)).

[0081] (1.5. Characteristic Configuration of the First Embodiment) Above, the configuration example and application example of the information processing system according to this embodiment have been described. Subsequently, the characteristic configuration of this embodiment will be described in detail below with reference to the drawings.

[0082] As described above, in this embodiment, in order to distinguish the fluorescence signal derived from the autofluorescent substance, which is a problem when photographing the tissue sample, from the fluorescence signal derived from the fluorescent substance to be analyzed, the spectral information (autofluorescent component (spectrum)) derived from the autofluorescent substance obtained when performing color separation processing on the stained specimen image is used to generate a pseudo autofluorescent component image (autofluorescent component correction image), and this is used to enable quantitative analysis of the stained specimen image.

[0083] More specifically, in this embodiment, based on the autofluorescent component extracted from the stained specimen image, an autofluorescent component correction image is generated using the corresponding autofluorescent reference spectrum, and the stained specimen image is processed using the autofluorescent component correction image to generate a fluorescence component image with higher-precision color separation. Note that the generated fluorescence component image may be displayed on the display unit 140, or predetermined processing (analysis processing, etc.) may be executed by the processing unit 130 or other configurations (for example, an analysis device connected via a network). Note that the predetermined processing may be, for example, processing such as detecting specific cells.

[0084] (1.5.1. Basic Operation Example) FIG. 6 is a flowchart showing a basic operation example of the information processing system according to this embodiment. Note that the following operations are executed, for example, by each unit operating under the control of the control unit 150.

[0085] As shown in FIG. 6, in the basic operation according to this embodiment, first, the information acquisition unit 111 of the acquisition unit 110 captures an image of the fluorescently stained specimen 30 to acquire a stained specimen image (step S101). The stained specimen image thus acquired is stored, for example, in the information storage unit 121 of the storage unit 120.

[0086] Further, the information acquisition unit 111 acquires reagent information and specimen information from the database 200 connected via the network (step S102). As described above, here, the specimen information includes information regarding the autofluorescence reference spectrum of one or more autofluorescent substances in the specimen 20, and the reagent information includes information regarding the fluorescence reference spectrum of the fluorescent substance in the fluorescently stained specimen 30. The acquired reagent information and specimen information are stored, for example, in the information storage unit 121 in the storage unit 120.

[0087] Next, the separation processing unit 132 acquires the stained specimen image, the reagent information, and the specimen information stored in the information storage unit 121, and performs color separation processing of the stained specimen image by performing fitting of the autofluorescence reference spectrum using, for example, the least squares method on the acquired stained specimen image (step S103). By this color separation processing, a fluorescence component image and an autofluorescence component image are generated.

[0088] Next, the separation processing unit 132 generates an autofluorescence component correction image using the autofluorescence component image generated in step S103 and the autofluorescence reference spectrum included in the specimen information acquired in step S102 (step S104). The generation of the autofluorescence component correction image will be described in more detail later.

[0089] Next, the separation processing unit 132 processes the stained specimen image using the autofluorescence component correction image to generate a fluorescence component image (step S105). In this way, by removing the autofluorescence component included in the stained specimen image using the generated autofluorescence component correction image using the autofluorescence reference spectrum, it is possible to generate a fluorescence component image with improved color separation accuracy, that is, with a reduced residual amount of the autofluorescence component (correction unit).

[0090] Then, the separation processing unit 132 transmits the generated fluorescence component image to the image generation unit 133, an external server, or the like (step S106). Thereafter, this operation ends.

[0091] (1.5.2. Example of autofluorescence component correction image generation flow) Next, an operation example for generating the autofluorescence component image described in step S104 of FIG. 6 will be described. FIG. 7 is a flowchart showing an operation example for generating a pseudo-autofluorescence component image according to the present embodiment. FIG. 8 is a diagram showing an example of the autofluorescence component image generated in step S103 of FIG. 6, and FIG. 9 is a diagram showing an example of the autofluorescence reference spectrum included in the specimen information acquired in step S102 of FIG. 6. Further, FIG. 10 is a diagram for explaining step S112 of FIG. 7, and FIG. 11 is a diagram for explaining step S114 of FIG. 7.

[0092] As shown in FIG. 7, in the autofluorescence component correction image generation flow, the separation processing unit 132 first selects one unselected image (this is defined as the autofluorescence component image of the autofluorescence channel CHn (n is a natural number)) from the autofluorescence component images generated in step S103 of FIG. 6 (see FIG. 8) (step S111). Note that the autofluorescence channel may be identification information assigned to each autofluorescence.

[0093] Next, as shown in FIG. 10, the separation processing unit 132 generates a spectral image related to the autofluorescence channel CHn from the autofluorescence component image of the autofluorescence channel CHn selected in step S111 and the autofluorescence reference spectrum of the autofluorescence channel CHn (see FIG. 9) included in the specimen information acquired in step S102 of FIG. 6 (step S112). The above-described NMF can be used to generate this spectral image.

[0094] Next, the separation processing unit 132 determines whether all the autofluorescence component images have been selected in step S111 (step S113). If not selected (NO in step S113), it returns to step S111, selects an unselected autofluorescence component image, and executes the subsequent operations.

[0095] On the other hand, when all the autofluorescence component images have been selected in step S111 (YES in step S113), as shown in FIG. 11, the separation processing unit 132 adds the spectral images of the respective autofluorescence channels CH generated in the repeated step S112 (step S114). Thereby, an autofluorescence component correction image is generated. Thereafter, the separation processing unit 132 ends the operation shown in FIG. 7.

[0096] (1.6. Extraction and analysis of analysis target region) Next, the extraction process of the analysis target region from the fluorescence component image according to the present embodiment and its analysis will be described using a comparative example. FIG. 12 is a flowchart for explaining the extraction process of the analysis target region according to the comparative example and its analysis, and FIG. 13 is a schematic diagram for explaining the extraction process of the analysis target region according to the comparative example shown in FIG. 12 and its analysis. FIG. 14 is a flowchart for explaining the extraction process of the analysis target region according to the present embodiment and its analysis, and FIG. 15 is a schematic diagram for explaining the extraction process of the analysis target region according to the present embodiment shown in FIG. 14 and its analysis.

[0097] (Comparative example) As shown in FIGS. 12 and 13, in the comparative example, first, a stained section is imaged to obtain a stained specimen image, and an unstained tissue section (hereinafter also referred to as an unstained tissue section or an unstained section) adjacent to or close to the stained section is imaged to obtain an image of the unstained section (hereinafter referred to as an unstained specimen image) (step S901). Subsequently, by performing color separation processing using the obtained stained specimen image and unstained specimen image, a fluorescence component image and an autofluorescence component image are generated (step S902).

[0098] Next, based on the autofluorescence component image, a threshold value of the number of antibodies for extracting a region (extraction target region) that is a candidate for the region to be analyzed (analysis target region) is set (step S903). The threshold value of the number of antibodies may be, for example, a threshold value for the pixel value of each pixel in the fluorescence component image. The extraction target region may be the region hatched with dots in step S903 of FIG. 13.

[0099] Next, the pixel value of each pixel in the stained specimen image is compared with the threshold value, and based on the comparison result, a mask image for extracting the analysis target area is generated (step S904). As a result, for example, a binary mask in which the pixel value is '1' if it is greater than or equal to the threshold value and '0' if it is less than the threshold value is generated as the mask image. Note that the pixel or area to which '1' is assigned may be a pixel or area that is a candidate for analysis, and the pixel or area to which '0' is assigned may be a pixel or area excluded from the analysis target.

[0100] Next, by performing a logical product operation between each pixel of the mask image generated in step S904 and each pixel of the fluorescence component image, the analysis target area is extracted from the fluorescence component image (step S905). As a result, an extraction image in which the area labeled with the fluorescent dye antibody (morphological information of the analysis target) is extracted using the value extracted from the autofluorescence component image as the threshold value is obtained.

[0101] Thereafter, for example, the analysis target area is sent to an analysis device such as an external server and quantitatively evaluated (step S906).

[0102] (This Embodiment) On the other hand, as shown in FIGS. 14 and 15, in this embodiment, first, a stained specimen image is acquired by imaging a stained section (step S1). That is, in this embodiment, it is not necessary to image an unstained section to acquire an unstained specimen image.

[0103] Next, in this embodiment, a fluorescence component image is generated by performing color separation processing on the stained specimen image (step S2). Also, in this embodiment, an autofluorescence component correction image is generated using the autofluorescence component image generated by the color separation processing and the autofluorescence reference spectrum included in the acquired specimen information (step S3).

[0104] Next, based on the autofluorescence component-corrected image, a threshold value for the number of antibodies for extracting an extraction target region that is a candidate for the analysis target region is set (step S4). Similar to the description using FIGS. 12 and 13, the threshold value for the number of antibodies may be, for example, a threshold value for the pixel value of each pixel in the fluorescence component image, and the extraction target region may be the region hatched with dots in step S4 of FIG. 15.

[0105] Thereafter, similar to steps S904 to S906 in FIGS. 12 and 13, the pixel value of each pixel in the stained specimen image is compared with the threshold value to generate a mask image (step S5). Subsequently, by performing a logical product operation between the mask image generated in step S5 and the fluorescence component image, the analysis target region is extracted from the fluorescence component image (step S6). As a result, an extracted image in which the region labeled with the fluorescent dye antibody (morphological information of the analysis target) is extracted using the value extracted from the autofluorescence component-corrected image as the threshold value is obtained. Then, the extracted analysis target region is sent to an analysis device such as an external server and quantitatively evaluated (step S7).

[0106] (1.7. Operations and Effects) As described above, according to the present embodiment, in the extraction process of the analysis target region, a value defined by the user from the autofluorescence component-corrected image or a numerical value having objectivity is set as the threshold value, and the analysis target region is extracted using the mask image (binary mask) generated based on the magnitude relationship between the pixel value in the stained specimen image and the threshold value.

[0107] In this way, by using the autofluorescence component-corrected image for setting the threshold value, in addition to being able to omit the labor of separately imaging a non-stained section to obtain a non-stained specimen image and the labor of generating an autofluorescence component image, etc., the spatial information in the stained specimen image (the signal distributions such as those derived from autofluorescence components and noise are the same) is retained. In other words, since the threshold value can be set based on the autofluorescence component-corrected image having a correlation with the stained specimen image and the fluorescence component image, it is possible to achieve the effect of being able to set a threshold value having a correlation with the stained specimen image and the fluorescence component image.

[0108] <2. Second Embodiment> Next, the second embodiment will be described. In the first embodiment described above, the threshold value for all the pixels of the stained specimen image was uniquely set from the generated autofluorescence component corrected image. In contrast, in the second embodiment, a threshold distribution image in which thresholds that are not necessarily the same are spatially distributed is generated by setting different threshold values for each pixel. In this embodiment, the configuration and basic operations of the information processing system may be the same as those of the information processing system according to the first embodiment, and thus detailed description thereof will be omitted here.

[0109] (2.1. Extraction of Analysis Target Region and Its Analysis) FIG. 16 is a flowchart for explaining the extraction process of the analysis target region and its analysis according to this embodiment, and FIG. 17 is a schematic diagram for explaining the extraction process of the analysis target region and its analysis according to this embodiment shown in FIG. 16.

[0110] As shown in FIGS. 16 and 17, in this embodiment, first, in the same manner as the operations described with reference to steps S1 to S3 in FIGS. 14 and 15 in the first embodiment, a stained specimen image is acquired by imaging a stained section (step S21), a fluorescence component image is generated by performing color separation processing on the stained specimen image (step S22), and an autofluorescence component corrected image is generated using the autofluorescence component image generated by the color separation processing and the autofluorescence reference spectrum included in the acquired specimen information (step S23).

[0111] Next, in this embodiment, a threshold distribution image in which different threshold values for the number of antibodies are set for each pixel is generated based on the autofluorescence component corrected image (step S24). As a method for generating a threshold distribution image in which the threshold values are different for each pixel, for example, a method of multiplying a coefficient preset for the pixel value of each pixel of the autofluorescence component corrected image can be adopted. Here, the coefficient may be determined, for example, within a numerically range considered appropriate. As a specific example, a value based on the maximum value or the average value of the pixel values in the autofluorescence component corrected image may be used as the coefficient.

[0112] Next, in the present embodiment, the pixel value of each pixel in the stained specimen image is compared with the threshold value of the corresponding pixel in the threshold distribution image, and based on the comparison result, a mask image for extracting the analysis target region is generated (step S25). As a result, for example, a binary mask in which the pixel value is '1' if it is equal to or greater than the threshold value and '0' if it is less than the threshold value is generated as the mask image. Similar to the first embodiment, the pixel or region to which '1' is assigned may be a pixel or region that is a candidate for analysis, and the pixel or region to which '0' is assigned may be a pixel or region excluded from the analysis target. Also, in the comparison result for each pixel shown in step S25 of FIG. 17, the pixels hatched with dots may be, for example, pixels whose pixel values were equal to or greater than the corresponding threshold value.

[0113] Thereafter, similar to steps S6 to S7 in FIGS. 14 and 15, by performing a logical product operation between the mask image generated in S25 and the fluorescence component image, the analysis target region is extracted from the fluorescence component image (step S26). As a result, an extraction image is obtained in which the region labeled with the fluorescent dye antibody (morphological information of the analysis target) is extracted using the value extracted from the autofluorescence component correction image as the threshold value. Then, the extracted analysis target region is sent to an analysis device such as an external server and quantitatively evaluated (step S27).

[0114] (2.2. Function and Effect) As described above, in the present embodiment, when generating a binary mask for extracting only the target region from the stained specimen image, instead of uniquely setting the threshold value from the autofluorescence component correction image as in the first embodiment, a threshold distribution image in which different threshold values can be spatially distributed is generated. As a result, it becomes possible to set the threshold value using the pixel values of the stained specimen image and the corresponding autofluorescence component correction image, so that it is possible to extract the analysis target region using a threshold distribution image that retains the spatial information of the noise that can occur in the system, such as the distribution of the autofluorescence component for each tissue region and the influence of the hardware. As a result, it becomes possible to further improve the analysis accuracy in fluorescence observation.

[0115] In addition, other configurations, operations, and effects may be the same as those in the above-described embodiments, and thus detailed descriptions thereof are omitted here.

[0116] <3. Third Embodiment> Next, a third embodiment will be described. In the above-described embodiments, the case of generating a self-fluorescence component correction image from a stained specimen image has been exemplified. In contrast, in this embodiment, in addition to the self-fluorescence component correction image, a pseudo fluorescence component image (hereinafter referred to as a fluorescence component correction image) is also generated, and the spectral intensity ratio thereof is calculated to obtain, as information, the ratio of the fluorescence dye intensity to the self-fluorescence. In this embodiment, the configuration and basic operation of the information processing system may be the same as those of the information processing system according to the above-described embodiments, and thus detailed descriptions thereof are omitted here.

[0117] (3.1. Generation and Analysis of Spectral Intensity Ratio Image) In this embodiment, the spectral intensity ratio is generated, for example, as spatially distributed information, that is, image data (spectral intensity ratio image). FIG. 18 is a flowchart for explaining the generation process and analysis of the spectral intensity ratio image according to this embodiment.

[0118] As shown in FIG. 18, in this embodiment, first, a self-fluorescence component correction image is generated (step S104) by executing the same operations as steps S101 to S104 in FIG. 6 in the first embodiment.

[0119] Next, in this embodiment, a fluorescence component correction image is generated (step S301) by applying, for example, the same operations as those when generating the self-fluorescence component correction image. Note that the fluorescence component correction image generation flow can be implemented by replacing the self-fluorescence component image with the fluorescence component image and replacing the self-fluorescence reference spectrum with the fluorescence reference spectrum in the operations described with reference to FIG. 7 in the first embodiment, and thus detailed descriptions thereof are omitted here.

[0120] Next, the separation processing unit 132 generates a spectral intensity ratio image by calculating the ratio (spectral intensity ratio) of corresponding pixels in the fluorescence component corrected image and the autofluorescence component corrected image (step S302).

[0121] Thereafter, for example, the spectral intensity ratio image is sent to an analysis device such as an external server and quantitatively evaluated (step S303).

[0122] (3.2. Method for generating spectral intensity image) In the present embodiment, the spectral intensity ratio image may be generated, for example, by generating a spectral intensity image indicating the spectral intensity of each pixel of the fluorescence component corrected image and a spectral intensity image indicating the spectral intensity of each pixel of the autofluorescence component corrected image, and calculating the ratio of the spectral intensities of corresponding pixels.

[0123] Here, the method for generating the spectral intensity image of each of the fluorescence component corrected image and the autofluorescence component corrected image is not particularly limited. Therefore, two generation methods are exemplified below.

[0124] (3.2.1 First generation method) FIG. 19 is a diagram for explaining the first generation method. As shown in FIG. 19, in the first generation method, a spectral intensity image is generated by summing up the pixel values for each wavelength channel of the fluorescence component corrected image / autofluorescence component corrected image in the data cube structure in the wavelength direction (step S31).

[0125] (3.2.2 Second generation method) FIG. 20 is a diagram for explaining the second generation method. As shown in FIG. 20, in the second generation method, a spectral intensity image is generated by extracting the pixel value with the maximum value in the wavelength direction from among the pixel values for each wavelength channel of the fluorescence component corrected image / autofluorescence component corrected image in the data cube structure (step S32).

[0126] (3.3. Function and effect) As described above, in the present embodiment, also in the case of the fluorescent component, a corrected fluorescent component image is generated by multiplying the fluorescence reference spectrum extracted by color separation processing for each pixel of the fluorescent component image, and spectral intensity images of the corrected fluorescent component image and the corrected autofluorescent component image are obtained. Then, from the spectral intensity ratio between each pixel of the corrected fluorescent component image and each pixel of the corrected autofluorescent component image, a spectral intensity ratio image, which is information spatially indicating the ratio of the fluorescent dye intensity to the autofluorescence, is generated. The spectral intensity ratio image generated in this way can be used for performance evaluation of the measurement system (corresponding to the acquisition unit 110) in the information processing system, color separation accuracy evaluation in the processing unit 130, and further for design / evaluation of the fluorescent reagent panel and the like.

[0127] Further, according to the present embodiment, for example, the spectral intensity ratio image generated as described above is associated with image acquisition conditions such as the shooting conditions when the acquisition unit 110 acquires the stained specimen image and the labeling conditions (staining conditions) of the fluorescent reagent 10 with respect to the specimen 20, and recorded in the storage unit 120 or the like, and can also be used as reference information when acquiring the stained specimen image using the same acquisition unit 110. Also, by storing the spectral intensity ratio image and the image acquisition conditions in a server on the network and sharing them with other information processing systems or the information processing apparatus 100, they can also be used as reference information when acquiring the stained specimen image using the same type of acquisition unit 110 in other information processing systems. Specifically, for example, the spectral intensity ratio image and the image acquisition conditions stored in the storage unit 120, the server, etc. may be used for inter-device baseline correction and calibration curve correction when using the same type of acquisition unit 110.

[0128] Note that other configurations, operations, and effects may be the same as those of the above-described embodiment, and thus detailed description thereof is omitted here.

[0129] <4. Fourth Embodiment> Next, a fourth embodiment will be described. In the above-described embodiments, for example, NMF, SVD, PCA, etc. have been used for extracting the autofluorescence spectrum and / or the fluorescence spectrum. In contrast, in this embodiment, instead of these, a case of extracting the autofluorescence spectrum and / or the fluorescence spectrum using machine learning will be described. Note that in this embodiment, the configuration and the basic operation of the information processing system may be the same as those of the information processing system according to the above-described embodiments, and thus detailed description thereof will be omitted here.

[0130] (4.1. Generation of Fluorescent Component Image Using Machine Learning) FIG. 21 is a flowchart for explaining the generation process of a fluorescent component image using machine learning according to this embodiment, and FIG. 22 is a schematic diagram for explaining the generation process of a fluorescent component image using machine learning according to this embodiment shown in FIG. 21.

[0131] As shown in FIGS. 21 and 22, in this embodiment, first, by executing the same operations as steps S1 to S3 in FIG. 14 in the second embodiment, an autofluorescent component corrected image is generated (steps S401 to S403).

[0132] Next, in this embodiment, the fluorescence component image generated by the color separation process in step S401 and the pseudo-autofluorescence component image generated in step S403 are input to the machine learning unit 401, and the machine learning unit 401 executes deep learning of unsupervised learning with the fluorescence component image and the pseudo-autofluorescence component image as input images, thereby extracting feature amounts (for example, autofluorescence spectrum) such as the signal intensity and distribution derived from the autofluorescence component from the stained specimen image (step S404). Note that as the deep learning, various machine learnings such as DNN (Deep Neural Network), CNN (Convolutional Neural Network), and RNN (Recurrent Neural Network) can be applied. Further, the machine learning unit 401 may be implemented, for example, in the processing unit 130 or the like in the information processing apparatus 100, or may be implemented in a cloud server or the like connected to the information processing apparatus 100 via a predetermined network.

[0133] Then, in this embodiment, using the feature amounts extracted in step S404, a fluorescence component image with an even lower remaining amount of the autofluorescence component and higher accuracy is generated (step S405), and this operation is terminated.

[0134] (4.2. Actions and effects) As described above, in this embodiment, by using the fluorescence component image generated by the color separation process for the stained specimen image and the autofluorescence component correction image generated therefrom as input images and applying unsupervised machine learning, feature amounts such as the signal intensity and distribution derived from the autofluorescence component are extracted from the stained specimen image, and a fluorescence component image with higher accuracy is further generated based on these extracted feature amounts. In this way, by executing machine learning that originates from the stained specimen image, in other words, that takes the autofluorescence component correction image generated from the stained specimen image as input, the association between the input image and the output image of the machine learning becomes easy, and thus it is possible to achieve the effect of easily enhancing the learning effect.

[0135] Note that other configurations, operations, and effects may be the same as those in the above-described embodiment, and thus detailed description thereof is omitted here.

[0136] <5. Configuration Example of Measurement System> Next, a configuration example of the measurement system in the information processing apparatus 100 according to the above-described embodiment (hereinafter simply referred to as the embodiment) will be described. FIG. 23 is a diagram showing an example of the measurement system of the information processing system according to the embodiment. Note that FIG. 23 shows an example of the measurement system when photographing a wide field of a fluorescently stained specimen 30 (or a specimen 20 that is an unstained specimen), such as WSI (Whole Slide Imaging). However, the measurement system according to the embodiment is not limited to the measurement system illustrated in FIG. 23, and may be variously modified as long as it is a measurement system capable of acquiring image data (hereinafter referred to as wide field image data) of sufficient resolution of the entire imaging region or a necessary region (also referred to as a region of interest) thereof, such as a measurement system that photographs the entire imaging region or a necessary region thereof at once, or a measurement system that acquires an image of the entire imaging region or the region of interest by line scanning.

[0137] As shown in FIG. 23, the measurement system according to the embodiment includes, for example, an information processing apparatus 100, an XY stage 501, an excitation light source 510, a beam splitter 511, an objective lens 512, a spectroscope 513, and a photodetector 514.

[0138] The XY stage 501 is a stage on which the fluorescently stained specimen 30 (or specimen 20) to be analyzed is placed, and may be, for example, a stage movable in a plane (XY plane) parallel to the placement surface of the fluorescently stained specimen 30 (or specimen 20).

[0139] The excitation light source 510 is a light source for exciting the fluorescently stained specimen 30 (or specimen 20), and emits, for example, a plurality of excitation lights having different wavelengths along a predetermined optical axis.

[0140] The beam splitter 511 is composed of, for example, a dichroic mirror or the like, reflects the excitation light from the excitation light source 510, and transmits the fluorescence from the fluorescently stained specimen 30 (or specimen 20).

[0141] The objective lens 512 irradiates the fluorescence-stained specimen 30 (or specimen 20) on the XY stage 501 with the excitation light reflected by the beam splitter 511.

[0142] The spectroscope 513 is composed of one or more prisms, lenses, etc., and spectroscopically disperses the fluorescence emitted from the fluorescence-stained specimen 30 (or specimen 20), transmitted through the objective lens 512 and the beam splitter 511, in a predetermined direction.

[0143] The photodetector 514 detects the light intensity for each wavelength of the fluorescence dispersed by the spectroscope 513, and inputs the fluorescence signal (fluorescence spectrum and / or autofluorescence spectrum) thus obtained to the fluorescence signal acquisition unit 112 of the information processing apparatus 100.

[0144] In the configuration as described above, when the entire imaging area, such as in WSI, exceeds an area (hereinafter referred to as the field of view) that can be imaged at once, the XY stage 501 is moved for each imaging to move the field of view, and imaging of each field of view is sequentially performed. Then, by tiling the image data (hereinafter referred to as field-of-view image data) obtained by imaging each field of view, wide-field image data of the entire imaging area is generated. The generated wide-field image data is stored, for example, in the fluorescence signal storage unit 122. Note that the tiling of the field-of-view image data may be executed in the acquisition unit 110 of the information processing apparatus 100, may be executed in the storage unit 120, or may be executed in the processing unit 130.

[0145] Then, the processing unit 130 according to the embodiment executes the above-described processing on the obtained wide-field image data to obtain the coefficient C, that is, the fluorescence separation image for each fluorescent molecule (or the autofluorescence separation image for each autofluorescent molecule).

[0146] <6. Method for Calculating the Number of Fluorescent Molecules (or Antibodies)> Next, a method for calculating the number of fluorescent molecules (or the number of antibodies) in one pixel in the above-described embodiment will be described. FIG. 24 is a schematic diagram for explaining a method for calculating the number of fluorescent molecules (or the number of antibodies) in one pixel in the embodiment. In the example shown in FIG. 24, when the imaging device and the sample are arranged via the objective lens, assume that the size of the bottom surface of the sample corresponding to 1 [pixel] of the imaging device is 13 / 20 (μm) × 13 / 20 (μm). And assume that the thickness of the sample is 10 (μm). In that case, the volume (m 3 ) of this rectangular parallelepiped is represented by 13 / 20 (μm) × 13 / 20 (μm) × 10 (μm). Note that the volume (liter) is represented by 13 / 20 (μm) × 13 / 20 (μm) × 10 (μm) × 10 3 .

[0147] And assume that the concentration of the number of antibodies (which may be the number of fluorescent molecules) contained in the sample is uniform and is 300 (nM). Then, the number of antibodies per pixel is represented by the following formula (11).

[0148]

Equation

[0149] In this way, by calculating the number of fluorescent molecules or the number of antibodies in the fluorescently stained sample 30 as a result of the fluorescence separation process, the operator can compare the number of fluorescent molecules among a plurality of fluorescent substances or compare the data imaged under different conditions. Also, while the luminance (or fluorescence intensity) is a continuous value, the number of fluorescent molecules or the number of antibodies is a discrete value. Therefore, the information processing apparatus 100 according to the modified example can reduce the data amount by outputting the image information based on the number of fluorescent molecules or the number of antibodies.

[0150] <7. Hardware Configuration Example> Above, the embodiments of the present disclosure and their modifications have been described. Subsequently, with reference to FIG. 25, a hardware configuration example of the information processing apparatus 100 according to each embodiment and modification will be described. FIG. 25 is a block diagram showing a hardware configuration example of the information processing apparatus 100. Various processes by the information processing apparatus 100 are realized by the cooperation of software and the hardware described below.

[0151] As shown in FIG. 25, the information processing apparatus 100 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, and a host bus 904a. Further, the information processing apparatus 100 includes a bridge 904, an external bus 904b, an interface 905, an input device 906, an output device 907, a storage device 908, a drive 909, a connection port 911, a communication device 913, and a sensor 915. The information processing apparatus 100 may have a processing circuit such as a DSP or an ASIC instead of or together with the CPU 901.

[0152] The CPU 901 functions as an arithmetic processing unit and a control unit, and controls the overall operation within the information processing apparatus 100 according to various programs. Also, the CPU 901 may be a microprocessor. The ROM 902 stores programs and arithmetic parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901 and parameters that appropriately change during the execution. The CPU 901 can embody, for example, at least the processing unit 130 and the control unit 150 of the information processing apparatus 100.

[0153] The CPU 901, ROM 902, and RAM 903 are interconnected by a host bus 904a including a CPU bus or the like. The host bus 904a is connected to an external bus 904b such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 904. Note that it is not always necessary to separately configure the host bus 904a, the bridge 904, and the external bus 904b, and these functions may be implemented in one bus.

[0154] The input device 906 is realized by a device into which information is input by an implementer, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever. Further, the input device 906 may be, for example, a remote control device using infrared rays or other radio waves, or an external connection device such as a mobile phone or PDA corresponding to the operation of the information processing apparatus 100. Furthermore, the input device 906 may include, for example, an input control circuit that generates an input signal based on information input by the implementer using the above-described input means and outputs the input signal to the CPU 901. By operating the input device 906, the implementer can input various data to the information processing apparatus 100 or instruct a processing operation. The input device 906 can embody, for example, at least the operation unit 160 of the information processing apparatus 100.

[0155] The output device 907 is formed of a device capable of visually or auditorily notifying the acquired information to the implementer. Such devices include display devices such as a CRT display device, a liquid crystal display device, a plasma display device, an EL display device, and a lamp, acoustic output devices such as a speaker and headphones, and a printer device. The output device 907 can embody, for example, at least the display unit 140 of the information processing apparatus 100.

[0156] The storage device 908 is a device for storing data. The storage device 908 is realized by, for example, a magnetic storage unit device such as an HDD, a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 908 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. This storage device 908 stores programs executed by the CPU 901, various data, and various data acquired from the outside. The storage device 908 can, for example, embody at least the storage unit 120 of the information processing device 100.

[0157] The drive 909 is a reader / writer for a storage medium and is built in or externally attached to the information processing device 100. The drive 909 reads information recorded on a removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory that is mounted and outputs it to the RAM 903. The drive 909 can also write information to the removable storage medium.

[0158] The connection port 911 is an interface connected to an external device and is a connection port for an external device capable of data transmission, for example, by USB (Universal Serial Bus).

[0159] The communication device 913 is a communication interface formed by, for example, a communication device for connecting to the network 920. The communication device 913 is, for example, a communication card for wired or wireless LAN (Local Area Network), LTE (Long Term Evolution), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 913 may also be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various communications. This communication device 913 can transmit and receive signals, for example, in accordance with a predetermined protocol such as TCP / IP between the Internet and other communication devices.

[0160] In this embodiment, the sensor 915 includes a sensor capable of acquiring a spectrum (for example, an imaging device, etc.), and may also include other sensors (for example, an acceleration sensor, a gyro sensor, a geomagnetic sensor, a pressure sensor, a sound sensor, or a distance measuring sensor, etc.). The sensor 915 can, for example, embody at least the fluorescence signal acquisition unit 112 of the information processing apparatus 100.

[0161] Note that the network 920 is a wired or wireless transmission path for information transmitted from a device connected to the network 920. For example, the network 920 may include a public line network such as the Internet, a telephone line network, or a satellite communication network, and various LANs (Local Area Networks) including Ethernet (registered trademark), WANs (Wide Area Networks), etc. Further, the network 920 may include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network).

[0162] Above, a hardware configuration example capable of realizing the functions of the information processing apparatus 100 has been shown. Each of the above components may be realized using general-purpose members, or may be realized by hardware specialized for the functions of each component. Therefore, it is possible to appropriately change the hardware configuration to be used according to the technical level at the time of implementing the present disclosure.

[0163] Note that it is possible to create a computer program for realizing each function of the information processing apparatus 100 as described above and install it on a PC or the like. Further, a computer-readable recording medium storing such a computer program can also be provided. The recording medium includes, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc. Further, the above computer program may be distributed via a network, for example, without using a recording medium.

[0164] The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings. However, the technical scope of the present disclosure is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field of the present disclosure can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and these are also naturally understood to belong to the technical scope of the present disclosure.

[0165] In addition, the effects described in this specification are merely illustrative or exemplary and not restrictive. That is, the technology according to the present disclosure can exhibit other effects that are obvious to those skilled in the art from the description of this specification, together with or instead of the above effects.

[0166] Note that the following configurations also belong to the technical scope of the present disclosure. (1) A separation unit that separates a fluorescence image observed from a specimen labeled with one or more fluorescent dyes into a fluorescence component image including one or more fluorescence components and a autofluorescence component image including one or more autofluorescence components; A generation unit that generates an autofluorescence component correction image using the reference spectrum of each of one or more autofluorescent substances contained in the specimen and the autofluorescence component image; A processing unit that processes the fluorescence component image based on the autofluorescence component correction image; An information processing apparatus comprising the same. (2) The information processing apparatus according to (1) above, wherein the processing unit generates a fluorescence component correction image obtained by correcting the fluorescence component image using the autofluorescence component correction image. (3) The information processing apparatus according to (1) or (2) above, wherein the generation unit generates the autofluorescence component correction image by adding the results of multiplying the autofluorescence component image and the reference spectrum for each of the autofluorescent substances. (4) The information processing apparatus according to any one of (1) to (3) above, wherein the processing unit generates region information indicating a region labeled with the fluorescent dye from the autofluorescence component correction image, and processes the fluorescence component image using the generated region information. (5) The information processing apparatus according to (4) above, wherein the region information is a binary mask indicating a region labeled with the fluorescent dye. (6) The information processing apparatus according to (5) above, wherein the processing unit generates the binary mask based on a magnitude relationship between a threshold value set based on the autofluorescence component corrected image and a pixel value of each pixel in the fluorescence image. (7) The information processing apparatus according to (5) above, wherein the processing unit sets a threshold value for each pixel based on a pixel value of each pixel in the autofluorescence component corrected image, and generates the binary mask based on a magnitude relationship between the threshold value for each pixel and a pixel value of each pixel in the fluorescence image. (8) The information processing apparatus according to any one of (1) to (7) above, wherein the generation unit further generates a fluorescence component corrected image using a reference spectrum of each of the one or more fluorescent dyes and the fluorescence component image. (9) The information processing apparatus according to (8) above, further comprising an evaluation unit that evaluates at least one of measurement accuracy of a measurement system for acquiring the fluorescence image based on the fluorescence component corrected image and the autofluorescence component corrected image, separation performance of the fluorescence component image and the autofluorescence component image by the separation unit, and staining performance of a fluorescence reagent panel composed of the one or more fluorescent dyes. (10) An image generation unit that generates a spectral intensity ratio image representing a ratio of a spectral intensity of each pixel in the fluorescence component corrected image to a spectral intensity of each pixel in the autofluorescence component corrected image; A storage unit that associates and stores imaging conditions at the time of imaging the specimen to obtain the fluorescence image and the spectral intensity ratio image; The information processing apparatus according to (8) or (9) above, further comprising: (11) The information processing apparatus further comprises a correction unit that corrects the fluorescence component image based on the fluorescence image, the fluorescence component image, and the autofluorescence component corrected image, The processing unit processes the corrected fluorescence component image based on the autofluorescence component corrected image. The information processing apparatus according to any one of (1) to (10) above. (12) The correction unit estimates a feature amount of the autofluorescent substance in the fluorescent image from the fluorescent component image and the autofluorescent component correction image, and corrects the fluorescent component image based on the estimated feature amount. The information processing apparatus according to (11) above. (13) The correction unit estimates the feature amount by using a learned model that takes the fluorescent component image and the autofluorescent component correction image as inputs. The information processing apparatus according to (12) above. (14) The learned model is a model trained by unsupervised learning. The information processing apparatus according to (13) above. (15) The information processing apparatus according to any one of (1) to (14) above, further comprising an acquisition unit that acquires the fluorescent image by imaging the specimen. (16) The processing unit detects specific cells included in the fluorescent image based on the processed fluorescent component image. The information processing apparatus according to any one of (1) to (15) above. (17) Separating a fluorescent image observed from a specimen labeled with one or more fluorescent dyes into a fluorescent component image including one or more fluorescent components and an autofluorescent component image including one or more autofluorescent components, Generating an autofluorescent component correction image using a reference spectrum of each of one or more autofluorescent substances included in the specimen and the autofluorescent component image, Processing the fluorescent component image based on the autofluorescent component correction image, An information processing method including this. (18) A process of separating a fluorescent image observed from a specimen labeled with one or more fluorescent dyes into a fluorescent component image including one or more fluorescent components and an autofluorescent component image including one or more autofluorescent components, A process of generating an autofluorescent component correction image using a reference spectrum of each of one or more autofluorescent substances included in the specimen and the autofluorescent component image, A process of processing the fluorescence component image based on the autofluorescence component correction image, A program for causing a computer to execute. (19) A microscope system including a light source that irradiates light onto a specimen labeled with one or more fluorescent dyes, an imaging device that observes fluorescence emitted from the specimen irradiated with the light, and a program for executing processing on the fluorescence image acquired by the imaging device, wherein the program, when executed on a computer, separates the fluorescence image into a fluorescence component image including one or more fluorescence components and an autofluorescence component image including one or more autofluorescence components, generates an autofluorescence component correction image using the reference spectrum of each of one or more autofluorescent substances contained in the specimen and the autofluorescence component image, processes the fluorescence component image based on the autofluorescence component correction image, and causes the computer to execute, a microscope system. (20) The information processing apparatus according to (1) above, an analysis apparatus that is connected to the information processing apparatus via a predetermined network and analyzes the fluorescence component image processed by the information processing apparatus, and an analysis system including the same.

Explanation of Signs

[0167] 10 Fluorescent reagent 11 Reagent identification information 20 Specimen 21 Specimen identification information 30 Fluorescently stained specimen 100 Information processing apparatus 110 Acquisition unit 111 Information acquisition unit 112 Fluorescence signal acquisition unit 120 Storage unit 121 Information storage unit 122 Fluorescence signal storage unit 130 Processing unit 132 Separation processing unit 133 Image generation unit 140 Display unit 150 Control unit 160 Operation unit 200 Database 401 Machine learning unit

Claims

1. a processing unit that generates an image based on a pixel value of each pixel in a cell image observed from a cell labeled with one or more fluorescent dyes and a threshold value for the pixel value of each pixel; An information processing device comprising:

2. The image generated by the processing unit is a binary mask image. The information processing device according to claim 1 .

3. The processing unit generates the binary mask image based on a magnitude relationship between a set threshold value and a pixel value of each pixel in the cell image. The information processing device according to claim 2 .

4. a separation unit that separates an autofluorescence component image including one or more autofluorescence components based on the cell image; a generation unit that generates spectral information of the autofluorescence using the autofluorescence component image and reference spectra of one or more autofluorescent substances contained in the cell; Further preparation, The threshold value is set based on spectral information of the autofluorescence. The information processing device according to claim 1 .

5. The autofluorescence spectral information generated by the generation unit is an autofluorescence component corrected image. The information processing device according to claim 4.

6. the generating unit generates the autofluorescence component corrected image by adding a result of multiplying the autofluorescence component image by the reference spectrum for each of the autofluorescence substances. The information processing device according to claim 5 .

7. The separation unit further separates a fluorescent component image including one or more fluorescent components based on the cell image, and The processing unit processes the fluorescence component image based on a threshold value. The information processing device according to claim 4.

8. The threshold value is set based on spectral information of the autofluorescence. The information processing device according to claim 7.

9. The separation unit performs separation processing using a least squares method or a weighted least squares method. The information processing device according to claim 4.

10. generating an image based on a pixel value of each pixel in a cell image observed from a cell labeled with one or more fluorescent dyes and a threshold value for the pixel value of each pixel; An information processing method comprising the steps of:

Images