Information processing device, microscope system, and information processing method

Abstract

[Problem] To provide an advantageous technology for determining a positivity threshold used in the analysis of fluorescence spectra of stained specimens. [Solution] This information processing device comprises a threshold determination unit that determines a positivity threshold that is compared with image data of a plurality of image sections included in a stained fluorescent component image and is a criterion for deciding whether each of the plurality of image sections corresponds to a positive cell image. The threshold determination unit determines the positivity threshold on the basis of an unstained fluorescent component image.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a microscope system, and an information processing method.

Background Art

[0002] In recent years, with the development of cancer immunotherapy and the like, fluorescence conversion and multiplex labeling of immunostaining have been widely used. For example, a measurement method is known in which an autofluorescence spectrum is extracted from an unstained section of the same tissue block, and fluorescence separation of a stained section is performed using the autofluorescence spectrum.

[0003] Also, a method for detecting positive cells in a stained section based on image analysis of the stained section has been proposed. Patent Document 1 discloses a method for detecting positive cells in a stained tissue specimen. According to the detection method of Patent Document 1, a region stained above a detection threshold with respect to a standardized image of the stained tissue specimen is detected, and the number and coordinates of the center of gravity of positive cell images selected from the detected region are recorded.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In a cell counting method using flow cytometry for measuring the light intensity value of each cell, the influence of autofluorescence derived from tissue cells is small. Also, in flow cytometry, setting a positive threshold for detecting a target sample as positive using a histogram of intensity values (background values) detected in a control sample such as an unlabeled cell is relatively easy if sufficient light intensity can be detected.

[0006] On the other hand, when performing image analysis such as cell counting based on images of tissue specimens without using flow cytometry, background values ​​are high due to physical signals from nonspecific adsorption of reagents, hardware-derived noise, and autofluorescence signals from tissue components and mounting materials. Furthermore, when estimating the positive threshold using the background value of a negative control sample used in flow cytometry in image analysis of tissue specimens, the analysis may become unstable and the positive threshold may not be estimated appropriately.

[0007] In flow cytometry, the cell population is divided into a "stained cell group" and a "negative control group" for measurement. On the other hand, if the stained specimen and the negative control specimen are serial sections, the stained specimen and the negative control specimen may have similar histological characteristics, but they represent different cell populations.

[0008] In measurement methods that utilize image analysis of tissue samples, determining the appropriate positive threshold is more difficult compared to measurement methods that utilize flow cytometry, making automation challenging. Therefore, in practice, the positive threshold is sometimes determined or adjusted based on the user's subjective judgment, but in such cases, it is difficult to perform highly accurate measurements consistently across different users.

[0009] The detection method disclosed in Patent Document 1 above uses only images of stained sections as input data for analysis, and detects cell images while gradually changing the detection threshold. Therefore, the detection method in Patent Document 1 cannot guarantee sufficient detection accuracy, for example, when the number of cells is extremely small or large, or when there is a large amount of background noise.

[0010] This disclosure provides a technique advantageous for determining the positive threshold used in the analysis of fluorescence spectra of stained specimens. [Means for solving the problem]

[0011] One aspect of the present disclosure relates to an information processing device comprising: a first separation unit that separates the fluorescence spectrum of a stained specimen, obtained by irradiating an excitation light onto a fluorescently stained specimen obtained by labeling a specimen with a fluorescent reagent, into a stained fluorescent component image containing a fluorescent reagent and a stained autofluorescent component image containing an autofluorescent component, using a fluorescence reference spectrum and an autofluorescent reference spectrum; a second separation unit that separates the fluorescence spectrum of an unstained specimen, obtained by irradiating an excitation light onto an unstained specimen not labeled with a fluorescent reagent, into an unstained fluorescent component image containing a fluorescent reagent and an unstained autofluorescent component image containing an autofluorescent component, using a fluorescence reference spectrum and an autofluorescent reference spectrum; a threshold determination unit that determines a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the unstained fluorescent component image and compared with the image data of multiple image sections included in the stained fluorescent component image; and a threshold output unit that outputs a positive threshold.

[0012] The first separation unit generates a pseudo-stained fluorescence spectrum based on the stained fluorescence component image and fluorescence reference spectrum, generates a pseudo-stained autofluorescence spectrum based on the stained autofluorescence component image and autofluorescence reference spectrum, generates a pseudo-stained specimen fluorescence spectrum based on the pseudo-stained fluorescence spectrum and pseudo-stained autofluorescence spectrum, generates a difference stained specimen fluorescence spectrum based on the difference between the stained specimen fluorescence spectrum and the pseudo-stained specimen fluorescence spectrum, separates the difference stained specimen fluorescence spectrum into a difference stained fluorescence component image containing the fluorescent reagent and a difference stained autofluorescence component image containing the autofluorescence component using the fluorescence reference spectrum and autofluorescence reference spectrum, and the second separation unit generates a pseudo-non-stained fluorescence component image based on the unstained fluorescence component image and fluorescence reference spectrum. A stained fluorescence spectrum is generated, a pseudo-unstained autofluorescence spectrum is generated based on the unstained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-unstained specimen fluorescence spectrum is generated based on the pseudo-unstained fluorescence spectrum and the pseudo-unstained autofluorescence spectrum, a differential unstained specimen fluorescence spectrum is generated based on the difference between the unstained specimen fluorescence spectrum and the pseudo-unstained specimen fluorescence spectrum, the differential unstained specimen fluorescence spectrum is separated into a differential unstained fluorescent component image containing the fluorescent reagent and a differential unstained autofluorescence component image containing the autofluorescence component using the fluorescence reference spectrum and the autofluorescence reference spectrum, and the threshold determination unit may correct the positive threshold based on the spectrum of the differential stained fluorescent component image and the spectrum of the differential unstained fluorescent component image.

[0013] The first separation unit generates a pseudo-stained fluorescence spectrum based on the stained fluorescence component image and fluorescence reference spectrum, generates a pseudo-stained autofluorescence spectrum based on the stained autofluorescence component image and autofluorescence reference spectrum, generates a pseudo-stained specimen fluorescence spectrum based on the pseudo-stained fluorescence spectrum and pseudo-stained autofluorescence spectrum, generates a difference stained specimen fluorescence spectrum based on the difference between the stained specimen fluorescence spectrum and the pseudo-stained specimen fluorescence spectrum, the second separation unit generates a pseudo-unstained fluorescence spectrum based on the unstained fluorescence component image and fluorescence reference spectrum, generates a pseudo-unstained autofluorescence spectrum based on the unstained autofluorescence component image and autofluorescence reference spectrum, generates a pseudo-unstained specimen fluorescence spectrum based on the pseudo-unstained fluorescence spectrum and pseudo-unstained autofluorescence spectrum, generates a difference unstained specimen fluorescence spectrum based on the difference between the unstained specimen fluorescence spectrum and the pseudo-unstained specimen fluorescence spectrum, and the threshold determination unit may correct the positive threshold based on the difference stained specimen fluorescence spectrum and the difference unstained specimen fluorescence spectrum.

[0014] The second separation unit generates a pseudo-unstained fluorescence spectrum based on the unstained fluorescence component image and fluorescence reference spectrum, generates a pseudo-unstained autofluorescence spectrum based on the unstained autofluorescence component image and autofluorescence reference spectrum, generates a pseudo-unstained specimen fluorescence spectrum based on the pseudo-unstained fluorescence spectrum and pseudo-unstained autofluorescence spectrum, generates a differential unstained specimen fluorescence spectrum based on the difference between the unstained specimen fluorescence spectrum and the pseudo-unstained specimen fluorescence spectrum, generates differential unstained norm data which is the norm data of the differential unstained specimen fluorescence spectrum, and the threshold determination unit may analyze the differential unstained norm data to obtain outlier data, correct the unstained fluorescence component image based on the outlier data, and determine a positive threshold based on the corrected unstained fluorescence component image.

[0015] The threshold determination unit may correct the positive threshold based on a predetermined correction value depending on the fluorescent reagent.

[0016] The threshold determination unit may obtain a correction value from a correction data storage unit that stores reagent identification information and correction values ​​in relation to each other, based on reagent identification information associated with the fluorescent reagent.

[0017] The threshold determination unit may correct the positive threshold based on a predetermined correction value depending on the combination of the fluorescent reagent and the object to be labeled with the fluorescent reagent.

[0018] The threshold determination unit may obtain a correction value from a correction data storage unit that stores the label target identification information associated with the sample, the reagent identification information associated with the fluorescent reagent, and the correction value in relation to each other, based on the label target identification information associated with the sample and the reagent identification information associated with the fluorescent reagent.

[0019] The threshold determination unit may determine a positive threshold for each of the multiple observation regions determined by dividing the stained fluorescent component image.

[0020] The threshold determination unit may determine a positive threshold for each of the multiple observation areas defined by the user.

[0021] The threshold determination unit may identify noise components included in the fluorescence spectrum of the stained specimen and define multiple observation regions by dividing the stained fluorescence component image according to the noise components.

[0022] The threshold determination unit may determine the correctable range of the positive threshold, and the threshold output unit may output information indicating the positive threshold and the correctable range.

[0023] Other aspects of the present disclosure include an optical irradiation unit that irradiates excitation light for exciting a fluorescent reagent, an imaging device that images a specimen irradiated with the excitation light to obtain a specimen fluorescence spectrum, and an information processing device that analyzes the specimen fluorescence spectrum. The information processing device irradiates excitation light on a fluorescence-stained specimen obtained by labeling a specimen with a fluorescent reagent, and uses a fluorescence reference spectrum and a autofluorescence reference spectrum to separate the fluorescence-stained specimen fluorescence spectrum into a stained fluorescence component image including a fluorescence reagent and a stained autofluorescence component image including an autofluorescence component. The information processing device irradiates excitation light on a non-fluorescence-stained specimen not labeled with a fluorescent reagent, and uses a fluorescence reference spectrum and a autofluorescence reference spectrum to separate the non-fluorescence-stained specimen fluorescence spectrum into a non-stained fluorescence component image including a fluorescence reagent and a non-stained autofluorescence component image including an autofluorescence component. Based on the non-stained fluorescence component image, a positive threshold for comparison with the image data of a plurality of image sections included in the stained fluorescence component image, which is a determination criterion for whether each of the plurality of image sections corresponds to a positive cell image, is determined by a threshold determination unit. It relates to a microscope system having the above components.

[0024] The microscope system may include a presentation information generation unit that generates presentation information displayed on a display unit, the presentation information including threshold information indicating a positive threshold.

[0025] The threshold determination unit determines a correctable range of the positive threshold, and the presentation information may include correctable range information indicating the correctable range.

[0026] The microscope system may include an analysis unit that performs analysis based on the positive threshold.

[0027] Another aspect of the present disclosure is to use a fluorescence reference spectrum and an autofluorescence reference spectrum to separate a stained sample fluorescence spectrum obtained by irradiating an excitation light to a fluorescence-stained sample obtained by labeling a sample with a fluorescent reagent into a stained fluorescence component image including a fluorescent reagent and a stained autofluorescence component image including an autofluorescence component, and irradiate an excitation light to a non-stained sample fluorescence spectrum obtained by irradiating an excitation light to a non-fluorescent stained sample not labeled with a fluorescent reagent, and use the fluorescence reference spectrum and the autofluorescence reference spectrum to separate the non-stained sample fluorescence spectrum into a non-stained fluorescence component image including a fluorescent reagent and a non-stained autofluorescence component image including an autofluorescence component, and based on the non-stained fluorescence component image, determine a positive threshold value that is compared with the image data of a plurality of image sections included in the stained fluorescence component image, and is a determination criterion for whether each of the plurality of image sections corresponds to a positive cell image, and output the positive threshold value. The present disclosure relates to an information processing method including the above steps.

Brief Description of Drawings

[0028] [Figure 1] FIG. 1 is a block diagram showing a configuration example of an information processing system. [Figure 2A] FIG. 2A is a specific example of a fluorescence spectrum acquired by a fluorescence signal acquisition unit. [Figure 2B] FIG. 2B is a specific example of a fluorescence spectrum acquired by a fluorescence signal acquisition unit. [Figure 2C] FIG. 2C is a specific example of a fluorescence spectrum acquired by a fluorescence signal acquisition unit. [Figure 2D] FIG. 2D is a specific example of a fluorescence spectrum acquired by a fluorescence signal acquisition unit. [Figure 3] FIG. 3 is a diagram for explaining an example of a method for generating a connected fluorescence spectrum by a connecting unit. [Figure 4] FIG. 4 is a diagram showing an example of a connected fluorescence spectrum generated from the fluorescence spectra shown in "A" to "D" of FIG. 3. [Figure 5] FIG. 5 is a diagram for explaining an outline of an example of NMF. [Figure 6]Figure 6 is a diagram illustrating an example of clustering. [Figure 7] Figure 7 shows an example of a functional configuration for determining a positive threshold in an information processing device. [Figure 8] Figure 8 shows an example of image spectral data obtained in an information processing device. [Figure 9] Figure 9 is a flowchart showing an example of image processing (particularly image processing based on the fluorescence spectrum of stained specimens) performed in an information processing device. [Figure 10] Figure 10 is a flowchart showing an example of image processing performed in an information processing device (particularly image processing based on the fluorescence spectrum of an unstained specimen). [Figure 11] Figure 11 shows a conceptual example of a stained autofluorescence component image. [Figure 12] Figure 12 shows a conceptual example of an autofluorescence reference spectrum. [Figure 13] Figure 13 shows a conceptual example of a calculation that generates a pseudo-stained autofluorescence spectrum from a stained autofluorescence component image and an autofluorescence reference spectrum. [Figure 14] Figure 14 shows an example of histograms for stained and unstained fluorescent component images. [Figure 15] Figure 15 shows an example of a difference unstained normed image. [Figure 16] Figure 16 shows an example of a region exhibiting outliers in a difference unstained normed image. [Figure 17] Figure 17 shows an example of an outlier region in an unstained fluorescence component image. [Figure 18] Figure 18 shows an example of a histogram of an unstained fluorescent component image. [Figure 19] Figure 19 shows an example of a histogram of the unstained fluorescence component image after correction based on outlier data. [Figure 20] Figure 20 shows an example of how image information is displayed on the display unit. [Figure 21]Figure 21 shows an example of how image information is displayed on the display unit. [Figure 22] Figure 22 shows an example of how image information is displayed on the display unit. [Figure 23] Figure 23 shows an example of a correction value stored in the correction data storage unit. [Figure 24] Figure 24 shows an example of a correction value stored in the correction data storage unit. [Figure 25] Figure 25 is a block diagram showing an example of a microscope system configuration. [Figure 26] Figure 26 is a schematic diagram illustrating an example of a method for calculating the number of fluorescent molecules or antibodies in a single pixel. [Figure 27] Figure 27 is a block diagram showing an example of the hardware configuration of an information processing device. [Modes for carrying out the invention]

[0029] Typical embodiments of this disclosure will be described below with reference to the drawings.

[0030] Referring to Figure 1, an example of the configuration of an information processing system according to one embodiment will be described. The information processing system shown in Figure 1 comprises an information processing device 100 and a database 200.

[0031] (Fluorescent reagent 10) The fluorescent reagent 10 is a chemical used to stain the sample 20. The fluorescent reagent 10 may include, for example, a fluorescent antibody (including a primary antibody used for direct labeling or a secondary antibody used for indirect labeling), a fluorescent probe, or a nuclear staining reagent, but the type of fluorescent reagent 10 is not limited to these. The fluorescent reagent 10 is managed by attaching identification information (hereinafter referred to as "reagent identification information 11") that can identify the fluorescent reagent 10 or the manufacturing lot of the fluorescent reagent 10. The reagent identification information 11 may be, for example, barcode information (one-dimensional barcode information or two-dimensional barcode information, etc.), but is not limited to this. Even if the fluorescent reagent 10 is the same product, its properties will differ from one manufacturing lot to another depending on the manufacturing method and the state of the cells from which the antibody was obtained. For example, in the fluorescent reagent 10, the spectrum, quantum yield, or fluorescence labeling rate may differ from one manufacturing lot to another. Therefore, in the information processing system according to this embodiment, the fluorescent reagent 10 is managed by each manufacturing lot by attaching reagent identification information 11. This allows the information processing device 100 to perform fluorescence separation that takes into account even slight differences in properties that appear from one manufacturing lot to another.

[0032] (specimen 20) Specimen 20 is prepared from a specimen or tissue sample taken from the human body for the purpose of pathological diagnosis, etc. Specimen 20 may be a tissue section, cells, or microparticles. With respect to Specimen 20, there are no limitations on the type of tissue (organ, etc.) used, the type of disease targeted, the attributes of the subject (age, sex, blood type, and race, etc.), or the subject's lifestyle (diet, exercise habits, and smoking habits, etc.). Tissue sections may include, for example, a section of the tissue section to be stained (hereinafter also simply referred to as "section") before staining, a section adjacent to a stained section, and a section different from the stained section in the same block (sampled from the same location as the stained section). Furthermore, tissue sections may include sections from different blocks of the same tissue (sampled from different locations than the stained section), and sections taken from different patients.

[0033] Sample 20 is managed by attaching identification information (hereinafter referred to as "sample identification information 21") that allows for the identification of sample 20. Sample identification information 21 is, like reagent identification information 11, for example, barcode information (one-dimensional barcode information, two-dimensional barcode information, etc.), but is not limited to this. The properties of sample 20 differ depending on the type of tissue used, the type of disease targeted, the attributes of the subject, or the lifestyle of the subject. For example, in sample 20, the measurement channel or spectrum may differ depending on the type of tissue used. In the information processing system according to this embodiment, samples 20 are managed individually by attaching sample identification information 21. This allows the information processing device 100 to perform fluorescence separation that takes into account even slight differences in properties that appear for each sample 20.

[0034] (Fluorescent stained specimen 30) A fluorescently stained specimen 30 is prepared by staining specimen 20 with a fluorescent reagent 10. In this embodiment, it is assumed that specimen 20 is stained with one or more fluorescent reagents 10 to produce the fluorescently stained specimen 30. However, the number of fluorescent reagents 10 used to stain specimen 20 is not limited. The staining method is determined by the combination of specimen 20 and fluorescent reagents 10, but is not particularly limited.

[0035] When using a specimen that has not been labeled with the fluorescent reagent 10 (hereinafter referred to as "unstained fluorescent specimen"), for example, specimen 20 can be used as an unstained fluorescent specimen without staining it with the fluorescent reagent 10.

[0036] (Information processing device 100) As shown in Figure 1, the information processing device 100 comprises an acquisition unit 110, a storage unit 120, a processing unit 130, a display unit 140, a control unit 150, and an operation unit 160. The information processing device 100 can be configured, for example, by a fluorescence microscope system, but is not necessarily limited to this and may include various devices. The information processing device 100 may also be configured, for example, by a PC (Personal Computer).

[0037] (Acquisition part 110) The acquisition unit 110 acquires information used for various processes of the information processing device 100. The acquisition unit 110 shown in Figure 1 includes an information acquisition unit 111 and a fluorescence signal acquisition unit 112.

[0038] (Information acquisition unit 111) The information acquisition unit 111 acquires information about the fluorescent reagent 10 (hereinafter referred to as "reagent information") and information about the specimen 20 (hereinafter referred to as "specimen information"). More specifically, the information acquisition unit 111 acquires reagent identification information 11 attached to the fluorescent reagent 10 used to produce the fluorescently stained specimen 30, and specimen identification information 21 attached to the specimen 20 used to produce the fluorescently stained specimen 30 and / or the non-fluorescently stained specimen. For example, the information acquisition unit 111 uses a barcode reader or the like to acquire the reagent identification information 11 and specimen identification information 21 attached to the fluorescent reagent 10 and specimen 20 as barcode information. Then, the information acquisition unit 111 acquires reagent information from the database 200 based on the reagent identification information 11, and acquires specimen information from the database 200 based on the specimen identification information 21. The information acquisition unit 111 stores this acquired information in the information storage unit 121, which will be described later.

[0039] In this embodiment, the sample information includes a linked autofluorescence reference spectrum, and the reagent information includes a linked fluorescence reference spectrum. The linked autofluorescence reference spectrum is obtained by linking the spectra of the autofluorescent substance in the sample 20 in the wavelength direction. The linked fluorescence reference spectrum is obtained by linking the spectra of the fluorescent substance in the fluorescently stained sample 30 in the wavelength direction. Note that the linked autofluorescence reference spectrum and the linked fluorescence reference spectrum are also simply referred to as "autofluorescence reference spectrum" and "fluorescence reference spectrum," respectively, and the linked autofluorescence reference spectrum and the linked fluorescence reference spectrum are collectively referred to as "reference spectrum."

[0040] (Database 200) Database 200 is a device that manages information such as reagent information and specimen information. More specifically, database 200 manages reagent identification information 11 in association with reagent information, and manages specimen identification information 21 in association with specimen information. The information acquisition unit 111 can acquire reagent information from database 200 based on the reagent identification information 11 of the fluorescent reagent 10, and can acquire specimen information from database 200 based on the specimen identification information 21 of the specimen 20. The database 200 shown in Figure 1 is connected to the information processing device 100 (particularly the acquisition unit 110 (information acquisition unit 111)) via a network.

[0041] The reagent information managed by database 200 is assumed to include, but is not necessarily limited to, the measurement channels and fluorescence reference spectra specific to the fluorescent substances in the fluorescent reagent 10. "Measurement channel" is a concept that indicates the fluorescent substance contained in the fluorescent reagent 10. Since the number of fluorescent substances varies depending on the fluorescent reagent 10, the measurement channels are managed as reagent information linked to each fluorescent reagent 10. Furthermore, the fluorescence reference spectra included in the reagent information are the fluorescence spectra of each fluorescent substance contained in the measurement channel.

[0042] The sample information managed by database 200 is assumed to include, but is not necessarily limited to, the measurement channels and autofluorescence reference spectra specific to the autofluorescent substances in sample 20. "Measurement channels" are a concept that indicates the autofluorescent substances contained in sample 20, and examples include Hemoglobin, Archidonic Acid, Catalase, Collagen, FAD, NADPH, and ProLongDiamond. Since the number of autofluorescent substances varies among samples 20, the measurement channels are managed as part of the sample information, linked to each sample 20. Furthermore, the autofluorescence reference spectra included in the sample information are the autofluorescence spectra of each autofluorescent substance contained in the measurement channels. Note that the information managed by database 200 is not necessarily limited to the above information.

[0043] (Fluorescence signal acquisition unit 112) The fluorescence signal acquisition unit 112 acquires multiple fluorescence signals (i.e., multiple fluorescence signals corresponding to each of the multiple excitation lights) obtained by irradiating the fluorescently stained specimen 30 with multiple excitation lights of different wavelengths. More specifically, the fluorescence signal acquisition unit 112 receives light from the fluorescently stained specimen 30 and outputs a detection signal corresponding to the amount of light received, thereby acquiring the fluorescence spectrum of the fluorescently stained specimen 30 based on the detection signal. Here, the characteristics of the excitation light (including, for example, wavelength and light intensity) are determined based on reagent information, etc. (i.e., information regarding the fluorescent reagent 10, etc.). Note that the fluorescence signal referred to here is not particularly limited as long as it is a signal derived from fluorescence, and may be, for example, a fluorescence spectrum.

[0044] The fluorescence signal acquisition unit 112 can acquire multiple fluorescence signals (fluorescence spectra) related to a non-fluorescent stained specimen by irradiating the specimen with multiple excitation lights in a similar manner.

[0045] Figures 2A to 2D show specific examples of fluorescence spectra acquired by the fluorescence signal acquisition unit 112. The fluorescence-stained specimens 30 shown in Figures 2A to 2D contain four types of fluorescent substances: DAPI, CK / AF488, PgR / AF594, and ER / AF647. Figures 2A to 2D show examples of fluorescence spectra acquired when excitation light with excitation wavelengths of 392 nm (Figure 2A), 470 nm (Figure 2B), 549 nm (Figure 2C), and 628 nm (Figure 2D) is irradiated onto the fluorescence-stained specimens 30. Note that because energy is released for fluorescence emission, the fluorescence wavelength shifts to a longer wavelength than the excitation wavelength (Stokes shift). Furthermore, the fluorescent substances contained in the fluorescence-stained specimens 30 and the excitation wavelengths of the irradiated excitation light are not limited to the examples above. The fluorescence signal acquisition unit 112 stores the acquired fluorescence spectra in the fluorescence signal storage unit 122 (see Figure 1), which will be described later.

[0046] (Preservation section 120) The storage unit 120 shown in Figure 1 stores information used in various processes of the information processing device 100 or information output by various processes. As shown in Figure 1, the storage unit 120 of this embodiment includes an information storage unit 121 and a fluorescence signal storage unit 122.

[0047] (Information storage section 121) The information storage unit 121 stores the reagent information and specimen information acquired by the information acquisition unit 111.

[0048] (Fluorescent signal storage unit 122) The fluorescence signal storage unit 122 stores the fluorescence signals of the fluorescently stained specimen 30 acquired by the fluorescence signal acquisition unit 112. The fluorescence signal storage unit 122 also stores the fluorescence signals of the unstained specimen acquired by the fluorescence signal acquisition unit 112.

[0049] (Processing unit 130) The processing unit 130 performs various processes, including fluorescence separation (i.e., color separation). As shown in Figure 1, the processing unit 130 comprises a connecting unit 131, a separation processing unit 132, and an image generation unit 133.

[0050] (Connection part 131) The coupling unit 131 generates a linked fluorescence spectrum by coupling at least a portion of the multiple fluorescence spectra acquired by the fluorescence signal acquisition unit 112 (i.e., the multiple fluorescence spectra stored in the fluorescence signal storage unit 122) in the wavelength direction. For example, for each of the four fluorescence spectra acquired by the fluorescence signal acquisition unit 112 (see labels "A" to "D" in Figure 3), the coupling unit 131 extracts data of a predetermined width from each fluorescence spectrum so as to include the maximum fluorescence intensity. The width of the wavelength band from which the coupling unit 131 extracts data may be determined based on reagent information, excitation wavelength, fluorescence wavelength, etc., and may differ for each fluorescent substance. That is, the width of the wavelength band from which the coupling unit 131 extracts data may differ among the fluorescence spectra shown in "A" to "D" in Figure 3. Then, as shown in "E" in Figure 3, the coupling unit 131 generates a single linked fluorescence spectrum by coupling the extracted data to each other in the wavelength direction. Note that since the linked fluorescence spectrum is composed of data extracted from multiple fluorescence spectra, the wavelengths are not necessarily continuous at the boundaries between the coupled data.

[0051] The coupling unit 131 of this embodiment couples multiple fluorescence spectra in the wavelength direction after aligning the intensity of the excitation light corresponding to each of the multiple fluorescence spectra (in other words, after correcting the multiple fluorescence spectra based on the intensity of the excitation light). More specifically, the coupling unit 131 couples the multiple fluorescence spectra after aligning the intensity of the excitation light corresponding to each of the multiple fluorescence spectra by dividing each fluorescence spectrum by the excitation power density, which indicates the intensity of the excitation light. This allows for the determination of the fluorescence spectrum that would be obtained if the fluorescence-stained specimen 30 were irradiated with excitation light of the same intensity. Furthermore, if the intensity of the irradiated excitation light is different, the intensity of the spectrum absorbed by the fluorescence-stained specimen 30 (hereinafter referred to as the "absorption spectrum") will also differ according to the intensity of the excitation light. Therefore, by aligning the intensity of the excitation light corresponding to each of the multiple fluorescence spectra as described above, it becomes possible to appropriately evaluate the absorption spectrum.

[0052] In this explanation, the intensity of the excitation light may also be the excitation power or excitation power density, as described above. The excitation power or excitation power density may be the power or power density obtained by actually measuring the excitation light emitted from the light source, or it may be the power or power density obtained from the driving voltage applied to the light source. In addition, the intensity of the excitation light in this explanation may be a value obtained by correcting the above excitation power density using the absorption rate of each excitation light of the intercept that is the object of observation, or the amplification rate of the detection signal in the detection system (fluorescence signal acquisition unit, etc.) that detects the fluorescence emitted from the intercept. That is, the intensity of the excitation light in this explanation may be the power density of the excitation light that actually contributed to the excitation of the fluorescent substance, or a value obtained by correcting that power density with the amplification rate of the detection system, etc. By considering the absorption rate, amplification rate, etc., it becomes possible to appropriately correct the intensity of the excitation light that changes in accordance with changes in machine state and environment, and thus it becomes possible to generate a linked fluorescence spectrum that enables higher accuracy color separation.

[0053] The correction value (also called the "intensity correction value") based on the intensity of the excitation light for each fluorescence spectrum is not limited to a value that equalizes the intensity of the excitation light corresponding to each of the multiple fluorescence spectra, but can be varied in various ways. For example, the signal intensity of a fluorescence spectrum with an intensity peak on the longer wavelength side (also called a "long-wavelength peak fluorescence spectrum") tends to be lower than the signal intensity of a fluorescence spectrum with an intensity peak on the shorter wavelength side (also called a "short-wavelength peak fluorescence spectrum"). Therefore, when a linked fluorescence spectrum contains both long-wavelength peak fluorescence spectra and short-wavelength peak fluorescence spectra, the long-wavelength peak fluorescence spectrum may be hardly considered, and the short-wavelength peak fluorescence spectrum may be extracted primarily. In this case, for example, by setting a larger intensity correction value for the long-wavelength peak fluorescence spectrum, it is possible to improve the separation accuracy of the short-wavelength peak fluorescence spectrum.

[0054] Furthermore, the coupling section 131 may correct the wavelength resolution of each of the coupled fluorescence spectra independently of the other fluorescence spectra. For example, the fluorescence spectra of AF546 and AF555 have almost the same spectral shape and peak wavelength. The difference between them is that the fluorescence spectrum of AF555 has a shoulder at the tail end on the high-wavelength side, while the fluorescence spectrum of AF546 does not have such a shoulder. In this way, when two fluorescence spectra are similar, a problem arises in that it becomes difficult to color-separate them by spectral extraction.

[0055] Such problems can sometimes be solved by increasing the wavelength resolution of the linked fluorescence spectrum. This indicates that even when using multiple fluorescence spectra with similar spectral shapes and peak wavelengths, it is possible to separate them by increasing the wavelength resolution.

[0056] However, increasing the wavelength resolution increases the amount of data in the concatenated fluorescence spectrum, which increases the required memory capacity and computational costs in fluorescence separation processing. Therefore, the concatenation unit 131 corrects the fluorescence spectra to be concatenated, adjusting the wavelength resolution of those spectra that are expected to be difficult to color separate, and adjusting the wavelength resolution of those spectra that are expected to be easy to color separate. This makes it possible to improve color separation accuracy while suppressing an increase in the amount of data.

[0057] Here, a specific example of how to generate a linked fluorescence spectrum using the linking unit 131 will be explained. In this example, a fluorescently stained specimen 30 containing four fluorescent substances, DAPI, CK / AF488, PgR / AF594, and ER / AF647, is used, similar to the method for generating a linked fluorescence spectrum described above using Figure 3. An example will be given in which the four fluorescence spectra obtained by irradiating the fluorescently stained specimen 30 with excitation light having excitation wavelengths of 392 nm, 470 nm, 549 nm, and 628 nm for each fluorescent substance are linked.

[0058] Figure 4 shows an example of a linked fluorescence spectrum generated from the fluorescence spectra shown in "A" to "D" in Figure 3. As shown in Figure 4, the linking unit 131 extracts fluorescence spectrum SP1 in the wavelength band between 392 nm and 591 nm from the fluorescence spectrum shown in "A" in Figure 3. The linking unit 131 also extracts fluorescence spectrum SP2 in the wavelength band between 470 nm and 669 nm from the fluorescence spectrum shown in "B" in Figure 3. The linking unit 131 also extracts fluorescence spectrum SP3 in the wavelength band between 549 nm and 748 nm from the fluorescence spectrum shown in "C" in Figure 3. The linking unit 131 also extracts fluorescence spectrum SP4 in the wavelength band between 628 nm and 827 nm from the fluorescence spectrum shown in "D" in Figure 3. Next, the coupling unit 131 corrects the wavelength resolution of the extracted fluorescence spectrum SP1 to 16 nm (without intensity correction), corrects the intensity of fluorescence spectrum SP2 by 1.2 times, and corrects its wavelength resolution to 8 nm. The coupling unit 131 also corrects the intensity of fluorescence spectrum SP3 by 1.5 times (without wavelength resolution correction), corrects the intensity of fluorescence spectrum SP4 by 4.0 times, and corrects its wavelength resolution to 4 nm. Then, the coupling unit 131 sequentially couples the corrected fluorescence spectra SP1 to SP4 to generate the coupled fluorescence spectrum shown in Figure 4.

[0059] Figure 4 shows a concatenated fluorescence spectrum obtained by the concatenation unit 131 extracting fluorescence spectra SP1 to SP4 within a predetermined bandwidth (200 nm width in Figure 4) from the excitation wavelengths acquired when each fluorescence spectrum was obtained and concatenating them. However, the bandwidths of the fluorescence spectra extracted by the concatenation unit 131 do not need to match between fluorescence spectra and may differ between fluorescence spectra. In other words, the region extracted by the concatenation unit 131 from each fluorescence spectrum only needs to include the peak wavelength of each fluorescence spectrum, and the wavelength band and bandwidth of the extracted region may be changed as appropriate. When changing the wavelength band and bandwidth of the extracted region, the shift in spectral wavelength due to Stokes shift may be taken into consideration. In this way, by extracting and using a limited wavelength band, it is possible to reduce the amount of data, and thus enable faster fluorescence separation processing.

[0060] (Separation processing unit 132) The separation processing unit 132 shown in Figure 1 performs color separation on the fluorescence signal of the fluorescently stained specimen 30 acquired by the fluorescence signal storage unit 122 (i.e., the fluorescence spectrum of the stained specimen). The separation processing unit 132 also performs color separation on the fluorescence signal of the unstained specimen (i.e., the fluorescence spectrum of the unstained specimen). As a result of the color separation processing, a stained fluorescence component image, created by extracting the fluorescence image of the fluorescent reagent from the fluorescence signal of the fluorescently stained specimen 30, and a stained autofluorescence component image, created by extracting the fluorescence image of the autofluorescence component, are derived as separate images. Similarly, an unstained fluorescence component image, created by extracting the fluorescence image of the fluorescent reagent from the fluorescence signal of the unstained specimen, and an unstained autofluorescence component image, created by extracting the fluorescence image of the autofluorescence component, are derived as separate images.

[0061] For color separation, methods such as least squares (LSM) or weighted least squares (WLSM) may be used. Furthermore, for the extraction of autofluorescence spectra and / or fluorescence spectra, methods such as non-negative matrix factorization (NMF), singular value decomposition (SVD), or principal component analysis (PCA) may be used.

[0062] (Regarding the least squares method) Here, we will explain the least squares method that can be used in the color separation processing by the separation processing unit 132. The least squares method is a calculation method that calculates the color mixing ratio by fitting a reference spectrum to the fluorescence spectrum, which is the pixel value of each pixel in the input sample fluorescence spectrum (for example, the fluorescence spectrum of a stained sample (stained sample image)). The color mixing ratio is an index that indicates the degree to which each substance is mixed. Equation (1) below is an equation that represents the residual obtained by subtracting the reference spectrum St (fluorescence reference spectrum and autofluorescence reference spectrum) mixed at a color mixing ratio a from the fluorescence spectrum (Signal). In equation (1), "Signal (1 × number of channels)" indicates that there are as many fluorescence spectra (Signal) as there are channels of wavelength. For example, Signal is a matrix that represents one or more fluorescence spectra. Also, "St (number of substances × number of channels)" indicates that there are as many reference spectra for each substance (fluorescent substance and autofluorescent substance) as there are channels of wavelength. For example, St is a matrix that represents one or more reference spectra. Furthermore, "a(1 × number of substances)" indicates that a color mixing ratio a is assigned to each substance (fluorescent substance and autofluorescent substance). For example, a is a matrix representing the color mixing ratios of each reference spectrum in the fluorescence spectrum.

[0063]

number

[0064] The separation processing unit 132 then calculates the mixing ratio a of each substance that minimizes the sum of squares of the residuals in equation (1). The sum of squares of the residuals is minimized when the result of the partial derivative of equation (1) with respect to the mixing ratio a is zero. Therefore, the separation processing unit 132 calculates the mixing ratio a of each substance that minimizes the sum of squares of the residuals by solving the following equation (2). In equation (2), "St'" represents the transpose matrix of the reference spectrum St. Also, "inv(St*St')" represents the inverse matrix of St*St'.

[0065]

number

[0066] Here, specific examples of the values ​​in equation (1) above are shown in equations (3) to (5) below. In the examples of equations (3) to (5), the case is shown in which the reference spectra (St) of three substances (3 substances) are mixed at different mixing ratios a in the fluorescence spectrum (Signal).

[0067]

number

[0068]

number

[0069]

number

[0070] Then, a concrete example of the calculation result of equation (2) above using the values ​​of equations (3) and (5) is shown in equation (6) below. As shown in equation (6), it can be seen that the calculation result correctly yields "a = (3 2 1)" (i.e., the same value as in equation (4) above).

[0071]

number

[0072] Furthermore, as described above, the separation processing unit 132 may extract the spectrum for each fluorescent substance from the fluorescence spectrum by performing calculations related to the weighted least squares method instead of the least squares method. In the weighted least squares method, weights are applied to emphasize errors at low signal levels by taking advantage of the fact that the noise of the measured fluorescence spectrum (Signal) follows a Poisson distribution. However, the upper limit for which weighting is not applied in the weighted least squares method is defined as the Offset value. The Offset value is determined by the characteristics of the sensor used for measurement, and separate optimization is required when an image sensor is used as the sensor. When the weighted least squares method is performed, the reference spectrum St in equations (1) and (2) above is replaced with St_ represented by the following equation (7). Note that the following equation (7) means that St_ is calculated by dividing each element (each component) of St, which is represented as a matrix, by the corresponding element (each component) in "Signal + Offset value", which is also represented as a matrix (in other words, elemental division).

[0073]

number

[0074] Here, when the Offset value is 1 and the values ​​of the reference spectrum St and the fluorescence spectrum Signal are expressed by equations (3) and (5) above, a specific example of St_ expressed by equation (7) above is shown in equation (8) below.

[0075]

number

[0076] A concrete example of the calculation result of the color mixing ratio a in this case is shown in equation (9) below. As can be seen in equation (9), the calculation result correctly yields "a = (3 2 1)".

[0077]

number

[0078] (Regarding non-negative matrix factorization (NMF)) The non-negative matrix factorization (NMF) used by the separation processing unit 132 for extracting the autofluorescence spectrum and / or fluorescence spectrum will be described below. However, the method is not limited to non-negative matrix factorization (NMF), and singular value decomposition (SVD), principal component analysis (PCA), etc., may also be used.

[0079] Figure 5 is a diagram illustrating the overview of NMF. As shown in Figure 5, NMF decomposes a non-negative N x M (N×M) matrix A into a non-negative N x k (N×k) matrix W and a non-negative k x M (k×M) matrix H. Matrices W and H are determined such that the mean square residual D between matrix A and the product of matrices W and H (W*H) is minimized. In this embodiment, matrix A corresponds to the spectrum before the autofluorescence reference spectrum is extracted (N is the number of pixels and M is the number of wavelength channels). Matrix H corresponds to the extracted autofluorescence reference spectrum (k is the number of autofluorescence reference spectra (in other words, the number of autofluorescent substances) and M is the number of wavelength channels). Here, the mean square residual D is expressed by the following equation (10). Note that "norm(D,'fro')" refers to the Frobenius norm of the mean square residual D.

[0080]

number

[0081] Factorization in NMF employs an iterative method starting with random initial values ​​for matrices W and H. While the value of k (number of autofluorescence reference spectra) is essential in NMF, the initial values ​​for matrices W and H are optional and can be set. When initial values ​​for matrices W and H are set, the solution becomes constant. Conversely, if initial values ​​for matrices W and H are not set, they are randomly determined, and the solution becomes non-constant.

[0082] The specimen 20 differs in nature and autofluorescence spectrum according to the type of tissue used, the type of disease targeted, the attributes of the subject, or the lifestyle of the subject. Therefore, by actually measuring the autofluorescence reference spectrum for each specimen 20 as described above, the information processing apparatus 100 can achieve more accurate color separation processing.

[0083] Note that the matrix A, which is the input of NMF, is a matrix consisting of the same number of rows as the number of pixels N (= Hpix × Vpix) of the stained specimen image and the same number of columns as the number of wavelength channels M. Therefore, when the number of pixels of the stained specimen image is large or the number of wavelength channels M is large, the matrix A becomes a very large matrix, increasing the calculation cost of NMF and lengthening the processing time.

[0084] In such a case, for example, as shown in FIG. 6, by clustering into the number of classes N (<Hpix × Vpix) with the number of pixels N (= Hpix × Vpix) of the stained image specified, it is possible to suppress the lengthening of the processing time due to the enlargement of the matrix A.

[0085] In clustering, for example, among the stained images, spectra similar in the wavelength direction or intensity direction are classified into the same class. As a result, an image with a smaller number of pixels than the stained image is generated, and thus it is possible to reduce the scale of the matrix A' with this image as the input.

[0086] (Image generation unit 133) The image generation unit 133 shown in Figure 1 generates image information based on image spectrum data (including stained fluorescence component images) obtained as a result of a series of processes (including color separation processing for fluorescence spectra) in the separation processing unit 132. For example, the image generation unit 133 can generate image information using fluorescence spectra corresponding to one or more fluorescent substances, or using autofluorescence spectra corresponding to one or more autofluorescent substances. The number and combination of fluorescent substances (molecules) or autofluorescent substances (molecules) used by the image generation unit 133 to generate image information are not particularly limited. Furthermore, if various processes (e.g., segmentation or S / N value calculation) are performed using the separated fluorescence spectra or autofluorescence spectra, the image generation unit 133 may generate image information showing the results of these processes.

[0087] (Display section 140) The display unit 140 presents the image information generated by the image generation unit 133 to the user by displaying it on the display. The type of display used for the display unit 140 is not particularly limited. Furthermore, although not described in detail here, the image information generated by the image generation unit 133 may also be presented to the user by projecting it using a projector (display unit 140) or printing it using a printer (display unit 140). In other words, the method of outputting the image information is not particularly limited.

[0088] (Operation unit 160) The operation unit 160 receives operational input from the user. More specifically, the operation unit 160 is equipped with various input means such as a keyboard, mouse, buttons, touch panel, and / or microphone, and the user can perform various inputs to the information processing device 100 by operating these input means. Information regarding the inputs made via the operation unit 160 is provided to the control unit 150.

[0089] (Control unit 150) The control unit 150 has a functional configuration that comprehensively controls all processing performed by the information processing device 100. For example, the control unit 150 controls the start and end of various processing as described above based on operation input from the user via the operation unit 160. Examples of such processing include adjusting the placement position of the fluorescently stained specimen 30, irradiating the fluorescently stained specimen 30 with excitation light, acquiring a spectrum, generating an autofluorescence component corrected image, color separation, generating image information, and displaying image information. The control content of the control unit 150 is not particularly limited. For example, the control unit 150 may control processing that is commonly performed in general-purpose computers, PCs, tablet PCs, etc. (for example, processing related to the OS (Operating System)).

[0090] The system configuration described above with reference to Figure 1 is merely an example, and the configuration of the information processing system described above is not limited to this example. For example, the information processing device 100 does not necessarily have to have all of the configurations shown in Figure 1, and may have configurations not shown in Figure 1.

[0091] (Detection of positive cell images) As described above, the separation processing unit 132 (see Figure 1) performs color separation processing to separate the fluorescence spectrum of the stained specimen into a stained fluorescence component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component. This effectively separates the fluorescence signal originating from the autofluorescent substance, which is problematic in images of fluorescently stained specimens, from the fluorescence signal originating from the fluorescent substance being analyzed.

[0092] Furthermore, by analyzing stained fluorescence component images from which autofluorescence components have been removed or reduced, positive cell images in stained fluorescence component images can be detected with high accuracy.

[0093] Specifically, by comparing the image data (e.g., characteristic data such as brightness values) of multiple image sections contained in the stained fluorescent component image with a positive threshold, it is possible to determine whether or not each image section corresponds to a positive cell image. Each of the multiple image sections referred to here may be composed of individual pixels that make up the stained fluorescent component image, or it may be composed of a set of two or more pixels.

[0094] Thus, in order to detect positive cell images in stained fluorescence component images, it is necessary to determine a positive threshold, which is the criterion for determining whether or not each image section corresponds to a positive cell image.

[0095] The following describes typical examples of devices and methods for determining the positive threshold for stained fluorescent component images.

[0096] Figure 7 shows an example of the functional configuration for determining the positive threshold in the information processing device 100. Figure 8 shows an example of image spectral data obtained in the information processing device 100. Figure 9 is a flowchart showing an example of image processing (particularly image processing based on the fluorescence spectrum of a stained specimen) performed in the information processing device 100. Figure 10 is a flowchart showing an example of image processing (particularly image processing based on the fluorescence spectrum of an unstained specimen) performed in the information processing device 100.

[0097] The separation processing unit 132 shown in Figure 7 includes a separation unit 40, a threshold determination unit 43, and a separation output unit 44.

[0098] (separation part 40) The separation unit 40 acquires fluorescence spectra D1 and D21 obtained by irradiating the sample with excitation light, and reference spectra R1 and R2 (S11 and S12 in Figure 9; S21 and S22 in Figure 10).

[0099] The specimens discussed here may include not only fluorescently stained specimens 30 obtained by labeling specimens with fluorescent reagents, but also unstained specimens that have not been labeled with fluorescent reagents. The fluorescence spectrum obtained by imaging a fluorescently stained specimen irradiated with excitation light is called the stained specimen fluorescence spectrum D1 (see Figure 8). On the other hand, the fluorescence spectrum obtained by imaging an unstained specimen irradiated with excitation light is called the unstained specimen fluorescence spectrum D21.

[0100] As described above, the reference spectrum includes a fluorescence reference spectrum R1 that shows the original spectrum of the fluorescent reagent 10 as a reference, and an autofluorescence reference spectrum R2 that shows the original spectrum of the autofluorescent substance of sample 20 as a reference.

[0101] The separation unit 40 obtains the fluorescence spectrum D1 of the stained specimen, the fluorescence spectrum D21 of the unstained specimen, the fluorescence reference spectrum R1, and the autofluorescence reference spectrum R2.

[0102] In this example, the separation unit 40 acquires the stained specimen fluorescence spectrum D1 and the unstained specimen fluorescence spectrum D21, which are generated as linked fluorescence spectra by the connecting unit 131 (see Figure 1) described above. Therefore, the separation unit 40 may acquire the stained specimen fluorescence spectrum D1 and the unstained specimen fluorescence spectrum D21 directly from the connecting unit 131. Alternatively, if the connecting unit 131 stores the stained specimen fluorescence spectrum D1 and the unstained specimen fluorescence spectrum D21 in the storage unit 120 shown in Figure 1, the separation unit 40 may acquire the stained specimen fluorescence spectrum D1 and the unstained specimen fluorescence spectrum D21 from the storage unit 120.

[0103] The separation unit 40 also acquires the fluorescence reference spectrum R1 and the autofluorescence reference spectrum R2 from the storage unit 120 (specifically, the information storage unit 121) shown in Figure 1.

[0104] The separation unit 40 then uses the reference spectra R1 and R2 to separate the fluorescence spectra D1 and D21 into fluorescence component images and autofluorescence component images (color separation processes P1 and P11 in Figure 8; S13 in Figure 9; S23 in Figure 10).

[0105] The color separation process for the fluorescence spectrum D1 of a stained specimen and the color separation process for the fluorescence spectrum D21 of an unstained specimen are basically carried out in the same manner. Therefore, a common separation unit 40 may be used to perform color separation on both the fluorescence spectrum D1 of the stained specimen and the fluorescence spectrum D21 of the unstained specimen. However, the first separation unit 41 may be used to perform color separation on the fluorescence spectrum D1 of the stained specimen, and a second separation unit 42, which is different from the first separation unit 41, may be used to perform color separation on the fluorescence spectrum D21 of the unstained specimen.

[0106] The fluorescence spectrum D1 of the stained specimen is separated by color separation processing P1 into a stained fluorescence component image D2 containing the fluorescent reagent and a stained autofluorescence component image D3 containing the autofluorescence component. Similarly, the fluorescence spectrum D21 of the unstained specimen is separated by color separation processing P11 into an unstained fluorescence component image D22 containing the fluorescent reagent and an unstained autofluorescence component image D23 containing the autofluorescence component.

[0107] (Threshold determination unit 43) The threshold determination unit 43 (see Figure 7) determines a positive threshold for the stained fluorescent component image D2 based on the image spectral data obtained by the processing in the separation unit 40 (including the color separation processes P1 and P11 described above) (positive threshold determination step).

[0108] The image spectral data referred to herein may include the fluorescence spectrum D1 of a stained specimen and the fluorescence spectrum D21 of an unstained specimen, as well as data derived from the fluorescence spectrum D1 of the stained specimen and the fluorescence spectrum D21 of the unstained specimen.

[0109] The threshold determination unit 43 can determine the positive threshold for the stained fluorescence component image D2 by performing arbitrary processing based on the image spectral data received from the separation unit 40.

[0110] A typical example of how to determine the positive threshold in the threshold determination unit 43 will be described later.

[0111] (Separation output section 44) The separation output unit 44 outputs the positive threshold determined by the threshold determination unit 43 (positive threshold output step).

[0112] In this example, the separation output unit 44 outputs the image spectral data obtained by processing in the separation unit 40, along with the positive threshold. That is, the separation output unit 44 outputs the image spectral data and the positive threshold in relation to each other.

[0113] The separation output unit 44 may also include, as separate units, an image spectrum output unit 45 that outputs image spectrum data and a threshold output unit 46 that outputs a positive threshold.

[0114] Furthermore, the separation output unit 44 outputs a positive threshold, but does not necessarily output image spectral data. In this case, the image spectral data obtained in the separation unit 40 may be sent from the separation unit 40 to the storage unit 120 (for example, the fluorescence signal storage unit 122) shown in Figure 1 and stored. The image spectral data stored in the storage unit 120 may be read out and used as appropriate by other devices (for example, the analysis unit 47 and the image generation unit 133 described later).

[0115] The destination of the positive threshold output by the separation output unit 44 is not limited. Typically, the separation output unit 44 outputs the positive threshold to the analysis unit 47 and / or the image generation unit 133, but it may also output the positive threshold to other devices or functional components.

[0116] (Analysis Department 47) The analysis unit 47 performs arbitrary analysis based on the positive threshold output from the separation output unit 44. Typically, the analysis unit 47 analyzes image spectral data (e.g., stained fluorescence component image D2) based on the positive threshold. The analysis unit 47 may include analysis software (application) that performs cell analysis processing, such as cell counting.

[0117] The positive threshold provided from the separation output unit 44 to the analysis unit 47 can be automatically set to the positive threshold used in the analysis process performed in the analysis unit 47.

[0118] However, the method by which the analysis unit 47 uses the positive threshold output from the separation output unit 44 is not limited.

[0119] The analysis unit 47 may use the positive threshold output from the separation output unit 44 as a fixed value or as an initial value. When the positive threshold output from the separation output unit 44 is used as an initial value in the analysis unit 47, the analysis unit 47 may use a corrected positive threshold as needed in its actual analysis.

[0120] The analysis unit 47 described above may be provided as part of the information processing device 100 (see Figure 1), or it may be provided as a separate unit from the information processing device 100.

[0121] (Image generation unit 133) The image generation unit 133 generates image information to be displayed on the display unit 140.

[0122] The image information includes presentation information based on a positive threshold, and the image generation unit 133 functions as a presentation information generation unit that generates the presentation information.

[0123] In this embodiment, the presented information includes threshold information indicating the positive threshold.

[0124] Users can confirm the positive threshold by looking at the information displayed on the display unit 140 (especially the threshold information).

[0125] The specific method for generating image information in the image generation unit 133 is not limited.

[0126] For example, the image generation unit 133 may generate image information (including presentation information) based on the positive threshold and stained fluorescent component image D2 received from the separation output unit 44.

[0127] The image generation unit 133 may also receive the results of the analysis performed by the analysis unit 47 and generate image information (including presentation information) based on the analysis results.

[0128] The image generation unit 133 shown in Figure 1 is provided as part of the information processing device 100, but the image generation unit 133 may also be provided as a separate unit from the information processing device 100.

[0129] (Display section 140) The display unit 140 displays the image information received from the image generation unit 133 and presents it to the user. Alternatively, the display unit 140 may receive image information based on the analysis results of the analysis unit 47 from the analysis unit 47 and display that image information.

[0130] Examples of how image information is displayed in the display unit 140 will be described later (see Figures 20 to 22), but the manner in which image information is displayed in the display unit 140 is not limited.

[0131] The display unit 140 shown in Figure 1 is provided as part of the information processing device 100, but the display unit 140 may also be provided as a separate unit from the information processing device 100.

[0132] Image information may be sent to other devices besides the display unit 140 (for example, analysis devices or servers connected via a network). In this case, the image information may be used for processing in the other devices (for example, analysis processing such as the detection of specific cells).

[0133] (Determination of the positive threshold) Next, we will explain the specific method for determining the positive threshold.

[0134] In addition to the color separation processes P1 and P11 described above, the separation unit 40 (see Figure 7) can perform the following processes.

[0135] In other words, the separation unit 40 can generate a pseudo-stained fluorescence spectrum D4 based on the stained fluorescence component image D2 and the fluorescence reference spectrum R1 described above (processing P2 in Figure 8; S14 in Figure 9). For example, the separation unit 40 can obtain a pseudo-stained fluorescence spectrum D4 as a pseudo-stained fluorescence spectrum by multiplying the stained fluorescence component image D2 by the fluorescence reference spectrum R1.

[0136] Furthermore, the separation unit 40 can generate a pseudo-stained autofluorescence spectrum D5 based on the stained autofluorescence component image D3 and the autofluorescence reference spectrum R2 described above (processing P3 in Figure 8; S14 in Figure 9). For example, the separation unit 40 can obtain a pseudo-stained autofluorescence spectrum D5 as a pseudo-stained autofluorescence spectrum by multiplying the stained autofluorescence component image D3 by the autofluorescence reference spectrum R2.

[0137] Furthermore, when the non-negative matrix factorization (NMF) described above is used in the color separation process P1 of the stained specimen fluorescence spectrum D1, the autofluorescence reference spectrum R2 is modified (corrected) by NMF to be optimized for the stained specimen fluorescence spectrum D1. In this case, when generating the pseudo-stained autofluorescence spectrum D5, using the optimized and corrected autofluorescence reference spectrum R2 can yield a more accurate pseudo-stained autofluorescence spectrum D5.

[0138] Thus, in the various processes performed to determine the positive threshold, the autofluorescence reference spectrum R2 stored in the storage unit 120 may be used as needed, or the autofluorescence reference spectrum R2 after optimization correction may be used. The term "autofluorescence reference spectrum R2" as used in the following description is a concept that may include not only the autofluorescence reference spectrum R2 stored in the storage unit 120, but also the autofluorescence reference spectrum R2 after optimization correction.

[0139] Figure 11 shows a conceptual example of a stained autofluorescence component image D3. Figure 12 shows a conceptual example of a stained autofluorescence reference spectrum R2. Figure 13 shows a conceptual example of a calculation to obtain a pseudo-stained autofluorescence spectrum D5 from the stained autofluorescence component image D3 and the autofluorescence reference spectrum R2.

[0140] The separation unit 40 selects one unselected image from the stained autofluorescence component images D3 (see Figure 11) (this is designated as the stained autofluorescence component image of autofluorescence channel CHn (where n is a natural number)). Note that the autofluorescence channel referred to here may be identification information assigned to each autofluorescence component.

[0141] The separation unit 40 then generates a pseudo-stained autofluorescence spectrum D5 from the stained autofluorescence component image of the selected autofluorescence channel CHn and the autofluorescence reference spectrum corresponding to the selected autofluorescence channel CHn.

[0142] The separation unit 40 may generate a stained specific channel brightness image D6 by determining the brightness value of the spectral data corresponding to a specific channel in the pseudo-stained autofluorescence spectrum D5 (processing P4 in Figure 8).

[0143] The separation unit 40 can then generate a pseudo-stained specimen fluorescence spectrum D7 based on the pseudo-stained fluorescence spectrum D4 and the pseudo-stained autofluorescence spectrum D5 (processing P5 in Figure 8; S15 in Figure 9). For example, the separation unit 40 can obtain the pseudo-stained specimen fluorescence spectrum D7 by adding the pseudo-stained fluorescence spectrum D4 and the pseudo-stained autofluorescence spectrum D5 together. The pseudo-stained specimen fluorescence spectrum D7 thus generated is a pseudo-stained specimen fluorescence spectrum.

[0144] The separation unit 40 then generates a differential stained specimen fluorescence spectrum D8 based on the difference between the stained specimen fluorescence spectrum D1 and the pseudo-stained specimen fluorescence spectrum D7 (processing P6 in Figure 8; S16 in Figure 9).

[0145] The separation unit 40 then obtains a norm image of the difference stained specimen fluorescence spectrum D8, which is the difference spectral data of the stained specimen fluorescence spectrum D1 and the pseudo-stained specimen fluorescence spectrum D7, as a difference stained norm image D9 (processing P7 in Figure 8; S17 in Figure 9). The difference stained norm image D9 is obtained by calculating the Euclidean norm of the difference stained specimen fluorescence spectrum D8 in the wavelength direction (depth direction). In this way, the separation unit 40 (second separation unit 42) generates a difference stained norm image (difference stained norm data) D9, which is the norm data of the difference stained specimen fluorescence spectrum D8.

[0146] The separation unit 40 can then separate the difference stained sample fluorescence spectrum D8 into a difference stained fluorescence component image D10 containing the fluorescent reagent and a difference stained autofluorescence component image D11 containing the autofluorescence component, using reference spectra R1 and R2 (processing P8 in Figure 8; S18 in Figure 9). Specifically, the separation unit 40 generates the difference stained fluorescence component image D10 and the difference stained autofluorescence component image D11 from the difference stained sample fluorescence spectrum D8 by performing the same color separation process as described above (see P1 in Figure 8).

[0147] As described above, the separation unit 40 can continuously perform a series of processes (P1 to P8) based on the fluorescence spectrum D1 of the stained specimen.

[0148] The separation unit 40 can perform a series of processes (P11-P18) based on the fluorescence spectrum D21 of the unstained specimen in a similar manner.

[0149] In other words, the separation unit 40 can generate a pseudo-unstained fluorescence spectrum D24 based on the unstained fluorescence component image D22 and the fluorescence reference spectrum R1 described above (processing P12 in Figure 8; S24 in Figure 10).

[0150] Furthermore, the separation unit 40 can generate a pseudo-unstained autofluorescence spectrum D25 based on the unstained autofluorescence component image D23 and the autofluorescence reference spectrum R2 described above (processing P13 in Figure 8; S24 in Figure 10).

[0151] The separation unit 40 may generate an unstained specific channel brightness image D26 by determining the brightness value of the spectral data corresponding to a specific channel in the pseudo-unstained autofluorescence spectrum D25 (processing P14 in Figure 8).

[0152] The separation unit 40 can then generate a pseudo-unstained specimen fluorescence spectrum D27 based on the pseudo-unstained fluorescence spectrum D24 and the pseudo-unstained autofluorescence spectrum D25 (processing P15 in Figure 8; S25 in Figure 10).

[0153] The separation unit 40 can then generate a differential unstained sample fluorescence spectrum D28 based on the difference between the unstained sample fluorescence spectrum D21 and the pseudo-unstained sample fluorescence spectrum D27 (processing P16 in Figure 8; S26 in Figure 10).

[0154] The separation unit 40 then obtains a norm image of the differential unstained sample fluorescence spectrum D28 as the differential unstained norm image D29 (processing P17 in Figure 8; S27 in Figure 10). The differential unstained norm image D29 is obtained by calculating the Euclidean norm of the differential unstained sample fluorescence spectrum D28 in the wavelength direction (depth direction). In this way, the separation unit 40 (second separation unit 42) generates a differential unstained norm image (differential unstained norm data) D29, which is the norm data of the differential unstained sample fluorescence spectrum D28.

[0155] The separation unit 40 then separates the differential unstained sample fluorescence spectrum D28 into a differential unstained fluorescent component image D30 containing the fluorescent reagent and a differential unstained autofluorescent component image D31 containing the autofluorescent component, using reference spectra R1 and R2 (processing P18 in Figure 8; S28 in Figure 10). In other words, the separation unit 40 generates the differential unstained fluorescent component image D30 and the differential unstained autofluorescent component image D31 from the differential unstained sample fluorescence spectrum D28 by performing the same color separation process as described above (see P11 in Figure 8).

[0156] (First method for determining the positive threshold) The threshold determination unit 43 can determine a positive threshold for the stained fluorescent component image D2 based on the unstained fluorescent component image D22 (see Figure 8) derived by the separation unit 40 as described above.

[0157] In this example, the positive threshold is determined based on the unstained fluorescent component image D22 obtained from the fluorescence spectrum D21 of the unstained specimen used as the negative control group. Therefore, the image section affected by fluorescence due to the fluorescent reagent 10 can be accurately distinguished from the image section not affected by the fluorescence in the stained fluorescent component image D2 and identified as a positive cell image.

[0158] The specific method for determining the positive threshold in this example is not limited.

[0159] For example, it is possible to determine the positive threshold based on the brightness value of the unstained fluorescent component image D22.

[0160] Figure 14 shows an example of histograms for stained fluorescence component image D2 and unstained fluorescence component image D22. In Figure 14, the X axis represents the brightness value and the Y axis represents the frequency.

[0161] The threshold determination unit 43 may, for example, determine the brightness value (see the symbol "T1" in Figure 14) corresponding to the edge (especially the high-brightness value side edge) of the histogram of the unstained fluorescent component image D22 as the positive threshold.

[0162] There are no specific methods for determining the edges of the histogram in the unstained fluorescent component image D22.

[0163] For example, the maximum brightness value among the unstained fluorescent component images D22 may be determined as the edge of the histogram of the unstained fluorescent component images D22.

[0164] Alternatively, the slope of the histogram of the unstained fluorescent component image D22 (see the symbol "G" in Figure 14) may be determined, and the edges of the histogram of the unstained sample fluorescence spectrum D21 may be determined based on this slope. In this case, the method for determining the "slope location for determining the slope" in the histogram of the unstained fluorescent component image D22 is not limited.

[0165] For example, the gradient location may be determined based on the frequency of brightness values ​​in the unstained fluorescent component image D22. Specifically, the gradient location can be determined in the same way as the method for determining the "positive threshold T2" described later.

[0166] Alternatively, the threshold determination unit 43 may determine and use a luminance value (see the symbol "T2" in Figure 14) determined based on the frequency of luminance values ​​in the unstained fluorescent component image D22 as the positive threshold. For example, the luminance value corresponding to a predetermined area (e.g., 95% of the area from the low-luminance side) of the area of ​​the histogram of the unstained fluorescent component image D22 may be determined as the positive threshold. Alternatively, the luminance value corresponding to a predetermined value (e.g., 95% from the low-luminance side) of the distance between both edges of the histogram of the unstained fluorescent component image D22 (X-axis distance in Figure 14) may be determined as the positive threshold.

[0167] As described above, in this example, the threshold determination unit 43 can determine the positive threshold based solely on the unstained fluorescent component image D22. Therefore, the threshold determination unit 43 can determine the positive threshold without the separation unit 40 performing the above-mentioned processes P2-P8 and P12-P18 (see Figure 8).

[0168] Therefore, the separation unit 40 does not need to perform the processes described above that do not contribute to determining the positive threshold (i.e., processes P2-P8 and P12-P18). In this case, the processing load on the separation unit 40 can be reduced, which can lead to an improvement in the overall processing speed and a reduction in processing time related to the calculation of the positive threshold.

[0169] (Second method for determining the positive threshold) The threshold determination unit 43 may first derive a positive threshold and then correct the positive threshold to determine the final positive threshold.

[0170] In this example, the threshold determination unit 43 derives a positive threshold from the unstained fluorescent component image D22 in the same manner as the first positive threshold determination method described above. Subsequently, the threshold determination unit 43 corrects the positive threshold based on the spectrum of the difference stained fluorescent component image D10 and the spectrum of the difference unstained fluorescent component image D30.

[0171] While there are no specific methods for correcting the positive threshold, typically, the positive threshold can be corrected based on the ratio of the spectrum of the difference unstained fluorescent component image D30 to the spectrum of the difference stained fluorescent component image D10.

[0172] For example, the correction value for the positive threshold can be determined based on a histogram of the spectrum based on its luminance values ​​and frequency (see Figure 14). The edge of the histogram of the spectrum of the differential stained fluorescent component image D10 (e.g., the high-luminance value side edge) is represented as "E1". The corresponding edge of the histogram of the spectrum of the differential unstained fluorescent component image D30 (e.g., the high-luminance value side edge) is represented as "E2". In this case, the threshold determination unit 43 may determine the final positive threshold by using "E1 / E2" as a correction value (correction coefficient) and applying it to the positive threshold (i.e., multiplying it).

[0173] In this example, the positive threshold is corrected using intermediate data derived from both the stained specimen fluorescence spectrum D1 and the unstained specimen fluorescence spectrum D21 (i.e., the difference stained fluorescence component image D10 and the difference unstained fluorescence component image D30).

[0174] Therefore, compared to the first positive threshold determination method described above, in which the positive threshold is determined solely based on intermediate data (i.e., the unstained fluorescence component image D22) derived from the fluorescence spectrum D21 of the unstained specimen, a more accurate positive threshold can be obtained stably. Thus, according to this example, the positive threshold can be corrected to compensate for calculation errors, making it possible to determine a more accurate positive threshold.

[0175] Furthermore, the difference stained fluorescence component image D10 and the difference unstained fluorescence component image D30, which are used to determine the correction value, are intermediate data obtained by processing the fluorescence spectrum D1 of the stained specimen and the fluorescence spectrum D21 of the unstained specimen.

[0176] Therefore, the correction value for the positive threshold can be calculated without requiring any input data other than the input data used to derive the stained fluorescent component image D2 and the unstained fluorescent component image D22. As a result, the deriving of the stained fluorescent component image D2 and the unstained fluorescent component image D22, as well as the deriving of the difference stained fluorescent component image D10 and the difference unstained fluorescent component image D30, can be performed within a series of calculations in the separation unit 40.

[0177] (Third method for determining the positive threshold) The threshold determination unit 43 in this example, similar to the second positive threshold determination method described above, derives a positive threshold from the unstained fluorescent component image D22, and then corrects the positive threshold to determine the final positive threshold.

[0178] In this example, the threshold determination unit 43 corrects the positive threshold based on the difference stained sample fluorescence spectrum D8 and the difference unstained sample fluorescence spectrum D28.

[0179] While there are no specific methods for correcting the positive threshold, typically, the positive threshold can be corrected based on the ratio of the fluorescence spectrum D28 of the unstained sample to the fluorescence spectrum D8 of the stained sample.

[0180] For example, similar to the second positive threshold determination method described above, a correction value for the positive threshold can be determined based on a histogram of the spectrum based on brightness values ​​and frequencies (see Figure 14). The edge of the histogram of the difference stained sample fluorescence spectrum D8 (e.g., the high brightness value side edge) is represented as "E3". The corresponding edge of the histogram of the difference unstained sample fluorescence spectrum D28 (e.g., the high brightness value side edge) is represented as "E4". In this case, the threshold determination unit 43 may determine the final positive threshold by using "E3 / E4" as a correction value (correction coefficient) and applying it to the positive threshold (i.e., multiplying it).

[0181] In this example as well, the positive threshold is corrected using intermediate data derived from both the stained sample fluorescence spectrum D1 and the unstained sample fluorescence spectrum D21 (i.e., the difference stained sample fluorescence spectrum D8 and the difference unstained sample fluorescence spectrum D28).

[0182] Therefore, a highly accurate positive threshold can be reliably obtained, and by correcting the positive threshold to compensate for calculation errors, it is possible to determine the positive threshold with even greater accuracy.

[0183] Furthermore, the positive threshold correction value can be calculated without requiring any input data other than the input data used to derive the stained fluorescent component image D2 and the unstained fluorescent component image D22.

[0184] (Fourth method for determining the positive threshold) In this example, the threshold determination unit 43 corrects the unstained fluorescent component image D22 before deriving the positive threshold from the unstained fluorescent component image D22. That is, the threshold determination unit 43 derives the positive threshold from the corrected unstained fluorescent component image D22.

[0185] Figure 15 shows an example of a difference unstained normed image D29. Figure 16 shows an example of an outlier region (hereinafter also referred to as the "outlier region Rh") in the difference unstained normed image D29. Figure 17 shows an example of the corresponding outlier region Rh in the unstained fluorescent component image D22. Figures 15 to 17 are images showing brightness.

[0186] As shown in Figure 15, the difference unstained normed image D29 may contain sparsely distributed areas with significantly high brightness values ​​(i.e., areas that are significantly brighter). Thus, the occurrence of "areas showing significantly high brightness values" that exist sparsely in space (pixels), for example in pathological specimens, can be caused by tissues that inherently exhibit strong autofluorescence (e.g., red blood cells).

[0187] The brightness values ​​exhibited by tissues that show such strong autofluorescence constitute error values ​​(outliers) that can occur abruptly in the fluorescence spectrum, which can hinder the determination of an appropriate positive threshold. In particular, when the image spectrum obtained by imaging a tissue sample with a low number of red blood cells is used for analysis, the brightness values ​​caused by red blood cells tend to be larger than the brightness values ​​of other regions. As a result, the influence of error values ​​caused by red blood cells on the determination of the positive threshold tends to be larger.

[0188] Furthermore, the optimization of the autofluorescence reference spectrum R2 using non-negative matrix factorization (NMF) described above, by its nature, applies to the entire image. Therefore, applying NMF specifically to reduce or eliminate localized errors caused by tissues exhibiting strong autofluorescence is practically difficult.

[0189] Therefore, the threshold determination unit 43 in this example analyzes the difference unstained normed image D29 to obtain outlier data.

[0190] While there are no specific methods for obtaining outlier data, typically, outlier data can be obtained using the following techniques.

[0191] For example, the threshold determination unit 43 may determine outliers in the difference unstained normed image D29 based on the average value of the pixel brightness values ​​of the difference unstained normed image D29. As an example, brightness values ​​that are 3σ (3 sigma) or more away from the average value of the pixel brightness values ​​of the difference unstained normed image D29 may be defined as outliers. Here, "σ" represents the standard deviation of the pixel brightness values ​​of the difference unstained normed image D29. In this example, the robustness may be inferior to the "example of determining outliers based on the median" exemplified below.

[0192] The threshold determination unit 43 may also determine outliers in the difference unstained normed image D29 based on the median of the pixel brightness values ​​of the difference unstained normed image D29. For example, brightness values ​​that are more than three times the MAD (median absolute deviation) away from the median of the pixel brightness values ​​of the difference unstained normed image D29 may be defined as outliers.

[0193] The threshold determination unit 43 may also determine outliers in the difference unstained normed image D29 based on the quantiles of the pixel brightness values ​​in the difference unstained normed image D29. For example, pixel brightness values ​​that exceed 1.5 times the interquartile range, starting from the upper quartile (75%) of the pixel brightness values ​​in the difference unstained normed image D29, may be defined as outliers.

[0194] The threshold determination unit 43 corrects the unstained fluorescent component image D22 based on the outlier data determined as described above. In other words, the threshold determination unit 43 corrects the unstained fluorescent component image D22 so that the influence of outliers in the unstained fluorescent component image D22 is reduced.

[0195] There are no specific methods for correcting the unstained fluorescence component image D22 based on outlier data, but it can be done as follows, for example.

[0196] Figure 18 shows an example of a histogram of the unstained fluorescent component image D22. Figure 19 shows an example of a histogram of the unstained fluorescent component image D22 after correction based on outlier data. In Figures 18 and 19, the X axis represents the luminance value and the Y axis represents the frequency.

[0197] The threshold determination unit 43 determines a mask threshold Tm based on outliers obtained from the difference unstained norm image D29 as described above (for example, "mask threshold Tm = outlier"). The threshold determination unit 43 may then correct the unstained fluorescent component image D22 by reducing the brightness values ​​of pixels that show brightness values ​​greater than the mask threshold Tm.

[0198] In the examples shown in Figures 18 and 19, pixels in the unstained fluorescent component image D22 that show a brightness value greater than the mask threshold Tm are assigned a "brightness value = 0 (zero)" by this correction method. As a result, the graph region showing a significantly large brightness value in Figure 18 disappears after correction, as shown in Figure 19. In other words, the unstained fluorescent component image D22 is corrected so that the graph region showing a brightness value greater than the mask threshold Tm in Figure 18 moves to the graph region showing "brightness value = 0" in Figure 19.

[0199] The threshold determination unit 43 determines the positive threshold based on the unstained fluorescent component image D22 corrected in this manner. The specific method for determining the positive threshold based on the corrected unstained fluorescent component image D22 is not limited. For example, when performing a correction that assigns "luminance value = 0 (zero)" to pixels in the unstained fluorescent component image D22 that show a luminance value greater than the mask threshold Tm, the positive threshold may be determined based on the maximum luminance value shown in the corrected unstained fluorescent component image D22. That is, the maximum luminance value shown in the corrected unstained fluorescent component image D22 may be determined as the positive threshold.

[0200] Furthermore, the positive threshold determination method in this example can also be applied to the first to third positive threshold determination methods described above.

[0201] This example demonstrates that a highly accurate positive threshold can be reliably obtained while minimizing the influence of outliers. Furthermore, the correction value for the positive threshold can be calculated without requiring any input data other than the input data used to derive the stained fluorescence component image D2 and the unstained fluorescence component image D22.

[0202] Furthermore, according to this example, the threshold determination unit 43 can determine the positive threshold based on the unstained fluorescent component image D22 and the difference unstained norm image D29. Therefore, the threshold determination unit 43 can determine the positive threshold without the separation unit 40 performing the above-mentioned processing P2 to P8 (see Figure 8), thus reducing the processing load on the separation unit 40.

[0203] (Fifth method for determining the positive threshold) In the first to fourth positive threshold determination methods described above, the positive threshold for the stained fluorescent component image D2 is derived based on the image spectral data (particularly the unstained fluorescent component image D22) obtained from the unstained specimen fluorescence spectrum D21.

[0204] On the other hand, it is also possible to derive a positive threshold for the stained fluorescent component image D2 based on image spectral data obtained from the stained specimen fluorescence spectrum D1. The threshold determination unit 43 can also derive and determine the positive threshold based, for example, on image spectral data derived from the stained fluorescent component image D2 and the fluorescence reference spectrum R1.

[0205] In this example of the positive threshold determination method, the positive threshold for the stained fluorescent component image D2 is determined based on the difference stained fluorescent component image D10 (see Figure 8).

[0206] The threshold determination unit 43 may determine the positive threshold, for example, based on the brightness values ​​corresponding to the edges of the histogram of "brightness value (X axis) - frequency (Y axis)" of the difference stained fluorescence component image D10.

[0207] The method for determining the edges of the histogram of the difference-stained fluorescent component image D10 is not limited. The threshold determination unit 43 can determine the edges of the histogram of the difference-stained fluorescent component image D10 by a method similar to the method for determining the edges of the histogram of the unstained fluorescent component image D22 described above (see Figure 14).

[0208] Alternatively, the threshold determination unit 43 may determine a luminance value determined based on the frequency of luminance values ​​in the difference stained fluorescence component image D10 as the positive threshold. For example, the luminance value corresponding to a predetermined area (e.g., 95% of the area from the low luminance value side) of the area of ​​the histogram of the difference stained fluorescence component image D10 may be determined as the positive threshold. Alternatively, the luminance value corresponding to a predetermined value (e.g., 95% from the low luminance value side) of the distance between both edges of the histogram of the difference stained fluorescence component image D10 from the low luminance value side or the high luminance value side may be determined as the positive threshold.

[0209] In this example, the threshold determination unit 43 can determine a positive threshold for the stained fluorescence component image D2 based on the stained specimen fluorescence spectrum D1, the fluorescence reference spectrum R1, and the autofluorescence reference spectrum R2.

[0210] Therefore, the unstained sample fluorescence spectrum D21 and the data derived from the unstained sample fluorescence spectrum D21 are not required to determine the positive threshold for the stained fluorescence component image D2. In other words, according to this example, the separation unit 40 can determine the positive threshold without performing the above-mentioned processing P11 to P18 (see Figure 8).

[0211] Therefore, the separation unit 40 does not need to perform the processes described above that do not contribute to determining the positive threshold (i.e., processes P11 to P18). Also, since the unstained sample fluorescence spectrum D21 is not required to determine the positive threshold, there is no need to prepare the unstained sample fluorescence spectrum D21 in the first place.

[0212] Furthermore, in this example, the difference stained fluorescence component image D10 used to determine the positive threshold corresponds to the calculation error in the color separation process P1 of the stained specimen fluorescence spectrum D1. Therefore, according to the positive threshold determination method in this example, it is possible to determine a positive threshold that is effective in reducing the influence of the calculation error.

[0213] (Example display) Next, an example of the display of image information in the display unit 140 will be explained with reference to Figures 20 to 22.

[0214] Figure 20 shows an example of the display of image information in the display unit 140.

[0215] The image information shown in Figure 20 includes sample image information J1 and presentation information J2.

[0216] The specimen image information J1 is image information based on the stained specimen fluorescence spectrum D1, and is typically a stained fluorescence component image D2 obtained by color separation processing of the stained specimen fluorescence spectrum D1. However, the specimen image information J1 may be an image other than the stained fluorescence component image D2 and is not particularly limited. For example, the specimen image information J1 may be a stained specimen image corresponding to the stained specimen fluorescence spectrum D1, or it may be another image generated based on the stained fluorescence component image D2 or a stained specimen image.

[0217] The stained specimen image referred to here may be, for example, an image obtained by photographing a fluorescently stained specimen 30 with an imaging device. An image obtained by photographing a non-fluorescently stained specimen with an imaging device is called a non-stained specimen image.

[0218] The specimen image information J1 displayed on the display unit 140 may be an image corresponding to the entire range of the fluorescently stained specimen 30 (especially the range to be photographed), or it may be an image corresponding to a part of the fluorescently stained specimen 30.

[0219] When only a portion of the fluorescently stained specimen 30 is displayed on the display unit 140 as specimen image information J1, it is preferable that the area including the cell image labeled with the fluorescent reagent 10 (hereinafter also referred to as the "labeled cell image") (for example, the area including the positive cell image K2) is displayed on the display unit 140.

[0220] Labeled cell images are classified into non-positive cell images K1, which are determined to be non-positive based on the positive threshold, and positive cell images K2, which are determined to be positive based on the positive threshold. For example, in the stained fluorescence component image D2, image sections showing brightness values ​​above the positive threshold (especially labeled cell images) can be classified as positive cell images K2, and image sections showing brightness values ​​below the positive threshold (especially labeled cell images) can be classified as non-positive cell images K1.

[0221] The classification of non-positive cell images K1 and positive cell images K2 may be performed, for example, by the image generation unit 133 (see Figures 1 and 7), or by any device such as the analysis unit 47.

[0222] In the example shown in Figure 20, each labeled cell image displayed on the display unit 140 is highlighted by a cell image position highlighting mark M1. However, the specific display manner of the cell image position highlighting mark M1 (e.g., color, thickness, pattern, and / or shape) is changed between the non-positive cell image K1 and the positive cell image K2. This allows a user viewing the specimen image information J1 displayed on the display unit 140 to intuitively recognize the non-positive cell image K1 and the positive cell image K2 in the specimen image information J1.

[0223] On the other hand, the information J2 displayed on the display unit 140 includes threshold information indicating the positive threshold.

[0224] In the example shown in Figure 20, the positive threshold is indicated by a gauge (indicator). Specifically, the positive threshold used to classify non-positive cell image K1 and positive cell image K2 is indicated by the positive threshold mark Q. The indicator in the presented information J2 in Figure 20 is represented by 16 bits (0 to 65535), and the lower the indicator (i.e., the larger the value), the higher the positive threshold corresponding to the brightness value.

[0225] The user can adjust the positive threshold used to classify non-positive cell images K1 and positive cell images K2 as appropriate by moving the positive threshold mark Q along the indicator via the control unit 160 (see Figure 1).

[0226] In this case, the control unit 150 (see Figure 1) controls the image generation unit 133 and / or the display unit 140 to adjust the position of the positive threshold mark Q displayed on the display unit 140 in response to an adjustment instruction signal input by the user via the operation unit 160.

[0227] Meanwhile, the image generation unit 133 obtains the adjusted positive threshold, corresponding to the adjustment instruction signal input via the operation unit 160, from, for example, the control unit 150. The image generation unit 133 then reclassifies the non-positive cell image K1 and the positive cell image K2 according to the adjusted positive threshold. The image generation unit 133 then generates image information (sample image information J1 and presentation information J2) corresponding to the reclassification result and the adjusted positive threshold, and sends it to the display unit 140.

[0228] As a result, image information generated based on the positive threshold adjusted by the user is displayed on the display unit 140.

[0229] Figure 21 shows another example of the display of image information in the display unit 140. In Figure 21, elements that are the same as or correspond to the elements shown in Figure 20 are denoted by the same reference numerals, and their detailed descriptions are omitted.

[0230] In the example shown in Figure 20 above, a common positive threshold is used throughout the entire sample image information J1 to classify non-positive cells K1 and positive cells K2.

[0231] On the other hand, in the example shown in Figure 21, the positive threshold is determined for each of the multiple observation regions (subregions) Rs1 and Rs2, which are defined by dividing the stained fluorescent component image D2 displayed in the sample image information J1.

[0232] The threshold determination unit 43 (see Figure 7) determines the positive threshold for each of the multiple observation regions Rs1 and Rs2, which are determined by dividing the stained fluorescent component image D2.

[0233] In wide-field tissue images such as those obtained using WSI (Whole Slide Imaging), unique features may appear in each of multiple sub-regions (e.g., regions with high and low background noise). Therefore, there is a need to set a positive threshold for each sub-region of the tissue image and analyze them accordingly.

[0234] The threshold determination unit 43 may, for example, obtain information indicating noise components contained in the stained specimen fluorescence spectrum D1 by analyzing the stained specimen fluorescence spectrum D1, the stained fluorescence component image D2, and / or the stained autofluorescence component image D3. In this case, the threshold determination unit 43 can define multiple observation regions Rs1 and Rs2 by dividing the stained fluorescence component image D2 according to the acquired noise components. This makes it possible to automatically set multiple observation regions Rs1 and Rs2 by dividing the image of the specimen image information J1 according to the magnitude of the background noise.

[0235] In the example shown in Figure 21, the second observation region Rs2, which exhibits relatively large background noise, is enclosed by the observation region highlighting mark M2. The region outside the observation region highlighting mark M2 is the first observation region Rs1, which exhibits relatively small background noise.

[0236] Furthermore, a user interface that allows users to adjust thresholds for each arbitrary region may be required. For this reason, the user interface may allow the user to specify multiple observation regions Rs1 and Rs2, and a positive threshold may be set for each of the specified observation regions Rs1 and Rs2.

[0237] In other words, the threshold determination unit 43 may determine a positive threshold for each of the multiple observation regions Rs1 and Rs2 defined by the user.

[0238] The method by which the user specifies multiple observation areas Rs1 and Rs2 is not limited. For example, the user may specify multiple observation areas Rs1 and Rs2 in any way they like by operating the control unit 160 (see Figure 1) while viewing the specimen image information J1 (stained fluorescence component image D2) on the display unit 140.

[0239] The control unit 150 can acquire information about multiple observation regions Rs1 and Rs2 specified by the user from the operation unit 160 and provide it directly or indirectly to the processing unit 130. The processing unit 130 (for example, the separation processing unit 132 and the image generation unit 133) may then determine the positive threshold and generate image information based on the information about the multiple observation regions Rs1 and Rs2. As a result, image information based on the multiple observation regions Rs1 and Rs2 specified by the user can be displayed on the display unit 140.

[0240] In the example shown in Figure 21, the user can adjust the positive threshold by moving the positive threshold marks Q1 and Q2 via the control unit 160.

[0241] Positive threshold marks may be provided for each observation region. The first positive threshold mark Q1 shown in Figure 21 is provided for the first observation region Rs1, and the second positive threshold mark Q2 is provided for the second observation region Rs2.

[0242] The user can change and adjust the positive thresholds assigned to the first observation area Rs1 and the second observation area Rs2, respectively, by moving the first positive threshold mark Q1 and the second positive threshold mark Q2 via the operation unit 160, similar to the positive threshold mark Q shown in Figure 20.

[0243] Figure 22 shows another example of the display of image information in the display unit 140. In Figure 22, elements that are the same as or correspond to the elements shown in Figures 20 and 21 are denoted by the same reference numerals, and their detailed descriptions are omitted.

[0244] In the example shown in Figure 22, the information J2 displayed on the display unit 140 includes not only threshold information indicating the positive threshold, but also correctable range information indicating the correctable range of the positive threshold.

[0245] The information J2 shown in Figure 22 includes the display of the correctable upper limit Lu and correctable lower limit Ld, in addition to the positive threshold mark Q.

[0246] The display of the upper limit value Lu and the lower limit value Ld that can be corrected indicates the upper limit value and the lower limit value of the correctable range of the positive threshold value, respectively. Therefore, in the indicator of the presentation information J2, the positive threshold mark Q basically points to somewhere within the range defined by the upper limit value Lu and the lower limit value Ld that can be corrected.

[0247] The correctable range of the positive threshold value determined by the upper limit value Lu and the lower limit value Ld that can be corrected can be displayed in any form. For example, the inside and outside of the correctable range of the positive threshold value may be displayed in different colors or patterns. Also, in the indicator of the presentation information J2, a display such as a line indicating the upper limit value Lu and the lower limit value Ld that can be corrected may be represented.

[0248] The user can adjust the positive threshold value by moving the positive threshold mark Q via the operation unit 160 while using the correctable range of the positive threshold value shown in the presentation information J2 as a reference.

[0249] The correctable range of the positive threshold value (that is, the upper limit value Lu and the lower limit value Ld that can be corrected) can be determined by the threshold determination unit 43 (see FIG. 7). Information indicating the correctable range determined by the threshold determination unit 43 is output from the separation output unit 44 and sent to the analysis unit 47, the image generation unit 133, and the like.

[0250] Note that the specific method for determining the correctable range of the positive threshold value is not limited.

[0251] As an example, the correctable range of the positive threshold value may be determined based on the positive threshold values determined by each of a plurality of positive threshold value determination methods. For example, the lower limit value Ld that can be corrected may be determined based on the minimum value among the positive threshold values determined by the first to fifth positive threshold value determination methods described above, and the upper limit value Lu that can be corrected may be determined based on the maximum value among the positive threshold values.

[0252] Alternatively, the correctable range of the positive threshold may be determined based on a correction value assigned to the positive threshold (for example, a predetermined correction value as described in the first modified example below (Figures 23 and 24)). In this case, the threshold determination unit 43 may obtain the data of the correctable range of the positive threshold by reading it from, for example, a storage unit (for example, a database 200 or an information storage unit 121).

[0253] The correctable range of the positive threshold determined in this way is displayed on the display unit 140 as described above, but it may also be sent to another device and used for processing by analysis software, etc.

[0254] Figures 20 to 22 above merely show examples of displays in the display unit 140, and the display unit 140 may display image information in any other form.

[0255] As described above, according to this embodiment (information processing device and information processing method), the positive threshold used in the analysis of the stained fluorescence component image D2 can be determined based on the sample fluorescence spectrum (stained sample fluorescence spectrum D1 and / or unstained sample fluorescence spectrum D21).

[0256] By using a positive threshold determined without subjective user input, it is possible to prevent variability in the analysis results of stained fluorescence component images D2 among users and obtain stable, highly accurate analysis results. Furthermore, even users who are not expert operators skilled in analysis can obtain highly accurate analysis results.

[0257] Furthermore, since the positive threshold can be automatically determined from the fluorescence spectrum D1 of stained specimens and / or the fluorescence spectrum D21 of unstained specimens, the preparation work for analysis can be streamlined. As a result, the effort required from the user for analysis processing and the time spent on user preparation can be reduced, promoting faster and more accurate result calculation in clinical research and diagnosis.

[0258] Furthermore, the determined positive threshold can be automatically displayed on the display unit 140. The user can check the positive threshold by looking at the presentation information J2 displayed on the display unit 140, and at the same time check the specimen image information J1 (especially the non-positive cell image K1 and the positive cell image K2) displayed on the display unit 140.

[0259] Furthermore, the above-described embodiment can determine the positive threshold regardless of the phenotype of the individual stained specimen fluorescence spectrum D1 (corresponding specimen image information J1), thus offering high applicability regardless of the XY spatial characteristics of the stained specimen fluorescence spectrum D1.

[0260] (First variation) The threshold determination unit 43 may correct the positive threshold based on a predetermined correction value.

[0261] The method for correcting the positive threshold using predetermined correction values ​​is not limited. Typically, it is possible to pre-determine the limit values ​​(i.e., upper and / or lower limits) that define the range of values ​​the positive threshold can take as correction values. It is also possible to pre-determine the correction coefficient used in multiplication of the positive threshold as a correction value.

[0262] Such correction values ​​can be determined, for example, depending on the fluorescent reagent, or depending on the combination of the fluorescent reagent and the object to be labeled with the fluorescent reagent.

[0263] In this context, "labeling target" refers to a substance that can be labeled with a fluorescent reagent (for example, a substance that emits fluorescence when reacted with a fluorescent reagent). Typically, antibodies and other targets may be included as labeling targets, but other cells and tissues (e.g., organs, cancer cells, and other cells / tissues) may also be included as labeling targets.

[0264] The database 200 and storage unit 120 (e.g., information storage unit 121) shown in Figure 1 can be used as a correction data storage unit for storing correction values ​​that can be used to correct the positive threshold. For example, the database 200 can pre-store reagent identification information 11 and corresponding correction values ​​in association with each of the multiple fluorescent reagents that can be used. The database 200 can also store label target identification information, reagent identification information 11 and correction values ​​in association with each of the multiple fluorescent reagents that can be used and each of the multiple objects that can be labeled.

[0265] Here, the tagged object identification information is information that identifies the tagged object. In this example, the tagged object identification information is included in the sample identification information 21, and the tagged object identification information is associated with the sample 20.

[0266] A single fluorescent reagent may be used for two or more different labeling targets. That is, the fluorescent reagent may be the same, but the labeling targets may differ, and database 200 can store and associate different correction values ​​for each of these cases.

[0267] Figures 23 and 24 show examples of correction values ​​stored in the correction data storage unit (e.g., database 200).

[0268] In the example shown in Figure 23, the corresponding "lower limit" and "upper limit" for each combination of "dye" and "antibody (labeled object)" are stored as correction values ​​in the correction data storage unit. On the other hand, in the example shown in Figure 24, the corresponding "coefficient" for each combination of "dye" and "antibody (labeled object)" is stored as a correction value in the correction data storage unit.

[0269] In the correction data storage unit, the correction values ​​are stored in any format, such as a lookup table.

[0270] The information acquisition unit 111 (see FIG. 1) can read and acquire a corresponding correction value from the database 200 based on the reagent identification information 11 associated with the fluorescent reagent 10 used in the fluorescently stained specimen 30. Alternatively, the information acquisition unit 111 can read and acquire a corresponding correction value from the database 200 based on the reagent identification information 11 and the specimen identification information 21 (particularly the labeled target identification information) associated with the fluorescent reagent 10 and the specimen 20 used in the fluorescently stained specimen 30.

[0271] Then, the information acquisition unit 111 stores the correction value read from the database 200 in the information storage unit 121.

[0272] The correction value stored in the information storage unit 121 is directly or indirectly acquired by the separation processing unit 132 (threshold determination unit 43 (see FIG. 7)) and is used for correcting the positive threshold in the threshold determination unit 43 (see FIG. 7). Thus, the threshold determination unit 43 can acquire a correction value from a correction data storage unit that mutually associates and stores the reagent identification information and the correction value based on the reagent identification information via the acquisition unit 110 and the storage unit 120. Similarly, the threshold determination unit 43 can acquire a correction value from a correction data storage unit that mutually associates and stores the labeled target identification information, the reagent identification information, and the correction value based on the labeled target identification information and the reagent identification information 11 via the acquisition unit 110 and the storage unit 120.

[0273] For example, in the example shown in FIG. 23, assume a case where the positive threshold derived by the threshold determination unit 43 is "600" when the fluorescently stained specimen 30 is derived from "AF488-CK".

[0274] In this case, as is clear from FIG. 23, the positive threshold exceeds the upper limit value (i.e., "500") assigned to "AF488-CK". Therefore, the threshold determination unit 43 corrects the positive threshold and changes it to "500".

[0275] The separation output unit 44 outputs the positive threshold after correction by the threshold determination unit 43, and the corrected positive threshold is used in subsequent devices (for example, the analysis unit 47 and image generation unit 133 in Figure 7).

[0276] By defining a limit for the positive threshold in this way, it is possible to prevent unexpectedly large or extremely small values ​​from being set as the positive threshold.

[0277] On the other hand, in the example shown in Figure 24, if the fluorescently stained specimen 30 is derived from "AF488-CK", there is a tendency for the positive threshold derived by the threshold determination unit 43 to be higher than the actual value.

[0278] In this case, the threshold determination unit 43 can suppress the influence of such a tendency by correcting the positive threshold by multiplying it by an appropriate coefficient smaller than 1 (0.92 in the example shown in Figure 24).

[0279] Furthermore, if the positive threshold derived by the threshold determination unit 43 tends to be smaller than the original value, an appropriate coefficient greater than 1 may be used as a correction value (see "AF532-CD68" in Figure 24).

[0280] In this example, correction values ​​corresponding to each reagent that can be used and / or each label target that can be detected are pre-stored as database information in the correction data storage unit. The threshold determination unit 43 can determine the final positive threshold by obtaining the correction value corresponding to the actual fluorescent stained specimen 30 from the correction values ​​pre-stored in the correction data storage unit and applying it to the positive threshold.

[0281] Therefore, the positive threshold is corrected according to the "fluorescent reagent" and the "combination of fluorescent reagent and labeling target" used in the fluorescently stained specimen 30. As a result, even if the threshold determination unit 43 mistakenly derives a value that deviates significantly from the original value as the positive threshold for some reason, it is possible to prevent such an incorrect value from being used as the positive threshold.

[0282] The "correction values ​​used to correct the positive threshold" stored in the correction data storage unit (e.g., database 200) may be updated as appropriate. For example, a user may update the correction values ​​stored in the correction data storage unit at an appropriate time (e.g., periodically).

[0283] (Examples of application) The above-described information processing system may include an imaging device (e.g., including a scanner) for acquiring fluorescence spectra and an information processing device for processing using fluorescence spectra. In this case, the fluorescence signal acquisition unit 112 shown in Figure 1 may be implemented by the imaging device, and the other components may be implemented by the information processing device. Alternatively, the above-described information processing system may include an imaging device for acquiring fluorescence spectra and software used for processing using fluorescence spectra. In other words, the information processing system does not need to have a physical configuration (e.g., memory or processor) for storing or executing the software. In this case, the fluorescence signal acquisition unit 112 shown in Figure 1 may be implemented by the imaging device, and the other components may be implemented by the information processing device on which the software is executed. The software may be provided to the information processing device via a network, for example, from a website or cloud server, or via any storage medium (e.g., a disk). The information processing device on which the software is executed may be various types of servers (e.g., cloud servers), general-purpose computers, PCs, or tablet PCs. The method by which the software is provided to the information processing device and the type of information processing device are not limited to those described above. Furthermore, it should be noted that the configuration of the information processing system described above is not necessarily limited to the above configuration, and that a configuration that can be conceived by a person skilled in the art may be applied based on the level of technology at the time of use.

[0284] (Examples of applications to microscope systems) The above information processing system may be implemented, for example, as a microscope system. Referring to Figure 25, an example configuration of a microscope system that implements the above information processing system will be described.

[0285] The microscope system shown in Figure 25 comprises a microscope 101 and a data processing unit 107. Figure 25 shows an example of a measurement system capable of capturing wide-field areas of fluorescently stained specimens 30 and unstained specimens, and this measurement system can also be applied to WSI, for example.

[0286] The microscope 101 comprises a stage 102, an optical system 103, a light source 104, a stage drive unit 105, a light source drive unit 106, and a fluorescence signal acquisition unit 112.

[0287] The stage 102 has a mounting surface on which fluorescently stained specimens 30 and non-fluorescently stained specimens can be placed, and is provided to be movable in a horizontal direction (xy plane direction) and a vertical direction (z axis direction) parallel to the mounting surface by the drive of the stage drive unit 105. The fluorescently stained specimen 30 has a thickness of, for example, several μm to several tens of μm in the Z axis direction, and is fixed by a predetermined method while being sandwiched between a glass slide SG and a cover glass (not shown).

[0288] An optical system 103 is positioned above the stage 102. The optical system 103 comprises an objective lens 103A, an imaging lens 103B, a dichroic mirror 103C, an emission filter 103D, and an excitation filter 103E. The light source 104 is, for example, a light bulb such as a mercury lamp or an LED (Light Emitting Diode), and emits light when driven by a light source drive unit 106. The light emitted from the light source 104 is guided through the optical system 103 to the fluorescently stained specimen 30 or a non-fluorescently stained specimen on the mounting surface of the stage 102.

[0289] The excitation filter 103E generates excitation light by transmitting only the light of the excitation wavelength that excites the fluorescent dye from the light source 104 when obtaining fluorescence images of the fluorescently stained specimen 30 and the non-fluorescently stained specimen. The dichroic mirror 103C reflects the excitation light that passes through the excitation filter 103E and guides it to the objective lens 103A. The objective lens 103A focuses the excitation light onto the fluorescently stained specimen 30. The objective lens 103A and the imaging lens 103B magnify the image of the fluorescently stained specimen 30 to a predetermined magnification and project the magnified image onto the imaging surface of the fluorescence signal acquisition unit 112.

[0290] When excitation light is shone on the fluorescently stained specimen 30, the staining agent (fluorescent reagent 10) and autofluorescent components bound to each tissue in the fluorescently stained specimen 30 emit fluorescence. This fluorescence passes through the objective lens 103A, then through the dichroic mirror 103C, and reaches the imaging lens 103B via the emission filter 103D. The emission filter 103D is magnified by the objective lens 103A, absorbing a portion of the light that has passed through the excitation filter 103E, and transmitting only a portion of the colored light. The image of the colored light, from which the ambient light has been lost, is magnified by the imaging lens 103B as described above and formed on the fluorescence signal acquisition unit 112.

[0291] Alternatively, a spectrometer (not shown) may be provided instead of the imaging lens 103B shown in Figure 25. This spectrometer can be constructed using one or more prisms, lenses, etc., and spectrally separates the fluorescence from the fluorescently stained specimen 30 or the non-fluorescently stained specimen in a predetermined direction. In this case, the fluorescence signal acquisition unit 112 is configured as a photodetector that detects the light intensity for each wavelength of fluorescence spectrally separated by the spectrometer, and inputs the detected fluorescence signal to the data processing unit 107.

[0292] The data processing unit 107 drives the light source 104 via the light source drive unit 106, acquires fluorescence spectra / fluorescence images of the fluorescently stained specimen 30 and the unstained specimen using the fluorescence signal acquisition unit 112, and performs various processing using the acquired fluorescence spectra / fluorescence images. More specifically, the data processing unit 107 can function as part or all of the information acquisition unit 111, storage unit 120, processing unit 130, display unit 140, control unit 150, operation unit 160, or database 200 of the information processing device 100 shown in Figure 1. The data processing unit 107 may also include an analysis unit 47 (see Figure 7) that performs analysis based on a positive threshold. For example, by functioning as the control unit 150 of the information processing device 100, the data processing unit 107 controls the driving of the stage drive unit 105 and the light source drive unit 106, and controls the acquisition of spectra by the fluorescence signal acquisition unit 112. Furthermore, the data processing unit 107 functions as the processing unit 130 of the information processing unit 100, generating image spectral data, calculating positive thresholds, performing analysis, and generating image information.

[0293] As described above, in the microscope system shown in Figure 25, at least the light source 104, excitation filter 103E, dichroic mirror 103C, and objective lens 103A function as a light irradiation unit that emits excitation light to excite the fluorescent reagent 10. The fluorescence signal acquisition unit 112 functions as an imaging device that captures the fluorescence spectrum of the specimen (fluorescent stained specimen 30 or unstained specimen) being irradiated with excitation light. The data processing unit 107 functions as an information processing device that performs analysis of the fluorescence spectrum of the specimen.

[0294] It should be noted that the apparatus described above with reference to Figure 25 is merely an example, and the measurement systems relating to the embodiments and modifications described above are not limited to the examples shown in Figure 25. For example, the microscope system does not necessarily have to have all of the configurations shown in Figure 25, and may have configurations not shown in Figure 25.

[0295] The above embodiments and modifications can be realized using a measurement system capable of acquiring image data (hereinafter referred to as "wide-field image data") with sufficient resolution for the entire area to be photographed or a required area within the area to be photographed (hereinafter also referred to as the "region of interest"). For example, the above embodiments and modifications can be realized using a measurement system capable of photographing the entire area to be photographed or a required area within the area to be photographed (hereinafter referred to as the "region of interest") at once, or a measurement system that acquires images of the entire area to be photographed or the region of interest by line scanning.

[0296] In the microscope system shown in Figure 25, in cases such as WSI where the entire imaging area exceeds the area from which image data can be acquired in a single shot (hereinafter referred to as "field of view"), the stage 102 is moved to shift the field of view with each shot, so that each field of view is captured sequentially. By tiling the image data obtained from each field of view (hereinafter referred to as "field of view image data"), a wide-field image data of the entire imaging area is generated. The generated wide-field image data is stored, for example, in the fluorescence signal storage unit 122 (see Figure 1). Note that the tiling of the field of view image data may be performed in the acquisition unit 110 of the information processing device 100, in the storage unit 120, or in the processing unit 130.

[0297] The processing unit 130 can then perform a series of processes on the obtained wide-field image data, including the process of acquiring a positive threshold.

[0298] (Method for calculating the number of fluorescent molecules or antibodies) Next, we will explain how to calculate the number of fluorescent molecules or antibodies in one pixel. Figure 26 is a schematic diagram illustrating the method for calculating the number of fluorescent molecules or antibodies in one pixel. In the example shown in Figure 26, when the image sensor and sample are arranged with an objective lens in between, let's assume that the size of the bottom surface of the sample corresponding to 1 [pixel] of the image sensor is 13 / 20 (μm) × 13 / 20 (μm). Let's also assume that the thickness of the sample is 10 (μm). In that case, the volume of this rectangular prism of sample is expressed as 13 / 20 (μm) × 13 / 20 (μm) × 10 (μm). The volume of this sample (liters) is expressed as 13 / 20 (μm) × 13 / 20 (μm) × 10 (μm) × 10³.

[0299] Furthermore, assuming that the concentration of antibodies (or fluorescent molecules) in the sample is uniform and 300 (nM), the number of antibodies per pixel can be expressed by the following equation (24).

[0300]

number

[0301] In this way, the number of fluorescent molecules or antibodies in the fluorescently stained specimen 30 is calculated as a result of the fluorescence separation process, allowing the user to compare the number of fluorescent molecules among multiple fluorescent substances or compare data imaged under different conditions. Furthermore, since brightness (or fluorescence intensity) is a continuous value while the number of fluorescent molecules or antibodies is a discrete value, the information processing device 100 can reduce the amount of data by outputting image information based on the number of fluorescent molecules or antibodies.

[0302] (Example hardware configuration) Referring to Figure 27, an example of the hardware configuration of the information processing device 100 will be described. Figure 27 is a block diagram showing an example of the hardware configuration of the information processing device 100. Various processes performed by the information processing device 100 are realized through the cooperation of software and the hardware described below.

[0303] As shown in Figure 27, the information processing device 100 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, and a host bus 904a. The information processing device 100 also includes a bridge 904, an external bus 904b, an interface 905, an input device 906, an output device 907, a storage device 908, a drive 909, a connection port 911, a communication device 913, and a sensor 915. The information processing device 100 may have a processing circuit such as a DSP or ASIC in place of, or together with, the CPU 901.

[0304] The CPU 901 functions as both an arithmetic processing unit and a control unit, controlling the overall operation of the information processing unit 100 according to various programs. The CPU 901 may also be a microprocessor. The ROM 902 stores programs and arithmetic parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901 and parameters that change as needed during its execution. For example, the CPU 901 can embody at least the processing unit 130 and the control unit 150 of the information processing unit 100.

[0305] The CPU 901, ROM 902, and RAM 903 are interconnected by a host bus 904a, which includes the CPU bus. The host bus 904a is connected to an external bus 904b, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 904. Note that the host bus 904a, bridge 904, and external bus 904b do not necessarily need to have separate configurations from each other; they may be implemented in a single configuration (e.g., a single bus).

[0306] The input device 906 can be implemented by a device into which information is input by the user, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever. Alternatively, the input device 906 may be a remote control device utilizing infrared or other radio waves, or an external device such as a mobile phone or PDA that is compatible with the operation of the information processing device 100. Furthermore, the input device 906 may include, for example, an input control circuit that generates an input signal based on the information input by the user using the above-mentioned input means and outputs it to the CPU 901. By operating this input device 906, the user can input various types of data to the information processing device 100 or instruct it to perform processing operations. The input device 906 can, for example, embody at least the operation unit 160 of the information processing device 100.

[0307] The output device 907 is formed of a device capable of visually or audibly notifying the implementer of the acquired information. Examples of such devices include display devices such as CRT displays, liquid crystal displays, plasma displays, EL displays, and lamps, as well as audio output devices such as speakers and headphones, and printers. The output device 907 can, for example, embody at least the display unit 140 of the information processing device 100.

[0308] The storage device 908 is a device for storing data. The storage device 908 can be implemented by, for example, a magnetic storage device such as an HDD, a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 908 may also include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deletion device for deleting data recorded on the storage medium. This storage device 908 stores programs executed by the CPU 901, various data, and various data acquired from external sources. The storage device 908 can, for example, embody at least the storage unit 120 of the information processing device 100.

[0309] Drive 909 is a reader / writer for storage media and is either built into or external to the information processing unit 100. Drive 909 reads information recorded on removable storage media such as magnetic disks, optical disks, magneto-optical disks, or semiconductor memory and outputs it to RAM 903. Drive 909 can also write information to removable storage media.

[0310] Connection port 911 is an interface for connecting to external devices, and is a connection port for external devices that can transmit data via, for example, USB (Universal Serial Bus).

[0311] The communication device 913 is, for example, a communication interface formed by a communication device for connecting to the network 920. The communication device 913 is, for example, a communication card for wired or wireless LAN (Local Area Network), LTE (Long Term Evolution), Bluetooth (registered trademark), or WUSB (Wireless USB). Alternatively, the communication device 913 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication. This communication device 913 can, for example, send and receive signals to and from the Internet or other communication devices in accordance with a predetermined protocol such as TCP / IP.

[0312] In this embodiment, the sensor 915 includes a sensor capable of acquiring a spectrum (e.g., an image sensor), but may also include other sensors (e.g., an acceleration sensor, a gyroscope, a geomagnetic sensor, a pressure sensor, a sound sensor, and a distance sensor). The sensor 915 can, for example, embody at least the fluorescence signal acquisition unit 112 of the information processing device 100.

[0313] Network 920 is a wired or wireless transmission path for information transmitted from devices connected to Network 920. For example, Network 920 may include public networks such as the Internet, telephone networks, and satellite communication networks, as well as various LANs (Local Area Networks) and WANs (Wide Area Networks), including Ethernet®. Network 920 may also include dedicated network lines such as IP-VPN (Internet Protocol - Virtual Private Network).

[0314] The above describes an example of a hardware configuration that can realize the functions of the information processing device 100. Each of the above components may be realized using general-purpose materials, or it may be realized using hardware specialized for the function of each component. Therefore, it is possible to change the hardware configuration used as appropriate, depending on the level of technology at the time of implementing this disclosure.

[0315] Furthermore, it is possible to create computer programs to realize each of the functions of the information processing device 100 as described above and implement them on a PC or the like. A computer-readable recording medium containing such computer programs can also be provided. The recording medium includes, for example, magnetic disks, optical disks, magneto-optical disks, and flash memory. Alternatively, the computer programs may be distributed without using a recording medium, for example, via a network.

[0316] It should be noted that the embodiments and modifications disclosed herein are illustrative in all respects and should not be construed restrictively. The embodiments and modifications described above may be omitted, substituted, and modified in various ways without departing from the scope and spirit of the appended claims. For example, the embodiments and modifications described above may be combined in whole or in part, and other embodiments may be combined with the embodiments or modifications described above. Furthermore, the effects described herein are illustrative, and other effects may result.

[0317] The technical categories that embody the above-described technical concept are not limited. For example, the above-described technical concept may be embodied by a computer program that causes a computer to execute one or more steps included in a method for manufacturing or using the above-described device. Alternatively, the above-described technical concept may be embodied by a computer-readable, non-transitory recording medium on which such a computer program is recorded.

[0318] This disclosure can also take the following configuration.

[0319] [Item 1] A first separation unit separates the fluorescence spectrum of a stained specimen, obtained by irradiating an excitation light onto a fluorescently stained specimen obtained by labeling a specimen with a fluorescent reagent, into a stained fluorescent component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component, using a fluorescence reference spectrum and an autofluorescence reference spectrum. A second separation unit separates the fluorescence spectrum of an unstained specimen, obtained by irradiating an unstained specimen not labeled with the fluorescent reagent with the excitation light, into an unstained fluorescent component image containing the fluorescent reagent and an unstained autofluorescent component image containing the autofluorescent component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. A threshold determination unit determines a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the unstained fluorescent component image, and is compared with the image data of multiple image sections included in the stained fluorescent component image. A threshold output unit that outputs the positive threshold, An information processing device equipped with the following features.

[0320] [Item 2] The first separation unit is, Based on the stained fluorescence component image and the fluorescence reference spectrum, a pseudo-stained fluorescence spectrum is generated. Based on the stained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-stained autofluorescence spectrum is generated. Based on the pseudo-stained fluorescence spectrum and the pseudo-stained autofluorescence spectrum, a pseudo-stained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the stained specimen and the fluorescence spectrum of the pseudo-stained specimen, a differential stained specimen fluorescence spectrum is generated. The fluorescence spectrum of the difference-stained specimen is separated into a difference-stained fluorescence component image containing the fluorescent reagent and a difference-stained autofluorescence component image containing the autofluorescence component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. The second separation unit is, Based on the unstained fluorescent component image and the fluorescence reference spectrum, a pseudo-unstained fluorescence spectrum is generated. Based on the unstained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-unstained autofluorescence spectrum is generated. Based on the pseudo-unstained fluorescence spectrum and the pseudo-unstained autofluorescence spectrum, a pseudo-unstained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the unstained specimen and the fluorescence spectrum of the pseudo-unstained specimen, a differential fluorescence spectrum of the unstained specimen is generated. The fluorescence spectrum of the difference unstained sample is separated into a difference unstained fluorescent component image containing the fluorescent reagent and a difference unstained autofluorescent component image containing the autofluorescent component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. The threshold determination unit corrects the positive threshold based on the spectrum of the difference stained fluorescent component image and the spectrum of the difference unstained fluorescent component image. The information processing device described in item 1.

[0321] [Item 3] The first separation unit is, Based on the stained fluorescence component image and the fluorescence reference spectrum, a pseudo-stained fluorescence spectrum is generated. Based on the stained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-stained autofluorescence spectrum is generated. Based on the pseudo-stained fluorescence spectrum and the pseudo-stained autofluorescence spectrum, a pseudo-stained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the stained specimen and the fluorescence spectrum of the pseudo-stained specimen, a differential stained specimen fluorescence spectrum is generated. The second separation unit is, Based on the unstained fluorescent component image and the fluorescence reference spectrum, a pseudo-unstained fluorescence spectrum is generated. Based on the unstained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-unstained autofluorescence spectrum is generated. Based on the pseudo-unstained fluorescence spectrum and the pseudo-unstained autofluorescence spectrum, a pseudo-unstained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the unstained specimen and the fluorescence spectrum of the pseudo-unstained specimen, a differential fluorescence spectrum of the unstained specimen is generated. The threshold determination unit corrects the positive threshold based on the difference stained sample fluorescence spectrum and the difference unstained sample fluorescence spectrum. The information processing device described in item 1.

[0322] [Item 4] The second separation unit is, Based on the unstained fluorescent component image and the fluorescence reference spectrum, a pseudo-unstained fluorescence spectrum is generated. Based on the unstained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-unstained autofluorescence spectrum is generated. Based on the pseudo-unstained fluorescence spectrum and the pseudo-unstained autofluorescence spectrum, a pseudo-unstained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the unstained specimen and the fluorescence spectrum of the pseudo-unstained specimen, a differential fluorescence spectrum of the unstained specimen is generated. The difference unstained norm data, which is the norm data of the fluorescence spectrum of the difference unstained sample, is generated. The threshold determination unit, The aforementioned difference unstained normed data is analyzed to obtain outlier data. Based on the outlier data, the unstained fluorescence component image is corrected. The positive threshold is determined based on the corrected unstained fluorescent component image. The information processing device described in item 1.

[0323] [Item 5] The threshold determination unit corrects the positive threshold based on a correction value predetermined according to the fluorescent reagent. An information processing device as described in any of items 1 to 4.

[0324] [Item 6] The threshold determination unit acquires the correction value from a correction data storage unit that stores the reagent identification information and the correction value in relation to each other, based on the reagent identification information associated with the fluorescent reagent. The information processing device described in item 5.

[0325] [Item 7] The threshold determination unit corrects the positive threshold based on a correction value predetermined according to the combination of the fluorescent reagent and the object to be labeled with the fluorescent reagent. An information processing device as described in any of items 1 to 3.

[0326] [Item 8] The threshold determination unit acquires the correction value from a correction data storage unit that stores the label target identification information associated with the sample, the reagent identification information associated with the fluorescent reagent, and the correction value in relation to each other, based on the label target identification information associated with the sample and the reagent identification information associated with the fluorescent reagent. The information processing device described in item 7.

[0327] [Item 9] The threshold determination unit determines the positive threshold for each of the multiple observation regions determined by dividing the stained fluorescent component image. An information processing device as described in any of items 1 to 8.

[0328] [Item 10] The threshold determination unit determines the positive threshold for each of the plurality of observation areas defined by the user. The information processing device described in item 9.

[0329] [Item 11] The information processing device according to item 9, wherein the threshold determination unit identifies noise components included in the fluorescence spectrum of the stained specimen and determines the plurality of observation regions by dividing the stained fluorescence component image according to the noise components.

[0330] [Item 12] The threshold determination unit determines the correctable range of the positive threshold, The threshold output unit outputs information indicating the positive threshold and the correctable range. An information processing device as described in any of items 1 through 11.

[0331] [Item 13] A light irradiation unit that irradiates excitation light to excite a fluorescent reagent, An imaging device that images a sample being irradiated with the excitation light and acquires the fluorescence spectrum of the sample, The system comprises an information processing device for analyzing the fluorescence spectrum of the sample, The aforementioned information processing device is A first separation unit separates the fluorescence spectrum of a stained specimen obtained by irradiating the fluorescently stained specimen, which is obtained by labeling the specimen with the fluorescent reagent, into a stained fluorescent component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component, using a fluorescence reference spectrum and an autofluorescence reference spectrum. A second separation unit separates the fluorescence spectrum of an unstained specimen, obtained by irradiating an unstained specimen not labeled with the fluorescent reagent with the excitation light, into an unstained fluorescent component image containing the fluorescent reagent and an unstained autofluorescent component image containing the autofluorescent component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. A threshold determination unit that determines a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the unstained fluorescent component image and compared with the image data of multiple image sections included in the stained fluorescent component image. Microscope system.

[0332] [Item 14] The display unit includes a display information generation unit that generates display information, which includes threshold information indicating the positive threshold, for display on the display unit. The microscope system described in item 13.

[0333] [Item 15] The threshold determination unit determines the correctable range of the positive threshold, The aforementioned information includes correctable range information indicating the correctable range. The microscope system described in item 14.

[0334] [Item 16] The system includes an analysis unit that performs analysis based on the positive threshold. A microscope system as described in any of items 13-15.

[0335] [Item 17] The process involves separating the fluorescence spectrum of a stained specimen obtained by irradiating an excitation light onto a fluorescently stained specimen obtained by labeling a specimen with a fluorescent reagent into a stained fluorescent component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component, using a fluorescence reference spectrum and an autofluorescence reference spectrum. A step of separating the fluorescence spectrum of an unstained specimen obtained by irradiating an unstained specimen not labeled with the fluorescent reagent with the excitation light into an unstained fluorescent component image containing the fluorescent reagent and an unstained autofluorescent component image containing the autofluorescent component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. A step of determining a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the unstained fluorescent component image, and which is compared with the image data of multiple image sections included in the stained fluorescent component image. The process of outputting the positive threshold, Information processing methods including

[0336] [Item 18] A first separation unit separates the fluorescence spectrum of a stained specimen, obtained by irradiating an excitation light onto a fluorescently stained specimen obtained by labeling a specimen with a fluorescent reagent, into a stained fluorescent component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component, using a fluorescence reference spectrum and an autofluorescence reference spectrum. A threshold determination unit determines a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the stained fluorescent component image and the fluorescence reference spectrum, and compares the image data of multiple image sections included in the stained fluorescent component image with the positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the stained fluorescent component image and the fluorescence reference spectrum. A threshold output unit that outputs the positive threshold, An information processing device equipped with the following features.

[0337] [Item 19] The first separation unit is, Based on the stained fluorescent component image and the fluorescence reference spectrum, a pseudo-stained fluorescent reagent spectrum is generated. Based on the aforementioned stained autofluorescence component image and the aforementioned autofluorescence reference spectrum, a pseudo-stained autofluorescence component spectrum is generated. Based on the pseudo-staining fluorescent reagent spectrum and the pseudo-staining autofluorescence component spectrum, a pseudo-staining specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the stained specimen and the fluorescence spectrum of the pseudo-stained specimen, a differential stained specimen fluorescence spectrum is generated. The fluorescence spectrum of the difference-stained specimen is separated into a difference-stained fluorescence component image containing the fluorescent reagent and a difference-stained autofluorescence component image containing the autofluorescence component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. The threshold determination unit determines the positive threshold based on the difference stained fluorescence component image. The information processing device described in item 18.

[0338] [Item 20] The threshold determination unit corrects the positive threshold based on a correction value predetermined according to the fluorescent reagent. Information processing device as described in item 18 or 19.

[0339] [Item 21] The threshold determination unit acquires the correction value from a correction data storage unit that stores the reagent identification information and the correction value in relation to each other, based on the reagent identification information associated with the fluorescent reagent. The information processing device described in item 20.

[0340] [Item 22] The threshold determination unit corrects the positive threshold based on a correction value predetermined according to the combination of the fluorescent reagent and the object to be labeled with the fluorescent reagent. An information processing device as described in any of items 18-21.

[0341] [Item 23] The threshold determination unit acquires the correction value from a correction data storage unit that stores the label target identification information associated with the sample, the reagent identification information associated with the fluorescent reagent, and the correction value in relation to each other, based on the label target identification information associated with the sample and the reagent identification information associated with the fluorescent reagent. The information processing device described in item 22.

[0342] [Item 24] The threshold determination unit determines the positive threshold for each of the multiple observation regions determined by dividing the stained fluorescent component image. An information processing device as described in any of items 18-23.

[0343] [Item 25] The threshold determination unit determines the positive threshold for each of the plurality of observation areas defined by the user. The information processing device described in item 24.

[0344] [Item 26] The information processing device according to item 24, wherein the threshold determination unit identifies noise components included in the fluorescence spectrum of the stained specimen and determines the plurality of observation regions by dividing the stained fluorescence component image according to the noise components.

[0345] [Item 27] The threshold determination unit determines the correctable range of the positive threshold, The threshold output unit outputs information indicating the positive threshold and the correctable range. An information processing device as described in any of items 18-26.

[0346] [Item 28] A light irradiation unit that irradiates excitation light to excite a fluorescent reagent, An imaging device that images a sample being irradiated with the excitation light and acquires the fluorescence spectrum of the sample, The system comprises an information processing device for analyzing the fluorescence spectrum of the sample, The aforementioned information processing device is A first separation unit separates the fluorescence spectrum of a stained specimen, obtained by irradiating an excitation light onto a fluorescently stained specimen obtained by labeling a specimen with a fluorescent reagent, into a stained fluorescent component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component, using a fluorescence reference spectrum and an autofluorescence reference spectrum. A threshold determination unit determines a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the stained fluorescent component image and the fluorescence reference spectrum, and compares the image data of multiple image sections included in the stained fluorescent component image with the positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the stained fluorescent component image and the fluorescence reference spectrum. The system includes a threshold output unit that outputs the positive threshold. Microscope system.

[0347] [Item 29] The display unit includes a display information generation unit that generates display information, which includes threshold information indicating the positive threshold, for display on the display unit. The microscope system described in item 28.

[0348] [Item 30] The threshold determination unit determines the correctable range of the positive threshold, The aforementioned information includes correctable range information indicating the correctable range. The microscope system described in item 29.

[0349] [Item 31] The system includes an analysis unit that performs analysis based on the positive threshold. A microscope system as described in any of items 28-30.

[0350] [Item 32] The process involves separating the fluorescence spectrum of a stained specimen obtained by irradiating an excitation light onto a fluorescently stained specimen obtained by labeling a specimen with a fluorescent reagent into a stained fluorescent component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component, using a fluorescence reference spectrum and an autofluorescence reference spectrum. A step of determining a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, which is compared with the image data of a plurality of image sections included in the stained fluorescent component image, based on the stained fluorescent component image and the fluorescence reference spectrum, image spectral data derived from the fluorescence reference spectrum, The process of outputting the positive threshold, Information processing methods including [Explanation of Symbols]

[0351] 10 Fluorescent reagent, 11 Reagent identification information, 20 Specimen, 21 Specimen identification information, 30 Fluorescent stained specimen, 40 Separation unit, 41 First separation unit, 42 Second separation unit, 43 Threshold determination unit, 44 Separation output unit, 45 Image spectrum output unit, 46 Threshold output unit, 47 Analysis unit, 100 Information processing unit, 101 Microscope, 102 Stage, 103 Optical system, 103A Objective lens, 103B Imaging lens, 103C Dichroic mirror, 103D Emission filter, 103E Excitation filter, 104 Light source, 105 Stage drive unit, 106 Light source drive unit, 107 Data processing unit, 110 Acquisition unit, 111 Information acquisition unit, 112 Fluorescence signal acquisition unit, 120 Storage unit, Lu Correctable upper limit, Ld Correctable lower limit, 121 Information storage unit, 122 Fluorescence signal storage unit, 130 Processing unit, 131 Linking unit, Q Positive threshold mark, 132 Separation processing unit, 133 Image generation unit, 140 Display unit, 150 Control unit, 160 Operation unit, 200 Database, D1 Stained specimen fluorescence spectrum, D2 Stained fluorescence component image, D3 Stained autofluorescence component image, D4 ​​Pseudo-stained fluorescence spectrum, D5 Pseudo-stained autofluorescence spectrum, D6 Stained specific channel brightness image, D7 Pseudo-stained specimen fluorescence spectrum, D8 Difference stained specimen fluorescence spectrum, D9 Difference stained norm image, D10 Difference stained fluorescence component image, D11 Difference stained autofluorescence component image, D21 Unstained specimen fluorescence spectrum, D22 Unstained fluorescence component image, D23 Unstained autofluorescence component image, D24 Pseudo-unstained fluorescence spectrum, D25 Pseudo-unstained autofluorescence spectrum, D26 Unstained specific channel brightness image, D27 Pseudo-unstained specimen fluorescence spectrum, D28 Differential unstained specimen fluorescence spectrum, D29 Differential unstained norm image, D30 Differential unstained fluorescence component image, D31 Differential unstained autofluorescence component image, J1 Specimen image information, J2 Presentation information, K1 Non-positive cell image, K2 Positive cell image, Ld Correctable lower limit, Lu Correctable upper limit, M1 Cell image position enhancement mark, M2 Observation area enhancement mark, Q Positive threshold mark, Q1 First positive threshold mark, Q2 Second positive threshold mark, R1 Fluorescence reference spectrum, R2 Autofluorescence reference spectrum, Rs1 First observation area, Rs2 Second observation area, Tm Mask threshold

Claims

1. A first separation unit separates the fluorescence spectrum of a stained specimen obtained by irradiating an excitation light onto a fluorescently stained specimen obtained by labeling a specimen with a fluorescent reagent into a stained fluorescent component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component, using a fluorescence reference spectrum and an autofluorescence reference spectrum. A second separation unit separates the fluorescence spectrum of an unstained specimen, obtained by irradiating an unstained specimen not labeled with the fluorescent reagent with the excitation light, into an unstained fluorescent component image containing the fluorescent reagent and an unstained autofluorescent component image containing the autofluorescent component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. A threshold determination unit determines a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the unstained fluorescent component image, and compares it with the image data of multiple image sections included in the stained fluorescent component image. A threshold output unit that outputs the positive threshold, An information processing device equipped with the following features.

2. The first separation unit is, Based on the stained fluorescence component image and the fluorescence reference spectrum, a pseudo-stained fluorescence spectrum is generated. Based on the stained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-stained autofluorescence spectrum is generated. Based on the pseudo-stained fluorescence spectrum and the pseudo-stained autofluorescence spectrum, a pseudo-stained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the stained specimen and the fluorescence spectrum of the pseudo-stained specimen, a differential stained specimen fluorescence spectrum is generated. The fluorescence spectrum of the difference-stained specimen is separated into a difference-stained fluorescence component image containing the fluorescent reagent and a difference-stained autofluorescence component image containing the autofluorescence component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. The second separation unit is, Based on the unstained fluorescent component image and the fluorescence reference spectrum, a pseudo-unstained fluorescence spectrum is generated. Based on the unstained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-unstained autofluorescence spectrum is generated. Based on the pseudo-unstained fluorescence spectrum and the pseudo-unstained autofluorescence spectrum, a pseudo-unstained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the unstained specimen and the fluorescence spectrum of the pseudo-unstained specimen, a differential fluorescence spectrum of the unstained specimen is generated. The fluorescence spectrum of the difference unstained sample is separated into a difference unstained fluorescent component image containing the fluorescent reagent and a difference unstained autofluorescent component image containing the autofluorescent component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. The threshold determination unit corrects the positive threshold based on the spectrum of the difference stained fluorescent component image and the spectrum of the difference unstained fluorescent component image. The information processing apparatus according to claim 1.

3. The first separation unit is, Based on the stained fluorescence component image and the fluorescence reference spectrum, a pseudo-stained fluorescence spectrum is generated. Based on the stained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-stained autofluorescence spectrum is generated. Based on the pseudo-stained fluorescence spectrum and the pseudo-stained autofluorescence spectrum, a pseudo-stained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the stained specimen and the fluorescence spectrum of the pseudo-stained specimen, a differential stained specimen fluorescence spectrum is generated. The second separation unit is, Based on the unstained fluorescent component image and the fluorescence reference spectrum, a pseudo-unstained fluorescence spectrum is generated. Based on the unstained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-unstained autofluorescence spectrum is generated. Based on the pseudo-unstained fluorescence spectrum and the pseudo-unstained autofluorescence spectrum, a pseudo-unstained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the unstained specimen and the fluorescence spectrum of the pseudo-unstained specimen, a differential fluorescence spectrum of the unstained specimen is generated. The threshold determination unit corrects the positive threshold based on the difference stained sample fluorescence spectrum and the difference unstained sample fluorescence spectrum. The information processing apparatus according to claim 1.

4. The second separation unit is, Based on the unstained fluorescent component image and the fluorescence reference spectrum, a pseudo-unstained fluorescence spectrum is generated. Based on the unstained autofluorescence component image and the autofluorescence reference spectrum, a pseudo-unstained autofluorescence spectrum is generated. Based on the pseudo-unstained fluorescence spectrum and the pseudo-unstained autofluorescence spectrum, a pseudo-unstained specimen fluorescence spectrum is generated. Based on the difference between the fluorescence spectrum of the unstained specimen and the fluorescence spectrum of the pseudo-unstained specimen, a differential fluorescence spectrum of the unstained specimen is generated. The difference unstained norm data, which is the norm data of the fluorescence spectrum of the difference unstained sample, is generated. The threshold determination unit, The aforementioned difference unstained normed data is analyzed to obtain outlier data. Based on the outlier data, the unstained fluorescence component image is corrected. The positive threshold is determined based on the corrected unstained fluorescent component image. The information processing apparatus according to claim 1.

5. The threshold determination unit corrects the positive threshold based on a correction value predetermined according to the fluorescent reagent. The information processing apparatus according to claim 1.

6. The threshold determination unit acquires the correction value from a correction data storage unit that stores the reagent identification information and the correction value in relation to each other, based on the reagent identification information associated with the fluorescent reagent. The information processing apparatus according to claim 5.

7. The threshold determination unit corrects the positive threshold based on a correction value predetermined according to the combination of the fluorescent reagent and the object to be labeled with the fluorescent reagent. The information processing apparatus according to claim 1.

8. The threshold determination unit acquires the correction value from a correction data storage unit that stores the label target identification information associated with the sample, the reagent identification information associated with the fluorescent reagent, and the correction value in relation to each other, based on the label target identification information associated with the sample and the reagent identification information associated with the fluorescent reagent. The information processing apparatus according to claim 7.

9. The threshold determination unit determines the positive threshold for each of the multiple observation regions determined by dividing the stained fluorescent component image. The information processing apparatus according to claim 1.

10. The threshold determination unit determines the positive threshold for each of the plurality of observation areas defined by the user. The information processing apparatus according to claim 9.

11. The information processing apparatus according to claim 9, wherein the threshold determination unit identifies noise components included in the fluorescence spectrum of the stained specimen and determines the plurality of observation regions by dividing the stained fluorescence component image according to the noise components.

12. The threshold determination unit determines the correctable range of the positive threshold, The threshold output unit outputs information indicating the positive threshold and the correctable range. The information processing apparatus according to claim 1.

13. A light irradiation unit that irradiates excitation light to excite a fluorescent reagent, An imaging device that images a sample being irradiated with the excitation light and acquires the fluorescence spectrum of the sample, The system comprises an information processing device for analyzing the fluorescence spectrum of the sample, The aforementioned information processing device is A first separation unit separates the fluorescence spectrum of a stained specimen obtained by irradiating the fluorescently stained specimen, which is obtained by labeling the specimen with the fluorescent reagent, into a stained fluorescent component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component, using a fluorescence reference spectrum and an autofluorescence reference spectrum. A second separation unit separates the fluorescence spectrum of an unstained specimen, obtained by irradiating an unstained specimen not labeled with the fluorescent reagent with the excitation light, into an unstained fluorescent component image containing the fluorescent reagent and an unstained autofluorescent component image containing the autofluorescent component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. A threshold determination unit that determines a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the unstained fluorescent component image and compared with the image data of multiple image sections included in the stained fluorescent component image. Microscope system.

14. The display unit includes a display information generation unit that generates display information, which includes threshold information indicating the positive threshold, for display on the display unit. The microscope system according to claim 13.

15. The threshold determination unit determines the correctable range of the positive threshold, The aforementioned information includes correctable range information indicating the correctable range. The microscope system according to claim 14.

16. The system includes an analysis unit that performs analysis based on the positive threshold. The microscope system according to claim 13.

17. The process involves separating the fluorescence spectrum of a stained specimen obtained by irradiating an excitation light onto a fluorescently stained specimen obtained by labeling a specimen with a fluorescent reagent into a stained fluorescent component image containing the fluorescent reagent and a stained autofluorescence component image containing the autofluorescence component, using a fluorescence reference spectrum and an autofluorescence reference spectrum. A step of separating the fluorescence spectrum of an unstained specimen obtained by irradiating an unstained specimen not labeled with the fluorescent reagent with the excitation light into an unstained fluorescent component image containing the fluorescent reagent and an unstained autofluorescent component image containing the autofluorescent component, using the fluorescence reference spectrum and the autofluorescence reference spectrum. A step of determining a positive threshold, which is a criterion for determining whether each of the multiple image sections corresponds to a positive cell image, based on the unstained fluorescent component image, and which is compared with the image data of multiple image sections included in the stained fluorescent component image. The process of outputting the positive threshold, Information processing methods including

Images