Fluorescence observation device and fluorescence observation method

Abstract

To provide a fluorescence observation device and a fluorescence observation method that can prevent an increase in photographing time associated with an increase in the number of pigments of an observation object.SOLUTION: A fluorescence observation device according to one form of the present technology is provided with a stage, an excitation unit, and a spectroscopic imaging unit. The stage can support a fluorescent-dyed pathological specimen. The excitation unit irradiates the pathological specimen on the stage with a plurality of rays of line illumination having different wavelengths from each other, having different axes, and parallel in a uniaxial direction. The spectroscopic imaging unit has one or more image pick-up devices that can individually receive fluorescence excited by the plurality of rays of line illumination.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present technology relates to a fluorescence observation device and a fluorescence observation method used for diagnosing pathological images, for example. [Background technology]

[0002] A pathological imaging diagnostic method using fluorescent staining has been proposed as a technique with excellent quantitative and polychromatic properties (see, for example, Patent Document 1). Fluorescent techniques are advantageous in that they facilitate multiplexing compared to color staining and provide detailed diagnostic information. Even in fluorescence imaging for purposes other than pathological diagnosis, an increase in the number of colors makes it possible to simultaneously examine various antigens expressed in a sample.

[0003] In general fluorescence imaging, excitation light of the dye's absorption wavelength (excitation wavelength) is irradiated, and the resulting dye spectrum is selectively captured using a bandpass filter. When there are multiple colors, the absorption wavelength (excitation wavelength) varies depending on the dye, so imaging is done by switching filters for each dye. However, because the absorption and emission spectra of dyes are broad and overlap, when stained with multiple colors, a single excitation wavelength will excite multiple dyes. Furthermore, fluorescence from neighboring dyes will leak into the bandpass filter, causing color mixing.

[0004] On the other hand, a method of capturing images by switching the wavelength of excitation light and the wavelength of detected fluorescence in a time-division manner is known (for example, Non-Patent Document 1). However, this method has the problem that the capturing time increases linearly as the number of colors increases. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 4452850 [Non-patent literature]

[0006] [Non-Patent Document 1] Edward C. Stack, "Multiplexed immunohistochemistry, imaging, and quantitation: A review, with an assessment of Tyramide signal amplification, multispectral imaging and multiplex analysis", Methods 70 (2014) 46-58 Summary of the Invention [Problem to be solved by the invention]

[0007] In view of the above circumstances, an object of the present technology is to provide a fluorescence observation device and a fluorescence observation method that can suppress an increase in imaging time that accompanies an increase in the number of dyes to be observed. [Means for solving the problem]

[0008] A fluorescence observation device according to an embodiment of the present technology includes a stage, an excitation unit, and a spectral imaging unit. The stage is configured to be able to support a fluorescently stained pathological specimen. The excitation unit irradiates the pathological specimen on the stage with a plurality of line illumination beams that are configured with different wavelengths and are parallel to one axis and have different axes. The spectral imaging unit has at least one image sensor capable of receiving individual fluorescent light excited by the plurality of line illuminators.

[0009] This makes it possible to suppress an increase in imaging time that would otherwise occur with an increase in the number of dyes to be observed.

[0010] The excitation unit may be configured to irradiate the pathological specimen with a plurality of line illuminators each configured with a combination of different wavelengths as the plurality of line illuminators.

[0011] The spectral imaging unit may further include a wavelength dispersive element that disperses the fluorescence excited by the plurality of line illuminators.

[0012] The spectral imaging unit may further include an observation slit having a plurality of slit portions through which the fluorescence excited by the plurality of line illuminators can pass.

[0013] The fluorescence observation device may further include a scanning mechanism that scans the stage with the plurality of line illuminations in a direction perpendicular to the one axial direction.

[0014] The fluorescence observation device may further include a processing unit having a storage section that stores spectroscopic data that indicates the correlation between the wavelengths of the plurality of line illuminations and the fluorescence received by the image sensor.

[0015] The processing unit may further include an image forming section that forms a fluorescent image of the pathological specimen based on the spectral data stored in the storage section and the intervals between the plurality of line illuminators.

[0016] The image forming section may be configured to form, as the fluorescent light image, an image in which the detection coordinates of the image capturing element are corrected by a value corresponding to the intervals between the plurality of line illuminators.

[0017] The processing unit may further include a data calibration unit that calibrates the spectroscopic data stored in the storage unit.

[0018] The storage unit may be configured to store in advance a standard spectrum of autofluorescence related to the pathological specimen and a standard spectrum of a single dye that stains the pathological specimen, and the image forming unit may be configured to output a component distribution of the spectroscopic data based on the standard spectra of the autofluorescence and the single dye.

[0019] The imaging element may include a plurality of imaging elements each capable of receiving the fluorescence that has passed through the observation slit.

[0020] The fluorescence observation device may further include a non-fluorescence observation unit having a light source that illuminates the pathological specimen on the stage and an imaging unit that acquires a non-fluorescence image of the pathological specimen.

[0021] The fluorescence observation device may further include a display unit that displays, for each of a plurality of line illuminators, a fluorescence spectrum excited by the plurality of line illuminators.

[0022] The display unit may have an operation area where wavelengths and outputs of the plurality of line illuminators can be set.

[0023] The display unit may have a display area that displays a detection wavelength range of the fluorescence spectrum.

[0024] A fluorescence observation method according to one aspect of the present technology includes: Multiple line lights with different wavelengths, each parallel to one axis and irradiated onto the pathological specimen on the stage, Fluorescent light excited by the plurality of line illuminators is received individually.

[0025] The fluorescence observation method may further comprise scanning the stage with the plurality of line illuminations in a direction perpendicular to the one axial direction.

[0026] The plurality of line illuminators may be configured with different combinations of wavelengths. [Effects of the Invention]

[0027] As described above, according to the present technology, it is possible to suppress an increase in imaging time that accompanies an increase in the number of dyes to be observed. The effects described here are not necessarily limited to those described herein, and may be any of the effects described in this disclosure. [Brief explanation of the drawings]

[0028] [Figure 1]1 is a schematic block diagram of a fluorescence observation device according to an embodiment of the present technology. [Figure 2] FIG. 2 is a diagram showing an example of an optical system in the fluorescence observation device. [Figure 3] FIG. 1 is a schematic diagram of a pathological specimen to be observed. [Figure 4] 3 is a schematic diagram showing a state of line illumination irradiating the observation object. FIG. [Figure 5] 10A and 10B are diagrams illustrating a method for acquiring spectral data when the imaging element in the fluorescence observation device is configured with a single image sensor. [Figure 6] FIG. 6 is a diagram showing wavelength characteristics of the spectral data acquired in FIG. 5. [Figure 7] 10A and 10B are diagrams illustrating a method for acquiring spectral data when the imaging element is configured with a plurality of image sensors. [Figure 8] 10A and 10B are conceptual diagrams illustrating a scanning method of the line illumination irradiating the observation object. [Figure 9] FIG. 1 is a conceptual diagram illustrating three-dimensional data (X, Y, λ) acquired with multiple line illumination devices. [Figure 10] 3A and 3B are diagrams illustrating an example of the wavelength configuration of an excitation unit in the fluorescence observation device. [Figure 11] FIG. 10 is a schematic diagram showing another example of the configuration of the spectral imaging unit in the fluorescence observation device. [Figure 12] 5 is a flowchart showing an example of a processing procedure executed in a processing unit in the fluorescence observation device. [Figure 13] FIG. 2 is a diagram illustrating a screen of a display unit in the fluorescence observation device. [Figure 14] FIG. 10 is a diagram showing an example of a screen configuration of a setting area of ​​the excitation unit on the display unit. [Figure 15] FIG. 10 is a diagram showing an example of a screen configuration of a detection setting area for a fluorescence spectrum from one line illumination on the display unit. [Figure 16] 10 is a diagram showing an example of a screen configuration of a detection setting area for a fluorescence spectrum from another line illumination on the display unit. FIG. [Figure 17]3 is a schematic diagram conceptually showing the relationship between fluorescence spectrum data and a fluorescence image displayed on a display unit. FIG. [Figure 18] 10 is a flowchart showing a modified example of the procedure of the process executed in the processing unit. [Figure 19] 10 is a flowchart showing another modified example of the procedure of the process executed in the processing unit. [Figure 20] FIG. 10 is a schematic block diagram showing a modified example of the fluorescence observation device. [Figure 21] FIG. 10 is a schematic block diagram showing another modified example of the fluorescence observation device. DETAILED DESCRIPTION OF THE INVENTION

[0029] Hereinafter, embodiments of the present technology will be described with reference to the drawings.

[0030] FIG. 1 is a schematic block diagram of a fluorescence observation apparatus according to an embodiment of the present technology, and FIG. 2 is a diagram showing an example of an optical system in the fluorescence observation apparatus.

[0031] [Overall configuration] The fluorescence observation device 100 of this embodiment includes an observation unit 1. The observation unit 1 includes an excitation unit 10 that irradiates a pathological specimen (pathological sample) with multiple line lights of different wavelengths that are arranged parallel to different axes, a stage 20 that supports the pathological specimen, and a spectral imaging unit 30 that acquires the fluorescence spectrum (spectral data) of the linearly excited pathological specimen.

[0032] Here, "different axes but parallel" means that the multiple line light sources are on different axes but parallel. "Different axes" means that they are not on the same axis, and the distance between the axes is not particularly limited. "Parallel" does not necessarily mean parallel in the strict sense, but also includes a state where the light sources are nearly parallel. For example, deviations from the parallel state due to distortions from the optical system such as lenses or manufacturing tolerances are acceptable, and these cases are also considered to be parallel.

[0033] The fluorescence observation device 100 further includes a processing unit 2. The processing unit 2 typically forms an image of a pathological specimen (hereinafter also referred to as sample S) or outputs a distribution of the fluorescence spectrum based on the fluorescence spectrum of the pathological specimen acquired by the observation unit 1. The image here refers to the component ratios of the pigments that make up the spectrum, autofluorescence derived from the sample, etc., the waveform converted into RGB (red-green-blue) colors, the luminance distribution of a specific wavelength band, etc.

[0034] The excitation unit 10 and the spectroscopic imaging unit 30 are connected to the stage 20 via an observation optical system 40, such as an objective lens 44. The observation optical system 40 has the function of tracking the optimum focus using a focus mechanism 60. A non-fluorescence observation unit 70, such as for dark-field observation or bright-field observation, may be connected to the observation optical system 40.

[0035] The fluorescence observation device 100 may be connected to a control unit 80 that controls the excitation unit (control of the LD and shutter), the XY stage which is the scanning mechanism, the spectroscopic imaging unit (camera), the focus mechanism (detector and Z stage), the non-fluorescence observation unit (camera), etc.

[0036] The excitation unit 10 includes a plurality of light sources L1, L2, ... that can output light of a plurality of excitation wavelengths Ex1, Ex2, .... The plurality of light sources are typically constituted by light-emitting diodes (LEDs), laser diodes (LDs), mercury lamps, etc., and each light is converted into a line illumination that is irradiated onto the sample S on the stage 20.

[0037] The sample S is typically a slide containing an observation target Sa such as a tissue section as shown in FIG. 3, but of course it can be something other than that. The sample S (observation target Sa) is stained with multiple fluorescent dyes. The observation unit 1 observes the sample S at a desired magnification. Enlarging part A in FIG. 3, the illumination unit has multiple line illuminators (two (Ex1, Ex2) in the illustrated example) arranged as shown in FIG. 4, and the imaging areas R1, R2 of the spectral imaging unit 30 are arranged so as to overlap with the respective illumination areas. The two line illuminators Ex1, Ex2 are each parallel to the Z-axis direction and spaced a predetermined distance (Δy) apart in the Y-axis direction.

[0038] The photographing areas R1 and R2 correspond to the slit sections of the observation slit 31 (FIG. 2) in the spectral imaging unit 30. That is, the same number of slit sections as the line illuminations are arranged in the spectral imaging unit 30. In FIG. 4, the illumination line width is wider than the slit width, but the relationship between these sizes is not critical. If the illumination line width is wider than the slit width, the alignment margin of the excitation unit 10 relative to the spectral imaging unit 30 can be increased.

[0039] The wavelengths constituting the first line illumination Ex1 and the second line illumination Ex2 are different from each other. The line-shaped fluorescence excited by these line illuminations Ex1 and Ex2 is observed by the spectral imaging unit 30 via the observation optical system 40.

[0040] The spectral imaging unit 30 has an observation slit 31 with multiple slit sections through which fluorescence excited by multiple line illuminations can pass, and at least one image sensor 32 that can individually receive the fluorescence that has passed through the observation slit 31. A two-dimensional imager such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) is used for the image sensor 32. By locating the observation slit 31 on the optical path, it is possible to detect the fluorescence spectra excited by each line without overlapping.

[0041] The spectral imaging unit 30 acquires fluorescence spectral data (x, λ) from each of the line illuminators Ex1 and Ex2, using the pixel array in one direction (for example, the vertical direction) of the image sensor 32 as a wavelength channel. The acquired spectral data (x, λ) is recorded in the processing unit 2 in a state where it is linked to the excitation wavelength from which each piece of spectral data was excited.

[0042] The processing unit 2 can be realized by hardware elements used in a computer, such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), as well as necessary software. Instead of or in addition to the CPU, a programmable logic device (PLD) such as a field programmable gate array (FPGA), a digital signal processor (DSP), or an application specific integrated circuit (ASIC) may be used.

[0043] The processing unit 2 has a memory 21 that stores spectroscopic data that indicates the correlation between the wavelengths of the multiple line illuminators Ex1 and Ex2 and the fluorescence received by the image sensor 32. The memory 21 uses a storage device such as a nonvolatile semiconductor memory or a hard disk drive, and stores in advance a standard spectrum of autofluorescence for the sample S and a standard spectrum of the single dye that stains the sample S. The spectroscopic data (x, λ) received by the image sensor 32 is acquired, for example, as shown in FIGS. 5 and 6 and stored in the memory 21. In this embodiment, the memory that stores the autofluorescence and standard spectra of the single dye of the sample S and the memory that stores the spectroscopic data (measured spectrum) of the sample S acquired by the image sensor 32 are configured in the common memory 21, but this is not limiting and they may be configured in separate memory units.

[0044] 5 and 6 are diagrams illustrating a method for acquiring spectral data when the image sensor 32 is configured with a single image sensor that commonly receives fluorescence that has passed through the observation slit 31. In this example, the fluorescence spectra Fs1 and Fs2 excited by the line illuminations Ex1 and Ex2 are ultimately imaged on the light-receiving surface of the image sensor 32 via a spectroscopic optical system (described later) with a shift in magnitude proportional to Δy (see FIG. 4).

[0045] As shown in Figure 5, the information obtained from the line illuminator Ex1 is recorded as Row_a and Row_b, and the information obtained from the line illuminator Ex2 is recorded as Row_c and Row_d. Data outside these areas is not read out. This makes it possible to make the frame rate of the image sensor 32 Row_full / (Row_b-Row_a+Row_d-Row_c) times faster than when reading out full frames.

[0046] As shown in Figure 2, a dichroic mirror 42 and a bandpass filter 45 are inserted in the optical path to prevent the excitation light (Ex1, Ex2) from reaching the image sensor 32. In this case, an intermittent portion IF occurs in the fluorescence spectrum Fs1 imaged on the image sensor 32 (see Figures 5 and 6). By excluding such an intermittent portion IF from the readout area, the frame rate can be further improved.

[0047] 2, the image sensor 32 may include a plurality of image sensors 32a and 32b each capable of receiving fluorescence that has passed through the observation slit 31. In this case, the fluorescence spectra Fs1 and Fs2 excited by the respective line illuminators Ex1 and Ex2 are acquired on the image sensors 32a and 32b as shown in FIG.

[0048] The line illuminators Ex1 and Ex2 are not limited to being composed of a single wavelength, and may each be composed of multiple wavelengths. When the line illuminators Ex1 and Ex2 are each composed of multiple wavelengths, the fluorescence excited by these illuminators also contains multiple spectra. In this case, the spectral imaging unit 30 has a wavelength dispersive element for separating the fluorescence into spectra derived from the excitation wavelengths. The wavelength dispersive element is composed of a diffraction grating, a prism, or the like, and is typically placed on the optical path between the observation slit 31 and the image sensor 32.

[0049] The observation unit 1 further includes a scanning mechanism 50 that scans the stage 20 with multiple line illuminators Ex1 and Ex2 in the Y-axis direction, i.e., in the arrangement direction of each line illuminator Ex1 and Ex2. Using the scanning mechanism 50, dye spectra (fluorescence spectra) excited with different excitation wavelengths, each spatially separated by Δy, on the sample S (observation target Sa) can be continuously recorded in the Y-axis direction. In this case, for example, as shown in Figure 8, the imaging region Rs is divided into multiple regions in the X-axis direction, and the sample S is scanned in the Y-axis direction, then moved in the X-axis direction, and further scanned in the Y-axis direction. This operation is repeated. A single scan can capture spectral images of a sample excited with several excitation wavelengths.

[0050] In the scanning mechanism 50, the stage 20 is typically scanned in the Y-axis direction, but multiple line illuminators Ex1 and Ex2 may also be scanned in the Y-axis direction by a galvanometer mirror disposed midway through the optical system. Ultimately, three-dimensional data of (X, Y, λ) as shown in Fig. 9 is acquired for each of the multiple line illuminators Ex1 and Ex2. Since the three-dimensional data derived from each of the line illuminators Ex1 and Ex2 has a coordinate shift of Δy about the Y-axis, the data is corrected and output based on a pre-recorded Δy or a value of Δy calculated from the output of the image sensor 32.

[0051] In the examples described so far, two line illuminators have been used as excitation light sources, but this is not a limitation and three, four, five, or more lines may be used. Each line illuminator may also include multiple excitation wavelengths selected to minimize degradation of color separation performance. Even with a single line illuminator, if the excitation light source is composed of multiple excitation wavelengths and each excitation wavelength is linked and recorded with the row data acquired by the image sensor, a multicolor spectrum can be obtained, although the separation ability is not as high as with parallel and different axes. For example, a configuration such as that shown in FIG. 10 may be used.

[0052] [Observation unit] Next, the observation unit 1 will be described in detail with reference to Fig. 2. Here, an example in which the observation unit 1 is configured in configuration example 2 in Fig. 10 will be described.

[0053] The pumping section 10 has a plurality of (four in this example) pumping light sources L1, L2, L3, and L4. Each of the pumping light sources L1 to L4 is composed of a laser light source that outputs laser light having wavelengths of 405 nm, 488 nm, 561 nm, and 645 nm, respectively.

[0054] The excitation unit 10 further includes a plurality of collimator lenses 11 and laser line filters 12 corresponding to the excitation light sources L1 to L4, dichroic mirrors 13a, 13b, and 13c, a homogenizer 14, a condenser lens 15, and an entrance slit 16.

[0055] The laser light emitted from excitation light source L1 and the laser light emitted from excitation light source L3 are each converted into parallel light by collimator lens 11, then transmitted through laser line filter 12, which cuts off the base of each wavelength band, and made coaxial by dichroic mirror 13a. The two coaxial laser lights are further beam-shaped by homogenizer 14, such as a fly-eye lens, and condenser lens 15 to become line illumination Ex1.

[0056] The laser light emitted from excitation light source L2 and the laser light emitted from excitation light source L4 are similarly made coaxial by dichroic mirrors 13b and 13c to become line illumination Ex2 having a different axis from line illumination Ex1. Line illuminations Ex1 and Ex2 form different-axis line illuminations (primary images) separated by Δy at entrance slit 16 (slit conjugate) having multiple slit portions through which each light can pass.

[0057] This primary image is irradiated onto the sample S on the stage 20 via the observation optical system 40. The observation optical system 40 has a condenser lens 41, dichroic mirrors 42 and 43, an objective lens 44, a bandpass filter 45, and a condenser lens 46. The line illuminations Ex1 and Ex2 are converted into parallel light by the condenser lens 41 paired with the objective lens 44, reflected by the dichroic mirrors 42 and 43, transmitted through the objective lens 44, and irradiated onto the sample S.

[0058] Illumination is formed on the surface of the sample S as shown in Fig. 4. Fluorescence excited by these illuminations is collected by the objective lens 44, reflected by the dichroic mirror 43, transmitted through the dichroic mirror 42 and a bandpass filter 45 that cuts the excitation light, collected again by the condenser lens 46, and enters the spectral imaging unit 30.

[0059] The spectral imaging unit 30 includes an observation slit 31, an image sensor 32 (32a, 32b), a first prism 33, a mirror , a diffraction grating 35 (wavelength dispersion element), and a second prism .

[0060] The observation slit 31 is positioned at the focal point of the condenser lens 46 and has the same number of slit sections as the number of excitation lines. The fluorescence spectra derived from the two excitation lines that have passed through the observation slit 31 are separated by the first prism 33 and are further separated into fluorescence spectra of each excitation wavelength by being reflected by the grating surface of the diffraction grating 35 via the mirror 34. The four fluorescence spectra separated in this way are incident on the image sensors 32a and 32b via the mirror 34 and the second prism 36 and are expanded into (x, λ) information as spectroscopic data.

[0061] The pixel size (nm / Pixel) of the image sensors 32a and 32b is not particularly limited and is set to, for example, 2 nm or more and 20 nm or less. This dispersion value may be realized optically by the pitch of the diffraction grating 35, or may be realized by using hardware binning of the image sensors 32a and 32b.

[0062] The stage 20 and the scanning mechanism 50 constitute an XY stage, and the sample S is moved in the X-axis direction and the Y-axis direction to acquire a fluorescent image of the sample S. In WSI (Whole Slide Imaging), the sample S is scanned in the Y-axis direction, then moved in the X-axis direction, and then scanned again in the Y-axis direction, and this operation is repeated (see FIG. 8).

[0063] The non-fluorescence observation section 70 is composed of a light source 71, a dichroic mirror 43, an objective lens 44, a condenser lens 72, an image sensor 73, etc. In the non-fluorescence observation system, an observation system using dark-field illumination is shown in FIG.

[0064] The light source 71 is disposed below the stage 20 and irradiates the sample S on the stage 20 with illumination light from the opposite side to the line illuminations Ex1 and Ex2. In the case of dark-field illumination, the light source 71 illuminates from outside the NA (numerical aperture) of the objective lens 44, and the light diffracted by the sample S (dark-field image) is captured by the image sensor 73 via the objective lens 44, dichroic mirror 43, and condenser lens 72. By using dark-field illumination, even seemingly transparent samples such as fluorescently stained samples can be observed with contrast.

[0065] This dark-field image may be observed simultaneously with the fluorescence and used for real-time focusing. In this case, the illumination wavelength may be selected so as not to affect the fluorescence observation. The non-fluorescence observation unit 70 is not limited to an observation system that acquires dark-field images, but may also be configured with an observation system capable of acquiring non-fluorescence images such as bright-field images, phase-contrast images, phase images, and in-line hologram images. For example, various observation methods, such as the Schlieren method, phase-contrast method, polarized light observation method, and epi-illumination method, can be used to acquire non-fluorescence images. The position of the illumination light source is not limited to below the stage, but may be above the stage or around the objective lens. In addition to a method that performs real-time focus control, other methods, such as a pre-focus map method in which the focus coordinate (Z coordinate) is recorded in advance, may also be used.

[0066] Figure 11 is a schematic diagram showing another example of the configuration of the spectral imaging unit. The spectral imaging unit 130 shown in the figure has a single image sensor 32. Each fluorescence that passes through an observation slit 31, which has slits matching the number of excitation lines, is re-imaged on the image sensor 32 via a relay optical system (first prism 33, mirrors 34 and 37) and a wavelength dispersive element (such as a prism) 38 placed along the way, and is expanded into (x, λ) data (see Figure 5). At this time, the value obtained by converting the excitation light interval Δy into pixels is determined so that the dispersed spectra do not overlap on the image sensor 32.

[0067] [Processing unit] The fluorescence spectrum acquired by the image sensor 32 (32a, 32b) is output to the processing unit 2. The processing unit 2 further includes a memory unit 21, a data calibration unit 22 that calibrates the spectroscopic data stored in the memory unit 21, and an image formation unit 23 that forms a fluorescence image of the sample S based on the spectroscopic data and the interval Δy between the multiple line illuminators Ex1 and Ex2.

[0068] FIG. 12 is a flowchart showing an example of a procedure of processing executed in the processing unit 2.

[0069] The memory unit 21 stores the spectroscopic data (fluorescence spectra Fs1, Fs2 (see FIGS. 5 and 7)) acquired by the spectral imaging unit 30 (step 101). The memory unit 21 stores in advance the standard spectra of autofluorescence and single dyes related to the sample S.

[0070] The storage unit 21 improves the recording frame rate by extracting only a wavelength region of interest from the pixel array in the wavelength direction of the image sensor 32. The wavelength region of interest corresponds to, for example, the visible light range (380 nm to 780 nm) or a wavelength range determined by the emission wavelength of the dye that stains the sample.

[0071] Examples of wavelength regions other than the wavelength region of interest include sensor regions containing light of unnecessary wavelengths, sensor regions with no apparent signal, and regions of excitation wavelengths that should be cut by the dichroic mirror 42 or bandpass filter 45 located along the optical path. Furthermore, the wavelength region of interest on the sensor may be switched depending on the status of the line illumination. For example, when there are few excitation wavelengths used for line illumination, the wavelength region on the sensor is also limited, and the frame rate can be increased accordingly.

[0072] The data calibration unit 22 converts the spectral data stored in the memory unit 21 from pixel data (x, λ) to wavelength, and calibrates the data so that all spectral data is interpolated and output in wavelength units ([nm], [μm], etc.) having common discrete values ​​(step 102).

[0073] The pixel data (x, λ) may not be neatly aligned with the pixel rows of the image sensor 32, but may be distorted by slight tilt or optical system distortion. Therefore, if a light source with a known wavelength is used to convert from pixels to wavelength units, different wavelengths (nm values) will be converted for each x coordinate. Because handling the data in this state is cumbersome, the data is converted to integer-aligned data using an interpolation method (e.g., linear interpolation or spline interpolation) (step 102).

[0074] Furthermore, sensitivity unevenness may occur in the long axis direction (X-axis direction) of the line illumination. Sensitivity unevenness occurs due to uneven illumination or variations in slit width, leading to uneven brightness in the captured image. Therefore, to eliminate this unevenness, the data calibration unit 22 uses an arbitrary light source and its representative spectrum (average spectrum or light source spectral radiance) to homogenize and output the data (step 103). This homogenization eliminates differences between instruments, and reduces the effort required to measure individual component spectra each time during spectral waveform analysis. Furthermore, it is possible to output an approximate quantitative value of the number of fluorescent dyes from the sensitivity-calibrated brightness value.

[0075] The calibrated spectrum is given by spectral radiance [W / (sr m 2 By using a standard spectrum (nm), the sensitivity of the image sensor 32 corresponding to each wavelength is also corrected. In this way, by calibrating to a standard spectrum, it is no longer necessary to measure the standard spectrum used in color separation calculations for each device. If the dye is stable from the same lot, it can be reused after one image capture. Furthermore, if the fluorescence spectral intensity per dye molecule is given in advance, it is possible to output an approximate value for the number of fluorescent dye molecules converted from the sensitivity-calibrated brightness value. This value also separates out autofluorescence components, making it highly quantitative.

[0076] The above processing is similarly performed for the illumination range of the sample S scanned in the Y-axis direction by the line illuminations Ex1 and Ex2. This results in obtaining spectroscopic data (x, y, λ) of each fluorescence spectrum for the entire range of the sample S. The obtained spectroscopic data (x, y, λ) is stored in the memory unit 21.

[0077] The image forming unit 23 forms a fluorescence image of the sample S based on the spectral data stored in the storage unit 21 (or the spectral data calibrated by the data calibrating unit 22) and the interval corresponding to the axial distance (Δy) between the excitation lines Ex1 and Ex2 (step 104). In this embodiment, the image forming unit 23 forms, as the fluorescence image, an image in which the detection coordinates of the image sensor 32 have been corrected by a value corresponding to the interval (Δy) between the multiple line illuminators Ex1 and Ex2.

[0078] The three-dimensional data derived from each of the line illuminators Ex1 and Ex2 has its coordinates shifted by Δy along the Y axis, and is therefore corrected and output based on a pre-recorded Δy or a Δy value calculated from the output of the image sensor 32. Here, the difference in detected coordinates at the image sensor 32 is corrected so that the three-dimensional data derived from each of the line illuminators Ex1 and Ex2 becomes data on the same coordinates.

[0079] The image forming unit 23 performs a process (stitching) to stitch the captured images together into one large image (WSI) (step 105). This allows a pathological image of the multiplexed sample S (observation object Sa) to be acquired. The formed fluorescent image is output to the display unit 3 (step 106).

[0080] Furthermore, the image forming unit 23 separates and calculates the component distributions of the autofluorescence and dye of the sample S from the captured spectroscopic data (measured spectrum) based on the standard spectra of the autofluorescence and dye alone of the sample S stored in advance in the memory unit 21. The calculation method may be the least squares method, weighted least squares method, or the like, and calculates coefficients such that the captured spectroscopic data becomes a linear sum of the above-mentioned standard spectra. The calculated coefficient distribution is stored in the memory unit 21 and is also output to the display unit 3 and displayed as an image (steps 107 and 108).

[0081] [summary] As described above, according to this embodiment, it is possible to provide a multiplex fluorescent scanner that does not increase the imaging time even if the number of dyes to be observed increases.

[0082] In other words, the data captured by the multiplexed fluorescence scanner is three-dimensional data (x, y, λ). Therefore, in the case of surface imaging, it is not possible to acquire all the data (x, y, λ) at once, and therefore imaging is required by switching λ in a time series. Furthermore, when irradiating with a surface light source (x, y), physical constraints necessitate a method for switching the excitation wavelength in a time series. To solve these issues, in this embodiment, fluorescence is excited by multiple line illuminators with different wavelengths arranged parallel to different axes. The line-shaped excited fluorescence spectra are each dispersed into (x, λ) and imaged simultaneously by a two-dimensional sensor (image sensor). This configuration allows the excitation wavelengths to be separated spatially, rather than temporally, so imaging time does not increase even if the number of dyes to be observed increases.

[0083] This method requires one-dimensional scanning (Y direction) to acquire two-dimensional data, which at first glance appears to result in slower imaging times than planar excitation. However, due to the characteristics of fluorescence, fluorescence intensity increases linearly with excitation power density until the dye's energy absorption saturates. Therefore, assuming the same light source output power as planar excitation can be used, line irradiation can achieve a higher power density due to the smaller area, allowing for brighter fluorescence excitation. As a result, exposure time is shortened, making it possible to image (x, y, λ) under theoretically the fastest conditions. (Because excited fluorescence is not discarded, energy efficiency is high and data acquisition is rapid.) Furthermore, this method allows for constant-speed scanning, making it superior to planar excitation methods, which require repeated stop-and-go cycles, for large-area imaging.

[0084] Furthermore, this embodiment enables high-speed imaging of fluorescent dyes with better energy efficiency than time-resolved area excitation spectroscopic imaging devices. Furthermore, since it is no longer necessary to measure the reference spectrum used for color separation calculations for each device, stable dyes from the same lot can be captured once and reused for subsequent measurements. Furthermore, quantitative measurement of dye molecules allows quantitative evaluation of the number of antigens on tissue and cell surfaces.

[0085] [Display] Next, the display unit 3 will be described.

[0086] 13 is a diagram illustrating the screen of the display unit 3. The display unit 3 may be configured as a monitor integrally attached to the processing unit 2, or may be a display device connected to the processing unit 2. The display unit 3 includes a display element such as a liquid crystal device or an organic EL device, and a touch sensor, and is configured as a UI (User Interface) that displays input settings for shooting conditions, shot images, etc.

[0087] 13, the display unit 3 has a main screen 301, a display screen 302 for thumbnail images, a display screen 303 for slide information, and a display screen 304 for a list of captured slides. The main screen 301 has a display area 305 for operation buttons (keys) for capturing images, a setting area 306 for the excitation laser (excitation unit 10), detection setting areas 307 and 308 for the fluorescence spectra from the line illuminations Ex1 and Ex2, etc. At least one of these display areas 305 to 308 is required at any one time, and one display area may include other display areas.

[0088] The fluorescence observation device 100 sequentially performs operations such as removing a slide (sample S) from a slide rack (not shown), reading slide information, photographing a thumbnail of the slide, and setting the exposure time. The slide information includes patient information, tissue location, disease, staining information, etc., and is read from a barcode or QR code (registered trademark) attached to the slide. The thumbnail image and slide information of the sample S are displayed on display screens 302 and 303, respectively. Display screen 304 displays a list of photographed slide information.

[0089] The main screen 301 displays the imaging status of the slide currently being imaged as well as the fluorescent image of the sample S. The excitation lasers (line illumination Ex1, Ex2) are displayed or set in the setting area 306, and the fluorescence spectra derived from the excitation lasers are displayed or set in the detection setting areas 307, 308.

[0090] 14 is a diagram showing an example of the screen configuration of the excitation laser setting area 306. Here, the output of each excitation light source L1 to L4 can be selected and switched on / off by touching a check box 81. The output magnitude of each light source is set via an operation unit 82. In this example, the line illumination Ex1 is set to the single wavelength of excitation light source L1.

[0091] Fig. 15 is a diagram showing an example of the screen configuration of detection setting area 307 for the fluorescence spectrum derived from line illuminator Ex1. Fig. 16 is a diagram showing an example of the screen configuration of detection setting area 308 for the fluorescence spectrum derived from line illuminator Ex2. The vertical axis represents brightness, and the horizontal axis represents wavelength.

[0092] 15 and 16, an index 83 indicates that the excitation light sources (L1, L2, L4) are lit, and the longer the index 83, the greater the power of the light source. The detection wavelength range of the fluorescence spectrum 85 is set by a setting bar 84.

[0093] The method for displaying the fluorescence spectrum 85 is not particularly limited, and for example, it is displayed as the average spectrum (wavelength x intensity) of all pixels of the image sensor 32. The fluorescence spectrum 85 can be set according to the wavelength and power of the excitation light source. The fluorescence spectrum 85 is displayed as the current average, or as a waveform calculated by taking into account setting changes from the last captured waveform.

[0094] 15 and 16, the fluorescence spectrum 85 may be displayed in a heat map format, in which the frequency information of values ​​is expressed by shading. In this case, it is possible to visualize the variance of the signal, which cannot be seen from the average value.

[0095] The vertical axis of the graph displaying the fluorescence spectrum 85 is not limited to a linear axis, but may be a logarithmic axis or a hybrid axis (biexponential axis).

[0096] The display unit 3 is configured to be able to display the fluorescence spectrum separately for each excitation line (Ex1, Ex2). The display unit 3 also has a UI with an operation area that explicitly displays the light source wavelength and power irradiated to each excitation line. The display unit 3 also has a UI that displays the detection wavelength range for each fluorescence spectrum. In other words, the readout area of ​​the image sensor 32 is configured to change based on the set wavelength range.

[0097] This allows the imaging conditions to be presented to the user in an easily understandable manner in a fluorescence observation device using a cross-axis excitation method. In particular, by providing the fluorescence spectrum detection setting areas 307 and 308 on the display unit 3, the relationship between the excitation line and excitation wavelength, and the relationship between the excitation wavelength and the imaging wavelength range can be displayed in an easily understandable manner, even in the case of cross-axis excitation.

[0098] The display unit 3 displays the fluorescence image of the sample S output from the image forming unit 23 on the main screen 301. The fluorescence image output from the image forming unit 23 to the display unit 3 is presented to the user with the value (the distance Δy between the line illuminators Ex1 and Ex2) corresponding to the difference in detection coordinates between the different-axis slits (each slit portion of the observation slit 31) corrected. Therefore, the user can recognize the image in which the respective decomposed image data are multiplexed, without being aware of the difference in the different-axis detection positions.

[0099] 17, for example, multiple decomposed images (image related to dye 1, image related to dye 2) are generated from spectral data derived from multiple line illuminations Ex1 and Ex2, and the images are displayed superimposed in different colors on the main screen 301. Here, the image related to dye 1 is superimposed on the image related to dye 2 by correcting the difference in the Y coordinate corresponding to Δy.

[0100] Each separation image corresponds to a standard spectrum, i.e., a staining dye, used in the separation calculation. In addition to the dye image in which each separation image is superimposed, the main screen 301 may also display a selection screen for displaying dyes. In this case, the image display switches in conjunction with the dye selection, and when dyes 1 and 2 are selected as shown in Figure 17, only images corresponding to those dyes are displayed.

[0101] The correction value for Δy is stored in the storage unit 21 and managed as internal information. The display unit 3 may be configured to be able to display information related to Δy, or may be configured to be able to change the displayed Δy. The correction value (Δy) may include not only correction for the distance between the slits (or the spacing between the line illuminations) but also the amount of distortion, such as distortion in the optical system. Furthermore, when the spectrum of each dye is detected by a different camera (image sensor), it may also include a correction amount related to the detection coordinate in the Y-axis direction for each camera.

[0102] <Modification>

[0103] In the above embodiment, the Δy image matching process (S104 in FIG. 12) and the stitching process (S105 in FIG. 12) are executed in the processing unit 2 before the color separation calculation process (S107 in FIG. 12), but this is not limiting. While this procedure has the advantage of obtaining stitched data of spectral data (spectral data) in addition to color-separated images, it also entails the storage unit 21 storing a huge amount of data. For this reason, at least one of the Δy image matching process and the stitching process may be executed after the color separation calculation process, as shown in FIGS. 18 and 19.

[0104] FIG. 18 shows an example of a processing procedure in which stitching processing (S105) is performed after color separation calculation processing (S107). In this example, stitching is not performed on large amounts of spectral data, and only images for which color separation calculation has been performed are stitched. In this case, the Δy image matching processing (S104) is performed on the spectral data before color separation calculation. According to this example, color separation is performed using data that combines spectral data from multiple excitation lines, thereby improving the signal-to-noise ratio of the separated images.

[0105] 19 shows an example of a processing procedure in which the Δy image alignment process (S104) and stitching process (S105) are performed after the color separation calculation process (S107). In this example, since stitching is not performed on the spectral data, an improvement in the S / N ratio of the separated images cannot be expected. However, since the Δy image alignment process is performed on separated images with a small amount of data, more advanced alignment correction, such as taking subpixels into consideration, is possible.

[0106] Next, a modified example of the configuration of the above-described fluorescence observation device 100 will be described.

[0107] Fig. 20 is a schematic block diagram of a fluorescence observation device 101 according to Modification 1, and Fig. 21 is a schematic block diagram of a fluorescence observation device 102 according to Modification 2. The fluorescence observation devices 101 and 102 each include an observation unit 1, a processing unit 2, a display unit 3, and a control program 81.

[0108] The control program 81 is a program that causes the fluorescence observation devices 101, 102 to execute functions similar to the control functions performed by the control unit 80 of the fluorescence observation device 100 described above. In the fluorescence observation device 101 shown in Fig. 20, the control program 81 is provided in a state stored on a recording medium such as a magnetic disk, optical disk, magneto-optical disk, or flash memory, and is used by being downloaded to a computer C or the like connected to the fluorescence observation device 101.

[0109] 21, a control program 81 distributed from outside via a network such as the Internet is downloaded to a computer C or the like for use. In this case, the fluorescence observation device 102 and a code for acquiring the control program 81 are provided in a packaged state.

[0110] The electronic computer C, to which the control program 81 has been downloaded, acquires various data for controlling the excitation unit 10, the spectral imaging unit 30, the scanning mechanism 50, the focusing mechanism 60, the non-fluorescence observation unit 70, etc., and executes the control algorithm of the downloaded control program 81 to calculate the control conditions of the fluorescence observation devices 101, 102. The electronic computer C issues commands to the fluorescence observation devices 101, 102 based on the calculated conditions, thereby automatically controlling the conditions of the fluorescence observation devices 101, 102.

[0111] The present technology can also be configured as follows. (1) a stage capable of supporting a fluorescently stained pathological specimen; an excitation unit that irradiates the pathological specimen on the stage with a plurality of line illuminations that are configured with different wavelengths and are parallel to one axis and different axes; a spectral imaging unit having at least one image sensor capable of individually receiving fluorescence excited by the plurality of line illuminations; A fluorescence observation device comprising: (2) The fluorescence observation device according to (1), The excitation unit irradiates the pathological specimen with a plurality of line illuminators each having a different wavelength combination as the plurality of line illuminators. Fluorescence observation device. (3) The fluorescence observation device according to (2), The spectral imaging unit further includes a wavelength dispersive element that disperses the fluorescence excited by the plurality of line illuminators. Fluorescence observation device. (4) The fluorescence observation device according to any one of (1) to (3) above, The spectral imaging unit further includes an observation slit having a plurality of slit portions through which the fluorescence excited by the plurality of line illuminations can pass. Fluorescence observation device. (5) The fluorescence observation device according to any one of (1) to (5) above, a scanning mechanism for scanning the stage with the plurality of line illuminations in a direction perpendicular to the one axial direction; Fluorescence observation device. (6) The fluorescence observation device according to any one of (1) to (5) above, The imaging device further includes a processing unit having a memory section for storing spectroscopic data representing a correlation between the wavelengths of the plurality of line illuminations and the fluorescence received by the imaging element. Fluorescence observation device. (7) The fluorescence observation device according to (6), The processing unit further includes an image forming unit that forms a fluorescent image of the pathological specimen based on the spectral data stored in the storage unit and the intervals between the plurality of line illuminators. Fluorescence observation device. (8) The fluorescence observation device according to (7), The image forming unit forms, as the fluorescent image, an image in which the detection coordinates of the image pickup element are corrected by a value corresponding to the intervals between the plurality of line illuminations. Fluorescence observation device. (9) The fluorescence observation device according to (6), The processing unit further includes a data calibration unit that calibrates the spectroscopic data stored in the storage unit. Fluorescence observation device. (10) The fluorescence observation device according to any one of (7) to (9) above, the storage unit stores in advance a standard spectrum of autofluorescence related to the pathological specimen and a standard spectrum of a single dye that stains the pathological specimen; The image forming unit outputs a component distribution of the spectral data based on the autofluorescence and the standard spectrum of the single dye. Fluorescence observation device. (11) The fluorescence observation device according to (4), The imaging element includes a plurality of imaging elements each capable of receiving the fluorescence that has passed through the observation slit. Fluorescence observation device. (12) The fluorescence observation device according to any one of (1) to (11) above, a non-fluorescence observation unit having a light source that illuminates the pathological specimen on the stage and an imaging unit that acquires a non-fluorescence image of the pathological specimen; Fluorescence observation device. (13) The fluorescence observation device according to any one of (1) to (12) above, The display unit further includes a display unit that displays, for each of the plurality of line illuminators, a fluorescence spectrum excited by the plurality of line illuminators. Fluorescence observation device. (14) The fluorescence observation device according to (13), The display unit has an operation area in which the wavelengths and outputs of the plurality of line illuminators can be set. Fluorescence observation device. (15) The fluorescence observation device according to (13) or (14), The display unit has a display area for displaying the detection wavelength range of the fluorescence spectrum. Fluorescence observation device. (16) Illuminating a pathological specimen on a stage with multiple line lights that are parallel to one axis and have different wavelengths, Fluorescence excited by the plurality of line illuminations is individually received. Fluorescence observation method. (17) The fluorescence observation method according to (16) above, further comprising: The stage is scanned with the plurality of line lights in a direction perpendicular to the one axial direction. Fluorescence observation method. (18) The fluorescence observation method according to (16) or (17), As the plurality of line lights, a plurality of line lights each composed of a combination of different wavelengths are used. Fluorescence observation method. [Explanation of symbols]

[0112] 1...Observation unit 2...Processing unit 3...Display section 10...Excitation section 20...Stage 21...Storage section 22...Data Proofreading Department 23...Image forming unit 30,130...Spectral imaging section 31...Observation slit 32, 32a, 32b...imaging element 35...Diffraction grating 38...Prism 50...Scanning mechanism 70...Non-fluorescent observation area 80...Control unit 81...Control program 100, 101, 102...Fluorescence observation device Ex1, Ex2...Line lighting S...Sample

Claims

1. an excitation unit that irradiates fluorescently stained cells with multiple line lights that are parallel to one axis and have different wavelengths; a spectral imaging unit having at least one image sensor capable of receiving individual fluorescence excited by the plurality of line illuminations; a processing unit that connects at least two of a plurality of spectral data that are received by the image sensor and that represent a correlation between the wavelengths of the plurality of line illuminations and the fluorescence received by the image sensor, and outputs a component distribution of the autofluorescence related to the cells and the dye that stains the cells from the spectral data; A fluorescence observation device comprising:

2. 2. The fluorescence observation device according to claim 1, The processing unit outputs a component distribution of the spectroscopic data based on a standard spectrum of autofluorescence related to the cell and a standard spectrum of a single dye that stains the cell. Fluorescence observation device.

3. 2. The fluorescence observation device according to claim 1, The excitation unit is configured to irradiate the cells with a plurality of line illuminators each having a different combination of wavelengths as the plurality of line illuminators. Fluorescence observation device.

4. 4. The fluorescence observation device according to claim 3, The spectral imaging unit further includes a wavelength dispersive element that disperses the fluorescence excited by the plurality of line illuminators. Fluorescence observation device.

5. 2. The fluorescence observation device according to claim 1, The spectral imaging unit further includes an observation slit having a plurality of slit portions through which the fluorescence excited by the plurality of line illuminations can pass. Fluorescence observation device.

6. 2. The fluorescence observation device according to claim 1, Further comprising a stage capable of supporting the fluorescently stained cells; a scanning mechanism for scanning the stage with the plurality of line illuminations in a direction perpendicular to the one axial direction; Fluorescence observation device.

7. 2. The fluorescence observation device according to claim 1, The processing unit has a storage unit that stores the spectroscopic data. Fluorescence observation device.

8. 8. The fluorescence observation device according to claim 7, The processing unit further includes an image forming unit that forms a fluorescent image of the cell based on the spectral data stored in the storage unit and the intervals between the plurality of line illuminators. Fluorescence observation device.

9. 9. The fluorescence observation device according to claim 8, The image forming unit forms, as the fluorescent image, an image in which the detection coordinates of the image pickup element are corrected by a value corresponding to the intervals between the plurality of line illuminations. Fluorescence observation device.

10. 8. The fluorescence observation device according to claim 7, The processing unit further includes a data calibration unit that calibrates the spectroscopic data stored in the storage unit. Fluorescence observation device.

11. 9. The fluorescence observation device according to claim 8, the storage unit stores in advance a standard spectrum of autofluorescence related to the cells and a standard spectrum of a single dye that stains the cells; The image forming unit outputs a component distribution of the spectral data based on the autofluorescence and the standard spectrum of the single dye. Fluorescence observation device.

12. 6. The fluorescence observation device according to claim 5, The imaging element includes a plurality of imaging elements each capable of receiving the fluorescence that has passed through the observation slit. Fluorescence observation device.

13. 2. The fluorescence observation device according to claim 1, a stage capable of supporting the fluorescently stained cells; a non-fluorescence observation unit having a light source that illuminates the cells on the stage and an imaging unit that acquires non-fluorescence images of the cells. Fluorescence observation device.

14. 2. The fluorescence observation device according to claim 1, The display unit further includes a display unit that displays, for each of the plurality of line illuminators, a fluorescence spectrum excited by the plurality of line illuminators. Fluorescence observation device.

15. The fluorescence observation device according to claim 14, The display unit has an operation area in which the wavelengths and outputs of the plurality of line illuminators can be set. Fluorescence observation device.

16. The fluorescence observation device according to claim 14, The display unit has a display area for displaying the detection wavelength range of the fluorescence spectrum. Fluorescence observation device.

17. Fluorescently stained cells are irradiated with multiple line lights that are parallel to one axis and have different wavelengths. receiving the fluorescence excited by the plurality of line illuminations individually; At least two of the plurality of spectral data representing the correlation between the wavelengths of the plurality of line illuminations received by an image sensor and the fluorescence received by the image sensor are linked together, and a component distribution of the autofluorescence related to the cells and the dye that stains the cells is output from the spectral data. Fluorescence observation method.

18. The fluorescence observation method according to claim 17, further comprising: the cells are supported by a stage; The stage is scanned with the plurality of line lights in a direction perpendicular to the one axial direction. Fluorescence observation method.

19. The fluorescence observation method according to claim 17, As the plurality of line lights, a plurality of line lights each composed of a combination of different wavelengths are used. Fluorescence observation method.

Images