Spectral imaging device and fluorescence observation device

Abstract

The spectral imaging device according to an embodiment of the present invention is provided with a spectral unit, an image sensor and a control unit. The spectral unit disperses incident light according to wavelength. The image sensor is configured so that an exposure time or a gain can be set for each pixel, and detects light of each wavelength dispersed in the spectral unit. The control unit is configured so that an exposure time or a gain of the image sensor can be set for each predetermined pixel area.

Description

Technical Field

[0001] The present technology relates to, for example, a spectral imaging device and a fluorescence observation device used for diagnosis of pathological images. Background Art

[0002] Pathological image diagnosis using fluorescent staining has been proposed as a highly quantitative and multicolor method (for example, see Patent Document 1). Compared with color staining, the advantage of the fluorescent method is that it is easy to multiplex and obtain detailed diagnostic information. In fluorescent imaging other than pathological diagnosis, the increase in the number of colors makes it possible to examine various antigens expressed in a sample at once.

[0003] Reference List

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 4452850 Summary of the invention

[0006] Technical issues

[0007] A spectral observation device that expands the horizontal axis of the area sensor to space and the vertical axis thereof (area sensor) to wavelength can easily obtain a spectrum of a line on a sample. However, in the case where a bright wavelength band, a very dark wavelength band, etc. are mixed in the spectrum, the dynamic range of the sensor itself is insufficient, the dark part collapses, or the bright part is saturated, resulting in failure to obtain sufficient data. On the other hand, in order to solve this problem, if a sensor with a large recording capacity is used, the storage capacity increases in a target such as a pathological image, where the number of entire pixels becomes huge, and new problems such as reduced accessibility of data and slow operation of the entire system arise.

[0008] In view of the above circumstances, an object of the present technology is to provide a spectral imaging device and a fluorescence observation device capable of recording in a high dynamic range while suppressing the recording capacity of a sensor.

[0009] Problem Solving

[0010] A spectral imaging device according to an embodiment of the present technology includes a spectral section, an image sensor, and a control unit.

[0011] The spectral unit disperses incident light for each wavelength.

[0012] The image sensor is configured to be able to set exposure time or gain in units of pixels, and detect light of each wavelength dispersed in a spectrum portion.

[0013] The control unit is configured to be able to set an exposure time or a gain of the image sensor in units of predetermined pixel areas.

[0014] According to the above-described spectral imaging apparatus, optimal exposure conditions can be obtained and the dynamic range of the spectrum to be recorded can be expanded.

[0015] The spectroscopy section may be configured to disperse incident light in one axial direction for each wavelength, and the control unit may be configured to set an exposure time of the image sensor in units of lines perpendicular to the one axial direction.

[0016] The image sensor may further include a pixel portion and a calculation portion that calculates a pixel value of the image data output from the pixel portion. In this case, the control unit is configured to set a gain for calculating the pixel value in units of a predetermined pixel area.

[0017] The control unit may include an evaluation unit that obtains an emission spectrum of incident light based on an output of the image sensor, and a storage unit that stores a plurality of reference component spectra and an autofluorescence spectrum. The evaluation unit is configured to calculate a component ratio of the emission spectrum so that a linear sum of the plurality of reference component spectra and the autoluminescence spectrum is obtained.

[0018] The evaluation section is configured to calibrate at least one of the emission spectrum or the component spectrum based on an exposure time or a gain set for each predetermined pixel area.

[0019] The evaluation section may be configured to determine whether there are pixels whose pixel values ​​are saturated from the captured spectrum, and exclude the saturated pixels from calculation of the component ratio of the captured spectrum.

[0020] A fluorescence observation apparatus according to an embodiment of the present technology includes a stage, an excitation section, a spectroscopy section, an image sensor, and a control unit.

[0021] The stage is configured to support fluorescently stained pathological specimens.

[0022] The excitation unit illuminates the pathological specimen on the stage with line illumination.

[0023] The spectral part disperses the fluorescence excited by the linear illumination for each wavelength.

[0024] The image sensor is configured to be able to set exposure time or gain in units of pixels, and detect light of each wavelength dispersed in a spectrum portion.

[0025] The control unit is configured to set an exposure time or a gain of the image sensor in units of predetermined pixel areas.

[0026] The fluorescence observation device may further include a display section for displaying the fluorescence spectrum based on the output of the image sensor.

[0027] The display section may have an operation area for receiving an input of an exposure time or a gain in units of a predetermined pixel area.

[0028] The display section may have a display area for displaying a spectrum and a histogram after setting based on the exposure time or the gain setting.

[0029] Advantageous Effects of the Invention

[0030] As described above, according to the present technology, it is possible to perform recording in a high dynamic range while suppressing the recording capacity of the sensor.

[0031] Note that the effects described here are not necessarily restrictive, and any effects described in the present disclosure may be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] [ Figure 1 ] Figure 1 : is a schematic diagram showing a basic configuration of a spectral imaging device according to an embodiment of the present technology.

[0033] [ Figure 2 ] Figure 2 is a schematic diagram showing an optical system of a fluorescence observation device provided with a spectral imaging device.

[0034] [ Figure 3 ] Figure 3 It is a schematic diagram of a pathological specimen of an observation target.

[0035] [ Figure 4 ] Figure 4 is a block diagram showing the configuration of a fluorescence observation apparatus.

[0036] [ Figure 5 ] Figure 5 This is a block diagram showing the arrangement of a detection unit and its periphery in a fluorescence observation device.

[0037] [ Figure 6 ] Figure 6 This is a schematic diagram for explaining the relationship between a pixel portion and an emission spectrum.

[0038] [ Figure 7 ] Figure 7 is an explanatory diagram showing the relationship between the emission spectrum and the dynamic range in the detection area.

[0039] [ Figure 8 ] Figure 8 : is a flowchart showing the processing procedure up to the component separation calculation of the emission spectrum executed in the control unit.

[0040] [ Fig. 9 ] Fig. 9 is a flowchart showing an example of a saturation processing procedure in the embodiment.

[0041] [ Fig.10 ] Fig.10 is a schematic diagram for explaining an example of saturation processing.

[0042] [ Fig.11 ] Fig.11 This is a schematic diagram of a display unit in a fluorescence observation device.

[0043] [ Fig.12 ] Fig.12 : is a diagram showing an example of a screen configuration of a setting area of ​​an excitation section in a display section.

[0044] [ Fig.13 ] Fig.13 : is a diagram showing an example of a screen configuration of a detection setting area of ​​a fluorescence spectrum from one line illumination in a display section.

[0045] [ Fig.14 ] Fig.14 : is a diagram showing an example of a screen configuration of a detection setting area of ​​a fluorescence spectrum from other line illumination in a display section.

[0046] [ Fig.15 ] Fig.15 It is a diagram for explaining the histogram window in the display unit.

[0047] [ Fig.16 ] Fig.16 : is a block diagram of the fluorescence observation apparatus for explaining the processing executed in the control unit.

[0048] [ Fig.17 ] Fig.17 is a schematic block diagram showing a modification of the fluorescence observation apparatus.

[0049] [ Fig.18 ] Fig.18 is a schematic block diagram showing other modifications of the fluorescence observation apparatus. DETAILED DESCRIPTION

[0050] Embodiments according to the present technology will be described below with reference to the drawings.

[0051] [Device Overview]

[0052] Figure 1 is a schematic diagram showing a basic configuration of a spectral imaging device 10 according to an embodiment of the present technology.

[0053] As shown in the same figure, the spectral imaging device 10 is a line scanning type imaging spectrometer, and includes a spectral section 11 and a detection section 12. The spectral section 11 has a slit 111 parallel to the X-axis direction and a wavelength dispersion element 112. The detection section 12 includes an image sensor (area sensor) 121, and the image sensor 121 includes a solid-state imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) and a CCD (Charge Coupled Device).

[0054] The slit 111 extracts the spatial component of the incident light (fluorescence) in the X-axis direction from a sample (not shown) on the xy plane. The wavelength dispersion element 112 disperses the incident light Lx passing through the slit 111 for each wavelength to image the image sensor 121. As the wavelength dispersion element 112, a prism or a diffraction grating is generally used to separate each wavelength band of the incident light Lx in the Y-axis direction. The image sensor 121 obtains a spectral image of (X, λ) of the incident light L1 wavelength-separated in the wavelength dispersion element 112. By combining a mechanism for scanning the sample in the Y-axis direction, a spectral image of (X, Y, λ) can be obtained.

[0055] The image sensor 121 is configured to be able to set the exposure time or gain for the pixel unit, as will be described later. By adjusting the exposure time or gain depending on the light receiving area of ​​the light in each wavelength band, the saturation of the light in the bright wavelength band can be suppressed, and a spectral image with sufficient sensitivity to the light in the dark wavelength band can be obtained.

[0056] In addition, the image sensor 121 is configured to read only a portion of the area from the readout area in the full frame. As a result, the frame rate can be increased by an amount corresponding to the reduction in the readout area. In addition, any area of ​​the readout area can be divided into multiple areas, and different gains and exposure times can be set in each area.

[0057] [Fluorescence observation device]

[0058] Figure 2 is a schematic diagram showing an optical system of a fluorescence observation apparatus 100 including the spectral imaging apparatus 10 of the present embodiment.

[0059] As shown in the figure, the fluorescence observation device 100 includes a spectroscopic section 11 , a detection section 12 , and a fluorescence excitation section 13 . The fluorescence excitation section 13 includes an excitation light optical system 131 , a filter block 132 , and an objective lens 133 .

[0060] The excitation light optical system 131 includes one or more light sources capable of emitting excitation light. As the light source, a light emitting diode (LED), a laser diode (LD), a mercury lamp, etc. are used. The excitation light is line-illuminated and irradiates the sample S on the stage 20 parallel to the xy plane.

[0061] The sample S is usually formed of a slide including an observation target Sa, such as Figure 3 However, it should be understood that the sample S may be formed of something other than such a slide glass. The sample S (observation target Sa) is stained with a plurality of fluorescent pigments excited by irradiation of excitation light.

[0062] The filter block 132 includes a dichroic mirror, a bandpass filter, etc. The dichroic mirror reflects the excitation light from the excitation light optical system 131 toward the objective lens 133, and transmits the fluorescence from the sample S that has passed through the objective lens 133 toward the spectrum section 11. The bandpass filter has a bandpass characteristic of cutting off the wavelength band of the excitation light from the sample S toward the spectrum section 11.

[0063] Figure 4 2 is a block diagram showing the configuration of the fluorescence observation apparatus 100 . The fluorescence observation apparatus 100 includes an apparatus main body 1 , a control unit 2 , and a display section 3 .

[0064] The apparatus body 1 includes a stage 20 , an excitation light source (excitation unit) 101 , a spectral imaging unit 102 , an observation optical system 103 , a scanning mechanism 104 , a focusing mechanism 105 , a non-fluorescence observation unit 106 , and the like.

[0065] The excitation light source 101 corresponds to the excitation optical system 131, and the spectral imaging section 102 corresponds to the spectral section 11 and the detection section 12. The observation optical system 103 corresponds to the filter block 132 and the objective lens 133.

[0066] The scanning mechanism 104 is generally formed of an XY moving mechanism that moves in at least two directions of the X-axis and the Y-axis in parallel with the stage 20. In this case, for example, Figure 3 As shown, the image capture area Rs is divided into a plurality of areas in the X-axis direction, and the operation is repeatedly performed, that is, the sample S is scanned in the Y-axis direction, then the sample S is moved in the X-axis direction, and further scanning is performed in the Y-axis direction. As a result, a spectral image of a large area can be obtained, for example, in the case of a pathological slide, etc., WSI (whole slide imaging) can be obtained.

[0067] The focusing mechanism 105 moves the stage 20 or the objective lens 133 to the best focus position in the direction perpendicular to the X-axis and the Y-axis. The non-fluorescence observation section 106 is used for dark field observation, bright field observation, etc. of the sample S, but can be omitted as needed.

[0068] The fluorescence observation device 100 can be connected to a control unit 80 for controlling a fluorescence excitation unit (control of a laser diode or a shutter), an XY stage as a scanning mechanism, a spectral imaging unit (camera), a focusing mechanism (a detection unit and a Z stage), a non-fluorescence observation unit (camera), etc.

[0069] [Image sensor]

[0070] Figure 5 2 is a block diagram showing the configuration of the detection unit 12 and its periphery.

[0071] As shown in the same figure, the detection section 12 includes an image sensor 121 and a signal processing circuit 122. The image sensor 121 includes a pixel section 30 and a calculation section 31.

[0072] The pixel section 30 outputs charge information corresponding to the exposure time by photoelectric conversion in each pixel of a Bayer array composed of, for example, RGB pixels. The pixel section 30 is set to different exposure times in units of pixel areas (for example, row (line) units) by control (shutter control) of the control unit 2. From the row to be subjected to long-time exposure, high-sensitivity pixel information 311 corresponding to the accumulated charge based on the long-time exposure is output. From the row to be subjected to short-time exposure, low-sensitivity pixel information 312 corresponding to the accumulated charge based on the short-time exposure is output.

[0073] The calculation section 31 calculates a pixel value from the image data output from the pixel section 30. In the present embodiment, the calculation section 31 inputs the high-sensitivity pixel information 311 and the low-sensitivity pixel information 312 output from the pixel section 30, and has an image information synthesis section 313 for generating a piece of image information based on the input information. The output of the image information synthesis section 313 is input to the signal processing circuit 122. The signal processing circuit 122 performs signal processing such as white balance (WB) adjustment and γ correction to generate an output image. The output image is provided to the control unit 2, stored in the storage section 21 described later, or output to the display section 3.

[0074] The image sensor 121 obtains fluorescence spectrum data (x, λ) using the pixel array in the Y-axis direction (vertical direction) of the pixel unit 30 as a channel of wavelength. The obtained spectrum data (x, λ) is recorded in the control unit 2 (storage unit 21) in a state where the spectrum data excited by the excitation wavelength is bound or not.

[0075] The exposure time of the pixel unit 30 is set for each predetermined pixel area by the control unit 2. In this embodiment, since the wavelength dispersion element 112 in the spectral unit 11 irradiates the incident light Lx (see Figure 1 ) wavelength separation, so that light with different wavelengths in the Y-axis direction (emission spectrum) reaches the pixel section 30 of the image sensor 121. Therefore, in the present embodiment, as described above, by the control of the control unit 2 (shutter control), the exposure time of the pixel section 30 is set to be a unit of a line perpendicular to the Y-axis direction and parallel to the X-axis direction.

[0076] The control unit 2 is also configured to be able to individually set, in units of pixel areas, a gain for sensitivity compensation to be multiplied by each of the high-sensitivity pixel information 311 and the low-sensitivity pixel information 312 in the image information synthesis section 313 of the calculation section 31. Thus, the sensitivity of the low-sensitivity pixel information can be improved while suppressing the saturation of the high-sensitivity pixel information 311.

[0077] The setting values ​​of exposure time and gain are not particularly limited, and may be arbitrary values ​​or values ​​based on the emission spectrum intensity of the pigment measured in advance. For example, when the exposure time and the gain of the pixel value of the low-sensitivity pixel area are set to 1, the exposure time and the gain of the pixel value of the high-sensitivity pixel information area are set to, for example, a range of about 1.5 to 5.0.

[0078] In addition, it is not limited to the case where both the exposure time and the gain are set, and it may be set that only the exposure time is adjustable, or it may be set that only the gain is adjustable. Alternatively, one of the exposure time and the gain is used as the main setting value, and the other may be a supplementary setting value. For example, by setting the exposure time as the main setting value, image data with good S / N can be obtained.

[0079] Figure 6 It is a schematic diagram for explaining the relationship between the pixel unit 30 and the emission spectrum.

[0080] As shown in the same figure, the control unit 2 determines the wavelength range of the emission spectrum and the transmission wavelength range of the filter block 132 (see Figure 2 ) and the entire readout area of ​​the image sensor 121 (pixel unit 30) to determine the detection area. In the case of fluorescence imaging, the filter block 132 generally has a bandpass characteristic for cutting off the excitation light. Therefore, if there are multiple excitation wavelengths, a band (opaque band DZ) where the wavelength is not transmitted as shown in the same figure is generated. The control unit 2 excludes the area that does not include such a signal to be detected from the detection area.

[0081] like Figure 6 As shown, when regions located above and below the light non-penetrating zone DZ are taken as ROI1 and ROI2, respectively, an emission spectrum (hereinafter, also referred to as a fluorescence spectrum) of a pigment having corresponding peaks is detected. Figure 7 It is an explanatory diagram showing the relationship between the emission spectrum in the detection area and the dynamic range, (a) of the same figure shows the data obtained before setting the exposure time and gain (the exposure time or gain is the same in each detection area), and (b) of the same figure shows the data obtained after setting the exposure time and gain respectively.

[0082] like Figure 7As shown in (a) of FIG. 1 , the pigment of ROI1 has a strong spectral intensity and is saturated beyond the dynamic range of detection, whereas the pigment of ROI2 has a weak intensity. Figure 7 As shown in (b) in FIG. 1 , the exposure time of the (X, λ) region corresponding to ROI1 is set to be relatively short (or the gain is set to be relatively small), and on the contrary, the exposure time of the (X, λ) region corresponding to ROI2 is set to be relatively long (or the gain is set to be relatively large). As a result, dark and light pigments can be captured with appropriate exposure. The coordinate information of the detection area, such as ROI1 and ROI2, the gain, and the information about the exposure time are stored in the storage unit 21 of the control unit 2.

[0083] [Control Unit]

[0084] The fluorescence spectrum obtained by the detection section 12 (spectral imaging section 102) including the image sensor 121 is output to the control unit 2. The captured data of a plurality of fluorescence spectra can be quantitatively evaluated by component analysis (color separation) based on the spectra of individual pigments or the like. The control unit 2, such as Figure 4 As shown, it includes a storage unit 21 and an evaluation unit 22 .

[0085] The control unit 2 can be implemented by hardware elements used in a computer, such as a CPU (central processing unit), a RAM (random access memory) and a ROM (read-only memory)) and necessary software. A PLD (programmable logic device) such as an FPGA (field programmable gate array), or a DSP (digital signal processor), other ASICs (application-specific integrated circuits (ASICs)), etc. can be used as a substitute or supplement to the CPU.

[0086] The storage section 21 stores in advance component spectra of a plurality of references of the pigment used alone to dye the sample S and the self-luminous spectrum of the sample S (hereinafter, also collectively referred to as standard spectrum). The evaluation section 22 separates the emission spectrum of the sample S obtained by the image sensor 121 into spectra derived from the pigment and the self-luminous spectrum based on the standard spectrum stored in the storage section 21, and calculates each component ratio. In the present embodiment, the component ratio of the emission spectrum of the captured sample S is calculated so as to be the linear sum of the standard spectrum.

[0087] On the other hand, the emission spectrum of the sample S obtained by the image sensor 121 is modulated from the original spectrum because the exposure time and gain are set individually for each detection area. Therefore, if the data obtained by the image sensor 121 is used as it is, the color separation calculation of the component spectra may not be accurately performed.

[0088] Therefore, the evaluation section 22 is configured to calibrate at least one of the emission spectrum or the reference component spectrum based on the exposure time and gain set for each predetermined pixel area (detection area) of the image sensor 121 .

[0089] Figure 8 1 is a flowchart showing a processing procedure up to component separation calculation of an emission spectrum performed in the control unit 2. Hereinafter, the emission spectrum of the sample S obtained by the image sensor 121 is also referred to as a captured spectrum.

[0090] As shown in the same figure, the control unit 2 sets the exposure time and gain of the detection area of ​​the pixel section 30 of the image sensor 121 (step 101). These setting values ​​are input by the user via the display section 3 described later. After recording the exposure time and gain settings to the storage section 21, the control unit 2 obtains the captured spectrum of the sample S via the image sensor 121 (steps 102 and 103).

[0091] The control unit 2 demodulates the captured spectrum based on the set gain and the set exposure time of each detection area, or calibrates the captured spectrum by modulating the standard spectrum stored in the storage unit 21 (step 104). In other words, the captured spectrum and the standard spectrum are converted to a common intensity axis based on the set exposure time and the set gain. The intensity axis includes, for example, the number of charges per unit time [e - ], spectral radiance [W / (sr·m 2 ·nm)], etc. When the standard spectrum is changed, the standard spectrum is multiplied by the relative intensity ratio of each detection area when captured. Thereafter, if necessary, the saturation processing (step 105) described later is performed, and then the component separation calculation of the captured spectrum is performed (step 106).

[0092] On the other hand, in the case of performing spectrum capture of multiple fluorescent samples by spectroscopy, it is important to set parameters such as exposure time and gain for each pixel so that capture can be performed in advance without saturation. However, in WSI (whole slide imaging) and the like, it is difficult to obtain optimal exposure in all areas of the sample, and the time loss is also large. When saturation occurs during capture, the peak of the spectrum reaches a limit at the AD (analog to digital conversion) maximum value of the sensor, making it impossible to capture the correct spectrum. Therefore, there arises a problem that the deviation from the component spectrum (standard spectrum) prepared in advance for color separation calculation becomes large, and correct calculation cannot be performed.

[0093] Therefore, in this embodiment, in addition to expanding the dynamic range by ROI (Region of Interest) setting, saturation processing described later is performed. This enables color separation calculation to be performed correctly even when there is some saturation in the captured spectrum, thereby reducing the number of capture retries.

[0094] The saturation processing in this embodiment performs processing to specify a pixel where saturation occurs and exclude the pixel from calculation. An example of the processing procedure is shown in FIG. Fig. 9 Shown in.

[0095] Fig. 9 is a flow chart showing the saturation process.

[0096] As shown in the same figure, the control unit 2 performs a process of generating a saturated detection array from the acquired captured spectrum (step 201). Fig.10 As shown, the presence or absence of saturation of the captured spectrum is determined for each wavelength (channel), and a saturation detection array is generated, where unsaturated channels are set to "1" and saturated channels are set to "0".

[0097] The presence or absence of saturation is determined by referring to the pixel value of each detection area and whether it reaches the maximum brightness value. Since the pixel area that is estimated to reach the maximum brightness value is saturated compared to the original correct spectrum, the channel of the reference spectrum corresponding to the pixel area (channel) is removed from the component separation calculation.

[0098] In general, the number of channels (CH number) of wavelengths recorded by spectral capture is usually greater than the number of components in the final output. Therefore, if the number of valid channels that are not saturated is greater than the number of components, component separation calculations can be performed even if the data of the saturated channels is removed from the calculation.

[0099] When the number of valid channels in the generation array (the number of channels is determined to be "1") is greater than the number of components (the number of channels) of the final output, a process of multiplying the saturated detection array by the captured spectrum and the reference spectrum is performed (steps 203 and 204). Otherwise, calculation is impossible, and therefore, the process is terminated without performing component separation calculation. As a result, since the channel in which saturation occurs is excluded from the calculation of the least squares method, the component ratio calculation can be performed using only the wavelength measured correctly.

[0100] According to the present embodiment as described above, with respect to the image sensor 121 capable of changing the gain setting and exposure time of any detection area, a spectral imaging optical system is provided to expand the horizontal axis of the image sensor 121 to space and the vertical axis thereof to wavelength. From each area of ​​the image sensor 121, by setting to read only the detection area, further dividing the detection area into two or more two-dimensional spatial ROIs of wavelength×space, and by setting a different combination of gain and exposure time for each detection area, the optimal exposure condition can be obtained, and the dynamic range of the spectrum to be recorded can also be expanded.

[0101] For example, when taking a multi-fluorescent dye sample, the blue fluorescent pigment can have a very high intensity compared to the red fluorescent pigment. Under such conditions, the exposure time of the blue wavelength band is shortened, the gain is set lower, and the exposure time of the red wavelength band is extended, and the gain is set higher. As a result, recording with a shallow bit range can be performed, so that recording with a high dynamic range can be performed while suppressing the recording capacity of the sensor.

[0102] The detection area of ​​the image sensor 121 is set according to the spectrum of the object to be measured within the sensitivity area of ​​the sensor. If there are non-transparent bands such as notch filters and areas where there is no light in the observation optical path, the recording frame rate can be increased by excluding them from the readout area.

[0103] Furthermore, according to the present embodiment, when the color mixing ratio of each pigment is calculated individually from the acquired spectrum, even if there is some saturation in the captured spectrum, it is possible to detect the saturated wavelength and other wavelengths by generating a saturation detection array (see Fig.10 ) to perform color separation by spectral fitting.

[0104] [Display]

[0105] The problem with setting capture parameters via ROI is that it is difficult for the user to understand the capture conditions. Because the data is three-dimensional in space and wavelength, it is difficult to see where saturation occurs and which wavelength has insufficient signal. The part that performs ROI setting and display needs to be able to comprehensively display and set the relationship between the setting parameters and the capture range, the relationship between the setting parameters and the sensor output, etc.

[0106] Therefore, in the present embodiment, the display section 3 is configured as follows, and the details of the display section 3 will be described below. Here, as an example, the configuration of the display section 3 assuming multiple times of fluorescent imaging will be described.

[0107] Fig.11 1 is a schematic diagram illustrating the display section 3. The display section 3 is configured to be able to display the fluorescence spectrum of the sample S based on the output of the image sensor 121. The display section 3 may be composed of a monitor integrally mounted to the control unit 2, or may be a display device connected to the control unit 2. The display section 3 includes a display element such as a liquid crystal device or an organic EL device and a touch sensor, and is configured as a UI (User Interface) that displays settings for inputting shooting conditions, shooting images, and the like.

[0108] like Fig.11As shown, the display section 3 includes a main screen 301, a thumbnail image display screen 302, a slide information display screen 303, and a captured slide list display screen 304. The main screen 301 includes a display area 305 of control buttons (keys) for capturing, a setting area 306 for excitation laser (line illumination), detection setting areas 307 and 308 for spectrum, a spectrum automatic setting control area 309, etc. At least one of these areas 305 to 309 may be present, and other display areas may be included in one display area.

[0109] The fluorescence observation device 100 sequentially performs the following operations: taking out a slide (sample S) from a slide rack (not shown), reading slide information, capturing a thumbnail of the slide, setting an exposure time, etc. The slide information includes patient information, tissue site, disease, staining information, etc., and is read from a barcode, QR code (registered trademark), etc. attached to the slide. The thumbnail image of the sample S and the slide information are displayed on display screens 302 and 303, respectively. The captured slide information is displayed as a list on a screen 304.

[0110] In addition to the fluorescent image of the sample S, the capture status of the currently captured slide is displayed on the main screen 301. The excitation laser is displayed or set in the setting area 306, and the fluorescent spectrum derived from the excitation laser is displayed or set in the detection setting areas 307 and 308.

[0111] Fig.12 3 is a diagram showing an example of the screen configuration of the setting area 306 for the excitation laser. Here, the output of each excitation line L1-L4 is selected or switched on / off by a touch operation on each check box 81. In addition, the magnitude of the output of each light source is set by the operation section 82.

[0112] Fig.13 An example of the screen configuration of the spectrum detection setting area 307 in the excitation line 1 is shown, and Fig.14 An example of the screen configuration of the spectrum detection setting area 308 in the excitation line 2 is shown. In each figure, the vertical axis represents brightness, and the horizontal axis represents wavelength. These detection setting areas 307 and 308 are each configured as an operation area for accepting input of exposure time and gain in predetermined units of pixels of the image sensor 121.

[0113] exist Fig.13 and Fig.14 In the figure, the indicator 83 indicates that the excitation light sources (L1, L2, and L4) are bright, and the longer the length of the indicator 83, the greater the power of the light source. The detection wavelength range of the fluorescence spectrum 85 is set by the operation bar 84. The display method of the fluorescence spectrum 85 is not particularly limited, for example, it is displayed as a total pixel average spectrum (wavelength×intensity) at the excitation lines 1 and 2.

[0114] like Fig.13 and Fig.14 As shown, the fluorescence spectrum 85 can be displayed by means of a heat map method, wherein the frequency information of the values ​​is indicated by shading. In this case, unclear signal distributions can also be visualized by the average value.

[0115] Note that the vertical axis of the graph for displaying the fluorescence spectrum 85 is not limited to a linear axis, and may be a logarithmic axis or a mixed axis (double exponential axis).

[0116] The fluorescence spectrum 85 can be set according to the wavelength and power of the excitation light source. The wavelength range of the fluorescence spectrum 85 can be arbitrarily changed by a cursor movement operation on the operation bar 84 using an input device such as a mouse. The fluorescence spectrum 85 is represented by a current average value or a waveform calculated from the last captured waveform in consideration of the setting change.

[0117] The control unit 2 sets the reading area of ​​the image sensor 121 based on the wavelength band (set value) input to the detection setting areas 307 and 308. Based on the wavelength band set by the detection setting areas 307 and 308 and the predetermined conversion formula (conversion formula to pixels corresponding to the wavelength) obtained in advance, the sensor coordinates are specified, and the exposure time and gain are set. Display areas that can set the exposure time and gain individually can be provided separately. After the settings are input based on the exposure time and gain and the settings are made via the operation bar 84, the detection setting areas 307 and 308 display the fluorescence spectrum 85.

[0118] Fig.15 An example of the screen configuration of the spectrum automatic setting control area 309 is shown. In the spectrum automatic setting control area 309, an automatic setting key 86, a histogram window 87, etc. are arranged. The automatic setting start key 86 automatically performs pre-sampling imaging and the above-mentioned spectrum detection setting. The histogram window 87 calculates and displays a histogram corresponding to the wavelength range of the spectrum set in the detection setting areas 307 and 308. The vertical axis of the histogram is frequency, and the horizontal axis is wavelength.

[0119] Referring to the histogram window 87, the occurrence of saturation and the presence or absence of insufficient signal (insufficient intensity) can be clearly confirmed when captured under the detection conditions of the spectrum set in the detection setting areas 307 and 308. In addition, the exposure time and gain can be changed while checking the histogram.

[0120] Fig.16 2 is a block diagram of the fluorescence observation apparatus 100 for explaining the processing executed in the control unit 2 .

[0121] The control unit 2 stores the parameters set in the respective setting areas 306 to 308 of the display section 3 in the storage section 21 (see Figure 4 ), and based on the parameters, the reading area (wavelength band), exposure time and gain are set for the image sensor 121 (S401).

[0122] The control unit 2 outputs the emission spectrum of the sample S obtained by the image sensor 121 to the display section 3 (S402), and the waveform of the spectrum is displayed in the detection setting areas 307 and 308 (see FIG. 404 ). Fig.13 and Fig.14 ).

[0123] In the automatic setting control mode, the control unit 2 performs optimization processing of the exposure time and the gain based on the captured data of the image sensor 121 (step 403), and repeats the processing of acquiring the captured data of the changed parameters.

[0124] On the other hand, when the component separation calculation of the captured spectrum is performed, the above-described component separation calculation is performed based on the captured data of the image sensor 121, and the result is displayed on the display section 3 (for example, the main screen 301) (S404).

[0125] As described above, according to the present embodiment, based on the set wavelength band, exposure time and gain, the spectrum and histogram after setting are captured and displayed in real time, and the spectrum and histogram under the new setting value are displayed from the obtained spectrum. Therefore, the relationship between the setting parameters and the capture range, the relationship between the setting parameters and the sensor output, etc. can be comprehensively displayed and set.

[0126] <Edit>

[0127] Next, a modification of the configuration of the above-described fluorescence observation apparatus 100 is described.

[0128] Fig.17 is a schematic block diagram of a fluorescence observation apparatus 200 according to Modification 1, and Fig.18 is a schematic block diagram of a fluorescence observation apparatus 300 according to Modification 2. The fluorescence observation apparatuses 200 and 300 include an apparatus main body 1 , a control unit 2 , a display section 3 , and a control program 81 , respectively.

[0129] The control program 81 is a program for causing the fluorescence observation apparatuses 200 and 300 to execute the same function as the control function executed by the control unit 80 of the above-mentioned fluorescence observation apparatus 100. Fig.17 In the illustrated fluorescence observation apparatus 200 , the control program 81 is provided in a state stored in a recording medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a flash memory, and is downloaded to and used by an electronic computer C or the like connected to the fluorescence observation apparatus 200 .

[0130] On the other hand, Fig.18In the fluoroscopy device 300 shown, a control program 81 distributed from the outside via a network such as the Internet is downloaded to an electronic computer C or the like and used. In this case, the fluoroscopy device 300 and a code for obtaining the control program 81 are packaged and provided.

[0131] The electronic computer C to which the control program 81 is downloaded obtains various data for controlling the excitation light source 101, the spectral imaging section 102, the scanning mechanism 104, the focusing mechanism 105, the non-fluorescence observation section 106, etc., and the control algorithm of the downloaded control program 81 is executed, and the control conditions of the fluorescence observation devices 200 and 300 are calculated. The electronic computer C issues commands to the fluorescence observation devices 200 and 300 based on the calculated conditions, so that the conditions of the fluorescence observation devices 200 and 300 are automatically controlled.

[0132] Although the embodiments of the present technology have been described above, it goes without saying that the present technology is not limited to the above-described embodiments, and various modifications can be made.

[0133] The present technology may also have the following structures.

[0134] (1) A spectral imaging device comprising:

[0135] The spectral part disperses the incident light for each wavelength;

[0136] an image sensor configured to be able to set exposure time or gain in units of pixels, the image sensor detecting light of each wavelength dispersed in a spectral portion; and

[0137] The control unit is configured to be able to set the exposure time or gain of the image sensor in units of predetermined pixel areas.

[0138] (2) The spectral imaging device according to (1), wherein

[0139] The spectral section is configured to disperse incident light in one axial direction for each wavelength, and

[0140] The control unit is configured to set an exposure time or a gain of the image sensor in units of lines perpendicular to the one axial direction.

[0141] (3) The spectral imaging device according to (1) or (2), wherein

[0142] The image sensor includes a pixel portion and a calculation portion that calculates a pixel value based on image data output from the pixel portion, and

[0143] The control unit is configured to set a gain for calculating a pixel value in units of a predetermined pixel area.

[0144] (4) The spectral imaging device according to any one of (1) to (3), wherein

[0145] The control unit includes an evaluation section that obtains an emission spectrum of incident light based on an output of the image sensor; a storage section that stores a plurality of reference component spectra and autofluorescence spectra, and

[0146] The evaluation section is configured to calculate a component ratio of the emission spectrum so that a linear sum of a plurality of reference component spectra and the self-luminous spectrum is obtained.

[0147] (5) The spectral imaging device according to (4), wherein

[0148] The evaluation section is configured to calibrate at least one of the emission spectrum or the component spectrum based on an exposure time or a gain set for each predetermined pixel area.

[0149] (6) The spectral imaging device according to (5), wherein

[0150] The evaluation section is configured to determine whether there are pixels whose pixel values ​​are saturated from the captured spectrum, and exclude the saturated pixels from calculation of the component ratio of the captured spectrum.

[0151] (7) A fluorescence observation device comprising:

[0152] A stage capable of supporting fluorescently stained pathological specimens;

[0153] The excitation section uses line illumination to illuminate the pathological specimen on the stage;

[0154] The spectral part disperses the fluorescence excited by the line illumination for each wavelength;

[0155] an image sensor configured to be able to set exposure time or gain in units of pixels, the image sensor detecting light of each wavelength dispersed in a spectral portion; and

[0156] The control unit is configured to set the exposure time or gain of the image sensor in units of predetermined pixel areas.

[0157] (8) The fluorescence observation device according to (7), further comprising:

[0158] The display unit is configured to display the fluorescence spectrum based on the output of the image sensor.

[0159] (9) The fluorescence observation device according to (8), wherein

[0160] The display section has an operation area for receiving an input of an exposure time or a gain in units of a predetermined pixel area.

[0161] (10) The fluorescence observation device according to (8) or (9), wherein

[0162] The display unit has a display area for displaying a spectrum and a histogram based on the set exposure time or gain setting.

[0163] Reference Mark List

[0164] 2 Control unit

[0165] 3 Display

[0166] 10 Spectral imaging device

[0167] 11 Spectrum

[0168] 12. Inspection Department

[0169] 13. Fluorescence excitation section

[0170] 20 Stage

[0171] 21 Storage

[0172] 22 Evaluation Department

[0173] 30 pixels

[0174] 31 Computing Department

[0175] 100, 200, 300 Fluorescence observation device

[0176] 121 Image Sensor

Claims

1. A spectral imaging device, comprising: a spectral section configured to disperse incident light in one axial direction for each wavelength; an image sensor configured to be able to set an exposure time or a gain in units of pixels, the image sensor detecting light of each wavelength dispersed in the spectral portion; and a control unit configured to set the exposure time or the gain of the image sensor in units of predetermined pixel areas, wherein The image sensor includes a pixel portion and a calculation portion that calculates a pixel value based on image data output from the pixel portion, and The control unit is configured to set the gain for calculating the pixel value in units of the predetermined pixel area, set the exposure time of the light receiving area having a strong spectral intensity and saturated beyond the dynamic range of detection to be relatively short or the gain to be relatively small, and set the exposure time of the light receiving area having a weak spectral intensity to be relatively long or the gain to be relatively large.

2. The spectral imaging device according to claim 1, wherein The control unit is configured to set the exposure time or the gain of the image sensor in units of lines perpendicular to the one axial direction.

3. The spectral imaging device according to claim 1, wherein The control unit includes an evaluation section that obtains an emission spectrum of the incident light based on an output of the image sensor, and a storage section that stores a plurality of reference component spectra and autofluorescence spectra, and The evaluation section is configured to calculate a component ratio of the emission spectrum so that a linear sum of a plurality of the reference component spectra and the self-luminous spectrum is obtained.

4. The spectral imaging device according to claim 3, wherein The evaluation section is configured to calibrate at least one of the emission spectrum or the component spectrum based on the exposure time or the gain set for each predetermined pixel area.

5. The spectral imaging device according to claim 4, wherein The evaluation section is configured to determine whether there are pixels whose pixel values ​​are saturated from the captured spectrum, and exclude the saturated pixels from calculation of the component ratio of the captured spectrum.

6. A fluorescence observation device, comprising: A stage capable of supporting fluorescently stained pathological specimens; an exciting section, irradiating the pathological specimen on the stage with line illumination; a spectroscopy section configured to disperse the fluorescence excited by the line illumination in one axial direction for each wavelength; an image sensor configured to be able to set an exposure time or a gain in units of pixels, the image sensor detecting light of each wavelength dispersed in the spectral portion; and a control unit configured to set the exposure time or the gain of the image sensor in units of predetermined pixel areas, wherein The image sensor includes a pixel portion and a calculation portion that calculates a pixel value based on image data output from the pixel portion, and The control unit is configured to set the gain for calculating the pixel value in units of the predetermined pixel area, set the exposure time of the light receiving area having a strong spectral intensity and saturated beyond the dynamic range of detection to be relatively short or the gain to be relatively small, and set the exposure time of the light receiving area having a weak spectral intensity to be relatively long or the gain to be relatively large.

7. The fluorescence observation device according to claim 6, further comprising: A display unit is configured to display a fluorescence spectrum based on the output of the image sensor.

8. The fluorescence observation device according to claim 7, wherein The display section has an operation area for receiving an input of an exposure time or a gain in units of the predetermined pixel area.

9. The fluorescence observation device according to claim 7, wherein The display unit has a display area for displaying a spectrum and a histogram based on the set exposure time or the set gain.

Images