Image acquisition device

Abstract

The image acquisition device includes a support portion that supports a sample, a light emitting portion that emits excitation light, a first light detection portion that detects fluorescence emitted from the sample, a first optical path switching portion that switches an optical path of the excitation light and an optical path of the fluorescence, and an optical filter portion that includes an optical filter that transmits the fluorescence. The first optical path switching unit is configured so as to switch the optical path of the excitation light from the first optical path to the second optical path, and to switch the optical path of the fluorescence from the second optical path to the third optical path. The optical filter has a transmission characteristic in which the transmittance varies in the fluorescence wavelength region. The optical filter unit is configured such that an optical filter is disposed on the third optical path.

Description

Technical Field

[0001] The present disclosure relates to an image acquisition device for acquiring a plurality of fluorescence images separated into respective plurality of wavelength regions. Background Art

[0002] An image acquisition device is known that obtains multiple fluorescence images of a sample by irradiating a sample such as pathological cells stained with multiple fluorescent pigments with excitation light, and performs demixing processing on the multiple multiple fluorescence images obtained to obtain multiple fluorescence images separated according to each of the multiple fluorescent pigments (for example, see Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application No. 2021-526220 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] To perform the aforementioned unmixing process, nonnegative matrix factorization (NFM) is sometimes used to obtain matrix data to be multiplied by multiple fluorescence images. In this case, if the image has a large number of pixels, obtaining the matrix data can take a considerable amount of time, potentially preventing efficient acquisition of multiple fluorescence images. Furthermore, if information about a particular fluorochrome is scarce, the accuracy of the matrix data decreases, potentially preventing high-precision acquisition of multiple fluorescence images.

[0008] An object of the present disclosure is to provide an image acquisition device capable of efficiently and accurately acquiring a plurality of fluorescence images separated into respective plurality of wavelength regions.

[0009] Technical means to solve the problem

[0010] An image acquisition device according to one aspect of the present disclosure is, [1] "an image acquisition device comprising: a support portion for supporting a sample; a light emitting portion for emitting excitation light; a first light detecting portion for detecting fluorescence emitted from the sample by irradiation with the excitation light; a first light path switching portion for switching the light path of the excitation light and the light path of the fluorescence; and an optical filter portion including an optical filter that transmits the fluorescence, wherein the light emitting portion is optically connected to the first light path switching portion via a first light path, the support portion is optically connected to the first light path switching portion via a second light path, and the first light detecting portion is optically connected to the first light path switching portion via a second light path. The three optical paths are optically connected to the first optical path switching portion, and the first optical path switching portion is configured to switch the optical path of the excitation light of multiple first wavelength regions that are different from each other from the first optical path to the second optical path, and to switch the optical path of the fluorescence of multiple second wavelength regions that are shifted from the multiple first wavelength regions from the second optical path to the third optical path, the optical filter has a transmission characteristic in which the transmittance changes in the fluorescence wavelength region including the multiple second wavelength regions, and the optical filter portion is configured to arrange the optical filter on the third optical path.

[0011] In the image acquisition device described in [1] above, fluorescence emitted from a sample can be detected via an optical filter having a transmittance characteristic whose transmittance varies within the fluorescence wavelength region. This allows efficient and high-precision acquisition of matrix data for performing demixing processing. Therefore, the image acquisition device described in [1] above can efficiently and high-precision acquire multiple fluorescence images separated into multiple wavelength regions.

[0012] An image acquisition device according to one aspect of the present disclosure may also be [2] "an image acquisition device according to the above-mentioned [1], wherein the first optical path switching unit includes a dichroic mirror, the dichroic mirror selectively reflecting the excitation light in the plurality of first wavelength regions and selectively transmitting the fluorescence in the plurality of second wavelength regions." According to the image acquisition device described in [2], the sample can be reliably irradiated with excitation light in each of the plurality of first wavelength regions. In addition, fluorescence can be reliably detected in the plurality of second wavelength regions.

[0013] An image acquisition device according to one aspect of the present disclosure may also be, [3] "the image acquisition device according to the above-mentioned [2], wherein the first optical path switching unit further includes a first multi-bandpass filter, the first multi-bandpass filter selectively transmitting the excitation light of the plurality of first wavelength regions incident on the dichroic mirror". According to the image acquisition device described in [3], the sample can be more reliably irradiated with excitation light in each of the plurality of first wavelength regions.

[0014] An image acquisition device according to one aspect of the present disclosure may also be [4] "an image acquisition device according to [2] or [3] above, wherein the first optical path switching unit further includes a second multi-bandpass filter that selectively transmits the fluorescence in the plurality of second wavelength regions emitted from the dichroic mirror." According to the image acquisition device described in [4], fluorescence can be more reliably detected in the plurality of second wavelength regions.

[0015] An image acquisition device according to one aspect of the present disclosure may be [5] "the image acquisition device according to any one of [1] to [4] above, wherein the optical filter has the transmission characteristic in which the transmittance varies linearly within the fluorescence wavelength region." According to the image acquisition device described in [5], matrix data for performing the demixing process can be acquired more efficiently and with higher accuracy.

[0016] An image acquisition device according to one aspect of the present disclosure may also be, [6] "an image acquisition device according to any one of the above [1] to [5], wherein the optical filter section includes a first optical filter and a second optical filter, the first optical filter being the optical filter having a first transmission characteristic as the transmission characteristic, the second optical filter having a second transmission characteristic different from the first transmission characteristic, and the optical filter section being configured to arrange each of the first optical filter and the second optical filter on the third optical path". According to the image acquisition device described in [6], by detecting fluorescence through each of the first optical filter and the second optical filter, matrix data for performing demixing processing can be acquired more efficiently and with higher accuracy.

[0017] An image acquisition device according to one aspect of the present disclosure may be [7] "the image acquisition device according to any one of [1] to [6] above, wherein the optical filter section is configured to remove the optical filter from the third optical path." According to the image acquisition device described in [7], by removing the optical filter from the third optical path after detecting the fluorescence used to obtain matrix data, fluorescence detection for obtaining multiple fluorescence images can be performed efficiently and with high precision.

[0018] An image acquisition device according to one aspect of the present disclosure may also be, [8] "an image acquisition device according to any one of [1] to [7] above, further comprising an imaging lens, the imaging lens being arranged on the third optical path to image the fluorescence on the first light detection unit, and the optical filter unit being configured to arrange the optical filter on the third optical path between the first optical path switching unit and the imaging lens". According to the image acquisition device described in [8], it is possible to prevent the fluorescence imaged by the imaging lens from being affected by the aberration of the optical filter.

[0019] An image acquisition device according to one aspect of the present disclosure may also be, [9] "an image acquisition device according to any one of the above [1] to [8], further comprising a fluorescence filter section, wherein the fluorescence filter section includes a light passing section and a plurality of single-bandpass filters, the light passing section allowing the fluorescence in the fluorescence wavelength region to pass through, the plurality of single-bandpass filters selectively allowing the fluorescence in each of the plurality of second wavelength regions to pass through, and the fluorescence filter section being configured such that the light passing section and each of the plurality of single-bandpass filters are arranged on the third optical path". According to the image acquisition device described in [9], by detecting fluorescence in a state where the light passing section is arranged on the third optical path, a plurality of fluorescence images can be acquired indirectly (i.e., by implementing a demixing process). In addition, by detecting fluorescence in a state where the plurality of single-bandpass filters are respectively arranged on the third optical path, a plurality of fluorescence images can be acquired directly.

[0020] An image acquisition device according to one aspect of the present disclosure may also be,

[10] "the image acquisition device according to the above-mentioned [9], further comprising an image processing unit, the image processing unit generating a first fluorescent image of the sample based on data output from the first light detection unit in a state where the light passing unit is arranged on the first optical path, generating a second fluorescent image of the sample based on data output from the first light detection unit in a state where the plurality of single-bandpass filters are respectively arranged on the first optical path, the image processing unit setting a shooting condition for generating the second fluorescent image based on the first fluorescent image". According to the image acquisition device described in

[10] , for example, by indirectly acquiring a plurality of fluorescent images as the first fluorescent image over the entire area of ​​the sample, the state of the entire area of ​​the sample can be efficiently grasped. In addition, by directly acquiring a plurality of fluorescent images as the second fluorescent image, for example, with respect to a specific area within the sample under the shooting condition set based on the first fluorescent image, the state of the specific area within the sample can be grasped with high precision.

[0021] An image acquisition device according to one aspect of the present disclosure may also be,

[11] "an image acquisition device according to any one of the above-mentioned [1] to [9], further comprising: an irradiation unit that irradiates the sample supported by the support unit with the excitation light at a position different from the position where the fluorescence is detected by the first light detection unit; a second light detection unit that detects the fluorescence emitted from the sample by the irradiation of the excitation light by the irradiation unit; and a second light path switching unit that is arranged on the first light path and switches the light path of the excitation light, the light emitting unit is optically connected to the second light path switching unit through a fourth light path that is a part of the first light path, the first light path switching unit is optically connected to the second light path switching unit through a fifth light path that is a part of the first light path, the irradiation unit is optically connected to the second light path switching unit through a sixth light path, and the second light path switching unit is configured to switch the light path of the excitation light from the fourth light path to each of the fifth light path and the sixth light path". According to the image acquisition device described in

[11] , by detecting fluorescence in the second light detection section using the excitation light emitted from the same light emitting section, it is possible to identify the presence range of the sample on the slide, for example.

[0022] An image acquisition device according to one aspect of the present disclosure may also be,

[12] "an image acquisition device according to the above-mentioned

[11] , wherein the second optical path switching unit includes a mirror, the mirror having a mirror surface arranged on the fourth optical path in a state inclined relative to the fourth optical path, and the second optical path switching unit is configured to rotate the mirror with the fourth optical path as the center line, and switch the state of the mirror to a state in which the fourth optical path and the fifth optical path intersect on the mirror surface and a state in which the fourth optical path and the sixth optical path intersect on the mirror surface". According to the image acquisition device described in

[12] , the optical path of the excitation light can be switched from the fourth optical path to each of the fifth optical path and the sixth optical path while saving space.

[0023] An image acquisition device according to one aspect of the present disclosure may also be,

[13] "an image acquisition device according to the above-mentioned

[11] or

[12] , wherein the image acquisition device further comprises an image processing unit, wherein the image processing unit generates a third fluorescent image of the sample based on the data output from the second light detection unit, and generates a fourth fluorescent image of the sample based on the data output from the first light detection unit, and the image processing unit sets a shooting condition for generating the fourth fluorescent image based on the third fluorescent image". According to the image acquisition device described in

[13] , for example, by acquiring a third fluorescent image for the entire area of ​​the carrier on which the sample is placed, the presence range of the sample on the carrier can be specified. In addition, by indirectly acquiring a plurality of fluorescent images as the fourth fluorescent image for the entire area of ​​the sample under the shooting conditions set based on the third fluorescent image, for example, the state of the entire area of ​​the sample can be efficiently grasped.

[0024] Effects of the Invention

[0025] According to the present disclosure, it is possible to provide an image acquisition device that can efficiently and accurately acquire a plurality of fluorescence images separated for each of a plurality of wavelength regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a structural diagram of an image acquisition device according to one embodiment.

[0027] Figure 2 yes Figure 1 The structural diagram of the sample supply unit is shown.

[0028] Figure 3 yes Figure 1 The structural diagram of the fluorescence macro image acquisition unit is shown.

[0029] Figure 4 yes Figure 1 The structural diagram of the bright field microscopic image acquisition unit is shown.

[0030] Figure 5 Is used to illustrate Figure 4 A diagram showing the scanning method in the bright field microscopic image acquisition section is shown.

[0031] Figure 6 yes Figure 1 The structural diagram of the fluorescence microscopic image acquisition unit is shown.

[0032] Figure 7 yes Figure 6 The structural diagram of the first optical path switching unit is shown.

[0033] Figure 8 Yes Figure 7 Graph showing transmission characteristics of the dichroic mirror, the first multi-bandpass filter, and the second multi-bandpass filter.

[0034] Figure 9 yes Figure 6 The structural diagram of the second optical path switching unit is shown.

[0035] Figure 10 yes Figure 6 Schematic diagram of the fluorescence microscopic image acquisition unit shown.

[0036] Figure 11 yes Figure 6 Schematic diagram of the fluorescence microscopic image acquisition unit shown.

[0037] Figure 12 Yes Figure 1 Flowchart of the image processing method in the image processing unit shown.

[0038] Figure 13 Is used to illustrate Figure 1 FIG. 1 is a diagram showing a method of specifying an imaging area in an image processing unit.

[0039] Figure 14 yes Figure 1 Schematic diagram of the image acquisition device shown.

[0040] Figure 15 Yes Figure 1 The block diagram of the hardware structure of the image processing unit is shown.

[0041] Figure 16 Yes Figure 1 A block diagram of the functional structure of the image processing unit shown.

[0042] Figure 17 Yes means through Figure 16 FIG. 1 shows an image of a pixel group clustered by the first clustering function of the clustering unit.

[0043] Figure 18 This is a graph showing the wavelength characteristics of the absorptivity of excitation light by a plurality of pigments contained in a sample.

[0044] Figure 19 It means by Figure 16 The graph shows the distribution of fluorescence wavelengths at the specific centroid of the cluster portion.

[0045] Figure 20 Yes means through Figure 16 FIG. 1 shows an image of a pixel group clustered by the second clustering function of the clustering unit.

[0046] Figure 21 It means by Figure 16 The diagram shows an image of matrix data and pigment matrix data regenerated by the statistical value calculation unit.

[0047] Figure 22 Graphs showing transmission characteristics of each of a first optical filter and a second optical filter included in an optical filter section according to a modified example.

[0048] Figure 23 Graphs showing transmission characteristics of each of a first optical filter and a second optical filter included in an optical filter section according to a modified example.

[0049] Figure 24 It is a structural diagram of the optical filter unit and the first light detection unit according to a modified example.

[0050] Figure 25 It is a structural diagram of the optical filter unit and the first light detection unit according to a modified example.

[0051] Figure 26 It is a structural diagram of the optical filter unit and the first light detection unit according to a modified example.

[0052] Figure 27 It is a structural diagram of the optical filter unit and the first light detection unit according to a modified example. DETAILED DESCRIPTION

[0053] Hereinafter, the embodiment of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in each figure, the same or corresponding parts are marked with the same reference numerals and repeated descriptions are omitted.

[0054] [Structure of Image Acquisition Device]

[0055] like Figure 1 As shown, the image acquisition device 1 includes a sample supply portion 2, a sample support portion 3, a fluorescence macro-image acquisition portion 4, a bright field micro-image acquisition portion 5, a fluorescence micro-image acquisition portion 6, an excitation light source 7, and an image processing device 8. The sample supply portion 2, the sample support portion 3, the fluorescence macro-image acquisition portion 4, the bright field micro-image acquisition portion 5, and the fluorescence micro-image acquisition portion 6 are configured in the same housing. In the image acquisition device 1, a sample S (for example, pathological cells stained with a plurality of fluorescent pigments, etc.) is processed while being placed on a carrier plate G, and various images of the sample S are acquired. Hereinafter, unless otherwise specified, the "carrier plate G carrying the sample S" will be referred to as the "carrier plate G." In addition, a predetermined horizontal direction will be referred to as the X-axis direction, a horizontal direction perpendicular to the X-axis direction will be referred to as the Y-axis direction, and a vertical direction will be referred to as the Z-axis direction.

[0056] like Figure 1 and Figure 2As shown, the sample supply unit 2 includes a cassette holder 21 and a conveyor table 22. A plurality of cassettes 21a are provided on the cassette holder 21. Each cassette 21a houses a plurality of carriers G. A gripping portion 22a is provided on the conveyor table 22. The gripping portion 22a grips the carriers G. The conveyor table 22 moves the gripping portion 22a in the X-axis direction, the Y-axis direction, and the Z-axis direction. In the image acquisition device 1, the carriers G (i.e., the sample S) are conveyed between each cassette 21a and the sample support unit 3 by moving the gripping portion 22a while the carriers G are gripped by the gripping portion 22a.

[0057] like Figure 1 As shown, the sample support 3 includes a support 31, a switching stage 32, and a moving stage 33. The support 31 is mounted on the upper side of the switching stage 32. The support 31 supports the slide G by holding it. In other words, the support 31 supports the sample S. The switching stage 32 is mounted on the upper side of the moving stage 33. The switching stage 32 moves the support 31 between the fluorescence macro-image acquisition section 4 and the bright-field micro-image acquisition section 5 (which also corresponds to the fluorescence micro-image acquisition section 6). The moving stage 33 moves the switching stage 32 in each of the X-axis direction and the Y-axis direction in the bright-field micro-image acquisition section 5 (which also corresponds to the fluorescence micro-image acquisition section 6).

[0058] In the image acquisition apparatus 1, the slide G (i.e., the specimen S) is transported between the fluorescence macro image acquisition section 4 and the brightfield micro image acquisition section 5 (also corresponding to the fluorescence micro image acquisition section 6) by moving the support section 31 by the switching stage 32 while the slide G is supported by the support section 31. Furthermore, in the image acquisition apparatus 1, the switching stage 32 is moved by the moving stage 33 while the slide G is supported by the support section 31, thereby scanning the field of view of the objective lens 55a (described later) with respect to the specimen S.

[0059] like Figure 1 and Figure 3 As shown, the fluorescence macro image acquisition unit 4 includes a bright field light source 41, a shutter 42, a plurality of dark field light sources 43, a light source for barcode shooting 44, an irradiation unit 45, an optical fiber 46, a light detection unit (second light detection unit) 47, an imaging lens 48, and a filter switching mechanism 49. The bright field light source 41 is arranged below the switching table 32. The bright field light source 41 emits light when a bright field image is acquired. The light emitted from the bright field light source 41 is irradiated onto the sample S through an opening formed in the switching table 32, an opening formed in the support portion 31, and a carrier plate G. The bright field light source 41 is, for example, a white light source such as an LED that emits white light through a diffuser. The shutter 42 is arranged between the bright field light source 41 and the switching table 32 except when a bright field image is acquired.

[0060] A plurality of dark-field light sources 43 are arranged below the switching table 32. The plurality of dark-field light sources 43 emit light when a dark-field image is acquired. The light emitted from the dark-field light source 43 is irradiated onto the sample S from obliquely above. Each dark-field light source 43 is, for example, a white light source such as an LED that emits white light. The barcode shooting light source 44 emits light toward the carrier G when acquiring information from a barcode attached to the carrier G. The irradiation unit 45 irradiates the sample S supported by the support unit 31 with excitation light in the fluorescence macro image acquisition unit 4 (a position different from the position in which fluorescence is detected by the light detection unit 61 of the fluorescence micro image acquisition unit 6). The irradiation unit 45 irradiates the carrier G with excitation light when a fluorescence macro image is acquired. The excitation light is emitted from the excitation light source 7 and is guided to the irradiation unit 45 by the optical fiber 46.

[0061] The light detection unit 47 is arranged above the switching table 32. The light detection unit 47 detects fluorescence emitted from the sample S by the irradiation of the excitation light by the irradiation unit 45. In addition, the light detection unit 47 detects light emitted from the bright field light source 41 and transmitted through the sample S, light emitted from the plurality of dark field light sources 43 and reflected by the sample S, and light emitted from the barcode imaging light source 44 and reflected by the barcode on the carrier plate G. The light detection unit 47 is, for example, an area image sensor that captures an image of the entire carrier plate G. In this case, the light detection unit 47 can be a color area image sensor or a black and white area image sensor.

[0062] The imaging lens 48 is mounted on the lower side of the light detection section 47. The imaging lens 48 forms an image of the entire carrier plate G on the light detection section 47. The filter switching mechanism 49 is arranged between the switching table 32 and the imaging lens 48. The filter switching mechanism 49 includes a light passing portion and a plurality of multi-bandpass filters. The light passing portion is, for example, an opening formed in the filter switching mechanism 49. When a bright field image is acquired and when a dark field image is acquired, the light passing portion of the filter switching mechanism 49 is arranged on the optical axis of the light detection section 47. When a fluorescent macroscopic image is acquired, each multi-bandpass filter of the filter switching mechanism 49 is arranged on the optical axis of the light detection section 47. As an example, the filter switching mechanism 49 is a rotation-type switching mechanism (for example, a filter wheel, etc.) or a direct-acting type switching mechanism (for example, a slider, etc.).

[0063] like Figure 1 and Figure 4As shown, the brightfield microscopic image acquisition unit 5 includes a brightfield light source 51, a brightfield optical system 52, a shutter 53, a light detection unit 54, an objective lens unit 55, and an imaging lens 56. The brightfield light source 51 is arranged below the switching stage 32 and the moving stage 33. The brightfield light source 51 emits light when a brightfield microscopic image is acquired. The light emitted from the brightfield light source 51 is irradiated onto the sample S via the brightfield optical system 52, the opening formed in the moving stage 33, the opening formed in the switching stage 32, the opening formed in the support portion 31, and the carrier plate G. The brightfield light source 51 is, for example, a white light source such as an LED that emits white light. The brightfield optical system 52 guides the light emitted from the brightfield light source 51 and irradiates the entire carrier plate G. The shutter 53 is arranged between the brightfield light source 51 and the brightfield optical system 52 except when a brightfield microscopic image is acquired.

[0064] The objective lens unit 55 is composed of a plurality of objective lenses 55a, a movable stage 55b and a switching stage 55c. The plurality of objective lenses 55a are mounted on the lower side of the movable stage 55b. The movable stage 55b is mounted on the lower side of the switching stage 55c. In the bright field microscopic image acquisition unit 5, the switching stage 55c is rotated so that any objective lens 55a is arranged on the optical axis of the light detection unit 54. In addition, in the bright field microscopic image acquisition unit 5, the objective lens 55a arranged on the optical axis of the light detection unit 54 is moved in the Z-axis direction by the movable stage 55b, thereby aligning the focus of the objective lens 55a arranged on the optical axis of the light detection unit 54 with the sample S. The magnification of each objective lens 55a is, for example, 10 times, 20 times, or 40 times. As an example, the switching stage 55c is a rotating type stage or a linear type stage.

[0065] The light detection section 54 detects light emitted from the bright field light source 51 and passing through the sample S. The light detection section 54 captures the image of the field of view of the objective lens 55a arranged on the optical axis of the light detection section 54. The light detection section 54 is, for example, a color area image sensor. The light detection section 54 can be a three-plate area image sensor or a linear image sensor. The imaging lens 56 forms the image of the field of view of the objective lens 55a arranged on the optical axis of the light detection section 54 on the light detection section 54. The imaging lens 56 is, for example, a cylindrical lens. In addition, when a bright field microscopic image is obtained, the optical path switching section 62 and the mirror 64 described later are removed from the optical axis of the light detection section 54, and the light passing portion (opening) of the fluorescence filter section 68 described later is arranged on the optical axis of the light detection section 54.

[0066] The scanning method in the bright field microscopic image acquisition unit 5 is as follows. Figure 5As shown, in the specified shooting area R, the field of view of the objective lens 55a is moved in the Y-axis direction from the area Ra of the first row and the first column to the area Ra of the first row and the mth column (m is an integer greater than or equal to 2). At this time, when the field of view of the objective lens 55a reaches each area Ra of the first row, the bright field light source 51 is flashed, and the image of each area Ra of the first row is sent from the light detection unit 54 to the image processing device 8. Next, the field of view of the objective lens 55a is moved in the Y-axis direction from the area Ra of the second row and the mth column to the area Ra of the second row and the first column. Similarly, the image of each area Ra of the second row is sent from the light detection unit 54 to the image processing device 8. Hereinafter, the movement of the field of view of the objective lens 55a, and the shooting and sending of the image of each area Ra are similarly implemented until the nth row (n is an integer greater than or equal to 2). In the image processing device 8, the images of all the areas Ra constituting the shooting area R are joined to generate a bright field microscopic image of the shooting area R.

[0067] In the above-mentioned scanning method, the movement of the field of view of the objective lens 55a is implemented as follows: while the carrier plate G is supported by the support portion 31, the switching stage 32 is moved by the moving stage 33. In the above-mentioned scanning method, the focus of the objective lens 55a relative to the sample S is aligned, for example, by a focus diagram method or a dynamic focus method. In addition, instead of emitting a flash light from the bright field light source 51, the shutter 53 can be opened and closed while the bright field light source 51 is continuously illuminated. In addition, when the field of view of the objective lens 55a reaches each area Ra, the movement of the field of view of the objective lens 55a can be stopped, and when the movement of the field of view of the objective lens 55a is stopped, the image of each area Ra can be captured by the light detection unit 54 (stitching scanning). In addition, a linear image sensor can be used as the light detection unit 54 to continuously implement the movement of the field of view of the objective lens 55a and the capture by the light detection unit 54. In addition, the field of view of the objective lens 55a can be moved in the same direction in all rows from the first row to the nth row. The matters related to the above scanning method also apply to the fluorescence microscopic image acquisition unit 6. However, as the scanning method in the fluorescence microscopic image acquisition unit 6, the above-mentioned stitching scanning is preferable.

[0068] like Figure 1 and Figure 6As shown, the fluorescence microscopic image acquisition section 6 includes a light detection section (first light detection section) 61, an optical path switching section (first optical path switching section) 62, an optical path switching section (second optical path switching section) 63, a mirror 64, a mirror 65, an imaging lens 66, an optical filter section 67 and a fluorescence filter section 68. The light emitting section 72 of the excitation light source 7 is optically connected to the optical path switching section 62 through the first optical path P1. The support section 31 is optically connected to the optical path switching section 62 through the second optical path P2. The light detection section 61 is optically connected to the optical path switching section 62 through the third optical path P3. That is, the first optical path P1 extends between the light emitting section 72 and the optical path switching section 62, the second optical path P2 extends between the support section 31 and the optical path switching section 62, and the third optical path P3 extends between the light detection section 61 and the optical path switching section 62.

[0069] As an example, in the fluorescence microscopic image acquisition section 6, the first optical path P1 extends along the Y-axis direction between the light emitting section 72 and the optical path switching section 62. The second optical path P2 extends along the Z-axis direction between the optical path switching section 62 and the support section 31. The third optical path P3 extends along the Z-axis direction between the optical path switching section 62 and the mirror 64, extends along the Y-axis direction between the mirror 64 and the mirror 65, and extends along the Z-axis direction between the mirror 65 and the light detection section 61.

[0070] The excitation light emitted from the light emitting portion 72 is incident on the optical path switching portion 62 along the first optical path P1. The excitation light incident on the optical path switching portion 62 is reflected by the optical path switching portion 62 and travels along the second optical path P2, and irradiates the sample S supported by the support portion 31. At this time, the excitation light passes through the objective lens 55a of the objective lens unit 55 arranged on the second optical path P2. The fluorescence emitted from the sample S by the irradiation of the excitation light is incident on the optical path switching portion 62 along the second optical path P2. At this time, the fluorescence passes through the objective lens 55a of the objective lens unit 55 arranged on the second optical path P2. The fluorescence incident on the optical path switching portion 62 travels along the third optical path P3 through the optical path switching portion 62 and is detected by the light detection portion 61.

[0071] As described above, the light detection unit 61 detects fluorescence emitted from the sample S by irradiation with the excitation light. The light detection unit 61 is, for example, a black-and-white area image sensor. The light detection unit 61 may be a color area image sensor or a color separation sensor (a multispectral sensor, a hyperspectral sensor, etc.). Using a color separation sensor as the light detection unit 61 facilitates handling of a large number of pigments (e.g., ten or more). The light detection unit 61 may also be a linear image sensor.

[0072] As described above, the optical path switching unit 62 switches the optical path of the excitation light and the optical path of the fluorescence. The optical path switching unit 62 is configured to switch the optical path of the excitation light in a plurality of first wavelength regions that are different from each other from the first optical path P1 to the second optical path P2, and to switch the optical path of the fluorescence in a plurality of second wavelength regions that are shifted from the plurality of first wavelength regions from the second optical path P2 to the third optical path P3. A plurality of first wavelength regions that are different from each other refer to a plurality of wavelength regions that are separated from each other. A plurality of second wavelength regions that are shifted from a plurality of first wavelength regions refer to a plurality of wavelength regions that do not substantially overlap with the plurality of first wavelength regions and are separated from each other. Each of the plurality of first wavelength regions and each of the plurality of second wavelength regions are arranged alternately.

[0073] The specific structure of the optical path switching unit 62 is as follows. Figure 7 As shown, the optical path switching portion 62 is formed into a cube shape by combining a pair of triangular prism-shaped light-transmitting components 621. A dichroic mirror 622 is formed on the boundary surface of the pair of light-transmitting components 621. The boundary surface of the pair of light-transmitting components 621 is a surface perpendicular to the surface containing the first optical path P1, the second optical path P2, and the third optical path P3, and is a surface inclined relative to the optical paths of the first optical path P1, the second optical path P2, and the third optical path P3. The dichroic mirror 622 selectively reflects the excitation light of multiple first wavelength regions and selectively transmits the fluorescence of multiple second wavelength regions. That is, the dichroic mirror 622 essentially has the function of reflecting only the excitation light of multiple first wavelength regions and the function of transmitting only the fluorescence of multiple second wavelength regions. The dichroic mirror 622 is, for example, a dielectric multilayer film.

[0074] A multi-bandpass filter (first multi-bandpass filter) 623 is formed on a surface of a light-transmitting member 621 perpendicular to the first optical path P1. Multi-bandpass filter 623 selectively transmits excitation light in multiple first wavelength regions incident on dichroic mirror 622. In other words, multi-bandpass filter 623 essentially transmits only excitation light in multiple first wavelength regions. Multi-bandpass filter 623 is, for example, a dielectric multilayer film.

[0075] A multi-bandpass filter (second multi-bandpass filter) 624 is formed on the surface of the other light-transmitting member 621, perpendicular to the third optical path P3. Multi-bandpass filter 624 selectively transmits fluorescence in multiple second wavelength regions emitted from dichroic mirror 622. In other words, multi-bandpass filter 624 essentially transmits only fluorescence in multiple second wavelength regions. Multi-bandpass filter 624 is, for example, a dielectric multilayer film.

[0076] Figure 8 (a) is a diagram showing the transmission characteristics of the dichroic mirror 622 (transmission characteristics indicated by a dotted line) and the transmission characteristics of the multi-bandpass filter 624 (transmission characteristics indicated by a solid line). Figure 8(b) is a diagram showing the transmission characteristics of the multi-bandpass filter 623. Figure 8 As shown in (a) and (b), the selective transmission of fluorescence in multiple second wavelength regions can be reliably achieved by using the dichroic mirror 622 and the multi-bandpass filter 624, and the selective transmission of excitation light in multiple first wavelength regions can be reliably achieved by using the dichroic mirror 622 and the multi-bandpass filter 623.

[0077] Furthermore, the fluorescence microscopic image acquisition section 6 includes a plurality of optical path switching sections 62 having different specifications (such as the number of wavelength regions, upper and lower limits of the wavelength regions), and is configured such that any one of the optical path switching sections 62 is disposed at the intersection of the first optical path P1, the second optical path P2, and the third optical path P3. Furthermore, the fluorescence microscopic image acquisition section 6 is configured such that each optical path switching section 62 can be attached and detached (replaced).

[0078] like Figure 1 and Figure 6 As shown, the optical path switching section 63 is arranged on the first optical path P1. The optical path switching section 63 switches the optical path of the excitation light emitted from the light emitting section 72. The light emitting section 72 is optically connected to the optical path switching section 63 via a fourth optical path P4, which is a part of the first optical path P1. The optical path switching section 62 is optically connected to the optical path switching section 63 via a fifth optical path P5, which is a part of the first optical path P1. The irradiation section 45 of the fluorescence macro image acquisition section 4 is optically connected to the optical path switching section 63 via a sixth optical path P6. That is, the fourth optical path P4 extends between the light emitting section 72 and the optical path switching section 63, the fifth optical path P5 extends between the optical path switching section 63 and the optical path switching section 62, and the sixth optical path P6 extends between the optical path switching section 63 and the irradiation section 45.

[0079] The optical path switching unit 63 is configured to switch the optical path of the excitation light from the fourth optical path P4 to each of the fifth optical path P5 and the sixth optical path P6. More specifically, Figure 9 As shown in (a) and (b), the optical path switching unit 63 includes a mirror 631 having a mirror surface 631a. The mirror surface 631a is arranged on the fourth optical path P4 in a state of being inclined relative to the fourth optical path P4. The optical path switching unit 63 is configured to rotate the mirror 631 with the fourth optical path P4 as the center line. By rotating the mirror 631, the optical path switching unit 63 switches the state of the mirror 631 to a state in which the fourth optical path P4 and the fifth optical path P5 intersect on the mirror surface 631a ( Figure 9 (a) and the state in which the fourth optical path P4 and the sixth optical path P6 intersect on the mirror surface 631a ( Figure 9In addition, the optical path switching unit 63 can also be configured to switch the configuration of the mirror on the first optical path P1 and the removal of the mirror from the first optical path P1 by, for example, a direct-acting mechanism, thereby switching the optical path of the excitation light from the fourth optical path P4 to each of the fifth optical path P5 and the sixth optical path P6.

[0080] like Figure 1 and Figure 6 As shown, the mirror 64 is arranged on the third optical path P3. The mirror 65 is arranged on the third optical path P3 between the mirror 64 and the light detection unit 61. The mirror 64 reflects the fluorescence of the multiple second wavelength regions emitted from the optical path switching unit 62 to the mirror 65 side. The mirror 65 reflects the fluorescence of the multiple second wavelength regions reflected by the mirror 64 to the light detection unit 61 side and makes it incident on the light detection unit 61. The imaging lens 66 is arranged on the third optical path P3 between the mirror 64 and the mirror 65. The imaging lens 66 images the fluorescence of the multiple second wavelength regions on the light detection unit 61. In other words, the imaging lens 66 images the image of the field of view of the objective lens 55a arranged on the first optical path P1 on the light detection unit 61. The imaging lens 66 is, for example, a cylindrical lens.

[0081] The optical filter unit 67 includes an optical filter 67a. The optical filter 67a transmits multiple fluorescence light in the second wavelength range. The optical filter unit 67 is configured to place the optical filter 67a on the third optical path P3 between the optical path switching unit 62 and the imaging lens 66. On the other hand, the optical filter unit 67 is configured to remove the optical filter 67a from the third optical path P3 between the optical path switching unit 62 and the imaging lens 66. The optical filter unit 67 places the optical filter 67a on the third optical path P3 when acquiring "matrix data for performing demixing processing" (described later). When acquiring a fluorescence microscopic image, the optical filter 67a is removed from the third optical path P3.

[0082] Optical filter 67a has a transmission characteristic in which transmittance varies within a fluorescence wavelength region (hereinafter referred to as the "fluorescence wavelength region") that includes multiple second wavelength regions. Optical filter 67a has a transmission characteristic in which wavelength and transmittance have a one-to-one correspondence within the fluorescence wavelength region. Optical filter 67a has a transmission characteristic in which transmittance varies linearly within the fluorescence wavelength region.

[0083] The fluorescence filter unit 68 includes a switching mechanism body 681, multiple single-bandpass filters 682, and a light-passing portion 683. Each single-bandpass filter 682 is positioned within a plurality of openings formed in the switching mechanism body 681. Each single-bandpass filter 682 selectively transmits fluorescence in each of the multiple second wavelength regions. In other words, each single-bandpass filter 682 essentially transmits fluorescence in only one second wavelength region. The light-passing portion 683 is an opening formed in the switching mechanism body 681 that transmits fluorescence in the fluorescence wavelength region. The light-passing portion 683 may also be a light-transmitting component having a transmission characteristic that allows fluorescence in the fluorescence wavelength region to pass, and is positioned within the opening formed in the switching mechanism body 681. The fluorescence filter unit 68 is configured such that, by switching the switching mechanism body 681, the light-passing portion 683 and each of the multiple single-bandpass filters 682 are positioned on the third optical path P3 (the third optical path P3 between the optical path switching unit 62 and the mirror 64). As an example, the switching mechanism main body 681 is a rotation type switching mechanism (for example, a filter wheel, etc.) or a direct-acting type switching mechanism (for example, a slider, etc.).

[0084] like Figure 1 As shown, the excitation light source 7 includes a light output unit 71, a light emitting unit 72, and an optical fiber 73. The light output unit 71 outputs excitation light in each of a plurality of first wavelength regions. The light emitting unit 72 causes the excitation light output from the light output unit 71 and guided by the optical fiber 73 to be emitted along the first optical path P1. The image processing device 8 includes an image processing unit 81 and a display unit 82. The image processing unit 81 is a computer that performs various processes described later. The display unit 82 is a display that displays various images of the sample S.

[0085] [Photographed in Sedat configuration and Pinkel configuration]

[0086] In the fluorescence microscopic image acquisition unit 6, the shooting under the Sedat configuration and the shooting under the Pinkel configuration are implemented. The shooting under the Sedat configuration is as follows. First, Figure 10 As shown, a single bandpass filter 682 is arranged on the optical path between the optical path switching unit 62 and the light detection unit 61. The single bandpass filter 682 selectively transmits fluorescence in a second wavelength range corresponding to an excitation light in a first wavelength range. In this state, an excitation light in a first wavelength range is emitted from the excitation light source 7 ( Figure 10 (arrows in the figure, dashed lines), an excitation light in the first wavelength region is reflected by the optical path switching unit 62 and irradiated onto the sample S. Then, the fluorescence ( Figure 10 , solid line arrows) passes through a single bandpass filter 682 via the optical path switching unit 62, whereby only fluorescence in a second wavelength region enters the light detection unit 61 and is detected by the light detection unit 61.

[0087] Next, another single-bandpass filter 682 is arranged on the optical path between the optical path switching section 62 and the light detection section 61. This other single-bandpass filter 682 selectively transmits fluorescence in the other second wavelength region corresponding to the excitation light in the other first wavelength region. In this state, excitation light in the other first wavelength region is emitted from the excitation light source 7, reflected by the optical path switching section 62, and irradiated onto the specimen S. Fluorescence emitted from the specimen S by the irradiation of the other excitation light then passes through the other single-bandpass filter 682 via the optical path switching section 62. As a result, only the fluorescence in the other second wavelength region is incident on the light detection section 61 and detected by the light detection section 61. Subsequently, the fluorescence in each of the multiple second wavelength regions corresponding to the excitation light in each of the multiple first wavelength regions is similarly detected, and the image processing section 81 directly generates multiple fluorescence images separated for each of the multiple wavelength regions.

[0088] The shooting in the Pinkel configuration is as follows. First, as Figure 11 As shown, a light passing portion 683 as an opening is arranged on the optical path between the optical path switching portion 62 and the light detecting portion 61. In this state, an excitation light in a first wavelength region ( Figure 11 (arrows in the figure, dashed lines), an excitation light in the first wavelength region is reflected by the optical path switching unit 62 and irradiated onto the sample S. Then, the fluorescence ( Figure 11 , solid arrow) passes through the light passing portion 683 via the optical path switching portion 62, whereby fluorescence containing fluorescence in a second wavelength region corresponding to excitation light in a first wavelength region enters the light detection portion 61 and is detected by the light detection portion 61.

[0089] Next, with the light-passing portion 683 disposed on the optical path between the optical path switching section 62 and the light detection section 61, excitation light in another first wavelength region is emitted from the excitation light source 7. The excitation light in another first wavelength region is reflected by the optical path switching section 62 and irradiated onto the specimen S. Fluorescence emitted from the specimen S by irradiation with the other excitation light then passes through the light-passing portion 683 via the optical path switching section 62. Consequently, fluorescence including fluorescence in another second wavelength region corresponding to the excitation light in another first wavelength region enters the light detection section 61 and is detected by the light detection section 61. Subsequently, detection of fluorescence including fluorescence in each of the multiple second wavelength regions corresponding to the excitation light in each of the multiple first wavelength regions is similarly performed, and multiple fluorescence images separated for each of the multiple wavelength regions are indirectly generated by the image processing section 81 (i.e., by performing the demixing process described later).

[0090] [Functions of the image processing unit]

[0091] like Figure 12 As shown, the image processing unit 81 performs photographing on the fluorescence macro image acquisition unit 4 (step S01). Next, the image processing unit 81 generates a fluorescence macro image (third fluorescence image) of the sample S based on the data output from the light detection unit 47 of the fluorescence macro image acquisition unit 4 (step S02). Here, a fluorescence macro image is generated for the entire area of ​​the carrier plate G on which the sample S is placed. Next, the image processing unit 81 sets the photographing conditions (photographing conditions for generating the fourth fluorescence image) under the Pinkel configuration performed by the fluorescence micro image acquisition unit 6 based on the fluorescence macro image of the carrier plate G (step S03). As the photographing conditions under the Pinkel configuration, Figure 13 As shown, the range of the sample S on the slide G is set as the scanning range R1. In addition, as imaging conditions in the Pinkel arrangement, multiple (for example, 9 points) focus measurement positions, exposure time, etc. within the scanning range R1 are set.

[0092] Then, if Figure 12 As shown, the image processing unit 81 performs shooting under the Pinkel configuration on the fluorescence microscopic image acquisition unit 6 (step S04). Then, the image processing unit 81 generates a fluorescence microscopic image (fourth fluorescence image, first fluorescence image) of the sample S under the Pinkel configuration based on the data output from the light detection unit 61 of the fluorescence microscopic image acquisition unit 6 (step S05). Here, for the entire area of ​​the sample S, a plurality of fluorescence microscopic images separated according to each of a plurality of wavelength regions are indirectly generated. Then, the image processing unit 81 sets the shooting conditions (shooting conditions for generating the second fluorescence image) under the Sedat configuration performed by the fluorescence microscopic image acquisition unit 6 based on the fluorescence microscopic image of the sample S under the Pinkel configuration (step S06). As the shooting conditions under the Sedat configuration, as Figure 13 As shown, a specific area within the sample S is set as the scanning range R2.

[0093] Then, if Figure 12 As shown, the image processing unit 81 performs SEDAT imaging on the fluorescence microscopic image acquisition unit 6 (step S07). Next, the image processing unit 81 generates a SEDAT-configured fluorescence microscopic image (second fluorescence image) of the sample S (step S08) based on the data output from the light detection unit 61 of the fluorescence microscopic image acquisition unit 6. Here, multiple fluorescence microscopic images separated into multiple wavelength regions are directly generated for a specific region within the sample S.

[0094] [Demixing processing with Pinkel configuration]

[0095] Figure 14 : is a perspective view showing the structure of the image acquisition device 1. Figure 14In FIG, the dotted line with an arrow indicates the optical path of the excitation light, and the solid line with an arrow indicates the optical path of the fluorescence. The image acquisition device 1 is configured to include an excitation light source 7, a multi-bandpass filter 623, a dichroic mirror 622, a multi-bandpass filter 624, an optical filter 67a, a light detection unit 61a, and a light detection unit 61b.

[0096] The excitation light source 7 is a light source that can switch between multiple bands (wavelength distributions) of excitation light and irradiate, and is, for example, an LED (Light Emitting Diode) light source, a light source composed of multiple monochromatic laser light sources, or a light source that combines a white light source and a wavelength selection optical element. The multi-bandpass filter 623 is a multi-bandpass filter that is arranged on the optical path of the excitation light of the excitation light source 7 and has the property of transmitting light of a predetermined multiple bands. The transmission band of the multi-bandpass filter 623 is set according to the multiple bands of excitation light that can be used. The dichroic mirror 622 is an optical component that is arranged between the multi-bandpass filter 623 and the sample S and has the property of reflecting the excitation light toward the sample S and transmitting the corresponding fluorescence emitted from the sample S. The multi-bandpass filter 624 is a multi-bandpass filter that is arranged on the optical path of the fluorescence transmitted by the dichroic mirror 622 and has the property of transmitting light of a predetermined multiple bands. The transmission band of the multi-bandpass filter 624 is set according to the band of fluorescence generated in the pigment of the sample S that can be included in the observation object.

[0097] The optical filter 67a is an optical system that is disposed on the optical path of the fluorescence transmitted by the multi-bandpass filter 624 and is used to obtain wavelength information of the fluorescence. Specifically, the optical filter 67a separates the fluorescence from the sample S into two optical paths with different wavelength characteristics. For example, a dichroic mirror having a wavelength characteristic in which the transmittance increases linearly with increasing wavelength can be used as the optical filter 67a. The optical filter 67a using such a dichroic mirror separates the fluorescence with different wavelength characteristics, reflecting a portion of the fluorescence with a wavelength characteristic in which the reflectance decreases with increasing wavelength, and transmitting a portion of the fluorescence with a wavelength characteristic in which the transmittance increases with increasing wavelength. A support mechanism (not shown) is provided on the optical filter 67a, which supports the optical filter 67a in a removable manner on the optical path of the fluorescence from the multi-bandpass filter 624.

[0098] The light detection unit 61a is an imaging device that captures a two-dimensional image composed of N pixels (N is an integer greater than or equal to 2, for example, 2048×2048). When the optical filter 67a is supported in the optical path of the fluorescence, the light detection unit 61a captures one component of the fluorescence separated by the optical filter 67a, thereby obtaining a one-sided separated fluorescence image. Furthermore, when the optical filter 67a is removed from the optical path of the fluorescence, the light detection unit 61a captures the fluorescence after it has passed through the multi-bandpass filter 624, thereby obtaining a fluorescence image. The light detection unit 61a outputs the obtained separated fluorescence image or fluorescence image to the image processing unit 81 using communication or a recording medium. The light detection unit 61b is an imaging device that captures a two-dimensional image composed of the same N pixels as the light detection unit 61a. When the optical filter 67a is supported in the optical path of the fluorescence, the light detection unit 61b captures the other component of the fluorescence separated by the optical filter 67a, thereby obtaining a second-sided separated fluorescence image. The light detection unit 61b outputs the obtained separated fluorescence image to the image processing unit 81 using communication or a recording medium.

[0099] Alternatively, the fluorescence image can be obtained by combining one separated fluorescence image obtained by the light detection unit 61a and the other separated fluorescence image obtained by the light detection unit 61b by the image processing unit 81. In this case, the support mechanism in the optical filter 67a can be eliminated.

[0100] Next, refer to Figure 15 and Figure 16 The configuration of the image processing unit 81 will be described. Figure 15 is a block diagram showing an example of the hardware configuration of the image processing unit 81. Figure 16 It is a block diagram showing the functional structure of the image processing unit 81.

[0101] like Figure 15 As shown, the image processing unit 81 is a computer that physically includes a CPU (Central Processing Unit) 101 as a processor, a RAM (Random Access Memory) 102 or a ROM (Read Only Memory) 103 as a recording medium, a communication module 104, and an input / output module 106, etc., all of which are electrically connected. Furthermore, the image processing unit 81 may include a display, keyboard, mouse, touch panel display, etc. as input / output devices, and may also include a data recording device such as a hard disk drive or semiconductor memory. Furthermore, the image processing unit 81 may be composed of multiple computers.

[0102] like Figure 16As shown, the image processing unit 81 includes an image acquisition unit 201 , a wavelength information acquisition unit 202 , a clustering unit 203 , a statistic calculation unit 204 , and an image generation unit 205 as functional components. Figure 16 Each functional unit of the image processing unit 81 shown is realized by reading a program (pigment image acquisition program of the embodiment) into the hardware such as the CPU 101 and the RAM 102, and operating the communication module 104 and the input / output module 106 under the control of the CPU 101, and reading and writing data in the RAM 102. The CPU 101 of the image processing unit 81 executes the computer program to Figure 16 Each functional part of the image processing unit 81 plays a role, and performs the processing corresponding to the pigment image acquisition method described below in sequence. In addition, CPU 101 can be a single hardware, or a component installed in a programmable logic such as FPGA as a soft processor. RAM and ROM can also be a single hardware, or a component built into a programmable logic such as FPGA. The various data required for executing this computer program, and the various data generated by executing this computer program are all stored in storage media such as built-in memories such as ROM 103, RAM 102, or hard disk drives. Below, the functions of the functional components of image processing unit 81 are described in detail.

[0103] The image acquisition unit 201 acquires C pre-specified (C is an integer greater than or equal to 2) fluorescence images of the specimen S from the image acquisition device 1. These C fluorescence images are generated by irradiating the specimen S with excitation light of C wavelength bands, with the optical filter 67a removed from the fluorescence optical path, and capturing the corresponding fluorescence generated from the specimen S. The number C of fluorescence images to be acquired (the number C of wavelength bands of excitation light irradiated on the specimen S) is pre-specified to be greater than the maximum number of pigments that can be contained in the specimen S.

[0104] Furthermore, the image acquisition unit 201 acquires a pre-specified set of C separated fluorescence images of the specimen S from the image acquisition device 1. These C sets of separated fluorescence images are a set of N pixels generated by irradiating the specimen S with excitation light of C wavelength bands while the optical filter 67a is supported on the optical path of the fluorescence, separating the fluorescence generated from the specimen S into two components, and capturing the images.

[0105] The wavelength information acquisition unit 202 estimates the centroid fluorescence wavelength, representing the centroid of the fluorescence wavelength distribution, by calculating the ratio of the fluorescence intensity (brightness value) of one separated fluorescence image to the fluorescence intensity of the other separated fluorescence image for each of the separated fluorescence images in group C. At this point, the wavelength information acquisition unit 202 calculates the average fluorescence intensity of one separated fluorescence image and the average fluorescence intensity of the other separated fluorescence image for the pixel groups clustered by the clustering unit 203 (described below), and calculates the ratio of these average values. The wavelength information acquisition unit 202 acquires the estimated centroid fluorescence wavelength as wavelength information related to the fluorescence wavelength.

[0106] The clustering unit 203 performs clustering on the N pixels constituting the C fluorescence images acquired by the image acquisition unit 201 and the wavelength information acquired by the wavelength information acquisition unit 202. Prior to the clustering process, the clustering unit 203 generates matrix data Y in which the fluorescence intensity values ​​of the N pixels constituting the C fluorescence images are arranged in parallel in a one-dimensional manner.

[0107] Next, the clustering unit 203 has the following function (first clustering function): based on the distribution information of each excitation light in the C wavelength bands of fluorescence intensity, clusters N pixels into C pixel groups. Specifically, the clustering unit 203 clusters pixels with the same wavelength band of excitation light having the maximum fluorescence intensity into the same pixel group. Figure 17 In FIG, an image of a pixel group clustered by the first clustering function of the clustering unit 203 is shown. Figure 18 The wavelength characteristics of the absorption rate of the excitation light of the multiple pigments contained in the sample S are shown in FIG. Figure 17 As shown in FIG. 1 , the sample S contains three types of pigments, namely pigment C1, pigment C2, and pigment C3. Assuming that six fluorescence images are obtained using excitation light of six wavelength bands, the clustering unit 203 clusters the N pixels contained in the six fluorescence images GC1 to GC6 into six pixel groups PGr1 to PGr6. Figure 18 As shown, different types of pigments typically have different wavelength characteristics for their absorptivity. The three pigments C1, C2, and C3 also have wavelength characteristics with different peak wavelengths, CW1, CW2, and CW3. Therefore, within the six wavelength bands of excitation light, EW1, EW2, EW3, EW4, EW5, and EW6, the pigment with the highest absorptivity is determined to be one of the three pigments C1, C2, and C3. For example, pigment C1 has the highest absorptivity for excitation light in wavelength band EW1, pigment C1 has the highest absorptivity for excitation light in wavelength band EW2, and pigment C2 has the highest absorptivity for excitation light in wavelength band EW3. Leveraging this property, clustering unit 203, using the first clustering function, can cluster N pixels into pixel groups with the same pigment distribution range. However, the six pixel groups PGr1 to PGr6 clustered using the first clustering function do not correspond one-to-one to the three pigments C1, C2, and C3.

[0108] Furthermore, the clustering unit 203 has the following function (second clustering function): based on wavelength information, it further clusters the C pixel groups clustered by the first clustering function into L pixel groups (L is an integer greater than 2 and less than N-1). The number of clustered pixel groups, L, corresponds to the number of pigment types that may be present in the sample S and is pre-set as a parameter stored in the image processing unit 81. Specifically, for each of the C pixel groups clustered by the first clustering function, the clustering unit 203 identifies the estimated centroid fluorescence wavelength for the wavelength band of the excitation light corresponding to that pixel group. More specifically, the clustering unit 203 obtains wavelength information from the wavelength information acquisition unit 202 for the pixel group clustered as having the maximum absorbance in a certain wavelength band and identifies the centroid fluorescence wavelength based on the obtained wavelength information. The wavelength information acquisition unit 202 then acquires the wavelength information using the average value of the fluorescence intensity within the pixel groups of the separated fluorescence image corresponding to that wavelength band. Furthermore, the clustering unit 203 determines the distance (value similarity) between the specific centroid fluorescence wavelengths for each of the C pixel groups, thereby clustering the C pixel groups into L pixel groups. The clustering unit 203 then divides the matrix data Y, which contains the fluorescence intensity values ​​of the pixels of the C fluorescence image arranged in parallel in a one-dimensional manner, into cluster matrices for each of the L pixel groups and regenerates the cluster matrix.

[0109] exist Figure 19 The distribution of the centroid fluorescence wavelengths specified by the clustering unit 203 is shown in Figure 20 ] represents an image of a pixel group clustered by the second clustering function of the clustering unit 203. Figure 19 and Figure 20 In the example shown, for each specific centroid fluorescence wavelength FW1 to FW6 of the six pixel groups PGr1 to PGr6 clustered by the first clustering function, the pixel group PGr1 and the pixel group PGr2 whose centroid fluorescence wavelengths are close to each other are clustered into a new pixel group PGr 01 Similarly, the pixel groups PGr3 and PGr4 are clustered into the pixel group PGr 02 , clustering pixel groups PGr5 and PGr6 into pixel group PGr 03 Thus, the pixels of the C fluorescence images can be divided into L pixel groups corresponding to the distribution of pixels assumed to be included in the sample S. However, the number of divisions L based on the second clustering function is set to be less than the number C of fluorescence images (the number C of wavelength bands of excitation light).

[0110] The statistical value calculation unit 204 obtains a mixing matrix A for generating K pigment images representing the distribution of K pigments (K is an integer greater than or equal to 2 and less than or equal to C) from the C fluorescence images based on the L cluster matrices obtained for the sample S. Typically, the mixing matrix A is used based on the relationship between matrix data Y, which is the observation matrix, and pigment matrix data X, which is a one-dimensional arrangement of the K pigment images arranged in parallel for each pixel, using the following equation:

[0111] Y=AX

[0112] Here, Y is the matrix data of C rows and N columns, A is the matrix data of C rows and K columns, and X is the matrix data of K rows and N columns. Conversely, if the value of the mixing matrix A is obtained, the pigment matrix data X can be obtained using the inverse matrix A of the mixing matrix A. -1 And matrix data Y, through the following formula:

[0113] X=A -1 Y

[0114] Derivation (This process is called demixing).

[0115] Here, the statistical value calculation unit 204 compresses the matrix data Y generated by the clustering unit 203 by pixel clusters, thereby regenerating matrix data Y'. Specifically, the statistical value calculation unit 204 calculates a statistical value for each pixel cluster in the clustered matrix based on the fluorescence intensity of each row of the matrix data Y, compressing each pixel cluster in each row into a single pixel having the calculated statistical value. Thus, the statistical value calculation unit 204 regenerates matrix data Y' as C rows and L columns of matrix data. The statistical value calculation unit 204 can calculate the average value of the cumulative value of the fluorescence intensity, the mode of the fluorescence intensity, or the median of the fluorescence intensity as the statistical value.

[0116] Furthermore, the statistical value calculation unit 204 uses the following equation containing the mixing matrix A in the regenerated matrix data Y′ and the color matrix data X′ compressed in the same manner from the color matrix data X:

[0117] Y'=AX'

[0118] The property also holds true, and the mixing matrix A is derived based on the matrix data Y'. Figure 21 , which is an image of the matrix data Y' regenerated by the statistical value calculation unit 204 and the corresponding pigment matrix data X'. Figure 21 Each square in the figure represents an element of the matrix data. 01 ~PGr 03 The pigment matrix data X and matrix data Y are based on the pixel group PGr 01~PGr 03 The statistical value of each is a representative value, and is compressed into three columns of pigment matrix data X' and matrix data Y'.

[0119] As follows, the statistic calculation unit 204 derives the mixing matrix A based on the matrix data Y'. Specifically, the statistic calculation unit 204 sets an initial value for the mixing matrix A, calculates the loss function (loss value) Los described below while sequentially changing the value of the mixing matrix A, and derives the mixing matrix A such that the value of the loss function Los decreases. Furthermore, a regularization term such as the L1 norm λ|A| (λ represents a coefficient indicating the degree of emphasis on the regularization term) may be added to this loss function.

[0120] [Mathematical formula 1]

[0121]

[0122] In the above formula, j is a parameter indicating the position of the matrix data row (corresponding to the wavelength band of the excitation light). The matrix subscript 1j represents the matrix data of the jth row of the first cluster matrix, the matrix subscript 2j represents the matrix data of the jth row of the second cluster matrix, and the matrix subscript 3j represents the matrix data of the jth row of the third cluster matrix. Furthermore, the parameters a, b, and c represent the average of the statistical values ​​of each column of the matrix data Y'.

[0123] As described above, the statistical value calculation unit 204 calculates a loss function for each of the L cluster matrices divided by the clustering unit 203, referring to the statistical values ​​of the C matrix data Y'. The loss function Los is calculated based on the sum of the L loss functions, and the mixing matrix A is obtained based on this loss function Los. At this time, the statistical value calculation unit 204 corrects the loss function calculated for each of the L cluster matrices by dividing it by the average values ​​a, b, and c of the statistical values ​​of the C matrix data Y'. The loss function Los is then calculated by summing the corrected loss functions. Alternatively, the statistical value calculation unit 204 may correct the row component of each wavelength band of the excitation light of the differential value Y'-AX' by dividing it by the C statistical values ​​corresponding to each wavelength band of the excitation light, thereby calculating the loss function for each of the L cluster matrices.

[0124] Furthermore, the above formula can also be generalized as follows. Specifically, the statistical value calculation unit 204 derives the mixing matrix A and the pigment matrix data X' based on the matrix data Y'. Specifically, the statistical value calculation unit 204 sets initial values ​​for the mixing matrix A and the pigment matrix data X', and while sequentially changing the values ​​of the mixing matrix A and the pigment matrix data X', calculates the loss function (loss value) Los using the following formula, thereby deriving the mixing matrix A and the pigment matrix data X' that reduce the value of the loss function Los. Furthermore, a regularization term such as the L1 norm λ|A| (λ is a coefficient indicating the degree of emphasis on the regularization term) may be added to this loss function. Furthermore, the calculation may be performed under the constraint that the mixing matrix A and the pigment matrix data X' are non-negative values.

[0125] [Mathematical formula 2]

[0126]

[0127] In the above formula, j is a parameter indicating the position of the row of the matrix data (corresponding to the wavelength band of the excitation light), and i is a parameter indicating the position of the column of the matrix data (corresponding to the i-th cluster). ij The weight of each element in the matrix data can be calculated based on the value of each element or its standard deviation. ij All are set to the same value regardless of the weight of each element. In addition, the average value of the statistical value of each column of the matrix data Y' in the above formula is set to a, b, c, ..., and replaced by w 1j =1 / a, w 2j =1 / b, w 3j =1 / c is the same as the formula of the loss function Los shown above.

[0128] As described above, the statistic calculation unit 204 calculates the loss function for each of the L cluster matrices divided by the clustering unit 203, referring to the statistic of the C matrix data Y', and calculates the loss function based on the L loss functions Los i The loss function Los is calculated based on the loss function Los, and the mixing matrix A is obtained based on the loss function Los. In addition, the statistical value calculation unit 204 can correct the row component of each wavelength band of the excitation light of the differential value Y'-AX' by dividing it with the C statistical values ​​corresponding to each wavelength band of the excitation light to calculate the loss function Los of each of the L cluster matrices. i .

[0129] The image generation unit 205 unmixes the C fluorescence images obtained with the sample S as the observation target using the mixing matrix A derived by the statistic calculation unit 204, thereby obtaining K pigment images. Specifically, the image generation unit 205 applies the inverse matrix A of the mixing matrix A to the matrix data Y generated by the clustering unit 203 based on the C fluorescence images.-1 , calculates the pigment matrix data X. The image generation unit 205 then regenerates K pigment images based on the pigment matrix data X and outputs the regenerated K pigment images. The output destination may be an output device of the image processing unit 81, such as a display or a touch panel display, or an external device connected to the image processing unit for data communication.

[0130] An example of the demixing process associated with shooting in a Pinkel arrangement has been described above, but the demixing process associated with shooting in a Pinkel arrangement is not limited to the above example.

[0131] For example, in the image processing unit 81 of this embodiment, the number of clustered pixel groups L and the number of pigment images K are pre-set as parameters based on the number of pigments contained in the sample S. However, the image processing unit 81 may also sequentially change the parameters L and K and repeatedly generate pigment images. For example, the number of clusters L = C-1 and the number of pigments K = C-1 may be set, and demixing may be performed to generate pigment images. If the accuracy of the resulting pigment images for each pigment is poor, the number of clusters L and the number of pigments K may be sequentially changed to C-2, C-3, and so on, and demixing may be repeated.

[0132] Alternatively, the image acquisition device 1 of this embodiment can directly use the C groups of separated fluorescence images obtained by the light detection units 61a and 61b to perform demixing and generate a pigment image. In this case, the number of fluorescence images to be demixed is 2×C. Alternatively, the image acquisition device 1 can switch between using M bandpass filters that transmit a single fluorescence band as the multi-bandpass filter 624 and perform demixing on the resulting M×C fluorescence images.

[0133] Alternatively, the image acquisition device 1 may include multiple excitation light sources 7 that simultaneously irradiate excitation light of multiple wavelength bands toward the sample S. The device 1 may irradiate excitation light of C different wavelength distributions while varying the intensity ratios between the multiple wavelength bands, thereby acquiring C fluorescence images. In this case, it is also possible to obtain a high-precision image of each of the multiple pigments.

[0134] When generating matrix data Y before clustering, the image processing unit 81 of this embodiment can generate data in which the N pixels constituting the image are arranged in the row direction according to a predetermined rule, or it can generate data in which the N pixels constituting the image are arranged in the row direction according to a random rule. However, the data of the fluorescence image constituting C rows of matrix data Y is set to data in which N pixels are arranged in the same rule. Using the matrix data Y generated by random arrangement, the same matrix data Y' can be regenerated through clustering.

[0135] Furthermore, the image processing unit 81 of the present embodiment may generate the matrix data Y before clustering by removing background pixels (pixels without pigment) included in the fluorescent image.

[0136] As a method of clustering pixel groups in the image processing unit 81 , a method using machine learning such as the K-means method, a method using deep learning, or the like can be adopted.

[0137] In addition, as a clustering method for pixel groups in the image processing unit 81, in addition to the K-means method, methods using machine learning such as decision trees, support vector machines, KNN (K nearest neighbors), self-organizing maps, spectral clustering, Gaussian mixture models, DBSCAN, Affinity Propagation, Mean Shift, Ward, Agglomerative Clustering, OPTICS, and BIRCH, or methods using deep learning, etc., can also be used. Furthermore, before clustering is applied, the matrix data Y can be pre-processed. For example, the C-dimensional data of each pixel can be reduced in dimensionality using Phasor Analysis, principal component analysis, singular value decomposition, independent component analysis, linear discriminant analysis, t-SNE, UMAP, or other machine learning methods.

[0138] In the image acquisition device 1 of this embodiment, in addition to the aforementioned tilted dichroic mirror having a wavelength characteristic in which transmittance changes linearly with wavelength, a single-band dichroic mirror that transmits one wavelength band or a multi-band dichroic mirror that transmits multiple wavelength bands can be used as the optical filter 67a. In this case, the aforementioned multi-bandpass filter can be used as the multi-bandpass filter 624, or a single-bandpass filter that transmits one wavelength band can be used.

[0139] Here, when a single bandpass filter is used as the multi-bandpass filter 624, and a tilted dichroic mirror having a transmittance characteristic of t(λ)=a1λ+b1 with respect to the wavelength λ is used as the optical filter 67a, the wavelength information acquisition unit 202 of the image processing unit 81 uses the following formula:

[0140] [Mathematical formula 3]

[0141]

[0142] , calculates wavelength information WLC representing the centroid fluorescence wavelength. Here, x1 is the fluorescence intensity of one separated fluorescence image, and x2 is the fluorescence intensity of the other separated fluorescence image. On the other hand, when a single bandpass filter is used as the multi-bandpass filter 624, and a single-band dichroic mirror or a multi-band dichroic mirror is used as the optical filter 67a, the wavelength information acquisition unit 202 of the image processing unit 81 uses the following formula:

[0143] WL ratio =x2 / x1

[0144] Calculate wavelength information WL ratio .

[0145] Furthermore, the optical filter 67a is not limited to a dichroic mirror; a filter set with the same wavelength characteristics can be used, or a beam splitter (such as a polarization beam splitter) that splits the fluorescence can be used. Furthermore, by switching between filters with different wavelength characteristics and capturing multiple images with a single camera, a single set of separated fluorescence images can be obtained. Furthermore, a single camera can be used to split the field of view and capture the two components of fluorescence separated by the optical filter 67a.

[0146] Furthermore, to obtain wavelength information related to fluorescence wavelengths, the wavelength information acquisition unit 202 of the image acquisition device 1 of this embodiment can process fluorescence images acquired using a camera capable of detecting at least two fluorescence wavelengths. Examples of such cameras include color sensors (color cameras) or multi-band sensors (multi-band cameras). For example, when using a color sensor, the wavelength information acquisition unit 202 can calculate and acquire information related to fluorescence wavelengths by comparing the three intensity values ​​of R, G, and B pixels acquired by the color sensor. Alternatively, when using a multi-band sensor, the wavelength information acquisition unit 202 can calculate and acquire information related to fluorescence wavelengths by comparing the intensity values ​​of each different wavelength acquired by the multi-band sensor. In this case, wavelength information is also acquired based on the fluorescence image captured, enabling high-precision analysis of the wavelength information. Consequently, the accuracy of pigment image separation can be improved.

[0147] [Function and Effect]

[0148] In the image acquisition device 1, fluorescence emitted from the sample S can be detected via an optical filter 67a having a transmission characteristic whose transmittance varies within the fluorescence wavelength region. This allows efficient and high-precision acquisition of matrix data (e.g., a mixing matrix) for performing unmixing processing. Consequently, the image acquisition device 1 can efficiently and accurately acquire multiple fluorescence images separated into multiple wavelength regions.

[0149] In the image acquisition device 1, the optical path switching unit 62 includes a dichroic mirror 622 that selectively reflects excitation light in multiple first wavelength regions and selectively transmits fluorescence in multiple second wavelength regions. This allows the sample S to be reliably irradiated with excitation light in each of the multiple first wavelength regions. Furthermore, fluorescence can be reliably detected in the multiple second wavelength regions.

[0150] In the image acquisition device 1, the optical path switching unit 62 includes a multi-bandpass filter 623 that selectively transmits excitation light in multiple first wavelength regions incident on the dichroic mirror 622. This allows the sample S to be more reliably irradiated with excitation light in each of the multiple first wavelength regions.

[0151] In the image acquisition device 1, the optical path switching unit 62 includes a multi-bandpass filter 624 that selectively transmits fluorescence in multiple second wavelength regions emitted from the dichroic mirror 622. This allows for more reliable detection of fluorescence in multiple second wavelength regions.

[0152] In the image acquisition device 1 , the optical filter 67 a has a transmission characteristic in which the transmittance changes linearly in the fluorescence wavelength region. This allows for more efficient and accurate acquisition of matrix data for performing the unmixing process.

[0153] In the image acquisition device 1 , the optical filter unit 67 is configured to remove the optical filter 67a from the third optical path P3 . Thus, by removing the optical filter 67a from the third optical path P3 after detecting the fluorescence used to acquire matrix data, fluorescence detection for acquiring multiple fluorescence images can be performed efficiently and accurately.

[0154] In the image acquisition device 1, the optical filter unit 67 is configured to arrange an optical filter 67a on the third optical path P3 between the optical path switching unit 62 and the imaging lens 66. This prevents the fluorescence imaged by the imaging lens 66 from being affected by aberrations of the optical filter 67a.

[0155] In the image acquisition device 1, the fluorescence filter unit 68 is configured such that the light-passing portion 683 and the plurality of single-bandpass filters 682 are each disposed on the third optical path P3. Thus, by detecting fluorescence with the light-passing portion 683 disposed on the third optical path P3, multiple fluorescence images can be acquired indirectly (i.e., through the implementation of the demixing process). Alternatively, by detecting fluorescence with the plurality of single-bandpass filters 682 disposed on the third optical path P3, multiple fluorescence images can be acquired directly.

[0156] In the image acquisition device 1, the optical path switching unit 63 is configured to switch the optical path of the excitation light from the fourth optical path P4 to each of the fifth optical path P5 and the sixth optical path P6. Thus, by using the excitation light emitted from the same light emitting unit 72, fluorescence is detected in the fluorescence macro-image acquisition unit 4, and, for example, the presence range of the sample S on the slide G can be determined.

[0157] In the image acquisition device 1, the optical path switching unit 63 is configured to rotate the mirror 631 about the fourth optical path P4 as the center line, switching the state of the mirror 631 to a state where the fourth optical path P4 and the fifth optical path P5 intersect on the mirror surface 631a, and a state where the fourth optical path P4 and the sixth optical path P6 intersect on the mirror surface 631a. In this way, the optical path of the excitation light can be switched from the fourth optical path P4 to each of the fifth optical path P5 and the sixth optical path P6 while saving space.

[0158] In the image acquisition device 1, the image processing unit 81 generates a fluorescence macro image of the slide G and sets the imaging conditions for the Pinkel configuration in the fluorescence micro image acquisition unit 6 based on the fluorescence macro image of the slide G. Thus, for example, by acquiring a fluorescence macro image of the entire area of ​​the slide G, the presence range of the specimen S on the slide G can be identified. Furthermore, under the imaging conditions set based on the fluorescence macro image, for example, a fluorescence micro image of the specimen S in the Pinkel configuration (a plurality of fluorescence micro images separated into a plurality of wavelength regions) can be indirectly acquired for the entire area of ​​the specimen S, thereby enabling efficient understanding of the state of the entire area of ​​the specimen S.

[0159] In the image acquisition device 1, the image processing unit 81 generates a fluorescence microscopic image of the specimen S in the Pinkel configuration, and sets the imaging conditions for the Sedat configuration in the fluorescence microscopic image acquisition unit 6 based on the fluorescence microscopic image of the specimen S in the Pinkel configuration. Thus, under the imaging conditions set based on the fluorescence microscopic image of the specimen S in the Pinkel configuration, for example, a fluorescence microscopic image of the specimen S in the Sedat configuration (a plurality of fluorescence microscopic images separated into a plurality of wavelength regions) is directly acquired for a specific region within the specimen S, thereby enabling the state of the specific region within the specimen S to be accurately grasped.

[0160] Furthermore, the time required for excitation light scanning in the Pinkel configuration is shorter than that required in the Sedat configuration. Furthermore, the wavelength separation accuracy of fluorescence microscopic images generated in the Sedat configuration is higher than that in the Pinkel configuration. As described above, by generating and observing fluorescence microscopic images of the entire area of ​​sample S using the Pinkel configuration and generating and observing fluorescence microscopic images of specific areas within sample S using the Sedat configuration, it is possible to appropriately analyze sample S.

[0161] [Modification]

[0162] The present disclosure is not limited to the above-mentioned embodiments. For example, the optical filter unit 67 may include a plurality of optical filters (a first optical filter, a second optical filter) 67a having different transmission characteristics, or may be configured to configure any one of the optical filters 67a on the third optical path P3. In this case, Figure 22 As shown in (a) and (b) of FIG. 1 , the transmission characteristics (first transmission characteristics, second transmission characteristics) of each optical filter 67a may be characteristics in which the transmittance changes linearly in the fluorescence wavelength region and the inclinations are different from each other in the fluorescence wavelength region. Figure 23 As shown in (a) and (b), it is also possible that the transmission characteristic (first transmission characteristic) of one optical filter 67a is a characteristic in which the transmittance changes linearly in the fluorescence wavelength region, while the transmission characteristic (second transmission characteristic) of the other optical filter 67a is a characteristic in which the transmittance is constant in the fluorescence wavelength region.

[0163] In other words, it suffices that at least one optical filter 67a has a transmission characteristic in which the transmittance changes within the fluorescence wavelength range. In this case, by detecting fluorescence through each of the multiple optical filters 67a, matrix data for performing the unmixing process can be acquired more efficiently and accurately. Furthermore, the fluorescence microscopic image acquisition unit 6 can be configured to allow for the installation and removal (replacement) of each optical filter 67a.

[0164] In addition, the optical filter unit 67 and the light detection unit 61 can be configured as follows. Figure 24 、 Figure 25 、 Figure 26 and Figure 27 As shown, the optical filter 67a can separate the fluorescence from the sample S into two optical paths with different wavelength characteristics ( Figure 14 (The same example as shown in the figure) In this case, for example, a dichroic mirror having a wavelength characteristic in which transmittance increases linearly with increasing wavelength can be used as the optical filter 67a. By detecting the fluorescence transmitted through the optical filter 67a and the fluorescence reflected by the optical filter 67a, matrix data for performing the demixing process can be obtained more efficiently and with higher accuracy.

[0165] exist Figure 24 In the example shown, the fluorescence transmitted through the optical filter 67a is transmitted through the imaging lens 66a, reflected by the mirror 65a, and detected by the light detection unit 61a. The fluorescence reflected by the optical filter 67a is transmitted through the imaging lens 66b, reflected by the mirror 65b, and detected by the light detection unit 61b. Figure 25 and Figure 26 In the example shown, the optical filter 67a is arranged inside the prism. Figure 25 and Figure 26In the example shown, the fluorescence that has passed through the optical filter 67a passes through the prism, is reflected by the mirror 65a, and is detected by the light detection unit 61a. The fluorescence that has been reflected by the optical filter 67a is reflected by the prism and is detected by the light detection unit 61b. When the optical filter 67a is provided inside the prism, the degree of freedom in the arrangement of the light detection units 61a and 61b is increased. Figure 27 In the illustrated example, the prism having the optical filter 67 a built therein, and the light detection section 61 a and the light detection section 61 b ​​are configured as one camera.

[0166] In addition, if the image acquisition device 1 has a light emitting portion 72, it may not have a light output portion 71. For example, the excitation light source 7 may be configured as an external device. In this case, the optical fiber 73 or the like may be connected to the light emitting portion 72 to guide the excitation light from the excitation light source 7 configured as an external device to the light emitting portion 72. In addition, the image acquisition device 1 may not have an image processing portion 81. For example, the image processing device 8 may be configured as an external device. In this case, the image acquisition device 1 may be connected to the image processing device 8 configured as an external device so as to be communicable. In addition, the optical path switching portion 62 may be configured to selectively transmit excitation light in a plurality of first wavelength regions and selectively reflect fluorescence in a plurality of second wavelength regions.

[0167] Explanation of symbols

[0168] 1…Image acquisition device, 31…Support unit, 45…Illumination unit, 47…Light detection unit (second light detection unit), 61…Light detection unit (first light detection unit), 62…Optical path switching unit (first light path switching unit), 622…Dichroic mirror, 623…Multi-bandpass filter (first multi-bandpass filter), 624…Multi-bandpass filter (second multi-bandpass filter), 63…Optical path switching unit (second light path switching unit), 631…Mirror, 631a…Mirror surface, 66…Imaging lens, 67…Optical filter unit, 67a…Optical filter (first optical filter, second optical filter), 68…Fluorescence filter unit, 682…Single-bandpass filter, 683…Light passing unit, 72…Light emitting unit, 81…Image processing unit, P1…First optical path, P2…Second optical path, P3…Third optical path, P4…Fourth optical path, P5…Fifth optical path, P6…Sixth optical path, S…Sample

Claims

1. An image acquisition device, wherein: have: a support portion that supports the specimen; a light emitting portion for emitting excitation light; a first light detection unit that detects fluorescence emitted from the sample by irradiation with the excitation light; a first optical path switching unit configured to switch an optical path of the excitation light and an optical path of the fluorescence; and an optical filter unit including an optical filter that transmits the fluorescence, The light emitting portion is optically connected to the first optical path switching portion via a first optical path, The supporting portion is optically connected to the first optical path switching portion via a second optical path, The first light detection unit is optically connected to the first light path switching unit via a third light path. The first optical path switching unit is configured to switch the optical path of the excitation light of a plurality of first wavelength regions that are different from each other from the first optical path to the second optical path, and to switch the optical path of the fluorescence of a plurality of second wavelength regions that are shifted from the plurality of first wavelength regions from the second optical path to the third optical path. The optical filter has a transmission characteristic in which the transmittance changes in the fluorescence wavelength region including the plurality of second wavelength regions. The optical filter unit is configured to arrange the optical filter on the third optical path.

2. The image acquisition device according to claim 1, wherein The first optical path switching unit includes a dichroic mirror that selectively reflects the excitation light in the plurality of first wavelength regions and selectively transmits the fluorescence in the plurality of second wavelength regions.

3. The image acquisition device according to claim 2, wherein: The first optical path switching unit further includes a first multi-bandpass filter configured to selectively transmit the excitation light in the plurality of first wavelength regions incident on the dichroic mirror.

4. The image acquisition device according to claim 2 or 3, wherein: The first optical path switching unit further includes a second multi-bandpass filter configured to selectively transmit the fluorescent light in the plurality of second wavelength regions emitted from the dichroic mirror.

5. The image acquisition device according to any one of claims 1 to 4, wherein The optical filter has the transmission characteristic in which the transmittance changes linearly in the fluorescence wavelength region.

6. The image acquisition device according to any one of claims 1 to 5, wherein: The optical filter unit includes a first optical filter and a second optical filter. The first optical filter is the optical filter having a first transmission characteristic as the transmission characteristic, The second optical filter has a second transmission characteristic different from the first transmission characteristic, The optical filter unit is configured to arrange the first optical filter and the second optical filter on the third optical path.

7. The image acquisition device according to any one of claims 1 to 6, wherein: The optical filter unit is configured to remove the optical filter from the third optical path.

8. The image acquisition device according to any one of claims 1 to 7, wherein: The device further includes an imaging lens, the imaging lens being arranged on the third optical path and forming an image of the fluorescence on the first light detection unit. The optical filter section is configured to arrange the optical filter on the third optical path between the first optical path switching section and the imaging lens.

9. The image acquisition device according to any one of claims 1 to 8, wherein It also includes a fluorescence filter unit including a light passing portion and a plurality of single-bandpass filters. The light passing portion allows the fluorescence in the fluorescence wavelength range to pass through. The plurality of single-bandpass filters selectively transmit the fluorescence in each of the plurality of second wavelength regions. The fluorescence filter unit is configured such that the light passing portion and each of the plurality of single-bandpass filters are arranged on the third optical path.

10. The image acquisition device according to claim 9, wherein: The apparatus further includes an image processing unit that generates a first fluorescent image of the sample based on data output from the first light detection unit when the light passing unit is disposed on the first optical path, and generates a second fluorescent image of the sample based on data output from the first light detection unit when the plurality of single-bandpass filters are respectively disposed on the first optical path. The image processing unit sets imaging conditions for generating the second fluorescent image based on the first fluorescent image.

11. The image acquisition device according to any one of claims 1 to 9, wherein Also features: an irradiation section configured to irradiate the sample supported by the support section with the excitation light at a position different from a position where the fluorescence is detected by the first light detection section; a second light detection section that detects fluorescence emitted from the sample due to irradiation of the excitation light by the irradiation section; as well as a second optical path switching unit, which is disposed on the first optical path and switches the optical path of the excitation light; The light emitting portion is optically connected to the second optical path switching portion via a fourth optical path which is a part of the first optical path. The first optical path switching portion is optically connected to the second optical path switching portion via a fifth optical path that is a part of the first optical path. The irradiation unit is optically connected to the second optical path switching unit via a sixth optical path. The second optical path switching unit is configured to switch the optical path of the excitation light from the fourth optical path to each of the fifth optical path and the sixth optical path.

12. The image acquisition device according to claim 11, wherein: The second optical path switching unit includes a mirror having a mirror surface arranged on the fourth optical path in a state inclined relative to the fourth optical path. The second optical path switching unit is configured to rotate the mirror with the fourth optical path as the center line, switching the state of the mirror to a state in which the fourth optical path and the fifth optical path intersect on the mirror surface and a state in which the fourth optical path and the sixth optical path intersect on the mirror surface.

13. The image acquisition device according to claim 11 or 12, wherein: further comprising an image processing unit configured to generate a third fluorescent image of the sample based on data output from the second light detection unit and a fourth fluorescent image of the sample based on data output from the first light detection unit, The image processing unit sets imaging conditions for generating the fourth fluorescent light image based on the third fluorescent light image.

Images