3D image capture device

Abstract

To enable three-dimensional images to be quickly captured, when capturing images using fluorescent light.SOLUTION: A three-dimensional image capturing device comprises: an irradiation unit that irradiates an observation target with strip-shaped excitation light at a prescribed time interval; a relative position change unit that changes the relative position of the observation target and the excitation light with regard to a prescribed scan direction; and an image capturing unit which, with the relative positions changed by the relative position change unit and a plurality of portions of the observation target irradiated with excitation light at a prescribed time interval by the irradiation unit, respectively, while the portions of the observation target irradiated with excitation light are changed with regard to the scan direction, captures images of the plurality of portions in one exposure by forming an image of fluorescence emitted from each of the plurality of portions at different positions on an image capturing element, respectively.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a three-dimensional image capturing device. [Background technology]

[0002] In imaging flow cytometry, it takes time to acquire three-dimensional images, so it was not possible to obtain three-dimensional images of cells flowing at the flow speeds (1-10 meters per second) used in general flow cytometry. A three-dimensional microscopy method is known that uses a light sheet tilted in the scanning direction to capture a cross section of an object to capture a three-dimensional image of the object at high speed (Non-Patent Document 1).Another known imaging method is one that uses strobe photography with multiple wavelengths of light to capture a three-dimensional image of an object in an extremely short imaging time of the order of femtoseconds to picoseconds (Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] "NATURE PHOTONICS", January 19, 2015, Volume 9, February 2015 issue, p.113-119 [Non-patent document 2] "NATURE PHOTONICS", August 10, 2014, Volume 8, September 2014 issue, p.695-700 Summary of the Invention [Problem to be solved by the invention]

[0004] In the 3D microscopy method described in Non-Patent Document 1, one frame of image is captured with one light sheet, so multiple frames are required to capture multiple cross sections. Therefore, a high frame rate is required when capturing images of cross sections of fast-flowing cells, as in flow cytometry. On the other hand, the imaging method described in Non-Patent Document 2 acquires signals for each wavelength, so it is not suitable for capturing fluorescence.

[0005] To implement 3D imaging in general flow cytometry, it is necessary to capture 3D images of cells flowing at the flow speeds used in general flow cytometry (1-10 meters per second) using the fluorescence emitted from the cells. As described above, when imaging using fluorescence, it is required to be able to capture three-dimensional images at high speed.

[0006] The present invention has been made in view of the above points, and provides a three-dimensional image capturing device that can capture three-dimensional images at high speed when capturing images using fluorescence. [Means for solving the problem]

[0007] The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is a three-dimensional image capturing device comprising: an irradiation unit that irradiates an object to be observed with band-shaped excitation light at predetermined time intervals; a relative position change unit that changes the relative position between the object to be observed and the excitation light in a predetermined scanning direction; and an imaging unit that images the plurality of portions with a single exposure by irradiating the excitation light onto each of a plurality of portions of the object to be observed at predetermined time intervals by the irradiation unit while changing the relative position by the relative position change unit so that the portion of the object to be observed that is irradiated with the excitation light is changed in the scanning direction, and thereby imaging the fluorescence emitted from each of the plurality of portions at different positions on an imaging element.

[0008] In addition, in one aspect of the present invention, in the above-mentioned three-dimensional image capturing device, the relative position changing unit changes the relative position in the scanning direction by moving the object to be observed in a movement direction that is a direction different from the scanning direction, and tilting the longitudinal direction of the strip-shaped excitation light irradiated by the irradiation unit by a predetermined angle with respect to the movement direction when viewed from the short side direction of the strip-shaped excitation light.

[0009] Furthermore, one aspect of the present invention is that, in the above-mentioned three-dimensional image capturing device, the imaging element is a collection of elements in which a plurality of elements are arranged in a matrix and signals are read out in units of rows or columns, and the imaging element is arranged such that the direction of the rows or columns in which the signals of the plurality of elements are read out corresponds to the direction in which different positions on the imaging element at which fluorescence emitted from each of the plurality of portions is imaged are lined up.

[0010] In addition, one aspect of the present invention is that, in the above-mentioned three-dimensional image capturing device, the relative position change unit changes the relative position in the scanning direction by changing the direction in which the excitation light is irradiated onto the fixed object to be observed. [Effects of the Invention]

[0011] According to the present invention, when imaging is performed using fluorescence, a three-dimensional image can be captured at high speed. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram showing the external configuration of a cell measuring system according to a first embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an example of the functional configuration of an imaging flow cytometer according to a first embodiment of the present invention. [Figure 3] 1A and 1B are diagrams illustrating an example of light sheet illumination light irradiated onto a flow path according to the first embodiment of the present invention. [Figure 4] 1 is a diagram showing an example of the positional relationship between light sheet illumination light and cells according to the first embodiment of the present invention. FIG. [Figure 5]FIG. 2 is a diagram showing an example of a captured image captured by an imaging section according to the first embodiment of the present invention. [Figure 6] FIG. 2 is a diagram showing an example of an imaging result according to the first embodiment of the present invention. [Figure 7] FIG. 2 is a diagram showing an example of a three-dimensional image generated by a three-dimensional image generating unit according to the first embodiment of the present invention. [Figure 8] FIG. 10 is a diagram showing an example of the configuration of a cell measuring system according to a second embodiment of the present invention. [Figure 9] FIG. 10 is a diagram showing an example of the positional relationship between light sheet illumination light and cells according to the second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] (First embodiment) A first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing the external configuration of a cell measurement system 1. The cell measurement system 1 includes an imaging flow cytometer 20 and a display unit 10. The imaging flow cytometer 20 includes at least one flow path through which an object to be observed flows. Note that, although this embodiment describes a case in which the object to be observed is a cell, the object to be observed is not limited to a cell. The object to be observed may be anything that transmits light. The cell in question is a fluorescently stained cell.

[0014] The imaging flow cytometer 20 generates a three-dimensional image of cells flowing through a flow path. The display unit 10 displays the three-dimensional image generated by the imaging flow cytometer 20. The display unit 10 is equipped with, for example, a liquid crystal display, and displays various images. The images displayed on the display unit 10 include the three-dimensional image of cells generated by the imaging flow cytometer 20.

[0015] [Functional configuration of the Imaging Flow Cytometer 20] Next, the functional configuration of the imaging flow cytometer 20 will be described with reference to FIG. 2 is a diagram showing an example of the functional configuration of an imaging flow cytometer 20 according to this embodiment. The imaging flow cytometer 20 includes the above-described flow channel 21, an irradiation unit 22, an optical system 23, an imaging unit 24, and a control unit 25.

[0016] The flow channel 21 moves the cells C1 in the direction of the flow velocity by causing the cells C1 to flow together with a fluid. The flow velocity of the fluid is, for example, 1-10 meters per second. The fluid is, for example, a sheath liquid.

[0017] The irradiation unit 22 irradiates the flow path 21 with excitation light P1. The irradiation unit 22 includes a light source 221. The light source generates band-shaped coherent light as the excitation light P1. The excitation light P1 is excitation light that has been formed into a band shape by narrowing down the coherent light. The specific shape of the band-shaped excitation light will be described later.

[0018] The irradiation unit 22 irradiates the cell C1 with a band-shaped excitation light at predetermined time intervals. The imaging flow cytometer 20 performs stroboscopic photography by irradiating the cell C1 with the band-shaped excitation light at predetermined time intervals.

[0019] In this embodiment, the light source 221 is, for example, a pulsed laser. In this embodiment, the irradiation unit 22 irradiates the cell C1 with pulsed light as excitation light P1. Each of the multiple pulses included in the excitation light P1, which is pulsed light, is a band-shaped excitation light. When the irradiation unit 22 irradiates pulsed light, the predetermined time interval is a predetermined period. The band-shaped excitation light is called light sheet illumination light LS1.

[0020] The light source 221 may be a CW (Continuous Wave) laser. When the light source 221 is a CW laser, the irradiating unit 22 irradiates the cell C1 with light obtained by modulating continuous wave laser light at a predetermined period as the excitation light P1.

[0021] The optical system 23 irradiates the excitation light P1 generated by the light source 221 onto a predetermined position in the flow path 21. The optical system 23 tilts the longitudinal direction of the strip-shaped light sheet illumination light LS1 irradiated by the irradiation unit 22 by a predetermined angle with respect to the direction of the flow velocity of the flow path 21 when viewed from the side of the flow path 21. In this embodiment, the predetermined position in the flow path 21 onto which the excitation light P1 is irradiated is determined in advance and does not change over time. The optical system 23 includes various lenses including, for example, a dichroic mirror and an objective lens.

[0022] In the following description, the position in the flow channel 21 where the excitation light P1 is irradiated will also be referred to as the irradiation position. When a cell C1 passes through this irradiation position, fluorescent molecules are excited by the excitation light P1, causing the cell C1 to emit light. The light resulting from this emission is fluorescence F1.

[0023] The imaging unit 24 captures an image of the fluorescence F1 from the cell C1 that has passed through the position in the flow path 21 where the excitation light P1 is irradiated. In this way, the imaging unit 24 captures an image of a certain portion of the cell C1. Here, the certain portion is a portion where fluorescent molecules are excited by the excitation light P1. The certain portion has a thickness approximately equal to the thickness of the excitation light, in response to the band-like excitation light being irradiated onto the cell C1. The top or bottom surface of the certain portion is also referred to as a cross-section of the cell C1.

[0024] The imaging unit 24 images multiple portions of the cell C1 with a single exposure. The imaging unit 24 includes, for example, a camera. The camera is preferably a camera capable of high-speed imaging. The camera is, for example, a CCD (Charge Coupled Device) camera or a CMOS (Complementary MOS) camera. The camera includes an imaging element 241. The imaging element 241 is a solid-state imaging element such as a CCD or CMOS.

[0025] In this embodiment, the image sensor 241 is a collection of elements in which a plurality of elements are arranged in a matrix and signals are read out in units of rows or columns. The image sensor 241 has a rectangular shape with more pixels in the horizontal direction than in the vertical direction. The readout circuit of the image sensor 241 is configured to acquire the detected light intensity for each horizontal pixel column of the image sensor 241. As will be described later, the image sensor 241 is arranged in an orientation such that the horizontal direction (row direction) in which signals from the plurality of elements are read out coincides with the direction in which different positions on the image sensor 241 are aligned and in which images of fluorescence F1 emitted from each of the plurality of portions of the cell C1 are formed.

[0026] In this example, the image sensor 241 is an image sensor configured with an sCMOS (Scientific CMOS; CMOS for scientific measurement), etc. The sCMOS can capture images faster and with better image quality than conventional CCDs or image sensors configured with CMOS. The image capturing unit 24 includes an optical system for collecting the fluorescence F1 from the cell C1 on the image capturing element 241. A part of the optical system may also serve as a part of the optical system 23 described above.

[0027] The control unit 25 includes, for example, a CPU, a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), etc., and performs various calculations and transfers of information. The control unit 25 includes a 3D image generation unit 251 as its functional unit.

[0028] The three-dimensional image generating unit 251 acquires a captured image from the imaging element 241. The three-dimensional image generating unit 251 generates a three-dimensional image of the cell C1 based on a plurality of cross-sectional images included in the acquired captured image. Generating a three-dimensional image is also referred to as reconstructing a three-dimensional image.

[0029] [Image capture method] 3 to 5, an imaging method using the imaging flow cytometer 20 will be described. Fig. 3 is a diagram showing an example of light sheet illumination light LS1 irradiated onto the flow path 21 according to this embodiment. Fig. 4 is a diagram showing an example of the positional relationship between the light sheet illumination light LS1 and a cell C1 according to this embodiment.

[0030] 3 and 4 show an xyz coordinate system as a three-dimensional Cartesian coordinate system. In this embodiment, the x-axis direction is the length direction of the flow channel 21. The x-axis direction is the direction in which the cells C1 flow in the flow channel 21. The cells C1 flow in the +x direction of the x-axis. The y-axis direction is the width direction (also referred to as the depth direction) of the flow channel 21. The z-axis direction is the height direction of the flow channel 21. In the following description, in the width direction of the flow channel 21, the positive side of the y-axis direction may be referred to as the "right side," and the negative side of the y-axis direction may be referred to as the "left side." The positive side of the z-axis direction may be referred to as the "upper side," and the negative side of the z-axis direction may be referred to as the "lower side."

[0031] FIG. 3(A) shows the flow channel 21 when viewed from above. FIG. 3(B) shows the flow channel 21 when viewed from the side (e.g., the left side). In this embodiment, the fluid flowing through the flow channel 21 is maintained in a steady flow state. Therefore, while the cell C1 flows through the flow channel 21, at least during the course of imaging by the imaging unit 24, the position of the cell C1 in the width direction of the flow channel 21, the position of the cell C1 in the height direction of the flow channel 21, and the orientation of the cell C1 relative to the top or bottom surface of the flow channel 21 do not change. In other words, the cell C1 flows straight through the flow channel 21 from upstream to downstream (+x direction) without rotating, at least during the course of imaging by the imaging unit 24.

[0032] As shown in Figure 3(B), the light sheet illumination light LS1 is rotated from a direction parallel to the bottom surface of the flow channel 21 (parallel to the xy plane) around a direction parallel to the width direction of the flow channel 21 (y-axis direction) as the rotation axis, and is irradiated at a predetermined angle relative to the direction in which the cell C1 flows in the flow channel 21 (x-axis direction). The optical system provided in the imaging flow cytometer 20 tilts the direction of the plane of the light sheet illumination light LS1 emitted by the irradiation unit 22 by a predetermined angle with respect to the direction of the flow velocity.

[0033] 3(B) illustrates an example in which the light sheet illumination light LS1 is inclined in a direction in which the height of the upstream side is lower than the height of the downstream side when viewed from the side of the flow path 21, but this is not limited to this. The light sheet illumination light LS1 may also be inclined in a direction in which the height of the upstream side is higher than the height of the downstream side when viewed from the side of the flow path 21.

[0034] As described above, in this embodiment, the excitation light P1 is pulsed light. The light sheet illumination light LS1 is one pulse contained in the excitation light P1. Therefore, when the pulse contained in the excitation light P1 is irradiated onto the flow path 21, the flow path 21 is in a state where it is irradiated with the light sheet illumination light LS1. Figure 4 corresponds to the case where the light sheet illumination light LS1 is irradiated onto the flow path 21. On the other hand, when the pulse is not irradiated onto the flow path 21, the flow path 21 is in a state where it is not irradiated with the light sheet illumination light LS1.

[0035] The flow path 21 and the optical system 23 are an example of a relative position change unit. In other words, in this embodiment, the relative position change unit moves the cell C1 through the flow path 21 in the direction of the flow velocity of the flow path 21, which is a direction different from the scanning direction, and changes the relative position between the cell C1 and the light sheet illumination light LS1 in the scanning direction by tilting the longitudinal direction of the strip-shaped light sheet illumination light LS1 irradiated by the irradiation unit 22 by a predetermined angle with respect to the direction of the flow velocity of the flow path 21 when viewed from the short-side direction of the strip-shaped light sheet illumination light LS1. Therefore, the relative position change unit changes the relative position between the cell C1 and the light sheet illumination light LS1 in the predetermined scanning direction. In this embodiment, the short-side direction of the strip-shaped light sheet illumination light LS1 coincides with the width direction (depth direction or y-axis direction) of the flow path 21.

[0036] In this embodiment, scanning refers to irradiating multiple sections (cross sections) of an object to be observed with excitation light at predetermined time intervals and capturing images of the multiple sections based on the fluorescence emitted from the multiple sections. The scanning direction is the direction in which the multiple sections (cross sections) of the object to be observed that are irradiated with excitation light at predetermined time intervals are lined up.

[0037] 4 shows a time series of light emission from each part of the cell C1 flowing through the flow path 21 at multiple times when the light sheet illumination light LS1 is irradiated onto the flow path 21. The multiple times are shown in FIG. 4 as six times from "t=1" to "t=6." In this embodiment, the excitation light P1 is pulsed light. That is, the light sheet illumination light LS1 is irradiated onto the flow path 21 at a predetermined cycle.

[0038] As described above, the light sheet illumination light LS1 is inclined at a predetermined angle with respect to the direction (x-axis direction) in which the cell C1 flows in the flow channel 21. Therefore, the portion of the cell C1 that is irradiated with the light sheet illumination light LS1 changes in a predetermined scanning direction as the cell C1 flows through the flow channel 21. The scanning direction is the normal direction to the plane of the light sheet illumination light LS1. The light sheet illumination light LS1, which is a plurality of pulses contained in the excitation light P1, is each irradiated to a different portion of the cell C1. In other words, the portion of the cell C1 flowing through the flow channel 21 is scanned in the normal direction to the plane of the strip-shaped light sheet illumination light LS1.

[0039] In this embodiment, the cell C1 flowing through the flow channel 21 passes through an area irradiated with the light sheet illumination light LS1 at a constant speed, the area being irradiated with the light sheet illumination light LS1, whose position does not change relative to the flow channel 21. Therefore, the light sheet illumination light LS1 is irradiated onto each portion (cross section) of the cell C1 at equal intervals in the scanning direction.

[0040] The area irradiated with the light sheet illumination light LS1 is a planar area where the strip-shaped light sheet illumination light LS1 and the cell C1 intersect. The area of ​​the cell C1 irradiated with the light sheet illumination light LS1 emits light. Because the area irradiated with the light sheet illumination light LS1 changes as the cell C1 flows through the flow path 21, the area of ​​the cell C1 that emits light changes as the cell C1 flows through the flow path 21.

[0041] As described above, the light sheet illumination light LS1 is a strip-shaped excitation light. The width direction of the light sheet illumination light LS1 coincides with the width direction of the flow path 21. The length direction of the light sheet illumination light LS1 is inclined at a predetermined angle with respect to the length direction of the flow path 21. The thickness (height) direction of the light sheet illumination light LS1 is inclined at a predetermined angle with respect to the height direction of the flow path 21 when the flow path 21 is viewed from the side. The predetermined angle is, for example, an angle greater than 0 degrees and less than 45 degrees.

[0042] The width of the light sheet illumination light LS1 is sufficiently longer than the diameter of the cell C1, which is the object of observation. In this embodiment, the width of the light sheet illumination light LS1 is longer than the width of the flow path 21. The height (thickness) of the light sheet illumination light LS1 is sufficiently shorter than the diameter of the cell C1. The position of the intersection between the light sheet illumination light LS1 and the cell C1 changes as the cell C1 flows through the flow path 21, but the area of ​​the light sheet illumination light LS1 consisting of these parts is narrow enough to be included in the vertical angle of view of the image sensor 241 included in the imaging unit 24. The direction of the vertical angle of view of the image sensor 241 is the length direction of the light sheet illumination light LS1.

[0043] The imaging unit 24 is installed so that the direction of the optical axis of the camera lens coincides with the direction of the normal to the surface of the light sheet illumination light LS1. Note that the direction of the optical axis of the camera lens provided in the imaging unit 24 may be tilted from the direction of the normal to the surface of the light sheet illumination light LS1. The imaging unit 24 images the multiple parts of the cell C1 with a single exposure by irradiating the light sheet illumination light LS1 onto each of the multiple parts of the cell C1 at a predetermined time interval and forming images of the fluorescence emitted from each of the multiple parts of the cell C1 at different positions on the imaging element 241. The imaging unit 24 maintains the exposure state while imaging the multiple parts of one cell C1.

[0044] The image capturing unit 24 preferably opens and closes the shutter based on, for example, the following shutter opening and closing conditions. Note that the following shutter opening and closing conditions are not essential. For example, the image capturing unit 24 opens the shutter to start exposure when the cell C1 flowing through the flow path 21 passes a predetermined position (exposure start position) in the flow path 21. The exposure start position is, for example, a position upstream of the irradiation position of the excitation light P1 in the flow path 21 and outside the angle of view of the image capturing element 241. The image capturing unit 24 closes the shutter and ends the exposure when a predetermined time has elapsed since the start of exposure. Note that the image capturing unit 24 may end the exposure when the cell C1 passes a predetermined position (exposure end position) in the flow path 21. The exposure end position is, for example, a position downstream of the irradiation position of the excitation light P1 in the flow path 21 and outside the angle of view of the image capturing element 241. The imaging unit 24 opens and closes the shutter in accordance with the position of the cell C1 flowing through the flow channel 21. Therefore, the imaging unit 24 opens and closes the shutter in synchronization with the movement of the object to be observed.

[0045] A plurality of cells C1 may be flowed through the flow channel 21, and the imaging unit 24 may capture an image of each of the plurality of cells C1. In this case, the imaging unit 24 captures an image of one cell C1 per exposure. The imaging unit 24 repeatedly opens and closes the shutter to sequentially capture an image of each of the plurality of cells C1 flowing through the flow channel 21.

[0046] When multiple cells C1 are flowed through the flow path 21, the distance between the multiple cells C1 is set to a level that prevents another cell C1 from entering the field of view of the imaging element 241 while the imaging unit 24 is imaging a certain cell C1. If the width of the flow channel 21 is sufficiently wide, images of a plurality of cells flowing at different positions in the width direction of the flow channel 21 may be captured in a single exposure.

[0047] 5 is a diagram showing an example of a captured image T1 captured by the imaging unit 24 according to this embodiment. FIG. 5 shows an XY coordinate system as a two-dimensional Cartesian coordinate system. In this embodiment, the X direction is the horizontal direction of the imaging element 241. The X direction coincides with the direction of the flow velocity of the flow channel 21 (the x-axis direction in FIGS. 3 and 4). The Y direction is the vertical direction of the imaging element 241.

[0048] 5, cross-sectional images, which are images of portions of cell C1 that have emitted light at multiple times, are captured as one frame of image at different positions on the image sensor 241. In the captured image T1, fluorescence emitted from portions of cell C1 at six times indicated by "t=1" to "t=6" is focused at different positions on the image sensor 241 and captured as one frame of image.

[0049] The imaging unit 24 images multiple parts of the cell C1 with a single exposure by irradiating the light sheet illumination light LS1 onto each of the multiple parts of the cell C1 at predetermined time intervals using the irradiation unit 22 while changing the relative position of the imaging unit 24 with the light sheet illumination light LS1 using the flow path 21 and the optical system 23, thereby changing the part of the cell C1 irradiated with the light sheet illumination light LS1 in the scanning direction, and forming images of the fluorescence emitted from each of the multiple parts of the cell C1 at different positions on the imaging element 241. In other words, the imaging unit 24 images multiple parts of the cell C1 with one frame without requiring multiple frames.

[0050] As described above, the image sensor 241 simultaneously acquires the detected light intensity for each pixel row in the horizontal direction (X direction). Furthermore, the image sensor 241 simultaneously acquires the light intensity for all horizontal pixel rows arranged in the vertical direction (Y direction). In other words, the camera provided in the imaging unit 24 uses a global shutter. Therefore, the imaging time required to capture one captured image T1 increases in proportion to the time required to scan in the vertical direction (Y direction). In other words, the imaging time required to capture one captured image T1 increases in proportion to the number of pixels in the vertical direction. Therefore, the number of images that the image sensor 241 can capture per unit time is inversely proportional to the number of pixels in the vertical direction and is independent of the number of pixels in the horizontal direction.

[0051] In this embodiment, the image sensor 241 is arranged in a direction such that the horizontal direction (X direction) in which signals from the multiple elements are read out coincides with the direction in which different positions on the image sensor 241 are aligned, where the fluorescence F1 emitted from each of the multiple portions of the cell C1 is imaged. Therefore, the imaging speed of the imaging flow cytometer 20 does not depend on the number of pixels in the horizontal direction of the image sensor 241. The imaging speed of the imaging flow cytometer 20 is not limited by the imaging speed of the camera itself used in the imaging unit 24.

[0052] The image sensor 241 may be arranged in an orientation other than the orientation in which the horizontal direction (X direction) in which signals from the multiple elements are read out coincides with the direction in which different positions on the image sensor 241, on which images of the fluorescence F1 emitted from each of the multiple portions of the cell C1, are aligned. Even in this case, the image sensor 241 is preferably arranged such that the row or column direction in which signals from the multiple elements are read out coincides with the direction in which different positions on the image sensor 241, on which images of the fluorescence F1 emitted from each of the multiple portions of the cell C1 are aligned. The direction in which the different positions are aligned is, for example, a direction tilted at an angle greater than 0 degrees and less than a predetermined angle (e.g., 45 degrees) from the direction in which the different positions are aligned.

[0053] In addition, in the present embodiment, the image sensor 241 is a collection of elements in which a plurality of elements are arranged in a matrix and signals are read out in units of rows or columns, and is rectangular in shape with more pixels arranged horizontally than vertically, but is not limited to this. The image sensor 241 may also be square in shape with the same number of pixels arranged horizontally and vertically.

[0054] In this embodiment, the three-dimensional image generating unit 251 combines multiple cross-sectional images of the cell C1 captured in the captured image T1 in an order based on the scanning direction. The three-dimensional image generating unit 251 acquires in advance information indicating the relationship between the scanning direction and the vertical and horizontal directions of the image sensor 241. The three-dimensional image generating unit 251 determines the order in which the multiple cross-sectional images captured in the captured image T1 should be combined, based on the information indicating this relationship. The three-dimensional image generating unit 251 generates a three-dimensional image by combining the multiple cross-sectional images based on the determined order.

[0055] The inclination of the light sheet illumination light LS1 is determined based on, for example, the flow velocity of the fluid flowing through the flow channel 21, the period of the pulses contained in the excitation light P1, and the desired resolution in the depth (thickness) direction of the portion of the cell C1. The depth (thickness) direction of the portion of the cell C1 is the direction in which the cell C1 is scanned by the light sheet illumination light LS1.

[0056] [Image results] 6 is a diagram showing an example of an imaging result according to this embodiment. Note that the captured image shown in FIG. 6 shows a scale of "30 μm." The captured image T10 is an image captured when the flow velocity of the fluid flowing through the flow channel 21 is 1.3 m / s and a CW laser modulated at a frequency of 50 kHz is used as the light source 221. The captured image T20 is an image captured when the flow velocity of the fluid flowing through the flow channel 21 is 11.8 m / s and a CW laser modulated at a frequency of 500 kHz is used as the light source 221. The captured image T30 is the result of imaging when the flow velocity of the fluid flowing through the flow channel 21 is 0.98 m / s and a Q-switched laser with a pulse frequency of 48 kHz is used as the light source 221.

[0057] Comparing the imaging conditions for captured image T20 with those for captured images T10 and T30, the flow velocity under the imaging conditions for captured image T20 is about 10 times that under the imaging conditions for captured images T10 and T30. Accordingly, in capturing captured image T20, the frequency of the CW laser is set to about 10 times the frequency of the CW laser used to capture captured image T10 or the frequency of the Q-switched laser used to capture captured image T30.

[0058] Generally, a Q-switched laser has a larger amount of light per pulse than a CW laser, so when a Q-switched laser is used as the light source 221, the contrast of the cross-sectional image of the cell C1 captured in the cross-sectional image can be increased compared to when a CW laser is used as the light source 221.

[0059] Fig. 7 is a diagram showing an example of a three-dimensional image generated by the three-dimensional image generating unit 251 according to this embodiment. Fig. 7 shows the result of reconstructing a three-dimensional image of a cell C1.

[0060] In the present embodiment, an example has been described in which the optical system 23 tilts the direction of the plane of the light sheet illumination light LS1 by a predetermined angle with respect to the direction of the flow velocity of the flow path 21, but this is not limited to this. The optical system 23 may make the plane of the light sheet illumination light LS1 parallel to the upper surface (or lower surface) of the flow path 21 and change the position of the light sheet illumination light LS1 in the depth direction.

[0061] In the present embodiment, an example has been described in which the irradiating unit 22 irradiates the cell C1 with the light sheet illumination light LS1 at a predetermined period, but this is not limited to this. The irradiating unit 22 may irradiate the cell C1 with the light sheet illumination light LS1 at non-periodic time intervals. For example, as described above, when the plane of the light sheet illumination light LS1 is parallel to the upper surface (or lower surface) of the flow channel 21 and the position of the light sheet is changed in the depth direction, the irradiating unit 22 may irradiate the cell C1 with the light sheet illumination light LS1 at non-periodic time intervals.

[0062] Furthermore, for example, if the movement speed of cell C1 changes while passing through an area illuminated by light sheet illumination light LS1 whose position does not change, the illumination unit 22 may irradiate light sheet illumination light LS1 onto cell C1 at non-periodic time intervals in accordance with the change in movement speed.

[0063] In this embodiment, an example has been described in which the cell C1 flows straight through the flow channel 21 from upstream to downstream (+x direction) without rotating, at least during the imaging by the imaging unit 24, but this is not limiting. For example, the control unit 25 may include a detection unit that detects a deviation in the orientation of the cell C1 from before the rotation when the cell C1 rotates. In the process of reconstructing a three-dimensional image, the three-dimensional image generation unit 251 corrects the deviation caused by the rotation of the cell C1 based on the deviation detected by the detection unit.

[0064] As described above, the three-dimensional image capturing device according to this embodiment (in this embodiment, the imaging flow cytometer 20) comprises an irradiation unit 22, a relative position change unit (in this embodiment, the flow path 21 and the optical system 23), and an imaging unit 24. The irradiating unit 22 irradiates an object to be observed (cell C1 in this embodiment) with band-shaped excitation light (light sheet illumination light LS1 in this embodiment) at predetermined time intervals. The relative position change unit (in this embodiment, the flow path 21 and the optical system 23) changes the relative position between the object to be observed (in this embodiment, the cell C1) and the excitation light (in this embodiment, the light sheet illumination light LS1) in a predetermined scanning direction (in this embodiment, the normal direction to the surface of the light sheet illumination light LS1). The imaging unit 24 images multiple parts (in this embodiment, multiple parts of the cell C1) with a single exposure by changing the relative position (in this embodiment, the relative position between the cell C1 and the light sheet illumination light LS1) using a relative position change unit (in this embodiment, the flow path 21 and the optical system 23), thereby changing the part of the object to be observed (in this embodiment, the cell C1) to be irradiated with the excitation light (in this embodiment, the light sheet illumination light LS1) in the scanning direction (in this embodiment, the normal direction of the surface of the light sheet illumination light LS1) while the irradiation unit 22 irradiates each of multiple parts of the object to be observed (in this embodiment, the cell C1) with the excitation light (in this embodiment, the light sheet illumination light LS1) at a predetermined time interval, and images the fluorescence emitted from each of the multiple parts (in this embodiment, multiple parts of the cell C1) at different positions on the imaging element 241.

[0065] With this configuration, the three-dimensional image capturing device according to this embodiment (the imaging flow cytometer 20 in this embodiment) can capture images of multiple portions of an object to be observed with a single exposure, allowing for high-speed capture of three-dimensional images when imaging using fluorescence. Here, "high speed" means faster than capturing images of multiple portions of an object to be observed in multiple frames. The three-dimensional image capturing device according to this embodiment does not require a high frame rate because it can capture images of multiple portions of an object to be observed with a single exposure.

[0066] In addition, in the three-dimensional image capturing device of this embodiment (in this embodiment, the imaging flow cytometer 20), the relative position change unit (in this embodiment, the flow path 21 and the optical system 23) moves the object to be observed (in this embodiment, the cell C1) in a movement direction (in this embodiment, the direction of the flow velocity of the flow path 21), which is a direction different from the scanning direction (in this embodiment, the normal direction of the surface of the light sheet illumination light LS1), and changes the relative position (in this embodiment, the relative position between the cell C1 and the light sheet illumination light LS1) in the scanning direction (in this embodiment, the normal direction of the surface of the light sheet illumination light LS1) by tilting the longitudinal direction of the band-shaped excitation light (in this embodiment, the light sheet illumination light LS1) irradiated by the irradiation unit 22 by a predetermined angle with respect to the movement direction (in this embodiment, the direction of the flow velocity of the flow path 21) when viewed from the short side direction of the band-shaped excitation light (in this embodiment, the light sheet illumination light LS1).

[0067] With this configuration, the 3D image capturing device according to this embodiment (the imaging flow cytometer 20 in this embodiment) can capture multiple parts of a moving object with a single exposure when capturing a 3D image of the object, enabling high-speed capture of 3D images when using fluorescence imaging. Here, "high-speed" means faster than capturing multiple parts of a moving object in multiple frames.

[0068] As explained above, the three-dimensional image capturing device according to this embodiment is suitable for use as an imaging flow cytometer 20. In flow cytometry using a sheath flow, cells flow through a flow channel at a flow velocity of 1-10 m / s. With the above configuration, the imaging flow cytometer 20 can capture images of cells flowing at a flow velocity of 1-10 m / s. Furthermore, the imaging flow cytometer 20 can instantly acquire three-dimensional images of cells flowing through the flow channel 21, making it possible to instantly distinguish cells using machine learning and to implement a cell sorter.

[0069] In this embodiment, an example has been described in which the cell C1, which is the observation object, is moved in a movement direction that is a direction different from the scanning direction by flowing the cell C1 through the flow channel 21, but the present invention is not limited to this. As long as the observation object can be moved in a movement direction that is a direction different from the scanning direction, a mechanism other than a flow channel may be used.

[0070] Furthermore, in the three-dimensional image capturing device of this embodiment (in this embodiment, the imaging flow cytometer 20), the image capturing element 241 is a collection of elements in which a plurality of elements are arranged in a matrix and signals are read out in units of rows or columns, and the image capturing element 241 is arranged such that the direction of the rows or columns in which signals from the plurality of elements are read out corresponds to the direction in which different positions on the image capturing element 241 are aligned at which fluorescence F1 emitted from each of a plurality of parts of the object to be observed (in this embodiment, the cell C1) is imaged.

[0071] With this configuration, in the three-dimensional image capturing device according to this embodiment (in this embodiment, the imaging flow cytometer 20), the imaging speed is not limited by the readout speed of the imaging element 241 compared to when the imaging element 241 is not arranged such that the row or column direction in which signals from the multiple elements are read out corresponds to the direction in which different positions on the imaging element 241 at which images of the fluorescence F1 emitted from each of the multiple portions of the observation object (in this embodiment, the cell C1) are lined up, and therefore the imaging speed of the three-dimensional image can be increased. In particular, when the imaging element 241 is arranged such that the row or column direction in which signals from the multiple elements are read out corresponds to the direction in which different positions on the imaging element 241 at which images of the fluorescence F1 emitted from each of the multiple portions of the observation object (in this embodiment, the cell C1) are lined up, the imaging speed is not limited by the readout speed of the imaging element 241.

[0072] (Second embodiment) The second embodiment of the present invention will be described in detail below with reference to the drawings. In the first embodiment, the three-dimensional image capturing device was described as a case where the relative position of the observation object and the excitation light (light sheet illumination light) changes as the observation object flows through the flow channel. In this embodiment, the position of the observation object is fixed, and a polygonal mirror is used to change the relative position of the excitation light with respect to the observation object. The cell measuring system according to this embodiment is referred to as a cell measuring system 1a.

[0073] Fig. 8 is a diagram showing an example of the configuration of a cell measurement system 1a according to this embodiment. Fig. 9 is a diagram showing an example of the positional relationship between the light sheet illumination light LS2 and the cell C2 according to this embodiment. 8 and 9 show an xyz coordinate system as a three-dimensional Cartesian coordinate system. The z-axis is oriented vertically upward. The x-axis and y-axis are parallel to the horizontal plane.

[0074] The cell measurement system 1a includes a light source D1, a camera CM1, and an optical system O1. The optical system O1 includes various lenses, various mirrors, an aperture A1, and the like. The various lenses included in the optical system O1 include a lens L1, a lens L2, a lens L3, an objective lens L4, a lens L5, a lens L6, and a lens L7. The various mirrors included in the optical system O1 include a dichroic mirror M1, a polygonal mirror M2, a half mirror M3, and a tilted mirror M4.

[0075] Lenses L1, L2, L3, L5, and L7 are, for example, tube lenses and achromatic lenses (achromatic lenses). An achromatic lens is a lens made by bonding together two optical elements, for example, crown glass, which has a low refractive index, and flint glass, which has a high refractive index.

[0076] The light source D1 emits excitation light E1. The excitation light E1 is pulsed light. Each pulse contained in the excitation light E1 is a band-shaped coherent light. The light source D1 is, for example, a pulsed laser. The excitation light E1 emitted from the light source D1 is reflected by the dichroic mirror M1. The excitation light E1 reflected by the dichroic mirror M1 is collected by the lens L1.

[0077] The excitation light E1 collected by the lens L1 is reflected by the polygonal mirror M2 as excitation light E2. The excitation light E2 reflected by the polygonal mirror M2 is collected by the lens L2, passes through the aperture A1, and then collected by the lens L3 to enter the objective lens L4. Each of the multiple pulses contained in the excitation light E2 is irradiated as light sheet illumination light LS2 onto the cell C2, which is the observation target, via the objective lens L4.

[0078] Here, the polygonal mirror M2 has multiple reflecting surfaces. In the example shown in Fig. 8, the polygonal mirror M2 has eight reflecting surfaces. The polygonal mirror M2 has a rotation mechanism and can rotate at high speed around a rotation axis parallel to the y-axis direction.

[0079] The polygonal mirror M2 rotates around its axis of rotation, changing the angle of the reflecting surface relative to the incident direction of the excitation light E2, and reflects the multiple pulses contained in the excitation light E2 from one of its reflecting surfaces. In the process of the multiple pulses being reflected by the reflecting surface, the angle of the reflecting surface relative to the incident direction of the excitation light E2 is changed, so that the multiple pulses are reflected in different directions. The polygonal mirror M2 rotates at a rotational speed such that each of the multiple pulses is irradiated onto a different part of the cell C2, which is the object of observation.

[0080] FIG. 9 shows an example of the positional relationship between the light sheet illumination light LS2 and the cell C2 according to this embodiment. The light sheet illumination light LS2 corresponds to one of the multiple pulses contained in the excitation light E2. As the orientation of the reflecting surface changes with the rotation of the polygonal mirror M2, the portion of the cell C2 irradiated with the light sheet illumination light LS2 changes in a predetermined direction. The predetermined direction is, for example, the normal direction of the surface of the light sheet illumination light LS2 (the direction indicated by the arrow AR1 in FIG. 9). In other words, the portion of the cell C2 is scanned in a predetermined direction (scanning direction).

[0081] The polygonal mirror M2 is an example of a relative position changer, which changes the direction in which the excitation light is irradiated onto a fixed observation target, thereby changing the relative position between the observation target and the excitation light in the scanning direction.

[0082] Returning to FIG. 8, the description of the configuration of the cell measuring system 1a will continue. Light sheet illumination light LS2 is irradiated onto a portion of cell C2, and fluorescence F1 emitted from that portion is incident on polygonal mirror M2 via objective lens L4, lens L3, aperture A1, and lens L2. The fluorescence F1 incident on polygonal mirror M2 is reflected by the reflective surface of polygonal mirror M2 toward lens L1.

[0083] Fluorescence F1 reflected by the reflective surface of polygonal mirror M2 and collected by lens L1 enters dichroic mirror M1 as fluorescence F2. Dichroic mirror M1 transmits the incident fluorescence F2 toward lens L5. Fluorescence F2 transmitted through dichroic mirror M1 is collected by lens L5 and enters half mirror M3 as fluorescence F3. Half mirror M3 transmits the incident fluorescence F3 toward lens L6.

[0084] Lens L6 focuses the incident fluorescence F3 to form an image of a portion (cross section) of cell C2. The fluorescence F3 imaged by lens L6 is reflected by tilted mirror M4, which is tilted at a predetermined angle (e.g., 45 degrees) with respect to the optical axis of lens L6.

[0085] The fluorescence F3 reflected by the tilted mirror M4 is incident on the half mirror M3 via the lens L6 as fluorescence F4. The fluorescence F4 incident on the half mirror M3 is reflected by the half mirror M3 towards the lens L7. The lens L7 collects the fluorescence F4 reflected by the half mirror M3. The fluorescence F4 collected by the lens L7 forms an image on the image sensor of the camera CM1.

[0086] The camera CM1 is irradiated with the light sheet illumination light LS2 at predetermined time intervals, and the fluorescence emitted from each of the multiple parts of the cell C2 is imaged at different positions on the imaging element, thereby capturing images of multiple parts of the cell C2 in a single exposure. The camera CM1 maintains the exposure state while capturing images of the multiple parts of one cell C2.

[0087] As described above, in the three-dimensional image capturing device of this embodiment (in this embodiment, cell measurement system 1a), the relative position change unit (in this embodiment, polygonal mirror M2) changes the direction in which excitation light (in this embodiment, light sheet illumination light LS2) is irradiated onto a fixed observation object (in this embodiment, cell C2), thereby changing the relative position (in this embodiment, the relative position between cell C2 and light sheet illumination light LS2) in the scanning direction (in this embodiment, the normal direction to the surface of light sheet illumination light LS2).

[0088] With this configuration, the 3D image capturing device of this embodiment (in this embodiment, the cell measurement system 1a) can capture multiple parts of a stationary object of observation with a single exposure when capturing a 3D image of the object, thereby capturing a 3D image faster than capturing multiple parts of the stationary object of observation each in multiple frames.

[0089] In the above-described embodiments, the scanning direction when irradiating the multiple portions of the observation object with the band-like excitation light is a direction indicated by a straight line has been described as an example, but this is not limiting. The scanning direction may also be an angular direction. In other words, excitation light may be irradiated at a predetermined time interval onto each of multiple portions (cross sections) of the observation object viewed from different directions in the angular direction of the observation object, and cross-sectional images of the multiple portions in the different directions may be captured.

[0090] In each of the above-described embodiments, the order in which the multiple portions of the observation object are scanned does not have to be consecutive in the scanning direction. For example, in the first embodiment, when the plane of the light sheet illumination light LS1 is not inclined with respect to the upper surface (lower surface) of the flow channel 21 and the position of the light sheet illumination light LS1 is changed in the depth direction, the order in which the multiple portions of the cell C1 are irradiated with the light sheet illumination light LS1 at predetermined time intervals does not have to be consecutive in the height direction (z-axis direction), which is the scanning direction.

[0091] In the above-described embodiments, the observation object, which is the object for capturing a three-dimensional image, is a cell, but the present invention is not limited to this. The observation object may be a particle, a macro object, or the like, as long as it is a substance that emits fluorescence when irradiated with excitation light.

[0092] Each of the above-mentioned devices has a computer built in. The processes of the above-mentioned devices are stored in the form of a program on a computer-readable recording medium, and the computer reads and executes the program to perform the above-mentioned processes. Here, computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, the computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute the program.

[0093] The program may also be for realizing part of the above-mentioned functions. Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program).

[0094] One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0095] 1, 1a... cell measurement system, 20... imaging flow cytometer, 21... flow path, 22... irradiation unit, 23... optical system, 24... imaging unit, 241... imaging element, LS1, LS2... light sheet illumination light, C1, C2... cells

Claims

1. an irradiation unit that irradiates the object to be observed with a strip of excitation light at predetermined time intervals; a relative position change unit that changes the relative position between the object to be observed and the excitation light in a predetermined scanning direction; an imaging unit that images the plurality of portions with a single exposure by irradiating the excitation light onto each of the plurality of portions of the observation object at predetermined time intervals using the irradiation unit while changing the relative position using the relative position change unit to change the portion of the observation object that is irradiated with the excitation light in the scanning direction, and forming images of fluorescence emitted from each of the plurality of portions at different positions on an imaging element; A three-dimensional image capturing device comprising:

2. The relative position change unit changes the relative position in the scanning direction by moving the object to be observed in a moving direction that is a direction different from the scanning direction, and tilting the longitudinal direction of the strip-shaped excitation light irradiated by the irradiation unit by a predetermined angle with respect to the moving direction when viewed from the lateral direction of the strip-shaped excitation light. The three-dimensional image capturing device according to claim 1 .

3. The imaging element is a group of elements in which a plurality of elements are arranged in a matrix and signals are read out in units of rows or columns, The imaging element is arranged such that the row or column direction in which signals from the plurality of elements are read out corresponds to the direction in which different positions on the imaging element in which images of fluorescence emitted from each of the plurality of portions are formed are aligned. The three-dimensional image capturing device according to claim 2 .

4. The relative position change unit changes the direction in which the excitation light is irradiated onto the fixed observation object, thereby changing the relative position in the scanning direction. The three-dimensional image capturing device according to claim 1 .

Images