Observation apparatus and observation method

Abstract

To provide an observation device capable of reducing variation in resolution in a moving direction of an observation object.SOLUTION: The observation device includes an irradiation optical system 22, a scanning unit 3, an imaging optical system 4, and an imaging device 5. The emission optical system 22 emits planar light L1 to the observation object B. The scanning unit 3 moves the observation object B at a constant speed in a direction intersecting the planar light L1 and causes the observation object B to pass through the emission surface La of the planar light L1. The imaging optical system 4 forms an image of the observation light P1 generated in the observation object B by the radiation of the planar light L1. The observation optical system 4 has an observation optical path L2 intersecting the radiation surface La. The imaging device 5 acquires image information L2 including an optical image of the observation light D1 every time the observation object B moves by a predetermined length. The scanning unit 3 outputs position information S for determining an exposure start timing when the imaging device 5 acquires the image D1, on the basis of the moving distance of the moving stage 33.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an observation device and an observation method. [Background technology]

[0002] Patent Document 1 discloses a sample observation device and a sample observation method. The sample observation device includes an illumination optical system, a scanning unit, an imaging optical system, an image acquisition unit, and an image generation unit. The illumination optical system irradiates the sample with planar light in the XZ plane. The scanning unit scans the sample in the Y axis direction so that the sample passes through the surface irradiated with the planar light. The imaging optical system has an observation axis tilted with respect to the irradiated surface, and forms an image of the observation light generated in the sample by the illumination with the planar light. The image acquisition unit acquires multiple XZ image data corresponding to the optical image of the observation light formed by the imaging optical system. The image generation unit generates observation image data of the sample based on the multiple XZ image data acquired by the image acquisition unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 256429 Summary of the Invention [Problem to be solved by the invention]

[0004] A light sheet microscope using SPIM (Selective Plane Illumination Microscopy) is known as one of the techniques for observing the interior of an object with a three-dimensional structure, such as a cell (see, for example, Patent Document 1). The light sheet microscope makes it possible to obtain information about the interior of an object while sufficiently reducing the influence of variations in the brightness value of background light.

[0005] In a light sheet microscope, an object to be observed is moved at a constant speed in a direction intersecting the planar light, and the object to be observed passes through the plane irradiated with the planar light. Then, image data including an optical image of the observation light generated on the object to be observed by the irradiation of the planar light is acquired each time the object to be observed moves a predetermined distance. However, if the timing of acquiring image data is controlled by time (in other words, if image data is acquired at regular intervals), variations in the moving speed of the object to be observed will also cause variations in the resolution in the moving direction (the Y-axis direction in Patent Document 1). As a result, there is a risk that the resolution of the observation image data in the moving direction will significantly deviate from the resolution in the direction perpendicular to the moving direction.

[0006] An object of the present disclosure is to provide an observation device and an observation method that can reduce variations in resolution in the movement direction of an observation object. [Means for solving the problem]

[0007] [1] An observation device according to an embodiment of the present disclosure includes an illumination optical system, a scanning unit, an imaging optical system, an imaging device, an image generation unit, and a control unit. The illumination optical system irradiates an observation object with planar light. The scanning unit moves the observation object at a constant speed in a direction intersecting the planar light, causing the observation object to pass through the illumination plane of the planar light. The imaging optical system has an observation axis intersecting the illumination plane and forms an image of the observation light generated on the observation object by the illumination of the planar light. The imaging device acquires image data including an optical image of the observation light formed by the imaging optical system every time the observation object moves a predetermined distance. The image generation unit generates observation image data including an observation image of the observation object based on the multiple image data. The scanning unit has an information output unit that outputs information for determining the timing of exposure start when the imaging device acquires image data, based on the movement distance of the observation object. The control unit controls the start of exposure of the imaging device based on the information. An observation method according to an embodiment of the present disclosure includes the steps of: irradiating an observation object with planar light, moving the observation object at a constant speed in a direction intersecting the planar light, causing the observation object to pass through a plane irradiated with the planar light, and using an imaging optical system having an observation axis intersecting the irradiated plane to form an image of the observation light generated on the observation object by the irradiation of the planar light, and acquiring image data including an optical image of the formed observation light using an imaging device every time the observation object moves a predetermined distance; and generating observation image data including an observation image of the observation object based on the multiple image data. The acquiring step includes a step of outputting information for determining timing of exposure start when the imaging device acquires the image data based on the movement distance of the observation object. In the acquiring step, the start of exposure of the imaging device is controlled based on the information.

[0008] In the observation device and observation method described above, the start of exposure of the imaging device is controlled based on the movement distance of the observed object, not on time. As a result, even if there is variation in the movement speed of the observed object, the exposure start timing changes in response to the variation in the movement speed, making it less likely that variation in resolution will occur in the movement direction. Therefore, with this observation device, it is possible to reduce the variation in resolution in the movement direction of the observed object, and accurately bring the resolution in the movement direction of the observed image data closer to the resolution in the direction perpendicular to the movement direction.

[0009] [2] In the observation device of [1] above, the control unit may determine the timing of starting exposure so that the predetermined distance is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system. Similarly, in the observation method of [1] above, the acquisition step may determine the timing of starting exposure so that the predetermined distance is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system. In this way, by making the interval (predetermined distance) between positions at which image data acquisition in the movement direction starts equal to or close to the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system, the resolution of the observation image data in the movement direction can be accurately brought close to the resolution in the direction perpendicular to the movement direction.

[0010] [3] In the observation device of [1] above, the control unit may determine the timing of exposure start so that the predetermined distance is 1 / N or N times (where N is an integer greater than or equal to 2) the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system. Similarly, in the observation method of [1] above, the acquiring step may determine the timing of exposure start so that the predetermined distance is 1 / N or N times (where N is an integer greater than or equal to 2) the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system. If the predetermined distance is N times the divided value, for example, when the imaging device performs a binning operation in which pixels in N rows and N columns are treated as a single pixel, the resolution of the observed image data in the movement direction can be easily matched with the resolution in the direction perpendicular to the movement direction, thereby eliminating the need for binning in the movement direction. Furthermore, if the predetermined distance is 1 / N times the divided value, the resolution in the movement direction can be increased compared to the resolution in the direction perpendicular to the movement direction. In addition, even if the exposure start timing is shifted for some reason, the position of the image data in the movement direction can be easily corrected.

[0011] [4] In the observation device of any one of [1] to [3] above, the control unit may determine the exposure time of the imaging device when acquiring image data based on the information. Similarly, in the observation method of any one of [1] to [3] above, the acquiring step may determine the exposure time of the imaging device when acquiring image data based on the information. In this case, the proportion of non-exposure time is minimized, and seamless observation image data can be acquired with minimal loss of information about the object of observation.

[0012] [5] In the observation device and observation method described in [1] to [4] above, the FWHM (full width at half maximum) of the light intensity distribution of the planar light in a direction perpendicular to the irradiation surface may be 130% or more and 160% or less of the predetermined distance. In this case, seamless observation image data can be acquired with minimal loss of information about the object being observed.

[0013] [6] In the observation devices of [1] to [5] above, the control unit may convert the information into a trigger signal indicating the timing of exposure start, and provide the trigger signal to the imaging device. Also, in the observation methods of [1] to [5] above, the information may be converted into a trigger signal indicating the timing of exposure start, and the trigger signal may be provided to the imaging device. In this case, the exposure start timing of the imaging device only needs to follow the trigger signal, and control of the imaging device can be simplified.

[0014] [7] In the observation device according to any one of [1] to [6] above, the information output unit may include an encoder. In this case, the information for determining the exposure start timing of the imaging device can be generated with high accuracy using a simple configuration. [Effects of the Invention]

[0015] According to the present disclosure, it is possible to provide an observation device and an observation method that can reduce variations in resolution in the movement direction of an observation object. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram schematically illustrating the configuration of an observation device according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a conceptual diagram showing observed image data including a plurality of image data. [Figure 3] FIG. 3 is a timing chart showing the relationship between the moving speed of the moving stage, the position and timing of the moving stage at which a control signal is output from the control unit, and the exposure period of the imaging device. [Figure 4] FIG. 4 is a flowchart showing an example of an observation method. [Figure 5] FIG. 5 is a diagram showing the relationship between the position at which position information is issued in the movement direction, the exposure section of the imaging device during one frame, and the intensity profile and composite profile of the planar light. [Figure 6]FIG. 6 is a diagram showing the relationship between the position where position information is issued in the movement direction, the exposure section of the imaging device during one frame, and the intensity profile and composite profile of the planar light. [Figure 7] FIG. 7 is a diagram for explaining an example in which a non-exposure section occurs. [Figure 8] FIG. 8 is a graph showing a composite profile obtained by varying the ratio of FWHM of the intensity profile of planar light in the movement direction and the ratio of exposure time, with one frame being 100%. [Figure 9] FIG. 9 is a graph showing a composite profile obtained by varying the ratio of FWHM of the intensity profile of planar light in the movement direction and the ratio of exposure time, with one frame being 100%. [Figure 10] FIG. 10 is a graph and its contour plot showing the ratio of information within an imaged pixel as shades of color, with the FWHM ratio on the vertical axis and the exposure time ratio on the horizontal axis. [Figure 11] FIG. 11 is a graph showing an enlarged portion of FIG. 10 where the exposure time ratio is 95% to 100%. [Figure 12] FIG. 12 is a graph and its contour map showing the minimum value of the information gradation (gradient) in shades of color, with the FWHM ratio on the vertical axis and the exposure time ratio on the horizontal axis. [Figure 13] FIG. 13 is a graph showing an enlarged portion of FIG. 12 where the exposure time ratio is 95% to 100%. [Figure 14] FIG. 14 is a graph and its contour map showing the percentage of the defective area with the FWHM percentage on the vertical axis and the exposure time percentage on the horizontal axis, with the percentage of the defective area shown in shades of color. [Figure 15] FIG. 15 is a graph showing an enlarged portion of FIG. 14 where the exposure time ratio is 95% to 100%. [Figure 16] FIG. 16 is a graph and its contour map showing the amount of information as a three-dimensional data set in shades of color, with the FWHM ratio on the vertical axis and the exposure time ratio on the horizontal axis. [Figure 17]FIG. 17 is a graph showing an enlarged portion of FIG. 16 where the exposure time ratio is 95% to 100%. [Figure 18] FIG. 18 is a diagram showing FIG. 10(b), FIG. 12(b), FIG. 14(b) and FIG. 16(b) arranged side by side. [Figure 19] FIG. 19 is a diagram showing FIG. 11(b), FIG. 13(b), FIG. 15(b) and FIG. 17(b) arranged side by side. [Figure 20] FIG. 20 is a diagram showing FIG. 11(b), FIG. 13(b), FIG. 15(b), and FIG. 17(c) arranged side by side. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0018] 1 is a diagram schematically illustrating the configuration of an observation device 1 according to an embodiment of the present disclosure. The observation device 1 irradiates an observation object B with planar light (sheet light) L1 and forms an image of observation light (e.g., fluorescence or scattered light) generated inside the observation object B on an imaging plane to acquire observation image data of the inside of the observation object B. The observation device 1 is, for example, a slide scanner that acquires and displays an image of the observation object B held on a slide glass, or a plate reader that acquires image data of the observation object B held on a microplate and analyzes the image data.

[0019] Observation object B is, for example, a human or animal cell, tissue, or organ, an animal or plant itself, or a plant cell or tissue. Observation object B is stained with a fluorescent material such as fluorescein-dextran (excitation wavelength: 494 nm / fluorescence wavelength: 521 nm) or tetramethylrhodamine (excitation wavelength: 555 nm / fluorescence wavelength: 580 nm). Observation object B may be stained with multiple fluorescent substances. Observation object B may be contained in a solution, a gel, or a substance with a refractive index different from that of observation object B.

[0020] As shown in FIG. 1, the observation device 1 includes a light irradiation unit 2, a scanning unit 3, an imaging optical system 4, an imaging device 5, an image generation unit 6, and a control unit .

[0021] The light irradiation unit 2 includes a light source 21 and an irradiation optical system 22. The light source 21 is a laser light source such as a laser diode or a solid-state laser light source, or may be a light-emitting diode, a superluminescent diode, or a lamp-based light source.

[0022] The illumination optical system 22 is optically coupled to the light source 21. Light output from the light source 21 is guided to the illumination optical system 22. The illumination optical system 22 shapes the light output from the light source 21 into planar light L1 and irradiates the observation object B with the planar light L1. In the following description, the optical axis of the illumination optical system 22 may be referred to as the optical axis of the planar light L1. The illumination optical system 22 includes a light shaping element such as a cylindrical lens, an axicon lens, or a spatial light modulator. The illumination optical system 22 may also include an objective lens. The planar light L1 formed by the illumination optical system 22 is irradiated onto the observation object B. In consideration of the resolution in the direction A1 perpendicular to the planar light L1, the planar light L1 is preferably thin, with a FWHM thickness of the light intensity of 2 mm or less. On the observation object B irradiated with the planar light L1, observation light L2 is generated at the irradiation surface La of the planar light L1. The observation light L2 is, for example, fluorescent light excited by the planar light L1, scattered light of the planar light L1, or diffusely reflected light of the planar light L1.

[0023] The scanning unit 3 is a mechanism that scans the observation object B with respect to the irradiation surface La of the planar light L1. The scanning unit 3 moves the observation object B at a constant speed along a direction A1 that intersects with the planar light L1, and causes the observation object B to pass through the irradiation surface La of the planar light L1. In this disclosure, a speed that varies from a target speed within ±10% is considered to be a "constant speed." The variation from the target speed is more preferably within ±5%.

[0024] In this embodiment, the scanning unit 3 has a holder 32 that holds the container 11 and a moving stage 33 that moves the container 11 while supporting the holder 32. The container 11 is, for example, a microplate, a slide glass, a petri dish, or the like, and is transparent to the planar light L1 and the observation light L2. In this embodiment, a microplate is used as an example. The container 11 has a frame-shaped main body 14 that forms a well 13 in which the observation object B is placed, and a plate-shaped transparent member 15 that is provided on one side of the main body 14 so as to close one end of the well 13. The main body 14 may form a plurality of wells 13 that are arranged one-dimensionally or two-dimensionally.

[0025] The well 13 is filled with a solution such as a culture medium, a fluorescent indicator, and a buffer, along with the observation object B. Autofluorescence is emitted from the solution. The transparent member 15 has an input surface 15a of the planar light L1 toward the observation object B placed in the well 13. The material of the transparent member 15 is not particularly limited as long as it is a member that is transparent to the planar light L1 and the observation light L2, and may be, for example, glass, quartz, or synthetic resin. The container 11 is arranged so that the input surface 15a is perpendicular to the optical axis of the planar light L1. The other end of the well 13 is open to the outside. The container 11 may be fixed to a moving stage 33.

[0026] The moving stage 33 moves the container 11 along a preset direction A1 in accordance with a control signal from the control unit 7. In this embodiment, the direction A1 is one direction in a plane perpendicular to the irradiation surface La of the planar light L1.

[0027] The imaging optical system 4 is an optical system that forms an image of the observation light L2 generated on the observation object B by irradiation with the planar light L1. The optical axis of the imaging optical system 4 coincides with the observation axis P1 of the observation light L2. The imaging optical system 4 includes, for example, an objective lens 41, a band-pass filter 42, and a coupling lens 43. The objective lens 41, the band-pass filter 42, and the coupling lens 43 are arranged in this order on the observation axis P1 of the observation light L2. The observation axis P1 of the imaging optical system 4 intersects with the irradiation surface La. In this embodiment, to avoid the main body 14, the observation axis P1 is inclined with respect to the irradiation surface La of the observation object B by the planar light L1. The inclination angle θ is, for example, within a range of 10° to 80°. From the viewpoint of improving the resolution of the observation image, the inclination angle θ is preferably within a range of 20° to 70°. Furthermore, from the viewpoints of improving the resolution of the observation image and stabilizing the field of view, the inclination angle θ is more preferably within a range of 30° to 65°.

[0028] The imaging device 5 is located on the optical axis and at the imaging plane of the imaging optical system 4 and is optically coupled to the imaging optical system 4. The imaging device 5 captures an image of the observation light L2 formed by the imaging optical system 4 each time the observation object B moves a predetermined distance using the scanning unit 3. The imaging device 5 generates multiple image data D1 including optical images of the observation light L2 at different positions in the direction A1. The imaging device 5 is, for example, an area image sensor such as a CMOS image sensor or a CCD image sensor. The imaging device 5 captures the optical image using, for example, a global shutter or a rolling shutter. The imaging device 5 generates multiple image data D1, which are two-dimensional image data related to the observation object B. The imaging device 5 outputs the multiple image data D1 to the image generation unit 6.

[0029] The imaging device 5 provides the generated multiple image data D1 to the image generation unit 6. The image generation unit 6 generates observation image data including an observation image of the observation object B based on the multiple image data D1. Specifically, when the image generation unit 6 receives the multiple image data D1 output from the imaging device 5, it generates image data including a luminance distribution as observation image data of the observation object B based on the multiple image data D1. When generating the observation image data, the image generation unit 6 generates observation image data D2 as a set of the multiple image data D1, as shown in FIG. 2. This observation image data D2 contains information about the three-dimensional luminance distribution of the observation object B. Note that in addition to the movement direction A1, FIG. 2 also shows two directions A2 and A3 orthogonal to the movement direction A1. The direction A2 is, for example, the row direction in the pixel array of the imaging device 5. The direction A3 is, for example, the column direction in the pixel array of the imaging device 5.

[0030] The scanning unit 3 further includes a position information output unit 31 (information output unit). The position information output unit 31 outputs position information S, which is necessary for determining the timing of exposure start when the imaging device 5 acquires image data D1, based on the movement distance of the moving stage 33. The position information S is constantly output from the position information output unit 31 at predetermined position intervals from before the start of observation to the end of observation. From the start of observation to the end of observation, the position information output unit 31 outputs the position information S at intervals of, for example, 0.01 μm. The position information output unit 31 may include an encoder that outputs the position information S according to the movement distance of the moving stage 33. The scanning unit 3 outputs the position information S to the control unit 7.

[0031] The control unit 7 generates a control signal (trigger signal) C1 for controlling the start of exposure of the imaging device 5 based on the position information S provided by the scanning unit 3, and sends the control signal C1 to the imaging device 5. The control unit 7 outputs the control signal C1 every time the moving stage 33 moves a predetermined distance (e.g., 10 μm). In addition, the control unit 7 controls the exposure time of the imaging device 5 when acquiring image data. The control unit 7 determines the exposure time of the imaging device 5 based on the position information S.

[0032] The control unit 7 receives a user's operation to start measurement and synchronously drives the light source 21, the moving stage 33, and the imaging device 5. The control unit 7 may control the light source 21 so that the light source 21 outputs continuous light while the moving stage 33 is moving the observation object B, or may control the light output by the light source 21 to be turned on / off in accordance with the imaging by the imaging device 5. If the irradiation optical system 22 is equipped with an optical shutter (not shown), the control unit 7 may turn on / off the irradiation of the observation object B with the planar light L1 by controlling the optical shutter.

[0033] The image generation unit 6 and the control unit 7 may be configured by a computer 8. The computer 8 physically includes memory such as RAM and ROM, a processor (arithmetic circuit) such as a CPU, a communication interface, a storage unit such as a hard disk, and a display unit such as a display. The computer 8 is, for example, a personal computer, a cloud server, or a smart device (smartphone, tablet terminal, etc.). The computer 8 functions as a controller that controls the operation of the light source 21 and the moving stage 33, the image generation unit 6 that generates observation image data of the observation object B, and the control unit 7 that controls the imaging device 5 by executing a program stored in the memory on the CPU of the computer system. Note that the trigger signal generation unit in the control unit 7 that outputs a control signal C1 based on the position information S may be hardware separate from the computer 8. The trigger signal generation unit may be included in the position information output unit 31. In this case, the control signal C1, which is a trigger signal, is input to the control unit 7 and output to the imaging device 5.

[0034] Figure 3 is a timing chart showing (a) the movement speed of the moving stage 33, (b) the position in direction A1 at which the control signal C1 is output from the control unit 7, (c) the timing at which the control signal C1 is output from the control unit 7, and (d) the exposure period of the imaging device 5.

[0035] 3(a), the moving speed of the moving stage 33 gradually increases immediately after the observation device 1 starts operating, and then remains constant from the observation start time T1 to the observation end time T2. After the observation end time T2, the moving speed of the moving stage 33 gradually decreases and eventually stops.

[0036] 3(b), the control unit 7 starts outputting a control signal C1 at observation start timing T1. Between observation start timing T1 and observation end timing T2, the control unit 7 outputs a control signal C1 based on position information S every time the observation object B (more precisely, the moving stage 33) moves a predetermined distance D. The distance between the control signals C1 is constant at the predetermined distance D. The control signal C1 is, for example, a pulse signal having a rectangular time waveform.

[0037] From the observation start timing T1 to the observation end timing T2, the observation object B (more precisely, the moving stage 33) moves at a constant speed. Therefore, as shown in FIG. 3(c), the control unit 7 outputs a control signal C1 at a constant time interval T during that time. However, when multiple observation devices 1 are manufactured, the moving speed of the observation object B will differ slightly for each observation device 1 due to various factors such as variations in the performance of the drive mechanism and manufacturing errors. Therefore, the time interval T will differ slightly for each observation device 1.

[0038] FIG. 3(d) shows a case where the imaging device 5 employs a rolling shutter system. The imaging device 5 has (Q×R) pixel groups 51 arranged across Q rows and R columns (where Q and R are integers equal to or greater than 2). In other words, the first through Qth row groups each include a first through Rth column group. Each pixel group 51 may consist of only one pixel, or may consist of an N-by-N pixel array (where N is an integer equal to or greater than 2). Operating N rows by N columns as a single pixel group 51 is called a binning operation. When each pixel group 51 consists of only one column of pixels, in order to bring the resolution in direction A1 closer to the resolution in directions A2 and A3, the control unit 7 determines the timing of exposure start so that the predetermined distance D is approximately equal to the pixel size of the imaging device 5 divided by the magnification of the imaging optical system 4 (specifically, between 0.95 and 1.05). Furthermore, when the pixels constituting each pixel group 51 are arranged in N rows and N columns, the control unit 7 determines the timing of exposure start so that the predetermined distance D is N times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. If necessary, the control unit 7 may also determine the timing of exposure start so that the predetermined distance D is 1 / N times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. When the predetermined distance D is 1 / N times the divided value, the resolution in the direction A1 is higher than the resolution in the directions A2 and A3, which is advantageous when more detailed observation is desired in the direction A1. In addition, even if the exposure start timing is delayed due to some factor, such as jitter J shown in FIG. 7(b) (described later), the predetermined distance D is 1 / N times the divided value and N pieces of image data are combined into one, making it possible to precisely correct any deviation in the image data due to a deviation in the exposure start timing.

[0039] When the position information S is input to the control unit 7 and the output interval of the position information S becomes constant, the control unit 7 controls the imaging device 5 to start exposing the first row group. This exposure is performed for a predetermined exposure time Ta. Next, when a predetermined time Δt has elapsed since the start of exposure of the first row group, the control unit 7 controls the imaging device 5 to start exposing the second row group. This exposure is also performed for the predetermined exposure time Ta. Thereafter, from the third row group to the Qth row group, the control unit 7 controls the imaging device 5 to start exposure at a timing shifted by a predetermined time Δt for each row group. The exposure time Ta for each row group is constant. The exposure time Ta is determined according to the time interval of the control signal C1. In one example, the exposure time Ta is greater than or equal to 90% and less than 100% of the time interval of the control signal C1.

[0040] The above description is for the case where the imaging device 5 has a rolling shutter system, but as mentioned above, the imaging device 5 may have a global shutter system. In that case, the control unit 7 controls the imaging device 5 so that the exposure start timings of the first row group to the Qth row group are synchronized with each other.

[0041] Next, an observation method according to an embodiment of the present disclosure will be described. This observation method can be suitably carried out using, for example, the observation device 1 described above. Fig. 4 is a flowchart showing an example of the observation method. As shown in the figure, this observation method includes an image data acquisition step ST1 and an observation image data generation step ST2.

[0042] In the image data acquisition step ST1, first, the observation object B is placed in the container 11 (step ST11). Then, irradiation with planar light L1 is started (step ST12). Next, movement of the observation object B by the scanning unit 3 is started (step ST13). Then, while the light irradiation unit 2 irradiates the observation object B with planar light L1, the scanning unit 3 moves the observation object B at a constant speed along a direction A1 intersecting with the planar light L1 (step ST14). In this way, the observation object B passes through the irradiation plane La of the planar light L1. At this time, the imaging optical system 4 having an observation axis P1 intersecting with the irradiation plane La forms an image of the observation light L2 generated on the observation object B by irradiation with the planar light L1 (step ST15). Next, in order to determine the exposure start timing when the imaging device 5 acquires image data D1, a control signal C1 (trigger signal) is output from the control unit 7, for example, every time the moving stage 33 moves a predetermined distance D, based on the position information S output from the position information output unit 31 (step ST16). The imaging device 5 acquires image data D1 including the optical image of the formed observation light L2 using the imaging device 5 every time the observation object B moves the predetermined distance D (step ST17). The exposure start timing and exposure time Ta of the imaging device 5 every time the observation object B moves the predetermined distance D are controlled by the control unit 7. The control unit 7 determines the exposure start timing and exposure time Ta of the imaging device 5 based on the position information S.

[0043] In the image data acquisition step ST1, the control unit 7 determines the timing to start exposure so that the predetermined distance D is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. Alternatively, the control unit 7 determines the timing to start exposure so that the predetermined distance D is 1 / N or N times (where N is an integer greater than or equal to 2) the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. In addition, the control unit 7 determines the exposure time of the imaging device 5 based on the time interval T of the control signal C1.

[0044] In the observation image data generating step ST2, the image generating unit 6 generates observation image data D2 including an observation image of the observation object B based on the plurality of image data D1. The computer 8 including the image generating unit 6 may perform various analyses on the observation object B based on the observation image data D2, as necessary.

[0045] The effects obtained by the observation device 1 and observation method according to this embodiment described above will now be described. In the observation device 1 and observation method according to this embodiment, the start of exposure of the imaging device 5 is controlled based on the movement distance of the observation object B, not on time. As a result, even if there is variation in the movement speed of the observation object B, the exposure start timing changes in response to the variation in the movement speed, so that variation in the resolution of the observation image data D2 in the movement direction A1 is unlikely to occur. Therefore, according to this observation device 1, variation in the resolution in the movement direction A1 of the observation object B can be reduced, and the resolution of the observation image data D2 in the movement direction A1 can be accurately brought closer to the resolution in the direction perpendicular to the movement direction A1.

[0046] As in the present embodiment, the control unit 7 and the image data acquisition step ST1 may determine the timing to start exposure so that the predetermined distance D is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. In this way, by making the interval (predetermined distance D) between positions at which acquisition of image data D1 in the movement direction A1 starts equal to or close to the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4, the resolution of the observed image data D2 in the movement direction A1 can be made to accurately approach the resolution in the direction perpendicular to the movement direction A1.

[0047] As in this embodiment, the control unit 7 and the image data acquisition step ST1 may determine the timing of the exposure start so that the predetermined distance D is 1 / N or N times (where N is an integer greater than or equal to 2) the pixel size of the imaging device 5 divided by the magnification of the imaging optical system 4. When the predetermined distance D is N times the divided value, for example, when the imaging device 5 performs a binning operation in which pixels in N rows and N columns are treated as a single pixel, the resolution of the observed image data D2 in the movement direction A1 can be easily matched with the resolution in the direction perpendicular to the movement direction A1, eliminating the need for binning in the movement direction. Furthermore, when the predetermined distance D is 1 / N times the divided value, the resolution in the movement direction A1 can be increased compared to the resolution in the directions A2 and A3 perpendicular to the movement direction A1. Additionally, even if the exposure start timing is shifted for some reason, the position of the image data D1 in the movement direction A1 can be easily corrected.

[0048] As in this embodiment, the control unit 7 and the image data acquisition step ST1 may determine the exposure time Ta of the imaging device 5 when acquiring the image data D1 based on a signal from the position information output unit 31 (for example, based on the position information S). In this case, the proportion of non-exposure time is minimized as much as possible, and seamless observation image data D2 can be acquired with minimal loss of information about the observation object B.

[0049] As in the present embodiment, the control unit 7 may convert the position information S into a control signal C1, which is a trigger signal indicating the timing to start exposure, and provide the control signal C1 to the imaging device 5. Alternatively, the position information S may be converted into a control signal C1, which is a trigger signal indicating the timing to start exposure, and the control signal C1 may be provided to the imaging device 5. In this case, the exposure start timing of the imaging device 5 only needs to follow the control signal C1, and the imaging device 5 can be easily controlled.

[0050] As in this embodiment, the position information output unit 31 may include an encoder. In this case, the position information S for determining the exposure start timing of the imaging device 5 can be generated with high accuracy using a simple configuration.

[0051] Here, a configuration for acquiring seamless observation image data D2 with reduced loss of information about the observation object B in the movement direction A1 will be further described. FIGS. 5(a) and 6(a) show the position from which the control signal C1 is emitted in the movement direction A1. FIGS. 5(b) and 6(b) show the exposure interval of the imaging device 5 from one control signal C1 to the next (=1 frame). The length of the exposure interval defines the resolution Yr in the movement direction A1. The length of the exposure interval is determined by the exposure time Ta described above. In FIG. 5(b), the exposure interval coincides with the interval (predetermined distance D) between the control signals C1. In contrast, in FIG. 6(b), the exposure interval is shorter than the interval (predetermined distance D) between the control signals C1. In an imaging device 5 such as an image sensor, operations other than exposure, such as reading out the image data D1, are required within one frame. Therefore, as shown in FIG. 6(b), the exposure interval is usually shorter than the interval (predetermined distance D) between the control signals C1.

[0052] 5(c) and 6(c) show an illumination area F formed by continuing the illumination surface La of the planar light L1 in the movement direction A1, intensity profiles Lb1 of the planar light L1 at multiple positions in the movement direction A1, and a composite profile Lb2 obtained by combining the intensity profiles Lb1 over one frame. When the length of the exposure interval matches the interval between the control signals C1, as shown in FIG. 5(b), the image data D1 can include information about the entire illumination area F in one frame in the movement direction A1. However, as shown in FIG. 6(c), the exposure interval is usually shorter than the interval between the control signals C1. Because the observation object B continues to move in intervals other than the exposure interval (hereinafter referred to as the non-exposure interval), information about the illumination area F in the non-exposure interval cannot be included in the image data D1. Therefore, information about the observation object B in the movement direction A1 is lost.

[0053] FIG. 7 is a diagram illustrating an example of a non-exposure interval. In FIG. 7, (a) shows the position where the control signal C1 is issued in the movement direction A1. (b) shows the exposure period for each row group of the imaging device 5. As shown in FIG. 7, a time lag occurs due to signal processing and the like between the issuance of the control signal C1 and the start of exposure of the first row group (this lag is called jitter J). Furthermore, in the case of a rolling shutter system, exposure of a certain row group starts after a delay time H from the start of exposure of the next row group. The delay time H is, for example, 13 μs. If the minimum time interval of the control signal C1 is 275 μs, the delay time H corresponds to 4.75% of that. The jitter J and delay time H also contribute to the non-exposure interval.

[0054] Below, the degree of information loss in the movement direction A1 will be quantitatively evaluated. Figures 8(a) to 8(f) and 9(a) to 9(f) are graphs showing the composite profile Lb2 obtained when the FWHM (full width at half maximum) ratio of the intensity profile Lb1 of the planar light L1 in the movement direction A1 and the ratio of the exposure time Ta are varied, with one frame being 100%. Figures 8(a) to 8(c) show the results when the FWHM ratio is 10%. Figures 8(d) to 8(f) show the results when the FWHM ratio is 50%. Figures 9(a) to 9(c) show the results when the FWHM ratio is 100%. Figures 9(d) to 9(f) show the results when the FWHM ratio is 200%. Also, Figures 8(a), 8(d), 9(a), and 9(d) show the results when the exposure time Ta ratio is 100%. Figures 8(b), 8(e), 9(b), and 9(e) show the case where the ratio of the exposure time Ta is 80%. Figures 8(c), 8(f), 9(c), and 9(f) show the case where the ratio of the exposure time Ta is 50%. The degree of information loss in real space within a pixel when the information distribution within one frame is quantized (imaged) is estimated from the composite profile Lb2. In the following explanation, the degree of information loss is defined by four indices: the information acquisition rate, the minimum value of the information gradation, the missing area within the observation area, and the amount of information as a three-dimensional data set.

[0055] [Information acquisition rate] The information acquisition rate refers to the ratio of the information in the imaged pixel to the actual information in the real space corresponding to the pixel. Specifically, the integral value of the composite profile Lb2 corresponds to the information in the imaged pixel. If the product of the maximum value of the composite profile Lb2 and a predetermined distance D is the actual information in the real space corresponding to the pixel, the ratio of the information in the imaged pixel can be found by calculating the ratio of the integral value of the composite profile Lb2 to that product.

[0056] Fig. 10(a) is a graph showing the ratio of information within an imaged pixel as shades of color, with the FWHM ratio on the vertical axis and the exposure time Ta ratio on the horizontal axis. Fig. 10(b) is a contour map of Fig. 10(a), showing 0% to 90% in 10% increments. Fig. 10(c) is a contour map of Fig. 10(a), showing 90% to 98% in 1% increments. Figs. 11(a) to 11(c) are enlarged graphs showing the portions of Figs. 10(a) to 10(c) where the exposure time Ta ratio is 95% to 100%, respectively.

[0057] [Minimum value of information gradient] The minimum value of the information gradation (gradient) refers to the minimum value of the spatial gradient of the information within a pixel. For example, in the graph shown in FIG. 8(a), point G1 is the minimum value of the gradation. FIG. 12(a) is a graph showing the minimum value of the information gradation (gradient) as a shade of color, with the FWHM ratio on the vertical axis and the exposure time Ta ratio on the horizontal axis. FIG. 12(b) is a contour map of FIG. 12(a), showing 0% to 90% in 10% increments. FIG. 12(c) is a contour map of FIG. 12(a), showing 70% to 90% in 5% increments. FIGS. 13(a) to 13(c) are enlarged graphs showing the portions of FIGS. 12(a) to 12(c) where the exposure time Ta ratio is 95% to 100%, respectively.

[0058] [Defective area within the observation area] A defective area within an observation area refers to an area that is not included in a pixel with respect to the actual information in real space corresponding to the pixel. Here, a defective area is defined as an area where the ratio of the composite profile to the maximum value is 10% or less. For example, in the graph shown in Figure 8(c), area G2 is a defective area. Figure 14(a) is a graph showing the ratio of the defective area with the FWHM ratio on the vertical axis and the exposure time Ta ratio on the horizontal axis, and the color shading used to represent the ratio of the defective area. Figure 14(b) is a contour map of Figure 14(a), showing 0% to 90% in 10% increments. Figure 14(c) is a contour map of Figure 14(a), showing 50% to 90% in 5% increments. Figures 15(a) to 15(c) are enlarged graphs showing the portions of Figures 14(a) to 14(c) where the ratio of the exposure time Ta is 95% to 100%, respectively.

[0059] [Amount of information as a three-dimensional data set] The amount of information in a three-dimensional data set refers to the amount of real-space information contained in the three-dimensional image when the entire measurement target (three-dimensional space) is three-dimensionally visualized. In other words, the amount of information in a three-dimensional data set refers to the sum of the information that can be acquired as information within a pixel and the information contained in other pixels. For example, in the graph shown in FIG. 8(a), the sum of the composite profile Lb2 present within the graph and the tail portion of the composite profile Lb2 present outside the graph corresponds to the amount of information in the three-dimensional data set.

[0060] Fig. 16(a) is a graph showing the amount of information as a three-dimensional data set using color shading, with the FWHM ratio on the vertical axis and the exposure time Ta ratio on the horizontal axis. Fig. 16(b) is a contour map of Fig. 16(a), showing 0% to 100% in 10% increments. Fig. 16(c) is a contour map of Fig. 16(a), showing 100% to 200% in 10% increments. Figs. 17(a) to 17(c) are enlarged graphs showing the portions of Figs. 16(a) to 16(c) where the exposure time Ta ratio is 95% to 100%, respectively.

[0061] [Overall evaluation of four indicators] FIG. 18 is a diagram showing the above-mentioned FIGS. 10(b), 12(b), 14(b), and 16(b) arranged side by side. FIG. 19 is a diagram showing the above-mentioned FIGS. 11(b), 13(b), 15(b), and 17(b) arranged side by side. FIG. 20 is a diagram showing the above-mentioned FIGS. 11(b), 13(b), 15(b), and 17(c) arranged side by side. Referring to these diagrams, it can be seen that when the FWHM ratio is 130% or more and 160% or less, the four indices reach desirable values, and the degree of information loss in the movement direction A1 is reduced. In other words, it is preferable that the FWHM of the light intensity distribution of the planar light L1 in the direction perpendicular to the irradiation surface La is 130% or more and 160% or less of the predetermined distance D. This makes it possible to acquire seamless observation image data with minimal loss of information about the observation object B.

[0062] The observation device and observation method according to the present disclosure are not limited to the above-described embodiment, and various other modifications are possible. For example, the control signal C1 output from the control unit 7 is not limited to a trigger signal, and may be various information that contributes to determining the timing of starting exposure. [Explanation of symbols]

[0063] 1...observation device, 2...light irradiation unit, 3...scanning unit, 4...imaging optical system, 5...imaging device, 6...image generation unit, 7...control unit, 8...computer, 11...container, 13...well, 14...main body, 15...transparent member, 15a...input surface, 21...light source, 22...irradiation optical system, 31...position information output unit (information output unit), 32...holding unit, 33...moving stage, 41...objective lens, 42...bandpass filter, 43...coupling lens, 51...pixel group, A1...moving direction, A2, A3...direction , B...object to be observed, C1...control signal, D...predetermined distance, D1...image data, D2...observation image data, F...irradiation area, G1...point, G2...area, H...delay time, J...jitter, L1...plane light, L2...observation light, La...irradiation surface, Lb1...intensity profile, Lb2...synthetic profile, P1...observation axis, S...position information, T...time interval, T1...observation start timing, T2...observation end timing, Ta...exposure time, Yr...resolution, Δt...predetermined time, θ...tilt angle.

Claims

1. an illumination optical system that illuminates the observation object with planar light; a scanning unit that moves the observation object at a constant speed in a direction intersecting the planar light, causing the observation object to pass through an irradiation surface of the planar light; an imaging optical system having an observation axis intersecting the irradiation surface and configured to form an image of observation light generated on the observation object by irradiation with the planar light; an imaging device that acquires image data including an optical image of the observation light formed by the imaging optical system every time the observation object moves a predetermined distance; an image generating unit that generates observation image data including an observation image of the observation object based on the plurality of image data; A control unit; Equipped with the scanning unit has an information output unit that outputs information for determining timing of starting exposure when the imaging device acquires the image data based on a moving distance of the observation object, The control unit controls the start of exposure of the imaging device based on the information.

2. 2. The observation device according to claim 1, wherein the control unit determines the timing of starting the exposure so that the predetermined distance is 0.95 to 1.05 times a value obtained by dividing a pixel size of the imaging device by a magnification of the imaging optical system.

3. 2. The observation device according to claim 1, wherein the control unit determines the timing of starting the exposure so that the predetermined distance is 1 / N times or N times (where N is an integer equal to or greater than 2) a value obtained by dividing a pixel size of the imaging device by a magnification of the imaging optical system.

4. 4. The observation device according to claim 1, wherein the control unit determines an exposure time of the imaging device when acquiring the image data based on the information.

5. 4. The observation device according to claim 1, wherein the FWHM of the light intensity distribution of the planar light in a direction perpendicular to the irradiation surface is 130% or more and 160% or less of the predetermined distance.

6. 4. The observation device according to claim 1, wherein the control unit converts the information into a trigger signal indicating the timing of starting the exposure, and provides the trigger signal to the imaging device.

7. 4. The observation device according to claim 1, wherein the information output unit includes an encoder.

8. a step of irradiating an object to be observed with planar light, moving the object to be observed at a constant speed in a direction intersecting the planar light, causing the object to pass through a surface irradiated with the planar light, and forming an image of the observation light generated on the object to be observed by the irradiation of the planar light using an imaging optical system having an observation axis intersecting the surface irradiated, and acquiring image data including an optical image of the formed observation light using an imaging device every time the object to be observed moves a predetermined distance; generating observation image data including an observation image of the observation object based on the plurality of image data; Including, the acquiring step includes a step of outputting information for determining timing of starting exposure when the imaging device acquires the image data, based on a movement distance of the observation object; In the acquiring step, the start of exposure of the imaging device is controlled based on the information.

9. 9. The observation method according to claim 8, wherein in the acquiring step, the timing of starting the exposure is determined so that the predetermined distance is 0.95 to 1.05 times a value obtained by dividing a pixel size of the imaging device by a magnification of the imaging optical system.

10. 9. The observation method according to claim 8, wherein in the acquiring step, the timing of starting the exposure is determined so that the predetermined distance is 1 / N times or N times (where N is an integer of 2 or more) a value obtained by dividing a pixel size of the imaging device by a magnification of the imaging optical system.

11. 11. The observation method according to claim 8, wherein in the acquiring step, an exposure time of the imaging device when acquiring the image data is determined based on the information.

12. 11. The observation method according to claim 8, wherein the FWHM of the light intensity distribution of the planar light in the direction perpendicular to the irradiation surface is 130% or more and 160% or less of the predetermined distance.

13. 11. The observation method according to claim 8, wherein the information is converted into a trigger signal indicating the timing of starting the exposure, and the trigger signal is provided to the imaging device.

Images