Observation apparatus and observation method

Abstract

An observation apparatus comprises an irradiation optical system 22, a scanning unit 3, an image-forming optical system 4, and an imaging device 5. The irradiation optical system 22 irradiates an observation object B with planar light L1. The scanning unit 3 moves the observation object B at a constant speed in a direction crossing the planar light L1, and allows the observation object B to pass through the irradiation surface La of the planar light L1. The image-forming optical system 4 has an observation axis P1 crossing the irradiation surface La, and forms an image from observation light L2 generated at the observation object B by irradiation with the planar light L1. Every time the observation object B moves by a prescribed distance, the imaging device 5 acquires image data D1 including an optical image of the observation light L2. The scanning unit 3 outputs, on the basis of the moving distance of a moving stage 33, position information S for determining the exposure start timing when the imaging device 5 acquires the image data D1.

Description

Observation device and observation method

[0001] This application claims priority to Japanese Patent Application No. 2024-110049, filed on July 9, 2024, and incorporates by reference all of the contents of said Japanese application.

[0002] Patent Document 1 discloses a sample observation device and a sample observation method. The sample observation device includes an illumination optical system, a scanning unit, an imaging optical system, an image acquisition unit, and an image generation unit. The illumination optical system irradiates the sample with planar light in the XZ plane. The scanning unit scans the sample in the Y axis direction so that the sample passes through the surface irradiated with the planar light. The imaging optical system has an observation axis tilted with respect to the irradiated surface, and forms an image of the observation light generated in the sample by the illumination with the planar light. The image acquisition unit acquires multiple XZ image data corresponding to the optical image of the observation light formed by the imaging optical system. The image generation unit generates observation image data of the sample based on the multiple XZ image data acquired by the image acquisition unit.

[0003] International Publication No. 2021 / 256429

[0004] As one of the techniques for observing the interior of an object having a three-dimensional structure such as a cell, a light sheet microscope using selective plane illumination microscopy (SPIM) is known (see, for example, Patent Document 1). The light sheet microscope makes it possible to obtain information about the interior of an object while sufficiently reducing the influence of variations in the brightness value of background light.

[0005] In a light sheet microscope, an object to be observed is moved at a constant speed in a direction intersecting the planar light, and the object passes through the plane irradiated with the planar light. Image data, including an optical image of the observation light generated on the object to be observed by the planar light, is acquired each time the object to be observed moves a predetermined distance. However, if the timing of image data acquisition is controlled by time (in other words, if image data is acquired at regular intervals), variations in the movement speed of the object to be observed also result in variations in the resolution in the movement direction (the Y-axis direction in Patent Document 1). As a result, there is a risk that the resolution of the observation image data in the movement direction will significantly deviate from the resolution in the direction perpendicular to the movement direction.

[0006] An object of the present disclosure is to provide an observation device and an observation method that can reduce variations in resolution in the movement direction of an observation object.

[0007] [1] An observation device according to an embodiment of the present disclosure includes an illumination optical system, a scanning unit, an imaging optical system, an imaging device, an image generation unit, and a control unit. The illumination optical system irradiates an observation object with planar light. The scanning unit moves the observation object at a constant speed in a direction intersecting the planar light, causing the observation object to pass through the illumination plane of the planar light. The imaging optical system has an observation axis intersecting the illumination plane, and forms an image of the observation light generated on the observation object by the illumination of the planar light. The imaging device acquires image data including an optical image of the observation light formed by the imaging optical system every time the observation object moves a predetermined distance. The image generation unit generates observation image data including an observation image of the observation object based on the multiple image data. The scanning unit has an information output unit that outputs information for determining the timing of exposure start when the imaging device acquires image data, based on the movement distance of the observation object. The control unit controls the start of exposure of the imaging device based on the information. An observation method according to an embodiment of the present disclosure includes the steps of: irradiating an observation object with planar light, moving the observation object at a constant speed in a direction intersecting the planar light, causing the observation object to pass through a plane irradiated with the planar light, and using an imaging optical system having an observation axis intersecting the irradiated plane to form an image of the observation light generated on the observation object by the irradiation of the planar light, and acquiring image data including an optical image of the formed observation light using an imaging device every time the observation object moves a predetermined distance; and generating observation image data including an observation image of the observation object based on the plurality of image data. The acquiring step includes a step of outputting information for determining timing of exposure start when the imaging device acquires the image data based on the movement distance of the observation object. In the acquiring step, the start of exposure of the imaging device is controlled based on the information.

[0008] In the observation device and observation method described above, the start of exposure of the imaging device is controlled based on the movement distance of the observed object, not on time. As a result, even if there is variation in the movement speed of the observed object, the exposure start timing changes in response to the variation in the movement speed, making it less likely that variation in resolution will occur in the movement direction. Therefore, with this observation device, it is possible to reduce the variation in resolution in the movement direction of the observed object, and accurately bring the resolution in the movement direction of the observed image data closer to the resolution in the direction perpendicular to the movement direction.

[0009] [2] In the observation device of [1] above, the control unit may determine the timing of starting exposure so that the predetermined distance is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system. Similarly, in the observation method of [1] above, the acquiring step may determine the timing of starting exposure so that the predetermined distance is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system. In this way, by having the interval (predetermined distance) between positions at which image data acquisition in the movement direction starts match or is close to the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system, the resolution of the observation image data in the movement direction can be accurately brought close to the resolution in a direction perpendicular to the movement direction.

[0010] [3] In the observation device of [1] above, the control unit may determine the timing of exposure start so that the predetermined distance is 1 / N or N times (where N is an integer greater than or equal to 2) the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system. Similarly, in the observation method of [1] above, the acquiring step may determine the timing of exposure start so that the predetermined distance is 1 / N or N times (where N is an integer greater than or equal to 2) the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system. If the predetermined distance is N times the divided value, for example, when the imaging device performs a binning operation in which pixels in N rows and N columns are treated as a single pixel, the resolution of the observation image data in the movement direction can be easily matched with the resolution in the direction perpendicular to the movement direction, thereby eliminating the need for binning in the movement direction. Alternatively, if the predetermined distance is 1 / N times the divided value, the resolution in the movement direction can be increased compared to the resolution in the direction perpendicular to the movement direction. In addition, even if the exposure start timing is shifted for some reason, the position of the image data in the movement direction can be easily corrected.

[0011] [4] In the observation device according to any one of [1] to [3] above, the control unit may determine the exposure time of the imaging device when acquiring image data based on the information. Similarly, in the observation method according to any one of [1] to [3] above, the acquiring step may determine the exposure time of the imaging device when acquiring image data based on the information. In this case, the proportion of non-exposure time is minimized, and seamless observation image data can be acquired with minimal loss of information about the object being observed.

[0012] [5] In the observation device and observation method according to [1] to [4] above, the FWHM (full width at half maximum) of the light intensity distribution of the planar light in a direction perpendicular to the irradiation surface may be 130% or more and 160% or less of the predetermined distance. In this case, seamless observation image data can be acquired with minimal loss of information about the object being observed.

[0013] [6] In the observation device of any one of [1] to [5] above, the control unit may convert the information into a trigger signal indicating the timing of exposure start, and provide the trigger signal to the imaging device. In the observation method of any one of [1] to [5] above, the information may be converted into a trigger signal indicating the timing of exposure start, and the trigger signal may be provided to the imaging device. In this case, the exposure start timing of the imaging device only needs to follow the trigger signal, and control of the imaging device can be simplified.

[0014] [7] In the observation device according to any one of [1] to [6] above, the information output unit may include an encoder. In this case, information for determining the exposure start timing of the imaging device can be generated with high accuracy using a simple configuration.

[0015] According to the present disclosure, it is possible to provide an observation device and an observation method that can reduce variations in resolution in the movement direction of an observation object.

[0016] FIG. 1 is a diagram schematically illustrating the configuration of an observation device according to an embodiment of the present disclosure. FIG. 2 is a diagram conceptually illustrating observation image data including a plurality of image data. FIG. 3 is a timing chart illustrating the relationship between the movement speed of a moving stage, the position and timing of the moving stage at which a control signal is output from a control unit, and the exposure period of an imaging device. FIG. 4 is a flowchart illustrating an example of an observation method. FIG. 5 is a diagram illustrating the relationship between the position from which position information is emitted in the movement direction, the exposure period of the imaging device within one frame, the intensity profile of planar light, and the composite profile. FIG. 6 is a diagram illustrating the relationship between the position from which position information is emitted in the movement direction, the exposure period of the imaging device within one frame, the intensity profile of planar light, and the composite profile. FIG. 7 is a diagram illustrating an example in which a non-exposure period occurs. FIG. 8 is a graph illustrating composite profiles obtained by varying the FWHM ratio of the intensity profile of planar light in the movement direction and the exposure time ratio, with one frame being 100%. FIG. 9 is a graph showing a composite profile when the FWHM ratio of the intensity profile of planar light in the movement direction and the exposure time ratio are variously changed, with one frame being 100%. FIG. 10 is a graph and its contour map showing the ratio of information within an imaged pixel in color shading, with the FWHM ratio on the vertical axis and the exposure time ratio on the horizontal axis. FIG. 11 is a graph showing an enlarged view of the portion of FIG. 10 where the exposure time ratio is 95% to 100%. FIG. 12 is a graph and its contour map showing the minimum value of the information gradation (gradient) in color shading, with the FWHM ratio on the vertical axis and the exposure time ratio on the horizontal axis. FIG. 13 is a graph showing an enlarged view of the portion of FIG. 12 where the exposure time ratio is 95% to 100%. FIG. 14 is a graph and its contour map showing the ratio of missing areas in color shading, with the FWHM ratio on the vertical axis and the exposure time ratio on the horizontal axis. Fig. 15 is a graph showing an enlarged view of the portion of Fig. 14 where the exposure time ratio is 95% to 100%. Fig. 16 is a graph showing the amount of information as a three-dimensional data set in shades of color, with the FWHM ratio on the vertical axis and the exposure time ratio on the horizontal axis, and a contour map of the graph. Fig. 17 is a graph showing an enlarged view of the portion of Fig. 16 where the exposure time ratio is 95% to 100%.Fig. 18 is a diagram showing part (b) of Fig. 10, part (b) of Fig. 12, part (b) of Fig. 14, and part (b) of Fig. 16 arranged side by side. Fig. 19 is a diagram showing part (b) of Fig. 11, part (b) of Fig. 13, part (b) of Fig. 15, and part (b) of Fig. 17 arranged side by side. Fig. 20 is a diagram showing part (b) of Fig. 11, part (b) of Fig. 13, part (b) of Fig. 15, and part (c) of Fig. 17 arranged side by side.

[0017] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicated explanations will be omitted.

[0018] 1 is a diagram schematically illustrating the configuration of an observation device 1 according to an embodiment of the present disclosure. The observation device 1 irradiates an observation object B with planar light (sheet light) L1, and forms an image of the observation light generated inside the observation object B on an imaging plane to acquire observation image data of the inside of the observation object B. The observation light is, for example, fluorescent light or scattered light. The observation device 1 is, for example, a slide scanner that acquires and displays an image of the observation object B held on a slide glass, or a plate reader that acquires and analyzes image data of the observation object B held on a microplate.

[0019] Observation object B is, for example, a human or animal cell, tissue, or organ, an animal or plant itself, or a plant cell or tissue. Observation object B is stained with a fluorescent material such as fluorescein-dextran or tetramethylrhodamine. Fluorescein-dextran has an excitation wavelength of 494 nm and a fluorescence wavelength of 521 nm. Tetramethylrhodamine has an excitation wavelength of 555 nm and a fluorescence wavelength of 580 nm. Observation object B may be stained with multiple fluorescent substances. Observation object B may be contained in a solution, a gel, or a substance with a refractive index different from that of observation object B.

[0020] As shown in FIG. 1, the observation device 1 includes a light irradiation unit 2, a scanning unit 3, an imaging optical system 4, an imaging device 5, an image generation unit 6, and a control unit 7.

[0021] The light irradiation unit 2 includes a light source 21 and an irradiation optical system 22. The light source 21 is a laser light source such as a laser diode or a solid-state laser light source, or may be a light-emitting diode, a superluminescent diode, or a lamp-based light source.

[0022] The illumination optical system 22 is optically coupled to the light source 21. Light output from the light source 21 is guided to the illumination optical system 22. The illumination optical system 22 shapes the light output from the light source 21 into planar light L1 and irradiates the observation object B with the planar light L1. In the following description, the optical axis of the illumination optical system 22 may be referred to as the optical axis of the planar light L1. The illumination optical system 22 includes a light shaping element such as a cylindrical lens, an axicon lens, or a spatial light modulator. The illumination optical system 22 may also include an objective lens. The planar light L1 formed by the illumination optical system 22 is irradiated onto the observation object B. In consideration of the resolution in a direction A1 perpendicular to the planar light L1, the planar light L1 is preferably thin, such that the FWHM thickness of the light intensity is 2 mm or less. On the observation object B irradiated with the planar light L1, observation light L2 is generated at an irradiation surface La of the planar light L1. The observation light L2 is, for example, fluorescent light excited by the planar light L1, scattered light of the planar light L1, or diffusely reflected light of the planar light L1.

[0023] The scanning unit 3 is a mechanism that scans the observation object B with respect to the irradiation surface La of the planar light L1. The scanning unit 3 moves the observation object B at a constant speed along a direction A1 intersecting the planar light L1, and causes the observation object B to pass through the irradiation surface La of the planar light L1. In the present disclosure, a "constant speed" is considered to be one in which the variation from the target speed is within ±10%. The variation from the target speed is more preferably within ±5%.

[0024] In this embodiment, the scanning unit 3 has a holder 32 that holds the container 11 and a moving stage 33 that moves the container 11 while supporting the holder 32. The container 11 is, for example, a microplate, a slide glass, a petri dish, or the like, and is transparent to the planar light L1 and the observation light L2. In this embodiment, a microplate is exemplified. The container 11 has a frame-shaped main body 14 that forms a well 13 in which the observation object B is placed, and a plate-shaped transparent member 15 that is provided on one surface of the main body 14 so as to close one end of the well 13. The main body 14 may form a plurality of wells 13 arranged one-dimensionally or two-dimensionally.

[0025] The well 13 is filled with a culture medium, a fluorescent indicator, a buffer, or other solution together with the observation object B. Autofluorescence is emitted from the solution. The transparent member 15 has an input surface 15a for the planar light L1 toward the observation object B placed in the well 13. The material of the transparent member 15 is not particularly limited as long as it is a material that is transparent to the planar light L1 and the observation light L2, and may be, for example, glass, quartz, or synthetic resin. The container 11 is disposed so that the input surface 15a is perpendicular to the optical axis of the planar light L1. The other end of the well 13 is open to the outside. The container 11 may be fixed to a moving stage 33.

[0026] The moving stage 33 moves the container 11 along a preset direction A1 in accordance with a control signal from the control unit 7. In this embodiment, the direction A1 is one direction in a plane perpendicular to the irradiation surface La of the planar light L1.

[0027] The imaging optical system 4 is an optical system that forms an image of the observation light L2 generated on the observation object B by irradiation with the planar light L1. The optical axis of the imaging optical system 4 coincides with the observation axis P1 of the observation light L2. The imaging optical system 4 includes, for example, an objective lens 41, a bandpass filter 42, and a coupling lens 43. The objective lens 41, the bandpass filter 42, and the coupling lens 43 are arranged in this order on the observation axis P1 of the observation light L2. The observation axis P1 of the imaging optical system 4 intersects with the irradiation plane La. In this embodiment, to avoid the main body 14, the observation axis P1 is inclined with respect to the irradiation plane La of the observation object B by the planar light L1. The inclination angle θ is, for example, within a range of 10° to 80°. From the viewpoint of improving the resolution of the observation image, the inclination angle θ is preferably within a range of 20° to 70°. From the viewpoint of improving the resolution of the observation image and stabilizing the field of view, the inclination angle θ is more preferably within a range of 30° to 65°.

[0028] The imaging device 5 is located on the optical axis and at the imaging plane of the imaging optical system 4 and is optically coupled to the imaging optical system 4. The imaging device 5 captures the observation light L2 formed by the imaging optical system 4 each time the observation object B moves a predetermined distance using the scanning unit 3. The imaging device 5 generates multiple pieces of image data D1 including optical images of the observation light L2 at different positions in the direction A1. The imaging device 5 is, for example, an area image sensor such as a CMOS image sensor or a CCD image sensor. The imaging device 5 captures the optical image using, for example, a global shutter or a rolling shutter. The imaging device 5 generates multiple pieces of image data D1, which are two-dimensional image data related to the observation object B. The imaging device 5 outputs the multiple pieces of image data D1 to the image generation unit 6.

[0029] The imaging device 5 provides the generated plurality of image data D1 to the image generation unit 6. The image generation unit 6 generates observation image data including an observation image of the observation object B based on the plurality of image data D1. Specifically, when the image generation unit 6 receives the plurality of image data D1 output from the imaging device 5, it generates image data including a luminance distribution as observation image data of the observation object B based on the plurality of image data D1. When generating the observation image data, the image generation unit 6 generates observation image data D2 as a set of the plurality of image data D1, as shown in FIG. 2. This observation image data D2 contains three-dimensional luminance distribution information of the observation object B. In addition to direction A1, FIG. 2 also shows two directions A2 and A3 orthogonal to direction A1. Direction A2 is, for example, the row direction in the pixel array of the imaging device 5. Direction A3 is, for example, the column direction in the pixel array of the imaging device 5.

[0030] The scanning unit 3 further includes a position information output unit 31 (information output unit). The position information output unit 31 outputs position information S, which is necessary for determining the timing of exposure start when the imaging device 5 acquires image data D1, based on the movement distance of the moving stage 33. The position information S is constantly output from the position information output unit 31 at predetermined position intervals from before the start of observation to the end of observation. From the start of observation to the end of observation, the position information output unit 31 outputs the position information S at intervals of, for example, 0.01 μm. The position information output unit 31 may include an encoder that outputs the position information S according to the movement distance of the moving stage 33. The scanning unit 3 outputs the position information S to the control unit 7.

[0031] The control unit 7 generates a control signal C1 (trigger signal) for controlling the start of exposure of the imaging device 5 based on the position information S provided by the scanning unit 3, and sends the control signal C1 to the imaging device 5. The control unit 7 outputs the control signal C1 every time the moving stage 33 moves a predetermined distance (e.g., 10 μm). In addition, the control unit 7 controls the exposure time of the imaging device 5 when acquiring image data. The control unit 7 determines the exposure time of the imaging device 5 based on the position information S.

[0032] The control unit 7 receives a user's operation to start measurement and synchronously drives the light source 21, the moving stage 33, and the imaging device 5. The control unit 7 may control the light source 21 so that the light source 21 outputs continuous light while the observation object B is being moved by the moving stage 33, or may control the light output by the light source 21 to be turned on / off in accordance with the imaging by the imaging device 5. If the irradiation optical system 22 is equipped with an optical shutter (not shown), the control unit 7 may turn on / off the irradiation of the observation object B with the planar light L1 by controlling the optical shutter.

[0033] The image generation unit 6 and the control unit 7 may be configured by a computer 8. The computer 8 physically includes memory such as RAM and ROM, a processor (arithmetic circuit) such as a CPU, a communication interface, a storage unit such as a hard disk, and a display unit such as a display. The computer 8 is, for example, a personal computer, a cloud server, or a smart device (smartphone, tablet terminal, etc.). The computer 8 functions as a controller that controls the operation of the light source 21 and the moving stage 33, the image generation unit 6 that generates observation image data of the observation object B, and the control unit 7 that controls the imaging device 5 by executing a program stored in the memory on the CPU of the computer system. A trigger signal generation unit in the control unit 7 that outputs a control signal C1 based on position information S may be hardware separate from the computer 8. The trigger signal generation unit may be included in the position information output unit 31. In this case, the control signal C1, which is a trigger signal, is input to the control unit 7 and output to the imaging device 5.

[0034] Figure 3 is a timing chart showing (a) the movement speed of the moving stage 33, (b) the position in direction A1 at which the control signal C1 is output from the control unit 7, (c) the timing at which the control signal C1 is output from the control unit 7, and (d) the exposure period of the imaging device 5.

[0035] 3(a), the moving speed of the moving stage 33 gradually increases immediately after the observation device 1 starts operating, and then remains constant from the observation start timing T1 to the observation end timing T2. After the observation end timing T2, the moving speed of the moving stage 33 gradually decreases and eventually stops.

[0036] 3(b), the control unit 7 starts outputting a control signal C1 at observation start timing T1. Between observation start timing T1 and observation end timing T2, the control unit 7 outputs a control signal C1 based on the position information S every time the observation object B (more precisely, the moving stage 33) moves a predetermined distance D. The distance between the control signals C1 is constant at the predetermined distance D. The control signal C1 is, for example, a pulse signal having a rectangular time waveform.

[0037] From the observation start timing T1 to the observation end timing T2, the observation object B (more precisely, the moving stage 33) moves at a constant speed. Therefore, as shown in part (c) of Figure 3, the control unit 7 outputs a control signal C1 at regular time intervals T during that time. However, when multiple observation devices 1 are manufactured, the moving speed of the observation object B will differ slightly for each observation device 1 due to various factors such as variations in the performance of the drive mechanism and manufacturing errors. Therefore, the time interval T will differ slightly for each observation device 1.

[0038] Part (d) of FIG. 3 illustrates a case where the imaging device 5 employs a rolling shutter system. Here, the imaging device 5 has (Q×R) pixel groups 51 spanning Q row groups and R column groups, where Q and R are integers greater than or equal to 2. In other words, the first through Qth row groups each include the first through Rth column groups. Each pixel group 51 may consist of only one pixel, or may consist of an N-row by N-column pixel array, where N is an integer greater than or equal to 2. Operating N-row by N-column pixels as a single pixel group 51 is called a binning operation. When each pixel group 51 consists of only one column of pixels, in order to bring the resolution in direction A1 closer to the resolution in directions A2 and A3, the control unit 7 determines the timing of exposure start so that the predetermined distance D is approximately equal to the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4, specifically, for example, 0.95 to 1.05 times. When the pixels constituting each pixel group 51 are arranged in N rows and N columns, the control unit 7 determines the timing of exposure start so that the predetermined distance D is N times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. If necessary, the control unit 7 may also determine the timing of exposure start so that the predetermined distance D is 1 / N times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. When the predetermined distance D is 1 / N times the divided value, the resolution in direction A1 is higher than the resolution in directions A2 and A3, which is advantageous when more detailed observation is desired in direction A1. In addition, even if the exposure start timing is delayed due to some factor such as jitter J shown in part (b) of FIG. 7 (described later), since the predetermined distance D is 1 / N times the divided value and the N pieces of image data are combined into one, it is possible to precisely correct any deviation in the image data due to a deviation in the exposure start timing.

[0039] When the position information S is input to the control unit 7 and the output interval of the position information S becomes constant, the control unit 7 controls the imaging device 5 to first start exposing the first row group. This exposure is performed for a predetermined exposure time Ta. Next, when a predetermined time Δt has elapsed after the start of exposure of the first row group, the control unit 7 controls the imaging device 5 to start exposing the second row group. This exposure is also performed for the predetermined exposure time Ta. Thereafter, the control unit 7 controls the imaging device 5 to start exposure for each row group, from the third row group to the Qth row group, with the timing shifted by the predetermined time Δt for each row group. The exposure time Ta for each row group is constant. The exposure time Ta is determined according to the time interval of the control signal C1. In one example, the exposure time Ta is greater than or equal to 90% and less than 100% of the time interval of the control signal C1.

[0040] The above description is for the case where the imaging device 5 has a rolling shutter system, but as mentioned above, the imaging device 5 may have a global shutter system. In that case, the control unit 7 controls the imaging device 5 so that the exposure start timings of the first row group to the Qth row group are synchronized with each other.

[0041] Next, an observation method according to an embodiment of the present disclosure will be described. This observation method can be suitably carried out using, for example, the observation device 1 described above. Fig. 4 is a flowchart showing an example of the observation method. As shown in the figure, this observation method includes an image data acquisition step ST1 and an observation image data generation step ST2.

[0042] In the image data acquisition step ST1, first, the observation object B is placed in the container 11 (step ST11). Then, irradiation with planar light L1 begins (step ST12). Furthermore, movement of the observation object B by the scanning unit 3 begins (step ST13). Then, while the light irradiation unit 2 irradiates the observation object B with planar light L1, the scanning unit 3 moves the observation object B at a constant speed along a direction A1 intersecting with the planar light L1 (step ST14). In this way, the observation object B passes through the irradiation plane La of the planar light L1. At this time, the imaging optical system 4, having an observation axis P1 intersecting with the irradiation plane La, forms an image of the observation light L2 generated on the observation object B by irradiation with the planar light L1 (step ST15). Next, in order to determine the exposure start timing when the imaging device 5 acquires image data D1, a control signal C1 (trigger signal) is output from the control unit 7, for example, each time the moving stage 33 moves a predetermined distance D, based on the position information S output from the position information output unit 31 (step ST16). The imaging device 5 acquires image data D1 including the optical image of the formed observation light L2 using the imaging device 5 each time the observation object B moves the predetermined distance D (step ST17). The exposure start timing and exposure time Ta of the imaging device 5 each time the observation object B moves the predetermined distance D are controlled by the control unit 7. The control unit 7 determines the exposure start timing and exposure time Ta of the imaging device 5 based on the position information S.

[0043] In the image data acquisition step ST1, the control unit 7 determines the timing of starting exposure so that the predetermined distance D is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. Alternatively, the control unit 7 determines the timing of starting exposure so that the predetermined distance D is 1 / N or N times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4, where N is an integer greater than or equal to 2. In addition, the control unit 7 determines the exposure time of the imaging device 5 based on the time interval T of the control signal C1.

[0044] In the observation image data generating step ST2, the image generating unit 6 generates, based on the plurality of image data D1, observation image data D2 including an observation image of the observation object B. The computer 8 including the image generating unit 6 may perform various analyses on the observation object B based on the observation image data D2, as necessary.

[0045] The effects obtained by the observation device 1 and observation method according to this embodiment described above will now be described. In the observation device 1 and observation method according to this embodiment, the start of exposure of the imaging device 5 is controlled based on the movement distance of the observation object B rather than on time. As a result, even if there is variation in the movement speed of the observation object B, the exposure start timing changes in response to that variation in movement speed, so that variation in the resolution of the observation image data D2 in the movement direction A1 is less likely to occur. Therefore, with this observation device 1, variation in the resolution of the observation object B in the movement direction A1 can be reduced, and the resolution of the observation image data D2 in the movement direction A1 can be accurately brought closer to the resolution in the direction perpendicular to the movement direction A1.

[0046] As in the present embodiment, the control unit 7 and the image data acquisition step ST1 may determine the timing of starting exposure so that the predetermined distance D is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. In this way, by making the interval (predetermined distance D) between positions at which acquisition of image data D1 in the movement direction A1 starts equal to or close to the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4, the resolution of the observation image data D2 in the movement direction A1 can be made to accurately approach the resolution in the direction perpendicular to the movement direction A1.

[0047] As in this embodiment, the control unit 7 and the image data acquisition step ST1 may determine the timing of the exposure start so that the predetermined distance D is 1 / N or N times the value obtained by dividing the pixel size of the imaging device 5 by the magnification of the imaging optical system 4. Here, N is an integer greater than or equal to 2. If the predetermined distance D is N times the divided value, for example, when the imaging device 5 performs a binning operation in which pixels in N rows and N columns are treated as a single pixel, the resolution of the observed image data D2 in the movement direction A1 can be easily matched with the resolution in the direction perpendicular to the movement direction A1, thereby eliminating the need for binning in the movement direction. Alternatively, if the predetermined distance D is 1 / N times the divided value, the resolution in the movement direction A1 can be increased compared to the resolution in the directions A2 and A3 perpendicular to the movement direction A1. In addition, even if the exposure start timing is shifted due to some factor, the position of the image data D1 in the movement direction A1 can be easily corrected.

[0048] As in this embodiment, the control unit 7 and the image data acquisition step ST1 may determine the exposure time Ta of the imaging device 5 when acquiring the image data D1 based on a signal from the position information output unit 31 (for example, based on the position information S). In this case, the proportion of non-exposure time is minimized as much as possible, and seamless observation image data D2 can be acquired with minimal loss of information about the observation object B.

[0049] As in the present embodiment, the control unit 7 may convert the position information S into a control signal C1, which is a trigger signal indicating the timing of exposure start, and provide the control signal C1 to the imaging device 5. The position information S may be converted into a control signal C1, which is a trigger signal indicating the timing of exposure start, and the control signal C1 may be provided to the imaging device 5. In this case, the exposure start timing of the imaging device 5 only needs to follow the control signal C1, and control of the imaging device 5 can be easily performed.

[0050] As in the present embodiment, the position information output unit 31 may include an encoder. In this case, the position information S for determining the exposure start timing of the imaging device 5 can be generated with high accuracy using a simple configuration.

[0051] Here, a configuration for acquiring seamless observation image data D2 with reduced loss of information about the observation object B in the movement direction A1 will be further described. Parts (a) of FIG. 5 and (a) of FIG. 6 show the position from which the control signal C1 is emitted in the movement direction A1. Parts (b) of FIG. 5 and (b) of FIG. 6 show the exposure period of the imaging device 5 from one control signal C1 to the next (= one frame). The length of the exposure period defines the resolution Yr in the movement direction A1. The length of the exposure period is determined by the exposure time Ta described above. In part (b) of FIG. 5, the exposure period coincides with the interval (predetermined distance D) between the control signals C1. In contrast, in part (b) of FIG. 6, the exposure period is shorter than the interval (predetermined distance D) between the control signals C1. In imaging devices 5 such as image sensors, operations other than exposure, such as reading the image data D1, are required within one frame. Therefore, as shown in part (b) of FIG. 6, the exposure period is usually shorter than the interval (predetermined distance D) between the control signals C1.

[0052] Parts (c) of Figure 5 and (c) of Figure 6 show an illumination area F formed by continuing the illumination surface La of the planar light L1 in the movement direction A1, intensity profiles Lb1 of the planar light L1 at multiple positions in the movement direction A1, and a composite profile Lb2 obtained by combining the intensity profiles Lb1 over one frame. When the length of the exposure interval matches the interval between the control signals C1, as shown in part (b) of Figure 5, the image data D1 can include information about the entire illumination area F in one frame in the movement direction A1. However, as shown in part (c) of Figure 6, the exposure interval is usually shorter than the interval between the control signals C1. Because the observation object B continues to move in intervals other than the exposure interval (hereinafter referred to as the non-exposure interval), information about the illumination area F in the non-exposure interval cannot be included in the image data D1. Therefore, information about the observation object B in the movement direction A1 is lost.

[0053] FIG. 7 is a diagram illustrating an example of a non-exposure interval. In FIG. 7, part (a) shows the position at which the control signal C1 is issued in the movement direction A1. In FIG. 7, part (b) shows the exposure period for each row group of the imaging device 5. As shown in FIG. 7, a time lag occurs due to signal processing and the like between the issuance of the control signal C1 and the start of exposure of the first row group. This lag is referred to as jitter J. In addition, in the case of a rolling shutter system, exposure of a certain row group begins after a delay time H from the start of exposure of the next row group. The delay time H is, for example, 13 μs. If the minimum time interval of the control signal C1 is 275 μs, the delay time H corresponds to 4.75% of that. The jitter J and delay time H also contribute to the non-exposure interval.

[0054] Below, the degree of information loss in the movement direction A1 will be quantitatively evaluated. Parts (a) to (f) of FIG. 8 and parts (a) to (f) of FIG. 9 are graphs showing a composite profile Lb2 obtained by varying the FWHM (full width at half maximum) ratio of the intensity profile Lb1 of the planar light L1 in the movement direction A1 and the ratio of the exposure time Ta, when one frame is set to 100%. Parts (a) to (c) of FIG. 8 show the case where the FWHM ratio is 10%. Parts (d) to (f) of FIG. 8 show the case where the FWHM ratio is 50%. Parts (a) to (c) of FIG. 9 show the case where the FWHM ratio is 100%. Parts (d) to (f) of FIG. 9 show the case where the FWHM ratio is 200%. Parts (a) of Figure 8, (d) of Figure 8, (a) of Figure 9, and (d) of Figure 9 show the case where the ratio of the exposure time Ta is 100%. Parts (b) of Figure 8, (e) of Figure 8, (b) of Figure 9, and (e) of Figure 9 show the case where the ratio of the exposure time Ta is 80%. Parts (c) of Figure 8, (f) of Figure 8, (c) of Figure 9, and (f) of Figure 9 show the case where the ratio of the exposure time Ta is 50%. The degree of information loss in real space within a pixel when the information distribution within one frame is quantized (imaged) is estimated from the composite profile Lb2. In the following description, the degree of information loss is defined by four indicators: the information acquisition rate, the minimum value of the information gradation (gradient), the missing area within the observation area, and the amount of information as a three-dimensional data set.

[0055] [Information Acquisition Rate] The information acquisition rate refers to the ratio of information within an imaged pixel to actual information in real space corresponding to the pixel. Specifically, the integral value of the composite profile Lb2 corresponds to the information within the imaged pixel. If the product of the maximum value of the composite profile Lb2 and a predetermined distance D is the actual information in real space corresponding to the pixel, the ratio of the integral value of the composite profile Lb2 to that product can be found to determine the ratio of the information within the imaged pixel.

[0056] Part (a) of Fig. 10 is a graph showing the ratio of information within an imaged pixel as a shade of color, with the FWHM ratio on the vertical axis and the exposure time Ta ratio on the horizontal axis. Part (b) of Fig. 10 is a contour map of part (a) of Fig. 10, showing 0% to 90% in 10% increments. Part (c) of Fig. 10 is a contour map of part (a) of Fig. 10, showing 90% to 98% in 1% increments. Parts (a) to (c) of Fig. 11 are enlarged graphs showing the portions of parts (a) to (c) of Fig. 10 where the exposure time Ta ratio is 95% to 100%, respectively.

[0057] [Minimum Value of Information Gradient] The minimum value of the information gradation refers to the minimum value of the spatial gradient of information within a pixel. For example, in the graph shown in part (a) of FIG. 8, point G1 is the minimum value of the gradation. Part (a) of FIG. 12 is a graph showing the minimum value of the information gradation in shades of color, with the FWHM ratio on the vertical axis and the exposure time Ta ratio on the horizontal axis. Part (b) of FIG. 12 is a contour map of part (a) of FIG. 12, showing 0% to 90% in 10% increments. Part (c) of FIG. 12 is a contour map of part (a) of FIG. 12, showing 70% to 90% in 5% increments. Parts (a) to (c) of FIG. 13 are enlarged graphs showing the portions of parts (a) to (c) of FIG. 12 where the exposure time Ta ratio is 95% to 100%, respectively.

[0058] [Defective Area in Observation Area] A defective area in an observation area refers to an area that is not included in a pixel relative to the actual information in real space corresponding to the pixel. Here, a defective area is defined as an area whose ratio to the maximum value of the composite profile is 10% or less. For example, in the graph shown in part (c) of FIG. 8, area G2 is a defective area. Part (a) of FIG. 14 is a graph showing the percentage of the defective area as a shade of color, with the FWHM percentage on the vertical axis and the exposure time Ta percentage on the horizontal axis. Part (b) of FIG. 14 is a contour map of part (a) of FIG. 14, showing 0% to 90% in 10% increments. Part (c) of FIG. 14 is a contour map of part (a) of FIG. 14, showing 50% to 90% in 5% increments. Parts (a) to (c) of FIG. 15 are graphs showing enlarged portions of parts (a) to (c) of FIG. 14 where the ratio of exposure time Ta is 95% to 100%, respectively.

[0059] [Amount of information as a three-dimensional data set] The amount of information as a three-dimensional data set refers to the amount of real-space information contained in the three-dimensional image when the entire measurement object (three-dimensional space) is three-dimensionally imaged. In other words, the amount of information as a three-dimensional data set refers to the sum of the information that can be obtained as information within a pixel and the information contained in other pixels. For example, in the graph shown in part (a) of Figure 8, the sum of the composite profile Lb2 present within the graph and the tail portion of the composite profile Lb2 present outside the graph corresponds to the amount of information as a three-dimensional data set.

[0060] Part (a) of Fig. 16 is a graph showing the amount of information as a three-dimensional data set by color shading, with the FWHM ratio on the vertical axis and the exposure time Ta ratio on the horizontal axis. Part (b) of Fig. 16 is a contour map of part (a) of Fig. 16, showing 0% to 100% in 10% increments. Part (c) of Fig. 16 is a contour map of part (a) of Fig. 16, showing 100% to 200% in 10% increments. Parts (a) to (c) of Fig. 17 are enlarged graphs showing the portions of parts (a) to (c) of Fig. 16 where the exposure time Ta ratio is 95% to 100%, respectively.

[0061] [Overall Evaluation of Four Indices] FIG. 18 is a diagram showing the above-mentioned (b) portions of FIG. 10, FIG. 12, FIG. 14, and FIG. 16 arranged side by side. FIG. 19 is a diagram showing the above-mentioned (b) portions of FIG. 11, FIG. 13, FIG. 15, and FIG. 17 arranged side by side. FIG. 20 is a diagram showing the above-mentioned (b) portions of FIG. 11, FIG. 13, FIG. 15, and FIG. 17 arranged side by side. Referring to these diagrams, it can be seen that when the FWHM ratio is 130% or more and 160% or less, the four indices have favorable values, and the degree of information loss in the movement direction A1 is low. That is, it is preferable that the FWHM of the light intensity distribution of the planar light L1 in the direction perpendicular to the irradiation surface La is 130% or more and 160% or less of the predetermined distance D. This makes it possible to acquire seamless observation image data with minimal loss of information about the observation object B.

[0062] The observation device and observation method according to the present disclosure are not limited to the above-described embodiment, and various other modifications are possible. For example, the control signal C1 output from the control unit 7 is not limited to a trigger signal, and may be various information that contributes to determining the timing of starting exposure.

[0063] While the principles of the present invention have been illustrated and described in preferred embodiments, it will be recognized by those skilled in the art that the present invention can be modified in arrangement and detail without departing from such principles. The present invention is not limited to the particular constructions disclosed herein. We therefore claim all modifications and variations that come within the scope and spirit of the following claims.

[0064] 1...observation device, 2...light irradiation unit, 3...scanning unit, 4...imaging optical system, 5...imaging device, 6...image generation unit, 7...control unit, 8...computer, 11...container, 13...well, 14...main body, 15...transparent member, 15a...input surface, 21...light source, 22...irradiation optical system, 31...position information output unit (information output unit), 32...holding unit, 33...moving stage, 41...objective lens, 42...bandpass filter, 43...coupling lens, 51...pixel group, A1...moving direction, A2, A3...direction , B...object of observation, C1...control signal, D...predetermined distance, D1...image data, D2...observation image data, F...irradiation area, G1...point, G2...area, H...delay time, J...jitter, L1...plane light, L2...observation light, La...irradiation surface, Lb1...intensity profile, Lb2...synthetic profile, P1...observation axis, S...position information, T...time interval, T1...observation start timing, T2...observation end timing, Ta...exposure time, Yr...resolution, Δt...predetermined time, θ...tilt angle.

Claims

1. An observation device comprising: an illumination optical system that illuminates an object to be observed with planar light; a scanning unit that moves the object to be observed at a constant speed in a direction intersecting the planar light, causing the object to pass through the illumination surface of the planar light; an imaging optical system that has an observation axis that intersects the illumination surface and that forms an image of the observation light generated on the object to be observed by the illumination of the planar light; an imaging device that acquires image data including an optical image of the observation light imaged by the imaging optical system each time the object to be observed moves a predetermined distance; an image generation unit that generates observation image data including an observation image of the object to be observed based on a plurality of the image data; and a control unit, wherein the scanning unit has an information output unit that outputs information for determining the timing of the start of exposure when the imaging device acquires the image data, based on the movement distance of the object to be observed, and the control unit controls the start of the exposure of the imaging device based on the information.

2. The observation device according to claim 1, wherein the control unit determines the timing of the start of exposure so that the specified distance is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system.

3. The observation device according to claim 1, wherein the control unit determines the timing of starting the exposure so that the predetermined distance is 1 / N or N times (where N is an integer of 2 or greater) the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system.

4. An observation device according to any one of claims 1 to 3, wherein the control unit determines the exposure time of the imaging device when acquiring the image data based on the information.

5. An observation device according to any one of claims 1 to 4, wherein the FWHM of the light intensity distribution of the planar light in a direction perpendicular to the irradiation surface is 130% or more and 160% or less of the predetermined distance.

6. An observation device according to any one of claims 1 to 5, wherein the control unit converts the information into a trigger signal indicating the timing of the start of exposure, and provides the trigger signal to the imaging device.

7. The observation device according to any one of claims 1 to 6, wherein the information output unit includes an encoder.

8. An observation method comprising the steps of: irradiating an object to be observed with planar light, moving the object to be observed at a constant speed in a direction intersecting with the planar light, causing the object to pass through the irradiation surface of the planar light, and using an imaging optical system having an observation axis intersecting with the irradiation surface to form an image of the observation light generated on the object to be observed by the irradiation of the planar light, and acquiring image data including an optical image of the formed observation light using an imaging device every time the object to be observed moves a predetermined distance; and generating observation image data including an observation image of the object to be observed based on a plurality of the image data, wherein the acquiring step includes a step of outputting information for determining the timing of exposure start when the imaging device acquires the image data, based on the movement distance of the object to be observed, and wherein the acquiring step controls the start of exposure of the imaging device based on the information.

9. An observation method according to claim 8, wherein in the acquiring step, the timing of starting the exposure is determined so that the predetermined distance is 0.95 to 1.05 times the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system.

10. An observation method according to claim 8, wherein in the acquiring step, the timing of starting the exposure is determined so that the predetermined distance is 1 / N or N times (where N is an integer of 2 or greater) the value obtained by dividing the pixel size of the imaging device by the magnification of the imaging optical system.

11. An observation method according to any one of claims 8 to 10, wherein in the acquiring step, the exposure time of the imaging device when acquiring the image data is determined based on the information.

12. An observation method according to any one of claims 8 to 11, wherein the FWHM of the light intensity distribution of the planar light in a direction perpendicular to the irradiation surface is 130% or more and 160% or less of the predetermined distance.

13. An observation method according to any one of claims 8 to 12, wherein the information is converted into a trigger signal indicating the timing of the start of exposure, and the trigger signal is provided to the imaging device.

Images