High-speed three-dimensional imaging device and method
The high-speed three-dimensional imaging device and method address the limitations of two-dimensional capture by using ultrashort pulses and spatial separation techniques to achieve continuous, single-shot three-dimensional imaging of high-speed phenomena, enabling detailed analysis of complex events with sub-nanosecond resolution.
Patent Information
- Application Number
- PCT/JP2024/017056
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-17
- Filing Date
- 2024-05-08
- Publication Date
- 2024-11-21
AI Technical Summary
Existing high-speed imaging methods are limited to two-dimensional capture, making it difficult to accurately analyze complex high-speed phenomena due to assumptions of axial symmetry, non-uniform materials, and stochastic events, especially when dealing with electromagnetic waves and ultra-high-speed events like plasmas and shock waves.
A high-speed three-dimensional imaging device and method that uses an ultrashort pulse light source, delay generators, multi-angle illuminators, and spatial separators to generate and separate elongated pulses by wavelength and angle, allowing for continuous single-shot imaging of three-dimensional profiles with sub-nanosecond resolution.
Enables the visualization and detailed analysis of ultra-high-speed phenomena by reconstructing three-dimensional images from multiple two-dimensional images acquired at various angles and wavelengths, overcoming the limitations of two-dimensional imaging and providing insights into complex high-speed events like laser-induced plasma and electrical discharges.
Smart Images

Figure JP2024017056_21112024_PF_FP_ABST
Abstract
Description
High-speed 3D imaging device and method
[0001] The present invention relates to a high-speed three-dimensional imaging apparatus and method.
[0002] Ultrafast imaging is a powerful tool for understanding the details of high-speed phenomena occurring in the femtosecond to nanosecond time domain, such as plasmas and shock waves. To overcome the limitations of high-speed cameras by completely eliminating mechanical and electrical motions and to capture phenomena that cannot be captured by high-speed cameras, optical high-speed imaging techniques with sub-nanosecond time resolution using ultrashort pulses have been developed (Patent Document 1, Non-Patent Document 1).
[0003] The three-dimensional shape of a target (high-speed phenomenon) is defined as the three-dimensional distribution of the target's optical properties, i.e., a three-dimensional profile. Existing high-speed imaging techniques with subnanosecond time resolution (Patent Document 1 and Non-Patent Document 1) are limited to two-dimensional acquisition of space. Therefore, when analyzing high-speed phenomena, it is necessary to assume a three-dimensional profile, taking into account the axial symmetry of the phenomenon, etc.
[0004] However, for complex phenomena that require measurement, it is difficult to assume a three-dimensional profile due to the inhomogeneity of electromagnetic waves and materials, as well as stochastic events. Conventional methods, therefore, are unable to obtain accurate three-dimensional profiles of three-dimensionally complex, high-speed phenomena. Non-Patent Document 2 discloses a method for spatially separating and acquiring information on a phenomenon from multiple angles using a lens array, and then reconstructing the three-dimensional distribution. However, this method does not allow for continuous shooting with a resolution of sub-nanoseconds. Therefore, an ultra-high-speed single-shot imaging technique capable of measuring the three-dimensional profile of a target is required.
[0005] Patent No. 6120280
[0006] K. Nakagawa, A. Iwasaki, Y. Oishi, R. Horisaki, A. Tsukamoto, A. Nakamura, K. Hirosawa, H. Liao, T. Ushida, K. Goda, F. Kannari and I. Sakuma, “Sequentially timed all-optical mapping photography (STAMP)” Nature Photonics 8, 695-700 (2014).KL Baker, “Tomographic reconstruction of high-energy-density plasmas with picosecond temporal resolution” Optics Letters 31(6), 730-732 (2006).H. Kudo et al., “Image reconstruction for sparse-view CT and interior CT-introduction to compressed sensing and differentiated” Quantitative imaging in medicine and surgery (2013)
[0007] An object of the present invention is to provide a technique capable of continuously capturing three-dimensional profiles of high-speed phenomena.
[0008] The first technical means adopted by the present invention is a high-speed three-dimensional imaging device comprising: a light source; a delay generator that stretches a pulse emitted from the light source to generate a stretched pulse consisting of a plurality of parts with different wavelengths; a multi-angle illuminator that divides the stretched pulse and makes it incident on an object from different angles; a spatial separator that spatially separates each stretched pulse, in which angle-dependent image information of the object is held in each part at a time corresponding to the wavelength, by wavelength and angle while the image information is held, to generate separated elements; an image sensor that detects each separated element holding image information as a plurality of two-dimensional image information; and three-dimensional reconstruction means that reconstructs a three-dimensional image from the plurality of two-dimensional image information at each wavelength and at a plurality of angles.
[0009] In one aspect, the light source is an ultrashort pulse light source. In one aspect, the stretched pulse has a chromatic dispersion (time delay according to wavelength) that allows image information to be acquired at frame intervals of 10 ns or less. In one aspect, the stretched pulse is a pulse train. In one aspect, the system includes a plurality of delay generators that can generate a plurality of types of stretched pulses that can provide image information at different frame intervals, and these delay generators may be selectable.
[0010] In one aspect, the multi-angle illuminator includes a diffractive optical element that splits the stretched pulse incident from the delay generator into multiple stretched pulses.
[0011] In one aspect, the spatial separator includes a first optical system that spatially separates each stretched pulse according to angle while preserving image information, and a second optical system that spatially separates each stretched pulse according to wavelength. In one aspect, the first optical system includes a lens and a lens array. In one aspect, the second optical system includes a diffractive optical element and a band-pass filter.
[0012] In one aspect, the spatial separator spatially separates each stretched pulse such that the plurality of two-dimensional image information is separately imaged on an image sensor.
[0013] In one embodiment, the image sensor is selected from a CCD image sensor and a CMOS image sensor. In one embodiment, the three-dimensional reconstruction means is a 3D tomographic reconstruction.
[0014] The second technical means adopted by the present invention is a high-speed imaging device comprising: a light source; a delay generator that stretches a pulse emitted from the light source to generate a stretched pulse consisting of a plurality of parts with different wavelengths; a multi-angle illuminator that divides the stretched pulse and makes it incident on an object from different angles; a spatial separator that spatially separates each stretched pulse, in which angle-dependent image information of the object is held in each part at a time corresponding to the wavelength, by wavelength and angle while the image information is held, to generate separated elements; and an image sensor that detects each separated element holding image information as a plurality of two-dimensional image information.
[0015] A third technical means adopted by the present invention is a high-speed three-dimensional imaging method, which includes: making stretched pulses consisting of multiple parts with different wavelengths incident on an object from different angles, so that each part of each stretched pulse incident from a different angle holds image information of the object that depends on the angle for a time corresponding to the wavelength; spatially separating the stretched pulses holding the image information for each wavelength and for each angle, and detecting them as multiple pieces of two-dimensional image information for each wavelength and for each angle; and reconstructing a three-dimensional image from the multiple pieces of two-dimensional image information at each wavelength and for each angle.
[0016] In one aspect, each stretched pulse is separated in a first separation step that spatially separates the pulses according to angle and in a second separation step that spatially separates the pulses according to wavelength while preserving image information, thereby obtaining a plurality of separated elements for each wavelength and for each angle. The temporal order in which the first separation step and the second separation step are performed is not limited, and the first separation step may be performed first, or the second separation step may be performed first, or the first separation step and the second separation step may be performed substantially simultaneously.
[0017] In one aspect, the plurality of pieces of two-dimensional image information are detected by forming an image of the separation element on an image sensor. In one aspect, the plurality of pieces of two-dimensional image information are separately formed on the image sensor.
[0018] In one aspect, the stretched pulse is obtained by stretching an ultrashort pulse, and in one aspect, the stretched pulse has a chromatic dispersion (wavelength-dependent time delay) that allows image information to be acquired at frame intervals of 10 ns or less.
[0019] A fourth technical means adopted by the present invention is a high-speed imaging method including: making stretched pulses consisting of multiple parts with different wavelengths incident on an object from different angles, so that each part of each stretched pulse incident from a different angle holds image information of the object that depends on the angle for a time corresponding to the wavelength; and spatially separating the stretched pulses holding the image information for each wavelength and each angle, and detecting them as multiple pieces of two-dimensional image information for each wavelength and each angle.
[0020] When considering obtaining a three-dimensional profile of a high-speed phenomenon, the obtained three-dimensional information may differ depending on the object. For example, when photographing an object whose image information is determined by the object's absorption coefficient, a three-dimensional absorption coefficient distribution is obtained. When photographing an object whose image information is determined by the object's scattering coefficient, a three-dimensional scattering coefficient is obtained. When photographing a non-transparent object, the three-dimensional distribution and shape of the object are obtained. Since the three-dimensional information obtained depending on the object is due to the optical properties of the photographed object, in this specification, the "three-dimensional profile of the object" can be rephrased as "the three-dimensional distribution of the object's optical properties."
[0021] The present invention enables single-shot continuous imaging of three-dimensional profiles of high-speed phenomena occurring in the subnanosecond time domain. Conventional methods are limited to two-dimensional acquisition of spatial data, so analyses are performed assuming three-dimensional profiles, such as axial symmetry. However, for complex phenomena that require measurement, such assumptions are difficult to make due to the heterogeneity of electromagnetic waves and materials, as well as stochastic events. By capturing three-dimensional profiles of high-speed phenomena of interest, the present invention enables detailed analysis of high-speed phenomena without the need to assume three-dimensional profiles.
[0022] This invention makes it possible to visualize the three-dimensional dynamics of ultrafast phenomena, the details of which have been unclear because the spatial data that can be obtained until now has been limited to two dimensions. This will enable the analysis of a wide range of high-speed phenomena, such as laser-induced plasma, dielectric breakdown, discharge, shock waves, crystal chemical reactions, and acoustic bioeffects, which are important in many fields of science and technology, including processing, energy, environment, and medicine.
[0023] FIG. 1 is an overall diagram of a high-speed 3D imaging system according to the present embodiment; FIG. 2 is a flow diagram illustrating a high-speed 3D imaging method according to the present embodiment; FIG. 3 is a conceptual diagram illustrating an example of a delay generator for realizing a frame interval of 10 ps or less (100 fs to 10 ps); FIG. 4 is a conceptual diagram illustrating an example of a delay generator for realizing a frame interval of 10 ps to 50 ps; FIG. 5 is a conceptual diagram illustrating an example of a delay generator for realizing a frame interval of 50 ps to several ns (10 ns); FIG. 6 illustrates stretched pulses (pulse trains) with different pulse intervals; FIG. 7 is a diagram illustrating a multi-angle illuminator (stretched pulse divider) according to the present embodiment; FIG. 8 is a diagram illustrating a multi-angle illuminator according to another embodiment; FIG. 9 is a diagram illustrating a multi-angle illuminator according to yet another embodiment; FIG. 11 is a diagram illustrating an illumination system (multi-angle illuminator) and a detection system (FIG. 14) according to the present embodiment; FIG. 12 is a conceptual diagram illustrating a detection system according to the present embodiment; FIG. 13 is a conceptual diagram illustrating image information (2D image) collected on an image sensor and a reconstructed 3D image; FIG. 9A is a conceptual diagram of the optical system according to FIG. 9B; FIG. 9B is a diagram illustrating the optical system according to FIG. 9C; FIG. 9C is a diagram illustrating the optical system according to FIG. 9C. 9A is a diagram similar to FIG. 8 and illustrates the design of an optical system. It shows a projection diagram on an image sensor of the optical system according to FIG. 9C. It illustrates an example of capturing a high-speed phenomenon in which all projection images for each wavelength and angle are spatially discretized. It is a diagram illustrating a 3D reconstruction method according to this embodiment. It is a diagram illustrating the correspondence between a stretched pulse consisting of multiple parts with different wavelengths and the time interval β of the wavelength components of the separation elements (image information) separated by a spatial separator, with the upper diagram illustrating the time interval α in a pulse train exemplified as a stretched pulse.
[0024] [A] Overview of High-Speed 3D Imaging System and Method [A-1] Overview of High-Speed 3D Imaging System An overview of the 3D high-speed imaging system according to this embodiment is shown in FIG. 1. The 3D high-speed imaging system according to this embodiment includes an illumination system that generates illumination light to illuminate an object, an imaging unit in which the object is placed, and a detection system that detects light emitted from the imaging unit (light transmitted through or reflected from the object). The illumination light from the illumination system illuminates the object (phenomenon) in the imaging unit. The light emitted from the imaging unit (transmitted light or reflected light) contains image information of the object (phenomenon), and the detection system detects the image information contained in the emitted light. The 3D high-speed imaging system according to this embodiment includes a 3D reconstruction means (3D moving image reconstruction device) that reconstructs a 3D image from multiple 2D images at multiple angles at each wavelength using image information (multiple 2D image information) detected by the detection system. Three-dimensional moving image information of the object is obtained by 3D reconstructing the detected image information (2D image information).
[0025] The illumination system consists of an ultrashort pulse light source, a delay generator (pulse stretcher) that stretches the pulse emitted from the ultrashort pulse light source to generate a stretched pulse consisting of multiple parts with different wavelengths, and a pulse splitter (multi-angle illuminator) that splits the stretched pulse and makes it incident on the object from different angles. In the illumination system, multiple illumination lights consisting of multiple parts with different wavelengths are prepared and these illumination lights are incident on the object from multiple angles.
[0026] In the imaging unit, multiple irradiation lights (stretched pulses) are irradiated onto the object from different angles, and each part (wavelength component) holds image information of the object depending on different angles at the time corresponding to the wavelength. For ultrafast phenomena, if stretched pulses consisting of multiple parts with different wavelengths (for example, a pulse train consisting of pulses with different wavelengths) are incident from multiple angles, the pulse with wavelength λ1 passes through the phenomenon at time t1, and the pulse with wavelength λ5 passes through the phenomenon at time t5, and each pulse (λ1, λ2, ... λ n ) passing timing t1 to t n Since the wavelengths are different, image information corresponding to the time periods is acquired. Each stretched pulse that holds the image information is, for example, transmitted light or reflected light, and is reflected from the imaging unit at different angles θ1 to θm It is emitted at.
[0027] The detection system detects different angles θ1 to θ m and an imaging element or image sensor that detects the light spatially separated by the spatial separator, i.e., the separated elements of the stretched pulses spatially separated by the spatial separator while the image information is maintained, as a plurality of pieces of two-dimensional image information. Typically, the emitted light or separated elements (two-dimensional image information) spatially separated by the spatial separator are imaged and detected at different positions on the light receiving surface of the image sensor.
[0028] In the detection system, a spatial separator aggregates all image information possessed by pulses of each angle and each wavelength as separation elements on an image sensor, thereby enabling acquisition of two-dimensional images of multiple angles at each wavelength, i.e., at each time, and performing tomographic reconstruction of a three-dimensional profile of the object from each two-dimensional image to acquire ultra-high-speed three-dimensional video images. Note that in the three-dimensional high-speed imaging system according to this embodiment, if the detection system is considered in a broad sense, the 3D video reconstruction device is a component of the detection system, while if the detection system is considered in a narrow sense, the three-dimensional high-speed imaging system according to this embodiment can be considered to consist of an illumination system (optical system), an imaging unit, a detection system (optical system), and a 3D video reconstruction device (which may be composed of a computer).
[0029] [A-2] Overview of High-Speed 3D Imaging Method The high-speed 3D imaging method according to this embodiment comprises: directing an elongated pulse consisting of multiple portions with different wavelengths onto an object from different angles, causing each portion of each elongated pulse incident from a different angle to retain angle-dependent image information of the object for a time corresponding to the wavelength; spatially separating the elongated pulses retaining the image information for each wavelength and angle, and detecting them as multiple pieces of 2D image information for each wavelength and angle; and reconstructing a 3D image from the multiple pieces of 2D image information at each wavelength and angle. In a case where the elongated pulse is a pulse train and image information is retained in the pulse train as projection light, the high-speed 3D imaging method comprises directing a pulse train with different wavelengths onto a phenomenon from multiple angles, thereby obtaining different projection images for each pulse depending on the timing and direction of incidence; and spatially separating and detecting all of the projection images for each time and angle, thereby continuously capturing a 3D profile of the high-speed phenomenon.
[0030] The frame interval achievable in high-speed imaging according to this embodiment is determined by the chromatic dispersion of the stretched pulse (time delay according to wavelength). The separated elements generated by the spatial separator have time information according to their wavelength components, and this time information depends on the chromatic dispersion of the stretched pulse (time delay according to wavelength). Figure 20 illustrates the correspondence between the stretched pulse consisting of multiple parts with different wavelengths generated by the delay generator and the time interval (i.e., frame interval) β of the wavelength components of the separated elements (image information) separated by the spatial separator. The upper diagram shows the time interval α in a pulse train exemplified as a stretched pulse. The frame interval β corresponds to and depends on the time interval α of the pulse train. Note that the time interval α of the pulse train can be substantially the same as the frame interval β, but this is not guaranteed.
[0031] An overview of the high-speed 3D imaging method will be described with reference to Figure 2. Ultrashort pulses emitted from an ultrashort pulse light source are stretched by a pulse stretcher (delay generator) to generate multiple n desired wavelength components λ1 to λ2. nA stretched pulse consisting of λ is generated. In Figure 2, a stretched pulse with five wavelength components is assumed. The stretched pulse is split by a multi-angle illuminator (pulse splitter) and incident (illuminated) on the object from multiple m different angles. In Figure 2, nine angles are assumed (m = 9), and stretched pulses with five wavelength components (n = 5) are incident on the object from nine different angles. The stretched pulses emitted from the object at nine different angles each hold image information in five wavelength components, and a total of 45 (m × n) pieces of image information are acquired depending on the incidence timing and direction. The stretched pulses emitted in nine directions are separated by wavelength and angle by a spatial separator to generate 45 (m × n) separated light beams (separation elements), and the 45 separated light beams are detected by an image sensor to acquire a 45 (m × n) two-dimensional image. The wavelength components λ1 to λ n By reconstructing 2D images into 3D images for each λ, n The times t1 to t correspond to (depend on) n In the example shown, at each time t, a 3D image at that time t is reconstructed from nine 2D images. In Figure 2, five time-series 3D images at times t1 to t5 corresponding to λ1 to λ5 are obtained.
[0032] The important points in the 3D high-speed imaging according to this embodiment are: (i) acquiring image information from multiple angles at a certain time and reconstructing a 3D profile at that time based on that image information; and (ii) acquiring image information at different times according to wavelengths from stretched pulses incident from the same angle. To reconstruct 3D image information at a certain time, specific wavelength components of stretched pulses incident from different angles must be simultaneously incident. This is because 3D reconstruction requires simultaneous acquisition of image information from multiple angles. Therefore, the image information to be acquired must be acquired from multiple angles at the same time.
[0033] As long as the above conditions are met, multiple stretched pulses incident on the target from different angles may contain stretched pulses with different wavelength components. For example, pulses incident from different angles may have different wavelengths. Specifically, a stretched pulse (pulse train) at one angle may be composed of wavelength components (λ1, λ2, λ3), while a stretched pulse at another angle may be composed of wavelength components (λ4, λ5, λ6). Furthermore, situations where the spatial profiles or polarizations of stretched pulses incident from different angles are not the same may also be considered. Furthermore, all stretched pulses do not need to be incident at the same time. For example, stretched pulses at angles θ1 to θ4 may be incident at the same time (time t1 to time t4), and stretched pulses at angles θ5 to θ8 may be incident at different times (time t5 to time t8). 3D reconstruction may be performed using the acquired time domains, resulting in a 3D video sequence from time t1 to time t8.
[0034] [B] Illumination System [B-1] Overview The illumination system according to this embodiment comprises an ultrashort pulse light source that emits ultrashort pulse light, a delay generator (pulse stretcher) that stretches the pulse emitted from the ultrashort pulse light source to generate a stretched pulse consisting of multiple parts with different wavelengths, and a pulse splitter (multi-angle illuminator) that splits the stretched pulse and makes it incident on the target from different angles. In the illumination system, first, an ultrashort pulse laser light is subjected to a first optical system that uses a material dispersion such as glass or a structural dispersion such as a grating pair to generate a different delay time for each wavelength. Next, a second optical system that uses a diffractive optical element or the like splits the stretched pulse into multiple stretched pulses, which are then incident on the target to be photographed from different angles. In this way, each wavelength component (λ1, λ2, ...λ n ) pulses arrive at different times (t1, t2, ... t) depending on the chromatic dispersion (time delay depending on the wavelength). n ) to obtain image information of the object from multiple angles.
[0035] Typically, the stretched pulse is a pulse train, but is not limited to this. As long as it is "illumination light (stretched pulse) consisting of multiple parts with different wavelengths" or "illumination light (stretched pulse) with a different delay time for each wavelength," it does not need to be discretized into multiple pulses as a pulse train, and pulses in which each wavelength is continuously stretched in time can also be used as illumination light.
[0036] [B-2] Light Source The light source according to this embodiment is typically a single ultrashort pulse light source, which generates a stretched pulse from an ultrashort pulse using a delay generator. It is also possible to operate light sources of multiple wavelengths individually. For example, a combination of multiple short-wavelength semiconductor pulse lasers (picosecond pulse lasers, nanosecond pulse lasers) may be used.
[0037] When stretching ultrashort pulses, current delay generators can achieve frame intervals of up to about 10 ns, and theoretically, frame intervals of 100 ns or longer are possible. When multiple light sources are used in combination, imaging with μs or ms frame intervals may also be possible. In principle, the present invention is applicable to any wave, not just light (visible light to near-infrared light), but also the entire electromagnetic wave range (X-rays, THz, radio waves, etc.) and electron beams. It will be understood by those skilled in the art that the term "light source" can be appropriately substituted depending on the radiation source.
[0038] [B-3] Delay Generator (Pulse Stretcher) The first optical system of the illumination system according to this embodiment is a delay generator (pulse stretcher) that stretches the ultrashort pulse laser light. Examples of the configuration of the delay generator include an optical system that uses material dispersion such as glass or an optical system that uses structural dispersion such as a grating pair, which generates a different delay time for each wavelength.
[0039] The generation of pulse trains with different wavelengths will be explained with reference to Figures 3 to 5. Figure 3 shows a conceptual diagram of a delay generator for achieving frame intervals between 100 fs and 10 ps. The delay generator shown in Figure 3 utilizes the material dispersion of glass, and an ultrashort pulse laser beam is propagated within the glass using a mirror pair. The desired frame interval Δt can be obtained according to the equation Δt = D(λ)LΔλ, where D(λ) is the dispersion coefficient, L is the path length, and Δλ is the wavelength difference. Such a delay generator can be constructed using glass blocks or glass rods.
[0040] Figure 4 shows a conceptual diagram of a delay generator for achieving frame intervals of 10 ps to 50 ps, constructed using a grating pair and a mirror. The desired frame interval Δt can be obtained according to Δt ≈ 2λΔs / (dc), where d is the groove spacing, λ is the wavelength, and c is the speed of light.
[0041] Figure 5 shows a conceptual diagram of a delay generator for achieving frame intervals of 50 ps to several ns. The upper diagram is constructed using a diffraction grating pair and a group of mirrors. The lower diagram is constructed using a mirror pair, a diffraction grating pair, and a mirror. The desired frame interval Δt can be obtained according to Δt ≈ 4L / c, where L is the distance between the mirror pairs and c is the speed of light.
[0042] In this embodiment, three different types of delay generators are used to realize stretched pulses with frame intervals of 100 ps to 10 ns. FIG. 6 shows an example of a stretched pulse (a pulse train consisting of multiple wavelength components) generated by stretching using a delay generator. In FIG. 6, "Interval" refers to the "time interval between pulse trains" or "time interval between adjacent pulses on the time axis." In FIG. 6, the time scales on the horizontal axis are different, and the interval between each pulse corresponds to the Interval. If a selector switch for switching the optical path is provided in the first optical system of the illumination system, it is possible to selectively use three types of delay generators depending on the desired frame interval. Alternatively, the delay generators may be appropriately reconfigured depending on the desired frame interval without using a selector switch. The delay generators (pulse stretchers) shown in FIGS. 3 to 5 are merely examples, and it is understood that a person skilled in the art can design a specific first optical system to achieve the desired frame interval.
[0043] [B-4] Pulse Splitter (Multi-Angle Illuminator) The second optical system of the illumination system according to this embodiment is a pulse splitter (multi-angle illuminator) that splits an elongated pulse and makes the multiple elongated pulses incident on the target from different angles. As shown in FIG. 7, the pulse splitter (multi-angle illuminator) according to this embodiment splits the elongated pulse into multiple parts using an optical system that uses a diffractive optical element and a lens, and makes each of the pulses incident on the target to be photographed (focused and illuminated) from multiple different angles. In this way, pulses of each wavelength component (λ1, λ2, ... λ n ) at different times (t1, t2, ... t n), projected images of the target are obtained from multiple angles. FIG. 7A shows a multi-angle illuminator according to another embodiment, which is composed of a mirror array and a lens, and in which a cross section of a single light beam is incident across multiple mirrors in the mirror array, thereby spatially separating the single light beam into multiple light beams, which are then incident on the target using a lens. FIG. 7B shows a multi-angle illuminator according to yet another embodiment, which is composed of a diffractive optical element and a curved mirror array, and in which the light beams split by the diffractive optical element are reflected by each curved mirror array and incident on the target. The pulse splitters (multi-angle illuminators) shown in FIGS. 7, 7(A), and 7(B) are merely examples, and it will be understood by those skilled in the art that specific optical systems for irradiating stretched pulses from multiple desired angles can be designed.
[0044] [C] Imaging Unit: A target (e.g., an ultrafast phenomenon) placed in the imaging unit is irradiated with multiple irradiation light beams (stretched pulses) from multiple angles. When stretched pulses (pulse trains) consisting of multiple different wavelength components are incident on the ultrafast phenomenon from multiple angles, the pulse with wavelength λ1 passes through the phenomenon at time t1, and the pulse with wavelength λ5 passes through the phenomenon at time t5. Each pulse (λ1, λ2, ... λ n ) passing times t1, t2,...t n Since the wavelengths are different, image information corresponding to the time of each wavelength is acquired. n ) at different times (t1, t2, ... t n ) to acquire image information of the object from multiple angles. Each stretched pulse is a time series (t1, t2, ... t n The extended pulses, which retain the time-series image information, are emitted from the imaging unit at different angles as, for example, transmitted light or reflected light.
[0045] [D] Detection System [D-1] Overview The detection system separates and acquires the image information obtained by illumination light incident from different angles on an image sensor. The detection system consists of a spatial separator that spatially separates each stretched pulse emitted from the imaging unit at different angles by wavelength and angle while preserving the image information, and an imaging element or image sensor that detects the light spatially separated by the spatial separator, i.e., each separated element of the stretched pulse spatially separated while preserving the image information, as multiple pieces of two-dimensional image information.
[0046] In the detection system, first, a spatial separator consisting of a lens array, a diffractive optical element, a band-pass filter, etc. is used to spatially separate two-dimensional image information for each angle and each wavelength, and then the information is captured on an image sensor that is kept in an exposed state. In this way, the two-dimensional image information for each wavelength, that is, each time (t1, t2, ... t n ) at multiple angles θ1 to θ m Using the two-dimensional image information (e.g., projection images) detected for each wavelength and each angle, a three-dimensional image is reconstructed (tomographic reconstruction) from two-dimensional images taken at multiple angles at each time (= wavelength), thereby obtaining a three-dimensional profile of the target (high-speed phenomenon).
[0047] [D-2] Spatial Separator The spatial separator according to this embodiment separates the stretched pulses that retain image information using both wavelength and angle parameters to capture a two-dimensional image on an image sensor. The following three examples of detection (capture) modes on the image sensor include: 1. When pulses at each angle and wavelength (= each time) are completely separated and detected as two-dimensional images (x, y) (FIGS. 9B and 9C). 2. When two-dimensional images (x, y) at each angle and wavelength (= each time) are not overlapped but are detected as separate two-dimensional images (x, y) (FIG. 9A). 3. When two-dimensional images (x, y) at each angle and wavelength (= each time) overlap, but the image is reconstructed using special assumptions and operations, such as compressed sensing.
[0048] One of the key aspects of the spatial separator according to this embodiment is how to consolidate all the information required for reconstruction into a limited two-dimensional area, such as an image sensor. In this embodiment, both wavelength and angle parameters are independently manipulated to separate the emitted light, which retains image information, and capture a two-dimensional image on the image sensor. This embodiment differs from the disclosures of Patent Document 1 and Non-Patent Document 1, which manipulate only the wavelength, and Non-Patent Document 2, which manipulate only the angle, in that it independently manipulates both the wavelength and angle parameters. The spatial separator according to this embodiment includes a first optical system that spatially separates each stretched pulse according to angle while retaining image information, and a second optical system that spatially separates each stretched pulse according to wavelength. This specification discloses three examples of the spatial separator. Please note that the configuration of the spatial separator described below (e.g., the combination of multiple optical elements constituting each optical system) is merely an example and does not limit the spatial separator according to the present invention.
[0049] [D-2-1] Spatial Separator (First Example) A first example of a spatial separator is shown in Figures 9(A) and 11. In this first example, the emitted light carrying image information is first largely separated by angle, and a two-dimensional space is arranged overall, with each point (focused spot) separated by time and imaged. The optical system (image formation method after passing through the phenomenon) according to this first example is composed of a lens array system that separates by angle, a microlens array system that separates the two-dimensional space as a focus point distribution, and a spectroscopic system that minutely displaces each wavelength. As a result, each parameter is separated and imaged as a focus point distribution on the image sensor.
[0050] As shown in Figure 11, the spatial separator according to the first embodiment comprises an angle separation means comprising an objective lens and a lens array, a spatial separation means comprising a microlens array, and a wavelength (time) separation means comprising a lens, a diffraction grating, and another lens. The lens and lens array work together to spatially separate the image information captured from each angle. The microlens array converts the two-dimensional space (x, y) into a distribution of focused spots. The lens, diffraction grating, and lens work together to separate the distribution of focused spots by wavelength. As with the second and third embodiments described below, the set of lens and lens array can be replaced with a set of lens and mirror array.
[0051] [D-2-2] Spatial Separator (Second Embodiment) A second embodiment of the spatial separator is shown in Figures 9(B), 12, and 13. As shown in Figure 9(B), in the second embodiment, the emitted light carrying image information is separated by angle on the image sensor, and then separated by time within that, and each is imaged in two-dimensional space. In Figure 12, the optical system is composed of a lens, a first lens array, a diffractive optical element, a band-pass filter, and a second lens array. As shown in Figure 13, in the second embodiment, the lens, lens array, diffractive optical element, and band-pass filter are considered to be an example of the minimum configuration that achieves this. In the optical system of Figure 13, the lens and lens array work together to spatially separate the image information captured from each angle. The diffractive optical element and band-pass filter further separate the image information of each wavelength. In the optical system shown in Figure 12, the lens, first lens array, diffraction grating, and band-pass filter play the same roles as in the optical system of Figure 13, and the second lens array is used to image each image information on the image sensor. In the embodiment shown in FIG. 13, the imaging function is included in the lens array.
[0052] In the optical system shown in Fig. 12, the set of a diffractive optical element and a band-pass filter can be replaced with a slice mirror system (a set of a diffraction grating, a lens, and a slice mirror) as described in Example 3. Furthermore, in the optical system shown in Fig. 12, the set of a lens and a first lens array can be replaced with the set of a lens and a mirror array as described in Example 3. Furthermore, when a set of a diffractive optical element and a band-pass filter is used, it is also possible to replace the second lens array behind it with a single lens to form an image.
[0053] [D-2-3] Spatial Separator (Third Embodiment) A third embodiment of the spatial separator is shown in FIGS. 9(C), 14, and 15. In the third embodiment, the emitted light carrying image information is separated by time on the image sensor, and then separated by angle, each of which is imaged in two-dimensional space. In FIG. 14, the optical system of the detection system is composed of a lens, a lens array, a diffractive optical element, a band-pass filter, and a lens. As shown in FIG. 15, in the third embodiment, the lens, lens array, diffractive optical element, and band-pass filter are considered to be an example of a minimum configuration that achieves this. In FIG. 14, another lens is used after the band-pass filter, but this is due to the actual optical design (issues such as magnification and the arrangement and specifications of each element). One of the minimum configurations from the perspective of separating angles and wavelengths is shown in FIG. 15.
[0054] In Figure 14, the lens and lens array work together to spatially separate the image information captured from each angle. The diffractive optical element and bandpass filter separate the image information for each wavelength. The diffractive optical element splits the pulse into multiple angular directions, the bandpass filter selects wavelengths for each (transmitting different wavelength components depending on the angle of incidence), and the lens serves to form an image on the image sensor. As shown in Figure 15, when no lens is included at the rear, the function of forming an image on the image sensor is included in the lens array.
[0055] In the optical system according to the third embodiment, a configuration that uses a set of a lens and a mirror array can be adopted as a configuration that can replace the set of a lens and a lens array, as described in the following document: S. Yeola et al., "Single-shot ultrafast imaging via spatiotemporal division of femtosecond laser pulses," Journal of the Optical Society of America B (2018).
[0056] A possible configuration that can replace the set of diffractive optical element, bandpass filter, and lens is a configuration using a slice mirror system, as described in the following document. Specifically, the order is (pick-off mirror →) diffraction grating 1 → lens 1 → slice mirror → lens 1 → diffraction grating 1 → (imaging) lens → image sensor. This configuration also makes it possible to separate images by wavelength. T. Saiki et al., "Sequentially timed all-optical mapping photography boosted by a branched 4f system with a slicing mirror," Optics Express (2020).
[0057] Referring to FIG. 16, the specific design of the optical system for the illumination system and detection system will be described. In FIG. 16, the illumination system consists of a diffractive optical element (DOE), lenses L1 and L2, and directs stretched pulses onto a target T from multiple angles. The detection system consists of lens L3, lens array LA, diffractive optical element (DOE), bandpass filter (BPF), and lens L4. In the figure, CP is the conjugate plane (intermediate image), D0 is the original beam diameter, D is the beam diameter after passing through the target (corresponding to the field of view), θ is the angle of incidence of the diffracted light on the target, and s is the distance from the target to the focal point of each pulse. Examples of parameters for designing the optical system shown in FIG. 16 include the focal length of each lens L1 to L4, the focal length of the lens array LA, D, θ, and θ0 (the diffraction angle of the DOE). Those skilled in the art can design an optimal optical system by adjusting each parameter. Note that the parameters that can be used may vary depending on the optical elements that make up the actual optical system.
[0058] To improve spatial resolution, three factors are important: (i) increasing the numerical aperture (NA), (ii) increasing the magnification of the imaging system, and (iii) preventing the pulses at each angle from overlapping. For (i), it is important to increase θ, and for (ii) and (iii), it is important to reduce D and use an optical arrangement that can separate each pulse in the lens array (LA) section.
[0059] For example, by designing the pulses so that they overlap just before focusing, it is possible to make D less than 50 μm. In order to observe microscale objects, it is important to narrow the beam diameter of each pulse to about several tens of μm at the target (T). In this embodiment, the optical design is such that probe pulses from different angles are incident on the target just before focusing, and all of them can overlap.
[0060] In this embodiment, to prevent overlap, each pulse is separated by a lens array (LA) and imaged at a conjugate plane (CP). In this embodiment, the distance between lens L3 and lens array LA is adjusted so that each pulse after passing through LA becomes a parallel beam. In this way, the pulses emitted from the imaging unit are incident on the rear optical system as parallel beams.
[0061] [D-3] Image Sensor In this embodiment, the spatial separator separates the emitted light carrying image information by wavelength and angle, and all image information contained in the pulses (separated light) at each angle and wavelength is collected on the image sensor. This allows for acquisition of two-dimensional images from multiple angles at each wavelength, i.e., at each time. These two-dimensional images are then tomographically reconstructed to obtain ultra-high-speed three-dimensional moving images, i.e., a three-dimensional profile of the target (high-speed phenomenon).
[0062] As shown in FIG. 9, the light receiving surface of the image sensor is 1 Multiple angles θ at 1 ~θ 4 Four 2D images from time t 2 Multiple angles θ at 1 ~θ 4 Four 2D images from time t 3 Multiple angles θ at 1 ~θ 4 Four 2D images from time t 4 Multiple angles θ at 1 ~θ 4 Four 2D images from time t 5 Multiple angles θ at 1 ~θ 4 Four spatially separated two-dimensional images are detected (imaged) from the image sensor 1. Each two-dimensional image is specified by its x and y coordinates.
[0063] Examples of imaging elements include CCD image sensors and CMOS image sensors, but imaging elements that can be used in this embodiment are not limited to these and include, in addition to CCD image sensors and CMOS image sensors, SPAD image sensors, image intensifiers, EMCCD cameras, InGaAs cameras, etc. The number of imaging elements is not limited to one, and an image of the target may be formed on multiple imaging elements to detect the image. For detecting two-dimensional images, a device with a light-receiving surface with a larger area (an area that allows multiple spatially separated light beams to be incident at different positions) or multiple CCD elements or photographic plates may be prepared.
[0064] When the spatial separator (third embodiment) is used, the emitted light carrying the image information is separated by time on the image sensor, and then by angle, forming two-dimensional images. Figure 17 shows an example in which illumination light is incident on a phenomenon from a 3 x 3 angle, and the acquired image information is separated into 45 pieces of image information by a 3 x 3 angle and five wavelengths using a spatial separator equipped with the optical system of the third embodiment, which are incident on different positions on a single image sensor and detected (projected images). Figure 18 shows a projected image of a high-speed phenomenon (laser ablation plasma) at a 3 x 3 angle and five wavelengths.
[0065] [E] Three-dimensional video reconstruction device (3D tomographic reconstruction) In the three-dimensional video reconstruction device according to this embodiment, a 3D image is reconstructed (tomographic reconstruction) from 2D images of multiple angles at each time (= wavelength), thereby obtaining an ultra-high-speed 3D video. As shown in FIG. 9, the three-dimensional reconstruction means reconstructs a 3D image at each time t 1 Multiple angles θ at 1 ~θ 4 From four 2D images at time t 1 A three-dimensional image is reconstructed at time t 2 Multiple angles θ at 1 ~θ 4 From four 2D images at time t 2 A three-dimensional image is reconstructed at time t 3 Multiple angles θ at 1 ~θ 4From four 2D images at time t 3 A three-dimensional image is reconstructed at time t 4 Multiple angles θ at 1 ~θ 4 From four 2D images at time t 4 A three-dimensional image is reconstructed at time t 5 Multiple angles θ at 1 ~θ 4 From four 2D images at time t 5 A three-dimensional image is reconstructed at time t 1 ~t 5 A three-dimensional video of the object can be obtained from each two-dimensional image in the image.
[0066] As shown in Figure 17, 3D reconstruction using multiple 2D images acquired by an image sensor can be performed using 3D tomographic reconstruction. The stretched pulse emitted from the imaging unit at an angle θ2 carries image information from times t1 to t5, ... corresponding to wavelengths λ1 to λ5, .... The spatial separator separates this image information into separate components, and each separate component is detected on the image sensor as a 2D image 2D(λ1,θ2), 2D(λ2,θ2), 2D(λ3,θ2), 2D(λ4,θ2), 2D(λ5,θ2), .... Similarly, the stretched pulse emitted at angle θ3 carries image information from times t1 to t5, corresponding to wavelengths λ1 to λ5. This image information is separated into separate components by the spatial separator, and each separate component is detected on the image sensor as two-dimensional images 2D(λ1,θ3), 2D(λ2,θ3), 2D(λ3,θ3), 2D(λ4,θ3), 2D(λ5,θ3), …. A three-dimensional image at that time is reconstructed using multiple two-dimensional images acquired at the same time. For example, a three-dimensional image 3D(t1) is reconstructed using two-dimensional images 2D(λ1,θ2), 2D(λ1,θ3), 2D(λ1,θ4), 2D(λ1,θ5), … acquired at time t1, corresponding to λ1. Similarly, three-dimensional images 3D(t2), 3D(t3), 3D(t4), 3D(t5) ... are reconstructed, and a high-speed three-dimensional video sequence is obtained from these three-dimensional images.
[0067] Tomographic reconstruction is a technique for performing reconstruction from images acquired at multiple angles, and is described, for example, in Non-Patent Documents 3 and 4. The 3D reconstruction unit can be configured with a computer (e.g., equipped with an input unit, memory, processor, output unit, display unit, etc.). Specific means for 3D reconstruction are known to those skilled in the art, and examples include Zernike polynomial expansion; maximum likelihood-expectation maximization (MLEM); minimum Fisher information (MFI); and deep neural network.
[0068] The high-speed imaging system of the present invention can be configured by combining an illumination system and a detection system that are made up of widely used optical elements and equipment, and the 3D reconstruction unit can be configured from a computer. The system is easy to build and has the flexibility to be optimized according to the purpose. It can also be applied to existing optical systems and equipment, particularly incorporation into microscopes. Furthermore, by combining it with an imaging element and computer that are capable of high-speed readout, it is possible to configure a high-grade high-speed video camera that can capture ultrafast phenomena for longer periods of time.
[0069] This invention provides a method for capturing 3D dynamic images with subnanosecond resolution and single-shot imaging. This method is useful for the development of products involving high-speed phenomena, such as laser processing machines and EDM machines. First, by visualizing high-speed phenomena during processing, we believe that clarifying the principles of processing phenomena will enable parameter design to achieve optimal processing. Furthermore, by implementing this method in actual processing machines, monitoring and feedback control are feasible. In laser processing and EDM, it is expected that errors from the design will occur in the processing process due to factors such as material inhomogeneity and instability of input parameters (laser, voltage). Since the input parameters and material surface shape are strongly related to the 3D dynamics of high-speed phenomena such as plasma and discharge during processing, high-speed 3D monitoring of these parameters is expected to enable error feedback and optimization of the processing process. Therefore, this method has a wide range of potential applications, including surface treatment and processing of composite materials and even laser treatment of biological tissue.
Claims
1. A high-speed 3D imaging device comprising: a light source; a delay generator which stretches a pulse emitted from the light source to generate a stretched pulse consisting of multiple parts with different wavelengths; a multi-angle illuminator which divides the stretched pulse and causes it to be incident on an object from different angles; a spatial separator which spatially separates each stretched pulse, in which angle-dependent image information of the object is held in each part at a time corresponding to the wavelength, for each wavelength and angle while the image information is held, to generate separated elements; an image sensor which detects each separated element holding image information as multiple two-dimensional image information; and 3D reconstruction means which reconstructs a 3D image from the multiple two-dimensional image information at multiple angles for each wavelength.
2. The high-speed 3D imaging device according to claim 1, wherein the light source is an ultrashort pulse light source.
3. The high-speed three-dimensional imaging device according to claim 1 or 2, wherein the stretched pulse has a wavelength dispersion that enables image information to be acquired at frame intervals of 10 ns or less.
4. The high-speed three-dimensional imaging device according to claim 1, wherein the multi-angle illuminator includes a diffractive optical element that splits the stretched pulse incident from the delay generator into a plurality of stretched pulses.
5. The high-speed 3D imaging device of claim 1, wherein the spatial separator includes a first optical system that spatially separates each stretched pulse according to angle while preserving image information, and a second optical system that spatially separates each stretched pulse according to wavelength.
6. The high-speed three-dimensional imaging device according to claim 5, wherein the first optical system includes a lens and a lens array.
7. The high-speed three-dimensional imaging device according to claim 5 or 6, wherein the second optical system includes a diffractive optical element and a band-pass filter.
8. The high-speed three-dimensional imaging device according to claim 1, wherein the spatial separator spatially separates each stretched pulse to generate the separated elements so that the plurality of two-dimensional image information are separately imaged on an image sensor.
9. A high-speed imaging device comprising: a light source; a delay generator that stretches a pulse emitted from the light source to generate an stretched pulse consisting of multiple parts with different wavelengths; a multi-angle illuminator that divides the stretched pulse and causes it to be incident on an object from different angles; a spatial separator that spatially separates each stretched pulse, in which angle-dependent image information of the object is held in each part at a time corresponding to the wavelength, by wavelength and angle while the image information is held, to generate separated elements; and an image sensor that detects each separated element holding image information as multiple two-dimensional image information.
10. A high-speed three-dimensional imaging method comprising: making an elongated pulse consisting of a plurality of parts with different wavelengths incident on an object from different angles, thereby causing each part of each elongated pulse incident from a different angle to hold angle-dependent image information of the object for a time corresponding to the wavelength; spatially separating the elongated pulses holding the image information for each wavelength and for each angle, and detecting them as a plurality of two-dimensional image information for each wavelength and for each angle; and reconstructing a three-dimensional image from the plurality of two-dimensional image information at the plurality of angles for each wavelength.
11. The high-speed three-dimensional imaging method according to claim 10, wherein each stretched pulse is separated in a first separation step for spatially separating the pulses according to angle while maintaining image information, and in a second separation step for spatially separating the pulses according to wavelength, thereby obtaining a plurality of separated elements for each wavelength and each angle.
12. The high-speed three-dimensional imaging method according to claim 11, wherein the plurality of two-dimensional image information are detected by forming an image of the separation elements on an image sensor.
13. The high-speed three-dimensional imaging method according to claim 10, wherein the stretched pulse has a wavelength dispersion that enables image information to be acquired at frame intervals of 10 ns or less.
14. A high-speed imaging method comprising: making an elongated pulse consisting of a plurality of parts with different wavelengths incident on an object from different angles, so that each part of each elongated pulse incident from a different angle holds image information of the object that depends on the angle for a time corresponding to the wavelength; and spatially separating the elongated pulses holding the image information for each wavelength and for each angle, and detecting them as a plurality of two-dimensional image information for each wavelength and for each angle.
Citation Information
Patent Citations
Apparatus and method of reconstituting three-dimensional shape on the basis of single captured image
JP2013192203A
Measuring device and measuring method
JP2023015388A
High-speed imaging system and method
JP6120280B2
