High-speed three-dimensional photographing device and method

The high-speed 3D imaging system and method address the limitations of two-dimensional capture by using stretched pulses and multi-angle illumination to reconstruct 3D images of high-speed phenomena, enabling detailed analysis of complex events with sub-nanosecond resolution.

JP2024165221APending Publication Date: 2024-11-28THE UNIV OF TOKYO

Patent Information

Application Number
JP2023081172
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-17
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Existing high-speed imaging methods are limited to two-dimensional capture of high-speed phenomena, making it difficult to analyze complex events with electromagnetic waves, non-uniform materials, and stochastic events, and require assumptions of three-dimensional profiles that are often inaccurate.

Method used

A high-speed 3D imaging system and method using a light source with a delay generator to generate stretched pulses of different wavelengths, a multi-angle illuminator, spatial separators, and an image sensor to capture and reconstruct 3D images from multiple angles, allowing continuous imaging of high-speed phenomena.

Benefits of technology

Enables single-shot continuous imaging of three-dimensional profiles of high-speed phenomena, providing detailed analysis of events like laser-induced plasma and shock waves with sub-nanosecond resolution, overcoming the limitations of two-dimensional imaging.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024165221000001_ABST
    Figure 2024165221000001_ABST
Patent Text Reader

Abstract

To provide a method capable of serial photographing a three-dimensional profile of a high-speed phenomenon.SOLUTION: A method includes: retaining image information of an object dependent on an angle at a time corresponding to wavelength in each part of each elongation pulse incident from a different angle by irradiating an elongation pulse, composed of a plurality of parts with different wavelengths, onto the object from different angles; detecting the elongation pulse which retains the image information as a plurality of pieces of two-dimensional image information for each wavelength and angle by spatially separating the elongation pulse; and reconstructing the three-dimensional image from the plurality of pieces of two-dimensional image information at the plurality of angles for each wavelength.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an apparatus and method for high speed three-dimensional imaging. [Background technology]

[0002] Ultrafast imaging is a powerful tool for understanding the details of high-speed phenomena occurring in the femtosecond to nanosecond time domain, such as plasmas and shock waves. By completely eliminating mechanical and electrical operations, optical high-speed imaging techniques with time resolution of less than nanoseconds have been developed to overcome the limitations of the imaging speed of high-speed cameras and to continuously capture phenomena that cannot be captured by high-speed cameras (Patent Document 1, Non-Patent Document 1).

[0003] The three-dimensional shape of an object (high-speed phenomenon) is defined as the three-dimensional distribution of the optical properties of the object, that is, as a three-dimensional profile. Here, the existing high-speed imaging techniques with time resolution of nanoseconds or less (Patent Document 1 and Non-Patent Document 1) are limited to two-dimensional capture of the space. Therefore, in order to analyze a high-speed phenomenon, it is necessary to assume a three-dimensional profile taking into account the axial symmetry of the phenomenon, etc.

[0004] However, for complex phenomena that require measurement, it is difficult to assume a three-dimensional profile due to the non-uniformity of electromagnetic waves and materials and stochastic events, and conventional methods cannot obtain accurate three-dimensional profiles of three-dimensionally complex high-speed phenomena. Non-Patent Document 2 discloses a method of spatially separating and acquiring information on a phenomenon from multiple angles using a lens array and reconstructing the three-dimensional distribution, but continuous shooting with a resolution of less than nanoseconds is not possible. Therefore, an ultra-high-speed single-shot shooting technology that can measure the three-dimensional profile of the target is required. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6120280 [Non-patent literature]

[0006] [Non-Patent Document 1] K. Nakagawa, A. Iwasaki, Y. Oishi, R. Horisaki, A. Tsukamoto, A. Nakamura, K. Hirosawa, H. Liao, T. Ushida, K. Goda, F. Kannari and I. Sakuma, “Sequentially timed all-optical mapping photography (STAMP)” Nature Photonics 8, 695-700 (2014). [Non-Patent Document 2] KL Baker, “Tomographic reconstruction of high-energy-density plasmas with picosecond temporal resolution” Optics Letters 31(6), 730-732 (2006). [Non-Patent Document 3] H. Kudo et al., “Image reconstruction for sparse-view CT and interior CT-introduction to compressed sensing and differentiated” Quantitative imaging in medicine and surgery (2013) DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]

[0007] An object of the present invention is to provide a technique capable of continuously capturing three-dimensional profiles of high-speed phenomena. [Means for solving the problem]

[0008] The first technical means adopted in the present invention is: A light source; a delay generator that stretches a pulse emitted from a light source to generate a stretched pulse having a plurality of portions with different wavelengths; a multi-angle illuminator for splitting the stretched pulse to be incident on a target from different angles; a spatial separator for spatially separating each stretched pulse, in which image information of an object depending on an angle is held in each portion at a time corresponding to a wavelength, for each wavelength and angle while the image information is held, to generate separated elements; an image sensor that detects each of the separated elements holding image information as a plurality of two-dimensional image information; a three-dimensional reconstruction means for reconstructing a three-dimensional image from a plurality of two-dimensional image information at a plurality of angles for each wavelength; It is a high-speed 3D imaging device consisting of:

[0009] In one embodiment, the light source is an ultrashort pulsed light source. In one embodiment, the stretched pulse has a chromatic dispersion (time delay as a function of wavelength) that allows image information to be acquired with frame intervals of 10 ns or less. In one embodiment, the stretched pulse is a pulse train. In one embodiment, a plurality of delay generators capable of generating a plurality of types of stretched pulses capable of providing image information with different frame intervals may be provided, and these delay generators may be selectable.

[0010] In one embodiment, the multi-angle illuminator includes a diffractive optical element that splits the stretched pulse provided from the delay generator into multiple stretched pulses.

[0011] In one embodiment, the spatial separator includes a first optical system that spatially separates each stretched pulse according to angle while preserving image information, and a second optical system that spatially separates each stretched pulse according to wavelength. In one embodiment, the first optical system includes a lens and a lens array. In one embodiment, the second optical system includes a diffractive optical element and a bandpass filter.

[0012] In one embodiment, the spatial separator spatially separates each stretched pulse such that the multiple two-dimensional image information are separately imaged on an image sensor.

[0013] In one embodiment, the image sensor is selected from a CCD image sensor, a CMOS image sensor. In one embodiment, said three-dimensional reconstruction means is a 3D tomographic reconstruction.

[0014] The second technical means adopted in the present invention is: A light source; a delay generator that stretches a pulse emitted from a light source to generate a stretched pulse having a plurality of portions with different wavelengths; a multi-angle illuminator for splitting the stretched pulse to be incident on a target from different angles; a spatial separator for spatially separating each stretched pulse, in which image information of an object depending on an angle is held in each portion at a time corresponding to a wavelength, for each wavelength and angle while the image information is held, to generate separated elements; an image sensor that detects each of the separated elements holding image information as a plurality of two-dimensional image information; This is a high-speed imaging device consisting of:

[0015] The third technical means adopted in the present invention is: By directing stretched pulses having a plurality of portions with different wavelengths at different angles to an object, each portion of the stretched pulses incident at a different angle holds image information of the object that is angle-dependent at a time corresponding to the wavelength; spatially separating the stretched pulses holding the image information for each wavelength and for each angle, and detecting the separated images as a plurality of pieces of two-dimensional image information for each wavelength and for each angle; Reconstructing a three-dimensional image from multiple two-dimensional image information at multiple angles for each wavelength; A high-speed 3D imaging method including:

[0016] In one embodiment, each stretched pulse is separated in a first separation step in which it is spatially separated according to angle while image information is preserved, and in a second separation step in which it is spatially separated according to wavelength, thereby obtaining multiple separation elements for each wavelength and angle. The temporal order in which the first separation step and the second separation step are performed is not limited, and the first separation step may be performed first, the second separation step may be performed first, or the first separation step and the second separation step may be performed substantially simultaneously.

[0017] In one embodiment, the plurality of two-dimensional image information is detected by imaging the separation elements onto an image sensor. In one embodiment, the plurality of pieces of two-dimensional image information are separately imaged on an image sensor.

[0018] In one embodiment, the stretched pulse is obtained by stretching an ultrashort pulse. In one embodiment, the stretched pulse has a chromatic dispersion (time delay as a function of wavelength) that allows image information to be acquired with frame intervals of 10 ns or less.

[0019] The fourth technical means adopted in the present invention is: By directing stretched pulses having a plurality of portions with different wavelengths at different angles to an object, each portion of the stretched pulses incident at a different angle holds image information of the object that is angle-dependent at a time corresponding to the wavelength; spatially separating the stretched pulses holding the image information for each wavelength and for each angle, and detecting the separated images as a plurality of pieces of two-dimensional image information for each wavelength and for each angle; A high-speed photography method comprising:

[0020] When considering obtaining a three-dimensional profile of a high-speed phenomenon, the obtained three-dimensional information may differ depending on the object. For example, when photographing an object whose image information is determined by the absorption coefficient of the object, a three-dimensional absorption coefficient distribution is obtained. When photographing an object whose image information is determined by the scattering coefficient, a three-dimensional scattering coefficient is obtained. When a non-transparent object is the target, the three-dimensional distribution and shape of the object are obtained. Since the three-dimensional information obtained depending on the object is due to the optical properties of the object to be photographed, in this specification, the "three-dimensional profile of the object" can be rephrased as the "three-dimensional distribution of the optical properties of the object." Effect of the Invention

[0021] The present invention enables single-shot continuous photography of three-dimensional profiles of high-speed phenomena occurring in the time domain of nanoseconds or less. In conventional methods, the space that can be acquired is limited to two dimensions, so analysis is performed by assuming a three-dimensional profile such as axial symmetry. However, for complex phenomena that require measurement, it is difficult to make such assumptions due to the inhomogeneity of electromagnetic waves and materials and stochastic events. The present invention enables detailed analysis of high-speed phenomena by photographing the three-dimensional profile of the target high-speed phenomenon, without the need to assume a three-dimensional profile of the high-speed phenomenon.

[0022] This invention makes it possible to visualize the three-dimensional dynamics of ultrafast phenomena, the details of which were previously unknown because the space that could be obtained was limited to two dimensions. This makes it possible to analyze a variety of high-speed phenomena, such as laser-induced plasma, dielectric breakdown, discharge, shock waves, crystal chemical reactions, and acoustic bioeffects, which are important in many fields of science and technology, including processing, energy, the environment, and medicine. [Brief description of the drawings]

[0023] [Figure 1] 1 is an overall view of a high-speed three-dimensional imaging system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a flow diagram showing a high-speed three-dimensional imaging method according to the present embodiment. [Diagram 3]FIG. 1 is a conceptual diagram showing an embodiment of a delay generator for realizing a frame interval of 10 ps or less (100 fs to 10 ps). [Figure 4] FIG. 1 is a conceptual diagram showing an embodiment of a delay generator for realizing a frame interval of 10 ps to 50 ps. [Diagram 5] FIG. 1 is a conceptual diagram showing an embodiment of a delay generator for realizing a frame interval of 50 ps to several ns (10 ns). [Figure 6] Stretched pulses (pulse trains) with different pulse intervals are illustrated. [Figure 7] FIG. 2 is a diagram showing a multi-angle illuminator (stretched pulse splitter) according to an embodiment of the present invention. [Figure 7A] FIG. 13 illustrates a multi-angle illuminator according to another embodiment. [Figure 7B] 13A and 13B are diagrams illustrating a multi-angle illuminator according to yet another embodiment. [Figure 8] The illumination system (multi-angle illuminator) and detection system according to this embodiment are shown in FIG. [Figure 9] FIG. 2 is a conceptual diagram illustrating a detection system according to the present embodiment. [Figure 10] FIG. 1 is a conceptual diagram showing image information (two-dimensional image) collected on an image sensor and a reconstructed three-dimensional image. [Figure 11] FIG. 10 is a conceptual diagram of the optical system according to FIG. [Figure 12] FIG. 10 is a diagram showing the optical system according to FIG. 9(B). [Figure 13] FIG. 10 is a diagram showing the optical system according to FIG. 9(B). [Figure 14] FIG. 10 is a diagram showing the optical system according to FIG. [Figure 15] FIG. 10 is a diagram showing the optical system according to FIG. [Figure 16] FIG. 9 is a diagram similar to FIG. 8 and illustrating the design of the optical system. [Figure 17] 9(C) shows a projection diagram of the optical system according to FIG. 9(C) onto an image sensor. [Figure 18] FIG. 13 is a diagram illustrating an example of imaging of a high-speed phenomenon in which all projected images at each wavelength and angle are spatially discretized. [Figure 19] 1A to 1C are diagrams illustrating a three-dimensional reconstruction method according to the present embodiment. [Figure 20] FIG. 1 is a diagram for explaining the correspondence between a stretched pulse consisting of multiple parts with different wavelengths and the time interval β of the wavelength components of the separation elements (image information) separated by the spatial separator. The upper diagram shows the time interval α in a pulse train exemplified as a stretched pulse. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] [A] Overview of high-speed 3D imaging system and method [A-1] Overview of high-speed 3D imaging system An outline of the three-dimensional high-speed photography system according to this embodiment is illustrated in FIG. 1. The three-dimensional high-speed photography system according to this embodiment includes an illumination system that generates illumination light for illuminating an object, an imaging section in which the object is placed, and a detection system that detects the emitted light (light transmitted or reflected by the object) from the imaging section, and the illumination light from the illumination system illuminates the object (phenomenon) in the imaging section. The emitted light (transmitted light or reflected light) from the imaging section holds image information of the object (phenomenon), and the image information held in the emitted light is detected in the detection system. The three-dimensional high-speed photography system according to this embodiment includes a three-dimensional reconstruction means (3D moving image reconstruction device) that reconstructs a three-dimensional image from multiple two-dimensional images at multiple angles at each wavelength using image information (multiple two-dimensional image information) detected by the detection system, and obtains three-dimensional moving image information of the object by three-dimensionally reconstructing the detected image information (two-dimensional image information).

[0025] The illumination system consists of an ultrashort pulse light source, a delay generator (pulse stretcher) that stretches the pulse emitted from the ultrashort pulse light source to generate a stretched pulse consisting of multiple parts with different wavelengths, and a pulse splitter (multi-angle illuminator) that splits the stretched pulse and makes it incident on the object from different angles. In the illumination system, multiple illumination lights consisting of multiple parts with different wavelengths are prepared, and these illumination lights are made to be incident on the phenomenon from multiple angles.

[0026] In the imaging section, multiple irradiation lights (stretched pulses) are irradiated onto the object from different angles, and each part (wavelength component) holds image information of the object that depends on different angles at the time corresponding to the wavelength. For an ultrafast phenomenon, when a stretched pulse consisting of multiple parts with different wavelengths (for example, a pulse train consisting of pulses with different wavelengths) is incident from multiple angles, the pulse with wavelength λ1 passes through the phenomenon at time t1, and the pulse with wavelength λ5 passes through the phenomenon at time t5, and each pulse (λ1, λ2, ... λ n ) passing timing t1~t n Since the wavelengths are different, image information corresponding to the times at which the respective wavelengths are held is acquired. Each stretched pulse that holds the image information is reflected from the imaging unit at different angles θ1 to θ2 as, for example, transmitted light or reflected light. m It is emitted at.

[0027] The detection system detects different angles θ1 to θ m The imaging device comprises a spatial separator that spatially separates each stretched pulse emitted from the imaging unit by wavelength and angle while image information is maintained, and an imaging element or image sensor that detects the light spatially separated by the spatial separator, i.e., the separated elements of the stretched pulses spatially separated while image information is maintained, as a plurality of two-dimensional image information. Typically, the emitted light or separated elements (two-dimensional image information) spatially separated by the spatial separator are imaged and detected at different positions on the light receiving surface of the image sensor.

[0028] In the detection system, all image information of pulses of each angle and each wavelength is collected on an image sensor as a separation element by a spatial separator, so that two-dimensional images of multiple angles at each wavelength, i.e., at each time, can be acquired, and a three-dimensional profile of the object can be tomographically reconstructed from each two-dimensional image to acquire an ultra-high-speed three-dimensional video. In the three-dimensional high-speed shooting system according to this embodiment, if the detection system is taken in a broad sense, the 3D video reconstruction device is a component of the detection system, and if the detection system is taken in a narrow sense, the three-dimensional high-speed shooting system according to this embodiment can be considered to be composed of an illumination system (optical system), an imaging unit, a detection system (optical system), and a 3D video reconstruction device (which can be composed of a computer).

[0029] [A-2] Overview of high-speed 3D imaging method The high-speed three-dimensional imaging method according to the present embodiment includes: making an elongated pulse consisting of a plurality of parts with different wavelengths incident on an object from different angles, so that each part of each elongated pulse incident from a different angle holds image information of the object depending on the angle at a time corresponding to the wavelength; spatially separating the elongated pulse holding the image information for each wavelength and for each angle, and detecting it as a plurality of two-dimensional image information for each wavelength and for each angle; and reconstructing a three-dimensional image from the plurality of two-dimensional image information at each wavelength and for each angle. In the case where the elongated pulse is a pulse train and image information is held in the pulse train as the projection light, the high-speed three-dimensional imaging method includes making a pulse train with different wavelengths incident on a phenomenon from a plurality of angles, so that each pulse acquires a different projection image depending on the incidence timing and incidence direction; and spatially separating and detecting all the projection images for each time and each angle, so as to continuously capture a three-dimensional profile of the high-speed phenomenon.

[0030] The frame interval achievable in the high-speed imaging according to the present embodiment is determined by the wavelength dispersion of the stretched pulse (time delay according to wavelength). The separation elements generated by the spatial separator have time information according to their wavelength components, and this time information depends on the wavelength dispersion of the stretched pulse (time delay according to wavelength). FIG. 20 is a diagram for explaining the correspondence between the stretched pulse consisting of a plurality of parts with different wavelengths generated by the delay generator and the time interval (i.e., frame interval) β of the wavelength components of the separation elements (image information) separated by the spatial separator, and the upper diagram shows the time interval α in a pulse train exemplified as the stretched pulse. The frame interval β corresponds to and depends on the time interval α of the pulse train. Note that the time interval α of the pulse train can be substantially the same as the frame interval β, but this is not guaranteed.

[0031] An overview of the high-speed 3D imaging method will be described with reference to Fig. 2. An ultrashort pulse emitted from an ultrashort pulse light source is stretched by a pulse stretcher (delay generator) to generate a desired number n of wavelength components λ1 to λ n In FIG. 2, an elongated pulse having five wavelength components is assumed. The elongated pulse is divided by a multi-angle illuminator (pulse splitter) and is incident (illuminated) on the target from multiple m different angles. In FIG. 2, nine angles are assumed (m=9), and an elongated pulse having five wavelength components (n=5) is incident on the target from nine different angles. The elongated pulses emitted from the target at nine different angles each hold image information in five wavelength components, and a total of 45 (m×n) pieces of image information are acquired depending on the incidence timing and incidence direction. The elongated pulses emitted in nine directions are separated by wavelength and angle by a spatial separator to generate 45 (m×n) separated lights (separation elements), and the 45 separated lights are detected by an image sensor to acquire a 45 (m×n) two-dimensional image. The wavelength components λ1 to λ n By reconstructing 2D images into 3D images for each λ1 to λ n The times t1 to t nIn the illustrated example, at each time t, a 3D image at that time t is reconstructed from nine 2D images. In Fig. 2, five time-series 3D images at times t1 to t5 corresponding to λ1 to λ5 are obtained.

[0032] Important points in the 3D high-speed imaging according to the present embodiment are: (i) acquiring image information from multiple angles at a certain time, and reconstructing a 3D profile at that time based on the image information; and (ii) acquiring image information at different times according to wavelengths from stretched pulses incident from the same angle. In order to reconstruct 3D image information at a certain time, it is necessary for specific wavelength components of stretched pulses incident from different angles to be simultaneously incident. This is because image information from multiple angles must be simultaneously acquired in order to perform 3D reconstruction. Therefore, it is necessary to acquire image information from multiple angles at the same time.

[0033] As long as the above conditions are satisfied, multiple stretched pulses incident on the target from different angles may include stretched pulses having different wavelength components. For example, the wavelengths of pulses incident from different angles may be different. Specifically, a stretched pulse (pulse train) at a certain angle may be composed of wavelength components (λ1, λ2, λ3), and a stretched pulse at another certain angle may be composed of wavelength components (λ4, λ5, λ6). In addition, a situation in which the spatial profile or polarization of stretched pulses incident from different angles is not the same may be assumed, other than the wavelength. Also, all stretched pulses do not have to be incident at the same timing. For example, stretched pulses at angles θ1 to θ4 may be incident at the same timing (time t1 to time t4), and stretched pulses at angles θ5 to θ8 may be incident at different timings (time t5 to time t8), and 3D reconstruction may be performed in the time domains acquired for each, resulting in a 3D video image from time t1 to time t8.

[0034] [B] Lighting system [B-1] Overview The illumination system according to this embodiment comprises an ultrashort pulse light source that emits ultrashort pulse light, a delay generator (pulse stretcher) that stretches the pulse emitted from the ultrashort pulse light source to generate a stretched pulse consisting of multiple parts with different wavelengths, and a pulse splitter (multi-angle illuminator) that splits the stretched pulse and makes it incident on the target from different angles. In the illumination system, first, an ultrashort pulse laser light is subjected to a first optical system using an optical system using material dispersion such as glass or a structural dispersion such as a grating pair to generate a different delay time for each wavelength. Next, a second optical system using a diffractive optical element or the like splits the stretched pulse into multiple stretched pulses, which are each incident on the target to be photographed from a different angle. In this way, each wavelength component (λ1, λ2, ...λ n ) pulses arrive at different times (t1, t2, ... t) depending on the chromatic dispersion (time delay according to wavelength). n ) to obtain image information of the target from multiple angles.

[0035] Typically, the stretched pulse is a pulse train, but is not limited to this. As long as it is "illumination light (stretched pulse) consisting of multiple parts with different wavelengths" or "illumination light (stretched pulse) with a different delay time for each wavelength", it does not need to be discretized into multiple pulses as a pulse train, and pulses in which each wavelength is continuously stretched in time can also be used as illumination light.

[0036] [B-2] Light source The light source according to this embodiment is typically a single ultrashort pulse light source, and generates an elongated pulse from an ultrashort pulse using a delay generator. Note that light sources of multiple wavelengths may be operated individually. For example, multiple short-wavelength semiconductor pulse lasers (picosecond pulse lasers, nanosecond pulse lasers) may be used in combination.

[0037] When performing pulse stretching from an ultrashort pulse, current delay generators realize frame intervals of up to about 10 ns, and theoretically frame intervals of 100 ns or more are also possible. When multiple light sources are used in combination, imaging at frame intervals of μs or ms may also be possible. In principle, the present invention is applicable not only to light (visible light to near-infrared light) but also to the entire electromagnetic wave range (X-rays, THz, radio waves, etc.) and electron beams as long as the wave is present, and it will be understood by those skilled in the art that the term "light source" can be appropriately replaced depending on the radiation source.

[0038] [B-3] Delay generator (pulse stretcher) The first optical system of the illumination system according to this embodiment is a delay generator (pulse stretcher) that stretches the ultrashort pulse laser light. The delay generator may be configured, for example, to generate a different delay time for each wavelength by an optical system using material dispersion such as glass or an optical system using structural dispersion such as a grating pair.

[0039] The generation of pulse trains with different wavelengths will be described with reference to Figs. 3 to 5. Fig. 3 is a conceptual diagram of a delay generator for realizing a frame interval of 100 fs to 10 ps. The delay generator shown in Fig. 3 utilizes the material dispersion of glass, and an ultrashort pulse laser light is propagated in the glass using a mirror pair. A desired frame interval Δt can be obtained according to Δt = D(λ)LΔλ, where D(λ) is the dispersion coefficient, L is the path length, and Δλ is the wavelength difference. Such a delay generator can be constructed using a glass block or a glass rod.

[0040] FIG. 4 is a conceptual diagram of a delay generator for achieving a frame interval of 10 ps to 50 ps, ​​which is constructed using a grating pair and a mirror. The desired frame spacing Δt can be obtained according to Δt≈2λΔs / (dc), where d is the groove spacing, λ is the wavelength, and c is the speed of light.

[0041] Figure 5 is a conceptual diagram of a delay generator for realizing a frame interval of 50 ps to several ns, where the embodiment shown in the upper figure is constructed using a diffraction grating pair and a group of mirrors. The embodiment shown in the lower figure is constructed using a mirror pair, a diffraction grating pair, and a mirror. The desired frame interval Δt can be obtained according to Δt ≈ 4L / c, where L is the distance between the mirror pairs and c is the speed of light.

[0042] In this embodiment, stretched pulses with a frame interval of 100 ps to 10 ns can be realized by using three different types of delay generators. FIG. 6 shows an example of a stretched pulse (a pulse train consisting of multiple wavelength components) generated by stretching using a delay generator. In FIG. 6, Interval means "time interval of a pulse train" or "time interval of adjacent pulses on the time axis". In FIG. 6, the time scales on the horizontal axis are different, and the intervals between each pulse correspond to the Interval. If a changeover switch for switching the optical path is provided in the first optical system of the illumination system, it is possible to selectively use three types of delay generators according to a desired frame interval. Also, the delay generators may be appropriately rearranged according to a desired frame interval without using a changeover switch. The delay generators (pulse stretchers) shown in FIG. 3 to FIG. 5 are examples, and it is understood that a person skilled in the art can design a specific first optical system for realizing a desired frame interval.

[0043] [B-4] Pulse divider (multi-angle illuminator) The second optical system of the illumination system according to this embodiment is a pulse splitter (multi-angle illuminator) that splits an extended pulse and makes the multiple extended pulses incident on the target from different angles. As shown in FIG. 7, the pulse splitter (multi-angle illuminator) according to this embodiment splits an extended pulse into multiple parts by an optical system using a diffractive optical element and a lens, and makes each of the pulses incident on the target to be photographed (focused and illuminated) from multiple different angles. In this way, pulses of each wavelength component (λ1, λ2, ... λ n ) at different times (t1, t2, ... t n) to obtain projected images of the target from multiple angles. FIG. 7A shows a multi-angle illuminator according to another embodiment, which is composed of a mirror array and a lens, and a cross section of one light beam is incident across multiple mirrors of the mirror array, so that the single light beam is spatially separated into multiple light beams, and then the single light beam is incident on the target using a lens. FIG. 7B shows a multi-angle illuminator according to yet another embodiment, which is composed of a diffractive optical element and a curved mirror array, and the light beam split by the diffractive optical element is reflected by each curved mirror array and incident on the target. The pulse splitters (multi-angle illuminators) shown in FIG. 7, FIG. 7(A), and FIG. 7(B) are examples, and it is understood that a person skilled in the art can design a specific optical system for irradiating stretched pulses from multiple desired angles.

[0044] [C] Imaging section A target (e.g., an ultrafast phenomenon) placed in the imaging section is irradiated with multiple irradiation lights (stretched pulses) from multiple angles. When stretched pulses (pulse trains) consisting of multiple different wavelength components are incident on the ultrafast phenomenon from multiple angles, the pulse with wavelength λ1 passes through the phenomenon at time t1, and the pulse with wavelength λ5 passes through the phenomenon at time t5. Each pulse (λ1, λ2, ... λ n ) passing times t1, t2,...t n Since the wavelengths are different, image information is acquired at the time corresponding to each wavelength. n ) at different times (t1, t2, ... t n ) to acquire image information of the object from multiple angles. Each stretched pulse is a time sequence (t1, t2, ... t n The expanded pulses, which hold the time-series image information, are output from the imaging section at different angles as, for example, transmitted light or reflected light.

[0045] [D] Detection system [D-1] Overview In the detection system, the image information obtained by the illumination light incident from different angles is separated and acquired on the image sensor. The detection system is composed of a spatial separator that spatially separates each stretched pulse emitted from the imaging unit at different angles for each wavelength and angle while maintaining the image information, and an imaging element or image sensor that detects the light spatially separated by the spatial separator, i.e., each separated element of the stretched pulse spatially separated while maintaining the image information, as multiple pieces of two-dimensional image information.

[0046] In the detection system, first, a spatial separator consisting of a lens array, a diffractive optical element, a bandpass filter, etc. is used to spatially separate the two-dimensional image information for each angle and each wavelength, and the information is captured on an image sensor that is kept in an exposed state. In this way, the two-dimensional image information for each wavelength, that is, each time (t1, t2, ...t n ) at multiple angles θ1 to θ m Using the two-dimensional image information (e.g., projected image) detected for each wavelength and angle, a three-dimensional image is reconstructed (tomographic reconstruction) from two-dimensional images taken at multiple angles at each time (=wavelength), thereby obtaining a three-dimensional profile of the target (high-speed phenomenon).

[0047] [D-2] Space separator The spatial separator according to this embodiment separates the stretched pulses that hold the image information by using both the wavelength and angle parameters, and captures a two-dimensional image on the image sensor. The following three modes of detection (capturing) on ​​the image sensor can be exemplified. 1. When the pulses for each angle and each wavelength (= each time) are completely separated and detected as a two-dimensional image (x, y) (Figure 9 (B) and (C)). 2. When the two-dimensional images (x, y) for each angle and each wavelength (= each time) are detected in a state where they are not overlapping, but are not grouped together as individual two-dimensional images (x, y) (Figure 9(A)). 3. Two-dimensional images (x, y) for each angle and each wavelength (= each time) overlap, but the image can be reconstructed using special assumptions and operations, such as compressed sensing.

[0048] One of the important points in the spatial separator according to this embodiment is how to aggregate all information required for reconstruction in a limited two-dimensional area called an image sensor. In this embodiment, by independently manipulating both wavelength and angle parameters, the emitted light that holds image information is separated to capture a two-dimensional image on the image sensor. In this embodiment, the disclosures of Patent Document 1 and Non-Patent Document 1, which manipulate only the wavelength, and the disclosures of Non-Patent Document 2, which manipulate only the angle, differ in that both wavelength and angle parameters are independently manipulated. The spatial separator according to this embodiment includes a first optical system that spatially separates each stretched pulse according to the angle while the image information is held, and a second optical system that spatially separates each stretched pulse according to the wavelength. This specification discloses three examples of the spatial separator. Please note that the configuration of the spatial separator described below (such as a combination of multiple optical elements that constitute each optical system) is an example and does not limit the spatial separator according to the present invention.

[0049] [D-2-1] Spatial separator (first embodiment) A first embodiment of the spatial separator is shown in Figures 9(A) and 11. In the first embodiment, first, the emitted light that holds the image information is largely separated by angle, a two-dimensional space is arranged overall, and each point (focused spot) is divided by time to form an image. The optical system (imaging method after passing through the phenomenon) according to the first embodiment is composed of a lens array system that separates by angle, a microlens array system that separates the two-dimensional space as a focus point distribution, and a spectroscopic system that minutely displaces each wavelength. As a result, each parameter is separated on the image sensor and imaged as a focus point distribution.

[0050] As shown in Fig. 11, the spatial separator according to the first embodiment comprises an angle separation means made up of an objective lens and a lens array, a spatial separation means made up of a microlens array, and a wavelength (time) separation means made up of a lens, a diffraction grating, and a lens. The lens and lens array work together to spatially separate image information captured from each angle. The microlens array converts the two-dimensional space (x, y) into a distribution of focused spots. The lens, diffraction grating, and lens work together to separate the distribution of focused spots by wavelength. As in the second and third embodiments described below, the set of lens and lens array can be replaced with a set of lens and mirror array.

[0051] [D-2-2] Spatial separator (second embodiment) A second embodiment of the spatial separator is shown in Figs. 9(B), 12, and 13. As shown in Fig. 9(B), in the second embodiment, the emitted light carrying image information is separated by angle on the image sensor, and then separated by time therein, and each is imaged as a two-dimensional space. In Fig. 12, the optical system is composed of a lens, a first lens array, a diffractive optical element, a bandpass filter, and a second lens array. As shown in Fig. 13, in the second embodiment, the lens, the lens array, the diffractive optical element, and the bandpass filter are considered to be an example of a minimum configuration that realizes this. In the optical system of Fig. 13, the lens and the lens array are a set and spatially separate the image information captured from each angle. The diffractive optical element and the bandpass filter further separate the image information of each wavelength. In the optical system shown in Fig. 12, the lens, the first lens array, the diffraction grating, and the bandpass filter have the same role as the optical system of Fig. 13, and the second lens array is used to image each image information on the image sensor. In the embodiment shown in Fig. 13, the function of imaging is included in the lens array.

[0052] In the optical system shown in Fig. 12, the set of the diffractive optical element and bandpass filter can be replaced with a slice mirror system (a set of a diffraction grating, a lens, and a slice mirror) as described in the third embodiment. Furthermore, in the optical system shown in Fig. 12, the set of the lens and the first lens array can be replaced with the set of the lens and the mirror array as described in the third embodiment. Furthermore, when using a set of the diffractive optical element and bandpass filter, it is also possible to replace the second lens array behind it with a single lens to form an image.

[0053] [D-2-3] Spatial separator (third embodiment) A third embodiment of the spatial separator is shown in Fig. 9(C), Fig. 14, and Fig. 15. In the third embodiment, the emitted light carrying image information is separated by time on the image sensor, and then separated by angle therein, and each is imaged as a two-dimensional space. In Fig. 14, the optical system of the detection system is composed of a lens, a lens array, a diffractive optical element, a bandpass filter, and a lens. As shown in Fig. 15, in the third embodiment, the lens, the lens array, the diffractive optical element, and the bandpass filter are considered to be an example of a minimum configuration that realizes this. In Fig. 14, one more lens is used behind the bandpass filter, but this is due to the actual optical design (problems of magnification and the arrangement and specifications of each element), and Fig. 15 shows one of the minimum configurations in terms of separating the angle and wavelength.

[0054] In Figure 14, the lens and lens array work together to spatially separate the image information captured from each angle. The diffractive optical element and bandpass filter separate the image information for each wavelength. The diffractive optical element splits the pulse into multiple angular directions, the bandpass filter selects the wavelength for each (transmitting different wavelength components according to the angle of incidence), and the lens is responsible for forming an image on the image sensor. As shown in Figure 15, when no lens is included at the rear, the function of forming an image on the image sensor is included in the lens array.

[0055] In the optical system according to the third embodiment, as a configuration in which the set of lens and lens array can be replaced, a configuration using a set of lens and mirror array as described in the following document can be adopted. S. Yeola et al., “Single-shot ultrafast imaging via spatiotemporal division of femtosecond laser pulses” Journal of the Optical Society of America B (2018).

[0056] As a configuration that can replace the set of diffractive optical element, bandpass filter, and lens, a configuration using a slice mirror system as described in the following document can be adopted. Specifically, it is (pick-off mirror →) diffraction grating 1 → lens 1 → slice mirror → lens 1 → diffraction grating 1 → (imaging) lens → image sensor. Even with this configuration, it is possible to separate images by wavelength. T. Saiki et al., “Sequentially timed all-optical mapping photography boosted by a branched 4f system with a slicing mirror” Optics Express (2020).

[0057] With reference to FIG. 16, a specific design of the optical system relating to the illumination system and the detection system will be mentioned. In FIG. 16, the illumination system is composed of a diffractive optical element DOE, a lens L1, and a lens L2, and causes an elongated pulse to be incident on a target T from a plurality of angles. The detection system is composed of a lens L3, a lens array LA, a diffractive optical element DOE, a bandpass filter BPF, and a lens L4. In the figure, CP is a conjugate plane (intermediate image), D0 is an original beam diameter, D is a beam diameter when passing through the target (corresponding to a field of view), θ is an incidence angle of the diffracted light on the target, and s is a distance from the target to a focusing point of each pulse. Examples of parameters for designing the optical system shown in FIG. 16 include the focal length of each lens L1 to L4, the focal length of the lens array LA, D, θ, and θ0 (diffraction angle of DOE). Those skilled in the art can design an optimal optical system by adjusting each parameter. It should be noted that the parameters that can be used may vary depending on the optical elements that constitute the actual optical system.

[0058] In order to increase the spatial resolution, three factors are important: (i) increasing the numerical aperture (NA), (ii) increasing the magnification of the imaging system, and (iii) preventing the pulses at each angle from overlapping. For (i), it is important to increase θ, and for (ii) and (iii), it is important to reduce D and use an optical arrangement that can separate each pulse in the lens array (LA) section.

[0059] For example, by designing the pulses to overlap just before they are focused, it is possible to make D less than 50 μm. In order to observe a micro-scale object, it is important to narrow the beam diameter of each pulse to about several tens of μm at the target (T). In this embodiment, the optical design is such that the probe pulses from different angles are incident on the target just before they are focused, and all of them can overlap.

[0060] In this embodiment, in order to prevent overlap, each pulse is separated by a lens array (LA) and imaged at a conjugate plane (CP). In this embodiment, the distance between the lens L3 and the lens array LA is adjusted so that each pulse after passing through LA becomes a parallel light. In this way, the pulse emitted from the imaging unit is made to enter the rear optical system as a parallel light.

[0061] [D-3] Image sensor In this embodiment, the spatial separator separates the emitted light that holds the image information by wavelength and angle, and all the image information of the pulses (separated light) at each angle and wavelength is collected on the image sensor. This makes it possible to obtain two-dimensional images from multiple angles at each wavelength, i.e., at each time. These two-dimensional images are then tomographically reconstructed to obtain ultra-high-speed three-dimensional moving images, i.e., a three-dimensional profile of the target (high-speed phenomenon).

[0062] 9, four two-dimensional images taken from multiple angles θ1 to θ4 at time t1, four two-dimensional images taken from multiple angles θ1 to θ4 at time t2, four two-dimensional images taken from multiple angles θ1 to θ4 at time t3, four two-dimensional images taken from multiple angles θ1 to θ4 at time t4, and four two-dimensional images taken from multiple angles θ1 to θ4 at time t5 are detected (imaged) in a spatially separated manner on the light receiving surface of the image sensor. Each two-dimensional image is specified by xy coordinates.

[0063] Examples of the imaging element include a CCD image sensor and a CMOS image sensor, but the imaging element that can be used in this embodiment is not limited, and examples include a SPAD image sensor, an image intensifier, an EMCCD camera, an InGaAs camera, etc., in addition to the CCD image sensor and the CMOS image sensor. The number of imaging elements is not limited to one, and an image of the target may be formed on multiple imaging elements to detect the image. In detecting a two-dimensional image, a device with a light receiving surface with a larger area (an area that allows multiple spatially separated light beams to be incident on different positions) or multiple CCD elements or photographic plates may be prepared.

[0064] When the spatial separator (third embodiment) is used, the emitted light carrying image information is separated by time on the image sensor, and then separated by angle therein, and each is imaged as a two-dimensional space. In Fig. 17, illumination light is incident on a phenomenon from a 3x3 angle, and the acquired image information is separated into 3x3 angles and five wavelengths by a spatial separator equipped with an optical system according to the third embodiment, resulting in 45 pieces of image information that are incident on different positions of one image sensor and detected (projected images). Fig. 18 shows a projection image of a high-speed phenomenon (laser ablation plasma) with an angle of 3x3 and five wavelengths.

[0065] [E] 3D video reconstruction device (3D tomographic reconstruction) In the three-dimensional video reconstruction device according to the present embodiment, a three-dimensional image is reconstructed (tomographic reconstruction) from 2D images of multiple angles at each time (=wavelength) to obtain an ultra-high-speed three-dimensional video. As shown in Fig. 9, the three-dimensional reconstruction means reconstructs a three-dimensional image at time t1 from four two-dimensional images from multiple angles θ1 to θ4 at time t1, reconstructs a three-dimensional image at time t2 from four two-dimensional images from multiple angles θ1 to θ4 at time t2, reconstructs a three-dimensional image at time t3 from four two-dimensional images from multiple angles θ1 to θ4 at time t3, reconstructs a three-dimensional image at time t4 from four two-dimensional images from multiple angles θ1 to θ4 at time t4, and reconstructs a three-dimensional image at time t5 from four two-dimensional images from multiple angles θ1 to θ4 at time t5, and a three-dimensional video of the target can be obtained from each two-dimensional image at times t1 to t5.

[0066] As shown in Fig. 17, 3D reconstruction using multiple 2D images acquired by an image sensor can be performed using 3D tomographic reconstruction. The stretched pulse emitted from the imaging unit at an angle θ2 holds image information at times t1 to t5, ... corresponding to wavelengths λ1 to λ5, ..., and the image information is separated into separation elements by the spatial separator, and each separation element is detected as a 2D image 2D(λ1,θ2), 2D(λ2,θ2), 2D(λ3,θ2), 2D(λ4,θ2), 2D(λ5,θ2), ... on the image sensor. Similarly, the stretched pulse emitted at angle θ3 holds image information from time t1 to t5, corresponding to wavelengths λ1 to λ5, and the image information is separated into separate elements by the spatial separator, and each separate element is detected as a two-dimensional image 2D(λ1,θ3), 2D(λ2,θ3), 2D(λ3,θ3), 2D(λ4,θ3), 2D(λ5,θ3), and so on on the image sensor. A three-dimensional image at a given time is reconstructed using multiple two-dimensional images acquired at the same time. For example, a three-dimensional image 3D(t1) is reconstructed using two-dimensional images 2D(λ1,θ2), 2D(λ1,θ3), 2D(λ1,θ4), 2D(λ1,θ5), and so on acquired at time t1 corresponding to λ1. Similarly, by reconstructing 3D images 3D(t2), 3D(t3), 3D(t4), 3D(t5) ..., a high-speed 3D video sequence is obtained from these 3D images.

[0067] Tomographic reconstruction is a technique in which reconstruction is performed from images obtained at multiple angles, and is described in, for example, Non-Patent Documents 3 and 4. The 3D reconstruction unit can be configured from a computer (e.g., equipped with an input unit, a memory, a processor, an output unit, a display unit, etc.). Specific means for 3D reconstruction are known to those skilled in the art, and examples include Zernike polynomial expansion; Maximum Likelihood-Expectation Maximization (MLEM); Minimum Fisher Information (MFI); and Deep Neural Network. [Industrial Applicability]

[0068] The high-speed imaging system according to the present invention can be configured by combining an illumination system and a detection system that are made up of widely used optical elements and equipment, and the 3D reconstruction unit can be configured from a computer. The system is easy to build and has the flexibility to be optimized according to the purpose. It can be applied to existing optical systems and equipment, particularly incorporation into microscopes. In addition, by combining it with an imaging element and computer that are capable of high-speed readout, it is possible to configure a high-grade high-speed video camera that can capture ultra-high-speed phenomena for a longer period of time.

[0069] The present invention provides a method capable of capturing 3D moving images with a single shot at a resolution of less than nanoseconds. The present invention is useful for the development of products involving high-speed phenomena (such as laser processing machines and electric discharge processing machines). First, by visualizing the high-speed phenomena during processing, it is believed that the theory of the processing phenomenon can be clarified, making it possible to design parameters to realize an optimal processing process. Furthermore, monitoring and feedback control by implementing it in an actual processing machine are possible. In laser processing and electric discharge processing, it is expected that errors from the design will occur in the processing process depending on the nonuniformity of the material and the instability of the input parameters (laser, voltage), and since the input parameters and material surface shape are strongly related to the three-dimensional dynamics of high-speed phenomena such as plasma and discharge during processing, it is expected that by monitoring these in high-speed 3D, the processing process can be optimized by feeding back the errors. Therefore, there is a wide range of possible applications, including surface treatment and processing of composite materials, and even laser treatment for biological tissue.

Claims

1. A light source; a delay generator that stretches a pulse emitted from a light source to generate a stretched pulse having a plurality of portions with different wavelengths; a multi-angle illuminator for splitting the stretched pulse to be incident on a target from different angles; a spatial separator for spatially separating each stretched pulse, in which image information of an object depending on an angle is held in each portion at a time corresponding to a wavelength, for each wavelength and angle while the image information is held, to generate separated elements; an image sensor that detects each of the separated elements holding image information as a plurality of pieces of two-dimensional image information; a three-dimensional reconstruction means for reconstructing a three-dimensional image from a plurality of two-dimensional image information at a plurality of angles for each wavelength; A high-speed 3D imaging device consisting of:

2. The light source is an ultrashort pulse light source.

2. The high-speed three-dimensional imaging device according to claim 1.

3. the stretched pulse has a chromatic dispersion that allows image information to be acquired at a frame interval of 10 ns or less; 3. The high-speed three-dimensional imaging device according to claim 1.

4. the multi-angle illuminator includes a diffractive optical element that splits the stretched pulse incident from the delay generator into a plurality of stretched pulses; 2. The high-speed three-dimensional imaging device according to claim 1.

5. 2. The high-speed 3D imaging device of claim 1, wherein the spatial separator includes a first optical system that spatially separates each stretched pulse according to angle while image information is preserved, and a second optical system that spatially separates each stretched pulse according to wavelength.

6. The first optical system includes a lens and a lens array.

6. The high-speed three-dimensional imaging device according to claim 5.

7. the second optical system includes a diffractive optical element and a bandpass filter; 7. The high-speed three-dimensional imaging device according to claim 5 or 6.

8. the spatial separator spatially separates each stretched pulse to generate the separated elements such that the plurality of two-dimensional image information are separately imaged on an image sensor; 2. The high-speed three-dimensional imaging device according to claim 1.

9. A light source; a delay generator that stretches a pulse emitted from a light source to generate a stretched pulse having a plurality of portions with different wavelengths; a multi-angle illuminator for splitting the stretched pulse to be incident on a target from different angles; a spatial separator for spatially separating each stretched pulse, in which image information of an object depending on an angle is held in each portion at a time corresponding to a wavelength, for each wavelength and angle while the image information is held, to generate separated elements; an image sensor that detects each of the separated elements holding image information as a plurality of pieces of two-dimensional image information; A high-speed photographing device consisting of

10. By directing stretched pulses having a plurality of portions with different wavelengths at different angles to an object, each portion of the stretched pulses incident at a different angle holds image information of the object that is angle-dependent at a time corresponding to the wavelength; spatially separating the stretched pulses holding the image information for each wavelength and for each angle, and detecting the separated pulses as a plurality of pieces of two-dimensional image information for each wavelength and for each angle; Reconstructing a three-dimensional image from a plurality of two-dimensional image information at a plurality of angles at each wavelength; A high-speed three-dimensional imaging method comprising:

11. Separating each stretched pulse in a first separation step in which the stretched pulses are spatially separated according to angle while image information is maintained, and in a second separation step in which the stretched pulses are spatially separated according to wavelength, thereby obtaining a plurality of separation elements for each wavelength and each angle. The method of high speed three-dimensional imaging according to claim 10.

12. The separation elements are imaged on an image sensor to detect the plurality of pieces of two-dimensional image information. The method of high speed three-dimensional imaging according to claim 11.

13. the stretched pulse has a chromatic dispersion that allows image information to be acquired at a frame interval of 10 ns or less; The method of high speed three-dimensional imaging according to claim 10.

14. By directing stretched pulses having a plurality of portions with different wavelengths at different angles to an object, each portion of the stretched pulses incident at a different angle holds image information of the object that is angle-dependent at a time corresponding to the wavelength; spatially separating the stretched pulses holding the image information for each wavelength and for each angle, and detecting the separated pulses as a plurality of pieces of two-dimensional image information for each wavelength and for each angle; A high-speed photography method including:

Citation Information

Patent Citations

  • Magnetic recording and reproducing system

    JP1986020280A

Cited By

  • Time-stretch spectroscopy device and time-stretch spectroscopy method

    WO2025178022A1