A compact long-sequence frequency domain reconstruction ultra-high-speed imaging system and method

Abstract

The present invention relates to a compact long-sequence frequency domain reconstruction ultra-high-speed imaging system and method, in order to solve the technical problem that the complexity of the system and the difficulty of operation increase when using multiple exposure frequency domain reconstruction technology to process a video with a large number of frames. The system includes a pulse light source and a structured light sequence generation unit that performs signal transmission in sequence, a high-speed scene generation unit, an image acquisition unit, and a data processing unit; the structured light sequence generation unit includes a beam splitting diffraction optical element, a first Fourier lens, a step-shaped delay line, and a second Fourier lens; the control unit is connected to the pulse light source and the image acquisition unit, respectively. The method includes: 1. The pulse light source emits a pulse; 2. N pairs of light pulses with different propagation delays are obtained; 3. A structured light sequence consisting of N sinusoidal stripe structured light pulses is obtained; 4. A structured signal light sequence superimposed image is obtained; 5. A reconstructed high-speed image of a dynamic scene is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of ultra-high-speed imaging, and in particular to a compact long-sequence frequency-domain reconstruction ultra-high-speed imaging system and method. Background Art

[0002] Digital high-speed cameras have been used to monitor a wide range of transient events, from laser ablation and plasma formation to cell dynamics and geological material studies. In order to increase the sensor frame rate, various electronic sensor architectures have been invented, such as charge-coupled devices (CCDs), complementary metal oxide sensors (CMOS), and in-situ storage image sensors (ISIS). However, this usually comes at a cost in the form of reduced sequence depth and / or pixel count. For example, the state-of-the-art Photron SA-Z records video with a sensor size of 1024×1024 pixels at a speed of 20kfps (fps, frames per second) (see reference: J. Manin, SASkeen, and LM Pickett, "Performance comparison of state-of-the-art high-speed video cameras for scientific applications," Opt. Eng. 57, 121706 (2018)), while at a frame rate of 2.1Mfps, only a sensor size of 128×8 pixels can be recorded. A fast sensor technology with similar pixel resolution (32×32 pixels) is able to achieve acquisition speeds up to 25Mfps, however, its sequence depth is significantly reduced at high speeds to 1220 frames (see: VTSDao, N. Ngo, AQ Nguyen, K. Morimoto, K. Shimonomura, P. Goetschalckx, L. Haspeslagh, P. De Moor, K. Takehara, and TGE Toh, "An image signal accumulation multi-collection-gate image sensor operating at 25Mfps with 32×32 pixels and 1220in-pixel frame memory," Sensors 18, 3112 (2018)). To overcome some of the limitations of single-sensor-based fast imaging, alternative camera solutions have emerged.For example, a multi-channel intensified camera can capture up to 16 1024×1024 pixel images at a speed of 200 Mfps (see reference: A. Roth, D. Frantz, W. Chaze, A. Corber, and E. Berrocal, “High-speed imaging database of water jet disintegration part I: quantitative imaging using liquid laser-induced fluorescence,” Int. J. Multiph. Flow 145, 103641 (2021)).

[0003] In contrast, frequency domain reconstruction of multiple exposures (FRAME) can achieve high temporal and spatial resolution and high frame rate. It is an active high-speed photography technology based on spectrum framing technology. However, as the sequence depth (number of framing) increases, FRAME faces the problem that the complexity of the system and the difficulty of operation will increase sharply. Summary of the invention

[0004] The present invention solves the technical problem of increased system complexity and operational difficulty when using multiple exposure frequency domain reconstruction technology to process a video with a large number of frames, and proposes a compact long-sequence frequency domain reconstruction ultra-high-speed imaging system and method.

[0005] The technical solution provided by the present invention is:

[0006] A compact long-sequence frequency domain reconstruction ultra-high-speed imaging system, which is special in that it includes a pulse light source, a structured light sequence generation unit, a high-speed scene generation unit, an image acquisition unit, a data processing unit and a control unit;

[0007] The pulse light source is used to emit a pulse and output it to the structured light sequence generating unit;

[0008] The structured light sequence generating unit includes a beam splitting diffractive optical element, a first Fourier lens, a stepped delay line and a second Fourier lens in sequence along the pulse transmission direction; the beam splitting diffractive optical element receives the pulse and outputs N pairs of light pulses, N≥2, N is an integer and is equal to the number of divided frames of the target scene to be measured in the high-speed scene generating unit, and each pair of light pulses is symmetrical with respect to the central axis of the light column formed by the N pairs of light pulses;

[0009] The distance between the first Fourier lens and the beam splitting diffractive optical element is F1, and F1 is the focal length of the first Fourier lens; the step delay line comprises a first sub-helical delay line and a second sub-helical delay line symmetrically arranged along the central axis, the first sub-helical delay line and the second sub-helical delay line are spiral bodies with flat lower surfaces, and the upper surfaces of the spiral bodies are spiral surfaces with an angle of 180°; the spiral lower end side surface of the first sub-helical delay line is connected to the spiral upper end side surface of the second sub-helical delay line, and the spiral upper end side surface of the first sub-helical delay line is connected to the spiral lower end side surface of the second sub-helical delay line; the central axis of the step delay line is perpendicular to the lower surfaces of the first sub-helical delay line and the second sub-helical delay line; it can be understood that the thickness of the two positions symmetrical with respect to the central axis of the step delay line is equal along the axial direction, so that when N pairs of optical pulses pass through the step delay line, the same pair of optical pulses obtain the same propagation delay on the step delay line, and different pairs of optical pulses obtain different propagation delays on the step delay line;

[0010] The distance between the second Fourier lens and the high-speed scene generation unit is F2, and F2 is the focal length of the second Fourier lens; the second Fourier lens focuses N pairs of light pulses and outputs them to the receiving end of the high-speed scene generation unit, so that the same pair of light pulses generates a sinusoidal stripe structure light pulse at the receiving end of the high-speed scene generation unit;

[0011] The control unit is connected to the pulse light source and the image acquisition unit respectively. The control unit receives a signal for starting the pulse light source and transmits the signal to the image acquisition unit for controlling the image acquisition unit to start after N pairs of light pulses pass through the high-speed scene generation unit. The signal output by the high-speed scene generation unit is received by the image acquisition unit, and the image acquisition unit is connected to the data processing unit. The data processing unit is used to receive the image signal collected by the image acquisition unit to reconstruct a high-speed image.

[0012] Furthermore, the upper surface of the spiral body is a 180° spiral profile, specifically: the upper surfaces of the first sub-spiral delay line and the second sub-spiral delay line are both provided with N steps of different thicknesses, and the steps symmetrical along the central axis have the same axial thickness.

[0013] Furthermore, the upper surface of the spiral body is a 180° spiral surface, specifically: the upper surfaces of the first sub-spiral delay line and the second sub-spiral delay line are spiral planes, and the axial thicknesses at positions symmetrical along the central axis are the same.

[0014] Furthermore, the step-shaped delay line is made of quartz glass, and the relationship between the axial thickness of the step-shaped delay line and the propagation delay is 1*10 -11 s / mm.

[0015] Furthermore, the image acquisition unit is at least one CCD camera or CMOS camera.

[0016] The present invention also provides a compact long sequence frequency domain reconstruction ultra-high speed imaging method, which is special in that the above-mentioned compact long sequence frequency domain reconstruction ultra-high speed imaging system is used, and the method comprises the following steps:

[0017] S1, the pulse light source emits a pulse;

[0018] S2, the pulse is split by the beam splitting diffractive optical element to generate N pairs of light pulses, N ≥ 2, N is an integer and is equal to the number of frames of the target scene to be measured in the high-speed scene generation unit;

[0019] S3, N pairs of light pulses enter the step-shaped delay line after passing through the first Fourier lens, the positions of the same pair of light pulses entering the step-shaped delay line are symmetrical with respect to the central axis, and have the same propagation delay, thereby obtaining N pairs of light pulses with different propagation delays; the time intervals of the N pairs of light pulses with different propagation delays are equal to the frame interval of the target scene to be measured in the high-speed scene generation unit, and the pulse width is equal to the time resolution;

[0020] S4, N pairs of light pulses with different propagation delays are focused by a second Fourier lens, and two light pulses in the same pair of light pulses generate sinusoidal stripe structured light pulses at the receiving end of the high-speed scene generation unit, thereby obtaining a structured light sequence consisting of N sinusoidal stripe structured light pulses, wherein each sinusoidal stripe structured light pulse has a different spatial frequency magnitude and direction;

[0021] The structured light sequence illuminates the target scene to be measured generated by the high-speed scene generation unit to form a structured light sequence carrying dynamic information of the target scene to be measured at different times;

[0022] S5, the image acquisition unit acquires images of the structured light sequence carrying the dynamic information of the target scene to be measured at different times, and performs one exposure to form a structured signal light sequence superposition image;

[0023] S6. The data processing unit receives the structured signal light sequence superposition image output by the image acquisition unit, and processes the structured signal light sequence superposition image by using a frequency domain reconstruction algorithm to obtain a reconstructed dynamic scene high-speed image.

[0024] Beneficial effects of the present invention:

[0025] 1. The compact long-sequence frequency domain reconstruction ultra-high-speed imaging system proposed in this invention generates terahertz-level (10 12 Hz) to petahertz level (10 15Hz) frame rate light pulse sequence, using the diffraction and interference of light to generate fringes, solves the complex problems of the FRAME imaging system. Compared with the traditional frequency domain reconstruction high-speed imaging system, the system provided by the present invention is dozens of times smaller. When the number of frames increases, the system provided by the present invention will be hundreds or even thousands of times smaller than the traditional system, and has obvious miniaturization advantages.

[0026] 2. The present invention generates stripe-shaped pulse light of different directions and spatial frequencies by setting a diffractive optical element, which can ensure the accuracy of the position of the image sequence in the frequency domain. The stepped delay line with a symmetrical 180° spiral surface can also ensure the accuracy of the sequence time interval. Therefore, compared with the traditional frequency domain reconstruction high-speed imaging system, the system provided by the present invention has high reliability and stability, and is more cost-effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of an embodiment of a compact long-sequence frequency domain reconstruction ultra-high-speed imaging system according to the present invention;

[0028] Figure 2 Schematic diagram of the structure of a structured light sequence generating unit in an embodiment of the present invention;

[0029] Figure 3 Schematic diagram of the structure of a stepped delay line in an embodiment of the present invention.

[0030] The reference numerals are as follows:

[0031] 1-pulse light source, 2-structured light sequence generating unit, 3-high-speed scene generating unit, 4-image acquisition unit, 5-data processing unit, 6-beam splitting diffraction optical element, 7-first Fourier lens, 8-step delay line, 9-second Fourier lens, 10-control unit. DETAILED DESCRIPTION

[0032] See also Figure 1 This embodiment provides a compact long-sequence frequency domain reconstruction ultra-high-speed imaging system, which includes a pulse light source 1, a structured light sequence generating unit 2, a high-speed scene generating unit 3, an image acquisition unit 4, a data processing unit 5 and a control unit 10.

[0033] The pulse light source 1 is used to generate a pulse and output it to the structured light sequence generating unit 2; see Figure 2 The structured light sequence generating unit 2 includes a beam splitting diffractive optical element 6, a first Fourier lens 7, a stepped delay line 8 and a second Fourier lens 9 in sequence along the pulse transmission direction.

[0034] The beam splitting diffraction optical element 6 (DOE, diffraction optical element) receives pulses and outputs four pairs of light pulses, the number of light pulse logarithm is equal to the number of frame divisions of the target scene to be measured in the high-speed scene generation unit 3, and each pair of light pulses is symmetrical with respect to the central axis of the light column formed by N pairs of light pulses; the distance between the first Fourier lens 7 and the beam splitting diffraction optical element 6 is F1, and F1 is the focal length of the first Fourier lens 7.

[0035] The step-shaped delay line 8 comprises a first sub-helical delay line and a second sub-helical delay line symmetrically arranged along the central axis. The first sub-helical delay line and the second sub-helical delay line are spiral bodies with flat lower surfaces, and the upper surfaces of the spiral bodies are spiral surfaces of the same shape of 180°; see Figure 3 In this embodiment, steps with axial thicknesses of h1, h2, h3 and h4 are respectively provided on the surfaces of the first sub-spiral delay line and the second sub-spiral delay line, and h1<h2<h3<h4, and the steps of the same thickness on the first sub-spiral delay line and the second sub-spiral delay line are symmetrical with respect to the central axis; it can be understood that the first sub-spiral delay line and the second sub-spiral delay line may also be provided with no steps on their surfaces, and both are spiral planes, and the axial thickness of the spiral body is set as required, so that the axial thicknesses passed by the same pair of optical pulses when passing through the stepped delay line 8 are the same; the spiral low end side surface of the first sub-spiral delay line is connected to the spiral high end side surface of the second sub-spiral delay line, and the first sub-spiral delay line is connected to the spiral high end side surface of the second sub-spiral delay line. The spiral high-end side of the spiral delay line is connected to the spiral low-end side of the second sub-spiral delay line; the central axis of the step-shaped delay line 8 is perpendicular to the lower surface of the first sub-spiral delay line and the second sub-spiral delay line; the thickness of the step-shaped delay line 8 along the axial direction is equal at two positions symmetrical to the central axis, so that when N pairs of light pulses pass through the step-shaped delay line 8, the same pair of light pulses obtain the same propagation delay on the step-shaped delay line 8, and different pairs of light pulses obtain different propagation delays on the step-shaped delay line 8; the step-shaped delay line 8 adjusts the propagation delay of N pairs of light pulses through the thickness, and controls the time interval of the light pulses (i.e., the frame interval of the target scene to be measured in the high-speed scene generation unit 3). In this embodiment, the step-shaped delay line 8 is made of quartz glass, and the axial propagation delay of different pairs of light pulses through the step-shaped delay line 8 is 1*10 -11 s / mm, that is, the axial thickness of the two pairs of optical pulses passing through the step delay line 8 differs by 1 mm, and the propagation delay is 1*10 -11 s; It can be understood that, in other embodiments, other suitable materials can also be selected to adjust the axial thickness of the stepped delay line 8 according to the light refractive index of the material.

[0036] The distance between the second Fourier lens 9 and the high-speed scene generation unit 3 is F2, and F2 is the focal length of the second Fourier lens 9; the second Fourier lens 9 focuses N pairs of light pulses and outputs them to the receiving end of the high-speed scene generation unit 3, so as to make the same pair of light pulses generate a sinusoidal stripe structured light pulse at the receiving end of the high-speed scene generation unit 3.

[0037] The control unit 10 is connected to the pulse light source 1 and the image acquisition unit 4 respectively. The control unit 10 receives a start signal of the pulse light source 1 and transmits the signal to the image acquisition unit 4 for controlling the image acquisition unit 4 to start after N pairs of light pulses pass through the high-speed scene generation unit 3; the signal output by the high-speed scene generation unit 3 is received by the image acquisition unit 4, and the image acquisition unit 4 is connected to the data processing unit 5; the data processing unit 5 is used to receive the image signal collected by the image acquisition unit 4 to reconstruct a high-speed image.

[0038] In this embodiment, N=2. It can be understood that the compact long-sequence frequency domain reconstruction ultra-high-speed imaging system provided in this embodiment is flexible and expandable, and the number of imaging frames and the frame rate can be set according to the needs of the diffractive optical element and the stepped delay line 8.

[0039] The present invention is provided with a diffractive optical element that generates N pairs of centrally symmetric light pulses and a stepped delay line 8 whose axial height is symmetric about the center. A camera composed of a single detector can achieve high frame rate (>1THz), multi-frame (>4 frames), and full-frame high-speed imaging. The image acquisition unit 4 can be a CCD camera or a CMOS camera; it can also be expanded according to the needs of the system and use multiple cameras.

[0040] The imaging method of the above-mentioned compact long-sequence frequency domain reconstruction ultra-high-speed imaging system comprises the following steps:

[0041] S1, pulse light source 1 emits a pulse;

[0042] S2, the pulse is split by the beam splitting diffractive optical element 6 to generate N pairs of light pulses, N ≥ 2, N is an integer and is equal to the number of frames of the target scene to be measured in the high-speed scene generation unit 3;

[0043] S3, N pairs of light pulses enter the step-shaped delay line 8 after passing through the first Fourier lens 7, and the positions of the same pair of light pulses entering the step-shaped delay line 8 are symmetrical with respect to the central axis and have the same propagation delay, thereby obtaining N pairs of light pulses with different propagation delays; the time intervals of the N pairs of light pulses with different propagation delays are equal to the frame interval of the target scene to be measured in the high-speed scene generation unit 3, and the pulse width and time resolution are equal;

[0044] S4, N pairs of light pulses with different propagation delays are focused by the second Fourier lens 9, and at the receiving end of the high-speed scene generation unit 3, two light pulses in the same pair of light pulses generate sinusoidal stripe structured light pulses at the receiving end of the high-speed scene generation unit 3, and a structured light sequence consisting of N sinusoidal stripe structured light pulses is obtained, wherein each sinusoidal stripe structured light pulse has a different spatial frequency magnitude and direction;

[0045] The structured light sequence illuminates the target scene to be measured generated by the high-speed scene generation unit 3 to form a structured light sequence carrying dynamic information of the target scene to be measured at different times;

[0046] S5, the image acquisition unit 4 acquires images of the structured light sequence carrying dynamic information of the target scene at different times, and performs one exposure to form a structured signal light sequence superimposed image;

[0047] S6. The data processing unit 5 receives the structured signal light sequence superposition image output by the image acquisition unit 4, and processes the structured signal light sequence superposition image using a frequency domain reconstruction algorithm to obtain a reconstructed dynamic scene high-speed image.

Claims

1. A compact long-sequence frequency domain reconstruction ultra-high-speed imaging system, characterized in that: It comprises a pulse light source (1), a structured light sequence generating unit (2), a high-speed scene generating unit (3), an image acquisition unit (4), a data processing unit (5) and a control unit (10); The pulse light source (1) is used to generate a pulse and output it to the structured light sequence generating unit (2); The structured light sequence generating unit (2) comprises, in sequence along the pulse transmission direction, a beam splitting diffractive optical element (6), a first Fourier lens (7), a step-shaped delay line (8) and a second Fourier lens (9); the beam splitting diffractive optical element (6) receives the pulse and outputs N pairs of light pulses, N≥2, N is an integer and is equal to the number of frames of the target scene to be measured in the high-speed scene generating unit (3), and each pair of light pulses is symmetrical with respect to the central axis of the light column formed by the N pairs of light pulses; The distance between the first Fourier lens (7) and the beam splitting diffractive optical element (6) is F1, and F1 is the focal length of the first Fourier lens (7); the step-shaped delay line (8) comprises a first sub-helical delay line and a second sub-helical delay line symmetrically arranged along the central axis, the first sub-helical delay line and the second sub-helical delay line are spiral bodies with flat lower surfaces, and the upper surfaces of the spiral bodies are 180° spiral surfaces; the spiral lower end side surface of the first sub-helical delay line is connected to the spiral upper end side surface of the second sub-helical delay line, and the spiral upper end side surface of the first sub-helical delay line is connected to the spiral lower end side surface of the second sub-helical delay line; the central axis of the step-shaped delay line (8) is perpendicular to the lower surfaces of the first sub-helical delay line and the second sub-helical delay line; The distance between the second Fourier lens (9) and the high-speed scene generation unit (3) is F2, and F2 is the focal length of the second Fourier lens (9); the second Fourier lens (9) focuses the N pairs of light pulses and outputs them to the receiving end of the high-speed scene generation unit (3), so that the same pair of light pulses generates a sinusoidal stripe structure light pulse at the receiving end of the high-speed scene generation unit (3); The control unit (10) is connected to the pulse light source (1) and the image acquisition unit (4) respectively; the control unit (10) receives a start signal of the pulse light source (1) and transmits the signal to the image acquisition unit (4) for controlling the image acquisition unit (4) to start after N pairs of light pulses pass through the high-speed scene generation unit (3); the signal output by the high-speed scene generation unit (3) is received by the image acquisition unit (4), and the image acquisition unit (4) is connected to a data processing unit (5); the data processing unit (5) is used to receive the image signal collected by the image acquisition unit (4) to reconstruct a high-speed image.

2. The compact long-sequence frequency domain reconstruction ultra-high-speed imaging system according to claim 1, characterized in that: The upper surface of the spiral body is a 180° spiral surface, specifically: the upper surfaces of the first sub-spiral delay line and the second sub-spiral delay line are both provided with N steps of different thicknesses, and the steps symmetrical along the central axis have the same axial thickness.

3. The compact long-sequence frequency domain reconstruction ultra-high-speed imaging system according to claim 1, characterized in that: The upper surface of the spiral body is a 180° spiral surface, specifically: the upper surfaces of the first sub-spiral delay line and the second sub-spiral delay line are spiral planes, and the axial thicknesses at positions symmetrical along the central axis are the same.

4. The compact long-sequence frequency domain reconstruction ultra-high-speed imaging system according to any one of claims 1 to 3, characterized in that: The step-shaped delay line (8) is made of quartz glass, and the relationship between the axial thickness of the step-shaped delay line (8) and the propagation delay is 1*10 -11 s / mm.

5. The compact long-sequence frequency domain reconstruction ultra-high-speed imaging system according to claim 4, characterized in that: The image acquisition unit (4) is at least one CCD camera or a CMOS camera.

6. A compact long-sequence frequency domain reconstruction ultra-high-speed imaging method, characterized in that: The compact long sequence frequency domain reconstruction ultra-high speed imaging system according to any one of claims 1 to 5 comprises the following steps: S1, a pulse light source (1) emits a pulse; S2, the pulse is split by the beam splitting diffractive optical element (6) to generate N pairs of light pulses, N≥2, N is an integer and is equal to the number of frames of the target scene to be measured in the high-speed scene generation unit (3); S3, N pairs of light pulses enter the step-shaped delay line (8) after passing through the first Fourier lens (7), the positions of the same pair of light pulses entering the step-shaped delay line (8) are symmetrical with respect to the central axis and have the same propagation delay, thereby obtaining N pairs of light pulses with different propagation delays; the time intervals of the N pairs of light pulses with different propagation delays are equal to the frame interval of the target scene to be measured in the high-speed scene generation unit (3), and the pulse width and time resolution are equal; S4, N pairs of light pulses with different propagation delays are focused by a second Fourier lens (9), and two light pulses in the same pair of light pulses generate sinusoidal stripe structured light pulses at the receiving end of the high-speed scene generation unit (3), thereby obtaining a structured light sequence consisting of N sinusoidal stripe structured light pulses, wherein each sinusoidal stripe structured light pulse has a different spatial frequency magnitude and direction; The structured light sequence illuminates the target scene to be measured generated by the high-speed scene generation unit (3), forming a structured light sequence carrying dynamic information of the target scene to be measured at different times; S5, the image acquisition unit (4) acquires images of the structured light sequence carrying dynamic information of the target scene to be measured at different times, and performs one exposure to form a structured signal light sequence superimposed image; S6. The data processing unit (5) receives the structured signal light sequence superposition image output by the image acquisition unit (4), and processes the structured signal light sequence superposition image using a frequency domain reconstruction algorithm to obtain a reconstructed dynamic scene high-speed image.

Images