Real-time holographic acquisition and reproduction system and method for real scene
By designing a real-time holographic acquisition and reproduction system, using infrared and visible light sources to alternately acquire and display, combined with light intensity transmission equation and spatial light modulator, the problem of holographic acquisition and display in the prior art is difficult to perform in real time, and efficient Three-dimensional reconstruction and display effects.
Patent Information
- Application Number
- CN202411235638.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-09-04
AI Technical Summary
The prior art is difficult to realize real-time holographic acquisition and display of real scenes, the optical holographic acquisition is poor in portability, the real-time holographic acquisition is poor in computing holographic acquisition, and it is difficult to collect and display at the same time.
A real-time holographic acquisition and reproduction system for real scenes is designed. Through infrared light source module, visible light source module, light guide device and infrared imaging device, combined with calculation control module, an alternating holographic acquisition and display process is realized, and the light intensity transmission equation is used. Process reflected light, generate coded holograms, and realize real-time three-dimensional reconstruction through spatial light modulators.
Real-time alternation of holographic acquisition and display is realized, avoiding light source interference, optimizing the calculation and processing flow, significantly improving real-time and three-dimensional display effects, and achieving a display effect with a 24Hz refresh rate.
Smart Images

Figure CN118938628A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of holographic technology, and in particular, relates to a real-time holographic acquisition and reproduction system and method for real scenes. Background Art
[0002] Since its invention in the 1940s, holographic technology has been regarded as the most promising technology in the field of 3D display. Holographic technology provides complete optical information to the human eye, ensuring comfortable focus adjustment and non-violated parallax cues. Holographic technology includes two processes, one is holographic acquisition and the other is holographic display. The holographic acquisition process and the holographic display process are often separated. There is almost no technology that can perform holographic acquisition and display in real time, that is, holographic acquisition and holographic display at the same time.
[0003] At present, there are two main ways of holographic acquisition for three-dimensional display. One method is optical holographic acquisition, which uses the interference effect of the object beam and the reference beam to record interference fringes with complete information of the object on the holographic dry plate. The other is the computational holographic method, which first converts the real object or scene into a three-dimensional model that can be understood by the computer program, and then encodes the three-dimensional model into a hologram. The disadvantages of optical acquisition are that it requires the construction of a complex optical path, resulting in poor portability of the system, and secondly, it is difficult to achieve dynamic acquisition. The disadvantage of computational holographic acquisition is that there is an intermediate process of a three-dimensional model between the real scene object and the hologram, which makes the real-time performance of the acquisition very poor.
[0004] In order to better apply holographic technology in the fields of augmented reality and virtual reality, the holographic acquisition and reproduction of real scenes is an indispensable part, and a digital holographic system that can reconstruct in real time is in great demand. Therefore, in the field of computational holography, it is urgent to solve the problem of real-time holographic acquisition and display.
[0005] Patent CN102024272A discloses a device and method for obtaining a computational hologram of a three-dimensional moving object. The device uses a computer to control a camera to obtain the object's spectrum information and depth information through a lens array, and finally encodes it into a hologram to achieve the acquisition of a computational hologram. However, when acquiring depth information, the device needs to take 18 pictures, and the hologram encoding through the depth element requires complex Fourier transform operations, which makes it difficult to achieve real-time acquisition, and thus it is difficult to achieve real-time holographic acquisition and holographic display.
[0006] Patent CN100429587C authorizes a holographic stereoscopic image recording and display method, device and application thereof. The authorized method uses a camera to continuously shoot scenes in circular motion to obtain two-dimensional images at different angles, and then converts the two-dimensional image set into a holographic negative film by optical methods, and finally uses it for holographic display. In theory, the three steps of continuous shooting of scenes, computer processing of two-dimensional images, and exposure of holographic negative films are difficult to complete in a short time, so it is difficult to achieve real-time holographic acquisition and holographic display. Summary of the invention
[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a real-time holographic acquisition and reproduction system and method for real scenes, which integrates the holographic acquisition device and the holographic display device into an integrated design so that the real scene can be holographically reproduced in real time, effectively solving the problem that holographic acquisition and holographic display of real scenes are difficult to perform in real time.
[0008] The technical solution of the present invention is as follows:
[0009] A real-time holographic acquisition and reproduction system for real scenes, which is characterized by:
[0010] an infrared light source module configured to provide collimated infrared light during holographic acquisition;
[0011] A visible light source module configured to provide collimated visible light during holographic display;
[0012] A light guiding device, configured as at least two beam splitters, for deflecting, splitting and guiding a light beam; specifically, in the process of holographic acquisition, the infrared light emitted by the infrared light source is guided to an object in a real scene, and the reflected light of the object is guided to an imaging device through a spatial light modulator, and in the process of holographic display, the visible light emitted by the visible light source module is guided to a spatial light modulator to generate a holographic image of the object;
[0013] Infrared imaging device, used to successively capture reflected light with different propagation distances carrying object information during holographic acquisition, and generate corresponding intensity maps;
[0014] The computing control module is respectively connected to the infrared light source module, the visible light source module, the light guide device, and the infrared imaging device and controls their working timings. Specifically, the computing control module controls the light emission timings of the infrared light source module and the visible light source module to ensure that the two emit light alternately without overlapping. During the holographic acquisition process, that is, the infrared light source module is turned on and the visible light source module is turned off, the infrared imaging device is controlled to capture reflected light at at least two different propagation distances to generate at least two corresponding intensity maps, wherein the absolute value of the difference between the second propagation distance and the first propagation distance is the effective propagation distance of the light intensity transmission equation, and the two intensity maps are processed and calculated to obtain a coded hologram carrying object information; during the holographic display process, that is, the infrared light source module is turned off and the visible light source module is turned on, the spatial light modulator is controlled to load the coded hologram to generate a holographically reconstructed three-dimensional image of the object.
[0015] Furthermore, the effective propagation distance of the light intensity transmission equation is 1 to 10 microns.
[0016] Preferably, the infrared light source module includes an infrared light source and an infrared collimating lens, the visible light source module includes a visible coherent light source and a visible collimating lens therewith, and the infrared imaging device includes an infrared imaging sensor and an imaging lens; the spatial light modulator is connected to the computing control module, and is used to modulate the object light wave during the holographic acquisition process to change the propagation distance of the light wave; and to load the encoded hologram during the holographic display process to generate a holographic image of the object; wherein, during the holographic acquisition process, the infrared imaging device, with the assistance of the spatial light modulator, successively captures reflected light with different propagation distances carrying object information and generates a corresponding intensity map.
[0017] Preferably, the infrared light source module includes an infrared light source and an infrared collimating lens, the visible light source module includes a visible coherent light source and a visible collimating lens therewith, and the infrared imaging device includes at least two groups of infrared imaging sensors and imaging lenses, so as to ensure that reflected light carrying object information and having different propagation distances is captured during the holographic acquisition process and a corresponding intensity map is generated.
[0018] The infrared imaging device is used to capture infrared light and generate corresponding electrical signals.
[0019] The light guiding device comprises at least three beam splitters.
[0020] Preferably, the light guiding device is composed of three beam splitters, and the light guiding device has, in accordance with the light propagation direction, the following in the holographic collection branch light path: an infrared light source module light inlet, a real scene light outlet, a real scene light inlet, a spatial light modulator light outlet, a spatial light modulator light inlet, and an infrared imaging device light outlet, wherein the spatial light modulator light outlet and the spatial light modulator light inlet coincide with each other in the holographic collection branch light path and the holographic display branch light path; the infrared imaging device is composed of a group of infrared imaging sensors and imaging lenses, and the infrared imaging sensors are located at the infrared imaging light outlet of the light guiding device.
[0021] Preferably, the light guide device is composed of four beam splitters, and the light guide device has two independent and non-overlapping infrared imaging device light outlets in the holographic collection branch light path; the infrared imaging device is composed of two groups of infrared imaging sensors and imaging lenses, and the two infrared imaging sensors correspond to the two infrared imaging light outlets of the light guide device. The axial distances between the two infrared imaging sensors and the two corresponding infrared imaging light outlets are different, and the difference between the two distances is the TIE propagation distance.
[0022] The spatial light modulator is reflective or transmissive, and can dynamically load the coded hologram generated by the computing control module to modulate the illumination light and reconstruct a holographic three-dimensional image of the object. In the holographic acquisition stage, the hologram is loaded to modulate the propagation distance, and in the holographic display stage, the display image hologram is loaded to modulate the holographic display. The spatial light modulator is located at the light outlet of the spatial light modulator of the light guide device or the light inlet of the spatial light modulator;
[0023] Preferably, the computing control module at least includes:
[0024] A light source control unit, used to control the on and off of the infrared light source and the visible light source according to the timing requirements of holographic acquisition and display;
[0025] An imaging control unit, used to control the capture timing of the imaging device to ensure that the reflected light is captured at a specific propagation distance to generate an intensity map;
[0026] The data processing unit is used to receive the intensity map generated by the imaging device, solve the complex amplitude of the object based on the light intensity transmission equation, and perform hologram encoding to generate a coded hologram for holographic display.
[0027] Second, the present invention also proposes a method for real-time holographic acquisition and reproduction of real scenes using the system, which is characterized by comprising:
[0028] S1: The computing control module controls the infrared light source module to turn on and the visible light source module to turn off, and starts the holographic acquisition process;
[0029] S2: The infrared light emitted by the infrared light source module illuminates the objects in the real scene through the light guide device, and the reflected light carrying the object information and having different propagation distances is captured by the infrared imaging device in succession, and a corresponding intensity map is generated; wherein the absolute value of the difference between the second propagation distance and the first propagation distance is the effective propagation distance of the light intensity transmission equation, and the capture of different propagation distances can be achieved by a set of infrared imaging devices in cooperation with a spatial light modulator or two sets of infrared imaging devices;
[0030] S3: The calculation control module receives at least two intensity images at different propagation distances, solves the complex amplitude of the object based on the light intensity transmission equation, and performs hologram encoding to generate a coded hologram;
[0031] S4: The computing control module controls the red infrared light source module to turn off and the visible light source module to turn on, and the holographic display process begins;
[0032] S5: The visible light emitted by the visible light source module is irradiated onto the spatial light modulator loaded with the coded hologram through the light guide device to reconstruct a holographic three-dimensional image of the object;
[0033] S6: Repeat S1 to S5 to achieve real-time alternation of holographic acquisition and display.
[0034] Further, the step S3 specifically includes:
[0035] S3.1 Substitute the two intensity images generated in step S2 into the light intensity transmission equation to obtain the phase information Φ0 at the first imaging sensor. Specifically:
[0036] S3.1.1 normalizes the two intensity maps into matrices, processes the intensity matrices into the same size, and S3.1.2 aligns the two intensity maps;
[0037] S3.1.3 Substitute the two intensity images into the light intensity transfer equation solver and solve for the phase;
[0038] S3.1.4 performs phase wrapping on the solved phase and outputs the phase.
[0039] S3.2 converts the intensity information captured in step S2 into amplitude information and combines it with the obtained phase information to obtain complex amplitude wavefront information Z1 of the light wavefront on the imaging plane;
[0040] S3.3 Substitute the complex amplitude wavefront information Z1 of the imaging plane into the spatial light propagation equation ASM to obtain the complex amplitude wavefront information Z0 at the real scene;
[0041] S3.4 Encode the complex amplitude wavefront information Z0 at the real scene to obtain a hologram H for holographic display that carries complete real scene optical information b .
[0042] The holographic acquisition process S1-S4 and the holographic display process S5-S6 meet certain timing control requirements, including:
[0043] The duration of the holographic display process S5-S6 is longer than the duration of the holographic acquisition process S1-S4. Preferably, the duration of the holographic display process S5-S6 accounts for more than 80% of the holographic reproduction cycle time S1-S6.
[0044] The switching frame rate between the holographic acquisition process S1-S4 and the holographic display process S5-S6 is greater than 24 Hz.
[0045] Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant advantages:
[0046] 1. The holographic acquisition method of the present invention directly processes two intensity images taken at close range through the light intensity transmission equation to obtain a coded hologram that can be used for holographic display. This process combines the advantages of optical holography without the need for three-dimensional modeling and fast encoding in computational holography. It avoids multiple Fourier transform calculations and the use of complex optical hardware, realizes holographic acquisition that retains the depth information of the real scene, and achieves the effect of three-dimensional display.
[0047] 2. The device of the present invention integrates the components commonly used in the holographic acquisition and holographic display processes, such as the spatial light modulator and the beam splitter, to obtain an integrated device that takes into account the functions of holographic acquisition and holographic display. By accurately controlling the working timing of the infrared light source module and the visible light source module, the alternation of acquisition and display is realized, avoiding light source interference, and optimizing the processing flow of the calculation control module, significantly improving the real-time performance of holographic acquisition and reproduction. By generating a coded hologram by calculation and quickly loading it using a spatial light modulator, real-time three-dimensional reconstruction is achieved.
[0048] 3. The device of the present invention uses an infrared light source for holographic acquisition and a visible light source for holographic display, thus avoiding the influence of visible stray light generated by holographic acquisition during the holographic display process. By calculating the timing control of the visible light source and the infrared light source by the control module, and considering the visual persistence effect of the human eye, a display effect with a refresh rate of 24Hz is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Attached Figure 1 This is a schematic diagram of the structure and optical path of Embodiment 1 of the real-time holographic acquisition and display device of the present invention;
[0050] Attached Figure 2 This is a schematic diagram of the structure and optical path of Embodiment 2 of the real-time holographic acquisition and display device of the present invention;
[0051] Attached Figure 3 A schematic flow chart of a real-time reconstruction method of a real-time holographic acquisition and display device provided in Embodiment 1 and Embodiment 2 of the present invention;
[0052] Attached Figure 4 This is a timing diagram of the real-time holographic acquisition and display method according to Embodiment 1 of the present invention;
[0053] Attached Figure 5 This is a timing diagram of the real-time holographic acquisition and display method of Example 2 of the present invention.
[0054] In the figure: 100 is an infrared light source module, 101 is an infrared light source, 102 is a first collimating lens; 110 is a first light guiding device, 111 is a first beam splitter, 112 is a second beam splitter, 113 is a third beam splitter, 114 is a fourth beam splitter; 120 is a single-camera infrared imaging device, 121 is a first imaging sensor, 122 is a first imaging objective lens, 123 is a second imaging sensor, 124 is a second imaging objective lens; 130 is a computing control module; 140 is a spatial light modulator.
[0055] 200 is a visible light source module, 201 is a visible light source, 202 is a second collimating lens; 210 is a second light guide device; 220 is a dual-camera imaging device.
[0056] It should be understood that the above drawings are only schematic and are not drawn to scale. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be described in detail below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0058] The above scheme is further described below in conjunction with specific implementation examples. The preferred embodiments of the present invention are described in detail as follows:
[0059] Example 1
[0060] See also Figure 1 , Figure 1 This is a schematic diagram of the structure and optical path of the embodiment 1 of the real-time holographic acquisition and display device of the present invention. Figure 1 The dashed arrow in the middle represents the holographic collection optical path of Example 1; Figure 1 The solid arrow in the middle represents the holographic display light path of Example 1.
[0061] like Figure 1As shown, the real-time holographic acquisition and display device of Example 1 includes an infrared light source module 100 , a first light guide device 110 , a single-camera infrared imaging device 120 , a calculation control module 130 , a spatial light modulator 140 , and a visible light source module 200 .
[0062] The infrared light source module 100 is located at the starting position in the holographic collection optical path. The infrared light source module 100 is used to provide collimated infrared light during the holographic collection process. The infrared light source module 100 is composed of an infrared light source 101 and a first collimating lens 102, and the first collimating lens 102 is located on the light outlet side of the infrared light source 101. The infrared light source 101 can be a laser infrared light source, a thermal radiation infrared light source, or a gas discharge infrared light source, etc. The infrared light source 101 provides a monochromatic light wave of any wavelength in the infrared band (greater than 780nm). The infrared light source 101 is connected to the computing control module 130. During the holographic imaging process, the computing control module 130 will control the light emission timing of the infrared light source 101. The first collimating lens 102 is used to collimate the light beam emitted by the infrared light source 101.
[0063] The visible light source module 200 is located at the starting position in the holographic display optical path. The visible light source module 200 is used to provide a visible light source for reconstructing the holographic image of the object during the holographic display process. The visible light source module 200 is composed of a visible light source 201 and a second collimating lens 202, and the second collimating lens 202 is located on the side of the light outlet of the visible light source 201. The visible light source 201 can be a laser, or an LED lamp, etc., wherein the laser provides a monochromatic light wave of any wavelength in the visible light band (380nm to 780nm). The second collimating lens 202 is used to collimate the light beam emitted by the visible light source 201.
[0064] The visible light source 201 is connected to the computing control module 130. During the holographic imaging process, the computing control module 130 controls the light emission timing of the visible light source 201. In order to ensure that there is no crosstalk between the holographic acquisition and holographic display processes during the holographic acquisition and display cycle, the infrared light source 101 and the visible light source 201 emit light alternately and do not overlap. Specifically, during the holographic acquisition process, the visible light source 201 is in the off state and the infrared light source 101 is in the on state, and during the holographic display process, the visible light source 201 is in the on state and the infrared light source 101 is in the off state.
[0065] The first light guide device 110 is composed of a first beam splitter 111, a second beam splitter 112, and a third beam splitter 113, wherein the coating surface of the first beam splitter 111 is placed parallel to the coating surface of the second beam splitter 112, and the coating surface of the second beam splitter 112 is placed symmetrically with the coating surface of the third beam splitter 113 along the symmetry plane of the two beam splitters. The first light guide device 110 has three light inlets and three light outlets in the holographic collection optical path, and has two light inlets and two light outlets in the holographic display optical path. In the holographic collection process, the first light guide device 110 is located between the infrared light source module 100 and the real scene that needs to be holographically collected, and is also located between the real scene that needs to be holographically collected and the single-camera infrared imaging device 120; in the holographic display process, the first light guide device 110 is located between the visible light source module 200 and the reconstructed image of the real scene displayed holographically. The first light guide device 110 can deflect the light beam, split the light beam and transmit it to a suitable position. In the holographic acquisition stage, the illumination light emitted by the infrared light source 101 can be guided to illuminate the real scene, and the reflected light of the illuminated object can be guided to enter the single-camera infrared imaging device 120. In the holographic display process, the illumination light emitted by the visible light source 201 can be guided to illuminate the spatial light modulator 140, so that the real object generates a holographic image.
[0066] The first light guiding device 110 has the following effects on the holographic acquisition process and the holographic display process: during the holographic acquisition process, the collimated infrared light emitted by the infrared light source module 100 first enters the first beam splitter 111, and a light beam reflected by the first beam splitter 111 irradiates an object in the real scene and is reflected, and the reflected light from the object enters the first beam splitter 111 again, and a light beam transmitted by the first beam splitter 111 enters the second beam splitter 112, and a light beam transmitted by the second beam splitter 112 irradiates the spatial light modulator 140 for modulation and reflection, and the reflected light enters the second beam splitter 112 again, and a light beam reflected by the second beam splitter 112 enters the third beam splitter 113, and a light beam transmitted by the third beam splitter 113 is captured and imaged by the single-camera infrared imaging device 120. During the holographic display process, the collimated visible light emitted by the visible light source module 200 first enters the third beam splitter 113, and a light beam reflected by the third beam splitter 113 enters the second beam splitter 112. A light beam reflected by the second beam splitter 112 irradiates the spatial light modulator 140 for modulation and reflection, and the reflected light enters the second beam splitter 112 again. A light beam transmitted by the second beam splitter 112 enters the first beam splitter 111, and a light beam transmitted by the first beam splitter 111 is projected into space to generate a holographic image of a real object.
[0067] The first beam splitter 111, the second beam splitter 112, and the third beam splitter 113 may be a combination of a polygonal lens and a coating for refracting or reflecting light, and are required to have the function of splitting a light beam into two or more different light beams at two angles.
[0068] The single-camera infrared imaging device 120 is located at the end of the optical path during the holographic acquisition process and does not participate in the optical path during the holographic display process. The single-camera infrared imaging device 120 is composed of a first imaging sensor 121 and a first imaging lens 122. The single-camera infrared imaging device 120 is used to quickly image the light waves that are reflected by the real scene and pass through the first light guide device 110 at two different propagation distances. After the light waves reflected by the real scene are guided by the light guide device 110 and the spatial light modulator 140 modulates the object light waves twice, the propagation distances traveled by the object light waves when they reach the single-camera infrared imaging device 120 are two different propagation distances.
[0069] At the same time, the single-camera infrared imaging device 120 is connected to the calculation control module 130, and the calculation control module 130 controls the capture time of the intensity map of the first imaging sensor 121 according to the timing of the hologram transformation of the spatial light modulator 140. Specifically, when the spatial light modulator 140 loads the first propagation distance hologram, the propagation distance of the object light wave when it reaches the single-camera infrared imaging device 120 is the first propagation distance, and the single-camera infrared imaging device 120 captures the first intensity map; when the spatial light modulator 140 loads the second propagation distance hologram, the propagation distance of the object light wave when it reaches the single-camera infrared imaging device 120 is the second propagation distance, and the single-camera infrared imaging device 120 captures the second intensity map. The first propagation distance is determined by the structure of the device and the relative spatial position with the real scene. It is a relative quantity and does not need to be determined during the holographic acquisition and display process. The absolute value of the difference between the second propagation distance and the first propagation distance is the effective propagation distance of the light intensity transmission equation, which is usually in the micrometer order.
[0070] The first imaging sensor 121 is located behind the first imaging lens 122. The first imaging sensor 121 is used to successively capture two intensity images with different propagation distances and convert infrared light intensity signals into electrical signals. The first imaging sensor 121 can be an infrared camera, a combination of multiple single-point infrared detectors, or any sensor device that can record infrared light wave intensity information. The first imaging sensor 121 is connected to the computing control module 130.
[0071] The first imaging lens 122 is used to focus the object light wave. The first imaging lens 122 makes the focal plane of the first imaging sensor 121 coincide with the equivalent plane position of the spatial light modulator 140 in the holographic collection optical path.
[0072] The calculation control module 130 is connected to the infrared light source 101 , the visible light source 201 , the first imaging sensor 121 , and the spatial light modulator 140 .
[0073] The calculation control module 130 is used to calculate the driver for the infrared light source 101, the visible light source 201, the first imaging sensor 121, and the spatial light modulator 140. When calculating the driver to load the hardware parameters, it is necessary to calculate the specific hardware characteristics of the infrared light source 101, the visible light source 201, the first imaging sensor 121, and the spatial light modulator 140, such as the minimum exposure time and sensitivity of the first imaging sensor 121, the modulation frame rate of the spatial light modulator 140, and the switching time characteristics of the infrared light source 101 and the visible light source 201. The driver controls the timing of the infrared light source 101, the visible light source 201, the first imaging sensor 121, and the spatial light modulator 140. The timing sequence must satisfy the following order: the infrared light source 101 emits light, the spatial light modulator 140 loads the first propagation distance hologram, the first imaging sensor 121 captures the first intensity map, the spatial light modulator 140 loads the second propagation distance hologram, the first imaging sensor 121 captures the second intensity map, the infrared light source 101 turns off the light, the visible light source 201 emits light, the spatial light modulator 140 loads the encoded hologram, and the visible light source 201 turns off the light. The above timing sequence repeats itself. The light emission time of the visible light source 201 must be longer than the light emission time of the infrared light source 101, that is, the duration of the holographic display process must be longer than the duration of the holographic acquisition process. Preferably, the duration of the holographic display process accounts for more than 80% of the cycle time of a holographic reproduction. The switching frame rate between the two light sources must be greater than 24Hz.
[0074] The calculation control module 130 is used to calculate the first propagation distance hologram and the second propagation distance hologram loaded by the spatial light modulator 140. The function of the first propagation distance hologram and the second propagation distance hologram is that, when they are loaded into the spatial light modulator 140, they can modulate the object light wave so that when the two object light waves successively reach the first imaging sensor 121, the object light wave actually propagates two different propagation distances, namely the first propagation distance and the second propagation distance. The specific calculation steps include: firstly, obtaining the first propagation distance hologram, namely the uniform phase hologram; according to the phase modulation effect formula of the lens, converting an ideal lens with a focal length of the propagation distance into an equivalent phase hologram or amplitude hologram, the second propagation distance hologram is the equivalent phase hologram or amplitude hologram of this lens.
[0075] The calculation control module 130 is used to calculate the coded hologram for three-dimensional display loaded by the spatial light modulator 140. According to the two intensity images captured by the first imaging sensor 121 and the light intensity transmission equation, the complete optical information including amplitude and phase at the real scene is calculated, and holographic encoding is performed to encode it into a coded hologram for computational holographic display. The specific calculation steps include: substituting the two intensity images into the light intensity transmission equation to obtain the phase information at the first imaging sensor 121; combining the captured intensity information with the obtained phase information to obtain the complex amplitude information of the light wavefront of the imaging plane; substituting the complex amplitude wavefront information of the imaging plane and the distance between the first imaging sensor 121 and the real scene into the spatial light propagation equation to obtain the complex amplitude wavefront information at the real scene; encoding the light wavefront information at the complex amplitude, using a diffusion algorithm or a dual-phase encoding method, to obtain a coded hologram for holographic display that carries complete real scene optical information.
[0076] The spatial light modulator 140 is located on the side of the first light guide device 110, and the second beam splitter 112 is located in front and behind in the optical path. In the holographic acquisition process, the spatial light modulator 140 is used to modulate the object light wave, so that the propagation distance of the light wave changes, and cooperates with the single-camera infrared imaging device 120 to capture the light waves reflected by the real object at two different propagation distances. In the holographic display stage, the spatial light modulator 140 loads the encoded hologram generated by the holographic acquisition process for three-dimensional display, modulates the light wavefront, and finally displays the holographic image. The spatial light modulator 140 can be an amplitude-type spatial light modulator or a phase-type spatial light modulator. In the present embodiment, the spatial light modulator 140 is a reflective spatial light modulator. When the present device is modified based on non-creative structure, the spatial light modulator 140 can also be a transmissive spatial light modulator.
[0077] The spatial light modulator 140 is connected to the calculation control module 130, and the calculation control module 130 is used to control the hologram loaded on the spatial light modulator 140. In the holographic acquisition process, the spatial light modulator 140 first loads the first propagation distance hologram, at which time the object light wave reaches the single-camera infrared imaging device 120 and is captured as the first intensity map. After the first intensity map is recorded, the spatial light modulator 140 loads the second propagation distance hologram, at which time the object light wave reaches the single-camera infrared imaging device 120 and is captured as the second intensity map. The first propagation distance hologram is the hologram loaded by the spatial light modulator 140 corresponding to the object light wave at the first propagation distance, and the second propagation distance hologram is the hologram loaded by the spatial light modulator 140 corresponding to the object light wave at the second propagation distance. The first propagation distance hologram and the second propagation distance hologram are calculated by the calculation control module 130 according to the phase modulation effect of the lens and the diffraction law of spatial light. During the holographic display stage, the spatial light modulator 140 is controlled by the computing control module 130 to load the encoded hologram generated through the holographic acquisition process for three-dimensional display.
[0078] The loading sequence of the hologram of the spatial light modulator 140 is as follows: during the holographic acquisition process, after the infrared light source 101 is turned on, the first propagation distance hologram is first loaded, and after the single-camera infrared imaging device 120 captures the first intensity image, the second propagation distance hologram is loaded; during the holographic display process, after the visible light source is turned on, the coded hologram used for three-dimensional display is loaded. The above process is continuously repeated.
[0079] Example 2
[0080] See also Figure 2 , Figure 2 Schematic diagram of the structure and optical path of embodiment 2 of the real-time holographic acquisition and display device of the present invention. Figure 2 The dashed arrow in the middle represents the holographic collection optical path of Example 2; Figure 2 The solid arrow in the middle represents the holographic display light path of Example 2.
[0081] like Figure 2 As shown, the real-time holographic acquisition and display device of Example 2 includes an infrared light source module 100, a second light guide device 210, a dual-camera infrared imaging device 220, a calculation control module 130, a spatial light modulator 140, and a visible light source module 200.
[0082] The second light guide device 210 is composed of a first beam splitter 111, a second beam splitter 112, a third beam splitter 113, and a fourth beam splitter 114. The second light guide device 210 has two light inlets and three light outlets in the holographic collection optical path, and has two light inlets and two light outlets in the holographic display optical path. In the holographic collection process, the second light guide device 210 is located between the infrared light source module 100 and the real scene that needs to be holographically collected, and is also located between the real scene that needs to be holographically collected and the dual-camera infrared imaging device 220; in the holographic display process, the second light guide device 210 is located between the visible light source module 200 and the reconstructed image of the real scene displayed holographically.
[0083] The second light guide device 210 can deflect the light beam, split the light beam and transmit it to a suitable position. In the holographic acquisition stage, the illumination light emitted by the infrared light source 101 can be guided to illuminate the real scene, and the reflected light of the illuminated object can be guided to enter the dual-camera infrared imaging device 220. In the holographic display process, the illumination light emitted by the visible light source 201 can be guided to illuminate the spatial light modulator 140, so that the real object generates a holographic image.
[0084] The effect of the second light guiding device 210 on the holographic acquisition process and the holographic display process is as follows: in the holographic acquisition process, the collimated infrared light emitted by the infrared light source module 100 first enters the third beam splitter 113, and a light beam transmitted through the third beam splitter 113 enters the second beam splitter 112. A light beam reflected through the second beam splitter 112 irradiates an object in the real scene and reflects. The reflected light from the object enters the second beam splitter 112 again, and a light beam transmitted through the second beam splitter 112 enters the fourth beam splitter 114. The light beam transmitted through the fourth beam splitter 114 is captured by the dual-camera infrared imaging device 220, and the light beam reflected through the fourth beam splitter 114 is captured by the dual-camera infrared imaging device 220. During the holographic display process, the collimated visible light emitted by the visible light source module 200 first enters the third beam splitter 113, and a light beam reflected by the third beam splitter 113 enters the second beam splitter 112. A light beam transmitted by the second beam splitter 112 enters the first beam splitter 111. A light beam reflected by the first beam splitter 111 irradiates the spatial light modulator 140 for modulation and reflection, and the reflected light enters the first beam splitter 111 again. The light beam transmitted by the first beam splitter 111 is projected into space to generate a holographic image of a real object.
[0085] The fourth beam splitter 114 may be a combination of a polygonal lens and a coating that refracts or reflects light, and is required to have the function of splitting the light beam into two or more different light beams at two angles.
[0086] The dual-camera infrared imaging device 220 is located at the end of the optical path during the holographic acquisition process and does not participate in the optical path during the holographic display process. The dual-camera infrared imaging device 220 is used to quickly image the light waves that pass through the second light guide device 210 after being reflected by the real scene at two different propagation distances. The dual-camera infrared imaging device 220 is composed of a first imaging sensor 121 and a first imaging lens 122 and a second imaging sensor 123 and a second imaging lens 124.
[0087] The distance between the first imaging sensor 121 and the fourth beam splitter 114 is different from the distance between the second imaging sensor 123 and the fourth beam splitter 114. The light wave reflected by the real scene is divided into two light beams after passing through the light guide device 110, and enters the first imaging sensor 121 and the second imaging sensor 123 respectively. The propagation distances traveled by the object light wave when reaching the first imaging sensor 121 and the second imaging sensor 123 of the dual-camera infrared imaging device 220 are two different propagation distances.
[0088] At the same time, the dual-camera infrared imaging device 220 is connected to the computing control module 130 , and the computing control module 130 controls the intensity map capture time of the first imaging sensor 121 and the second imaging sensor 123 according to the timing of the visible light source 201 and the infrared light source 101 .
[0089] Specifically, when the infrared light source 101 is turned on, when the object light wave reaches the first imaging sensor 121 in the dual-camera infrared imaging device 220, the propagation distance it passes is the first propagation distance, and at this time, the first imaging sensor 121 captures the first intensity map. At the same time, when the object light wave reaches the second imaging sensor 123 in the dual-camera infrared imaging device 220, the propagation distance it passes is the second propagation distance, and at this time, the second imaging sensor 123 captures the second intensity map. The first propagation distance is determined by the distance between the first imaging sensor 121 and the real scene, and the second propagation distance is determined by the distance between the second imaging sensor 123 and the real scene. The second propagation distance and the first propagation distance do not need to be determined during the holographic acquisition and display process. The absolute value of the difference between the second propagation distance and the first propagation distance is the effective propagation distance of the light intensity transmission equation, which is usually in the micrometer order. It is only necessary to determine the difference between the second propagation distance and the first propagation distance.
[0090] The second imaging sensor 123 is located behind the second imaging lens 124. The second imaging sensor 123 is used to successively capture two intensity images with different propagation distances and convert infrared light intensity signals into electrical signals. The second imaging sensor 123 can be an infrared camera, a combination of multiple single-point infrared detectors, or any sensor device capable of recording infrared light wave intensity information. The second imaging sensor 123 is connected to the computing control module 130.
[0091] The second imaging lens 124 is used to focus the object light wave. The focal length of the second imaging lens 124 is the same as that of the first imaging lens 122.
[0092] The calculation control module 130 is connected to the infrared light source 101 , the visible light source 201 , the first imaging sensor 121 , the second imaging sensor 123 , and the spatial light modulator 140 .
[0093] The calculation control module 130 is used to calculate the driver programs for the infrared light source 101, the visible light source 201, the first imaging sensor 121, the second imaging sensor 123, and the spatial light modulator 140. When calculating the driver program, it is necessary to consider the specific hardware characteristics of the infrared light source 101, the visible light source 201, the first imaging sensor 121, the second imaging sensor 123, and the spatial light modulator 140, such as the minimum exposure time and sensitivity of the first imaging sensor 121, the modulation frame rate of the spatial light modulator 140, and the switching time characteristics of the infrared light source 101 and the visible light source 201. The driver program controls the timing of the infrared light source 101, the visible light source 201, the first imaging sensor 121, the second imaging sensor 123, and the spatial light modulator 140. The timing sequence must meet the following order: the infrared light source 101 emits light, the first imaging sensor 121 captures the first intensity image and the second imaging sensor 123 captures the second intensity image, the infrared light source 101 turns off the light, the visible light source 201 emits light, the spatial light modulator 140 loads the encoded hologram, and the visible light source 201 turns off the light. The above timing sequence is repeated.
[0094] The calculation control module 130 is used to calculate the coded hologram for three-dimensional display loaded by the spatial light modulator 140. According to the two intensity images captured by the first imaging sensor 121 and the second imaging sensor 123 and the light intensity transmission equation, the complete optical information including amplitude and phase at the real scene is calculated, and holographic encoding is performed to encode it into a coded hologram for computational holographic display.
[0095] The spatial light modulator 140 is located on the side of the second light guide device 210, and the first beam splitter 111 is located in front and behind in the optical path. In the holographic display stage, the spatial light modulator 140 loads the coded hologram for three-dimensional display generated through the holographic acquisition process, modulates the light wavefront, and finally displays the holographic image. The spatial light modulator 140 is connected to the computing control module 130, and the computing control module 130 is used to control the hologram loaded into the spatial light modulator 140. In the holographic display stage, the spatial light modulator 140 is controlled by the computing control module 130 to load the coded hologram for three-dimensional display generated through the holographic acquisition process. The loading sequence of the hologram of the spatial light modulator 140 is: in the holographic acquisition process, after the infrared light source 101 is turned on, the uniform phase hologram is loaded first; in the holographic display process, after the visible light source is turned on, the coded hologram for three-dimensional display is loaded. The above process is continuously cycled.
[0096] The real-time holographic acquisition and display method based on the light intensity transmission equation provided in embodiments 1 and 2 of the present invention is as follows Figure 3 As shown, the method includes:
[0097] Step 1: The calculation control module controls the infrared light source to emit light and the visible light source to shut off light;
[0098] Step 2: The calculation control module calculates the first hologram H1 and the second hologram H2. The first hologram is calculated as a fixed uniform phase hologram. According to the phase modulation effect of the lens and the diffraction propagation law, the second hologram H2 is calculated as an equivalent phase hologram or amplitude hologram of an ideal lens with a focal length equal to the propagation distance;
[0099] Step 3: The computing control module controls the spatial light modulator to load the first hologram H1 and the second hologram H2 respectively, and controls the imaging device to correspondingly capture the first intensity map A1 and the second intensity map A2 respectively;
[0100] Step 4: The calculation control module calculates the coded hologram H for holographic display b . The two intensity images are regularized and aligned, and then substituted into the light intensity transmission equation solver to solve the phase, and the solved phase is phase wrapped and output as phase Φ0. The captured intensity information is converted into amplitude information and combined with the obtained phase Φ0 to obtain the complex amplitude wavefront information Z1 of the light wavefront on the imaging plane. The complex amplitude wavefront information of the imaging plane is substituted into the spatial light propagation equation to obtain the complex amplitude wavefront information Z0 at the real scene. The light wavefront information at the complex amplitude is encoded to obtain a hologram H for holographic display that carries complete real scene optical information. b ;
[0101] Step 5: The calculation control module controls the infrared light source to close the light and the visible light source to emit light;
[0102] Step 6: The illumination light from the visible light source is irradiated onto the spatial light modulator, and the encoded hologram H for holographic display loaded by the spatial light modulator b After modulation, it is projected to a specific location in space to form a holographic image of the real scene Z0 ’ ;
[0103] Step 7: Repeat steps 1 to 6 to realize real-time holographic acquisition and display.
[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present invention as described above. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A real-time holographic acquisition and reproduction system for real scenes, characterized in that: include: an infrared light source module configured to provide collimated infrared light during holographic acquisition; A visible light source module configured to provide collimated visible light during holographic display; A light guiding device, configured as at least three beam splitters, for deflecting, splitting and guiding the light beam; Specifically, the method comprises transmitting the infrared light emitted by the infrared light source to an object in a real scene during holographic acquisition, guiding the reflected light of the object to be transmitted to an imaging device via a spatial light modulator, and transmitting the visible light emitted by the visible light source module to a spatial light modulator during holographic display to generate a holographic image of the object; Infrared imaging device, used to successively capture reflected light with different propagation distances carrying object information during holographic acquisition, and generate corresponding intensity maps; The computing control module is respectively connected to the infrared light source module, the visible light source module, the light guide device, and the infrared imaging device and controls their working timings. Specifically, the computing control module controls the light emission timings of the infrared light source module and the visible light source module to ensure that the two emit light alternately without overlapping. During the holographic acquisition process, that is, the infrared light source module is turned on and the visible light source module is turned off, the infrared imaging device is controlled to capture reflected light at at least two different propagation distances to generate at least two corresponding intensity maps, wherein the absolute value of the difference between the second propagation distance and the first propagation distance is the effective propagation distance of the light intensity transmission equation, and the two intensity maps are processed and calculated to obtain a coded hologram carrying object information; during the holographic display process, that is, the infrared light source module is turned off and the visible light source module is turned on, the spatial light modulator is controlled to load the coded hologram to generate a holographically reconstructed three-dimensional image of the object.
2. The real-time holographic acquisition and reproduction system of the real scene according to claim 1, characterized in that: The effective propagation distance of the light intensity transmission equation is 1 to 10 microns.
3. The real-time holographic acquisition and reproduction system of a real scene according to claim 1 or 2, characterized in that: The infrared light source module includes an infrared light source and an infrared collimating lens, the visible light source module includes a visible coherent light source and a visible collimating lens, and the infrared imaging device includes an infrared imaging sensor and an imaging lens; the spatial light modulator is connected to the computing control module and is used to modulate the object light wave during the holographic acquisition process to change the propagation distance of the light wave; And loading the encoded hologram during the holographic display process to generate a holographic image of the object; wherein, during the holographic acquisition process, the infrared imaging device, with the assistance of a spatial light modulator, successively captures reflected light with different propagation distances carrying object information and generates a corresponding intensity map.
4. The real-time holographic acquisition and reproduction system of a real scene according to claim 1 or 2, characterized in that: The infrared light source module includes an infrared light source and an infrared collimating lens, the visible light source module includes a visible coherent light source and a visible collimating lens, and the infrared imaging device includes at least two groups of infrared imaging sensors and imaging lenses, ensuring that reflected light with different propagation distances carrying object information is captured during the holographic acquisition process and a corresponding intensity map is generated.
5. The real-time holographic acquisition and reproduction system of a real scene according to any one of claims 1 to 4, characterized in that: The infrared imaging device is used to capture infrared light and generate corresponding electrical signals.
6. The real-time holographic acquisition and reproduction system of a real scene according to any one of claims 1 to 4, characterized in that: The spatial light modulator is reflective or transmissive and can dynamically load the coded hologram generated by the computing control module to modulate the illumination light and reconstruct a holographic three-dimensional image of the object.
7. The real-time holographic acquisition and reproduction system of a real scene according to any one of claims 1 to 4, characterized in that: The computing control module at least includes: A light source control unit, used to control the on and off of the infrared light source and the visible light source according to the timing requirements of holographic acquisition and display; An imaging control unit, used to control the capture timing of the imaging device to ensure that the reflected light is captured at a specific propagation distance to generate an intensity map; The data processing unit is used to receive the intensity map generated by the imaging device, solve the complex amplitude of the object based on the light intensity transmission equation, and perform hologram encoding to generate a coded hologram for holographic display.
8. A method for real-time holographic acquisition and reproduction of real scenes using the system described in any one of claims 1 to 7, characterized in that: include: S1: The computing control module controls the infrared light source module to turn on and the visible light source module to turn off, and starts the holographic acquisition process; S2: The infrared light emitted by the infrared light source module illuminates the objects in the real scene through the light guide device, and the reflected light carrying the object information and having different propagation distances is captured by the infrared imaging device in succession, and a corresponding intensity map is generated; wherein the absolute value of the difference between the second propagation distance and the first propagation distance is the effective propagation distance of the light intensity transmission equation, and the capture of different propagation distances can be achieved by a set of infrared imaging devices in cooperation with a spatial light modulator or two sets of infrared imaging devices; S3: The calculation control module receives at least two intensity images at different propagation distances, solves the complex amplitude of the object based on the light intensity transmission equation, and performs hologram encoding to generate a coded hologram; S4: The computing control module controls the red infrared light source module to turn off and the visible light source module to turn on, and the holographic display process begins; S5: The visible light emitted by the visible light source module is irradiated onto the spatial light modulator loaded with the coded hologram through the light guide device to reconstruct a holographic three-dimensional image of the object; S6: Repeat S1 to S5 to achieve real-time alternation of holographic acquisition and display.
9. The real-time holographic acquisition and reproduction method of a real scene according to claim 8, characterized in that: The step S3 specifically includes: S3.1 Substitute the two intensity images generated in step S2 into the light intensity transmission equation to obtain the phase information Φ0 at the first imaging sensor; S3.2 converts the intensity information captured in step S2 into amplitude information and combines it with the obtained phase information to obtain complex amplitude wavefront information Z1 of the light wavefront on the imaging plane; S3.3 Substitute the complex amplitude wavefront information Z1 of the imaging plane into the spatial light propagation equation ASM to obtain the complex amplitude wavefront information Z0 at the real scene; S3.4 Encode the complex amplitude wavefront information Z0 at the real scene to obtain a hologram H for holographic display that carries complete real scene optical information b .
10. The real-time holographic acquisition and reproduction method of a real scene according to claim 8, characterized in that: S3.1 substitutes the two intensity images into the light intensity transmission equation to obtain the phase information Φ0 at the first imaging sensor, which specifically includes the following steps: S3.1.1 Regularize the two intensity maps into matrices and process the intensity matrices into the same size. S3.1.2 align the two intensity images; S3.1.3 Substitute the two intensity images into the light intensity transfer equation solver and solve for the phase; S3.1.4 performs phase wrapping on the solved phase and outputs the phase.
Citation Information
Patent Citations
Holographic three-D image recording and display method and its device and application
CN100429587C
Device and method for acquiring computed hologram of three-dimensional motion object
CN102024272A
Reflective digital holographic fingerprint imaging device
CN102436577A
Holographic fluctuation microscopy apparatus and method for determining mobility of particle and / or cell dispersions
CN103025886A
Digital holograph collecting and processing method based on phase shift interference
CN103955126A
