A multi-depth 3D computer hologram synthesis method based on interlayer edge processing
By performing layered processing on RGB-D images, supplementing interlayer edge information and smoothing phase, the problem of unclear occlusion relationships between holographic layers is solved, generating high-quality multi-depth 3D holograms, reducing computational costs and improving reconstruction results.
Patent Information
- Application Number
- CN202411483339.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-23
- Publication Date
- 2025-11-28
- Estimated Expiration
- 2044-10-23
AI Technical Summary
Existing technologies lack proper handling of occlusion relationships between layers when generating holograms, resulting in artifacts at the edges of the holograms, especially black stripes at the edges between layers, making it impossible to correctly reconstruct complex 3D scenes.
By using a layer-by-layer edge processing method, the RGB-D image is first layered to supplement the edge information of the deepest slice. The complex amplitude field is then used for propagation, and optical diffraction calculations are performed using the angular spectrum method. This process is repeated layer by layer until the final complex amplitude image is generated, supplementing the interlayer edge information and smoothing the phase information.
The generated holograms have correct occlusion relationships, significantly reduce interlayer edge artifacts, improve the clarity of the reconstructed 3D scene, closely resemble the optical diffraction phenomenon of real scenes, and have low computational cost.
Smart Images
Figure CN119559363B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of computer-generated holography (CGH), and particularly relates to a multi-depth 3D computer-generated hologram synthesis method based on inter-layer edge processing. BACKGROUND
[0002] Holographic technology can encode the spatial information of a three-dimensional scene into a hologram, and through diffraction, it can reconstruct the original scene, providing all the natural visual depth cues required by the human visual system, including occlusion, eye accommodation and stereopsis. However, due to the limitations of experimental device complexity and performance flexibility, it has always been a challenge to reconstruct large and complex three-dimensional scenes, especially virtual three-dimensional scenes, using optical holography. Since the invention of holography by Dennis Gabor in the late 1940s, holographic technology has made great progress. With the advent of stable laser sources, computers and spatial light modulators (SLMs), it has gradually become possible to generate high-quality phase-only holograms (POHs) based on computers. Computer-generated holography essentially simulates optical diffraction propagation by using a computer to establish an accurate propagation model, then establishes a link between the phase hologram and the target real light field through the diffraction propagation model, and finally inversely solves the target phase pattern. Computer-generated holography can provide multiple layers of accurate depth information for the generated image. Since it is a simulation of optical diffraction propagation, it is less likely to cause visual fatigue than other display technologies, and the picture has a stronger sense of reality, so computer-generated holography has a wide range of applications in the display field.
[0003] The method of generating a computer-generated hologram based on depth layering slices uses a depth map to process RGB images in layers, then propagates the image layer farthest from the previous layer using a mask to achieve an occlusion effect, and finally obtains a 3D hologram. However, in real-world scenarios, light propagation is not point-to-point propagation, but point-to-surface diffraction propagation. The lack of optical diffraction of the occluded part of the edge of the previous layer results in a lack of information about the occluded object. The boundary between the slices is equivalent to the edge of the light field, which leads to coherent superposition at the edge of the current layer in the hologram obtained by the above method, resulting in black stripes at the edge of the diffraction field, and the occlusion relationship between the layers is not correctly handled. SUMMARY
[0004] In order to overcome the shortcomings of the prior art, the present application provides a multi-depth 3D computer-generated hologram synthesis method based on inter-layer edge processing, which can synthesize multi-depth 3D computer-generated holograms at low computational cost, has a clear three-dimensional scene with correct occlusion relationship, and greatly reduces the inter-layer edge artifacts of the hologram.
[0005] The technical scheme adopted by the present application to solve its technical problems is:
[0006] A multi-depth 3D computer hologram synthesis method based on interlayer edge processing, comprising the following steps:
[0007] Step one, first load the RGB-D image, generate five binary mask templates according to the depth range of the depth map and five binary inverse mask templates opposite to the binary values, and divide the image into five layers using the mask;
[0008] Step two, supplement the edge information of the slice corresponding to the deepest depth, and then combine the supplemented slice with the initial zero phase to form a complex amplitude field;
[0009] Step three, propagate the complex amplitude field to the next depth plane using the angular spectrum method, and obtain a complex amplitude field;
[0010] Step four, use the inverse mask to block the slice position of the complex amplitude field corresponding to the depth of the layer and replace it with the slice of the depth layer;
[0011] Step five, supplement the information of the amplitude blocked edge part of the complex amplitude obtained in step four, and smooth the phase blocked part, and then recombine the processed amplitude and phase into a complex amplitude;
[0012] Step six, repeat steps three, four and five until the final complex amplitude image is obtained in step five at the last depth plane to complete the synthesis of the hologram.
[0013] Further, in step one, the RGB-D image is two images: one is a normal RGB three-channel color image, and the other is a Depth image, which is similar to a grayscale image, and each pixel value is the relative distance of the sensor from the object.
[0014] Further, in step two, the complex amplitude field refers to an image composed of complex numbers, which can be split into an amplitude image and a phase image by transformation, and is usually used for optical diffraction calculation.
[0015] Further, in step one, the binary mask template is a binary four-dimensional tensor consistent with the tensor size of the input image generated by equally dividing the normalized depth map according to the depth value. The binary inverse mask template is opposite in value to the binary mask template.
[0016] In the fifth step, the information supplement of the amplitude blocked edge part is to use the information of the pixels around the edge to supplement the edge area of the blocked part, so that the supplemented image edge extends a part to compensate for the optical diffraction of the blocked part of the image edge, and the supplemented image edge extends as similar as possible to the existing edge information.
[0017] In the fifth step, the smooth supplement processing of the phase blocked part is to first use the information of the pixels around the edge to supplement the edge area of the blocked part, and then use the information of the supplemented part to fill the internal blocked phase, so that the obtained phase diagram not only ensures the same phase of the slice area of the layer, but also ensures the smooth contact of the slice phase edge, and eliminates the edge phase mutation of the phase diagram as much as possible.
[0018] The technical concept of the present application is that for the slice obtained based on depth layering, first, the information of the blocked edge part of the picture slice is supplemented, the smooth supplement processing of the phase blocked part is performed, then the processed amplitude and phase are combined into a complex amplitude field, propagation is performed by using the angular spectrum method, similar processing is performed when reaching the next layer, and the whole depth layer is processed to obtain a complex amplitude hologram. Compared with the original method, the multi-depth 3D hologram generated by supplementing the interlayer edge information and smoothing the phase information can reconstruct a clear three-dimensional scene with correct occlusion relationship, and the generated hologram has clear front and back relationship, and the edges of the focusing surface and the non-focusing surface are clearer and closer to the real scene. In addition, this method greatly reduces the interlayer edge artifacts of the hologram, and the generated hologram has natural optical diffraction phenomenon, which is expected to be applied in holographic display, VA / AR and other scenes.
[0019] The beneficial effects of the present application mainly include:
[0020] 1. Low computational cost multi-depth 3D computer hologram synthesis: The present application uses a multi-depth 3D computer hologram synthesis method based on interlayer edge processing, which can generate holograms with lower computational cost and obtain hologram quality comparable to other existing methods.
[0021] 2. Clear three-dimensional scene with correct occlusion relationship: The present application supplements the image edge blocked part by using edge pixel supplement to compensate for the optical diffraction of the image edge blocked part, correctly handles the occlusion effect in the discontinuous edge of the segmented depth map object, and generates a hologram with clear front and back relationship, and the edges of the focusing surface and the non-focusing surface are clearer and closer to the real scene.
[0022] 3, greatly reduce the interlayer edge artifacts of hologram: the present application by for the phase is blocked part of the smooth supplement processing, as far as possible to eliminate the edge phase of the phase diagram mutation. The complex amplitude hologram obtained by the phase replacement method greatly reduces the interlayer edge artifacts of hologram, has the effect of being closer to the real optical propagation, and the image obtained by propagation is more consistent with the real scene. BRIEF DESCRIPTION OF DRAWINGS
[0023] Figure 1 is a flowchart schematic diagram of the present application;
[0024] Figure 2 is a schematic diagram of generating binary mask and inverse mask and corresponding depth layer slice of the present application;
[0025] Figure 3 is the generation result of different depths of the present application and other methods, wherein (a) represents the generation result of traditional layer-based multi-depth 3D hologram, (b) (c) (d) (e) are the pictures obtained by focusing on the lower, left, right and upper four different block parts respectively by the results obtained by the amplitude phase processing in the generation process of the method of the present application;
[0026] Figure 4 is a detailed comparison of the real scene results generated by the present application and other methods, wherein (a) is the hologram obtained by the traditional method, and (b) is the result obtained by the method of the present application. DETAILED DESCRIPTION
[0027] The present application will be further described below with reference to the accompanying drawings.
[0028] Reference Figures 1-4 A multi-depth 3D computer hologram synthesis method based on interlayer edge processing, the flowchart of the present application is shown in Figure 1 , comprising the following steps:
[0029] A multi-depth 3D computer hologram synthesis method based on interlayer edge processing, comprising the following steps:
[0030] Step one, first load RGB-D image, generate five binary mask templates and five binary inverse mask templates opposite to the binary according to the depth range of the depth map, and divide the image into five layers by using the mask;
[0031] The RGB-D image is two images: one is a normal RGB three-channel color image, and the other is a depth image. The depth image is similar to a gray image, and each pixel value of the depth image is the relative distance of the sensor from the object.
[0032] The binary mask template is a binary four-dimensional tensor consistent with the size of the input image tensor, which is generated by equally dividing the normalized depth map by depth value.
[0033] In this step, first load the RGB-D image, generate five binary mask templates and five binary anti-mask templates opposite to the binary mask templates according to the depth range of the depth map Figure 2 Schematic diagram, M n (x, y) is a normal mask, is an anti-mask), and the image is divided into five layers using the mask, and the slicing effect diagram is shown in Figure 2 The formula of the binary mask is:
[0034]
[0035]
[0036] Where, depth(x, y) is the depth distribution of the 3D scene, x, y are the horizontal and vertical coordinates of the depth image matrix, z n-1 and z n are the boundaries of the depth layer, and the boundary values of each interval are obtained by dividing the entire depth range "0-1" into five intervals.
[0037] Step two, the edge of the slice corresponding to the deepest depth is supplemented with information, and then the supplemented slice and the initial zero phase are combined into a complex amplitude field;
[0038] The complex amplitude field refers to an image composed of complex numbers, which can be split into amplitude images and phase images by transformation, and the complex amplitude field is usually used for optical diffraction calculation;
[0039] Step three, propagate the complex amplitude field to the next depth plane using the angular spectrum method, and obtain the same complex amplitude field;
[0040] The complex amplitude field u0(x, y) is propagated to the next depth plane using the angular spectrum method, and the same complex amplitude field is obtained. The formula of the angular spectrum method is:
[0041] The angular spectrum method (ASM) is used as a diffraction propagation method to calculate the reconstructed amplitude and phase of the target plane.
[0042] The expression of ASM is given by the following equation:
[0043]
[0044] Where, f ASM (φ) is the propagation operator, i represents the imaginary unit, φ(x, y) is the phase value, u0(x, y) is the complex amplitude field distribution on the holographic plane generated by the incident coherent light source. denotes the Fourier transform, is the transfer function in ASM, The expression of is written as:
[0045]
[0046] where λ is the wavelength, f x , f y are the spatial frequencies in x and y directions, respectively, and z is the distance between the hologram and the target plane.
[0047] Step four, using the inverse mask to shield the complex amplitude field obtained by propagation corresponding to the slice position of the depth of the layer and replace it with the slice of the depth layer;
[0048] Step five, the amplitude of the edge part of the complex amplitude obtained in step four is supplemented with information, and the phase of the shielded part is supplemented with smoothing processing, and then the processed amplitude and phase are recombined into a complex amplitude;
[0049] The information supplementing of the amplitude of the edge part is to use the information of the pixels around the edge to supplement the edge area of the shielded part, so that the extended image edge can compensate for the optical diffraction of the shielded part of the image edge, and the extended image edge is as similar as possible to the existing edge information.
[0050] The smoothing supplementing of the phase of the shielded part is first to use the information of the pixels around the edge to supplement the edge area of the shielded part, and then to use the information of the supplemented part to fill the internal shielded phase, so that the obtained phase diagram can ensure that the phase of the slice area is the same and the phase of the slice edge is smooth, and the phase edge mutation of the phase diagram is eliminated as much as possible.
[0051] The amplitude of the edge part of the complex amplitude obtained is supplemented with information, and the phase of the shielded part is supplemented with smoothing processing, and then the processed amplitude and phase are recombined into a complex amplitude; the expression of the above process is given by the following equation:
[0052]
[0053]
[0054]
[0055] where C′ n is the complex amplitude field propagated by the angular spectrum method (ASM), C′ n a is the amplitude component of the complex amplitude field C′ n , C′ np is the phase component of the complex amplitude field C' n Inpaint(·) represents image occluded edge supplementation (IOES), PsInpaint(·) phase occlusion smooth supplementation (POSS), is the IOES processed amplitude map, is the POSS processed phase map;
[0056] Step six, repeat step three, step four, step five until the final step five is completed in the last depth plane to get the final complex amplitude image to complete the synthesis of the hologram.
[0057] In the embodiments of the present application, the method shown in Figure 1 is used, Figure 3 Figure 4 The resolution of the original image is set to 1024*1024, 1920*1080 respectively, and the three-color wavelengths are 638nm, 520nm, and 450nm respectively. The RGB image is divided into five layers according to the depth map, and the interval Δd of the five layers is set to 0.4mm. The simulation results are shown in Figure 3 , 4 , Figure 3 (a) is the generation result of the traditional layer-based multi-depth 3D hologram, Figure 3 (b)-(e) are the results obtained by the amplitude and phase processing of the method described in the present application, and the pictures obtained by focusing on the four different block parts below, left, right, and above respectively, Figure 4 (a) is the hologram obtained by the traditional method, Figure 4 (b) is the result obtained by the method described in the present application, Figure 4 Both holograms are focused on the front figure model. It can be seen that the present application can reconstruct a clear three-dimensional scene with correct occlusion relationship compared with the traditional method, and the generated hologram has a clear front-back relationship, and the edges of the focused and non-focused surfaces are clearer and closer to the real scene. In addition, the results also show that the method greatly reduces the edge artifacts between the hologram layers, and the non-focused surface of the generated hologram has a natural optical diffraction phenomenon.
[0058] The content described in the embodiments of the present specification is only a list of implementation forms of the inventive concept, and is only for illustrative purposes. The protection scope of the present application should not be regarded as being limited to the specific forms described in the present embodiments, and the protection scope of the present application also extends to the equivalent technical means that can be thought of by those skilled in the art according to the inventive concept.
Claims
1. A method for synthesizing multi-depth 3D computational holograms based on inter-layer edge processing, characterized in that, The method includes the following steps: Step 1: First, load the RGB-D image, generate five binary mask templates and five binary inverse mask templates based on the depth range of the depth map, and use the masks to divide the image into five layers. Step 2: Supplement the information of the edge of the slice corresponding to the deepest depth, and then combine the supplemented slice with the initial zero phase to form a complex amplitude field; Step 3: Use the angular spectrum method to propagate the complex amplitude field to the next depth plane, which also yields a complex amplitude field. Step 4: Use a reverse mask to block the slice position corresponding to the depth of the propagated complex amplitude field and replace it with a slice of the layer at that depth; Step 5: Supplement the information of the occluded edge portion of the amplitude in the complex amplitude obtained in Step 4, and perform smoothing supplementation on the occluded phase portion. Then, recombine the processed amplitude and phase to form a complex amplitude. Supplementing the information of the occluded edge portion of the amplitude is done by using the information of the pixels around the edge to supplement the edge region of the occluded portion. Smoothing supplementation on the occluded phase portion is done by first using the information of the pixels around the edge to supplement the edge region of the occluded portion, and then using the information of the supplemented portion to smoothly fill all the occluded phases inside. Step 6: Repeat steps 3, 4, and 5 until step 5 is completed in the last depth plane to obtain the final complex amplitude image and complete the hologram synthesis.
2. The method for synthesizing multi-depth 3D computational holograms based on inter-layer edge processing as described in claim 1, characterized in that, In step one, the RGB-D image consists of two images: one is a regular RGB three-channel color image, and the other is a Depth image. Each pixel value in the Depth image represents the relative distance between the sensor and the object.
3. A method for synthesizing multi-depth 3D computational holograms based on interlayer edge processing as described in claim 1 or 2, characterized in that, In step two, the complex amplitude field refers to an image composed of complex numbers, which can be transformed into an amplitude image and a phase image. The complex amplitude field is usually used for optical diffraction calculations.
4. A method for synthesizing multi-depth 3D computational holograms based on interlayer edge processing as described in claim 1 or 2, characterized in that, In step one, the binary mask template is a binary four-dimensional tensor with the same size as the input image tensor, generated by dividing the normalized depth map into intervals with equal depth values. The binary inverse mask template is numerically opposite to the binary mask template.