Semantic rasterization geometric occlusion elimination method suitable for dynamic interactive holographic display

Abstract

The invention provides a semantic rasterization geometric occlusion elimination method suitable for dynamic interactive holographic display, and relates to the technical field of computer generated holography (CGH), and the method comprises the steps: obtaining three-dimensional point clouds, carrying out the grouping of the three-dimensional point clouds based on semantic tags, projecting the three-dimensional point clouds to a hologram plane, and carrying out the grid division, calculating the average depth of each grid, carrying out semantic shielding judgment based on the average depth, determining visible grid units, optimizing shielding boundaries, retaining shielded points around the boundaries of the visible grid units, and generating a complete scene hologram based on the optimized visible grid units. According to the method, efficient and accurate occlusion relation updating and artifact elimination are realized through semantic grouping, rasterizing depth calculation, occlusion judgment and boundary optimization.

Description

Technical Field

[0001] This invention relates to the field of computer-generated holography (CGH) technology, and in particular to a semantic rasterized geometric occlusion culling method suitable for dynamic interactive holographic displays. Background Technology

[0002] Virtual reality (VR) and augmented reality (AR) technologies aim to build an interactive environment that deeply integrates the virtual and real worlds, providing users with a richer visual experience through precise and natural human-computer interaction and real-time feedback mechanisms. In recent years, with the rapid popularization and development of 3D display applications, holographic display technology, due to its superior 3D scene restoration capabilities, has gradually become one of the core technologies for enhancing the immersive experience of VR and AR.

[0003] Unlike traditional 3D light field display technologies based on geometric optics principles, holographic displays reconstruct 3D information through wavefront diffraction, enabling a more natural and realistic stereoscopic visual experience. Among these technologies, computer-generated holograms (CGH) numerically simulate the propagation of light waves in a 3D scene and, combined with a spatial light modulator, achieve dynamic control of the wavefront, promoting the deep integration of holographic displays with VR and AR technologies.

[0004] However, existing CGH methods have the following shortcomings in dynamic interaction scenarios: 1. Traditional CGH methods rely on predefined 3D models, and global occlusion needs to be recalculated after interaction, resulting in slow response speed; 2. Existing occlusion removal algorithms for point cloud-based CGH methods do not consider semantic relationships and are prone to generating dark band artifacts at occlusion boundaries; 3. Lacking a "semantic layering" mechanism, it is difficult to adapt to the occlusion logic in complex scenarios such as semi-transparent and complete occlusion.

[0005] Therefore, it is essential to design a semantic rasterized geometric occlusion culling method suitable for dynamic interactive holographic displays. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the purpose of this invention is to provide a semantic rasterized geometric occlusion culling method suitable for dynamic interactive holographic displays.

[0007] To achieve the above objectives, the present invention provides the following solution: This invention provides a semantic rasterized geometric occlusion culling method suitable for dynamic interactive holographic displays, comprising: Step 1: Obtain 3D point cloud; Step 2: Group the 3D point cloud based on semantic labels; Step 3: Project the 3D point cloud onto the hologram plane and divide it into meshes, then calculate the average depth of each mesh. Step 4: Perform semantic occlusion judgment based on average depth to determine visible grid cells; Step 5: Optimize occlusion boundaries, retaining occluded points around the boundaries of visible mesh cells; Step 6: Generate a complete scene hologram based on the optimized visible grid cells.

[0008] Preferably, in step 1, obtaining the 3D point cloud specifically involves: The acquired 3D point cloud of the scene is represented as a set. ,for (1) In the formula, Let be the spatial coordinates of the i-th point in the 3D point cloud. This refers to the color information of that point, i.e., its RGB value. Let L be the semantic label for that point, L be the set of semantic labels, and N be the total number of points in the point cloud.

[0009] Preferably, in step 2, the 3D point cloud is grouped based on semantic labels, specifically as follows: Based on semantic tags Grouping the point cloud yields a subset of the 3D point cloud. This enables point cloud separation of objects with different semantic meanings.

[0010] Preferably, in step 3, the 3D point cloud is projected onto the hologram plane and meshed, specifically as follows: Each 3D point cloud subset Projected onto the xy-plane, where the hologram plane is perpendicular to the z-axis, and divided into a uniform grid with a fixed spatial resolution, each point... Corresponding grid index for: , (2).

[0011] Preferably, in step 3, the average depth of each grid is calculated, specifically as follows: For each semantic tag The grid cells below Calculate the average depth of the corresponding point set, which is used as the depth representation of the grid, as follows: (3).

[0012] Preferably, in step 4, semantic occlusion is determined based on the average depth to identify visible mesh cells, specifically as follows: For those with the same grid index Multiple semantic levels are sorted in ascending order of depth value. Let the set of grids corresponding to all semantic categories at this position be: (4) In the formula, N is the number of semantic label categories contained at that position, for any grid ,satisfy For sets Two consecutive grid cells in and By calculating the transmittance coefficient To determine whether the latter is obscured by the former; like ,express quilt Completely covered All subsequent grid cells will be discarded; like ,reserve And continue to perform the testing; Get the index The set of visible mesh cells at that location is: (5).

[0013] Preferably, in step 5, the occlusion boundary is optimized, retaining the occluded points around the visible mesh cell boundary, specifically: The occlusion boundary is determined by detecting the depth difference between adjacent grid cells. A parameter r is introduced, which represents the expansion radius centered on the occlusion boundary grid cell. The Chebyshev distance is used to measure the proximity relationship between grid cells. When performing occlusion culling, in addition to retaining visible points, occluded points within a range of r around the occlusion boundary are also retained.

[0014] Preferably, in step 6, a complete scene hologram is generated based on the optimized visible grid cells, specifically as follows: For the visible point cloud of each semantic object, calculate the complex amplitude sub-hologram; The sub-holograms are superimposed to construct a complete scene hologram.

[0015] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention provides a semantic rasterized geometric occlusion removal method suitable for dynamic interactive holographic displays. The method includes acquiring a 3D point cloud, grouping the 3D point cloud based on semantic tags, projecting the 3D point cloud onto a hologram plane and performing grid division, calculating the average depth of each grid, performing semantic occlusion judgment based on the average depth, identifying visible grid units, optimizing occlusion boundaries, retaining occluded points around the boundaries of visible grid units, and generating a complete scene hologram based on the optimized visible grid units. This invention achieves efficient updating of occlusion relationships and artifact elimination in dynamic interactive scenes through semantic tag grouping, rasterized depth calculation, semantic occlusion judgment, and boundary optimization. It can adapt to complex scenes such as semi-transparent and completely occluded scenes, significantly improving the real-time performance, realism, and interactivity of holographic displays. Numerical simulations and optical experiments show that this invention performs excellently in non-Lambertian scenes such as mirror images and underwater objects, producing high-quality reconstructed images with clear and natural occlusion boundaries. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the overall framework of interactive holography; Figure 1 a is a schematic diagram of a 3D point cloud scene constructed from an RGB image. Figure 1 b is a hologram of the scene generated based on the occlusion processing model. Figure 1 c is a schematic diagram based on the editing of the object's point cloud; Figure 2 A schematic diagram illustrating the impact of different input settings on the quality of holographic reconstruction. Figure 2 a is a schematic diagram comparing the impact of different input image resolutions on the reconstruction results. Figure 2 b is a schematic diagram comparing the impact of different SGCM parameters on the reconstruction results. Figure 2 c is a schematic diagram illustrating the influence of image resolution on the PSNR and SSIM of the reconstructed image. Figure 2 d represents the effect curve of SGCM parameters on the PSNR of the reconstructed image; Figure 3 This is a schematic diagram of the occlusion modeling algorithm proposed in this invention. Figure 3 'a' is a schematic diagram of wavefront occlusion modeling; according to SA-PBM, the light emitted from the brick wall is blocked by the rabbit's surface but can pass through the glass bottle's surface. Figure 3 b is a schematic diagram illustrating the independent calculation of the wavefronts of each object using an adaptive wavefront recording plane to reduce computational redundancy. Figure 3c is a schematic diagram of geometric occlusion modeling. According to SGCM, the object behind the glass bottle is visible, while the brick wall behind the rabbit is completely occluded. Figure 3 d is a schematic diagram showing that, in order to ensure accurate wavefront contribution, a portion of the occluded object points near the occlusion boundary are retained; Figure 4 A numerical simulation result of a scene consisting of a rabbit, brick walls, and glass windows is shown in the figure. Figure 4 'a' is the image used to construct the 3D scene. Figure 4 b is the simulated depth image corresponding to the three frames of interactive operation. Figure 4 c represents the multi-focal plane reconstructed image sequence of the scene; Figure 5 A schematic diagram illustrating the results of an optical experiment in a scene consisting of a rabbit, a brick wall, and a glass window; Figure 6 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The purpose of this invention is to provide a semantic rasterized geometric occlusion culling method suitable for dynamic interactive holographic displays. Through semantic grouping, rasterized depth calculation, occlusion judgment and boundary optimization, it achieves efficient and accurate occlusion relationship updating and artifact elimination.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 6 As shown, this invention provides a semantic rasterized geometric occlusion culling method suitable for dynamic interactive holographic displays, comprising: Step 1: Obtain 3D point cloud; Step 2: Group the 3D point cloud based on semantic labels; Step 3: Project the 3D point cloud onto the hologram plane and divide it into meshes, then calculate the average depth of each mesh. Step 4: Perform semantic occlusion judgment based on average depth to determine visible grid cells; Step 5: Optimize occlusion boundaries, retaining occluded points around the boundaries of visible mesh cells; Step 6: Generate a complete scene hologram based on the optimized visible grid cells.

[0022] In step 1, the 3D point cloud is obtained, specifically as follows: The acquired 3D point cloud of the scene is represented as a set. ,for (1) In the formula, Let be the spatial coordinates of the i-th point in the 3D point cloud. This refers to the color information of that point, i.e., its RGB value. Let L be the semantic label for that point, L be the set of semantic labels, and N be the total number of points in the point cloud.

[0023] In step 2, the 3D point cloud is grouped based on semantic labels, specifically as follows: Each point Includes three-dimensional spatial coordinates and semantic tags Based on semantic tags Grouping the point cloud yields a subset of the 3D point cloud. This enables point cloud separation of objects with different semantic meanings.

[0024] In step 3, the 3D point cloud is projected onto the hologram plane and meshed, specifically as follows: Each 3D point cloud subset Projected onto the xy plane, where the hologram plane is perpendicular to the z-axis, at a fixed spatial resolution. Divided into a uniform grid, such as Figure 3 As shown in c, each point Corresponding grid index for: , (2).

[0025] In step 3, the average depth of each grid cell is calculated, specifically as follows: For each semantic tag The grid cells below Calculate the average depth of the corresponding point set, which is used as the depth representation of the grid, as follows: (3).

[0026] Figure 3 The occlusion modeling algorithm proposed in this invention, Figure 3 According to SA-PBM, light emitted from the brick wall is blocked by the rabbit's surface but can pass through the glass bottle's surface. Figure 3 To reduce computational redundancy, b introduces an adaptive wavefront recording plane to independently calculate the wavefront of each object. Figure 3 According to SGCM, the object behind the glass bottle is visible, while the brick wall behind the rabbit is completely obscured. Figure 3To ensure accurate wavefront contribution, a portion of the occluded object points near the occlusion boundary are preserved.

[0027] In step 4, semantic occlusion is determined based on the average depth to identify visible mesh cells, specifically as follows: For those with the same grid index Multiple semantic levels are sorted in ascending order of depth value. Let the set of grids corresponding to all semantic categories at this position be: (4) In the formula, N is the number of semantic label categories contained at that position, for any grid ,satisfy For sets Two consecutive grid cells in and By calculating the transmittance coefficient To determine whether the latter is obscured by the former; like ,express quilt Completely covered All subsequent grid cells will be discarded; like ,reserve And continue to perform the testing; Get the index The set of visible mesh cells at that location is: (5).

[0028] In step 5, the occlusion boundary is optimized, retaining the occluded points around the visible mesh cell boundary, specifically: The occlusion boundary is determined by detecting the depth difference between adjacent grid cells. A parameter *r* is introduced, representing the expansion radius centered on the occlusion boundary grid cell. The Chebyshev distance is used to measure the proximity relationship between grid cells, where, for example... Figure 3 As shown in d, when performing occlusion culling, in addition to retaining visible points, occluded points within a range r around the occlusion boundary are also retained.

[0029] In step 6, a complete scene hologram is generated based on the optimized visible grid cells, specifically as follows: For the visible point cloud of each semantic object, calculate the complex amplitude sub-hologram, and superimpose the sub-holograms to construct the complete scene hologram, as follows: (20) (6) in, This represents the complex amplitude hologram of the i-th object in the scene.

[0030] Figure 1 The overall framework of interactive holography. Figure 1 A 3D point cloud scene is constructed based on RGB images. Point cloud models of a rabbit, a brick wall, and a glass bottle are generated from three images respectively. These object point clouds will be used to construct the 3D point cloud scene. Figure 1 b represents the hologram of the scene generated based on the occlusion processing model. The blue arrows indicate geometric occlusion processing, constructed through forward propagation; the red arrows indicate wavefront occlusion processing, constructed through backpropagation. Figure 1 c is based on editing object point clouds, which can freely construct 3D point cloud scenes and further realize holographic reconstruction.

[0031] Figure 2 A schematic diagram illustrating the impact of different input settings on the quality of holographic reconstruction. Figure 2 a shows the comparison of the impact of different input image resolutions on the reconstruction results. Input images of different resolutions are generated by a 3D mesh through a renderer, and a 1920×1080 resolution input image is used as the target image. Figure 2 b shows a comparison of the impact of different SGCM parameters on the reconstruction results. The input images of the rabbit and the brick wall were generated by the 3D mesh through the renderer, while the target amplitude image and the target depth image were generated by the complete point cloud scene through the renderer. Figure 2 c represents the curve showing the effect of image resolution on the PSNR and SSIM of the reconstructed image. Figure 2 d represents the effect curve of SGCM parameters on the PSNR of the reconstructed image.

[0032] Figure 4 This is a numerical simulation result of a scene consisting of a rabbit, brick walls, and glass windows. Figure 4 Image 'a' is used to construct the 3D scene. The RGB image is generated by the renderer from the 3D mesh model, and the lower left and upper right corners show the corresponding image segmentation mask and depth map, respectively. Figure 4 b is a simulated depth image corresponding to the three interactive operations, showing the overall reconstruction effect. Figure 4 c represents a sequence of multi-focal plane reconstructed images of the scene, showing the reconstruction results at different depths and their local magnified details.

[0033] Figure 5 This is a schematic diagram illustrating the results of an optical experiment on a scene consisting of a rabbit, a brick wall, and a glass window.

[0034] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0035] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A semantic rasterized geometric occlusion culling method suitable for dynamic interactive holographic displays, characterized in that, include: Step 1: Obtain 3D point cloud; Step 2: Group the 3D point cloud based on semantic labels; Step 3: Project the 3D point cloud onto the hologram plane and divide it into meshes, then calculate the average depth of each mesh. Step 4: Perform semantic occlusion judgment based on average depth to determine visible grid cells; Step 5: Optimize occlusion boundaries, retaining occluded points around the boundaries of visible mesh cells; Step 6: Generate a complete scene hologram based on the optimized visible grid cells.

2. The method according to claim 1, characterized in that, In step 1, the 3D point cloud is obtained, specifically as follows: The acquired 3D point cloud of the scene is represented as a set. ,for （1） In the formula, Let be the spatial coordinates of the i-th point in the 3D point cloud. This refers to the color information of that point, i.e., its RGB value. Let L be the semantic label for that point, L be the set of semantic labels, and N be the total number of points in the point cloud.

3. The method according to claim 2, characterized in that, In step 2, the 3D point cloud is grouped based on semantic labels, specifically as follows: Based on semantic tags Grouping the point cloud yields a subset of the 3D point cloud. This enables point cloud separation of objects with different semantic meanings.

4. The method according to claim 3, characterized in that, In step 3, the 3D point cloud is projected onto the hologram plane and meshed, specifically as follows: Each 3D point cloud subset Projected onto the xy-plane, where the hologram plane is perpendicular to the z-axis, and divided into dimensions of a fixed spatial resolution. A uniform grid, each point Corresponding grid index for: , （2）。 5. The method according to claim 4, characterized in that, In step 3, the average depth of each grid cell is calculated, specifically as follows: For each semantic tag The grid cells below Calculate the average depth of the corresponding point set, which is used as the depth representation of the grid, as follows: （3）。 6. The method according to claim 5, characterized in that, In step 4, semantic occlusion is determined based on the average depth to identify visible mesh cells, specifically as follows: For those with the same grid index Multiple semantic levels are sorted in ascending order of depth value. Let the set of grids corresponding to all semantic categories at this position be: （4） In the formula, N is the number of semantic label categories contained at that position, for any grid ,satisfy For sets Two consecutive grid cells in and By calculating the transmittance coefficient To determine whether the latter is obscured by the former; like ,express quilt Completely covered All subsequent grid cells will be discarded; like ,reserve And continue to perform the testing; Get the index The set of visible mesh cells at that location is: （5）。 7. The method according to claim 6, characterized in that, In step 5, the occlusion boundary is optimized, retaining the occluded points around the visible mesh cell boundary, specifically: The occlusion boundary is determined by detecting the depth difference between adjacent grid cells. A parameter r is introduced, which represents the expansion radius centered on the occlusion boundary grid cell. The Chebyshev distance is used to measure the proximity relationship between grid cells. When performing occlusion culling, in addition to retaining visible points, occluded points within a range of r around the occlusion boundary are also retained.

8. The method according to claim 7, characterized in that, In step 6, a complete scene hologram is generated based on the optimized visible grid cells, specifically as follows: For the visible point cloud of each semantic object, calculate the complex amplitude sub-hologram; The sub-holograms are superimposed to construct a complete scene hologram.

Images