Method for quickly generating all-round holographic image based on motion compensation

Abstract

The invention provides a generation method for accelerating a holographic video in 360-degree holographic display based on a rotation motion compensation algorithm of a segmented wave field. The method comprises the following steps: firstly, according to a given three-dimensional object, rotating to an mth reference visual angle along a horizontal direction, and rendering a three-dimensional scene under the visual angle; secondly, the three-dimensional object is divided into a plurality of basic objects in the depth direction (z-axis), each basic object comprises point cloud data with the depth range being L, and the corresponding central plane is located at the position where z = + / -nL; for each basic object, independently calculating wavefront distribution of the basic object on a holographic plane by adopting a traditional fast holographic algorithm to obtain a corresponding basic hologram; overlapping all the basic holograms to generate an mth period reference frame, and taking the mth period reference frame as an mkth frame in the holographic video at the same time; on the basis of the mth period reference frame, within a certain horizontal view angle interval, executing a rotation motion compensation operation based on frequency spectrum mapping on each basic hologram, generating a compensation frame, and performing compensation before and after the mkth frame; and repeating the above operations until all holographic frames are generated in a 360-degree full-view angle range, and finally outputting a holographic video sequence which is high in quality and frame rate and retains depth and motion parallax information. The algorithm has the advantages of high calculation speed, high-quality 360-degree holographic video generation and the like.

Description

Technical Field

[0001] This invention relates to a method for accelerating the generation of holographic video in 360-degree holographic displays by using a segmented wave field rotational motion compensation algorithm, and belongs to the field of display technology. Background Technology

[0002] Holographic displays are considered the ultimate 3D display technology because they can provide all the depth cues required by the human visual system. Computer-generated holography, which eliminates the need for complex optical recording processes and enables the display of virtual objects, has become the dominant technology in holographic displays. However, due to the limited spatial bandwidth product of commercial spatial light modulators, the reconstructed images suffer from narrow viewing angles and insufficient motion parallax.

[0003] To address these challenges, 360-degree holographic display technology was developed. This technology seamlessly stitches together multiple viewpoints using a spatiotemporal multiplexing method to form a circular viewing area. This method requires computing computer-generated holograms for hundreds of horizontal viewpoints to ensure smooth motion parallax as the observer's viewpoint changes. These holograms, serving as the basic holographic frames for holographic video, are sequentially displayed on a digital micromirror device at a high refresh rate. However, the heavy computational burden associated with this process limits real-time viewing of 360-degree holographic displays.

[0004] Currently, various fast algorithms have been proposed to accelerate CGH computation, such as point source methods, line source methods, area source methods, tomography methods, GPU-based methods, and deep learning-based methods. However, most of these methods focus on accelerating the computation of a single hologram, meaning that the generation of holographic videos with a large number of frames still requires frame-by-frame computation. Therefore, as the number of frames increases, the computation time increases significantly, becoming a major bottleneck for real-time applications. To overcome this drawback, utilizing the correlation between consecutive frames and reducing redundant computation through block-based motion compensation is an effective solution that has been widely used in traditional video processing.

[0005] However, due to the diffraction properties, any object motion causes changes in the entire hologram, making this approach ineffective when applied to holographic frames. To date, only a few attempts have been made to address this issue. Some methods based on novel lookup tables (NLUTs) have been proposed, utilizing the translation invariance of LUTs to compensate for translational motion parallel to the holographic plane. For 3D rotational motion, effective compensation can be achieved by using the rotational properties of spherical holograms. However, converting a planar hologram to a spherical representation requires complex approximations. Recently, a general rigid body motion compensation method was proposed for efficient holographic frame generation. This method achieves motion compensation on a reference frame with accumulated motion vectors by treating the holographic changes caused by 3D rotational motion as a wavefield transformation of the central rotational plane. However, this approximation process perturbs the original depth information of the 3D object, resulting in a 2D planarization effect in the reconstructed image. Therefore, the reconstruction quality and motion parallax effect of the compensated holographic frame suffer significant degradation.

[0006] In view of the above, the present invention aims to provide a rotational motion compensation algorithm to accelerate the generation of 360-degree surround-view holographic videos, so as to solve one or more of the above-mentioned technical problems. Summary of the Invention

[0007] To address one or more technical problems in the prior art, according to one aspect of the present invention, a rotational motion compensation algorithm for accelerating the generation of 360-degree holographic videos is provided. It is characterized by comprising the following steps:

[0008] Given a 3D object, rotate it horizontally to the reference viewpoint mθ of the m-th cycle. p It also performs rendering processing on the 3D scene from that perspective, including lighting calculation and occlusion relationship determination.

[0009] The 3D object is divided into multiple basic objects along the depth direction (z-axis). Each basic object contains point cloud data with a depth range of L, and its corresponding center plane is located at z = ±nL.

[0010] For each basic object, its wavefront distribution on the holographic plane is independently calculated using a traditional fast holographic algorithm to obtain the corresponding basic hologram. All basic holograms are then superimposed to generate the m-th periodic reference frame, which also serves as the m-th periodic reference frame in the holographic video. k frame.

[0011] Based on the m-th periodic reference frame, within a certain horizontal viewing angle range, a rotational motion compensation operation based on spectral mapping is performed on each basic hologram to generate a compensation frame. k Compensation is performed before and after each frame.

[0012] Repeat the above steps until all holographic frames within the 360-degree field of view are generated, ultimately outputting a high-quality, high-frame-rate holographic video sequence that retains depth and motion parallax information.

[0013] According to another aspect of the present invention, the three-dimensional object can be of any geometric structure, supports static or dynamic rotating scenes, and is suitable for monochrome or color holographic display systems.

[0014] According to another aspect of the present invention, the conventional fast holographic algorithm includes, but is not limited to: algorithms based on angular spectrum propagation, algorithms based on point spread function superposition, algorithms based on GPU parallel acceleration, and end-to-end generation algorithms based on deep neural networks.

[0015] According to another aspect of the present invention, the segmentation depth L of the basic object can be dynamically adjusted according to the system reconstruction accuracy requirements. The finer the segmentation, the smaller the compensation error and the higher the reconstruction quality.

[0016] According to another aspect of the invention, the periodic reference frame interval θ p The maximum compensation error allowed by the system is determined, and the value is usually in the range of 0°–10°, to ensure that the error caused by the approximate planarization process is acceptable within this angle range.

[0017] According to another aspect of the present invention, the spectrum mapping employs an interpolation algorithm to resample the non-uniform spectrum to an equally spaced spectrum grid, supporting bilinear interpolation, cubic spline interpolation, or nearest neighbor interpolation.

[0018] According to another aspect of the invention, the rotational motion compensation function includes spectral coordinate transformation, phase offset correction, and Jacobian determinant correction factor cosθ to maintain energy conservation and wavefront continuity.

[0019] According to another aspect of the present invention, the method is compatible with existing spatial light modulator hardware platforms such as DMD and LCOS, and supports high refresh rate (>60fps) dynamic holographic display.

[0020] Compared with the prior art, the present invention has one or more of the following technical effects:

[0021] First, this invention uses the innovative concept of "segmented wave field approximation" to decompose the complex 3D object wave field rotation transformation into multiple simple 2D plane wave field transformations, which greatly reduces the computational complexity of 360-degree holographic video generation. It breaks through the dependence of traditional methods on high-performance computing hardware for 3D hologram calculation, enabling ordinary processors to efficiently generate continuous holographic videos.

[0022] Secondly, an error compensation mechanism is established based on periodic reference frames. This mechanism not only controls the loss of depth information caused by planar approximation through accurate calculation of reference frames, but also periodically updates the occlusion relationship of 3D objects. This solves the problem of error accumulation with rotation angle in traditional motion compensation methods and significantly improves the reconstruction accuracy of holographic videos within a large field of view.

[0023] Finally, the motion compensation method for 360-degree holographic video of the present invention has good compatibility and scalability: on the one hand, it is compatible with different types of hologram calculation algorithms (traditional fast methods) and display devices (DMD, LCoS-SLM, etc.); on the other hand, by adjusting parameters such as the number of segments and the reference frame interval, a flexible balance can be struck between generation efficiency and reconstruction accuracy to meet the needs of 360-degree holographic display in different scenarios.

[0024] This invention provides a novel technical approach for achieving high-quality, high-efficiency, and low-cost 360-degree dynamic three-dimensional holographic display, and has significant academic value and broad application prospects. Attached Figure Description

[0025] To understand the details of the above-described features of the present invention, a more detailed description of the invention, briefly summarized above, can be obtained by referring to the embodiments. The accompanying drawings relate to preferred embodiments of the invention and are described below:

[0026] Figure 1 A flowchart illustrating the operation method of the motion compensation algorithm for 360-degree holographic three-dimensional display proposed in this invention.

[0027] Figure 2 This is a schematic diagram illustrating the relationship between the holographic frame and rotational motion according to the present invention;

[0028] Figure 3 This is a schematic diagram of the two-dimensional object plane wave field rotation transformation according to the present invention;

[0029] Figure 4 This is a schematic diagram of the segmented wave field-based rotational transformation of a 3D object according to the present invention. Specific Implementation

[0030] Various embodiments will now be described in detail, one or more examples of which are illustrated in the figures. The examples are provided for illustrative purposes and are not intended to be limiting. For example, features illustrated or described as part of one embodiment can be used in or combined with any other embodiment to produce yet another embodiment. The invention is intended to include such modifications and variations.

[0031] In the following description of the accompanying drawings, the same reference numerals indicate the same or similar structures. Generally, only the differences between individual embodiments will be described. Unless otherwise expressly indicated, the description of parts or aspects of one embodiment can also be applied to corresponding parts or aspects of another embodiment.

[0032] Example 1

[0033] See Figure 1 It illustrates a method for accelerating the generation of holographic videos in 360-degree holographic displays by using a segmented wavefield-based rotational motion compensation algorithm, characterized by the following steps:

[0034] First, based on the given 3D object, rotate it horizontally to the reference viewpoint mθ of the m-th cycle. p It also performs rendering processing on the 3D scene from that perspective, including lighting calculation and occlusion relationship determination.

[0035] The 3D object is divided into multiple basic objects along the depth direction (z-axis). Each basic object contains point cloud data with a depth range of L, and its corresponding center plane is located at z = ±nL.

[0036] For each basic object, the wavefront distribution on the holographic plane is independently calculated using the traditional fast holographic algorithm to obtain the corresponding basic hologram; all basic holograms are superimposed to generate the m-th periodic reference frame, which is also used as the mk-th frame in the holographic video.

[0037] Based on the m-th period reference frame, within a certain horizontal viewing angle range θ p Within, a rotational motion compensation operation based on spectral mapping is performed on each basic hologram to generate a compensation frame, for the m-th... k Compensation is performed before and after each frame.

[0038] Repeat the above steps until all holographic frames within the 360-degree field of view are generated, ultimately outputting a high-quality, high-frame-rate holographic video sequence that retains depth and motion parallax information.

[0039] According to a preferred embodiment of the present invention, the three-dimensional object in step 01 can be of any geometric structure, supports static or dynamic rotating scenes, and is suitable for monochrome or color holographic display systems.

[0040] According to a preferred embodiment of the present invention, the conventional fast holographic algorithm in step 03 includes, but is not limited to: algorithms based on angular spectrum propagation, algorithms based on point spread function superposition, algorithms based on GPU parallel acceleration, and end-to-end generation algorithms based on deep neural networks.

[0041] According to a preferred embodiment of the present invention, the segmentation depth L of the basic object in step 03 can be dynamically adjusted according to the system reconstruction accuracy requirements. The finer the segmentation, the smaller the compensation error and the higher the reconstruction quality.

[0042] According to a preferred embodiment of the present invention, in step 04, the periodic reference frame interval θ p The maximum compensation error allowed by the system is determined, and the value is usually in the range of 0°–10°, to ensure that the error caused by the approximate planarization process is acceptable within this angle range.

[0043] According to a preferred embodiment of the present invention, in step 04, the spectrum mapping uses an interpolation algorithm to resample the non-uniform spectrum to an equally spaced spectrum grid, supporting bilinear interpolation, cubic spline interpolation, or nearest neighbor interpolation.

[0044] According to a preferred embodiment of the present invention, the rotational motion compensation function in step 04 includes spectral coordinate transformation, phase offset correction and Jacobian determinant correction factor cosθ, so as to maintain energy conservation and wavefront continuity.

[0045] Figure 2 This is a schematic diagram illustrating the relationship between holographic frames and rotational motion according to the present invention. To accelerate the generation process of holographic video, this embodiment employs a motion compensation strategy: based on the reference frame H0, a compensation frame Hn for intermediate viewpoints is generated by establishing a mapping relationship between the rotational motion of the three-dimensional object and the changes in the hologram, thereby avoiding the need to perform complete holographic calculations for each viewpoint. This method significantly reduces computational complexity while maintaining the motion parallax and viewpoint continuity of the reconstructed image.

[0046] Specifically, H0 represents the wavefront distribution of a 3D object propagating to the holographic plane from the initial horizontal viewing angle; the horizontal viewing angle sampling interval is denoted as Δθ, which is the viewing angle resolution of the 360-degree holographic display system. Under these conditions, the nth frame in the holographic video can be represented as a hologram Hn corresponding to a horizontal viewing angle of nΔθ degrees. The visual effect obtained by reconstructing this holographic video is equivalent to the 3D object undergoing continuous rotation around the y-axis.

[0047] Figure 3 This is a schematic diagram of the wave field transformation of a two-dimensional object plane under rotational motion. To simplify the analysis of the impact of three-dimensional rotation on the hologram, this embodiment first studies the wave field changes on the holographic plane caused by the rotation of a two-dimensional object plane at a fixed depth around the y-axis. Based on this, a compensation model is established to compensate for adjacent viewpoints.

[0048] Specifically, suppose the object plane is initially located at depth d0, with its center at the origin of the local coordinate system. When this plane is rotated counterclockwise by an angle θ around the y-axis, its diffraction wavefront on the holographic plane also changes. According to the theory of angular spectrum diffraction and the principle of coordinate rotation transformation, the rotated hologram can be obtained from the original hologram through frequency domain mapping and phase correction, without the need to recalculate the complete wavefront propagation.

[0049] Specifically, the rotation operation in the frequency domain manifests as a nonlinear remapping of spectral coordinates, accompanied by a phase shift and an energy correction factor cosθ related to the rotation angle. This process can be achieved through the following steps:

[0050] First, a Fourier transform is performed on the original hologram. Then, a coordinate transformation is performed on the spectrum in the frequency domain according to the rotation angle, and an interpolation method is used to resample the non-uniform spectrum to the standard grid. Next, a linear phase factor caused by the coordinate system offset is introduced and multiplied by a Jacobian correction term to maintain the conservation of wave field energy. Finally, a compensated hologram under the rotating viewpoint is obtained through an inverse Fourier transform.

[0051] This method transforms the complex three-dimensional rotation problem into a frequency domain transformation of a two-dimensional plane wave field, avoiding the high overhead of frame-by-frame holographic computation, while effectively preserving the phase continuity and viewpoint consistency of the reconstructed image. The two-dimensional rotational motion compensation model established in this embodiment lays the theoretical foundation for subsequently segmenting three-dimensional objects along the depth direction and applying this compensation strategy layer by layer.

[0052] Figure 4 This is a schematic diagram of the piecewise wave field-based rotation transformation of a 3D object according to the present invention. Based on the aforementioned two-dimensional object plane rotation motion compensation theory, this embodiment proposes a piecewise wave field rotation transformation method for three-dimensional objects to expand its application in three-dimensional scenes. The three-dimensional object is divided into multiple basic objects along the z-axis, each basic object containing point cloud data within a certain depth range, with center point O. +n To O -n Arranged coaxially along the z-axis. Each basic object corresponds to an initial hologram H0. +n ...H0 -n , representing the wavefront distribution propagating from each basic object to the holographic plane.

[0053] The specific operating steps are as follows:

[0054] First, based on the required reconstruction accuracy, the 3D object is divided into several basic objects along the z-axis, each with a fixed depth range L. For example, if the number of divisions is n, then the central plane of the nth basic object is located at z = +nL, denoted as L. +n .

[0055] Secondly, for each basic object, its initial hologram H0 is calculated based on angular spectrum theory. +n The wavefront spectrum is calculated, and rotational motion compensation is performed on this spectrum. This step is achieved through frequency domain mapping and phase correction, transforming the wavefront changes under the rotating perspective into spectral coordinate transformation and corresponding energy correction factor processing, thereby generating the compensated hologram H1. +n .

[0056] Then, after performing the above steps sequentially on all basic objects, the compensated holograms of each basic object under the new horizontal viewpoint are superimposed to form the hologram H1 of the entire 3D object under that viewpoint. By adjusting the number of segments, the amount of depth information contained in each basic object can be controlled, thereby ensuring the quality of the reconstructed image while maintaining computational efficiency.

[0057] The final spatial domain rotational motion compensation process is achieved by accumulating the compensation holograms of each basic object. This method simplifies the complex three-dimensional rotation problem into a series of rotational transformations of two-dimensional plane wave fields, significantly reducing the computational resources required to directly calculate the three-dimensional rotating hologram.

[0058] This embodiment introduces an error compensation mechanism based on periodic reference frames to address the loss of depth information and distortion of occlusion relationships caused by planarization processing under large-angle rotation in the piecewise wavefield approximation. This mechanism effectively suppresses error accumulation by periodically recalculating the precise hologram within specific viewing intervals, using it as a new reference frame.

[0059] Specifically, firstly, based on the system's requirements for reconstruction quality, the maximum allowable angle range ±θ for single rotational motion compensation is set. p / 2, where θ p The reference frame interval (typically 2°–10°). At each mθ p At viewpoints (e.g., 0°, 5°, 10°, etc.), traditional fast holographic algorithms (such as angular spectrum or GPU-accelerated point source methods) are used to accurately re-render the 3D object. This includes lighting calculations and occlusion relationship determination under the current viewpoint, and the basic objects are re-segmented along the depth direction to ensure that each basic object corresponds strictly in space to its corresponding basic hologram. At this point, the segmentation of the basic objects is based on the true geometric projection range after rotation, rather than the axial depth range under the initial viewpoint, thus accurately reflecting the structure and occlusion state of the object under that viewpoint.

[0060] Subsequently, using this precisely calculated hologram as the reference frame for the m-th period, within its neighborhood view interval [mθ] p -θ p / 2,mθ p +θ pWithin [ / 2], the segmented wavefield rotation motion compensation method is applied again to generate intermediate compensation frames. Since each compensation starts from the latest reference frame, the planarization approximation error is limited to a small angle range, and the occlusion relationship is updated periodically, which significantly improves the reconstruction fidelity of the large-view holographic video.

[0061] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Technical features in these embodiments that do not contradict each other can be combined with each other. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for accelerating the generation of holographic videos in 360-degree holographic display based on a piecewise wavefield-based rotational motion compensation algorithm, characterized in that The method comprises the following steps: According to the given three-dimensional object, rotate to the mth period reference view angle mθ in the horizontal direction p And render the three-dimensional scene under this view angle, including light calculation and occlusion relationship determination; The three-dimensional object is divided into a plurality of basic objects along the depth direction (z axis), each basic object containing point cloud data with a depth range of L and a corresponding central plane located at z = ± nL; For each basic object, a traditional fast holographic algorithm is used to independently calculate the wavefront distribution on the holographic plane to obtain a corresponding basic hologram; all basic holograms are superimposed to generate the mth periodic reference frame, which simultaneously serves as the mkth frame in the holographic video; Based on the mth periodic reference frame, within a certain horizontal viewing angle range, a rotation motion compensation operation based on spectral mapping is performed on each basic hologram to generate a compensation frame, which is compensated for the mkth frame and the frames before and after the mkth frame; The above operations are repeated until the generation of all holographic frames in the 360-degree full viewing angle range is completed, and finally a holographic video sequence with high quality, high frame rate, and preserved depth and motion parallax information is output.

2. The method of claim 1, wherein the segmented wavefield-based rotational motion compensation algorithm is used to accelerate the generation of holographic video in 360-degree holographic display. The three-dimensional object can be of any geometric structure, supports static or dynamic rotation scenes, and is suitable for monochrome or color holographic display systems.

3. The method of claim 1, wherein the segmented wavefield-based rotational motion compensation algorithm is used to accelerate the generation of holographic video in 360-degree holographic display. The traditional fast holographic algorithm includes but is not limited to: an algorithm based on angular spectrum propagation, an algorithm based on point spread function superposition, an algorithm based on GPU parallel acceleration, and an end-to-end generation algorithm based on a deep neural network.

4. The method of claim 1, wherein the segmented wavefield-based rotational motion compensation algorithm is used to accelerate the generation of holographic video in 360-degree holographic display. The segmentation depth L of the basic object can be dynamically adjusted according to the system reconstruction accuracy requirement. The finer the segmentation, the smaller the compensation error, and the higher the reconstruction quality.

5. The method of claim 1, wherein the segmented wavefield-based rotational motion compensation algorithm is used to accelerate the generation of holographic video in 360-degree holographic display. The periodic reference frame interval θ p The maximum compensation error allowed by the system is determined, typically in the range 0° - 10°, ensuring that the error caused by the near-planarization process is acceptable within this angular interval.

6. The method of claim 1, wherein the segmented wavefield-based rotational motion compensation algorithm is used to accelerate the generation of holographic video in 360-degree holographic display. The spectral mapping uses an interpolation algorithm to realize the resampling of non-uniform spectrum to an equidistant spectrum grid, supporting bilinear interpolation, cubic spline interpolation, or nearest neighbor interpolation.

7. The method of claim 1, wherein the segmented wavefield-based rotational motion compensation algorithm is used to accelerate the generation of holographic video in 360-degree holographic display. The rotation motion compensation function contains spectral coordinate transformation, phase offset correction, and Jacobian determinant correction factor cosθ to maintain energy conservation and wavefront continuity.

8. The method of claim 1, wherein the segmented wavefield-based rotational motion compensation algorithm is used to accelerate the generation of holographic video in 360-degree holographic display. The method is compatible with existing DMD, LCOS, and other spatial light modulator hardware platforms, and supports high refresh rate (> 60 fps) dynamic holographic display.

Images