A holographic imaging system and method for a VR holographic theater

Abstract

The application discloses a kind of holographic imaging system and method for VR holographic theatre, it is related to imaging technical field, the present application, obtain depth information, texture information and light field parameter constructs initial light field, based on audience position data, using machine learning algorithm generates adaptive light field, and real-time adjustment light field parameter, so that holographic image can be dynamically optimized according to the viewing angle of audience, reduce the image drift and distortion caused by position change, introduce frustum mapping algorithm to optimize parallax, and combine with structured light projection and optical modulator dynamic adjustment projection angle, compensate the optical offset of different viewing positions;Combining refracting mirror array and AI optical adjustment optimizes beam path, and projects to holographic display medium, so that the light field distribution of holographic image is more uniform, while detecting the illumination condition in theatre through ambient light sensing unit, dynamically adjusts the brightness, contrast and shadow adaptation of image based on detection result, so that it can maintain visual consistency under different illumination environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of imaging technology, in particular to a holographic imaging system and method for a VR holographic theater. BACKGROUND

[0002] With the continuous development of VR technology and the diversification of social needs, a VR holographic theater has emerged, through which a realistic three-dimensional image can be seen on a holographic screen, and can be freely viewed at different angles.

[0003] The current holographic imaging method mainly uses specific angle light field projection or interference imaging to provide clear and stable three-dimensional images at a fixed viewing angle. When the audience moves or views from a larger angle, the image will mostly be distorted, shifted, or even disappear. The image shape seen by the audience at the edge area does not match the actual content, and the three-dimensional sense is weakened.

[0004] It can be seen that the current holographic theater position has a great impact on the viewing experience. The core factor causing this problem is the calculation method of light field projection. The traditional holographic projection is preset with a projection angle. When the audience deviates from the designed viewing angle, the light path calculation is insufficient, and the imaging algorithm cannot be corrected in real time. To solve this problem, some traditional solutions increase the number of projection arrays to enable the audience to obtain relatively consistent three-dimensional images within a certain range, or use head tracking technology to dynamically adjust the projection angle according to the audience's viewing angle. However, these methods have extremely high calculation costs and are difficult to completely synchronize in a multi-person scenario. Therefore, how to enable audiences at different positions to see more stable and realistic holographic images is a technical problem that needs to be solved urgently. SUMMARY

[0005] In view of the above existing problems, the present application is proposed.

[0006] The present application provides a holographic imaging system and method for a VR holographic theater to solve the problem that the traditional solution is difficult to adapt to different viewing angles in real time in a multi-person viewing and free movement scenario, resulting in image distortion, drift or disappearance, and affecting the immersive experience.

[0007] To solve the above technical problems, the present application provides the following technical solutions:

[0008] In a first aspect, the present application embodiment provides a holographic imaging system for a VR holographic theater, comprising,

[0009] a three-dimensional data acquisition module for acquiring three-dimensional data of a holographic image and constructing an initial light field based on the three-dimensional data;

[0010] a light field calculation module receiving the initial light field and generating an adaptive light field based on audience position data through a machine learning algorithm;

[0011] An optical adjustment module, based on adaptive light field, uses a frustum mapping algorithm to optimize parallax, and combines structured light projection to adjust projection angle through an optical modulator to generate an optimized light field;

[0012] An optical modulation module, according to the optimized light field, optimizes light beam path based on a refractive mirror array and AI optical adjustment, and projects the light field to a holographic display medium;

[0013] An interactive system receives the optically optimized light field, combines audience position, motion and gaze tracking data, and adjusts light field parameters in real time.

[0014] As a preferred scheme of the holographic imaging system for the VR holographic theater, the optical modulation module comprises an ambient light perception unit for obtaining theater lighting conditions.

[0015] The optical modulation module further combines the ambient light perception unit to detect the theater lighting conditions, and adjusts the light field brightness, contrast and shadow adaptation based on the detection results.

[0016] In a second aspect, the present application provides a holographic imaging method for a VR holographic theater, comprising,

[0017] Step S1, obtaining three-dimensional data of holographic images, and constructing an initial light field based on the three-dimensional data;

[0018] Step S2, inputting the initial light field into a light field calculation module, generating an adaptive light field based on audience position data through a machine learning algorithm, and dynamically adjusting light field parameters;

[0019] Step S3, inputting the adaptive light field into an optical adjustment module, using a frustum mapping algorithm to optimize parallax, combining structured light projection, adjusting projection angle through an optical modulator, compensating for viewing position offset, and generating an optimized light field;

[0020] Step S4, inputting the optimized light field into an optical modulation module, optimizing light beam path based on a refractive mirror array and AI optical adjustment, and projecting to a holographic display medium;

[0021] In step S4, the ambient light perception unit detects the theater lighting conditions, and further adjusts the light field brightness, contrast and shadow adaptation based on the detection results.

[0022] Step S5, inputting the optically optimized light field of step S4 into an interactive system, combining audience position, motion and gaze tracking data, and adjusting light field parameters in real time.

[0023] In step S5, multi-view synchronous calibration is performed.

[0024] As a preferred scheme of the holographic imaging method for the VR holographic theater, the three-dimensional data comprises depth information, texture information and light field parameters.

[0025] As a preferred scheme of the holographic imaging method for the VR holographic theater, the step of constructing the initial light field based on the three-dimensional data comprises,

[0026] Supposing that the three-dimensional data of the holographic image is composed of depth information, texture information and light field parameters, and the spatial coordinate system of the target scene is defined as P(x, y, z):

[0027] P(x, y, z) = (D(x, y), T(x, y), L(x, y, θ, φ)),

[0028] wherein P(x, y, z) represents a three-dimensional coordinate point in the scene, x, y and z respectively represent spatial coordinate components, D(x, y) represents the depth information, describing the distance from the scene point to the reference plane, T(x, y) represents the texture information, containing color, illumination and reflection characteristics, L(x, y, θ, φ) represents the light field parameters, defined as the light intensity and propagation characteristics in the direction (θ, φ) from the point (x, y),

[0029] The light intensity distribution is calculated based on the light field rendering equation, and the light intensity distribution formula is:

[0030] I(x, y, θ, φ) = ∫ λ R(x, y, λ)L(x, y, θ, φ, λ)dλ,

[0031] wherein I(x, y, θ, φ) represents the light field intensity at the point (x, y) in the direction (θ, φ), λ represents the wavelength of light, ∫ λ · represents the wavelength integration in the spectral range, obtaining the final visible light intensity, R(x, y, λ) represents the reflectivity of the object surface to different wavelengths of light, and L(x, y, θ, φ, λ) represents the light field parameters varying with the wavelength,

[0032] The phase information of the light field is calculated by using the light wave propagation formula, and the formula is:

[0033] Ψ(x, y, θ, φ) = e ikD(x,y) ,

[0034] wherein Ψ(x, y, θ, φ) represents the phase information of the light wave, e is the base of natural logarithm, i represents the imaginary unit, and i 2 =-1.

[0035] is the wave number, describing the spatial frequency of the light wave, and D(x, y) represents the depth information.

[0036] The initial light field formula is:

[0037] F(x, y, θ, φ) = I(x, y, θ, φ) · Ψ(x, y, θ, φ),

[0038] Where F(x, y, θ, φ) is a complex representation of the initial light field intensity, combining light field intensity and phase information.

[0039] As a preferred scheme of the holographic imaging method for the VR holographic theater, the step of generating an adaptive light field based on audience position data through a machine learning algorithm and dynamically adjusting light field parameters is,

[0040] Collecting audience position data and establishing an observation coordinate system, defined as V(u, v, w):

[0041] V(u, v, w) = (X v ,Y v ,Z v ,α v ,β v ),

[0042] Where V(u, v, w) represents the position state of the audience in the theater, u, v, w are spatial position components in the observation coordinate system, X v ,Y v ,Z v represent the three-dimensional coordinates of the audience, and α v ,β v represent the viewing angle direction, defined as the horizontal angle and the vertical angle,

[0043] Based on the collected audience position information V(u, v, w), a neural network M NN is used to calculate the adaptive adjustment parameters of the light field, and the formula is:

[0044] Θ(x, y, u, v, w) = M NN (V(u, v, w), L0(x, y, θ, φ)),

[0045] Where Θ(x, y, u, v, w) represents the adaptive adjustment parameters of the light field, M NN is a neural network model for light field optimization, and L0(x, y, θ, φ) is the initial light field.

[0046] Using the light field adjustment parameters Θ(x, y, u, v, w) predicted by the neural network, the light field is optimized in real time, and the optimization formula is:

[0047] L1(x, y, θ, φ) = L0(x, y, θ, φ) · Θ(x, y, u, v, w),

[0048] Wherein, L1(x, y, θ, φ) is the dynamic adjusted light field, and the adaptive adjustment of the initial light field L0(x, y, θ, φ) is realized through Θ(x, y, u, v, w).

[0049] In order to cope with the real-time change of the audience's viewing angle, the adjusted light field is optimized and compensated, and the compensation formula is:

[0050]

[0051] Wherein, C(x, y, θ, φ) represents the dynamic compensation term of the light field, the influence of the audience position change V(u, v, w) on the light field is calculated,

[0052] The final calculation obtains the optimized adaptive light field:

[0053] L opt (x, y, θ, φ) = L1(x, y, θ, φ) + C(x, y, θ, φ),

[0054] Wherein, L opt (x, y, θ, φ) is the final adaptive light field.

[0055] As a preferred scheme of the holographic imaging method for the VR holographic theater, wherein: the step of utilizing the frustum mapping algorithm to optimize parallax, combining with structured light projection, adjusting the projection angle through the optical modulator, compensating the viewing position offset, and generating the optimized light field is,

[0056] In the holographic theater, the viewing position of each audience is different, here the frustum mapping parameters are calculated according to the audience position, and the light field is optimized by using the parallax compensation algorithm, and the frustum mapping of the audience is C v (u, v, w):

[0057] C v (u, v, w) = (θ v , φ v , d v ),

[0058] Wherein, C v (u, v, w) represents the frustum mapping parameters of the audience, u, v, w are the spatial position components in the observation coordinate system, θ v is the horizontal viewing angle range of the audience, φ v is the vertical viewing angle range of the audience, and d v is the distance from the audience to the display plane,

[0059] Based on the frustum mapping parameters of the audience, the parallax compensation factor is calculated, and the formula is:

[0060]

[0061] where S(x, y, u, v, w) represents the parallax compensation factor, which is calculated using gradient:

[0062] The impact of the change of the audience's viewing angle on the light field is calculated;

[0063] The optimized light field is represented as:

[0064] L view (x, y, θ, φ) = L opt (x, y, θ, φ) + S(x, y, u, v, w),

[0065] where L view (x, y, θ, φ) is the parallax-optimized light field;

[0066] To improve the uniformity of the light field, a structured light projection is used to optimize the light field, and a structured light projection function P struct is defined:

[0067] P struct (x, y, θ, φ) = L view (x, y, θ, φ) · G(x, y),

[0068] where P struct (x, y, θ, φ) represents the structured light projection-optimized light field, and G(x, y) is a structured light modulation function, G(x, y) = cos(k x x + k y y + φ g ), where k x , k y are spatial frequency components, and φ g is a phase shift phase;

[0069] The structured light-optimized light field is further adjusted by an optical modulator to align the projection angle with the audience's position, and an optical modulation function T adj is defined:

[0070] O mod (x, y, θ, φ) = P struct (x, y, θ, φ) · T adj (x, y, θ, φ),

[0071] where O mod (x, y, θ, φ) is the light field adjusted by the optical modulator, and T adj (x, y, θ, φ) is an optical modulator adjustment function used to compensate for the viewing position offset. The optical modulation uses a phase control based on a liquid crystal spatial light modulator (SLM):

[0072] T adj (x,y,θ,φ)=e iΦ(x,y,θ,φ) ,

[0073] Among them, T adj (x,y,θ,φ) is the optical modulator adjustment function, Φ(x,y,θ,φ) is the phase modulation distribution controlling the beam direction, e is the base of the natural logarithm, and i is the imaginary unit, satisfying i 2 =-1,

[0074] The final optimized light field was calculated as follows:

[0075] L proj (x,y,θ,φ)=O mod (x,y,θ,φ),

[0076] Among them, L proj (x,y,θ,φ) represents the light field that is ultimately projected onto the holographic display medium.

[0077] As a preferred embodiment of the holographic imaging method for VR holographic theaters described in this invention, the step of optimizing the beam path based on a refractive mirror array and AI optical adjustment, and projecting it onto the holographic display medium, is as follows:

[0078] Before a holographic light field is projected onto a display medium, the beam path needs to be optimized by a refractive mirror array. Let the refractive mirror array be M. r (x,y), the formula for calculating the beam refraction angle is defined as:

[0079]

[0080] Where, θ r (x,y) is the refraction angle of the light beam after passing through the array of refracting mirrors, θ proj (x,y) represents the incident angle of the beam projected onto the refracting mirror, and n1 and n2 represent the refractive indices of air and the refracting mirror material, respectively.

[0081] The corrected light field is:

[0082] L refract (x,y,θ r ,φ)=L proj (,y,θ proj ,φ)·T r (x,y),

[0083] Among them, L refract (x,y,θ r φ) represents the light field after passing through the array of refracting mirrors, and T represents the light field. r (x,y) represents the transmission coefficient of the refraction mirror, describing the energy loss of the light beam.

[0084] To further optimize the light beam path, an AI optical adjustment model is used to adjust the direction of the light beam, making the light field more uniform. The AI optimization function is defined as A opt , and the light beam adjustment parameters are calculated:

[0085] Θ AI (x,y)=A opt (L refract (x,y,θ r ,φ)),

[0086] wherein,

[0087] Θ AI (x,y) is the light beam adjustment parameter calculated by AI optimization, A opt is the AI optical adjustment model,

[0088] The optimized light field is:

[0089] L AI (x,y,θ r ,φ)=L refract (x,y,θ r ,φ)·Θ AI (x,y),

[0090] wherein, L AI (x,y,θ r ,φ) is the light field after AI optical adjustment optimization;

[0091] The final optimized light field is projected onto the holographic display medium, and the projected light field is defined as:

[0092] L holo (x,y,θ r ,φ)=L AI (x,y,θ r ,φ)·T h (x,y),

[0093] wherein, L holo (x,y,θ r ,φ) is the light field finally projected onto the holographic display medium, and T h (x,y) is the transmission characteristics of the holographic medium.

[0094] As a preferred scheme of the holographic imaging method for VR holographic theater, in step S5, based on the interactive system, the light field parameters are adjusted in real time combined with the audience position, action and line-of-sight tracking data, and multi-view synchronous calibration is performed, specifically:

[0095] The interactive system obtains the position, action and line-of-sight data of the audience in real time through a sensor array, including three-dimensional position, head orientation, line-of-sight direction and gaze point position, for judging the current viewing angle and interaction state of the audience;

[0096] The collected data are processed by using a neural network model to predict the dynamic adjustment parameters required by the light field, the input of the neural network being the interaction state and the current holographic projection light field of the audience, and the output being real-time light field adjustment coefficients for correcting the projection light field to match the current observation viewing angle and interaction behavior of the audience;

[0097] According to the adjustment parameters output by the neural network, the current holographic display light field is updated, and the update content includes viewing angle adaptive adjustment, brightness and contrast compensation and light field focus optimization;

[0098] The interactive system calculates the light field error under different viewing angles of the audience, and corrects it through a synchronous calibration algorithm, the synchronous calibration method including error equalization optimization and global compensation algorithm, and finally generates an optimized synchronous light field.

[0099] As a preferred scheme of the holographic imaging method for the VR holographic theater, the interactive system continuously monitors the behavior changes of the audience, and if the position, head posture or line-of-sight direction of the audience changes, the interactive system immediately calculates new light field adjustment parameters and updates the projection light field.

[0100] The error equalization optimization adjusts the consistency of light fields under different viewing angles, and the global compensation algorithm corrects the distortion of light fields under multiple viewing angles.

[0101] The sensor array includes an infrared camera, a depth camera and an eye movement tracking sensor.

[0102] The present application has the following advantages: the present application obtains depth information, texture information and light field parameters through a three-dimensional data acquisition module, constructs an initial light field, generates an adaptive light field based on audience position data by using a machine learning algorithm through a light field calculation module, and adjusts light field parameters in real time, so that the holographic image can be dynamically optimized according to the viewing angle of the audience, reducing image drift and distortion caused by position changes; in addition, a frustum mapping algorithm is introduced to optimize parallax, and the projection angle is dynamically adjusted by combining structured light projection and an optical modulator to compensate for optical offset at different viewing positions, thereby improving the consistency of the image at each viewing angle.

[0103] The application, the optical modulation module combines the refractive mirror array and the AI optical adjustment to optimize the light beam path, and projects to the holographic display medium, so that the light field distribution of the holographic image is more uniform, and the ambient light sensing unit detects the lighting conditions in the theater, dynamically adjusts the brightness, contrast and shadow of the image based on the detection result, so that it can maintain visual consistency under different lighting environments. In addition, the interactive system combines audience position, action and line of sight tracking data, adjusts the light field parameters in real time, and performs multi-view synchronous calibration to ensure the stability and consistency of the holographic image when multiple people watch.

[0104] The application focuses on improving the viewing angle adaptation capability of the holographic image, so that it can be dynamically adjusted according to the viewing angle of different audiences, reducing image distortion, drift or disappearance problems, while combining with environmental light adaptation and interactive optimization, improving the immersion of the holographic image, so that it can be more realistically integrated into the actual application environment of the VR holographic theater. BRIEF DESCRIPTION OF DRAWINGS

[0105] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creating laborious work.

[0106] Figure 1 The frame schematic diagram of the holographic imaging system for the VR holographic theater of the application.

[0107] Figure 2 The flowchart schematic diagram of the holographic imaging method for the VR holographic theater of the application. DETAILED DESCRIPTION

[0108] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings of the specification.

[0109] In the following description, many specific details are set forth in order to provide a thorough understanding of the application, but the application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the application. Therefore, the application is not limited to the specific embodiments disclosed below.

[0110] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0111] Embodiment 1, refer to Figure 1 and Figure 2 The embodiment provides a holographic imaging system for a VR holographic theater, comprising:

[0112] a three-dimensional data acquisition module for acquiring three-dimensional data of a holographic image and constructing an initial light field based on the three-dimensional data;

[0113] a light field calculation module receiving the initial light field and generating an adaptive light field based on audience position data through a machine learning algorithm;

[0114] an optical adjustment module for optimizing parallax based on the adaptive light field using a frustum mapping algorithm, and adjusting the projection angle through an optical modulator in combination with structured light projection to generate an optimized light field;

[0115] an optical modulation module for optimizing light beam paths based on a refractive mirror array and AI optical adjustment according to the optimized light field, and projecting the light field to a holographic display medium;

[0116] The optical modulation module comprises an ambient light perception unit for acquiring theater lighting conditions;

[0117] The optical modulation module further detects theater lighting conditions in combination with the ambient light perception unit, and adjusts light field brightness, contrast and shadow adaptation based on the detection results;

[0118] an interactive system receiving the optically optimized light field, combining audience position, motion and gaze tracking data, and adjusting light field parameters in real time.

[0119] The embodiment also provides a holographic imaging method for the holographic imaging system for a VR holographic theater as described above, comprising:

[0120] Step S1, acquiring three-dimensional data of a holographic image and constructing an initial light field based on the three-dimensional data;

[0121] The three-dimensional data includes depth information, texture information and light field parameters;

[0122] The step of constructing an initial light field based on three-dimensional data is,

[0123] Let the three-dimensional data of the holographic image be composed of depth information, texture information and light field parameters, and define the spatial coordinate system of the target scene as P(x, y, z):

[0124] P(x, y, z) = (D(x, y), T(x, y), L(x, y, θ, φ)),

[0125] Wherein, P(x, y, z) represents a three-dimensional coordinate point in the scene, x, y, z respectively represent the spatial coordinate components, D(x, y) represents the depth information, describes the distance from the scene point to the reference plane, T(x, y) represents the texture information, contains color, illumination and reflection characteristics, L(x, y, θ, φ) represents the light field parameter, defined as the light intensity and propagation characteristics in the direction (θ, φ) from the point (x, y),

[0126] The light intensity distribution is calculated based on the light field rendering equation, and the light intensity distribution formula is:

[0127] I(x, y, θ, φ) = ∫ λ R(x, y, λ) L(x, y, θ, φ, λ) dλ,

[0128] Wherein, I(x, y, θ, φ) represents the light field intensity of the point (x, y) in the direction (θ, φ), λ represents the wavelength of light, ∫ λ · represents the wavelength integration in the spectral range, and the final visible light intensity is obtained, R(x, y, λ) represents the reflectivity of the object surface to different wavelengths of light, and L(x, y, θ, φ, λ) represents the light field parameter varying with wavelength,

[0129] The phase information of the light field is calculated by using the light wave propagation formula, and the formula is:

[0130] Ψ(x, y, θ, φ) = e ikD(x,y) ,

[0131] Wherein, Ψ(x, y, θ, φ) represents the phase information of the light wave, e is the base of natural logarithm, i represents the imaginary unit, and i 2 =-1,

[0132] Is the wave number, which describes the spatial frequency of the light wave, and D(x, y) represents the depth information;

[0133] Then the initial light field formula is:

[0134] F(x, y, θ, φ) = I(x, y, θ, φ) · Ψ(x, y, θ, φ),

[0135] Wherein, F(x, y, θ, φ) is a complex representation of the initial light field intensity, which combines the light field intensity and phase information;

[0136] Specifically, the light field model is constructed from three-dimensional data here; the spatial position of the scene point is calculated by using the depth information, and the color and reflection characteristics are determined by combining the texture information, the distribution of the light in different directions is calculated by using the light field parameter, and the final light field intensity is calculated by using the light field rendering equation;

[0137] Step S2, input the initial light field into the light field calculation module, generate an adaptive light field based on the audience position data through a machine learning algorithm, and dynamically adjust the light field parameters;

[0138] The step of generating an adaptive light field based on the audience position data through a machine learning algorithm and dynamically adjusting the light field parameters is,

[0139] Collect audience position data and establish an observation coordinate system, defined as V(u,v,w):

[0140] V(u,v,w)=(X v ,Y v ,Z v ,α v ,β v ),

[0141] Wherein, V(u,v,w) represents the position state of the audience in the theater, u,v,w are the spatial position components in the observation coordinate system, X v ,Y v ,Z v represent the three-dimensional coordinates of the audience, and α v ,β v represent the viewing angle direction of the audience, defined as the horizontal angle and the vertical angle,

[0142] Based on the collected audience position information V(u,v,w), the neural network M NN is used to calculate the adaptive adjustment parameters of the light field, and the formula is:

[0143] Θ(x,y,u,v,w)=M NN (V(u,v,w),L0(x,y,θ,φ)),

[0144] Wherein, Θ(x,y,u,v,w) represents the light field adaptive adjustment parameter, M NN is a neural network model for light field optimization, and L0(x,y,θ,φ) is the initial light field;

[0145] Using the light field adjustment parameter Θ(x,y,u,v,w) predicted by the neural network, the light field is optimized in real time, and the optimization formula is:

[0146] L1(x,y,θ,φ)=L0(x,y,θ,φ)·Θ(x,y,u,v,w),

[0147] Wherein, L1(x,y,θ,φ) is the dynamically adjusted light field, and Θ(x,y,u,v,w) realizes the adaptive adjustment of the initial light field L0(x,y,θ,φ);

[0148] In order to cope with the real-time change of the audience's viewing angle, the adjusted light field is optimized and compensated, and the compensation formula is:

[0149]

[0150] wherein C(x, y, θ, φ) represents a light field dynamic compensation term, the influence of the audience position change V(u, v, w) on the light field is calculated,

[0151] The final calculation obtains the optimized adaptive light field:

[0152] L opt (x, y, θ, φ) = L1(x, y, θ, φ) + C(x, y, θ, φ),

[0153] wherein L opt (x, y, θ, φ) is the final adaptive light field;

[0154] Specifically, the light field is dynamically adjusted based on the audience position information here, so that it adapts to different viewing angles. Specifically, the position information of the audience is defined, the light field adjustment parameters are calculated through a neural network, and the light field is optimized. Further, the light field dynamic compensation term is calculated to correct the light field distortion caused by the audience position change, and finally the optimized adaptive light field is generated.

[0155] Step S3, input the adaptive light field into the optical adjustment module, optimize the parallax by using the frustum mapping algorithm, combine with the structured light projection, adjust the projection angle through the optical modulator, compensate for the viewing position offset, and generate an optimized light field.

[0156] The step of generating an optimized light field by using the frustum mapping algorithm to optimize the parallax and combining with the structured light projection to adjust the projection angle through the optical modulator to compensate for the viewing position offset is,

[0157] In the holographic theater, the viewing position of each audience is different. Here, the frustum mapping parameters are calculated according to the audience position, and the light field is optimized by using the parallax compensation algorithm. It is assumed that the frustum mapping of the audience is C v (u, v, w):

[0158] C v (u, v, w) = (θ v , φ v , d v ),

[0159] wherein C v (u, v, w) represents the frustum mapping parameters of the audience, u, v, and w are spatial position components in the observation coordinate system, θ v is the horizontal viewing angle range of the audience, φ v is the vertical viewing angle range of the audience, and d v is the distance from the audience to the display plane,

[0160] Based on the parameters of the audience's view frustum, the parallax compensation factor is calculated, with the formula:

[0161]

[0162] where S(x, y, u, v, w) represents the parallax compensation factor, which is calculated using the gradient:

[0163] The effect of the audience's perspective change on the light field is calculated.

[0164] The optimized light field is represented as:

[0165] L view (x, y, θ, φ) = L opt (x, y, θ, φ) + S(x, y, u, v, w),

[0166] where L view (x, y, θ, φ) is the parallax-optimized light field.

[0167] To improve the uniformity of the light field, a structured light projection is used to optimize the light field, with the structured light projection function P struct defined as:

[0168] P struct (x, y, θ, φ) = L view (x, y, θ, φ) · G(x, y),

[0169] where P struct (x, y, θ, φ) represents the structured light projection-optimized light field, and G(x, y) is the structured light modulation function, G(x, y) = cos(k x x + k y y + φ g ), where k x , k y are spatial frequency components, and φ g is the phase shift phase.

[0170] The structured light-optimized light field is further adjusted by an optical modulator to align the projection angle with the audience's position, with the optical modulation function T adj defined as:

[0171] O mod (x, y, θ, φ) = P struct (x, y, θ, φ) · T adj (x, y, θ, φ),

[0172] where O mod (x, y, θ, φ) is the light field adjusted by the optical modulator, and T adj(x,y, θ, φ) is an optical modulator adjustment function, used to compensate for viewing position offset, and the optical modulation is phase control based on a liquid crystal spatial light modulator (SLM):

[0173] T adj (x,y, θ, φ) = e iΦ(x,y,θ,φ) ,

[0174] where T adj (x,y, θ, φ) is an optical modulator adjustment function, Φ(x,y, θ, φ) is a phase modulation distribution, e is the base of the natural logarithm, i is the imaginary unit, and i 2 = -1,

[0175] The optimized light field is finally calculated as:

[0176] L proj (x,y, θ, φ) = O mod (x,y, θ, φ),

[0177] where L proj (x,y, θ, φ) is the light field finally projected to the holographic display medium,

[0178] Specifically, the optimal viewing angle of the audience is calculated by the frustum mapping, and the light field is optimized based on the parallax compensation algorithm to align the light field with different viewing positions; the uniformity of the light field is enhanced by using structured light projection, the phase grating function is used for modulation, and finally the projection angle is adjusted by the liquid crystal spatial light modulator (SLM), the optical modulator adjustment function is calculated by phase control, and the light beam direction is optimized, so that the light field projected to the holographic medium can adapt to the viewing angle of different audiences, and the realism and immersion of the holographic image are improved;

[0179] In step S4, the optimized light field is input into the optical modulation module, the light beam path is optimized based on the refractive mirror array and AI optical adjustment, and is projected to the holographic display medium;

[0180] In step S4, the environmental light perception unit detects the theater lighting conditions, and further adjusts the light field brightness, contrast and shadow adaptation based on the detection results;

[0181] The step of optimizing the light beam path based on the refractive mirror array and AI optical adjustment and projecting to the holographic display medium is,

[0182] Before the holographic light field is projected to the display medium, the light beam path needs to be optimized by the refractive mirror array, and the refractive mirror array is defined as M r (x,y), and the light beam refraction angle calculation formula is defined as:

[0183]

[0184] where θr (x,y) is the light beam incident angle before projection to the refractive mirror, n1, n2 are the refractive indexes of air and refractive mirror material, proj (x,y) is the light beam incident angle before projection to the refractive mirror, n1, n2 are the refractive indexes of air and refractive mirror material,

[0185] The corrected light field is:

[0186] L refract (x,y,θ r ,φ)=L proj (x,y,θ proj ,φ)·T r (x,y),

[0187] Wherein, L refract (x,y,θ r ,φ) is the light field after passing through the refractive mirror array, T r (x,y) is the refractive mirror transmission coefficient, describing the light beam energy loss,

[0188] In order to further optimize the light beam path, an AI optical adjustment model is used to adjust the light beam direction, so that the light field is more uniform, and the AI optimization function is set as A opt , The light beam adjustment parameter is calculated:

[0189] Θ AI (x,y)=A opt (L refract (x,y,θ r ,φ)),

[0190] Wherein,

[0191] Θ AI (x,y) is the light beam adjustment parameter calculated by AI optimization, A opt is the AI optical adjustment model,

[0192] The optimized light field is:

[0193] L AI (x,y,θ r ,φ)=L refract (x,y,θ r ,φ)·Θ AI (x,y),

[0194] Wherein, L AI (x,y,θ r ,φ) is the light field after AI optical adjustment optimization;

[0195] The final optimized light field is projected to the holographic display medium, and the projected light field is defined as:

[0196] L holo (x,y,θr , φ) = L AI (x, y, θ r , φ) · T h (x, y),

[0197] wherein L holo (x, y, θ r , φ) is the final light field projected to the holographic display medium, T h (x, y) is the transmission characteristics of the holographic medium;

[0198] Specifically, the light beam direction is adjusted by the refractive mirror array here to make it more suitable for the angle of holographic projection: the light beam refraction angle is calculated according to the refraction law and the light field is corrected, then the light beam optimization parameters are calculated using the AI optical adjustment model and the light field is adjusted, and finally the optimized light field is projected to the holographic display medium;

[0199] Step S5, input the optical optimized light field of step S4 into the interactive system, combine the audience position, action and gaze tracking data, and adjust the light field parameters in real time;

[0200] In step S5, multi-view synchronous calibration is performed;

[0201] In step S5, based on the interactive system, the audience position, action and gaze tracking data are combined to adjust the light field parameters in real time, and multi-view synchronous calibration is performed, specifically:

[0202] The interactive system acquires the position, action and gaze data of the audience in real time through a sensor array, including three-dimensional position, head orientation, gaze direction and gaze point position, which are used to determine the current viewing angle and interaction state of the audience;

[0203] The collected data is processed using a neural network model to predict the dynamic adjustment parameters required by the light field. The input of the neural network is the interaction state of the audience and the current holographic projection light field, and the output is the real-time light field adjustment coefficient, which is used to correct the projection light field to match the current observation angle and interaction behavior of the audience;

[0204] According to the adjustment parameters output by the neural network, the current holographic display light field is updated, and the update content includes angle adaptive adjustment, brightness and contrast compensation, and light field focus optimization;

[0205] The interactive system calculates the light field error under different audience viewing angles and corrects it through a synchronous calibration algorithm. The synchronous calibration method includes error equalization optimization and global compensation algorithm, and finally generates an optimized synchronous light field;

[0206] The interactive system continuously monitors the behavior changes of the audience. If the audience position, head posture or gaze direction changes, the interactive system immediately calculates new light field adjustment parameters and updates the projection light field;

[0207] Error equalization optimizes the consistency of light fields of different perspectives, and a global compensation algorithm corrects light field distortion under multiple perspectives.

[0208] The sensor array includes an infrared camera, a depth camera, and an eye tracking sensor.

[0209] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. A holographic imaging system for a VR holographic theater, characterized by: Comprising, a three-dimensional data acquisition module for acquiring three-dimensional data of the holographic image and constructing an initial light field based on the three-dimensional data; a light field calculation module receiving the initial light field and generating an adaptive light field through a neural network based on audience position data; an optical adjustment module optimizing parallax using a frustum mapping algorithm based on the adaptive light field and adjusting projection angles through an optical modulator in combination with structured light projection to generate an optimized light field; an optical modulation module optimizing light beam paths based on a refractive mirror array and AI optics according to the optimized light field and projecting the light field to a holographic display medium; an interactive system receiving the optically optimized light field and adjusting light field parameters in real time in combination with audience position, motion, and gaze tracking data; the adaptive light field is generated in particular as follows: Collecting audience position data and establishing an observation coordinate system, defined as : ， wherein represents a position state of the audience in the theater, is a spatial position component in the observation coordinate system, represents a three-dimensional coordinate of the audience, represents a viewing angle direction of the audience, defined as a horizontal angle and a vertical angle, Based on the collected audience position information , using a neural network to calculate the adaptive adjustment parameters of the light field, the formula is: ， wherein, denotes a light field adaptive adjustment parameter, is a neural network model for light field optimization, is an initial light field; denotes a spatial sampling coordinate on a plane, denotes a light direction angle from the point, is a polar / zenith angle, is an azimuth angle; Light field adjustment parameters predicted using neural networks Optimizing light field in real time with the formula: ， wherein, is the dynamically adjusted light field, by implementing adaptive adjustment on the initial light field ; To cope with real-time changes in audience viewing angles, the adjusted light field is optimized and compensated, with the compensation formula being: ， wherein, represents a light field dynamic compensation term, which calculates the change of the audience position the impact on the light field; The final calculation obtains the optimized adaptive light field as follows: ， wherein, is the final adaptive light field.

2. A holographic imaging system for a VR holographic theater as claimed in claim 1, characterized in that: The optical modulation module includes an ambient light perception unit for acquiring theater lighting conditions; The optical modulation module further detects theater lighting conditions in combination with the ambient light perception unit and adjusts light field brightness, contrast, and shadow adaptation based on the detection results.

3. A holographic imaging method for a VR holographic theater, based on the holographic imaging system for a VR holographic theater of any one of claims 1-2, characterized in that, Comprising, Step S1: Acquiring three-dimensional data of the holographic image and constructing an initial light field based on the three-dimensional data; Step S2: Inputting the initial light field into a light field calculation module, generating an adaptive light field through a neural network based on audience position data, and dynamically adjusting light field parameters; Step S3: Inputting the adaptive light field into an optical adjustment module, optimizing parallax using a frustum mapping algorithm, adjusting projection angles through an optical modulator in combination with structured light projection, compensating for viewing position shifts, and generating an optimized light field; Step S4: Inputting the optimized light field into an optical modulation module, optimizing light beam paths based on a refractive mirror array and AI optics, and projecting to a holographic display medium; In Step S4, the ambient light perception unit detects theater lighting conditions, and further adjusts light field brightness, contrast, and shadow adaptation based on the detection results; Step S5: Inputting the optically optimized light field of Step S4 into an interactive system, adjusting light field parameters in real time in combination with audience position, motion, and gaze tracking data; In Step S5, multi-view synchronous calibration is performed.

4. A holographic imaging method for a VR holographic theater as claimed in claim 3, characterized by: The three-dimensional data includes depth information, texture information, and light field parameters.

5. A holographic imaging method for a VR holographic theater as claimed in claim 4, characterized by: The step of constructing an initial light field based on three-dimensional data is as follows: The three-dimensional data of the holographic image is composed of depth information, texture information and light field parameters, and the spatial coordinate system of the target scene is defined as : ， in, Represents the three-dimensional coordinates of a point in the scene. They represent the spatial coordinate components, It represents depth information, describing the distance from a point in the scene to a reference plane. This represents texture information, including color, lighting, and reflection properties. Represents the light field parameters, defined from point Starting point, angle The intensity and propagation characteristics of light rays in different directions. Calculate light intensity distribution based on the light field rendering equation, with the formula being: ， wherein, represents a point at which the light field intensity, in the direction represents the wavelength of the light, represents the integration over the wavelengths in the spectral range, resulting in the final visible light intensity, represents the reflectivity of the object surface for different wavelengths of light, represents the light field parameter as a function of the wavelength, Calculate the phase information of the light field using the light wave propagation formula, with the formula being: ， wherein denotes the phase information of the light wave, is the base of the natural logarithm, denotes the imaginary unit, satisfying , k is the wave number, describing the spatial frequency of the light wave, represents the depth information; The initial light field formula is: ， wherein, is a complex representation of the initial light field intensity, combining light field intensity and phase information.

6. A holographic imaging method for a VR holographic theater as claimed in claim 5, characterized by: The step of optimizing parallax using a frustum mapping algorithm, adjusting projection angles through an optical modulator in combination with structured light projection, compensating for viewing position shifts, and generating an optimized light field is as follows: In the holographic theater, the viewing position of each audience is different, here the viewing cone mapping parameters are calculated according to the audience position, and the light field is optimized by using the parallax compensation algorithm, and the viewing cone mapping of the audience is, : ， wherein represents a frustum mapping parameter of the viewer, is a spatial position component in the observation coordinate system, is a horizontal viewing angle range of the viewer, is a vertical viewing angle range of the viewer, is a distance of the viewer to the display plane, Calculate the parallax compensation factor based on the audience's frustum mapping parameters, with the formula being: ， wherein denotes the parallax compensation factor, calculated using the gradient: calculating the effect of the change in the viewer perspective on the light field; The optimized light field is represented as: ， wherein, is a parallax optimized light field; To improve the uniformity of the light field, a structured light projection is used here to optimize the light field, and a structured light projection function is defined : ， wherein, represents the optimized light field of the structured light projection, is a structured light modulation function, wherein, are spatial frequency components, respectively, is a phase shift phase; The light field after structured light optimization is further adjusted by an optical modulator to align the projection angle with the audience position, defining an optical modulation function : ， wherein, is the light field adjusted by the optical modulator, is an optical modulator adjustment function to compensate for viewing position offset, the optical modulation being phase control based on a liquid crystal spatial light modulator SLM: ， wherein is an optical modulator adjustment function, is a phase modulation profile, controlling the light beam direction, is the base of the natural logarithm, is the imaginary unit, satisfying , The final calculation obtains the optimized light field as follows: ， wherein, is the light field that is ultimately projected to the holographic display medium.

7. A holographic imaging method for a VR holographic theater as claimed in claim 6, characterized by: The step of optimizing light beam paths based on a refractive mirror array and AI optics and projecting to a holographic display medium is as follows: Before the holographic light field is projected to the display medium, the light beam path needs to be optimized by a refractive mirror array, the refractive mirror array is , and a light beam refraction angle calculation formula is defined as: ， wherein, is the angle of refraction of the light beam after the array of refractive mirrors, is the angle of incidence of the light beam before the projection onto the refractive mirror, are the refractive indices of air and the material of the refractive mirror, The corrected light field is: ， wherein, is the light field after the refractive lens array, is the refractive lens transmission coefficient, describing the loss of light beam energy, To further optimize the light beam path, an AI optical adjustment model is used to adjust the direction of the light beam, making the light field more uniform. The AI optimization function is set as , and the light beam adjustment parameters are calculated. ， wherein, to optimize the computed beam adjustment parameters for AI, for an AI optical adjustment model, The optimized light field is: ， wherein, is the light field optimized by AI optical adjustment; The final optimized light field is projected to a holographic display medium, with the projected light field being defined as ， wherein, is the light field as finally projected to the holographic display medium, is the transmission characteristic of the holographic medium.

8. A holographic imaging method for a VR holographic theater as claimed in claim 7, characterized by: In step S5, the interactive system combines audience position, motion and gaze tracking data to adjust light field parameters in real time and perform multi-view synchronization calibration. Specifically: The interactive system acquires audience position, motion and gaze data in real time through a sensor array, including three-dimensional position, head orientation, gaze direction and fixation point position, to determine the audience's current viewing angle and interaction state; A neural network model is used to process the collected data and predict the dynamic adjustment parameters required for the light field. The input of the neural network is the audience's interaction state and the current holographic projection light field, and the output is the real-time light field adjustment coefficient, which is used to correct the projection light field to match the audience's current observation angle and interaction behavior; According to the adjustment parameters output by the neural network, the current holographic display light field is updated, including perspective adaptive adjustment, brightness and contrast compensation, and light field focal point optimization; The interactive system calculates the light field error under different audience viewing angles and corrects it through a synchronization calibration algorithm. The synchronization calibration method includes error equalization optimization and global compensation algorithm, and finally generates an optimized synchronized light field.

9. A holographic imaging method for a VR holographic theater as claimed in claim 8, characterized by: The interactive system continuously monitors the audience's behavior changes. If the audience's position, head posture or gaze direction changes, the interactive system immediately calculates new light field adjustment parameters and updates the projection light field. The error equalization optimization adjusts the consistency of light fields at different viewing angles, and the global compensation algorithm corrects the distortion of light fields under multi-view; The sensor array includes an infrared camera, a depth camera and an eye tracking sensor.

Images