A holographic imaging system and method for VR holographic theater

Abstract

The present invention discloses a holographic imaging system and method for VR holographic theater, which relates to the field of holographic imaging technology and solves the problem that traditional holographic imaging relies on fixed optical compensation devices and is difficult to cope with the color distortion problem caused by changes in ambient light intensity. The holographic imaging system includes a holographic background module, a dynamic fusion module, an intelligent light source control module and an adaptive interaction module. The holographic background module utilizes a holographic dynamic perception strategy to superimpose and project three-dimensional stereoscopic images based on the spatial layout of multiple screens in combination with theater interpretation. The dynamic fusion module utilizes a holographic fusion mechanism to track the actor's movements, and combines character elements and plot elements with the three-dimensional stereoscopic image to obtain a holographic image. The multi-screen collaborative display effect of the holographic image on multiple screens is improved, and the probability of color distortion of the holographic image is reduced.

Description

Technical Field

[0001] The present invention relates to the field of holographic imaging technology, and in particular to a holographic imaging system and method for VR holographic theater. Background Art

[0002] With the deep integration of virtual reality (VR) technology and holographic imaging technology, VR holographic theaters have gradually become an important carrier for immersive entertainment, education, and industrial simulation. However, traditional holographic theaters face significant technical bottlenecks in achieving precise synchronization between scenes and performances. Early holographic imaging systems mostly relied on manual staged control, using preset scripts to independently operate display media such as LED dynamic media walls and projection screens. This model has two core problems: First, manual intervention leads to a synchronization error rate of up to 3%-5% between scene switching and actor movements. Especially in dynamic scenes (such as fast transitions or complex interactions), the delay phenomenon will destroy the sense of immersion; second, the coordinated control of multimodal display media (such as holographic interactive transparent screens and lifting screens) lacks a dynamic optimization mechanism. The brightness and transparency parameters of different media are difficult to match the scene requirements in real time, resulting in a chaotic visual hierarchy.

[0003] Patent No. CN202411463277.8 discloses a VR multi-terminal collaborative interaction method, system, computer equipment and storage medium, including: displaying a VR scene image through a VR holographic imaging screen, when an interactive scene appears in the VR scene image after a set time value, displaying several interactive selection pop-up windows linearly in the lower edge area; receiving interactive commands input by several interactive control terminals, and installing interactive control terminals on the front seats of the VR holographic theater, and each interactive selection pop-up window corresponds to a front seat; fusing the interactive commands to obtain fusion data, and determining the plot scene of the corresponding interactive scene based on the fusion data; when the VR scene image is played and displayed, generating a VR program list selection pop-up window, receiving the VR program selection instruction input by the interactive control terminal, and determining the next VR program to be played. In this way, more connections are created between the plot and the audience, and the audience's sense of participation will be stronger.

[0004] Patent No. CN202411577875.8 discloses a holographic imaging system and method for VR holographic theater. The embodiment of the present invention receives content editing operations based on the built-in design resource pool, performs three-dimensional scene rendering, generates VR scene data; performs preview acquisition to obtain preview acquisition data; performs synchronous segmentation to obtain multiple stage transition features; performs performance motion capture and recognition, synchronizes current scene data; and performs holographic imaging control on the LED main screen, holographic lifting projection gauze screen, multiple front and rear wing-shaped transparent screens, and multiple LED extension screens. It is possible to synchronously segment the VR scene data and the preview acquisition data, obtain multiple stage transition features, and perform performance motion capture and recognition, synchronize the current scene data, and perform corresponding holographic imaging control during the official performance of the VR holographic theater, thereby avoiding synchronization control errors and improving the viewing experience of the VR holographic theater.

[0005] Education is placing higher demands on the precision of virtual-reality fusion (e.g., ±0.05mm error tolerance in spacecraft assembly) for dynamic 3D displays at the molecular level (e.g., chemical reaction processes) and industrial simulation. However, existing systems are limited by static scene rendering engines, unable to seamlessly switch between microscopic and macroscopic scenes and lacking the ability to simulate light and shadow according to physical laws. Furthermore, traditional holographic imaging relies on fixed optical compensation devices, making it difficult to cope with color distortion caused by changes in ambient light intensity. This is especially true when the refractive index of the holographic interactive transparent screen deviates, resulting in image distortion when viewed from multiple angles. Summary of the Invention

[0006] The purpose of the present invention is to provide a holographic imaging system and method for VR holographic theater, which can combine multiple screens and actor motion capture through a holographic dynamic perception strategy and a holographic fusion mechanism to improve the multi-screen collaborative display effect of holographic images on multiple screens and reduce the probability of color distortion of holographic images.

[0007] The present invention utilizes the following technical solutions:

[0008] A holographic imaging system for VR holographic theater, including a holographic background module, a dynamic fusion module, an intelligent light source control module and an adaptive interaction module; wherein,

[0009] The holographic background module is used to project three-dimensional images based on the spatial layout of multiple screens in a borderless virtual space, combining theatrical interpretation and holographic dynamic perception strategies;

[0010] The screens include dynamic media walls, fog screen projection screens, holographic interactive transparent screens and tilted holographic films; theatrical performances include background elements, character elements, plot elements and special effects elements;

[0011] The dynamic fusion module is used to track the actor's movements, combine character elements and plot elements, and use the holographic fusion mechanism to fuse them with the three-dimensional image to obtain a holographic image;

[0012] Intelligent light source control module, used to dynamically adjust the color, brightness and projection angle of the light source array according to the screen quality of different screens;

[0013] The adaptive interaction module is used to sense the audience's visual focus and gesture commands, and adjust the presentation angle and plot content of the holographic image.

[0014] Preferably, the workflow of the holographic background module includes the following steps:

[0015] First, the spatial layout and screen size of each screen are detected to obtain screen data;

[0016] Then, the digital twin algorithm is used to virtually construct the screen data to obtain a digital twin screen. At the same time, the knowledge graph is used to separate and associate background elements based on the deductive time sequence and spatial orientation to obtain a background transformation map.

[0017] Then, the holographic dynamic perception strategy is combined with plot elements to map the background transformation map to the digital twin screen to obtain a virtual background demonstration projection. The ray tracing engine and discrete element method are used to perform edge enhancement and global deduction on the virtual demonstration projection to obtain the optimal background projection.

[0018] Finally, the optimal background projection is superimposed and projected onto the corresponding screen according to the spatial layout, and then a three-dimensional stereoscopic image is displayed on each screen.

[0019] Preferably, the workflow of the holographic dynamic perception strategy includes the following steps:

[0020] S1: Extract plot elements according to time sequence using information frame extraction algorithm to obtain several image time series frames;

[0021] S2: Use the information matching algorithm to match and fuse the image time sequence frame with the background transformation map to obtain the scene fusion map;

[0022] S3: Classify the scene fusion atlas according to spatial orientation, obtain an orientation correlation time series table containing several image time series frames, and extract the projection offset angle at the same time;

[0023] S4: Compare and judge the projection offset angle with the preset azimuth conversion range;

[0024] If the projection offset angle is greater than the upper limit of the azimuth conversion range or the projection offset angle is less than the lower limit of the azimuth conversion range, the digital twin screen is rotated until the projection offset angles of all image timing frames in the azimuth association timing table are within the azimuth conversion range;

[0025] If the projection offset angles are all within the azimuth conversion range, the display effects of the image time series frames on the digital twin screen are compared and corrected.

[0026] Preferably, the workflow of the holographic dynamic perception strategy further includes the following steps:

[0027] S5: If screen tearing occurs on the digital twin screen, the timestamp interpolation algorithm is used to compensate for the inter-frame delay and reconstruct the continuous background transformation trajectory;

[0028] If color shift occurs on the digital twin screen, a natural language network is used to analyze the event nodes of the plot elements, match the color tone, and then update the global illumination of the background;

[0029] If dynamic blur mismatch occurs in the digital twin screen, the optical flow method is used to calculate the pixel motion vectors between adjacent frames, and the edge enhancement of the image time sequence frames is performed in combination with the super-resolution reconstruction algorithm and the temporal anti-aliasing method;

[0030] S6: The image time-series frames that have completed contrast correction are virtually rendered using a three-dimensional rendering algorithm to obtain a virtual background demonstration projection of each digital twin screen;

[0031] S7: Global inference is performed using the discrete element method for the virtual background demonstration projections with the same timestamp: the pixel peak signal-to-noise ratio and pixel structure similarity of the overlapping area between the digital twin screens are compared with the signal-to-noise threshold and the structure threshold respectively;

[0032] S8: If either the pixel peak signal-to-noise ratio or the pixel structure similarity is less than the signal-to-noise threshold and the structure threshold, it is determined that the timing between the digital twin screens is inconsistent, and the display effect of the image timing frame on the digital twin screen is corrected again;

[0033] If the pixel peak signal-to-noise ratio and pixel structure similarity are both greater than or equal to the signal-to-noise threshold and structure threshold, the timing between the digital twin screens is determined to be consistent, and the color error and color difference threshold of the overlapping area are judged;

[0034] S9: If the color error is greater than or equal to the color difference threshold, gamma linearization is performed on the overlapping area or color correction is performed on the digital twin screen;

[0035] If the color error is less than the color difference threshold, the color of the overlapping area is determined to be normal, and the optimal background projection is obtained.

[0036] Preferably, the workflow of the holographic fusion mechanism is:

[0037] T1: Special effect elements are divided into particle effects, rigid body effects, fluid effects, hair effects, cloth effects and soft body effects according to their expression forms;

[0038] T2: Extract plot turning points from plot elements based on Sid Field's theory to obtain several special effects plot timing points;

[0039] T3: Arrange the character animations in time sequence, and use the time segmentation algorithm according to the special effects plot timing points to extract the corresponding character action frames;

[0040] T4: Decompose the character action frames into independent motion units based on the discrete element principle, assign mass and elastic coefficients according to the character type, and generate discrete trajectory data;

[0041] T5: Define the discrete element parameters of the special effect elements, calculate the contact quantization value between the character action frame and the special effect elements, and preset the connection threshold based on the character type and special effect type;

[0042] T6: If the timing of the contact quantization value and the connection threshold is different, the timestamps of the character action frames and / or special effects elements are corrected using the dynamic time warping algorithm based on the plot elements;

[0043] If the timing of the contact quantization value and the connection threshold is the same, the contact quantization value is compared with the preset connection threshold;

[0044] T7: If the contact quantization value is greater than or equal to the connection threshold, physical feedback is triggered; if the contact quantization value is less than the connection threshold, a smooth transition algorithm is used to compensate for the inertia of special effect elements and / or character action frames;

[0045] T8: Adaptively blur and sharpen the contact edges between the character action frames and special effect elements, and fuse the character action frames, special effect elements and three-dimensional stereo images to complete the virtual character driving and holographic image construction.

[0046] Preferably, the workflow of the intelligent light source control module includes the following steps:

[0047] First, all light sources within the borderless virtual space are divided according to the spatial layout of the screen to obtain a light source array. The screen quality of all screens is then tested to obtain the screen resolution, refresh rate, response time, color gamut, brightness, contrast, color accuracy, and viewing angle.

[0048] Then, based on the emotional tone of the plot elements and the screen quality, the chromaticity, brightness and projection angle of the light source array are dynamically adjusted to compensate for the light source of the holographic theater. At the same time, a variational autoencoder is used to detect the color of the special effect elements based on the display effect to obtain a color distortion set.

[0049] The color distortion set is then parsed into the three primary colors of light, and combined with plot and background elements, the light source array is controlled to perform dynamic quantitative light compensation.

[0050] Preferably, the workflow of the adaptive interaction module includes the following steps:

[0051] First, the audience in the holographic theater is counted to obtain the audience distribution density; at the same time, the light source array is measured to obtain the theater's lighting intensity;

[0052] At the same time, smart glasses or handheld devices are used to capture the audience's movements and behaviors to obtain the focus of sight and gesture instructions;

[0053] Then, the visual field and seating arrangement of each audience member are analyzed according to the audience distribution density to obtain the audience perspective diffusion map and audience interaction limit map;

[0054] Finally, the multi-resolution holographic model is used to adjust the presentation angle of the holographic image according to the line of sight and the audience's perspective diffusion map; at the same time, the plot interaction model is used to provide interactive feedback on the holographic image and trigger hidden plots according to gesture instructions and the audience's interactive limit map.

[0055] Preferably, the workflow of the adaptive interaction module further includes the following steps:

[0056] Based on the screen function, combined with the eye focus and gesture instructions, a layered projection strategy is used to dynamically adjust each screen:

[0057] The distant environment of the background elements is presented on the dynamic media wall, and the close environment of the background elements is superimposed on the holographic interactive transparent screen;

[0058] The horizontal field of view in the plot elements is expanded on the tilted holographic film, and the special effects elements are demonstrated and simulated using a fog screen projection screen with a smoke machine and wind effect equipment, thereby constructing a borderless virtual space.

[0059] A holographic imaging method for a VR holographic theater, applied to a holographic imaging system, comprises the following steps:

[0060] A: The adaptive interaction module initializes and performs functional tests on all screens in the holographic theater to build a borderless virtual space;

[0061] B: After the screen is initialized, the holographic background module superimposes and projects 3D images based on the spatial layout of several screens, combining theatrical performance and holographic dynamic perception strategies;

[0062] C: The dynamic fusion module tracks the actor's movements and combines character and plot elements with the 3D image using a holographic fusion mechanism to construct a holographic image.

[0063] D: The intelligent light source control module dynamically adjusts the color, brightness, and projection angle of the light source array based on the screen quality of different screens, combined with plot elements and special effects elements;

[0064] E: Adaptive interaction module, which is used to dynamically adjust the spatial layout of the screen and the plot content of the holographic image based on the screen function, the audience's visual focus and gesture commands, and the layered projection strategy.

[0065] The present invention utilizes a holographic dynamic perception strategy through a holographic background module to project three-dimensional stereoscopic images based on the spatial layout of several screens in combination with theatrical interpretation; utilizes a holographic fusion mechanism through a dynamic fusion module to track the actors' movements, combines character elements and plot elements with the three-dimensional stereoscopic images to obtain a holographic image; improves the multi-screen collaborative display effect of the holographic image on several screens, and reduces the probability of color distortion of the holographic image. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0067] Figure 1 This is the principle block diagram of the holographic imaging system;

[0068] Figure 2 This is a flow chart of the holographic dynamic perception strategy;

[0069] Figure 3 This is a flow chart of the holographic fusion mechanism;

[0070] Figure 4 Flowchart of the holographic imaging method. DETAILED DESCRIPTION

[0071] The present invention is described in detail below with reference to the accompanying drawings and embodiments:

[0072] like Figures 1 to 3 As shown, the holographic imaging system for VR holographic theater described in the present invention includes a holographic background module, a dynamic fusion module, an intelligent light source control module and an adaptive interaction module; wherein,

[0073] The holographic background module is used to project three-dimensional images based on the spatial layout of multiple screens in a borderless virtual space, combining theatrical interpretation and holographic dynamic perception strategies;

[0074] In the present invention, the screen includes a dynamic media wall, a fog screen projection screen, a holographic interactive transparent screen and a tilted holographic film; the theater performance includes background elements, character elements, plot elements and special effect elements;

[0075] In this embodiment, background elements refer to the static or dynamic environment that constitutes the scene, including spatial layout, scene hierarchy (foreground, midground, background) and lighting conditions, which are used to build an immersive feeling in the virtual world;

[0076] Character elements encompass character appearance, movements, and interactive behaviors, and are the core vehicle for driving narrative and connecting with user emotions.

[0077] Plot elements refer to the story structure, event logic, and character relationship design, which determine the narrative coherence and user engagement of the virtual world;

[0078] Special effects elements include visual enhancement technologies such as lighting, particle effects, and material reflections, which are used to enhance the atmosphere or create surreal scenes.

[0079] The dynamic fusion module is used to track the actor's movements, combine character elements and plot elements, and use the holographic fusion mechanism to fuse them with the three-dimensional image to obtain a holographic image;

[0080] Intelligent light source control module, used to dynamically adjust the color, brightness and projection angle of the light source array according to the screen quality of different screens;

[0081] The adaptive interaction module is used to sense the audience's visual focus and gesture commands, and adjust the presentation angle and plot content of the holographic image.

[0082] In the present invention, the workflow of the holographic background module includes the following steps:

[0083] First, the spatial layout and screen size of each screen are detected to obtain screen data;

[0084] Then, the digital twin algorithm is used to virtually construct the screen data to obtain a digital twin screen. At the same time, the knowledge graph is used to separate and associate background elements based on the deductive time sequence and spatial orientation to obtain a background transformation map.

[0085] Then, the holographic dynamic perception strategy is combined with plot elements to map the background transformation map to the digital twin screen to obtain a virtual background demonstration projection. The ray tracing engine and discrete element method are used to perform edge enhancement and global deduction on the virtual demonstration projection to obtain the optimal background projection.

[0086] Finally, the optimal background projection is superimposed and projected onto the corresponding screen according to the spatial layout, and then a three-dimensional stereoscopic image is displayed on each screen.

[0087] In this embodiment, digital twin algorithms, knowledge graphs, ray tracing engines, and discrete element methods are all commonly used technical means in this field and will not be described in detail here.

[0088] Deductive chronology refers to the chronological order in which the plot is presented in a work, and is the creator's strategy for arranging the timeline of the story material.

[0089] Edge enhancement refers to technology that improves the clarity of object outlines in an image or scene. Its core principle is to use algorithms to enhance the contrast or sharpness of edge areas, making key structures in the projected content (such as object boundaries, text, and symbols) easier to discern.

[0090] Global inference refers to predicting or generating reasonable information of missing / fuzzy areas by analyzing the overall spatiotemporal relationship of the projected content to maintain the coherence and authenticity of the scene.

[0091] In the present invention, the workflow of the holographic dynamic perception strategy includes the following steps:

[0092] S1: Extract plot elements according to time sequence using information frame extraction algorithm to obtain several image time series frames;

[0093] S2: Use the information matching algorithm to match and fuse the image time sequence frame with the background transformation map to obtain the scene fusion map;

[0094] S3: Classify the scene fusion atlas according to spatial orientation, obtain an orientation correlation time series table containing several image time series frames, and extract the projection offset angle at the same time;

[0095] S4: Compare and judge the projection offset angle with the preset azimuth conversion range;

[0096] If the projection offset angle is greater than the upper limit of the azimuth conversion range or the projection offset angle is less than the lower limit of the azimuth conversion range, the digital twin screen is rotated until the projection offset angles of all image timing frames in the azimuth association timing table are within the azimuth conversion range;

[0097] If the projection offset angles are all within the azimuth conversion range, the display effects of the image time series frames on the digital twin screen are compared and corrected;

[0098] S5: If screen tearing occurs on the digital twin screen, the timestamp interpolation algorithm is used to compensate for the inter-frame delay and reconstruct the continuous background transformation trajectory;

[0099] If color shift occurs on the digital twin screen, a natural language network is used to analyze the event nodes of the plot elements, match the color tone, and then update the global illumination of the background;

[0100] If dynamic blur mismatch occurs in the digital twin screen, the optical flow method is used to calculate the pixel motion vectors between adjacent frames, and the edge enhancement of the image time sequence frames is performed in combination with the super-resolution reconstruction algorithm and the temporal anti-aliasing method;

[0101] S6: The image time-series frames that have completed contrast correction are virtually rendered using a three-dimensional rendering algorithm to obtain a virtual background demonstration projection of each digital twin screen;

[0102] S7: Global inference is performed using the discrete element method for the virtual background demonstration projections with the same timestamp: the pixel peak signal-to-noise ratio and pixel structure similarity of the overlapping area between the digital twin screens are compared with the signal-to-noise threshold and the structure threshold respectively;

[0103] S8: If either the pixel peak signal-to-noise ratio or the pixel structure similarity is less than the signal-to-noise threshold and the structure threshold, it is determined that the timing between the digital twin screens is inconsistent, and the display effect of the image timing frame on the digital twin screen is corrected again;

[0104] If the pixel peak signal-to-noise ratio and pixel structure similarity are both greater than or equal to the signal-to-noise threshold and structure threshold, the timing between the digital twin screens is determined to be consistent, and the color error and color difference threshold of the overlapping area are judged;

[0105] S9: If the color error is greater than or equal to the color difference threshold, gamma linearization is performed on the overlapping area or color correction is performed on the digital twin screen;

[0106] If the color error is less than the color difference threshold, the color of the overlapping area is determined to be normal, and the optimal background projection is obtained.

[0107] In this embodiment, the information frame extraction algorithm, information matching algorithm, natural language network, timestamp interpolation algorithm, optical flow method, super-resolution reconstruction algorithm, time domain anti-aliasing method, and three-dimensional rendering algorithm are all commonly used technical means in this field and will not be repeated here.

[0108] In the present invention, the workflow of the dynamic fusion module includes the following steps:

[0109] First, a multi-camera array is used to shoot each actor from multiple angles to obtain several actor performance videos. At the same time, an inertial sensor is used to capture each actor's action nodes to obtain several actor action sequence tables.

[0110] Then, the spatial projection conversion algorithm is used to convert the performance videos of each actor into several character slices according to the performance role. The semantic segmentation network is used to perform pixel-level classification on the character slices to obtain the character action timing diagram.

[0111] Then, a generative adversarial network is used in combination with a texture optimization algorithm and a multi-light rendering algorithm to generate a virtual character based on the character slices;

[0112] Finally, the virtual character is combined with the character action timing diagram and the actor action sequence table, and the character dynamic diagram is generated according to the character elements. At the same time, the holographic fusion mechanism is used according to the plot elements and special effects elements to fuse the character dynamic diagram and / or the actor with the three-dimensional stereo image, thereby completing the virtual character driving and holographic image construction.

[0113] In this embodiment, multi-camera arrays, inertial sensors, spatial projection conversion algorithms, semantic segmentation networks, generative adversarial networks, texture optimization algorithms, and multi-light source rendering algorithms are all commonly used technical means in this field and will not be elaborated here.

[0114] In the present invention, the workflow of the holographic fusion mechanism is as follows:

[0115] T1: Special effect elements are divided into particle effects, rigid body effects, fluid effects, hair effects, cloth effects and soft body effects according to their expression forms;

[0116] In this embodiment, particle effects: From sandstorms to rain, to various light effects, particle effects bring a rich visual experience to the film. Through particle replacement, you can create spectacular effects such as thousands of swords flying together and snow falling all over the sky;

[0117] Rigid body shattering: glass shattering, wall collapsing, rigid body shattering special effects will break the whole object into multiple small objects, adding realism to the film;

[0118] Fluid effects: floods, tsunamis, or the flowing water effects in commercials, fluid effects make water elements more vivid;

[0119] Hair special effects: Through the hair special effects system, you can easily create furry effects such as hair and grass, eliminating the tedious model making;

[0120] Cloth special effects: animation effects of fluttering ribbons and flowing cloth. The cloth special effects system makes animation production easier.

[0121] Soft body effects: animation effects of elastic and soft objects. Through the soft body effects system, you can easily achieve natural and smooth animations.

[0122] T2: Extract plot turning points from plot elements based on Sid Field's theory to obtain several special effects plot timing points;

[0123] In this embodiment, Syd Field's screenwriting theory system takes the "three-act structure" as its core, combines narrative diagnosis and treatment methods with character creation principles, and lays the foundation for modern film screenwriting.

[0124] A plot turning point refers to a key event or information in the narrative that breaks the linear development of the plot. It reconstructs the story's logical chain through time / space jumps, perspective switching, causal reversal, and other techniques, forcing the audience to recalibrate their understanding of the character's fate or the core of the theme.

[0125] T3: Arrange the character animations in time sequence, and use the time segmentation algorithm according to the special effects plot timing points to extract the corresponding character action frames;

[0126] T4: Decompose the character action frames into independent motion units based on the discrete element principle, assign mass and elastic coefficients according to the character type, and generate discrete trajectory data;

[0127] In this embodiment, the quality coefficient refers to the narrative weight of the character in the story; the elasticity coefficient represents the possibility of the character breaking the established destiny trajectory.

[0128] T5: Define the discrete element parameters of the special effect elements, calculate the contact quantization value between the character action frame and the special effect elements, and preset the connection threshold based on the character type and special effect type;

[0129] In this embodiment, the contact quantization value (CQ) refers to a composite indicator of the energy transfer efficiency and narrative impact weight generated when a character animation keyframe physically or symbolically interacts with a special effect element (particle / fluid / rigid body) within a specific time window. The calculation formula is:

[0130] CQ = (E_transfer × α) + (N_weight × β), where E_transfer (energy transfer rate) represents the percentage of change in the kinetic energy of the special effect element caused by the action frame, N_weight (narrative weight) represents the intensity of the impact of the special effect on the character's fate line (0-1 scale), and α and β represent the domain adjustment coefficients.

[0131] T6: If the timing of the contact quantization value and the connection threshold is different, the timestamps of the character action frames and / or special effects elements are corrected using the dynamic time warping algorithm based on the plot elements;

[0132] If the timing of the contact quantization value and the connection threshold is the same, the contact quantization value is compared with the preset connection threshold;

[0133] T7: If the contact quantization value is greater than or equal to the connection threshold, physical feedback is triggered; if the contact quantization value is less than the connection threshold, a smooth transition algorithm is used to compensate for the inertia of special effect elements and / or character action frames;

[0134] T8: Adaptively blur and sharpen the contact edges between the character action frames and special effect elements, and fuse the character action frames, special effect elements and three-dimensional stereo images to complete the virtual character driving and holographic image construction.

[0135] In this embodiment, the time series segmentation algorithm and the dynamic time warping algorithm are both commonly used technical means in this field and will not be described in detail here.

[0136] In the present invention, the workflow of the intelligent light source control module includes the following steps:

[0137] First, all light sources within the borderless virtual space are divided according to the spatial layout of the screen to obtain a light source array. The screen quality of all screens is then tested to obtain the screen resolution, refresh rate, response time, color gamut, brightness, contrast, color accuracy, and viewing angle.

[0138] Then, based on the emotional tone of the plot elements and the screen quality, the chromaticity, brightness and projection angle of the light source array are dynamically adjusted to compensate for the light source of the holographic theater. At the same time, a variational autoencoder is used to detect the color of the special effect elements based on the display effect to obtain a color distortion set.

[0139] The color distortion set is then parsed into the three primary colors of light, and combined with plot and background elements, the light source array is controlled to perform dynamic quantitative light compensation.

[0140] In this embodiment, the variational autoencoder is a commonly used technical means in this field and will not be described in detail here.

[0141] Emotional tone refers to the sustained emotional atmosphere created by a narrative through the integration of audiovisual symbols, textual rhythm, and character interaction. It serves as the emotional filter through which viewers and players perceive the core of the story. Essentially, it represents the creator's emotional encoding strategy for conveying information, directly influencing the audience's psychological engagement and the direction of their interpretation.

[0142] In the present invention, the adaptive interaction module first counts the audience in the holographic theater to obtain the audience distribution density; at the same time, it measures the light source array to obtain the theater lighting intensity; at the same time, it uses smart glasses or handheld devices to capture the audience's movements and behaviors to obtain the visual focus and gesture instructions; then, based on the audience distribution density, it analyzes the visual field and seating arrangement of each audience member to obtain the audience perspective diffusion map and the audience interaction limit map; finally, it uses a multi-resolution holographic model to adjust the presentation angle of the holographic image based on the visual focus and the audience perspective diffusion map; at the same time, it uses a plot interaction model to provide interactive feedback on the holographic image and trigger hidden plots based on gesture instructions and the audience interaction limit map;

[0143] The workflow of the adaptive interaction module also includes the following steps:

[0144] Based on the screen function, combined with the eye focus and gesture commands, a layered projection strategy is used to dynamically adjust each screen:

[0145] The distant environment of the background elements is presented on the dynamic media wall, and the close environment of the background elements is superimposed on the holographic interactive transparent screen;

[0146] The horizontal field of view in the plot elements is expanded on the tilted holographic film, and the special effects elements are demonstrated and simulated using a fog screen projection screen with a smoke machine and wind effect equipment, thereby constructing a borderless virtual space.

[0147] In this embodiment, the multi-resolution holographic model and the layered projection strategy are both commonly used technical means in this field and will not be described in detail here.

[0148] like Figure 4 As shown, the present invention also includes a holographic imaging method for VR holographic theater, which is applied to a holographic imaging system and includes the following steps in sequence:

[0149] A: The adaptive interaction module initializes and performs functional tests on all screens in the holographic theater to build a borderless virtual space;

[0150] B: After the screen is initialized, the holographic background module superimposes and projects 3D images based on the spatial layout of several screens, combining theatrical performance and holographic dynamic perception strategies;

[0151] C: The dynamic fusion module tracks the actor's movements and combines character and plot elements with the 3D image using a holographic fusion mechanism to construct a holographic image.

[0152] D: The intelligent light source control module dynamically adjusts the color, brightness, and projection angle of the light source array based on the screen quality of different screens, combined with plot elements and special effects elements;

[0153] E: Adaptive interaction module, based on the screen function combined with the audience's visual focus and gesture instructions, uses a layered projection strategy to dynamically adjust the screen's spatial layout and the plot content of the holographic image.

[0154] Example:

[0155] In the holographic imaging system, the adaptive interaction module dynamically adjusts each screen according to the screen function combined with the line of sight and gesture instructions, using a layered projection strategy: the distant environment of the background elements is presented on the dynamic media wall, the close environment of the background elements is superimposed on the holographic interactive transparent screen, the horizontal field of view of the plot elements is expanded on the tilted holographic film, and the special effects elements are demonstrated and simulated using a fog screen projection screen with a smoke machine and wind effect equipment, thereby constructing a borderless virtual space.

[0156] The holographic background module first detects the spatial layout and screen size of each screen to obtain screen data; then uses the digital twin algorithm to virtually construct the screen data to obtain a digital twin screen, and uses the knowledge graph to separate and associate background elements according to the deductive sequence and spatial orientation to obtain a background transformation map; then uses the holographic dynamic perception strategy in combination with plot elements to map the background transformation map to the digital twin screen to obtain a virtual background demonstration projection, and uses the ray tracing engine and discrete element method to perform edge enhancement and global deduction on the virtual demonstration projection to obtain the optimal background projection; finally, the optimal background projection is superimposed and projected onto the corresponding screen according to the spatial layout, and then a three-dimensional stereoscopic image is displayed on each screen.

[0157] The dynamic fusion module first uses a multi-camera array to shoot each actor from multiple angles to obtain several actor performance videos, and at the same time uses inertial sensors to capture each actor's action nodes to obtain several actor action sequence tables; then uses a spatial projection conversion algorithm to convert each actor's performance video into several role slices according to the performance role, and uses a semantic segmentation network to perform pixel-level classification on the role slices to obtain a role action timing diagram; then uses a generative adversarial network combined with a texture optimization algorithm and a multi-light source rendering algorithm to generate a virtual character based on the role slices; finally, the virtual character is combined with the role action timing diagram and the actor action sequence table to generate a character dynamic diagram based on the character elements, and at the same time, a holographic fusion mechanism is used to fuse the character dynamic diagram and / or the actor with the three-dimensional stereo image based on the plot elements and special effects elements, thereby completing the virtual character drive and holographic image construction.

[0158] The intelligent light source control module first divides all light sources in the borderless virtual space according to the spatial layout of the screen to obtain a light source array; it then tests the screen quality of all screens to obtain the screen resolution, refresh rate, response time, color gamut, brightness, contrast, color accuracy and viewing angle; then, based on the emotional tone of the plot elements and the screen quality, it dynamically adjusts the chromaticity, brightness and projection angle of the light source array to compensate for the light source of the holographic theater. At the same time, it uses a variational autoencoder to perform color detection on special effects elements based on the display effect to obtain a color distortion set; the color distortion set is then parsed into the three primary colors of light, and combined with plot elements and background elements, the light source array is controlled to perform dynamic quantitative light compensation.

[0159] After the performance, the adaptive interaction module first counts the audience in the holographic theater to obtain the audience distribution density; at the same time, it measures the light source array to obtain the theater lighting intensity; at the same time, it uses smart glasses or handheld devices to capture the audience's movements and behaviors to obtain the line of sight and gesture instructions; then, according to the audience distribution density, it analyzes the field of view and seating of each audience member to obtain the audience perspective diffusion map and the audience interaction limit map; finally, it uses the multi-resolution holographic model to adjust the presentation angle of the holographic image according to the line of sight focus and the audience perspective diffusion map; at the same time, it uses the plot interaction model to provide interactive feedback on the holographic image and trigger hidden plots according to gesture instructions and the audience interaction limit map.

[0160] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A holographic imaging system for VR holographic theater, characterized by: It includes holographic background module, dynamic fusion module, intelligent light source control module and adaptive interaction module; among them, The holographic background module is used to project three-dimensional images based on the spatial layout of several screens, combining theatrical interpretation and holographic dynamic perception strategies; The screens include dynamic media walls, fog screen projection screens, holographic interactive transparent screens and tilted holographic films; theatrical performances include background elements, character elements, plot elements and special effects elements; The dynamic fusion module is used to track the actor's movements, combine character elements and plot elements, and use the holographic fusion mechanism to fuse them with the three-dimensional image to obtain a holographic image; Intelligent light source control module, used to dynamically adjust the color, brightness and projection angle of the light source array according to the screen quality of different screens; Adaptive interaction module, which is used to dynamically adjust the spatial layout of the screen and the plot content of the holographic image based on the screen function, the audience's visual focus and gesture instructions, and the layered projection strategy; The workflow of the holographic fusion mechanism is as follows: T1: Special effect elements are divided into particle effects, rigid body effects, fluid effects, hair effects, cloth effects and soft body effects according to their expression forms; T2: Extract plot turning points from plot elements based on Sid Field's theory to obtain several special effects plot timing points; T3: Arrange the character animations in time sequence, and use the time segmentation algorithm according to the special effects plot timing points to extract the corresponding character action frames; T4: Decompose the character action frames into independent motion units based on the discrete element principle, assign mass and elastic coefficients according to the character type, and generate discrete trajectory data; T5: Define the discrete element parameters of the special effect elements, calculate the contact quantization value between the character action frame and the special effect elements, and preset the connection threshold based on the character type and special effect type; T6: If the timing of the contact quantization value and the connection threshold is different, the timestamps of the character action frames and / or special effects elements are corrected using the dynamic time warping algorithm based on the plot elements; If the timing of the contact quantization value and the connection threshold is the same, the contact quantization value is compared with the preset connection threshold; T7: If the contact quantization value is greater than or equal to the connection threshold, physical feedback is triggered; if the contact quantization value is less than the connection threshold, a smooth transition algorithm is used to compensate for the inertia of special effect elements and / or character action frames; T8: Adaptively blur and sharpen the contact edges between the character action frames and special effect elements, and fuse the character action frames, special effect elements and three-dimensional stereo images to complete the virtual character driving and holographic image construction.

2. The holographic imaging system for VR holographic theater according to claim 1, characterized in that: The workflow of the holographic background module includes the following steps: The spatial layout and screen size of each screen are detected to obtain screen data; then the screen data is virtually constructed to obtain a digital twin screen; The background elements are separated and associated according to the deductive time sequence and spatial orientation to obtain the background transformation map; By using holographic dynamic perception strategies combined with plot elements, the background transformation map is mapped to the digital twin screen to obtain a virtual background demonstration projection. The virtual demonstration projection is then edge-enhanced and globally deduced to obtain the optimal background projection. The optimal background projection is superimposed and projected onto the corresponding screen according to the spatial layout, and then a three-dimensional stereoscopic image is displayed on each screen.

3. The holographic imaging system for VR holographic theater according to claim 1, characterized in that: The workflow of the holographic dynamic perception strategy includes the following steps: S1: Extract plot elements according to time sequence to obtain several image time series frames; S2: Match and fuse the image time series frames with the background transformation map to obtain the scene fusion map; S3: Classify the scene fusion atlas according to spatial orientation, obtain an orientation correlation time series table containing several image time series frames, and extract the projection offset angle at the same time; S4: Compare and judge the projection offset angle with the preset azimuth conversion range; If the projection offset angle is greater than the upper limit of the azimuth conversion range or the projection offset angle is less than the lower limit of the azimuth conversion range, the digital twin screen is rotated until the projection offset angles of all image timing frames in the azimuth association timing table are within the azimuth conversion range; If the projection offset angles are all within the azimuth conversion range, the display effects of the image time series frames on the digital twin screen are compared and corrected.

4. The holographic imaging system for VR holographic theater according to claim 3, characterized in that: The workflow of the holographic dynamic perception strategy is also The following steps are involved: S5: If screen tearing occurs on the digital twin screen, the inter-frame delay is compensated and the continuous background transformation trajectory is reconstructed; If color shift occurs on the digital twin screen, the event nodes of the plot elements are analyzed to match the color tone and update the global illumination of the background; If a dynamic blur mismatch occurs in the digital twin screen, the pixel motion vectors between adjacent frames are calculated and the edges of the image time sequence frames are enhanced; S6: Virtually render the image time-series frames that have completed the contrast correction to obtain the virtual background demonstration projection of each digital twin screen; S7: Global inference of virtual background demonstration projections with the same timestamp: The pixel peak signal-to-noise ratio and pixel structure similarity of the overlapping area between digital twin screens are compared with the signal-to-noise threshold and structure threshold respectively; S8: If either the pixel peak signal-to-noise ratio or the pixel structure similarity is less than the signal-to-noise threshold and the structure threshold, it is determined that the timing between the digital twin screens is inconsistent, and the display effect of the image timing frame on the digital twin screen is corrected again; If the pixel peak signal-to-noise ratio and pixel structure similarity are both greater than or equal to the signal-to-noise threshold and structure threshold, the timing between the digital twin screens is determined to be consistent, and the color error and color difference threshold of the overlapping area are judged; S9: If the color error is greater than or equal to the color difference threshold, gamma linearization is performed on the overlapping area or color correction is performed on the digital twin screen; If the color error is less than the color difference threshold, the color of the overlapping area is determined to be normal, and the optimal background projection is obtained.

5. The holographic imaging system for VR holographic theater according to claim 1, characterized in that: The workflow of the dynamic fusion module includes the following steps: Each actor is filmed from multiple angles to obtain several performance videos. At the same time, each actor's action nodes are captured to obtain several actor action sequence lists. The performance videos of each actor are converted into several character slices according to the role they play, and the character slices are classified at the pixel level to obtain the character action timing diagram, and a virtual character is generated based on the character slices; Combine the virtual character with the character action timing diagram and the actor action sequence table to generate the character dynamic diagram based on the character elements; According to the plot elements and special effects elements, the holographic fusion mechanism is used to integrate the character dynamic graphics and / or actors with the three-dimensional stereoscopic images, thereby completing the virtual character driving and holographic image construction.

6. The holographic imaging system for VR holographic theater according to claim 1, characterized in that: The workflow of the intelligent light source control module includes the following steps: Divide all light sources in the borderless virtual space according to the spatial layout of the screen to obtain a light source array; Test the screen quality of all screens to obtain the screen resolution, refresh rate, response time, color gamut, brightness, contrast, color accuracy and viewing angle; According to the emotional tone of the plot elements and the screen quality, the color, brightness and projection angle of the light source array are dynamically adjusted to compensate for the light source of the holographic theater. Perform color detection on special effect elements according to the display effect to obtain a color distortion set; The color distortion set is analyzed into the three primary colors of light, and combined with plot elements and background elements, the light source array is controlled to perform dynamic quantitative light compensation.

7. The holographic imaging system for VR holographic theater according to claim 1, characterized in that: The workflow of the adaptive interaction module includes the following steps: Count the audience in the holographic theater to get the audience distribution density; measure the light source array to get the theater lighting intensity; and capture the audience's movements to get the focus of vision and gesture instructions; Analyze each audience member's field of view and seating arrangement based on audience distribution density to obtain audience perspective diffusion maps and audience interaction limit maps; Adjust the presentation angle of the holographic image based on the sight focus and the audience's perspective diffusion map; According to gesture commands and the audience interaction limit map, interactive feedback on the holographic image and hidden plot triggering are carried out.

8. The holographic imaging system for VR holographic theater according to claim 1, characterized in that: The workflow of the adaptive interaction module further includes the following steps: Based on the screen function, combined with the eye focus and gesture instructions, a layered projection strategy is used to dynamically adjust each screen: The distant environment of the background elements is presented on the dynamic media wall, and the close environment of the background elements is superimposed on the holographic interactive transparent screen. The horizontal field of view in the plot elements is expanded on the tilted holographic film, and the special effects elements are demonstrated and simulated using a fog screen projection screen with a smoke machine and wind effect equipment, thereby constructing a borderless virtual space.

9. A holographic imaging method for a VR holographic theater, applied to the holographic imaging system according to any one of claims 1 to 8, characterized in that: The following steps are included in sequence: A: The adaptive interaction module initializes and performs functional tests on all screens in the holographic theater to build a borderless virtual space; B: After the screen is initialized, the holographic background module superimposes and projects 3D images based on the spatial layout of several screens, combining theatrical performance and holographic dynamic perception strategies; C: The dynamic fusion module tracks the actor's movements and combines character and plot elements with the 3D image using a holographic fusion mechanism to construct a holographic image. D: The intelligent light source control module dynamically adjusts the color, brightness, and projection angle of the light source array based on the screen quality of different screens, combined with plot elements and special effects elements; E: Adaptive interaction module, which is used to dynamically adjust the spatial layout of the screen and the plot content of the holographic image based on the screen function, the audience's visual focus and gesture instructions, and the layered projection strategy.

Images