Holographic imaging system and method for VR holographic theater

Abstract

The invention discloses a holographic imaging system and method for a VR holographic theater, relates to the technical field of holographic imaging, and solves the problem that traditional holographic imaging depends on a fixed optical compensation device and is difficult to deal with color distortion caused by ambient light intensity change. The holographic imaging system comprises a holographic background module, a dynamic fusion module, an intelligent light source control module and a self-adaptive interaction module. A three-dimensional stereoscopic image is projected in a superposed manner through a holographic background module by utilizing a holographic dynamic sensing strategy according to the spatial layout of a plurality of screens in combination with theater deduction; tracking actions of actors by utilizing a holographic fusion mechanism through a dynamic fusion module, and fusing character elements and plot elements with the three-dimensional image to obtain a holographic image; the multi-screen collaborative display effect of the holographic image on a plurality of screens is improved, and the color distortion probability of the holographic image is reduced.

Description

Technical Field

[0001] The present invention relates to the field of holographic imaging technology, and particularly to a holographic imaging system and method for a VR holographic theater. Background Art

[0002] With the deep integration of virtual reality (VR) technology and holographic imaging technology, VR holographic theaters have gradually become an important carrier for immersive entertainment, education, and industrial simulation. However, traditional holographic theaters face significant technical bottlenecks in achieving precise synchronization of scenes and performances. Early holographic imaging systems mostly relied on manual phased control, independently operating display media such as LED dynamic media walls and projection screens through preset scripts. This mode has two core problems: First, manual intervention results in a synchronization error rate of up to 3%-5% between scene switching and actor movements. Especially in dynamic scenes (such as rapid scene transitions or complex interactions), latency can disrupt the immersion. Second, the collaborative control of multi-modal display media (such as holographic interactive transparent screens, lifting screens) lacks a dynamic optimization mechanism, and it is difficult to match the brightness and transparency parameters of different media to the scene requirements in real time, leading to a chaotic visual hierarchy.

[0003] Patent No. CN202411463277.8 discloses a VR multi-terminal collaborative interaction method, system, computer device, and storage medium, including: displaying VR scene images through a VR holographic imaging screen; when an interaction scenario appears after a set time value, linearly displaying a number of interaction selection pop-up windows in the lower edge area; receiving interaction commands input by a number of interaction control terminals, with interaction control terminals installed on the front row seats of the VR holographic theater, and each interaction selection pop-up window corresponding to a front row seat; performing fusion processing on the interaction commands to obtain fusion data, and determining the plot scene of the corresponding interaction scenario according to the fusion data; when the VR scene image playback is completed, generating a VR program list selection pop-up window, receiving a VR program selection instruction input by the interaction control terminal, and determining the next VR program to be played. In this way, more connections are established between the plot and the audience, and the audience's sense of participation will be stronger.

[0004] Patent No. CN202411577875.8 discloses a holographic imaging system and method for a VR holographic theater. In the embodiments of the present invention, based on a built-in design resource pool, content editing operations are received and three-dimensional scene rendering is performed to generate VR scene data; preview acquisition is carried out to obtain preview acquisition data; synchronous segmentation is performed to obtain multiple-stage transition features; performance action capture and recognition are carried out to synchronize the current scene data; and holographic imaging control is performed on the LED main screen, holographic lifting projection screen, multiple front and rear wing-shaped transparent screens, and multiple LED extended screens. It is possible to synchronously segment the VR scene data and the preview acquisition data to obtain multiple-stage transition features. During the formal performance of the VR holographic theater, performance action capture and recognition are carried out, the current scene data is synchronized, and corresponding holographic imaging control is performed to avoid errors in synchronous control and improve the viewing experience of the VR holographic theater.

[0005] The education field has put forward higher requirements for molecular-level three-dimensional dynamic display (such as the chemical reaction process), and industrial simulation has put forward higher requirements for the accuracy of virtual-real fusion (such as the ±0.05mm error tolerance for spacecraft assembly). However, the existing systems are limited by static scene rendering engines, unable to achieve seamless switching between microscopic and macroscopic scenes, and lacking the ability to simulate light and shadow according to physical laws. In addition, traditional holographic imaging relies on fixed optical compensation devices, making it difficult to cope with the color distortion problem caused by changes in ambient light intensity. Especially under the refractive index deviation of the holographic interactive transparent screen, image distortion will occur when observing from multiple angles. Summary of the Invention

[0006] The purpose of the present invention is to provide a holographic imaging system and method for a VR holographic theater, which can improve the multi-screen collaborative display effect of holographic images on several screens and reduce the probability of color distortion of holographic images through a holographic dynamic perception strategy and a holographic fusion mechanism combined with the action capture of several screens and actors.

[0007] The present invention utilizes the following technical solutions: A holographic imaging system for a VR holographic theater, including a holographic background module, a dynamic fusion module, an intelligent light source control module, and an adaptive interaction module; wherein, The holographic background module is used to project three-dimensional stereoscopic images in a borderless virtual space according to the spatial layout of several screens, combined with theater deduction and a holographic dynamic perception strategy; The screens include a dynamic media wall, a fog screen projection screen, a holographic interactive transparent screen, and an inclined holographic film; the theater deduction includes background elements, character elements, plot elements, and special effect elements; The dynamic fusion module is used to track the actions of actors, and combine character elements and plot elements to fuse with the three-dimensional stereoscopic images using a holographic fusion mechanism to obtain holographic images; An intelligent light source control module, which is used to dynamically adjust the chromaticity, brightness and projection angle of the light source array according to the screen quality of different screens; An adaptive interaction module, which is used to perceive the focus of the audience's line of sight and gesture instructions, and adjust the presentation angle and plot content of the holographic image.

[0008] Preferably, the working process of the holographic background module includes the following steps: First, detect the spatial layout and screen size of each screen to obtain screen data; Then, use the digital twin algorithm to virtually construct the screen data to obtain a digital twin screen. At the same time, use the knowledge graph to separate and associate the background elements according to the deductive time sequence and in combination with the spatial orientation to obtain a background transformation map; Subsequently, use the holographic dynamic perception strategy in combination with the plot elements to map the background transformation map to the digital twin screen to obtain a virtual background demonstration projection, and use the ray tracing engine and the discrete element method to perform edge enhancement and global derivation on the virtual demonstration projection to obtain an optimal background projection; Finally, superimpose and project the optimal background projection onto the corresponding screen according to the spatial layout, and then display three-dimensional stereoscopic images on each screen.

[0009] Preferably, the working process of the holographic dynamic perception strategy includes the following steps: S1: Extract the plot elements using the information frame extraction algorithm according to the time sequence to obtain a number of image time sequence frames; S2: Use the information matching algorithm to match and fuse the image time sequence frames with the background transformation map to obtain a scene fusion map; S3: Classify the scene fusion map according to the spatial orientation to obtain an azimuth correlation time sequence table containing a number of image time sequence frames, and at the same time extract the projection offset angle; S4: Compare and judge the projection offset angle with the preset azimuth conversion range; If the projection offset angle is greater than the upper limit of the azimuth conversion range or the projection offset angle is less than the lower limit of the azimuth conversion range, rotate the digital twin screen until the projection offset angles of all the image time sequence frames in the azimuth correlation time sequence table are within the azimuth conversion range; If the projection offset angles are all within the azimuth conversion range, compare and correct the display effect of the image time sequence frames on the digital twin screen.

[0010] Preferably, the working process of the holographic dynamic perception strategy further includes the following steps: S5: If image tearing occurs in the digital twin screen, use the timestamp interpolation algorithm to compensate for the frame delay and reconstruct a continuous background transformation trajectory; If there is a color offset in the digital twin screen, the event nodes of the plot elements are parsed using a natural language network to match the color tone, and then the global illumination of the background is updated; If there is a dynamic blur mismatch in the digital twin screen, the optical flow method is used to calculate the pixel motion vectors between adjacent frames, and the edge enhancement of the image time series frames is performed by combining the super-resolution reconstruction algorithm and the temporal anti-aliasing method; S6: The image time series frames after comparison and correction are virtually rendered using a 3D rendering algorithm to obtain the virtual background demonstration projections of each digital twin screen; S7: The virtual background demonstration projections with the same timestamp are globally derived using the discrete element method: the peak signal-to-noise ratio and pixel structural similarity of the overlapping area between the digital twin screens are respectively compared with the signal-to-noise threshold and the structural threshold; S8: If either the peak signal-to-noise ratio or the pixel structural similarity is less than the signal-to-noise threshold and the structural threshold, it is determined that the time series between the digital twin screens is inconsistent, and the display effect of the image time series frames on the digital twin screens is corrected again; If both the peak signal-to-noise ratio and the pixel structural similarity are greater than or equal to the signal-to-noise threshold and the structural threshold, it is determined that the time series between the digital twin screens is consistent, and the color error in the overlapping area is judged against the color difference threshold; S9: If the color error is greater than or equal to the color difference threshold, linearize the gamma value for the overlapping area or correct the color of the digital twin screen; If the color error is less than the color difference threshold, it is determined that the color in the overlapping area is normal, and then the optimal background projection is obtained.

[0011] Preferably, the workflow of the holographic fusion mechanism is as follows: T1: The special effect elements are divided into particle special effects, rigid body special effects, fluid special effects, hair special effects, cloth special effects and soft body special effects according to the manifestation form; T2: The plot turning nodes in the plot elements are extracted according to the Syd Field theory to obtain several special effect plot time series points; T3: The character dynamic images are arranged in a time series, and the corresponding character action frames are extracted using the time series segmentation algorithm according to the special effect plot time series points; T4: The character action frames are disassembled into independent motion units according to the discrete element principle, and mass and elastic coefficients are assigned according to the character type to generate discrete trajectory data; T5: Define the discrete element parameters of the special effect elements, and at the same time measure the contact quantization value between the character action frames and the special effect elements, and preset the connection threshold according to the character type and the special effect type; T6: If the timings of the contact quantization value and the connection threshold are different, correct the timestamps of the character action frames and / or the special effect elements using the dynamic time warping algorithm according to the plot elements; If the timing of the contact quantization value is the same as the connection threshold, compare the contact quantization value with the preset connection threshold; T7: If the contact quantization value is greater than or equal to the connection threshold, trigger physical feedback; if the contact quantization value is less than the connection threshold, use a smooth transition algorithm to perform inertial compensation on the special effect elements and / or character action frames; T8: Perform adaptive blurring-sharpening on the contact edges of the character action frames and the special effect elements, and at the same time fuse the character action frames, the special effect elements, and the three-dimensional stereoscopic images, thereby completing virtual character driving and holographic image construction.

[0012] Preferably, the working process of the intelligent light source control module includes the following steps: First, divide all the light sources in the borderless virtual space according to the spatial layout of the screen to obtain a light source array; and detect the screen quality of all the screens to obtain the resolution, refresh rate, response time, color gamut, brightness, contrast, color accuracy, and viewing angle of the screens; Then, dynamically adjust the chromaticity, brightness, and projection angle of the light source array according to the emotional tone in the plot elements combined with the screen quality, thereby performing light source compensation on the holographic theater. At the same time, use a variational autoencoder to perform color detection on the special effect elements according to the display effect to obtain a color distortion set; Subsequently, analyze the color distortion set into the three primary colors of light, and combine the plot elements and the background elements to control the light source array to perform dynamic quantization light compensation.

[0013] Preferably, the working process of the adaptive interaction module includes the following steps: First, count the audience in the holographic theater to obtain the audience distribution density; at the same time, measure the light source array to obtain the theater illumination intensity; At the same time, use smart glasses or handheld devices to capture the action behaviors of the audience to obtain the line of sight focus and gesture commands; Then, analyze the field of view and seating order of each audience according to the audience distribution density to obtain an audience perspective diffusion map and an audience interaction limit map; Finally, use a multi-resolution holographic model to adjust the presentation angle of the holographic image according to the line of sight focus, combined with the audience perspective diffusion map; at the same time, use a plot interaction model to perform interaction feedback and hidden plot triggering on the holographic image according to the gesture commands, in cooperation with the audience interaction limit map.

[0014] Preferably, the working process of the adaptive interaction module further includes the following steps: Dynamically adjust each screen according to the screen function combined with the line of sight focus and gesture commands, using a hierarchical projection strategy: Present the distant view environment in the background elements on the dynamic media wall, and superimpose the near view environment in the background elements on the holographic interactive transparent screen; Expand the horizontal field of view in the plot elements on the inclined holographic film, and use the fog screen projection screen in cooperation with the smoke machine and wind effect equipment to demonstrate and simulate the special effect elements, thereby constructing a borderless virtual space.

[0015] A holographic imaging method for a VR holographic theater, applied to a holographic imaging system, includes the following steps: A: The adaptive interaction module initializes and performs function detection on all the screens in the holographic theater to construct a borderless virtual space; B: After the screens are initialized, the holographic background module superimposes and projects three-dimensional stereoscopic images according to the spatial layout of several screens, in combination with theater performances and holographic dynamic perception strategies; C: The dynamic fusion module tracks the actions of the actors, and at the same time combines the character elements and plot elements to fuse with the three-dimensional stereoscopic images using the holographic fusion mechanism to construct holographic images; D: The intelligent light source control module dynamically adjusts the chromaticity, brightness, and projection angle of the light source array according to the screen quality of different screens, in combination with the plot elements and special effect elements; E: The adaptive interaction module is used to dynamically adjust the spatial layout of the screen and the plot content of the holographic image according to the screen function, combined with the viewer's line of sight focus and gesture commands, using the hierarchical projection strategy.

[0016] In the present invention, the holographic background module uses the holographic dynamic perception strategy to superimpose and project three-dimensional stereoscopic images according to the spatial layout of several screens, in combination with theater performances; the dynamic fusion module uses the holographic fusion mechanism to track the actions of the actors, and combines the character elements and plot elements to fuse with the three-dimensional stereoscopic images to obtain holographic images; improving the multi-screen collaborative display effect of the holographic images on several screens and reducing the probability of color distortion of the holographic images. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following will briefly introduce the drawings required to be used in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0018] Figure 1 It is a schematic block diagram of the holographic imaging system; Figure 2 It is a flow chart of the holographic dynamic perception strategy; Figure 3 It is a flow chart of the holographic fusion mechanism; Figure 4 It is a flowchart of a holographic imaging method. Specific implementation manners

[0019] The present invention will be described in detail below in conjunction with the accompanying drawings and embodiments: As Figures 1 to 3 shown, a holographic imaging system for a VR holographic theater according to the present invention includes a holographic background module, a dynamic fusion module, an intelligent light source control module, and an adaptive interaction module; wherein, The holographic background module is used to project three-dimensional stereoscopic images in a borderless virtual space according to the spatial layout of several screens, in combination with theater performances and holographic dynamic perception strategies; In the present invention, the screens include a dynamic media wall, a fog screen projection screen, a holographic interactive transparent screen, and an inclined holographic film; the theater performances include background elements, character elements, plot elements, and special effect elements; In this embodiment, the background elements refer to the static or dynamic environment that constitutes the scene, including the spatial layout, scene levels (foreground, middle ground, background), and lighting conditions, and are used to construct the immersion of the virtual world; The character elements cover the appearance, actions, and interaction behaviors of the characters, and are the core carriers for promoting the narrative and connecting with the user's emotions; The plot elements refer to the story structure, event logic, and character relationship design, which determine the narrative coherence of the virtual world and the user's participation; The special effect elements include visual enhancement technologies such as lighting, particle effects, and material reflections, and are used to enhance the atmosphere or represent surreal scenes.

[0020] The dynamic fusion module is used to track the actions of the actors, and combine the character elements and plot elements to fuse with the three-dimensional stereoscopic images using a holographic fusion mechanism to obtain a holographic image; The intelligent light source control module is used to dynamically adjust the chromaticity, brightness, and projection angle of the light source array according to the screen quality of different screens; The adaptive interaction module is used to sense the focus of the audience's line of sight and gesture commands, and adjust the presentation angle and plot content of the holographic image.

[0021] In the present invention, the working process of the holographic background module includes the following steps: First, detect the spatial layout and screen size of each screen to obtain screen data; Then, use the digital twin algorithm to virtually construct the screen data to obtain a digital twin screen, and at the same time use the knowledge graph to separate and associate the background elements according to the performance timing and in combination with the spatial orientation to obtain a background transformation graph; Subsequently, using the holographic dynamic perception strategy combined with plot elements, the background transformation map is mapped onto the digital twin screen to obtain a virtual background demonstration projection. Then, the ray tracing engine and the discrete element method are used to perform edge enhancement and global derivation on the virtual demonstration projection to obtain the optimal background projection. Finally, the optimal background projection is superimposed and projected onto the corresponding screen according to the spatial layout, and then three-dimensional stereoscopic images are displayed on each screen.

[0022] In this embodiment, the digital twin algorithm, knowledge graph, ray tracing engine, and discrete element method are all common technical means in the art and will not be elaborated here.

[0023] The deductive time sequence refers to the time sequence in which the plot appears in the work and is the time line arrangement strategy of the creator for the story materials.

[0024] Edge enhancement refers to a technology that enhances the clarity of the object contour in an image or scene. Its core logic is to strengthen the contrast or sharpness of the edge region through an algorithm, making the key structures (such as object boundaries, text, symbols, etc.) in the projection content more easily recognizable.

[0025] Global derivation refers to analyzing the overall spatio-temporal relationship of the projection content to predict or generate reasonable information for missing / blurred regions and maintaining the coherence and authenticity of the scene.

[0026] In the present invention, the workflow of the holographic dynamic perception strategy includes the following steps: S1: Extract the plot elements using the information frame extraction algorithm according to the time sequence to obtain a number of image time sequence frames. S2: Use the information matching algorithm to match and fuse the image time sequence frames with the background transformation map to obtain a scene fusion map. S3: Classify the scene fusion map according to the spatial orientation to obtain an orientation-related time sequence table containing a number of image time sequence frames, and at the same time extract the projection offset angle. S4: Compare and judge the projection offset angle with the preset azimuth conversion range. If the projection offset angle is greater than the upper limit value of the azimuth conversion range or the projection offset angle is less than the lower limit value of the azimuth conversion range, rotate the digital twin screen until the projection offset angles of all the image time sequence frames in the orientation-related time sequence table are within the azimuth conversion range. If the projection offset angles are all within the azimuth conversion range, compare and correct the display effect of the image time sequence frames on the digital twin screen. S5: If image tearing occurs on the digital twin screen, use the timestamp interpolation algorithm to compensate for the frame delay and reconstruct the continuous background transformation trajectory. If color deviation occurs in the digital twin screen, the event nodes of the plot elements are parsed using the natural language network to match the color tone, and then the global illumination of the background is updated; If dynamic blur mismatch occurs in the digital twin screen, the optical flow method is used to calculate the pixel motion vectors between adjacent frames, and the edge enhancement of the image time series frames is performed by combining the super-resolution reconstruction algorithm and the temporal anti-aliasing method; S6: The image time series frames after comparison and correction are virtually rendered using the three-dimensional rendering algorithm to obtain the virtual background demonstration projections of each digital twin screen; S7: The virtual background demonstration projections with the same time stamp are globally derived using the discrete element method: the peak signal-to-noise ratio and pixel structural similarity of the overlapping area between the digital twin screens are respectively compared with the signal-to-noise threshold and the structural threshold; S8: If either the peak signal-to-noise ratio or the pixel structural similarity is less than the signal-to-noise threshold and the structural threshold, it is determined that the time series between the digital twin screens is inconsistent, and the display effect of the image time series frames in the digital twin screens is corrected again; If both the peak signal-to-noise ratio and the pixel structural similarity are greater than or equal to the signal-to-noise threshold and the structural threshold, it is determined that the time series between the digital twin screens is consistent, and the color error and color difference threshold of the overlapping area are judged; S9: If the color error is greater than or equal to the color difference threshold, gamma value linearization is performed on the overlapping area or color correction is performed on the digital twin screen; If the color error is less than the color difference threshold, it is determined that the color of the overlapping area is normal, and then the optimal background projection is obtained.

[0027] In this embodiment, the information frame extraction algorithm, the information matching algorithm, the natural language network, the time stamp interpolation algorithm, the optical flow method, the super-resolution reconstruction algorithm, the temporal anti-aliasing method, and the three-dimensional rendering algorithm are all commonly used technical means in the art and will not be elaborated here.

[0028] In the present invention, the working process of the dynamic fusion module includes the following steps: First, a multi-camera array is used to take multi-angle shots of each actor to obtain a number of actor performance videos, and at the same time, an inertial sensor is used to capture the action nodes of each actor to obtain a number of actor action sequence lists; Then, the spatial projection conversion algorithm is used to convert each actor performance video into a number of role slices according to the performance role, and the semantic segmentation network is used to perform pixel-level classification on the role slices to obtain the role action time series diagram; Subsequently, the generative adversarial network is combined with the texture optimization algorithm and the multi-light source rendering algorithm to generate virtual characters according to the role slices; Finally, the virtual character is combined with the character action timing diagram and the actor action sequence list, and the character dynamic diagram is generated according to the character elements. At the same time, according to the plot elements and special effect elements, the holographic fusion mechanism is used to fuse the character dynamic diagram and / or the actor with the three-dimensional stereoscopic image, thereby completing the virtual character driving and holographic image construction.

[0029] In this embodiment, the multi-camera array, inertial sensor, spatial projection conversion algorithm, semantic segmentation network, generative adversarial network, texture optimization algorithm, and multi-light source rendering algorithm are all commonly used technical means in the art and will not be elaborated here.

[0030] In the present invention, the working process of the holographic fusion mechanism is as follows: T1: The special effect elements are divided into particle special effects, rigid body special effects, fluid special effects, hair special effects, cloth special effects, and soft body special effects according to the manifestation form; In this embodiment, for particle special effects: from sandstorms to rain, and then to various light effects, particle special effects bring a rich visual experience to the film. Through particle substitution, spectacular effects such as flying swords and snowflakes floating in the sky can be created; Rigid body fragmentation: When glass breaks or a wall collapses, the rigid body fragmentation special effect breaks the whole object into multiple small objects, adding a sense of reality to the film; Fluid special effects: For floods, tsunamis, or the flowing water effect in an advertisement, fluid special effects make the water element more vivid; Hair special effects: Through the hair special effect system, it is easy to create furry effects such as hair and grass, saving the trouble of model making; Cloth special effects: For the animated effects of fluttering ribbons and flowing cloth, the cloth special effect system makes animation production easier; Soft body special effects: For the animated effects of elastic and soft objects, through the soft body special effect system, natural and smooth animations can be easily achieved.

[0031] T2: Extract the plot turning nodes in the plot elements according to Syd Field's theory to obtain several special effect plot timing points; In this embodiment, Syd Field's screenwriting theory system takes the "three-act structure" as the core, combines narrative diagnosis methods and character creation principles, and lays the foundation for modern film screenwriting.

[0032] The plot turning node refers to the key event or information that breaks the linear development of the plot in the narrative. By means of time / space jumps, perspective switches, causal reversals, etc., the story logic chain is reconstructed, forcing the audience to recalibrate their understanding of the character's fate or the theme core.

[0033] T3: Arrange the character dynamic diagrams in time series, and use the timing segmentation algorithm according to the special effect plot timing points to extract the corresponding character action frames; T4: Decompose the character action frames into independent motion units according to the discrete element principle, assign mass and elastic coefficients according to the character type, and generate discrete trajectory data; In this embodiment, the mass coefficient refers to the narrative weight of the character in the story; the elastic coefficient characterizes the possibility of the character breaking the established fate trajectory.

[0034] T5: Define the discrete element parameters of the special effect elements, measure the contact quantization value between the character action frames and the special effect elements, and preset the connection threshold according to the character type and the special effect type; In this embodiment, the contact quantization value CQ refers to a composite index of the energy transfer efficiency and the narrative influence weight when the key frames of the character animation and the special effect elements (particles / fluids / rigid bodies) have physical or symbolic interactions within a specific time window. The measurement formula is: CQ = (E_transfer × α) + (N_weight × β), where E_transfer (energy transfer rate) represents the percentage of the kinetic energy change of the special effect elements by the action frames, N_weight (narrative weight) represents the influence intensity of the special effects on the character's fate line (0-1 scale), and α, β represent the domain adjustment coefficients.

[0035] T6: If the time sequence of the contact quantization value is different from the connection threshold, correct the timestamps of the character action frames and / or the special effect elements according to the plot elements using the dynamic time warping algorithm; If the time sequence of the contact quantization value is the same as the connection threshold, compare the contact quantization value with the preset connection threshold; T7: If the contact quantization value is greater than or equal to the connection threshold, trigger physical feedback; if the contact quantization value is less than the connection threshold, perform inertial compensation on the special effect elements and / or the character action frames using the smooth transition algorithm; T8: Perform adaptive blur-sharpening on the contact edges between the character action frames and the special effect elements, and fuse the character action frames, the special effect elements, and the three-dimensional stereoscopic images, thereby completing the virtual character drive and the holographic image construction.

[0036] In this embodiment, the time sequence segmentation algorithm and the dynamic time warping algorithm are both commonly used technical means in the art and will not be elaborated here.

[0037] In the present invention, the working process of the intelligent light source control module includes the following steps: First, divide all the light sources in the borderless virtual space according to the spatial layout of the screen to obtain a light source array; and detect the screen quality of all the screens to obtain the resolution, refresh rate, response time, color gamut, brightness, contrast, color accuracy, and viewing angle of the screens; Then, according to the emotional tone in the plot elements and combined with the screen quality, dynamically adjust the chromaticity, brightness, and projection angle of the light source array, thereby compensating the light source of the holographic theater. At the same time, use the variational autoencoder to detect the colors of the special effect elements according to the display effect, and obtain the color distortion set; Subsequently, parse the color distortion set into the three primary colors of light, and combine the plot elements and background elements to control the light source array to perform dynamic quantization of light compensation.

[0038] In this embodiment, the variational autoencoder is a commonly used technical means in the art and will not be elaborated here.

[0039] The emotional tone refers to the continuous emotional atmosphere created by narrative works through integrating audiovisual symbols, text rhythm, and character interactions. It is the emotional filter for the audience / player to perceive the core of the story. Its essence is the emotional encoding strategy of the creator for information transmission, which directly affects the psychological participation degree of the audience and the direction of meaning interpretation.

[0040] In the present invention, the adaptive interaction module first counts the audience in the holographic theater to obtain the audience distribution density; at the same time, measures the light source array to obtain the theater illumination intensity; at the same time, uses smart glasses or handheld devices to capture the action behaviors of the audience to obtain the line of sight focus and gesture commands; then analyzes the field of view and seat number of each audience according to the audience distribution density to obtain the audience perspective diffusion map and the audience interaction limit map; finally, uses the multi-resolution holographic model to adjust the presentation angle of the holographic image according to the line of sight focus, in combination with the audience perspective diffusion map; at the same time, uses the plot interaction model to perform interactive feedback on the holographic image and trigger hidden plots according to the gesture commands, in coordination with the audience interaction limit map; The working process of the adaptive interaction module further includes the following steps: According to the screen function, combined with the line of sight focus and gesture commands, dynamically adjust each screen using the hierarchical projection strategy: Present the distant view environment in the background elements on the dynamic media wall, and superimpose the near view environment in the background elements on the holographic interactive transparent screen; Expand the horizontal field of view in the plot elements on the inclined holographic film, and demonstrate and simulate the special effect elements using the fog screen projection screen in cooperation with the smoke machine and wind effect equipment, thereby constructing a borderless virtual space.

[0041] In this embodiment, the multi-resolution holographic model and the hierarchical projection strategy are both commonly used technical means in the art and will not be elaborated here.

[0042] As Figure 4 shown, the present invention also includes a holographic imaging method for a VR holographic theater, which is applied to a holographic imaging system and sequentially includes the following steps: A: The adaptive interaction module initializes and performs functional detection on all the screens in the holographic theater to construct a borderless virtual space; B: After the screens are initialized, the holographic background module projects three-dimensional stereoscopic images by superimposing them according to the spatial layout of several screens, in combination with theater performances and holographic dynamic perception strategies; C: The dynamic fusion module tracks the movements of the actors and, at the same time, combines human and plot elements to fuse with the three-dimensional stereoscopic images using a holographic fusion mechanism to construct holographic images; D: The intelligent light source control module dynamically adjusts the chromaticity, brightness, and projection angle of the light source array according to the screen quality of different screens, in combination with plot elements and special effect elements; E: The adaptive interaction module, based on the screen functions, combines the line-of-sight focus and gesture commands of the audience, and dynamically adjusts the spatial layout of the screens and the plot content of the holographic images using a hierarchical projection strategy.

[0043] Example: In the holographic imaging system, the adaptive interaction module dynamically adjusts each screen according to the screen functions, combining the line-of-sight focus and gesture commands, using a hierarchical projection strategy: presenting the distant view environment in the background elements on the dynamic media wall, superimposing the near-view environment in the background elements on the holographic interactive transparent screen, expanding the horizontal field of view in the plot elements on the inclined holographic film, and demonstrating and simulating the special effect elements using the fog screen projection screen in cooperation with the smoke machine and wind effect equipment, thereby constructing a borderless virtual space.

[0044] The holographic background module first detects the spatial layout and screen size of each screen to obtain screen data; then uses the digital twin algorithm to virtually construct the screen data to obtain digital twin screens, and at the same time uses the knowledge graph to separate and associate the background elements according to the performance timing, in combination with the spatial orientation to obtain the background transformation graph; subsequently, uses the holographic dynamic perception strategy to combine with the plot elements to map the background transformation graph to the digital twin screens to obtain virtual background demonstration projections, and uses the ray tracing engine and the discrete element method to perform edge enhancement and global derivation on the virtual demonstration projections to obtain the optimal background projections; finally, superimposes and projects the optimal background projections onto the corresponding screens according to the spatial layout, and then displays three-dimensional stereoscopic images on each screen.

[0045] The dynamic fusion module first uses a multi-camera array to capture each actor from multiple angles, obtaining several actor performance videos. At the same time, it uses inertial sensors to capture the motion nodes of each actor, obtaining several actor motion sequence lists. Then, it uses the spatial projection conversion algorithm to convert each actor performance video into several role slices according to the performing role, and uses a semantic segmentation network to perform pixel-level classification on the role slices, obtaining a role action time series graph. Subsequently, it uses a generative adversarial network combined with a texture optimization algorithm and a multi-light source rendering algorithm to generate virtual characters according to the role slices. Finally, it combines the virtual characters with the role action time series graph and the actor motion sequence lists to generate a role dynamic graph according to the character elements. At the same time, according to the plot elements and special effect elements, it uses a holographic fusion mechanism to fuse the role dynamic graph and / or the actor with three-dimensional stereoscopic images, thus completing virtual character driving and holographic image construction.

[0046] The intelligent light source control module first divides all the light sources in the borderless virtual space according to the spatial layout of the screen, obtaining a light source array. It also detects the screen quality of all the screens, obtaining the resolution, refresh rate, response time, color gamut, brightness, contrast, color accuracy, and viewing angle of the screens. Then, it dynamically adjusts the chromaticity, brightness, and projection angle of the light source array according to the emotional tone in the plot elements combined with the screen quality, thus performing light source compensation on the holographic theater. At the same time, it uses a variational autoencoder to perform color detection on the special effect elements according to the display effect, obtaining a color distortion set. Subsequently, it analyzes the color distortion set into the three primary colors of light, and combines the plot elements and background elements to control the light source array to perform dynamic quantization light compensation.

[0047] After the performance plot ends, the adaptive interaction module first counts the audience in the holographic theater, obtaining the audience distribution density. At the same time, it measures the light source array, obtaining the theater light intensity. It also uses smart glasses or handheld devices to capture the motion behaviors of the audience, obtaining the line of sight focus and gesture commands. Then, it analyzes the field of view and seating order of each audience according to the audience distribution density, obtaining an audience perspective diffusion map and an audience interaction limit map. Finally, it uses a multi-resolution holographic model to adjust the presentation angle of the holographic image according to the line of sight focus, combined with the audience perspective diffusion map. At the same time, it uses a plot interaction model to perform interaction feedback and hidden plot triggering on the holographic image according to the gesture commands, in coordination with the audience interaction limit map.

[0048] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A holographic imaging system for a VR holographic theater, characterized in that: including a holographic background module, configured to project three-dimensional stereoscopic images in a borderless virtual space according to the spatial layout of a plurality of screens, in combination with theater performance and holographic dynamic perception strategies; The screens include a dynamic media wall, a fog screen projection screen, a holographic interactive transparent screen, and an inclined holographic film; The theater performance includes background elements, character elements, plot elements, and special effect elements; a dynamic fusion module, configured to track the actions of actors, and use a holographic fusion mechanism to fuse with the three-dimensional stereoscopic images in combination with character elements and plot elements to obtain holographic images; an intelligent light source control module, configured to dynamically adjust the chromaticity, brightness, and projection angle of a light source array according to the screen quality of different screens; an adaptive interaction module, configured to dynamically adjust the spatial layout of the screens and the plot content of the holographic images according to the screen functions, in combination with the line-of-sight focus and gesture commands of the audience, using a hierarchical projection strategy.

2. The holographic imaging system for a VR holographic theater according to claim 1, wherein: The working process of the holographic background module includes the following steps: Detect the spatial layout and screen size of each screen to obtain screen data; then perform virtual construction on the screen data to obtain a digital twin screen; Separate and correlate the background elements according to the performance timing and spatial orientation to obtain a background transformation map; Map the background transformation map to the digital twin screen by using a holographic dynamic perception strategy in combination with plot elements to obtain a virtual background demonstration projection, and perform edge enhancement and global derivation on the virtual demonstration projection to obtain an optimal background projection; Superimpose and project the optimal background projection onto the corresponding screen according to the spatial layout, and then display three-dimensional stereoscopic images on each screen.

3. The holographic imaging system for a VR holographic theater according to claim 1, characterized in that: The working process of the holographic dynamic perception strategy includes the following steps: S1: Extract the plot elements in time series to obtain a plurality of image time series frames; S2: Use to match and fuse the image time series frames with the background transformation map to obtain a scene fusion map; S3: Classify the scene fusion map according to the spatial orientation to obtain an orientation correlation time series table containing a plurality of image time series frames, and extract the projection offset angle at the same time; S4: Compare and judge the projection offset angle with a preset orientation conversion range; If the projection offset angle is greater than the upper limit value of the orientation conversion range or the projection offset angle is less than the lower limit value of the orientation conversion range, rotate the digital twin screen until the projection offset angles of all the image time series frames in the orientation correlation time series table are within the orientation conversion range; If the projection offset angles are all within the orientation conversion range, compare and correct the display effect of the image time series frames in the digital twin screen.

4. The holographic imaging system for a VR holographic theater according to claim 3, wherein: The working process of the holographic dynamic perception strategy also includes the following steps: S5: If picture tearing occurs in the digital twin screen, compensate for the frame interval delay and reconstruct a continuous background transformation trajectory; If color offset occurs in the digital twin screen, analyze the event nodes of the plot elements, match the color tone, and then update the global illumination of the background; If dynamic blur mismatch occurs in the digital twin screen, calculate the pixel motion vector between adjacent frames and perform edge enhancement on the image time series frames; S6: Perform virtual rendering on the image time series frames after the comparison and correction to obtain the virtual background demonstration projection of each digital twin screen; S7: Perform global derivation on virtual background demonstration projections with the same timestamp: Compare the peak signal-to-noise ratio of pixels and the pixel structural similarity in the overlapping area between digital twin screens with the signal-to-noise threshold and the structural threshold respectively; S8: If either the peak signal-to-noise ratio of pixels or the pixel structural similarity is less than the signal-to-noise threshold and the structural threshold, it is determined that the time sequence between digital twin screens is inconsistent, and the display effect of the image time sequence frames on the digital twin screens is corrected again; If both the peak signal-to-noise ratio of pixels and the pixel structural similarity are greater than or equal to the signal-to-noise threshold and the structural threshold, it is determined that the time sequence between digital twin screens is consistent, and the color error in the overlapping area is judged against the color difference threshold; S9: If the color error is greater than or equal to the color difference threshold, perform gamma value linearization on the overlapping area or correct the color of the digital twin screens; If the color error is less than the color difference threshold, it is determined that the color in the overlapping area is normal, and thus the optimal background projection is obtained.

5. The holographic imaging system for a VR holographic theater according to claim 1, characterized in that: The working process of the dynamic fusion module includes the following steps: Take multi-angle shots of each actor to obtain several actor performance videos, and at the same time capture the action nodes of each actor to obtain several actor action sequence lists; Convert each actor performance video into several role slices according to the performing role, perform pixel-level classification on the role slices to obtain a role action time sequence diagram, and generate virtual characters according to the role slices; Combine the virtual characters with the role action time sequence diagram and the actor action sequence list to generate a role dynamic diagram according to the human elements; Use the holographic fusion mechanism according to the plot elements and special effect elements to fuse the role dynamic diagram and / or the actor with the three-dimensional stereoscopic image, and thus complete virtual character driving and holographic image construction.

6. The holographic imaging system for a VR holographic theater according to claim 1, wherein: The working process of the holographic fusion mechanism is as follows: T1: Divide the special effect elements into particle special effects, rigid body special effects, fluid special effects, hair special effects, cloth special effects and soft body special effects according to the manifestation form; T2: Extract the plot turning nodes in the plot elements according to the Syd Field theory to obtain several special effect plot time sequence points; T3: Arrange the role dynamic diagrams in time sequence, and use the time sequence segmentation algorithm according to the special effect plot time sequence points to extract the corresponding role action frames; T4: Decompose the role action frames into independent motion units according to the discrete element principle, assign mass and elastic coefficients according to the role type, and generate discrete trajectory data; T5: Define the discrete element parameters of the special effect elements, and at the same time measure the contact quantization value between the role action frames and the special effect elements, and preset the connection threshold according to the role type and the special effect type; T6: If the time sequence of the contact quantization value is different from the connection threshold, correct the timestamps of the role action frames and / or the special effect elements according to the plot elements using the dynamic time warping algorithm; If the time sequence of the contact quantization value is the same as the connection threshold, compare the contact quantization value with the preset connection threshold; T7: If the contact quantization value is greater than or equal to the connection threshold, trigger physical feedback; if the contact quantization value is less than the connection threshold, perform inertial compensation on the special effect elements and / or the role action frames using the smooth transition algorithm; T8: Adaptive blur - sharpening is performed on the contact edges between the character action frames and the special effect elements, and at the same time, the character action frames, special effect elements, and three - dimensional stereoscopic images are fused to complete virtual character driving and holographic image construction.

7. The holographic imaging system for a VR holographic theater according to claim 1, characterized in that: The working process of the intelligent light source control module includes the following steps: All light sources in the borderless virtual space are divided according to the spatial layout of the screen to obtain a light source array; The screen qualities of all screens are detected to obtain the resolution, refresh rate, response time, color gamut, brightness, contrast, color accuracy, and viewing angle of the screens; Combined with the screen qualities according to the emotional tone in the plot elements, the chromaticity, brightness, and projection angle of the light source array are dynamically adjusted to perform light source compensation on the holographic theater; Color detection is performed on the special effect elements according to the display effect to obtain a color distortion set; The color distortion set is analyzed into the three primary colors of light, and combined with the plot elements and background elements, the light source array is controlled to perform dynamic quantization light compensation.

8. The holographic imaging system for a VR holographic theater according to claim 1, characterized in that: The working process of the adaptive interaction module includes the following steps: The audience in the holographic theater is counted to obtain the audience distribution density; at the same time, the light source array is measured to obtain the theater illumination intensity; at the same time, the action behaviors of the audience are captured to obtain the line - of - sight focus and gesture commands; Based on the audience distribution density, the field of view and seating order of each audience are analyzed to obtain the audience perspective diffusion map and the audience interaction limit map; Based on the line - of - sight focus and combined with the audience perspective diffusion map, the presentation angle of the holographic image is adjusted; Based on the gesture commands and combined with the audience interaction limit map, interactive feedback and hidden plot triggering are performed on the holographic image.

9. The holographic imaging system for a VR holographic theater according to claim 1, characterized in that: The working process of the adaptive interaction module further includes the following steps: Based on the screen functions, combined with the line - of - sight focus and gesture commands, each screen is dynamically adjusted using a hierarchical projection strategy: The distant view environment in the background elements is presented on the dynamic media wall, and the near - view environment in the background elements is superimposed on the holographic interactive transparent screen, The horizontal field of view in the plot elements is extended on the inclined holographic film, and the special effect elements are demonstrated and simulated using a fog screen projection screen in cooperation with a smoke machine and a wind effect device to construct a borderless virtual space.

10. A holographic imaging method for a VR holographic theater, applied to the holographic imaging system according to any one of claims 1 to 9, characterized in that: It successively includes the following steps: A: The adaptive interaction module initializes and performs function detection on all screens in the holographic theater to construct a borderless virtual space; B: After the screens are initialized, the holographic background module superimposes and projects three - dimensional stereoscopic images according to the spatial layout of several screens, combined with theater performance and holographic dynamic perception strategies; C: The dynamic fusion module tracks the actions of the actors, and at the same time, combines the character elements and plot elements to fuse with the three - dimensional stereoscopic images using a holographic fusion mechanism to construct a holographic image; D: The intelligent light source control module dynamically adjusts the chromaticity, brightness, and projection angle of the light source array according to the screen qualities of different screens, combined with the plot elements and special effect elements; E: The adaptive interaction module is used to dynamically adjust the spatial layout of the screen and the plot content of the holographic image according to the screen functions, combined with the line - of - sight focus and gesture commands of the audience, using a hierarchical projection strategy.

Images