A real-time editing and rendering method for visualizing a film and television stage scene

Abstract

The application discloses a film and television stage scene visualization real-time editing and rendering method, comprising the following steps: S1, collecting stage scene construction requirements, covering scene theme, element type, space layout and performance budget information, thus, the application realizes intelligent construction of a three-dimensional stage scene based on a trained and improved generative artificial intelligence model, can effectively shorten the scene production cycle, reduce labor cost and technical threshold according to structured requirements such as scene theme, space layout and performance budget, dynamically corrects virtual rendering parameters by collecting real-time shooting data and establishing a coordinate mapping relationship, can adapt to differentiated display requirements of various terminals, intelligently predicts the field of view range in combination with an LSTM network, can realize dynamic preloading of effective scene resources and real-time uninstalling of low-weight redundant resources, and greatly improves the real-time rendering fluency and stability of a complex dynamic stage scene.

Description

Technical Field

[0001] This application relates to the technical field of film and television stage, and in particular to a method for real-time editing and rendering of film and television stage scenes. Background Technology

[0002] With the rapid iteration and upgrading of digital film and television production, immersive performances, and XR virtual stage technologies, the traditional stage film and television production model of physical set construction and green screen keying has been gradually replaced by real-time virtual production technology. Virtual stage scenes, with their advantages of rapid customization, no physical construction losses, rich visual presentation effects, and real-time debugging and modification, are applied to film and television shooting, gala performances, online live broadcasts, immersive exhibitions, and other scenarios. Traditional manual production and fixed rendering technologies are no longer able to meet the industry's high-quality development needs.

[0003] Traditional virtual stage scenes rely heavily on designers for manual modeling, prop adaptation, lighting adjustment, and material mapping. The overall process is cumbersome, has a long production cycle, high labor costs, and requires highly skilled operators. It is difficult to quickly generate customized 3D scene assets based on scene theme, spatial layout, and performance requirements.

[0004] Meanwhile, most existing rendering methods use a single, universal, fixed rendering pipeline, which is not adapted to the hardware characteristics, interaction modes, and differentiated display needs of different display terminals. They lack targeted and differentiated rendering control strategies, and cannot achieve real-time and accurate matching between device position and virtual rendering perspective. This can easily lead to problems such as deviation in virtual scene perspective and insufficient fusion of virtual and real images.

[0005] Furthermore, scene resource scheduling only employs a basic optimization method of fixed view frustum culling, lacking a multi-level detailed model adaptation mechanism and resource loading priority determination logic. For dynamic moving viewpoints, it cannot predict the field of view range in advance, preload scene resources, or unload low-weight redundant resources outside the field of view in a timely manner, which easily leads to problems such as scene loading stuttering, frame rate fluctuations, and device memory overload. Summary of the Invention

[0006] This application aims to address, at least to some extent, the technical problems in the related art.

[0007] To achieve the above objectives, this application proposes a method for real-time visual editing and rendering of film and television stage scenes, comprising the following steps:

[0008] S1. Collect stage scene construction requirements, including scene theme, element type, spatial layout and performance budget information. Rebuild the model based on generative artificial intelligence model to generate initial 3D stage scene assets, including stage geometry model, prop model, lighting configuration and material texture.

[0009] S2. Differentiated rendering schemes are adopted for LED facade screens, XR extended reality headsets, and planar monitors. The LED screen relies on camera pose data to dynamically adjust virtual camera parameters to ensure real-time matching between virtual scene perspective and physical camera position. The XR headset uses SLAM algorithm to obtain the user's six degrees of freedom pose and realize adaptive field of view clipping and layered rendering. The planar monitor uses standard perspective projection pipeline output.

[0010] S3. Divide the stage scene into blocks and set multi-level detail models for each block. Collect the position and motion parameters of various viewpoints such as cameras, audience, and directors in real time. Combine the relative position and motion state of the viewpoints and scene blocks to determine the resource loading priority and detail level, and dynamically complete the loading and unloading of scene resources. At the same time, predict the frustum range through the LSTM network, eliminate redundant scene resources, and achieve rendering load balancing.

[0011] S4 monitors performance data such as GPU utilization, rendering frame rate, and memory usage in real time. When the indicators exceed the threshold, it reduces scene complexity parameters such as model accuracy and texture resolution as needed. At the same time, it recognizes user editing commands such as adding or deleting elements, adjusting poses, and modifying materials, and generates editing suggestions or automatically completes editing operations with the help of AI models.

[0012] S5 synthesizes differentiated views and editing results across multiple devices, generates terminal rendering screens, establishes a real-time feedback loop for editing and display, supports users to view editing effects in real time, and completes synchronous real-time display of screens across multiple devices.

[0013] In addition, the application may also include the following additional technical features:

[0014] Specifically, in step S1, the generative artificial intelligence model is a 3D scene generation model with a 3D convolutional generative adversarial network architecture. It is pre-trained using a massive dataset of film and television stage scenes, which includes samples of different stage themes, prop types, lighting schemes, and spatial layouts. After receiving structured requirement parameters, the generative artificial intelligence model automatically completes scene topology construction, prop adaptation, lighting baking, and material mapping. At the same time, it performs lightweight preprocessing on the initial 3D stage scene assets in conjunction with the performance budget information, removing oversized redundant model patches and high-precision invalid textures to generate initial scene assets that meet the performance budget standards.

[0015] Specifically, in step S2, the method for dynamically adjusting the virtual camera parameters on the LED screen is as follows: real-time acquisition of the position, rotation angle, focal length, field of view, and pose data of the physical camera; establishment of a coordinate mapping matrix between the physical camera position and the virtual camera; and real-time correction of the perspective matrix and projection matrix of the virtual camera based on the mapping matrix to achieve millisecond-level matching between the virtual scene perspective effect and the physical shooting angle.

[0016] Specifically, in step S2, the XR headset uses a visual inertial SLAM algorithm to obtain the user's six degrees of freedom pose, which includes three-dimensional translational and three-dimensional rotational degrees of freedom. The adaptive field of view culling locks the visible frustum region based on the user's real-time pose and directly shields the scene blocks outside the frustum from rendering. The layered rendering divides the scene resources into a core interaction layer, a distant view display layer, and an environment preparation layer. The core interaction layer uses high-precision real-time rendering, while the distant view display layer and the environment preparation layer use low-precision asynchronous rendering.

[0017] Specifically, in step S3, the resource loading priority is determined as follows: the closer the scene block is to the current effective viewpoint, the higher the motion frequency, and the larger the visual proportion, the higher the resource loading priority; for static viewpoints, only high-precision scene resources within the current field of view are loaded; for dynamic moving viewpoints, scene resources in the motion trajectory prediction area are preloaded in advance, while unloading operations are performed in real time for scene resources outside the field of view and with low weight to avoid memory overload.

[0018] Specifically, in step S4, the semantic recognition model analyzes the user's mouse, touch, and peripheral device control editing commands in real time to accurately identify various editing needs such as adding or deleting elements, adjusting model posture, modifying material colors, adjusting lighting parameters, and scaling and panning scenes. Combining stage scene aesthetic rules and industry standards, the AI ​​model automatically generates parameter optimization suggestions. For conventional standardized editing commands, one-click automatic editing is achieved. For personalized and complex editing commands, multiple editing schemes are pushed for the user to select and confirm.

[0019] Specifically, in step S5, building a real-time feedback closed loop for editing and display involves: establishing a data synchronization bus based on a subscription model, where user editing instructions on any terminal are encapsulated as standardized scene update events and published to the bus, which then distributes the events to all rendering terminals that have subscribed to the scene. Each terminal performs the same scene state update locally and triggers incremental re-rendering, while simultaneously storing the editing instructions in the operation log queue, supporting multi-user collaborative editing and operation playback.

[0020] The beneficial effects of the real-time editing and rendering method for visualizing film and television stage scenes proposed in this application are as follows:

[0021] 1. Based on a well-trained generative artificial intelligence model, it realizes intelligent construction of 3D stage scenes. It can automatically complete the entire process of scene topology construction, prop adaptation, lighting baking and material mapping according to structured requirements such as scene theme, spatial layout and performance budget, effectively shortening the scene production cycle and reducing labor costs and technical thresholds.

[0022] 2. To address the hardware characteristics and interactive needs of various display terminals, a differentiated adaptive rendering mechanism is constructed. By collecting camera position data in real time and establishing coordinate mapping relationships, the virtual rendering parameters are dynamically corrected to achieve millisecond-level precise matching between physical camera positions and virtual viewpoints. This mechanism can adapt to the differentiated display needs of multiple terminals while balancing image quality and rendering stability.

[0023] 3. By determining the priority of resource loading based on the position and motion state of multiple viewpoints and combining the LSTM network to intelligently predict the field of view, it is possible to dynamically preload effective scene resources and unload low-weight redundant resources in real time, effectively avoiding problems such as screen stuttering, frame rate fluctuations, and memory overload, achieving dynamic balance of rendering load, and greatly improving the real-time rendering smoothness and stability of complex dynamic stage scenes. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0025] Figure 1 This is a flowchart illustrating a method for real-time visual editing and rendering of film and television stage scenes according to this application;

[0026] Figure 2 This is a flowchart illustrating the 3D stage scene generation process of the real-time visual editing and rendering method for film and television stage scenes proposed in this application.

[0027] Figure 3 This application presents a multi-terminal collaborative editing flowchart for a real-time visualization editing and rendering method for film and television stage scenes. Detailed Implementation

[0028] To make the technical means, inventive features, objectives, and effects of this application easier to understand, the application is further described below with reference to specific illustrations. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0029] The present application will now be described in further detail with reference to the accompanying drawings.

[0030] like Figures 1-3 As shown in the figure, a method for real-time editing and rendering of a film and television stage scene according to an embodiment of this application includes the following steps:

[0031] S1. Collect stage scene construction requirements, including scene theme, element type, spatial layout and performance budget information. Rebuild the model based on generative artificial intelligence model to generate initial 3D stage scene assets, including stage geometry model, prop model, lighting configuration and material texture.

[0032] It should be noted that the requirements are broken down through interactive interfaces, forms, and business system integration: the scene theme defines the visual style and artistic tone; the element types define the material categories such as stage props, stage equipment, and special effects components; the spatial layout constrains the three-dimensional spatial coordinates, layers, sizes, and arrangement logic; and the performance budget pre-limits the overall number of faces, texture specifications, and rendering load limits, setting hard boundaries for subsequent modeling and rendering to avoid assets exceeding the equipment's carrying capacity.

[0033] Asset generation is accomplished by relying on a multimodal generative AI model: First, the model performs semantic parsing on the structured requirements, transforming textual descriptions into three-dimensional spatial semantics and visual feature instructions; then, based on a three-dimensional geometry generation algorithm, it batch-builds geometric mesh models of the main stage structure, props, etc.; combined with a style matching algorithm, it automatically generates PBR physical materials and texture maps to simulate realistic physical effects such as metal, fabric, and light and shadow reflections; finally, based on stage design industry rules and spatial lighting logic, it automatically matches lighting configuration parameters such as light type, illumination angle, brightness, and color temperature.

[0034] S2. Differentiated rendering schemes are adopted for LED facade screens, XR extended reality headsets, and planar monitors. The LED screen relies on camera pose data to dynamically adjust virtual camera parameters to ensure real-time matching between virtual scene perspective and physical camera position. The XR headset uses SLAM algorithm to obtain the user's six degrees of freedom pose and realize adaptive field of view clipping and layered rendering. The planar monitor uses standard perspective projection pipeline output.

[0035] It should be noted that the complete process of a film and television stage project includes three core stages: on-site presentation, immersive rehearsal, and backstage editing and monitoring. LED facade screens, XR extended reality headsets, and flat monitors correspond to different usage scenarios.

[0036] LED screens are fixed physical facade display carriers, with the audience's vision relying on the view captured by physical cameras on site. The system collects the position, angle, focal length, and distortion parameters of the physical cameras in real time, and uses this data to drive the virtual camera in the rendering engine to dynamically correct parameters such as the field of view, perspective matrix, and clipping plane. Following the principle of perspective matching, it ensures that the perspective relationships, proportions, and composition of the virtual stage scene are perfectly aligned with the images captured by the physical cameras. The virtual content presented on the LED screen is seamlessly integrated with the actual scene, without any glitches or visual shifts.

[0037] XR headsets are wearable immersive devices where the user's head can freely rotate and move, exhibiting six degrees of freedom (DOF) spatial motion. The system utilizes SLAM (Simultaneous Localization and Mapping) algorithms to scan the physical space in real-time, calculate the headset's current six-DOF pose, and accurately locate the user's field of view. Combined with field-of-view clipping and layered rendering techniques, it renders only the scene content within the user's current field of view, clips redundant models outside the field of view, and renders scenes in layers according to distance and importance, significantly reducing the rendering pressure per frame while balancing immersive visual effects with device computing power limitations.

[0038] Ordinary monitors are fixed-view desktop display devices with no need for spatial movement or immersive viewing, so they directly use the standard computer graphics perspective projection pipeline. Following the traditional 3D rendering workflow, a projection matrix is ​​constructed, geometric transformations are performed, rasterization is applied, and pixel shading is applied to output a standard 2D perspective image, meeting the visual needs of directors and backstage operators for routine viewing and basic editing.

[0039] S3. Divide the stage scene into blocks and set multi-level detail models for each block. Collect the position and motion parameters of various viewpoints such as cameras, audience, and directors in real time. Combine the relative position and motion state of the viewpoints and scene blocks to determine the resource loading priority and detail level, and dynamically complete the loading and unloading of scene resources. At the same time, predict the frustum range through the LSTM network, eliminate redundant scene resources, and achieve rendering load balancing.

[0040] It should be noted that the overall 3D stage scene is divided into several independent scene blocks according to the spatial partitioning algorithm, realizing modular resource management. Each block is equipped with a multi-level LOD model: high-precision model is used for close-up viewing, medium / low-precision simplified model is used for medium to long distance, and ultra-simplified model is used for distant / blind spots. The topological structure of different precision models is consistent, the visual transition is natural, and there is no obvious jump when switching.

[0041] The system collects real-time 3D coordinates, movement speed, and direction parameters from various viewpoints, including cameras, audience members, and the director's perspective. It calculates the straight-line distance and relative orientation of each viewpoint to different scene blocks. Based on the principle of closer proximity, smoother movement, and higher attention, resource loading priorities are assigned to blocks: high-priority blocks load high-precision models and complete textures, low-priority blocks load low-precision models, and blocks far from all viewpoints undergo resource unloading to free up video memory and computing power.

[0042] LSTM (Long Short-Term Memory) networks possess temporal data prediction capabilities. They perform temporal modeling of the continuous motion trajectories of the camera and viewpoint, predicting the visible frustum range in the next moment and within a short future period. Based on the prediction results, scene resources outside the view frustum are unloaded in advance, blocks that are about to enter the field of view are preloaded, and redundant models and textures that will not appear in the current and future field of view are actively removed, continuously balancing the overall rendering load.

[0043] S4 monitors performance data such as GPU utilization, rendering frame rate, and memory usage in real time. When the indicators exceed the threshold, it reduces scene complexity parameters such as model accuracy and texture resolution as needed. At the same time, it recognizes user editing commands such as adding or deleting elements, adjusting poses, and modifying materials, and generates editing suggestions or automatically completes editing operations with the help of AI models.

[0044] It should be noted that the rendering engine has a built-in hardware metric acquisition module that continuously reads core parameters such as GPU utilization, frame rate, RAM usage, and VRAM consumption, and presets safety thresholds. When the metrics exceed the thresholds, the system triggers an adaptive complexity reduction strategy: prioritizing the reduction of model geometric precision, compressing texture map resolution, disabling non-core lighting and shadow effects, and simplifying particle effects, thereby reducing the computational load per frame without compromising the main visual effect, forcibly pulling rendering performance back to the normal range, and ensuring continuous and stable system operation.

[0045] Through the human-computer interaction command parsing module, the system identifies user editing operations such as adding or deleting stage elements, adjusting model posture, and modifying materials / lighting, extracting the operation intent, modified parameters, and target objects. The editing commands are then input into the AI ​​model, which, based on the stage design rules, visual matching logic, and industry aesthetic experience learned by the model, automatically pushes reasonable editing optimization suggestions and performs fully automated editing of routine standardized operations, reducing repetitive manual operations and improving editing efficiency.

[0046] S5 synthesizes differentiated views and editing results across multiple devices, generates terminal rendering screens, establishes a real-time feedback loop for editing and display, supports users to view editing effects in real time, and completes synchronous real-time display of screens across multiple devices.

[0047] It should be noted that by aggregating the differentiated rendering views generated by LED screens, XR headsets, and flat monitors, and combining them with image layering and compositing algorithms and color unification correction technology, the screen resolution, color space, brightness, and image size of different terminals are uniformly adapted to ensure that the artistic style and content logic of the same stage scene are completely consistent on different devices, while only retaining the terminal-specific differences in perspective and immersion effects.

[0048] Every edit performed by the user is synchronized to the rendering engine in real time. The engine immediately updates the scene assets and re-renders the image, and the changes are instantly displayed on all display terminals. Operators can observe the editing effect and determine the direction of adjustments immediately.

[0049] Leveraging low-latency network synchronization protocols and status frame synchronization technology, the rendering clock, scene status, and camera parameters of all terminals are unified. Scene data and parameter change commands generated by the master terminal editor are quickly distributed to all slave devices via the local area network, ensuring that the frame sequence of screens from terminals in different locations and of different types is aligned and that content is updated synchronously.

[0050] In one embodiment of this application, in step S1, the generative artificial intelligence model is a 3D scene generation model with a 3D convolutional generative adversarial network architecture. It is pre-trained using a massive dataset of film and television stage scenes, which covers different stage themes, prop types, lighting schemes, and spatial layout samples. After receiving structured requirement parameters, the generative artificial intelligence model automatically completes scene topology construction, prop adaptation, lighting baking, and material mapping. At the same time, it performs lightweight preprocessing on the initial 3D stage scene assets in conjunction with performance budget information, removing oversized redundant model patches and high-precision invalid textures to generate initial scene assets that meet performance budget standards.

[0051] It should be noted that the 3D convolutional generative adversarial network model has undergone iterative optimization to meet the spatial structure, visual presentation, and real-time rendering requirements of film and television stages. The model is trained on a massive, finely annotated dataset of film and television stage scenes, covering various stage themes, prop categories, lighting schemes, and spatial layout styles. The samples are diverse and industry-adaptable, making it suitable for various stage creation scenarios. In practical applications, after receiving structured requirements parameters such as stage size, performance theme, style positioning, and lighting scheme, the model can automatically complete the entire 3D scene construction process, including scene topology building, intelligent prop adaptation and arrangement, global lighting baking, and accurate mapping of scene materials. Meanwhile, the model can combine the system's preset hardware computing power, rendering frame rate and other performance budget parameters to carry out intelligent lightweight processing of the initial high-precision 3D stage assets. By accurately judging the threshold, redundant model patches, invalid high-precision textures and other oversized redundant resources are eliminated. While preserving the core visual texture and detail effects of the scene, the asset size is reduced and the rendering computing power consumption is reduced. Finally, the initial 3D stage scene assets that are adapted to the terminal performance and can run efficiently are generated, which solves the problem of excessive computing power consumption of traditional high-precision 3D scenes and the difficulty of running smoothly on ordinary terminals.

[0052] In one embodiment of this application, in step S2, the specific method for dynamically adjusting the virtual camera parameters on the LED screen is as follows: real-time acquisition of the position, rotation angle, focal length, and field of view pose data of the physical camera, establishing a coordinate mapping matrix between the physical camera position and the virtual camera, and real-time correction of the perspective matrix and projection matrix of the virtual camera based on the mapping matrix to achieve millisecond-level matching between the perspective effect of the virtual scene and the physical shooting perspective.

[0053] It should be noted that the LED screen achieves precise fusion of virtual and real images by dynamically adjusting virtual camera parameters, featuring high precision, low latency, and real-time synchronization, making it suitable for XR scene fusion shooting needs. In specific implementation, the system uses high-precision sensors to collect real-time pose data of the physical camera, including its spatial position, rotation angle, focal length, and field of view. Based on the high-frequency collected real-time camera position data, a coordinate mapping matrix between the physical camera and the virtual camera is constructed, completing the precise alignment and binding of physical space and virtual 3D space.

[0054] The system relies on this mapping matrix to correct the perspective and projection matrices of the virtual camera in real time. It can synchronize the movement, rotation, and zoom of the physical camera in milliseconds, ensuring a precise match between the perspective effect of the virtual scene and the physical shooting perspective, with a latency controlled within 10ms. This method effectively solves problems such as misalignment of virtual and real images, perspective deviation, and camera synchronization lag in traditional XR shooting, achieving seamless integration of LED virtual images and real-world shooting images, significantly improving the overall visual effect and immersive experience of stage shooting.

[0055] In one embodiment of this application, in step S2, the XR head-mounted display uses a visual inertial SLAM algorithm to obtain the user's six degrees of freedom pose. The six degrees of freedom include three-dimensional translational degrees of freedom and three-dimensional rotational degrees of freedom. The adaptive view field culling is to lock the visible view frustum region based on the user's real-time pose and directly shield the scene blocks outside the view frustum for rendering. The layered rendering is to divide the scene resources into a core interaction layer, a distant view display layer, and an environment preparation layer. The core interaction layer uses high-precision real-time rendering, while the distant view display layer and the environment preparation layer use low-precision asynchronous rendering.

[0056] It should be noted that the six degrees of freedom include three-dimensional translational degrees of freedom along the X, Y, and Z axes, and three-dimensional rotational degrees of freedom along the same axes, enabling comprehensive capture of user movement, rotation, and other perspective changes. Based on real-time user pose data, the system employs a dual optimization strategy of adaptive field-of-view clipping and layered rendering: on the one hand, it locks the user's visible frustum area in real time, shielding invalid scene blocks outside the frustum from rendering and eliminating unnecessary computational waste; on the other hand, it divides scene resources into a core interaction layer, a distant view display layer, and an environmental preparation layer, implementing differentiated rendering strategies. The core interaction layer, such as the stage performance area and core interactive props, uses high-precision real-time rendering to ensure smooth interaction and clear details; the low-priority scene layer, such as distant backgrounds and environmental atmosphere, uses low-precision asynchronous rendering, effectively reducing the rendering pressure on the XR headset without affecting the overall visual experience, balancing image quality and device performance, and avoiding problems such as frame drops, device overheating, and perspective delay.

[0057] In one embodiment of this application, in step S3, the resource loading priority is determined by the following rules: the closer the scene block is to the current effective viewpoint, the higher the motion frequency, and the larger the visual proportion, the higher the resource loading priority; for static viewpoints, only high-precision scene resources within the current field of view are loaded; for dynamic moving viewpoints, scene resources in the motion trajectory prediction area are preloaded in advance, and scene resources outside the field of view and with low weight are unloaded in real time to avoid memory overload.

[0058] It should be noted that the system has set up intelligent resource loading priority judgment rules, with the distance between scene blocks and effective viewpoint, dynamic motion frequency, and visual proportion of the screen as the core judgment indicators. Blocks that are closer to the viewpoint, have higher motion frequency, and larger visual proportion are given higher loading priority, and high-precision core resources are loaded first to ensure the image quality and smoothness of interaction in the user's visual focus area.

[0059] The system employs differentiated loading strategies for different viewpoint states: when the user's viewpoint is stationary, only high-precision scene resources within the current view domain are loaded, while redundant resources outside the domain are shielded, saving device computing power and memory; when the user's viewpoint is dynamically moving, the system predicts the user's movement range through a trajectory prediction algorithm and preloads scene resources in the predicted area, eliminating stuttering during viewpoint switching. Simultaneously, the system incorporates a dynamic resource scheduling mechanism that unloads redundant, low-weight scene resources outside the view domain in real time, continuously releasing memory and video memory to avoid device resource overload and ensure smooth and stable dynamic interaction throughout the 3D scene.

[0060] In one embodiment of this application, in step S4, the user's mouse, touch, and peripheral device control editing commands are analyzed in real time through a semantic recognition model to accurately identify various editing needs such as adding or deleting elements, adjusting model posture, modifying material colors, adjusting lighting parameters, and scaling and panning scenes. Combining stage scene aesthetic rules and industry standards, the AI ​​model automatically generates parameter optimization suggestions. For conventional standardized editing commands, one-click automatic editing is achieved. For personalized and complex editing commands, multiple editing schemes are pushed for the user to select and confirm.

[0061] It should be noted that the system is equipped with a lightweight, high-precision semantic recognition model, which can analyze editing commands from various input devices such as mice, touch screens, and external peripherals in real time, and accurately identify all types of stage editing needs, including adding or deleting scene elements, adjusting model postures, modifying material colors, adjusting lighting parameters, and panning and scaling scenes.

[0062] The system incorporates professional stage aesthetic rules and industry standards, and can automatically generate parameter optimization suggestions based on real-time scene conditions, correcting issues such as layout imbalance, lighting conflicts, and color inconsistencies. For standardized and routine editing commands such as adding and deleting elements and adjusting basic parameters, the system supports one-click automatic editing, significantly improving basic editing efficiency. For complex and personalized editing needs such as multi-element linkage, style reconstruction, and adjustment of lighting and material combinations, the system uses algorithms to generate multiple compliant editing solutions for users to choose from, fine-tune, and confirm, balancing intelligent editing efficiency with user-personalized creative freedom, and achieving intelligent scene creation through human-machine collaboration.

[0063] In one embodiment of this application, step S5, establishing a real-time feedback closed loop for editing and display, specifically involves: establishing a data synchronization bus based on a subscription model, where user editing instructions on any terminal are encapsulated as standardized scene update events and published to the bus, and the bus distributes the events to all rendering terminals that have subscribed to the scene, with each terminal performing the same scene state update locally and triggering incremental re-rendering, while simultaneously storing the editing instructions in the operation log queue, supporting multi-user collaborative editing and operation playback.

[0064] It should be noted that the system is based on a subscription-based data synchronization bus, establishing a closed loop for real-time editing feedback across multiple terminals, enabling synchronized editing and display of scenes by multiple users and terminals. The system unifies the data transmission protocols for various terminals, encapsulating user-initiated editing operations and scene state changes on any terminal into standardized scene update events with complete parameters and timestamps, and publishing them to the synchronization bus.

[0065] Based on subscription relationships, the bus accurately distributes update events to all associated rendering terminals. Each terminal synchronously updates the scene state locally and uses incremental re-rendering to update only the changed blocks, eliminating the need for a global refresh. This significantly reduces synchronization computational power consumption and ensures real-time, error-free synchronization of multi-terminal screens. Simultaneously, all editing operations are stored in an operation log queue in real time, completely preserving the operation trajectory. This supports multi-user remote collaborative editing without interference, as well as subsequent operation playback, version rollback, and process traceability, forming a closed-loop real-time feedback system across the entire editing-synchronization-rendering-preservation chain.

[0066] The present application and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present application. The actual structure is not limited to this. In conclusion, if a person skilled in the art is inspired by this description and designs a similar structure and embodiment without departing from the spirit of the present application, such design should fall within the protection scope of the present application.

Claims

1. A method for real-time visual editing and rendering of film and television stage scenes, characterized in that, Includes the following steps: S1. Collect stage scene construction requirements, covering scene theme, element type, spatial layout and performance budget information, and reconstruct the model based on generative artificial intelligence model to generate initial 3D stage scene assets, including stage geometry model, prop model, lighting configuration and material texture. S2. Differentiated rendering schemes are adopted for LED facade screens, XR extended reality head-mounted devices, and planar monitors. The LED screen end relies on camera pose data to dynamically adjust virtual camera parameters to ensure real-time matching between virtual scene perspective and physical camera position. The XR head-mounted device uses SLAM algorithm to obtain the user's six degrees of freedom pose and realize adaptive cropping and layered rendering of the field of view. The planar monitor end uses standard perspective projection pipeline output. S3. Divide the stage scene into blocks and set multi-level detail models for each block. Collect the position and motion parameters of various viewpoints such as cameras, audience, and directors in real time. Combine the relative position and motion state of the viewpoints and scene blocks to determine the resource loading priority and detail level, dynamically complete the loading and unloading of scene resources, and predict the view frustum range through the LSTM network to eliminate redundant scene resources and achieve rendering load balancing. S4. Real-time monitoring of performance data such as GPU utilization, rendering frame rate, and memory. When the indicators exceed the threshold, the scene complexity parameters such as model accuracy and texture resolution are reduced as needed. At the same time, it recognizes user editing commands such as adding or deleting elements, adjusting poses, and modifying materials, and generates editing suggestions or automatically completes editing operations with the help of AI models. S5 synthesizes differentiated views and editing results across multiple devices, generates terminal rendering screens, establishes a real-time feedback loop for editing and display, supports users to view editing effects in real time, and completes synchronous real-time display of screens across multiple devices.

2. The method for real-time editing and rendering of film and television stage scenes according to claim 1, characterized in that, In step S1, the generative artificial intelligence model is a 3D scene generation model with a 3D convolutional generative adversarial network architecture. It is pre-trained using a massive film and television stage scene dataset, which covers different stage themes, prop types, lighting schemes and spatial layout samples. After receiving the structured requirements parameters, the generative artificial intelligence model automatically completes scene topology construction, prop adaptation, lighting baking, and material mapping. At the same time, it combines the performance budget information to perform lightweight preprocessing on the initial 3D stage scene assets, removes oversized redundant model patches and high-precision invalid textures, and generates initial scene assets that meet the performance budget standards.

3. The method for real-time editing and rendering of film and television stage scenes according to claim 1, characterized in that, In step S2, the specific method for dynamically adjusting the virtual camera parameters on the LED screen is as follows: real-time acquisition of the position, rotation angle, focal length, field of view, and pose data of the physical camera; establishment of a coordinate mapping matrix between the physical camera position and the virtual camera; and real-time correction of the perspective matrix and projection matrix of the virtual camera based on the mapping matrix to achieve millisecond-level matching between the perspective effect of the virtual scene and the physical shooting perspective.

4. The method for real-time editing and rendering of film and television stage scenes according to claim 1, characterized in that, In step S2, the XR headset uses a visual inertial SLAM algorithm to obtain the user's six degrees of freedom pose, which includes three-dimensional translational and three-dimensional rotational degrees of freedom. The adaptive field of view culling locks the visible frustum region based on the user's real-time pose and directly shields the scene blocks outside the frustum from rendering. The layered rendering divides the scene resources into a core interaction layer, a distant view display layer, and an environment preparation layer. The core interaction layer uses high-precision real-time rendering, while the distant view display layer and the environment preparation layer use low-precision asynchronous rendering.

5. The method for real-time editing and rendering of film and television stage scenes according to claim 1, characterized in that, In step S3, the rule for determining the resource loading priority is: the closer the scene block is to the current effective viewpoint, the higher the motion frequency, and the larger the visual proportion, the higher the resource loading priority. For static viewpoints, only high-precision scene resources within the current field of view are loaded. For dynamic moving viewpoints, scene resources in the motion trajectory prediction area are preloaded in advance. At the same time, scene resources outside the field of view and with low weight are unloaded in real time to avoid memory overload.

6. The method for real-time editing and rendering of film and television stage scenes according to claim 1, characterized in that, In step S4, the semantic recognition model analyzes the user's mouse, touch, and peripheral device control editing commands in real time, accurately identifying various editing needs such as adding or deleting elements, adjusting model posture, modifying material colors, adjusting lighting parameters, and scaling and panning scenes. Combining stage scene aesthetic rules and industry standards, the AI ​​model automatically generates parameter optimization suggestions. For conventional standardized editing commands, one-click automatic editing is achieved. For personalized and complex editing commands, multiple editing schemes are pushed for the user to choose and confirm.

7. The method for real-time editing and rendering of film and television stage scenes according to claim 1, characterized in that, In step S5, the real-time feedback closed loop for editing and display is established as follows: a data synchronization bus based on a subscription model is established. User editing instructions on any terminal are encapsulated as standardized scene update events and published to the bus. The bus distributes the events to all rendering terminals that have subscribed to the scene. Each terminal performs the same scene state update locally and triggers incremental re-rendering. At the same time, the editing instructions are synchronously stored in the operation log queue, supporting multi-user collaborative editing and operation playback.

Images