A miniaturized and portable system for virtual film production on large LED screens

Abstract

This application relates to a small, portable system for virtual film production on large LED screens, encompassing virtual film production and display control. The system includes an integrated processing host, a device identification module, a scene rendering module, a unified synchronization control module, an adaptive display scheduling module, and a multi-interface output module. The system automatically acquires LED display device parameters and establishes display mapping relationships through the device identification module. The scene rendering module performs multi-channel real-time rendering based on the virtual scene and camera pose, generating unified frame identification data. The unified synchronization control module establishes a logical clock and calculates the synchronization time window, performing consistency judgment and dynamic buffer adjustment for display frames. The adaptive display scheduling module optimizes output timing based on device refresh rate and latency. The multi-interface output module outputs synchronized data and dynamically corrects synchronization parameters based on feedback information. This invention enables high-precision synchronized display under heterogeneous multi-device conditions.

Description

Technical Field

[0001] This application relates to the field of virtual film production and display control technology, and in particular to a small and portable virtual film production system for large LED screens. Background Technology

[0002] With the development of virtual production technology, real-time shooting systems based on LED screens are gradually replacing traditional green screen solutions and are widely used in film and television production and virtual studios. Existing large-scale LED virtual production systems typically rely on multiple servers, independent synchronization control equipment, and complex wiring structures, resulting in problems such as large system size, complex deployment, and limited synchronization accuracy. Especially when multiple LED display devices are spliced ​​together, differences in refresh rates, display latency, and interface transmission delays between different devices can easily lead to screen tearing, desynchronization, or frame errors. Furthermore, existing systems rely heavily on manual configuration of display parameters and topology relationships during device connection, resulting in low automation and hindering rapid deployment. In addition, the lack of a hierarchical rendering mechanism based on camera perspective during virtual scene rendering leads to low utilization of computing resources. Summary of the Invention

[0003] Therefore, it is necessary to provide a small and portable system for virtual film production on large LED screens to address the aforementioned technical problems.

[0004] The present application provides a small and portable virtual film production system for large LED screens, which adopts the following technical solution: including an integrated processing host, a device identification module, a scene rendering module, a unified synchronization control module, an adaptive display scheduling module, and a multi-interface output module;

[0005] The integrated processing host is used to integrate a graphics processing unit, a central processing unit and a display control unit, and to provide a unified data processing platform;

[0006] The device identification module is communicatively connected to the multi-interface output module and is used to automatically obtain the device information of each LED display device when the LED display device is connected. The device information includes at least the display resolution, refresh rate, display topology, display delay parameters and interface type, and establishes a corresponding display device mapping relationship based on the device information.

[0007] The scene rendering module is used to perform multi-channel real-time rendering based on virtual scene data and camera pose information, generate corresponding multi-channel display frame data, and assign a unified frame identifier to each display frame data.

[0008] The unified synchronization control module is used to establish a unified logical clock, generate a corresponding synchronization time window based on the unified logical clock, determine the time consistency of each display frame data with the same frame identifier, and dynamically adjust the buffer depth of each display frame data according to the determination result, so that each display frame data can be synchronously output within the synchronization time window.

[0009] The adaptive display scheduling module is used to perform output timing adjustment, buffer compensation and refresh rate matching processing on the corresponding display frame data according to the display device mapping relationship and the refresh rate and display delay parameters of each LED display device, and generate output control data for the corresponding display device.

[0010] The multi-interface output module is used to send display frame data and synchronization control information to each LED display device for display via the corresponding interface according to the output control data;

[0011] The unified synchronization control module dynamically adjusts the synchronization time window based on the display status feedback information of each LED display device to maintain synchronized display between different LED display devices.

[0012] Preferably, the device identification module includes an interface detection unit, a device information reading unit, and a topology construction unit;

[0013] in,

[0014] The interface detection unit is used to detect the connection status of the LED display device;

[0015] The device information reading unit is used to read the device parameters of each LED display device;

[0016] The topology building unit automatically generates the display topology structure based on the device connection relationship and establishes the output mapping relationship.

[0017] Preferably, the unified synchronization control module includes: a logical clock generation unit, a synchronization window calculation unit, a frame consistency determination unit, and a dynamic cache adjustment unit;

[0018] in,

[0019] The synchronization window calculation unit calculates a unified synchronization time window based on the refresh cycle of each display device.

[0020] The frame consistency determination unit is used to determine whether each display frame meets the synchronous output condition.

[0021] The dynamic cache adjustment unit adjusts the cache depth of each display frame based on the judgment result.

[0022] Preferably, the dynamic cache adjustment unit adjusts the cache depth based on at least one of the following parameters:

[0023] LED display device refresh cycle; display delay; image rendering time; interface transmission delay; synchronization error of the previous output cycle.

[0024] Preferably, the adaptive display scheduling module establishes a corresponding output timing model based on the refresh cycle of each LED display device, and determines the output time and buffer release time of each display frame data according to the output timing model.

[0025] Preferably, the scene rendering module constructs a real-time view frustum region based on the camera pose information, and performs hierarchical rendering of the virtual scene based on the view frustum region, wherein:

[0026] The first rendering precision is used within the view frustum region;

[0027] Outside the view frustum region, a second rendering precision, lower than the first rendering precision, is used.

[0028] Preferably, the integrated processing host integrates the central processing unit, graphics processing unit, synchronization control unit and interface output unit in the same chassis, and is equipped with a portable power supply interface and a quick deployment interface to reduce the number of devices deployed on site.

[0029] The control method is as follows:

[0030] S1. Automatically identify the connected LED display devices, obtain the display parameters and device connection relationships of each LED display device, and establish a display device mapping relationship;

[0031] S2. Acquire virtual scene data and camera pose information, perform multi-channel real-time rendering, and generate display frame data with unified frame identifiers;

[0032] S3. Establish a unified logical clock and calculate the synchronization time window based on the unified logical clock;

[0033] S4. Based on the synchronization time window, perform consistency determination on display frame data with the same frame identifier, and dynamically adjust the cache depth of each display frame data according to the determination result.

[0034] S5. Based on the refresh rate, display delay, and output timing model of each LED display device, perform buffer compensation and output scheduling on the display frame data;

[0035] S6. Synchronously output the data of each display frame according to the output scheduling result, and correct the synchronization time window in real time according to the feedback information from the LED display device.

[0036] Preferably, in step S4, the synchronization time window is dynamically updated based on the following parameters: display refresh cycle; cached output error; interface transmission delay; and error of the previous synchronization cycle.

[0037] Preferably, in step S6, when the output delay of any LED display device exceeds a preset threshold, the corresponding device buffer depth and output time are recalculated so that all LED display devices complete the screen refresh within the updated synchronization time window.

[0038] In summary, this application includes the following beneficial technical effects:

[0039] By setting up an integrated processing host, central processing, graphics rendering, synchronization control, and interface output functions are integrated into a single device, significantly reducing system size and deployment complexity, and improving portability and on-site adaptability. The device identification module enables automatic identification and topology construction of LED display devices, reducing manual configuration workload and improving system intelligence. A unified synchronization control module establishes a logical clock and combines it with a dynamic cache adjustment mechanism to effectively eliminate refresh rate differences and latency effects between different devices, significantly improving the synchronization accuracy of multi-screen displays. An adaptive display scheduling module provides fine-grained control of output timing, achieving stable matching across refresh rate devices. Simultaneously, a hierarchical rendering strategy based on the view frustum region reduces system computational load and improves rendering efficiency. Overall, this invention has advantages such as high synchronization accuracy, strong stability, convenient deployment, and high resource utilization, and has good engineering application value. Detailed Implementation

[0040] This embodiment provides a small, portable virtual production system for large LED screens, suitable for film and television shooting sets, mobile virtual production environments, and rapid deployment scenarios. The system adopts an integrated architecture design, centralizing rendering, synchronization control, and display scheduling on a single processing platform to achieve high-precision synchronous display across multiple LED screens.

[0041] The system as a whole includes an integrated processing host, a device identification module, a scene rendering module, a unified synchronization control module, an adaptive display scheduling module, and a multi-interface output module. The modules interact with each other through a high-speed internal bus (such as PCIe bus or high-speed Ethernet).

[0042] The integrated processing host adopts a compact industrial computer structure, integrating the central processing unit (CPU), graphics processing unit (GPU), display control unit, and interface output unit into the same chassis. (Specific implementation details are as follows:)

[0043] The CPU can be a multi-core processor (e.g., 8 cores or more) used to execute device management, scheduling control and synchronization algorithms;

[0044] The GPU is used to perform real-time rendering of virtual scenes and supports multi-channel parallel rendering (at least 4 channels).

[0045] The display control unit uses an FPGA or a dedicated display control chip to achieve frame synchronization control and output signal shaping;

[0046] The memory uses high-speed DDR4 / DDR5 (≥32GB) to cache multi-processor rendering frame data;

[0047] The storage module uses NVMe solid-state drives to store virtual scene data and rendering resources;

[0048] Configure a portable power supply interface (supporting 220V AC and battery power switching) and set up quick deployment interfaces (such as quick-plug network ports and fiber optic interfaces) to enable rapid on-site access;

[0049] Through the above integrated design, the number of server clusters and external synchronization devices in traditional virtual production is reduced, achieving miniaturization and portability.

[0050] The device identification module automatically identifies device information and establishes a mapping relationship when an LED display device is connected. Internally, it includes an interface detection unit, a device information reading unit, and a topology construction unit. The interface detection unit detects the status of each physical interface, including HDMI, DP, SDI, Ethernet, and fiber optic interfaces, through a polling mechanism or interrupt triggering. When a change in interface level or a link establishment signal is detected, it determines that an LED display device has been connected. The device information reading unit, after detecting device connection, reads device parameters via the EDID protocol or a custom communication protocol, including: display resolution (e.g., 1920×1080, 3840×2160); refresh rate (e.g., 60Hz, 120Hz); display latency (in milliseconds); interface type; LED panel splicing information; and receiver card number. For LED devices that do not support standard protocols, configuration files can be read via control card communication (such as serial or Ethernet protocols); Topology building unit: Based on device connection relationships and physical splicing positions, construct the display topology structure, such as: horizontal splicing matrix (NxM structure); irregular splicing structure (arc, L-shaped), and generate a display device mapping table, including: device ID → display area coordinates → input channel → output interface; this mapping relationship is used for subsequent rendering partitioning and output scheduling.

[0051] The scene rendering module implements multi-channel real-time rendering based on a virtual engine, specifically as follows: It receives virtual scene data and camera pose information (position, orientation, focal length); constructs the camera's view frustum; and performs hierarchical rendering of the scene according to the view frustum region. Hierarchical rendering strategy: Within the view frustum region (main view area): high-precision rendering (4K resolution, full illumination calculation, real-time shadows); Outside the view frustum region (edge ​​area): low-precision rendering (reduced resolution or simplified lighting model); Multi-channel output: Based on the display topology, the rendered screen is divided into multiple sub-viewports, each corresponding to an LED display device, generating multiple display frame data streams. Frame identification mechanism: A unified frame identifier (FrameID, such as a timestamp or incrementing sequence number) is assigned to each frame of data for subsequent synchronization control.

[0052] The unified synchronization control module is used to realize time synchronization between multiple LED screens. It includes a logic clock generation unit, a synchronization window calculation unit, a frame consistency determination unit, and a dynamic buffer adjustment unit.

[0053] The system comprises several functional units: a logic clock generation unit, a synchronization window calculation unit, and a dynamic buffer adjustment unit. The logic clock is based on a high-precision system timer (e.g., 1μs level). The synchronization window calculation unit calculates the synchronization time window based on the refresh cycle Ti of each LED device: T_sync = max(Ti) + Δt, where Δt is a safety margin (e.g., 1~3ms). The frame consistency determination unit checks whether frame data with the same FrameID arrives in the output queue within the same time window; if a timeout or premature arrival occurs, it is marked as an out-of-sync frame. The dynamic buffer adjustment unit dynamically adjusts the buffer depth based on the following parameters: LED refresh cycle, display latency, rendering time (GPU return time), interface transmission latency, and synchronization error of the previous cycle. The specific adjustment strategy is: BufferDepth_i = BaseDepth + α·Delay_i + β·Error_prev, where α and β are adjustment coefficients.

[0054] The adaptive display scheduling module is used to match the refresh characteristics of different LED devices and realize output scheduling. Specifically, it includes: establishing an output timing model, creating an output timing model for each LED device: OutputTime_i = n × RefreshPeriod_i + Offset_i, where Offset_i is the device delay compensation value; scheduling processing, output timing adjustment: ensuring that frame data is triggered in the corresponding refresh cycle; buffer compensation: releasing frame data in advance or with delay; refresh rate matching: performing frame interpolation or frame discarding processing for devices with different refresh rates, and finally generating output control data, including the transmission timestamp, frame ID, and interface identifier.

[0055] The multi-interface output module is used to send data to the LED display device. Specifically, it supports multiple interfaces: HDMI 2.1, DP 1.4, SDI, and 10G Ethernet. The corresponding interface is selected according to the mapping relationship. The output content includes display frame data and synchronization control signals (FrameID, timestamp). DMA is used during the output process to reduce CPU load.

[0056] Synchronous feedback and dynamic correction: Each LED display device feeds back display status information through the backhaul link, including: current display frame ID, actual refresh time, and delay deviation. The unified synchronization control module dynamically corrects the synchronization window based on the feedback: when the delay of a device exceeds the threshold (e.g., 5ms): the cache depth is recalculated, the output time Offset_i is adjusted, and the synchronization time window is updated to ensure that all devices complete the refresh within the new synchronization window.

[0057] Based on the above system, the control method is as follows: First, the LED device access status is automatically detected and device parameters are obtained to construct the display topology and mapping relationship; then, virtual scene data is loaded and camera pose information is obtained, multi-channel real-time rendering is performed, and frame data with unified frame identifiers is generated; then, a unified logical clock is established and the synchronization time window is calculated; next, consistency judgment is performed on the same frame identifier data, and the buffer depth is dynamically adjusted according to the result; then, output scheduling is performed according to the refresh rate and latency of each device; finally, the display frame is synchronously output, and the synchronization time window is corrected in real time according to feedback information. During the synchronization process, the synchronization window is dynamically updated according to the refresh cycle, buffer error, and transmission delay; when an abnormal output of a device is detected, its buffer and output time are readjusted to ensure overall synchronization.

[0058] In the actual testing of this embodiment, the integrated processing host and device identification module were first jointly functionally verified. Under the condition of accessing various LED display devices of different specifications, the interface detection unit monitored the HDMI, DP, and network port link status in real time, and was able to complete access identification within 50ms after the device was powered on. The device information reading unit successfully obtained the resolution, refresh rate, and latency parameters through EDID and control card protocol, and the topology construction unit automatically generated the display topology structure. In the test environment, a 3×2 splicing LED array was constructed. The system was able to accurately establish the display mapping relationship, and the mapping relationship was automatically updated during device hot-plugging without manual intervention, verifying the stability and real-time performance of the device identification module.

[0059] In the scene rendering module test, a highly complex virtual scene (including dynamic lighting and particle effects) was selected for multi-channel rendering verification. By acquiring camera pose information in real time to construct the view frustum region, the system uses high-precision rendering in the main view frustum region and automatically reduces rendering precision in non-view frustum regions. Test results show that under the same GPU load, this hierarchical rendering mechanism can reduce rendering resource usage by approximately 25% while ensuring no significant quality loss in the main view frustum region. During multi-channel output, each channel's frame data was successfully appended with a unified frame identifier, the frame numbers were consecutive, and there were no dropped frames, providing a reliable data foundation for subsequent synchronization control.

[0060] For the unified synchronization control module and the adaptive display scheduling module, the focus was on synchronization performance testing under heterogeneous conditions with multiple devices. Under different refresh rates (mixed 60Hz and 120Hz) and different display latencies (5ms–20ms), the system generated a unified time base through a logical clock, and the synchronization window calculation unit dynamically generated a synchronization time window. The frame consistency determination unit accurately identified asynchronous frames and triggered the dynamic cache adjustment unit to adjust the cache depth. Test data showed that after introducing the dynamic cache adjustment strategy, the frame synchronization error between display devices decreased from the original ±8ms to within ±1ms. The adaptive display scheduling module precisely controlled the output time of each device through an output timing model, achieving stable synchronized display across refresh rate devices.

[0061] During the overall system integration test, the rendering frame data and synchronization control information were sent to each LED display device through a multi-interface output module, and the display status information was obtained in real time through the feedback link. When a single device delay (simulating network jitter or processing lag) was artificially introduced and exceeded the preset threshold, the system was able to complete the recalculation of the synchronization window and reconfiguration of cache parameters within two refresh cycles, allowing all devices to re-enter the synchronization state. Long-term operation testing (8 hours) showed that the system did not exhibit significant tearing, frame errors, or desynchronization.

Claims

1. A miniaturized and portable virtual film production system for large LED screens, characterized in that: It includes an integrated processing host, a device identification module, a scene rendering module, a unified synchronization control module, an adaptive display scheduling module, and a multi-interface output module; The integrated processing host is used to integrate a graphics processing unit, a central processing unit and a display control unit, and to provide a unified data processing platform; The device identification module is communicatively connected to the multi-interface output module and is used to automatically obtain the device information of each LED display device when the LED display device is connected. The device information includes at least the display resolution, refresh rate, display topology, display delay parameters and interface type, and establishes a corresponding display device mapping relationship based on the device information. The scene rendering module is used to perform multi-channel real-time rendering based on virtual scene data and camera pose information, generate corresponding multi-channel display frame data, and assign a unified frame identifier to each display frame data. The unified synchronization control module is used to establish a unified logical clock, generate a corresponding synchronization time window based on the unified logical clock, determine the time consistency of each display frame data with the same frame identifier, and dynamically adjust the buffer depth of each display frame data according to the determination result, so that each display frame data can be synchronously output within the synchronization time window. The adaptive display scheduling module is used to perform output timing adjustment, buffer compensation and refresh rate matching processing on the corresponding display frame data according to the display device mapping relationship and the refresh rate and display delay parameters of each LED display device, and generate output control data for the corresponding display device. The multi-interface output module is used to send display frame data and synchronization control information to each LED display device for display via the corresponding interface according to the output control data; The unified synchronization control module dynamically adjusts the synchronization time window based on the display status feedback information of each LED display device to maintain synchronized display between different LED display devices.

2. The large-scale LED screen virtual film production miniaturized and portable system according to claim 1, characterized in that: The device identification module includes an interface detection unit, a device information reading unit, and a topology construction unit; in, The interface detection unit is used to detect the connection status of the LED display device; The device information reading unit is used to read the device parameters of each LED display device; The topology building unit automatically generates the display topology structure based on the device connection relationship and establishes the output mapping relationship.

3. The large-scale LED screen virtual film production miniaturized and portable system according to claim 1, characterized in that: The unified synchronization control module includes: a logical clock generation unit, a synchronization window calculation unit, a frame consistency determination unit, and a dynamic cache adjustment unit; in, The synchronization window calculation unit calculates a unified synchronization time window based on the refresh cycle of each display device. The frame consistency determination unit is used to determine whether each display frame meets the synchronous output condition. The dynamic cache adjustment unit adjusts the cache depth of each display frame based on the judgment result.

4. The large-scale LED screen virtual film production miniaturized and portable system according to claim 1, characterized in that: The dynamic cache adjustment unit adjusts the cache depth based on at least one of the following parameters: LED display device refresh cycle; display delay; image rendering time; interface transmission delay; synchronization error of the previous output cycle.

5. The miniaturized and portable virtual film production system for large LED screens according to claim 1, characterized in that: The adaptive display scheduling module establishes a corresponding output timing model based on the refresh cycle of each LED display device, and determines the output time and buffer release time of each display frame data according to the output timing model.

6. The miniaturized and portable virtual film production system for large LED screens according to claim 1, characterized in that: The scene rendering module constructs a real-time view frustum based on the camera pose information, and performs hierarchical rendering of the virtual scene based on the view frustum, wherein: The first rendering precision is used within the view frustum region; Outside the view frustum region, a second rendering precision, lower than the first rendering precision, is used.

7. The large-scale LED screen virtual film production miniaturized and portable system according to claim 1, characterized in that: The integrated processing host integrates the central processing unit, graphics processing unit, synchronization control unit and interface output unit in the same chassis, and is equipped with a portable power supply interface and a quick deployment interface to reduce the number of devices deployed on site.

8. A method for controlling virtual film production on a large LED screen based on the system described in any one of claims 1 to 7, characterized in that, The specific steps are as follows: S1. Automatically identify the connected LED display devices, obtain the display parameters and device connection relationships of each LED display device, and establish a display device mapping relationship; S2. Acquire virtual scene data and camera pose information, perform multi-channel real-time rendering, and generate display frame data with unified frame identifiers; S3. Establish a unified logical clock and calculate the synchronization time window based on the unified logical clock; S4. Based on the synchronization time window, perform consistency determination on display frame data with the same frame identifier, and dynamically adjust the cache depth of each display frame data according to the determination result. S5. Based on the refresh rate, display delay, and output timing model of each LED display device, perform buffer compensation and output scheduling on the display frame data; S6. Synchronously output the data of each display frame according to the output scheduling result, and correct the synchronization time window in real time according to the feedback information from the LED display device.

9. The virtual film production control method for a large LED screen according to claim 8, characterized in that: In step S4, the synchronization time window is dynamically updated based on the following parameters: display refresh cycle; cached output error; interface transmission delay; and error of the previous synchronization cycle.

10. The virtual film production control method for a large LED screen according to claim 9, characterized in that: In step S6, when the output delay of any LED display device exceeds the preset threshold, the corresponding device buffer depth and output time are recalculated so that all LED display devices complete the screen refresh within the updated synchronization time window.