Multi-source signal fusion display method, device, system and program product

Abstract

The application relates to the display field, in particular to a multi-source signal fusion display method, device, system and program product. A system for fusion display comprises a rendering server and multiple service servers, the method comprises the following steps: the service servers generate and output original video signals with different frame rates; the rendering server collects the original video signals output by the service servers through multiple built-in collection interfaces and performs frame-level synchronization alignment processing; the rendering server converts the original video signals of all channels which have completed frame-level synchronization alignment into target video signals with a preset target output frame rate through an adaptive frame repetition strategy; and the rendering server fuses the target video signals of all channels which have completed frame rate conversion and synchronously outputs the target video signals to a driving control system for display. The method can effectively improve service compatibility and expansibility, frame rate compatibility and synchronization precision.

Description

Technical Field

[0001] This application relates to the field of display, and in particular to multi-source signal fusion display methods, apparatus, systems and program products. Background Technology

[0002] LED dome screens, with their superior immersive experience, are becoming the preferred choice for high-end cultural tourism and science popularization displays (such as 5D cinemas, digital exhibition halls, and sports broadcasting venues). The modern "one venue, multiple uses" operation model of venues requires display systems to be flexible and compatible with various business scenarios such as movie playback, live sports broadcasts, real-time interaction, and stage performances. This brings the challenge of simultaneously processing multiple heterogeneous (different frame rates, different formats, and different sources) signal sources.

[0003] Currently, mainstream solutions in the industry include: 1) Dedicated dome players have limited functionality, cannot access real-time signals, and have poor scalability; 2) Single ultra-high-performance servers have performance bottlenecks and single-point-of-failure risks, making it difficult to handle ultra-high-resolution rendering and multi-channel real-time acquisition; 3) When multiple servers are combined with an external fusion unit, the frame timing of each server cannot be precisely aligned, resulting in screen tearing and flickering at the projection blending zone, severely damaging the immersive experience, and exhibiting poor frame rate compatibility. Therefore, current solutions generally suffer from poor business compatibility, poor scalability, poor frame rate compatibility, and low synchronization accuracy. Summary of the Invention

[0004] In view of this, embodiments of this application provide a multi-source signal fusion display method, apparatus, system, and program product to solve the problems of poor service compatibility and scalability, poor frame rate compatibility, and low synchronization accuracy in the prior art.

[0005] A first aspect of this application provides a multi-source signal fusion display method. The system for fusion display includes a rendering server and multiple service servers. The method includes: the service servers generating and outputting raw video signals with different frame rates; the rendering server acquiring the raw video signals output by each service server through multiple built-in acquisition interfaces, and performing frame-level synchronization alignment processing on the acquired multi-channel video signals; the rendering server converting the frame-level synchronized raw video signals to target video signals with a preset target output frame rate through an adaptive frame repetition strategy; and the rendering server fusing the multi-channel target video signals after frame rate conversion and synchronously outputting them to a drive control system for display.

[0006] Furthermore, the conversion of the target video signal to a preset target output frame rate using an adaptive frame repetition strategy includes: acquiring the original frame rate of the original video signal and acquiring the target output frame rate; determining the number of repetitive frames to be inserted into the original video signal based on the ratio of the target output frame rate to the original frame rate; and inserting repetitive frames into the original video signal according to the number of repetitive frames to be inserted, thereby generating the target video signal. This adaptive frame repetition strategy dynamically determines the number of repetitive frames by calculating the frame rate ratio, achieving unified processing of signals with different frame rates.

[0007] Furthermore, based on the ratio of the target output frame rate to the original frame rate, the number of repeated frames inserted into the original video signal is determined, including: when the ratio of the target output frame rate to the original frame rate is an integer N, determining that the number of repeated frames inserted after each original frame is N-1; when the ratio of the target output frame rate to the original frame rate is not an integer, determining a different number of repeated frames to be inserted after at least two consecutive original frames. This classification processing strategy employs different frame repetition rules for integer and non-integer ratios, minimizing the motion discontinuity introduced by frame repetition.

[0008] Furthermore, the target output frame rate is 120Hz; when the ratio of the target output frame rate to the original frame rate is not an integer, the number of repeated frames inserted after at least two consecutive original frames is determined, including: when the original frame rate is 48Hz, the total number of repeated frames inserted after two consecutive original frames is determined to be 3; when the original frame rate is 50Hz, the total number of repeated frames inserted after five consecutive original frames is determined to be 7. 120Hz as the target output frame rate is a common multiple or close to a common multiple of common input frame rates, providing smooth motion performance. For frame rates with non-integer ratios such as 48Hz and 50Hz, repeated frames are inserted through a specific cyclical pattern, ensuring the smoothness of the image and the continuity of motion.

[0009] Furthermore, the rendering server acquires video signals output by each service server through multiple built-in acquisition interfaces, including: when a new signal source is detected, allocating at least a first buffer and a second buffer for each new signal source's video signal; when the first buffer has stored a predetermined number of frames of images, reading the images from the first buffer and writing the received images through the second buffer; when the second buffer has stored a predetermined number of frames of images, reading the images from the second buffer and writing the received images through the first buffer. This dual-buffer ping-pong mechanism ensures continuous data reading and writing while providing a time buffer window for fluctuations in the frame arrival times of each signal, preventing data loss caused by brief jitter.

[0010] Furthermore, frame-level synchronization alignment processing is performed on the acquired multi-channel video signals, including: generating a synchronization reference signal through a synchronization signal generator; obtaining the first timestamp of the buffered frames of the multi-channel video signals and the second timestamp of the synchronization reference signal, and determining the time deviation between the first timestamp and the second timestamp; and performing frame-level alignment processing on the multi-channel video signals according to the time deviation and a preset deviation threshold. This synchronization alignment mechanism ensures that signals from different service servers, different frame rates, and different formats are processed uniformly within the same clock cycle by comparing a unified time reference and timestamps, eliminating screen tearing and flickering in the blending band.

[0011] Furthermore, the multi-channel video signals are subjected to frame-level alignment processing based on the time deviation and a preset deviation threshold. This includes: when the time deviation of all video signals is less than or equal to the deviation threshold, all video signals are determined to have completed synchronization alignment processing, and the aligned video signals are output; if the time deviation of any video signal is greater than the deviation threshold, the system waits for the next frame of that video signal for deviation verification, or if the waiting time exceeds a predetermined duration threshold, the frame is determined to have timed out and is skipped. This timeout frame processing strategy improves the robustness of the system. When a signal source experiences a frame timeout due to network jitter or encoding delay, skipping frames or waiting for the next frame ensures that other signal sources are not blocked, maintaining the stability of the overall output frame rate.

[0012] A second aspect of this application provides a multi-source signal fusion display device. The system for fusion display includes a rendering server and multiple service servers. The device includes: a raw video signal generation unit, used by the service servers to generate and output raw video signals with different frame rates; a synchronization alignment unit, used by the rendering server to acquire the raw video signals output by each service server through multiple built-in acquisition interfaces, and to perform frame-level synchronization alignment processing on the acquired multi-channel video signals; a frame rate conversion unit, used by the rendering server to convert the raw video signals that have completed frame-level synchronization alignment to a target video signal with a preset target output frame rate through an adaptive frame repetition strategy; and a fusion display unit, used by the rendering server to fuse the multi-channel target video signals after frame rate conversion, and synchronously output them to the drive control system for display.

[0013] A third aspect of this application provides a multi-source signal fusion display system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the multi-source signal fusion display system implements the method described in any of the first aspects.

[0014] A fourth aspect of this application provides a computer program product that, when run on a computer, causes the computer to execute the methods described in the first aspect or its various implementations.

[0015] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any of the first aspects.

[0016] A sixth aspect of this application provides a chip for implementing the methods in the various implementations of the first aspect described above. Specifically, the chip includes a processor for calling and running a computer program from a memory, causing a device equipped with the chip to perform the methods as described in the first aspect or its various implementations.

[0017] The beneficial effects of this application embodiment compared with the prior art are as follows: By adopting a distributed architecture of multiple business servers and rendering servers, the business functions and rendering functions are decoupled. Each business server independently runs business logic and outputs video signals to the rendering server, which can effectively support the dynamic expansion of business servers, thereby meeting the flexible operation needs of modern venues for "one venue, multiple uses" and significantly improving the system's scalability and business compatibility. Through frame-level synchronization processing and an adaptive frame repetition strategy, heterogeneous signals with different frame rates are aligned and uniformly converted to the target output frame rate, which can effectively solve the problems of screen tearing and flickering caused by frame timing asynchrony when multiple source signals are accessed in LED dome display systems. It also reduces motion discontinuity introduced by frame repetition and significantly improves the system's frame rate compatibility and screen smoothness. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the structure of a multi-source signal fusion display system provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the implementation process of a multi-source signal fusion display method provided in an embodiment of this application; Figure 3 This is a schematic diagram of a synchronization alignment process provided in an embodiment of this application; Figure 4 This is a schematic diagram of a dual-buffer area provided in an embodiment of this application; Figure 5 This is a schematic diagram of multi-source signal synchronization timing provided in an embodiment of this application; Figure 6 This is another schematic diagram of multi-source signal synchronization timing provided in the embodiments of this application; Figure 7 This is a schematic diagram illustrating the implementation process of a method provided in an embodiment of this application; Figure 8 This is a schematic diagram illustrating the implementation process of a method provided in an embodiment of this application; Figure 9 This is a schematic diagram of a multi-source signal fusion display device provided in an embodiment of this application; Figure 10 This is a schematic diagram of a multi-source signal fusion display system provided in an embodiment of this application. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0021] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0022] LED dome screens, as an emerging immersive display technology, are widely used in scenarios such as 5D cinemas, stage concert halls, digital exhibition halls, and sports broadcasting venues. With the continuous expansion of application scenarios, dome screen display systems need to simultaneously access multiple types of signal sources. For example, in a 5D cinema scenario, a 24fps movie playback signal is required; in a stage concert hall scenario, a 60fps real-time visual effects signal and a 30fps interactive content signal are needed; and in a sports broadcasting venue scenario, a 50fps live broadcast signal is required. The modern "one venue, multiple uses" operation model of venues requires display systems to be flexible and compatible with multiple business scenarios, quickly switching between or simultaneously displaying different types of signal sources.

[0023] The adopted solutions include: Option 1: Dedicated Dome Player. This option uses a player device specifically designed for dome screens, typically integrating dome geometry correction and edge blending functions, and supporting playback of pre-rendered content. However, this option has the following main drawbacks: it only supports playback of pre-rendered content, cannot access real-time signal sources, and cannot meet the real-time business needs of stage performances, live sports broadcasts, etc.; it cannot simultaneously connect to multiple business servers, resulting in poor business scalability; and its closed system architecture makes it difficult to customize and expand according to the venue's needs.

[0024] Option 2: Single Ultra-High-Performance Server. This option uses a single server equipped with a high-performance GPU to implement multi-signal access, fusion processing, and dome screen correction via software. While this option can handle multiple signals, it has the following main drawbacks: Limited by single-machine rendering capabilities, performance bottlenecks easily occur when multiple high-resolution signals are input, leading to excessive GPU load and frame rate drops; it is difficult to simultaneously meet the needs of ultra-high-resolution rendering and multi-service expansion; the single-server solution carries a single point of failure risk, resulting in lower system reliability; and the business logic and rendering processing are coupled on the same server, requiring redeployment of the entire system when expanding services.

[0025] Option 3: Multiple servers connected with an external fusion unit. In this option, each server independently runs its business logic and outputs video signals, while the external fusion unit completes the image stitching. However, this option has the following main drawbacks: each server has an independent clock, lacking a unified synchronization reference, resulting in inconsistent timing of multiple signal outputs; when different frame rate signal sources are connected, the difference in the arrival time of each signal at the display terminal causes image tearing at the seams of the dome screen; the external fusion unit typically uses hardware for signal processing, resulting in significant processing latency and inability to achieve precise frame-level synchronization; and the inconsistent timing of multiple signals at the fusion boundary causes flickering in the fusion band, severely compromising the immersive experience.

[0026] Therefore, the above solution has the following drawbacks: poor business compatibility, unable to flexibly access multiple real-time signal sources, making it difficult to meet the operational needs of "one venue for multiple uses"; poor system scalability, with high coupling between the business server and the rendering server, requiring redeployment of the entire system when expanding business; low synchronization accuracy of multiple signals, lacking a unified time base and frame-level synchronization mechanism, resulting in screen tearing and flickering in the blending band; and poor frame rate compatibility, unable to uniformly handle heterogeneous signal sources with different frame rates such as 24fps, 30fps, 48fps, 50fps, and 60fps.

[0027] To address the aforementioned problems, this application proposes a multi-source signal fusion display method. Figure 1 The diagram illustrates a system for implementing this method, which includes a rendering server, multiple business servers, a synchronization signal generator, a drive control system, and a display terminal.

[0028] The display terminal may include large-screen or heterogeneous screen display terminals such as LED dome display terminals.

[0029] Multiple service servers include a movie playback server, an MR interactive server, a sports event live streaming server, and a stage performance server. Each service server runs its business logic independently and can generate and output raw video signals with different frame rates. For example, the movie playback server outputs a 24fps movie playback signal, the MR interactive server outputs a 60fps interactive content signal, the sports event live streaming server outputs a 50fps live streaming signal, and the stage performance server outputs a 60fps stage effects signal. Each service server outputs video signals to the rendering server via an SDI interface.

[0030] A synchronization signal generator is used to generate a unified synchronization reference signal. The frequency of the synchronization reference signal output by the synchronization signal generator can be 60Hz. The timing of the video output frames of all service servers and the output refresh rate of the rendering server can be locked to this synchronization reference signal to ensure physical-level timing consistency.

[0031] The rendering server can acquire raw video signals output from various service servers through multiple built-in acquisition interfaces and perform frame-level synchronization alignment on the acquired multi-channel video signals. The number of acquisition interfaces can exceed the number of service servers, facilitating the dynamic integration of new service servers. For each raw video signal that has completed frame-level synchronization alignment, the rendering server can convert it to a target video signal with a preset target output frame rate using an adaptive frame repetition strategy. The multi-channel target video signals after frame rate conversion are then fused and synchronously output to the drive control system for display, thereby facilitating the generation of smooth playback.

[0032] Figure 2 The diagram shown is a schematic representation of the implementation flow of a multi-source signal fusion display method provided in an embodiment of this application. This method is based on... Figure 1 The multi-source signal fusion system shown below is described in detail below: In S201: The service server generates and outputs raw video signals with different frame rates.

[0033] In this embodiment, multiple business servers independently run business logic, generating and outputting raw video signals with different frame rates. Specifically, the movie playback server runs movie playback software, decodes movie files, and outputs a raw video signal at 24fps; the MR interactive server runs a mixed reality interactive program, receives user input, renders interactive content in real time, and outputs a raw video signal at 60fps; the event live streaming server receives remote live stream signals, decodes them, and outputs a raw video signal at 50fps; and the stage performance server runs stage visual effects software such as Notch, generates stage lighting, particle effects, and other content in real time, and outputs a raw video signal at 60fps.

[0034] The video output frames of each service server are time-locked to a unified synchronization reference signal output by a synchronization signal generator. Specifically, the synchronization signal generator outputs a 60Hz synchronization reference signal (square wave), and each service server has a built-in synchronization locking module that locks the timing of the video output frames to the rising or falling edge of this synchronization reference signal. For example, the 24fps output of a movie playback server corresponds to the rising edge of the 1st, 3rd, 5th, 8th, and 10th cycles of the synchronization reference signal for each frame output (distributed in a 5:5:5:5:5 pattern). This unified locking of the hardware time reference ensures the timing consistency of all signals at the physical level, providing a prerequisite for subsequent software synchronization.

[0035] Each service server outputs video signals to the rendering server via the SDI interface. The SDI interface adopts the serial digital interface standard and supports high-bandwidth, low-latency video signal transmission. For example, the 3G-SDI interface supports 1920×1080@60fps video transmission with a transmission bandwidth of 3Gbps.

[0036] In S202: The rendering server acquires the raw video signals output by each service server through multiple built-in acquisition interfaces, and performs frame-level synchronization and alignment processing on the acquired multi-channel video signals.

[0037] like Figure 3 The schematic diagram of the synchronization alignment process shown includes: acquiring the original video signal, dual buffer processing, and frame-level synchronization alignment processing.

[0038] In S301: Acquire the raw video signal.

[0039] The rendering server acquires raw video signals from various service servers through multiple built-in acquisition interfaces. Specifically, the SDI capture card receives video signals from multiple service servers via signal connections. For example, four SDI capture cards can be configured, each supporting four inputs, for a total of 16 inputs. The SDI capture card directly transfers video frame data to system memory via the PCIe bus using DMA. DMA bypasses the CPU interrupt handling mechanism, avoiding frame timing jitter caused by CPU scheduling uncertainties, resulting in an acquisition latency of less than 1ms.

[0040] In S302, dual buffer processing is used.

[0041] The rendering server acquires video signals output by each service server through multiple built-in acquisition interfaces, including: when a new signal source is detected, allocating at least a first buffer and a second buffer for the video signal of each new signal source; when a predetermined number of frames of images have been stored in the first buffer, reading the images in the first buffer and writing the received images through the second buffer; when a predetermined number of frames of images have been stored in the second buffer, reading the images in the second buffer and writing the received images through the first buffer.

[0042] See Figure 4 A diagram illustrating a dual-buffer system. Figure 4 As shown, the dual-buffer module allocates dual-buffer space in system memory for each input signal. When a new signal source is detected, the main control synchronization module detects changes in the input status of the acquisition interface through a polling mechanism. The polling period can be a predetermined duration, such as 10ms. After detecting parameters such as the video format and frame rate of the new signal source, the main control synchronization module automatically allocates dual-buffer resources for the new signal source. It allocates the first and second buffers from the reserved memory pool, with the size of each buffer calculated based on the video resolution. For example, for a 1920×1080 resolution video signal, each buffer caches 2 frames, allocating 1920×1080×3×2 bytes, approximately 12MB of space, for a total of approximately 24MB for both buffers.

[0043] A ping-pong caching mechanism is employed: when a predetermined number of image frames have been stored in the first buffer (e.g., two complete image frames have been stored), the images in the first buffer are read for subsequent processing, and the newly received image is written through the second buffer; when a predetermined number of image frames have been stored in the second buffer (e.g., two complete image frames have been stored), the images in the second buffer are read for subsequent processing, and the newly received image is written through the first buffer. Alternating switching enables continuous read and write operations.

[0044] The specific working process is as follows: Initially, both the first and second buffers are empty. When the Nth frame of video data arrives, it is written to the first buffer. After the first buffer is full for two frames (the Nth and N+1th frames), the main control synchronization module reads the Nth and N+1th frames from the first buffer, and simultaneously, the N+2th frame of video data arrives and is written to the second buffer. After the second buffer is full for two frames (the N+2th and N+3th frames), the main control synchronization module reads the N+2th and N+3th frames from the second buffer, and simultaneously, the N+4th frame of video data arrives and is written to the first buffer (at this point, the Nth and N+1th frames of the first buffer have been read, and the space has been released). This cycle continues, achieving continuous read and write operations.

[0045] The dual-buffer mechanism effectively absorbs frame arrival time fluctuations caused by network jitter, encoding delays, and other factors. For example, if a signal's 5th frame arrives 10ms late due to network jitter, the dual-buffer mechanism allows the main control synchronization module to first read and process the 4th frame from the first buffer, waiting for the 5th frame to arrive in the second buffer before reading it, thus avoiding frame loss. It also provides a time buffer window for synchronization alignment. When a signal's frame arrives earlier than other signals, that frame is temporarily stored in the buffer, awaiting the arrival of frames from other signals for unified synchronization alignment. Furthermore, it prevents data loss due to brief jitter. When a signal is briefly interrupted (interruption time < 1 frame period), the data in the buffer ensures uninterrupted output.

[0046] In S303, frame-level synchronization alignment is handled.

[0047] The frame-level synchronization and alignment processing of the acquired multi-channel video signals includes: generating a synchronization reference signal through a synchronization signal generator; obtaining a first timestamp of the buffered frames of the multi-channel video signals and a second timestamp of the synchronization reference signal, and determining the time deviation between the first timestamp and the second timestamp; and performing frame-level alignment processing on the multi-channel video signals according to the time deviation and a preset deviation threshold.

[0048] Specifically, the synchronization signal generator produces a synchronization reference signal. The synchronization reference signal can be a 60Hz square wave signal with a period of approximately 16.7ms. The main control synchronization module uses the synchronization reference signal as a precise time reference to timestamp all video frames in the buffer.

[0049] The system acquires the first timestamp of the buffered frames from multiple video signals and the second timestamp of the synchronization reference signal. Specifically, the main control synchronization module uses a continuous clock provided by a high-precision Genlock synchronization signal generator as the timestamp reference, with an accuracy down to the microsecond level. When a video frame from a certain signal arrives in the buffer, the main control synchronization module records the absolute arrival time T_i. The second timestamp of the synchronization reference signal is the current reference clock time T_genlock.

[0050] When determining the time deviation between the first timestamp and the second timestamp, the main control synchronization module calculates the absolute difference ΔT_i = |T_i - T_genlock| between the first timestamp T_i of each signal and the current reference time T_genlock, as the time deviation. For example, if the current reference time T_genlock = 1000.000ms and the arrival time T_i of a certain signal frame = 999.999ms, then the time deviation ΔT_i = 0.001ms (1 microsecond).

[0051] The multi-channel video signals are aligned at the frame level based on the time deviation and a preset deviation threshold. Specifically, the preset deviation threshold set by the main control synchronization module is 1ms. The main control synchronization module determines that the current frame of all signal sources has reached frame-level synchronization only when the time deviation ΔT_i of all signal sources is less than or equal to 1ms, and then releases this set of frame data through the synchronization gate to enter the subsequent processing flow. If any signal times out, the system will adopt a waiting or skipping mechanism to ensure output continuity.

[0052] See Figure 5 The diagram shows a timing sequence for synchronizing multiple signal sources. Figure 5 As shown, after adopting this synchronization alignment mechanism, the arrival time of each signal frame is constrained within an extremely narrow time window. For example, the deviations of the first timestamps of signals 1, 2, and 3 from the same reference time are calculated to be 0.3ms, 0.5ms, and 0.8ms, respectively, all less than the deviation threshold of 1ms. The main control synchronization module determines that the timestamps are aligned and then outputs the frame data of each signal in a unified manner.

[0053] The multi-channel video signals are aligned at the frame level based on the time deviation and a preset deviation threshold. This includes: when the time deviation of all video signals is less than or equal to the deviation threshold, all video signals are determined to have completed synchronization alignment, and the aligned video signals are output; if the time deviation of any video signal is greater than the deviation threshold, the next frame of that video signal is waited for for deviation verification, or if the waiting time exceeds a predetermined duration threshold, the frame is determined to have timed out and is skipped.

[0054] Specifically, when the time deviation of all video signals is less than or equal to the deviation threshold, the main control synchronization module determines that all video signals have completed synchronization alignment processing and uniformly outputs the aligned video signals to enter the subsequent frame rate conversion process. For example, if the time deviations of the three signals are 0.3ms, 0.5ms, and 0.8ms, respectively, all less than the deviation threshold of 1ms, the main control synchronization module uniformly outputs the current frame of these three signals when the synchronization gate alignment point is triggered.

[0055] If the time deviation of any video signal exceeds the deviation threshold, the system waits for the next frame of that video signal for deviation verification. Alternatively, if the waiting time exceeds a predetermined time threshold, the frame is determined to have timed out and is skipped. Specifically, the main control synchronization module sets the predetermined time threshold to the theoretical maximum waiting time for the next frame. For example, for a 60Hz signal, the time threshold can be 16.7ms. The following describes two cases with accompanying examples: Case 1, see Figure 5 The diagram shows a timing sequence for synchronizing multiple signal sources. Figure 5 The diagram illustrates the ideal scenario where all signal sources are properly synchronized. In this scenario, the first timestamps of signals 1, 2, and 3 deviate from the same reference time (Genlock reference clock) by 0.3ms, 0.5ms, and 0.8ms, respectively (the waiting time for the next frame of signal 3 is less than 16.7ms). All three deviations are less than the preset deviation threshold of 1ms. Therefore, the main control synchronization module determines that the timestamps of these three signals are aligned and then outputs the frame data of each signal uniformly through the synchronization gate. Because each signal achieves precise synchronization at the frame level, when the images from different signal sources are fused and output to the drive control system for display, the timing of the images at the projection fusion band is completely consistent, thus eliminating image tearing and fusion band flickering caused by frame timing misalignment.

[0056] Case 2, see Figure 6 The diagram shown is another example of multi-source signal synchronization timing provided in an embodiment of this application. Figure 6 The diagram illustrates an anomaly where a signal source experiences a timeout. For example, the current frame of signal 3 (frame 2 in the diagram) fails to arrive for an extended period due to network jitter or encoding delay, exceeding 16.7ms. The main control synchronization module determines that the current frame of signal 3 has timed out, skips the frame, and ceases waiting. To ensure the continuity of the fused output, the main control synchronization module can employ the following strategies: copy the previous frame of signal 3 (frame 1 in the diagram) as the current frame (shown in the diagram), aligning it with the current frames of signals 1 and 2 (frame 2 in the diagram); or, it can output a black frame or a static image as the current frame of signal 3, aligning it with the current frames of signals 1 and 2.

[0057] The technical effects of this timeout frame processing strategy are: improved system robustness; when a signal source experiences a timeout due to network jitter or encoding delay, other signal sources are not blocked by skipping frames or copying the previous frame; the overall output frame rate is kept stable, avoiding a drop in the overall output frame rate due to a timeout of a signal source; and the visual discontinuity caused by skipping frames is reduced by copying the previous frame.

[0058] In S203, the rendering server will complete the frame-level synchronization and alignment of each original video signal, and convert it into a target video signal with a preset target output frame rate through an adaptive frame repetition strategy.

[0059] The method of converting a target video signal to a preset target output frame rate using an adaptive frame repetition strategy includes: obtaining the original frame rate of the original video signal and obtaining the target output frame rate; determining the number of repetitive frames to be inserted into the original video signal based on the ratio of the target output frame rate to the original frame rate; and inserting repetitive frames into the original video signal according to the number of repetitive frames to be inserted to generate the target video signal.

[0060] Specifically, the frame rate adaptive conversion module obtains the original frame rate of the original video signal. The original frame rate can be read from the metadata of the video signal or calculated by analyzing the timestamps of consecutive frames. For example, the original frame rate of the original video signal output by the movie playback server is 24fps, the original frame rate of the original video signal output by the MR interactive server is 60fps, and the original frame rate of the original video signal output by the sports live streaming server is 50fps.

[0061] The frame rate adaptive conversion module obtains the target output frame rate. The target output frame rate is the output refresh rate of the rendering server, which can be fixed at 120Hz. When selecting the target output frequency, it can be determined based on the least common multiple of the original frame rates output by the current business server, or an approximate least common multiple. For example, if the original frame rates output by the business server are 24fps, 30fps, 48fps, 50fps, and 60fps, 120 is a least common multiple of 24, 30, and 60, and an approximate least common multiple of 48 and 50. Furthermore, 120fps is compatible with the refresh rates of mainstream LED displays, providing smooth motion performance.

[0062] The number of duplicate frames inserted into the original video signal is determined based on the ratio of the target output frame rate to the original frame rate. Specifically, the ratio is calculated as: Ratio = Target Output Frame Rate / Original Frame Rate. For example, if the original frame rate is 24fps and the target output frame rate is 120Hz, the ratio = 120 / 24 = 5.

[0063] Determining the number of duplicate frames inserted into the original video signal based on the ratio of the target output frame rate to the original frame rate includes: when the ratio of the target output frame rate to the original frame rate is an integer N, determining that the number of duplicate frames inserted after each original frame is N-1; when the ratio of the target output frame rate to the original frame rate is not an integer, determining different numbers of duplicate frames to be inserted after at least two consecutive original frames.

[0064] Case 1: When the ratio of the target output frame rate to the original frame rate is an integer N, the number of duplicate frames to be inserted after each original frame is determined to be N-1.

[0065] For example, the original frame rate is 24fps, the target output frame rate is 120Hz, and the ratio is 5 (an integer). The number of duplicate frames inserted after each original frame is determined to be 5-1=4. Specifically, 4 duplicate frames are inserted after the first original frame, resulting in frames 1-5; 4 duplicate frames are inserted after the second original frame, resulting in frames 6-10; and so on. See also... Figure 7 The diagram illustrates the frame rate adaptive strategy (24Hz→120Hz). Figure 7 As shown, the 24fps input repeats in a 5:5:5:5:5 pattern, that is, each original frame is repeated 5 times (including the original frame itself once + the inserted repeated frames 4 times), which is converted to 120fps.

[0066] For example, if the original frame rate is 30fps and the target output frame rate is 120Hz, the ratio is 4 (an integer). The number of repeating frames inserted after each original frame is determined to be 4-1=3. Specifically, 3 repeating frames are inserted after the first original frame, resulting in frames 1-4; 3 repeating frames are inserted after the second original frame, resulting in frames 5-8; and so on. That is, the 30fps input repeats in a 4:4 pattern, meaning each original frame appears 4 times (including the original frame itself once + the 3 times the inserted repeating frames), which is converted to 120fps.

[0067] For example, if the original frame rate is 60fps and the target output frame rate is 120Hz, the ratio is 2 (an integer). The number of duplicate frames inserted after each original frame is determined to be 2-1=1. Specifically, one duplicate frame is inserted after the first original frame, resulting in frames 1-2; one duplicate frame is inserted after the second original frame, resulting in frames 3-4; and so on. The 60fps input repeats in a 2:2 pattern, meaning each original frame appears twice (including the original frame itself once plus the inserted duplicate frame once), converting it to 120fps.

[0068] Case 2: When the ratio of the target output frame rate to the original frame rate is not an integer, determine the different number of duplicate frames to be inserted after at least two consecutive original frames.

[0069] The target output frame rate is 120Hz; when the ratio of the target output frame rate to the original frame rate is not an integer, the different number of duplicate frames inserted after at least two consecutive original frames is determined, including: when the original frame rate is 48Hz, the total number of duplicate frames inserted after two consecutive original frames is determined to be 3; when the original frame rate is 50Hz, the total number of duplicate frames inserted after five consecutive original frames is determined to be 7.

[0070] Case 2.1: When the original frame rate is 48Hz, the total number of duplicate frames inserted after two consecutive original frames is determined to be 3.

[0071] Specifically, such as Figure 8 The diagram illustrates an adaptive frame rate strategy. The original frame rate is 48fps, and the target output frame rate is 120Hz. The ratio is 120 / 48 = 2.5 (not an integer). A 2:3 cyclical pattern is used: after the first original frame, one duplicate frame is inserted (repeated twice), resulting in frames 1-2; after the second original frame, two duplicate frames are inserted (repeated three times), resulting in frames 3-5, and this pattern continues. Alternatively, two duplicate frames can be inserted after the first original frame, and one duplicate frame after the second. The total number of duplicate frames inserted after two consecutive original frames (frame 1 and frame 2) is 1 + 2 = 3 frames. Since 48fps × 2.5 = 120fps, every two original frames correspond to 5 output frames, meaning the total number of duplicate frames inserted after two consecutive original frames is 3.

[0072] Case 2.2: When the original frame rate is 50Hz, the total number of duplicate frames inserted after 5 consecutive original frames is determined to be 7.

[0073] Specifically, the original frame rate is 50fps, and the target output frame rate is 120Hz, with a ratio of 120 / 50 = 2.4 (not an integer). A 2:2:2:3:3 cyclic pattern can be used: the first original frame appears twice, the second original frame appears twice, the third original frame appears twice, the fourth original frame appears three times, and the fifth original frame repeats this pattern three times, and so on. Five consecutive original frames will output 2 + 2 + 2 + 3 + 3 = 12 frames.

[0074] Of course, the order in which the repetitions occur can be set flexibly, for example, a 3:3:2:2:2 cycle pattern can be used for repetition.

[0075] The target video signal is generated by inserting repeating frames into the original video signal according to the number of repeating frames to be inserted. Specifically, the frame rate adaptive conversion module inserts a corresponding number of repeating frames after each original frame of the original video signal according to the repeating pattern determined above. The content of the repeating frames is exactly the same as that of the original frames, that is, it copies the pixel data of the original frames. For example, for a 24fps input, four repeating frames (all with pixel data of Frame1) are inserted after the first original frame (pixel data of Frame1), and the output is frames 1-5 (Frame1, Frame1, Frame1, Frame1, Frame1).

[0076] The aforementioned adaptive frame repetition strategy unifies the heterogeneous frame rate signals to a 120Hz output clock domain. The technical benefits of this strategy include: minimizing motion discontinuities introduced by frame repetition. For frame rates with integer ratios (e.g., 24fps, 30fps, 60fps), the number of repetitions for each original frame is fixed, resulting in optimal motion continuity. For frame rates with non-integer ratios (e.g., 48fps, 50fps), a cyclical pattern is used, alternating between 2 and 3 repetitions, resulting in good motion continuity and minimal visual jitter. This ensures compatibility between the output frame rate and the refresh rate of the drive control system and the LED display screen. Using 120Hz as the target output frame rate ensures compatibility with the refresh rates of mainstream LED displays (typically 60Hz, 120Hz, or higher), providing smooth motion performance.

[0077] In S204: The rendering server fuses the multi-channel target video signals after frame rate conversion and outputs them synchronously to the drive control system for display.

[0078] Specifically, the fusion processing unit in the rendering server performs fusion processing on the multiple target video signals after frame rate conversion. The fusion processing includes layer overlay, geometric correction, and edge blending.

[0079] Layer overlay: The fusion processing unit can overlay multiple target video signals according to a preset layer order. For example, in a stage concert hall application scenario, interactive content can be used as the foreground (highest on the Z-axis), stage visual effects as the midground, and lighting signals as the background. Layer overlay can use alpha blending, where the transparency parameter of the foreground layer determines the visibility of the background layer.

[0080] Geometric Correction: The fusion processing unit can call pre-calibrated geometric correction parameters to geometrically deform the superimposed image, adapting it to the curved shape of the LED dome. The geometric correction parameters include an image deformation mapping table based on the dome's surface model. This mapping table can be implemented using a Bezier grid or UV mapping, calculated based on the dome's center coordinates, radius, and projector position. For example, for a 20-meter diameter dome, the center coordinates are (0, 0, 10) (assuming the bottom of the dome is on the ground and the center height is 10 meters), and the radius is 10 meters. Projectors are located outside the dome, projecting onto the dome surface from different angles using multiple projectors. The purpose of geometric correction is to deform the planar image into a curved image, so that the image projected onto the dome surface appears as a normal planar image from the viewer's perspective.

[0081] Edge blending: The core rendering server calls pre-calibrated edge blending parameters to process the final image, eliminating uneven brightness and color caused by performance differences between different video splicing processors or LED display modules, thus achieving overall visual consistency of the screen. Edge blending parameters mainly include: Brightness decay curve (Gamma curve): Corrects the non-linearity of brightness for different display areas or modules to ensure a consistent brightness transition from low gray to high gray. This is usually achieved through a global Gamma value (such as 2.2) and local gain / offset fine-tuning.

[0082] Color consistency calibration matrix: calibrates the color coordinates of different display areas to ensure that white and all primary colors are uniform and without color difference across the entire dome screen.

[0083] Black level compensation coefficient: calibrates the black reference of each display area to avoid problems such as "grayish dark areas" or "inconsistent dark area colors".

[0084] After completing layer overlay, geometric correction, and edge blending, the fusion processing unit outputs the data to the drive control system via a multi-channel output interface. The output interface uses an SDI interface or a fiber optic interface, supporting high-bandwidth, low-latency video signal transmission.

[0085] All output timing is locked to the synchronization reference signal. Specifically, the output refresh rate of the fusion processing unit is fixed at 120Hz, and the output timing is locked to the rising edge of the synchronization reference signal output by the synchronization signal generator. For example, if the rising edge of the Nth cycle of the synchronization reference signal occurs at time T0, the fusion processing unit outputs the fused video signal of the Mth frame at time T0. This unified locking of output timing ensures an end-to-end frame-level synchronization path from the service server output → acquisition interface → rendering server processing → drive control system → LED dome display terminal.

[0086] The drive control system receives the video signal output from the rendering server, performs video signal splicing processing, and then drives the LED dome display terminal for display. The refresh rate of the LED dome display terminal can be synchronized with the output refresh rate of the rendering server, both being 120Hz, thus ensuring the smoothness and synchronization of the image.

[0087] In a specific implementation, a large-scale stage concert hall uses an LED dome screen as the stage backdrop. It requires simultaneous access to: 60fps real-time visual effects (stage performance server, running Notch software); 60fps lighting control signals (lighting server); and 30fps interactive content (interactive server, for audience mobile phone interaction).

[0088] Special requirements include: real-time visual effects synchronized with music beats, with a synchronization error of less than 1 frame; lighting effects linked with visual effects; low-latency response of interactive content, with an end-to-end latency of less than 100ms.

[0089] The implementation process includes: The first step is to deploy the system according to the embodiment of this application. This includes configuring a synchronization signal generator (outputting a 60Hz synchronization reference), 3 service servers (stage performance server, lighting server, and interactive server), 1 rendering server (configured with 4 SDI acquisition cards, 16 inputs, using 3 of them), a drive control system (16 inputs), and an LED dome display terminal.

[0090] The second step involves the stage performance server running the Notch software, outputting a 60fps SDI signal locked to the synchronization reference signal. The signal resolution is 1920×1080. The signal content includes stage lighting effects, particle effects, background animations, etc.

[0091] The third step involves the lighting server controlling stage lights, such as LED PAR lights and moving head lights, via the DMX512 protocol. Simultaneously, it outputs visual lighting control signals via the SDI interface. The signal operates at 60fps with a resolution of 1920×1080. The signal content is a visual representation of the light's color and brightness.

[0092] The fourth step involves the interactive server receiving interactive data from viewers' mobile phones. Viewers upload their interactive data via WiFi or 5G networks. The interactive server then generates interactive visuals in real time, such as a wall of viewer avatars, live comments, and voting results. It outputs a 30fps SDI signal at a resolution of 1920×1080.

[0093] Fifth, the rendering server completes frame-level synchronization. The SDI acquisition card acquires three signals using DMA. Each signal is allocated a dual buffer. The main control synchronization module timestamps the buffered frames based on the synchronization reference signal.

[0094] Step 6, Frame Rate Conversion. 30fps interactive content is converted to 120fps by repeating in a 4:4 ratio, appearing 4 times per frame. 60fps signals are converted to 120fps by repeating in a 2:2 ratio, appearing twice per frame.

[0095] Step 7, Blending Process. Layer Overlay: Interactive content as foreground (highest Z-axis), visual effects as midground, and light signals as background. Geometric Correction: A pre-calibrated dome surface deformation mapping table is used, calculated based on the dome's center coordinates, radius, and projector position. Edge Blending: A brightness gradient curve is applied to the overlapping areas of the 16 outputs, using a Gamma=2.2 gradient curve.

[0096] Step 8: Synchronous Output. The rendering server outputs to the drive control system via 16 SDI interfaces. Each channel has a resolution of 1920×1080, resulting in a total resolution of 8192×4096 after splicing. All output timings are locked to the synchronization reference signal.

[0097] The ninth step is to drive the LED dome display terminal through the drive control system to achieve synchronous fusion display of the three signals.

[0098] Compared with the prior art, the embodiments of this application have the following beneficial effects: 1. By adopting a distributed architecture with multiple business servers and rendering servers, the business functions and rendering functions are decoupled. Each business server independently runs its business logic and outputs video signals to the rendering server, supporting dynamic expansion of the business servers. The rendering server's acquisition interface reserves idle interfaces. The main control synchronization program automatically allocates dual buffers for new signal sources and includes them in the synchronization alignment management queue by detecting changes in the input status of the acquisition interface, achieving seamless business expansion and meeting the flexible operation needs of modern venues for "multi-purpose use". Compared with the shortcomings of existing dedicated dome screen players that only support pre-rendered source playback and cannot access real-time signal sources, and the problem that single-server solutions are limited by single-machine rendering capabilities and cannot simultaneously meet the needs of ultra-high resolution rendering and multi-service expansion, the embodiments of this application significantly improve the system's scalability and business compatibility.

[0099] 2. A unified synchronization reference signal is generated by a synchronization signal generator. The output frame timing of all service servers and the output refresh rate of the rendering server are locked to this synchronization reference signal, providing a unified hardware time reference for multiple heterogeneous signal sources and ensuring the timing consistency of each signal at the physical level. Combined with the deterministic acquisition method of the acquisition interface, low-latency acquisition of video frame data is achieved. A ping-pong mechanism with dual buffers allocates a first buffer and a second buffer to each signal, alternating between reading and writing, absorbing the frame arrival time fluctuations caused by network jitter, encoding delays, and other factors, providing a time buffer window for synchronization alignment. The main control synchronization program timestamps each buffered frame with the synchronization reference signal as the time reference. When the time deviation of all signal sources meets a preset threshold, the frame data of each signal is output uniformly, achieving frame-level synchronization alignment of multiple heterogeneous signals. The embodiments of this application are significantly superior to existing technical solutions in terms of synchronization determinism, long-term stability, and picture quality, effectively solving the problems of image tearing and flickering caused by frame timing asynchrony when multiple source signals are accessed in existing LED dome display systems.

[0100] 3. An adaptive frame repetition strategy selects the repetition pattern based on the input signal frame rate, uniformly converting heterogeneous signals with different frame rates to the target output frame rate. The rendering server's output refresh rate is fixed at 120Hz, which is a common multiple or close to a common multiple of common input frame rates and is compatible with the refresh rates of the drive control system and the LED display screen. Different frame repetition patterns are used for different input frame rates, minimizing motion discontinuity introduced by frame repetition. This embodiment significantly improves the system's frame rate compatibility and image smoothness.

[0101] 4. After layer overlay, geometric correction, and edge blending are completed through fusion processing, the data is synchronously output to the drive control system. All output timings are locked to the synchronization reference signal, achieving an end-to-end frame-level synchronization path from the business server output to the drive control system. While ensuring frame-level synchronization, the geometric correction and edge blending required for irregularly shaped displays are completed, ensuring an immersive viewing experience. Compared to existing technologies where each server clock is independent and lacks a unified synchronization reference, leading to inconsistent timing of multiple signal outputs and image tearing and flickering at the seams of the dome screen, this application significantly improves the synchronization and immersion of the dome screen display.

[0102] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0103] Figure 9 This is a schematic diagram of a multi-source signal fusion display device proposed in an embodiment of this application. The system for fusion display includes a rendering server and multiple service servers. The device includes: The raw video signal generation unit 901 is used to generate and output raw video signals with different frame rates by the service server.

[0104] The synchronization alignment unit 902 is used by the rendering server to collect the original video signals output by each service server through multiple built-in acquisition interfaces, and to perform frame-level synchronization alignment processing on the collected multi-channel video signals.

[0105] The frame rate conversion unit 903 is used by the rendering server to convert the original video signals that have completed frame-level synchronization and alignment into target video signals with a preset target output frame rate through an adaptive frame repetition strategy.

[0106] The fusion display unit 904 is used by the rendering server to fuse multiple target video signals after frame rate conversion and synchronously output them to the drive control system for display.

[0107] Figure 9The multi-source signal fusion display device shown is, with Figure 2 The multi-source signal fusion display shown corresponds to this.

[0108] Figure 10 This is a schematic diagram of a multi-source signal fusion display system provided in an embodiment of this application. Figure 10 As shown, the multi-source signal fusion display system 10 of this embodiment includes: a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and executable on the processor 100, such as a multi-source signal fusion display program. When the processor 100 executes the computer program 102, it implements the steps in the various multi-source signal fusion display method embodiments described above. Alternatively, when the processor 100 executes the computer program 102, it implements the functions of each module / unit in the various device embodiments described above.

[0109] For example, the computer program 102 may be divided into one or more modules / units, which are stored in the memory 101 and executed by the processor 100 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 102 in the multi-source signal fusion display system 10.

[0110] The multi-source signal fusion display system may include, but is not limited to, a processor 100 and a memory 101. Those skilled in the art will understand that... Figure 10 This is merely an example of a multi-source signal fusion display system 10 and does not constitute a limitation on the multi-source signal fusion display system 10. It may include more or fewer components than shown, or combine certain components, or different components. For example, the multi-source signal fusion display system may also include input / output devices, network access devices, buses, etc.

[0111] The processor 100 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0112] The memory 101 can be an internal storage unit of the multi-source signal fusion display system 10, such as a hard disk or memory of the multi-source signal fusion display system 10. The memory 101 can also be an external storage device of the multi-source signal fusion display system 10, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the multi-source signal fusion display system 10. Furthermore, the memory 101 can include both internal storage units and external storage devices of the multi-source signal fusion display system 10. The memory 101 is used to store the computer program and other programs and data required by the multi-source signal fusion display system. The memory 101 can also be used to temporarily store data that has been output or will be output.

[0113] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0114] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0115] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0117] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0118] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0119] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0120] In addition, this application also provides a computer program product that, when run on a computer, causes the computer to execute the methods in the above-described implementations.

[0121] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multi-source signal fusion display method, characterized in that, The system for fused display includes a rendering server and multiple service servers, and the method includes: The service server generates and outputs raw video signals with different frame rates; The rendering server acquires the raw video signals output by each service server through multiple built-in acquisition interfaces, and performs frame-level synchronization and alignment processing on the acquired multi-channel video signals. The rendering server will complete the frame-level synchronization and alignment of each original video signal, and convert it into a target video signal with a preset target output frame rate through an adaptive frame repetition strategy. The rendering server fuses the multi-channel target video signals after frame rate conversion and outputs them synchronously to the drive control system for display.

2. The method according to claim 1, characterized in that, The target video signal is converted to a preset target output frame rate through an adaptive frame repetition strategy, including: Obtain the original frame rate of the original video signal, and obtain the target output frame rate; The number of duplicate frames inserted into the original video signal is determined based on the ratio of the target output frame rate to the original frame rate. The target video signal is generated by inserting repeating frames into the original video signal according to the number of repeating frames to be inserted.

3. The method according to claim 2, characterized in that, Determining the number of duplicate frames to be inserted into the original video signal based on the ratio of the target output frame rate to the original frame rate includes: When the ratio of the target output frame rate to the original frame rate is an integer N, the number of duplicate frames inserted after each original frame is determined to be N-1. When the ratio of the target output frame rate to the original frame rate is not an integer, a different number of duplicate frames to be inserted after at least two consecutive original frames is determined.

4. The method according to claim 3, characterized in that, The target output frame rate is 120Hz; When the ratio of the target output frame rate to the original frame rate is not an integer, determining the different number of duplicate frames inserted after at least two consecutive original frames includes: When the original frame rate is 48Hz, the total number of duplicate frames inserted after two consecutive original frames is determined to be 3. When the original frame rate is 50Hz, the total number of repeated frames inserted after 5 consecutive original frames is determined to be 7.

5. The method according to claim 1, characterized in that, The rendering server acquires video signals output by each service server through multiple built-in acquisition interfaces, including: When a new signal source is detected, at least a first buffer and a second buffer are allocated for the video signal of each new signal source. When a predetermined number of image frames have been stored in the first buffer, the image in the first buffer is read and the received image is written through the second buffer. When a predetermined number of image frames have been stored in the second buffer, the image in the second buffer is read and the received image is written through the first buffer.

6. The method according to claim 1, characterized in that, Frame-level synchronization and alignment processing is performed on the acquired multi-channel video signals, including: A synchronization reference signal is generated by a synchronization signal generator; Obtain the first timestamp of the buffered frames of the multiple video signals and the second timestamp of the synchronization reference signal, and determine the time deviation between the first timestamp and the second timestamp; The multi-channel video signals are aligned at the frame level based on the time deviation and a preset deviation threshold.

7. The method according to claim 6, characterized in that, The multi-channel video signals are aligned at the frame level based on the time deviation and a preset deviation threshold, including: When the time deviation of all video signals is less than or equal to the deviation threshold, it is determined that all video signals have completed the synchronization alignment process, and the aligned video signal is output. If the time deviation of any video signal is greater than the deviation threshold, the system waits for the next frame of that video signal to perform deviation verification, or if the waiting time exceeds a predetermined duration threshold, the frame is determined to have timed out and is skipped.

8. A multi-source signal fusion display device, characterized in that, The system for fused display includes a rendering server and multiple service servers, and the device includes: The raw video signal generation unit is used to generate and output raw video signals with different frame rates by the service server. The synchronization alignment unit is used by the rendering server to acquire the original video signals output by each service server through multiple built-in acquisition interfaces, and to perform frame-level synchronization alignment processing on the acquired multi-channel video signals. The frame rate conversion unit is used by the rendering server to convert the original video signals that have completed frame-level synchronization and alignment into a target video signal with a preset target output frame rate through an adaptive frame repetition strategy. The fusion display unit is used by the rendering server to fuse multiple target video signals after frame rate conversion and synchronously output them to the drive control system for display.

9. A multi-source signal fusion display system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the multi-source signal fusion display system to implement the method as described in any one of claims 1-7.

10. A computer program product comprising computer program instructions, characterized in that, When the computer program is run, the method as described in any one of claims 1-7 is performed.

Images