An apparatus and method for multi-source video fusion processing

Abstract

The present application discloses a method for multi-source video fusion processing, which relates to the field of video fusion and includes: collecting multi-source video data; using the preprocessed multi-source video data as input, cascading multiple IPG gateways to form a signal scheduling matrix, and performing first scheduling and fusion processing on the multi-source video data; setting up a master control cross-point matrix of 256x256; using the signal processed by the signal scheduling matrix as the input signal, and performing secondary scheduling and processing on the input signal by controlling the switching and scheduling of virtual cross-points; creating different cross-point applications for each output channel in the master control cross-point matrix according to different application scenarios; customizing hierarchical grouping for the output channels of the master control cross-point matrix through a switching control module, setting an output channel in multiple groups according to different application scenarios, and creating new groups according to the preset groups; aiming at the poor scalability of video fusion processing devices in the prior art, the present application improves scalability.

Description

Technical Field

[0001] This application relates to the field of video fusion, and particularly to a device and method for multi-source video fusion processing. Background Art

[0002] With the continuous expansion of the video application field, the requirements for video processing devices in scenarios such as video surveillance, command and dispatch, live broadcast, and remote conferencing are getting higher and higher. Most traditional video processing devices adopt a single input / output interface, such as SDI or HDMI, with poor scalability and are difficult to meet the needs of large-scale and diversified video fusion processing.

[0003] Specifically, the existing video fusion processing devices have the following deficiencies: First, the types of video input / output interfaces are single, usually only supporting a single type such as SDI or HDMI, and cannot be compatible with other heterogeneous video sources, resulting in limited applicability of the devices. Second, the system scalability is poor. When it is necessary to expand the scale of video input / output, limited by the hardware architecture, major modifications need to be made to the device, with high expansion costs and low flexibility.

[0004] Therefore, there is an urgent need for a new type of video fusion processing device that can support the access and fusion of multi-source heterogeneous videos, and has flexible expansion capabilities and dynamic scheduling mechanisms to adapt to the continuously developing diversified video application requirements. Summary of the Invention

[0005] Aiming at the problem of poor scalability of video fusion processing devices in the prior art, this application provides a device and method for multi-source video fusion processing, which uses cascaded IPG gateways to construct a signal scheduling matrix for the first scheduling and fusion, and then performs secondary scheduling processing through a master control cross-point matrix, and performs grouped switching control on the output channels according to different application scenarios, etc., improving the scalability of the device.

[0006] The objectives of this application are achieved through the following technical solutions.

[0007] One aspect of the present application provides a device for multi-source video fusion processing, including: an acquisition module that acquires multi-source video data, including an SDI or CVBS acquisition unit, a network communication unit, and an interface processing unit, which are respectively used for SDI camera resources, external IP network video media resources, and device desktop media resources; a preprocessing module that preprocesses the acquired multi-source video data; a control module that performs fusion processing on the acquired multi-source video data, including: a signal scheduling matrix composed of multiple cascaded IPG gateways for performing the first scheduling and fusion processing on the multi-source video data; a master control cross-point matrix that sets virtual cross-points according to the output of the signal scheduling matrix for performing secondary scheduling and processing on the signals processed by the signal scheduling matrix; a switching control module that performs custom hierarchical grouping and switching control on the output channels of the master control cross-point matrix; and a transmission module that transmits the data processed by the control module through a wireless communication link, a satellite communication link, or a wired IP communication link to a display device.

[0008] Another aspect of the present application further provides a method for multi-source video fusion processing, including: obtaining SDI camera resources through an SDI or CVBS acquisition unit, obtaining external IP network video media resources through a network communication unit, and obtaining device desktop media resources through an interface processing unit to obtain multi-source video data; preprocessing the multi-source video data; using the preprocessed multi-source video data as input, cascading multiple IPG gateways to form a signal scheduling matrix for performing the first scheduling and fusion processing on the multi-source video data; setting a 256x256 master control cross-point matrix according to the output of the signal scheduling matrix, where the master control cross-point matrix includes virtual cross-points; using the signals processed by the signal scheduling matrix as input signals and inputting them into the master control cross-point matrix, and performing secondary scheduling and processing on the input signals by controlling the switching and scheduling of the virtual cross-points; creating different cross-point applications for each output channel according to different application scenarios in the master control cross-point matrix, where the applications include testing, live broadcast, and standby broadcast; customizing hierarchical grouping of the output channels of the master control cross-point matrix through a switching control module, setting one output channel in multiple groups according to different application scenarios, and creating new groups according to the preset groups; and transmitting the video data processed by the control module through a wireless communication link, a satellite communication link, or a wired IP communication link to a display device for display.

[0009] Further, multi-source video data is obtained, including: obtaining SDI video data or CVBS video data by collecting SDI signals or CVBS signals output by SDI cameras through SDI or CVBS acquisition units; establishing a communication connection with an external IP network through a network communication unit and receiving network video media data transmitted by the external IP network to obtain network video data; establishing a connection with a video acquisition interface inside the device through an interface processing unit and collecting device desktop media data to obtain desktop video data.

[0010] Among them, SDI (Serial Digital Interface) is a widely used digital video interface standard for transmitting uncompressed digital video signals. CVBS (Composite Video Baseband Signal) is an analog video signal, also known as composite video signal. IP network (Internet Protocol Network) is a data communication network based on the Internet Protocol (IP), which realizes data transmission and exchange through interconnected network devices and communication protocols. Desktop video data refers to the video content on the display screen of a computer or other device. Desktop video data includes displayed graphics, text, images, and videos, etc., reflecting the current display state of the device. Through video acquisition interfaces such as VGA, DVI, HDMI, or DisplayPort, etc., the desktop video data of the device can be collected. Network video media data refers to video content data transmitted through an IP network. Network video media data usually adopts specific video coding formats such as H.264, H.265, or VP9, etc., to achieve efficient data compression and transmission. Network video media data can be transmitted through various network protocols such as RTSP (Real-Time Streaming Protocol), RTMP (Real-Time Messaging Protocol), HLS (HTTP Live Streaming), etc. Common network video media data sources include network cameras, video servers, streaming media platforms, etc., and these data can be received and decoded through the network communication unit.

[0011] Further, preprocessing is performed on the multi-source video data, including: performing encoding processing on SDI video data, CVBS video data, network video data, and desktop video data to obtain encoded multi-source video data, where audio data is encoded using the AAC encoding algorithm, and video data is encoded using the H.264 encoding algorithm; encapsulating the encoded multi-source video data using a streaming media transmission protocol; the streaming media protocol includes at least one of the RTMP protocol, HLS protocol, and HTTP-FLV protocol.

[0012] Among them, the AAC encoding algorithm (Advanced Audio Coding), AAC is an efficient audio compression encoding algorithm used to compress audio data into a smaller data volume while maintaining high-quality audio restoration. The H.264 encoding algorithm (H.264 / MPEG-4 Advanced Video Coding) is an efficient video compression encoding algorithm, also known as MPEG-4 AVC (Advanced Video Coding). The RTMP protocol (Real-Time Messaging Protocol) is an application layer protocol based on TCP used for real-time transmission of audio and video data. The HLS protocol (HTTP Live Streaming) is a streaming media transmission protocol based on HTTP introduced by Apple. HLS divides video content into a series of short-duration video segments (usually 5 - 10 seconds) and generates a playlist file (M3U8) to manage these segments. The HTTP-FLV protocol (HTTP-Flash Video) is a streaming media transmission protocol based on HTTP used for real-time transmission of FLV-format audio and video data.

[0013] Preferably, SDI video data, CVBS video data, network video data, and desktop video data are encoded to obtain encoded multi-source video data. Among them, the audio data is encoded using the AAC encoding algorithm, and the video data is encoded using the H.264 encoding algorithm; the encoded multi-source video data is encapsulated using a streaming media transmission protocol, and the streaming media transmission protocol is at least one of the RTMP protocol, the HLS protocol, and the HTTP-FLV protocol; among them, the encoding process includes: inputting SDI video data, CVBS video data, network video data, and desktop video data into corresponding encoders respectively; encoding the video data using an H.264 encoder to obtain H.264-encoded video data; encoding the audio data using an AAC encoder to obtain AAC-encoded audio data; inputting the H.264-encoded video data and the AAC-encoded audio data into a multiplexer, and using the multiplexer to synthesize the encoded audio and video data and generate an encapsulation format conforming to the streaming media transmission protocol; among them, the encapsulation process of the streaming media transmission protocol includes: when using the RTMP protocol, generating RTMP-encapsulated format data packets, specifically including: splitting the H.264-encoded video data into multiple NALU units, adding NALU header information, and constructing RTMP video data packets; adding ADTS header information to the AAC-encoded audio data to construct RTMP audio data packets; multiplexing the constructed audio and video data packets according to timestamps to generate RTMP stream data.

[0014] When the HLS protocol is adopted, data packets in the TS encapsulation format and the M3U8 index file are generated, specifically including: splitting the H.264-encoded video data into multiple NALU units, adding NALU header information, and constructing TS video data packets; constructing PES audio data packets from the AAC-encoded audio data; multiplexing the constructed audio and video PES packets according to timestamps and dividing them into multiple TS shards, and generating the corresponding M3U8 index file to establish the mapping relationship between the TS shards and the index; when the HTTP-FLV protocol is adopted, data packets in the FLV encapsulation format are generated, specifically including: splitting the H.264-encoded video data into multiple NALU units and constructing FLV video TAGs; constructing FLV audio TAGs from the AAC-encoded audio data; multiplexing the constructed audio and video TAGs alternately according to timestamps to generate FLV stream data. H.264 and AAC are used to efficiently encode video and audio respectively, and combined with mainstream streaming media encapsulation protocols such as RTMP, HLS, and HTTP-FLV, unified encapsulation of multi-source heterogeneous video data is achieved. By adopting mature and common codec and encapsulation standards, the system compatibility and data transmission adaptability can be significantly improved. At the same time, according to the characteristics of different protocols, a differentiated encapsulation processing flow is designed, which improves the flexibility of data encapsulation and transmission while ensuring video quality.

[0015] Further, taking the preprocessed multi-source video data as input, cascading multiple IPG gateways to form a signal scheduling matrix, and performing the first scheduling and fusion processing on the multi-source video data, including: each IPG module has an internal switching matrix, denoted as A(i), where i represents the i-th IPG module, and 1 ≤ i ≤ N. The dimension of the switching matrix A(i) is X×Y, that is, the number of input ports is X and the number of output ports is Y. Input the multi-source video data obtained by the SDI acquisition unit, network communication unit, and interface processing unit into the input interfaces of the corresponding IPG modules respectively. Let the input video data of the i-th IPG module be V(i, j), where 1 ≤ j ≤ X, representing the video data input to the j-th input port of the i-th IPG module. For the i-th IPG module, according to the preset scheduling strategy, by configuring the switching matrix A(i), schedule and fuse the input X-channel video data V(i, j). The scheduling strategy can be to perform operations such as exchanging, synthesizing, or splitting video data from different sources according to a predetermined rule. The output result of the switching matrix A(i) is Y-channel fused video data, denoted as U(i, k), where 1 ≤ k ≤ Y, representing the video data of the k-th output port of the i-th IPG module. Cascade the output data U(i, k) of the N IPG modules to form the output signal of the signal scheduling matrix. Let the cascaded output signal be W(l), where 1 ≤ l ≤ N×Y, representing the video data of the l-th output port of the signal scheduling matrix. The output signal W(l) of the signal scheduling matrix can be expressed as: W(l) = U(i, k), where k = (l - 1) mod Y + 1; in the above formula, represents the ceiling operation, and mod represents the modulo operation.

[0016] Among them, the IPG module (Integrated Processing Gateway) is an integrated video processing device used to implement the input, processing, and output of multiple video signals. The IPG module usually has multiple input interfaces, such as SDI or HDMI interfaces, for receiving video signals from different video sources. The IPG module also has multiple output interfaces for outputting the processed video signals, which can be SDI, HDMI, or network interfaces, etc. The IPG module integrates a video processing unit internally, which can perform various processes on the input video signals, such as encoding, decoding, scaling, mixing, overlaying, etc. The IPG module supports flexible configuration and control, and its functions and parameters can be set and adjusted through software or a control interface. The IPG module can be used alone or cascaded with other IPG modules to build a larger-scale video processing system. The signal scheduling matrix is a video signal processing and distribution system composed of multiple cascaded IPG modules. The signal scheduling matrix forms a matrix structure with multiple input interfaces and output interfaces by interconnecting multiple IPG modules. The input interfaces of the signal scheduling matrix are used to receive multi-source video data from different video sources, such as the video data obtained by the SDI acquisition unit, the network communication unit, and the interface processing unit. By controlling the video processing unit inside the IPG module, the signal scheduling matrix can perform scheduling and fusion processing on the input multi-source video data. The scheduling processing includes routing the input video data to different IPG modules to achieve the distribution and switching of video signals. The fusion processing includes operations such as mixing, overlaying, and picture segmentation on multiple video signals to generate the fused video signal. The output interfaces of the signal scheduling matrix are used to output the video signals after scheduling and fusion processing, and can be connected to display devices, recording devices, or other video processing devices.

[0017] Further, taking the preprocessed multi-source video data as input, cascading multiple IPG gateways to form a signal scheduling matrix, and performing the first scheduling and fusion processing on the multi-source video data, further including: setting 2(N + 1) IPG modules, where 2N IPG modules are used as main channels and 2 IPG modules are used as backup channels. The IPG module numbers of the main channels are M(i), and the IPG module numbers of the backup channels are B(j), where 1 ≤ i ≤ 2N and 1 ≤ j ≤ 2. Each main-channel IPG module M(i) has X input interfaces and Y output interfaces, and each backup-channel IPG module B(j) has X input interfaces and Y output interfaces. Input the preprocessed multi-source video data V(i, k) into the input interfaces of the main-channel IPG module M(i), where 1 ≤ k ≤ X, indicating the video data input to the k-th input port of the i-th main-channel IPG module. The main-channel IPG module M(i) performs the first scheduling and fusion processing on the input video data to obtain Y output signals U(i, m), where 1 ≤ m ≤ Y, indicating the video data at the m-th output port of the i-th main-channel IPG module. To achieve primary-backup switching, input the Y output signals U(i, m) of each main-channel IPG module M(i) into the input interfaces of the corresponding backup-channel IPG module B(j) simultaneously, where that is, the output signals of the 1st to the N-th main-channel IPG modules are input to the backup-channel IPG module B(1), and the output signals of the (N + 1)-th to the 2N-th main-channel IPG modules are input to the backup-channel IPG module B(2). The backup-channel IPG module B(j) performs backup processing on the input main-channel output signals to obtain Y backup output signals R(j, n), where 1 ≤ n ≤ Y, indicating the video data at the n-th output port of the j-th backup-channel IPG module. By controlling the main-channel IPG module M(i) and the backup-channel IPG module B(j), the output signal U(i, m) of the main channel can be switched to the input interface of the corresponding backup-channel IPG module B(j) to achieve the switching and backup of the primary and backup channels. The output signals of the signal scheduling matrix include NxY output signals of the main-channel IPG modules and 2xY output signals of the backup-channel IPG modules, totaling NxY + 2Y output signals, denoted as W(l), where 1 ≤ l ≤ NxY + 2Y. The correspondence between the output signal U(i, m) of the main-channel IPG module and the output signal W(l) of the signal scheduling matrix is: W(l) = U(i, m), where m = (l - 1) mod Y + 1, 1 ≤ l ≤ NxY; the correspondence between the output signal R(j, n) of the backup-channel IPG module and the output signal W(l) of the signal scheduling matrix is: W(l) = R(j, n), where n = (l - NxY - 1) mod Y + 1, NxY + 1 ≤ l ≤ NxY + 2Y.

[0018] Further, according to the output of the signal scheduling matrix, a 256x256 master control cross-point matrix is set up. The master control cross-point matrix includes virtual cross-points. The signal processed by the signal scheduling matrix is used as the input signal and input into the master control cross-point matrix. Through controlling the switching and scheduling of the virtual cross-points, the input signal is secondarily scheduled and processed, including: setting up a 256x256 master control cross-point matrix, denoted as C. The master control cross-point matrix includes virtual cross-points. The virtual cross-points are used to realize the scheduling and switching of signals without involving actual physical connections. The NxY + 2Y path output signals W(l) of the signal scheduling matrix are used as the input signals of the master control cross-point matrix C, denoted as I(l), where 1 ≤ l ≤ NxY + 2Y. Set S channels, and each channel is provided with a primary and a backup channel. Therefore, 2(N + S) IPG modules need to be set up, where 2N IPG modules are for the primary channels and 2S IPG modules are for the backup channels. By controlling the virtual cross-points in the master control cross-point matrix C, the input signal I(l) is secondarily scheduled and processed to obtain 2SxY path output signals O(k, m), which are used as the primary and backup channel signals of the S channels, where 1 ≤ k ≤ 2S, 1 ≤ m ≤ Y. Each channel's primary and backup channels each have Y path signals output. The relationship between the input signal I(l) and the output signal O(k, m) of the master control cross-point matrix C can be expressed as: O(k, m) = C[I(l)], where C[·] represents the mapping relationship of the master control cross-point matrix, which is determined by the connection state of the virtual cross-points. The 2SxY path output signals O(k, m) are respectively output to the corresponding IPG modules, where: the SxY path signals of the primary channels are output to 2N primary channel IPG modules, denoted as M(i), 1 ≤ i ≤ 2N. The SxY path signals of the backup channels are output to 2S backup channel IPG modules, denoted as B(j), 1 ≤ j ≤ 2S. The correspondence between the input signal of the primary channel IPG module M(i) and the output signal O(k, m) of the master control cross-point matrix C is: M(i) = O(k, m), where m = (i - 1) mod Y + 1, 1 ≤ i ≤ 2N; the correspondence between the input signal of the backup channel IPG module B(j) and the output signal O(k, m) of the master control cross-point matrix C is: B(j) = O(k, m), where m = (j - 1) mod Y + 1, 1 ≤ j ≤ 2S.

[0019] The output signals of the signal scheduling matrix are secondarily scheduled and processed using the master control cross-point matrix, and the primary and backup channel signals of S channels are obtained. Each channel has independent primary and backup channels, which can be flexibly switched and scheduled as needed. At the same time, through the setting of virtual cross-points, arbitrary combination and distribution of signals can be achieved, improving the flexibility and reconfigurability of the system. In addition, by outputting the primary and backup channel signals to the corresponding primary channel and backup channel IPG modules respectively, channel-level backup and redundancy can be realized, further improving the reliability and fault tolerance of the system. When a fault occurs in the primary channel of a certain channel, it can be quickly switched to the corresponding backup channel to ensure the continuity and stability of the video transmission of that channel.

[0020] Among them, a channel corresponds to an independent video signal output channel in this solution, and each channel can be set with one or more output signals according to application requirements. For example, in the broadcast system of a television station, each channel can correspond to a TV program, such as a news channel, a sports channel, a movie channel, etc.; in the video transmission system of a large event or conference, each channel can correspond to different venues or different perspectives, such as a main venue channel, a sub-venue channel, a panoramic channel, a close-up channel, etc. The setting of channels is closely related to application requirements. In different application scenarios, the number and content of channels may vary. For example, in the television station broadcast system, the number of channels may be relatively large, and the content of each channel is relatively fixed; while in the video transmission system of a large event or conference, the number of channels may be relatively small, but the content of each channel may change at any time, requiring real-time scheduling and switching. In this solution, in order to improve the reliability and availability of the system, each channel is provided with primary and backup channels in both the master control and broadcast. The primary channel is used for normal signal transmission and broadcast, and the backup channel is used to take over the work of the primary channel when the primary channel fails or needs maintenance to ensure the continuity and stability of the signal.

[0021] Among them, the master control cross-point matrix is a large-scale video signal scheduling and processing system used to achieve flexible scheduling, switching, and backup of multiple video signals. The master control cross-point matrix usually adopts the concept of virtual cross-points, that is, logically connecting and switching video signals through software or a control system, rather than physically hardware connections. The scale of the master control cross-point matrix can be set according to requirements. For example, 256x256 means the matrix has 256 input ports and 256 output ports. In the master control cross-point matrix, video signals can be routed from any input port to any output port to achieve flexible signal scheduling and distribution. The master control cross-point matrix can be connected to the IPG module, taking the output signal of the IPG module as the input of the matrix and transmitting the output signal of the matrix to the IPG module for further processing. By controlling the virtual cross-points in the master control cross-point matrix, the input video signals can be rescheduled and processed to generate multiple output signals. The master control cross-point matrix supports the setting of primary and backup channels, that is, outputting the signals of the same channel to the primary channel and the backup channel simultaneously to achieve signal redundancy backup. The primary channel and the backup channel can be connected to different IPG modules to process and transmit signals respectively, improving the reliability and fault tolerance of the system.

[0022] Further, according to different application scenarios in the master control cross-point matrix, different cross-point applications are created for each output channel. The applications include testing, live broadcast, and standby broadcast. In the master control cross-point matrix C, different application scenarios are created for each output channel, including testing, live broadcast, and standby broadcast. By using the SDI interface, HDMI interface, and IP interface of the IPG module, the switching and scheduling of SDI signals, HDMI signals, and IP signals are carried out. According to different application scenarios, by combining the cross-points in the master control cross-point matrix C, virtual cross-point matrices of different sizes are created, denoted as V_test, V_live, and V_backup respectively, corresponding to the testing, live broadcast, and standby broadcast application scenarios. The output signal O(k, m) of the master control cross-point matrix C is used as the input signal and input into the corresponding virtual cross-point matrix. According to different application scenarios, the signal is processed and controlled twice through the virtual cross-point matrix: Testing application scenario: The signal is tested and monitored through V_test, and signal quality parameters such as bit error rate and signal-to-noise ratio are recorded to achieve fault detection and alarm. Live broadcast application scenario: The main and standby signals are switched and scheduled through V_live, and the optimal signal source is dynamically selected according to the preset strategy or manual intervention to ensure the smoothness and stability of the live broadcast. Standby broadcast application scenario: The SDI, HDMI, and IP signals are backed up through V_backup, and the key video sources are redundantly stored so that they can be quickly restored in case of a failure. Through the control of the virtual cross-point matrix, the output signal O(k, m) of the master control cross-point matrix C is processed twice to obtain the output signals corresponding to the application scenarios, denoted as follows: Output signal for the testing application scenario: T(p, q), 1 ≤ p ≤ P, 1 ≤ q ≤ Q; Output signal for the live broadcast application scenario: L(r, s), 1 ≤ r ≤ R, 1 ≤ s ≤ S; Output signal for the standby broadcast application scenario: B(u, v), 1 ≤ u ≤ U, 1 ≤ v ≤ V; where P, Q, R, S, U, and V respectively represent the number of output channels and the number of output paths for each channel in different application scenarios. The processed output signals are output through the SDI interface, HDMI interface, and IP interface of the IPG module, denoted as follows: Testing application scenario: T(p, q) is output to the testing device and monitoring device. Live broadcast application scenario: L(r, s) is output to the encoder, transmission device, and broadcast device. Standby broadcast application scenario: B(u, v) is output to the storage device and standby broadcast device. The relationship between the virtual cross-point matrix and the input and output signals can be expressed as follows: Testing application scenario: T(p, q) = V_test[O(k, m)], where V_test[·] represents the mapping relationship of the virtual cross-point matrix in the testing application scenario. Live broadcast application scenario: L(r, s) = V_live[O(k, m)], where V_live[·] represents the mapping relationship of the virtual cross-point matrix in the live broadcast application scenario.Standby broadcast application scenario: B(u, v) = V_backup[O(k, m)], where V_backup[·] represents the virtual crosspoint matrix mapping relationship in the standby broadcast application scenario. The virtual crosspoint matrix is used to perform secondary processing and control on the output signals of the master control crosspoint matrix. According to different application scenarios, virtual matrices for testing, live broadcast, and standby broadcast are dynamically created to flexibly schedule signal resources and meet various application requirements.

[0023] Among them, the virtual crosspoint matrix is a logical matrix created by combining the crosspoints in the master control crosspoint matrix according to different application scenarios based on the master control crosspoint matrix. The virtual crosspoint matrix is a flexible video signal scheduling and processing mechanism that can be dynamically created and configured according to actual needs. Each virtual crosspoint matrix can have different sizes and functions for implementing video signal processing and control in specific application scenarios. The virtual crosspoint matrix receives the output signals of the master control crosspoint matrix as inputs, performs secondary processing and scheduling on the input signals, and generates output signals corresponding to the application scenarios. The virtual crosspoint matrix can be configured and controlled through software or a control system to achieve flexible signal switching, scheduling, and processing. The virtual crosspoint matrix can be connected to different interfaces of the IPG module (such as SDI interface, HDMI interface, and IP interface) to achieve switching and scheduling of different types of signals.

[0024] Further, the switching control module performs custom hierarchical grouping on the output channels of the master control cross-point matrix. According to different application scenarios, an output channel is set in multiple groups, and new groups are created according to the preset groups, including: grouping and managing the output channels of the master control cross-point matrix C, and performing scenario settings on the output channels according to different application scenarios. Preset multiple groups, and each group corresponds to a unique group number. There are G groups, and the group number is G(i), where 1 ≤ i ≤ G. Preset multiple application scenarios, and each application scenario corresponds to a unique scenario number. There are S application scenarios, and the scenario number is S(j), where 1 ≤ j ≤ S. Establish a mapping table M between the application scenarios and the groups, and set the mapping relationship between the output channels and the groups under different application scenarios in the mapping table M. The rows of the mapping table M represent the application scenarios, the columns represent the groups, and the element M(j, i) represents the mapping relationship that the output channel belongs to the group G(i) under the application scenario S(j), and the value is 0 or 1, where: M(j, i) = 1 means that under the application scenario S(j), the output channel belongs to the group G(i); M(j, i) = 0 means that under the application scenario S(j), the output channel does not belong to the group G(i). Set the output channel configuration table T, and set the corresponding relationship between the output channel entries and the group numbers in the output channel configuration table T, and associate an output channel entry with multiple group numbers. The rows of the output channel configuration table T represent the output channel entries, the columns represent the group numbers, and the element T(k, i) represents the corresponding relationship between the output channel entry k and the group number G(i), and the value is 0 or 1, where: T(k, i) = 1 means that the output channel entry k is associated with the group number G(i); T(k, i) = 0 means that the output channel entry k is not associated with the group number G(i). The switching control module queries the mapping table M between the application scenarios and the groups according to the current application scenario S(j), obtains the mapping relationship between the output channels and the groups in this scenario, and gets a group mapping vector M_j, where M_j(i) = M(j, i). The switching control module dynamically generates new groups according to the group mapping vector M_j and the output channel configuration table T, and assigns the output channel entries to the corresponding groups. For each output channel entry k, it is grouped according to the following rules: if there exists a group number G(i) such that M_j(i) = 1 and T(k, i) = 1, then the output channel entry k is assigned to the group G(i); if for all group numbers G(i), M_j(i) = 0 or T(k, i) = 0, then the output channel entry k is assigned to a new group, and a new group number is assigned to this group. The switching control module updates the output channel configuration table T and the mapping table M between the application scenarios and the groups according to the generated new groups, reflecting the latest corresponding relationship between the output channel entries and the groups. The switching control module sends the group information to the master control cross-point matrix C to perform group control and switching on the output channels.The master control cross-point matrix C dynamically adjusts the allocation and connection relationship of output channels according to the received packet information to achieve signal scheduling and distribution in different application scenarios. Through the above solution, the custom hierarchical grouping of the output channels of the master control cross-point matrix is realized, and the mapping relationship between the output channels and the groups can be flexibly configured according to different application scenarios. By introducing the mapping table of application scenarios and groups and the output channel configuration table, new groups can be dynamically generated, and an output channel entry can be associated with multiple groups to meet diverse application requirements.

[0025] Further, create a new group according to a preset grouping, including: setting up a grouping configuration table G to store the configuration information of the groups. The configuration information includes the group number and the group name. Each row of the grouping configuration table G represents a group and contains the following fields: G(i).id: the group number, which uniquely identifies a group, and its value range is from 1 to G_max, where G_max is the maximum number of groups; G(i).name: the group name, which is used to describe the attributes or uses of the group and is user-defined. Create a new group in the grouping configuration table G according to the user's instruction to create a new group: the user issues an instruction to create a new group, and the instruction contains the name new_name of the new group; the switching control module queries the grouping configuration table G to find an unused group number new_id, and its value range is from 1 to G_max; add a new row in the grouping configuration table G, and set G(new_id).id = new_id and G(new_id).name = new_name, indicating that a new group is created. Establish an association between the new group number and the output channel entry in the output channel configuration table T, and associate the output channel to the new group: the switching control module selects the output channel entry to be associated to the new group according to the user instruction or preset rules, denoted as T(k); in the output channel configuration table T, set the value of T(k, new_id) to 1, indicating that the output channel entry k is associated to the new group new_id. Generate a grouping control instruction reflecting the grouping relationship of the output channels according to the grouping configuration table G and the output channel configuration table T: the switching control module traverses each row of the grouping configuration table G, and for each group G(i), generates a grouping control instruction C(i); the grouping control instruction C(i) contains the following information: C(i).group_id: the group number, that is, G(i).id; C(i).channel_list: the list of output channel entries belonging to the group G(i), which is generated according to the output channel entry k where T(k, i) = 1 in the output channel configuration table T. The switching control module sends the generated grouping control instruction C(i) to the master control cross-point matrix C. The master control cross-point matrix C synchronously switches and controls the output channels belonging to the same group according to the received grouping control instruction C(i): for each grouping control instruction C(i), the master control cross-point matrix C extracts the list of output channel entries C(i).channel_list belonging to the group; by controlling the cross-point switch, synchronously switch and control the audio and video data of the output channels in C(i).channel_list to achieve independent control of the group G(i); the output channels of different groups can be switched and controlled independently without affecting each other. Through the above solution, creating a new group according to a preset grouping and flexibly associating the output channels to the new group are realized. Through the cooperation of the grouping configuration table and the output channel configuration table, the switching control module can dynamically generate grouping control instructions to reflect the grouping relationship of the output channels.The master control cross-point matrix independently and synchronously switches and controls the output channels of different groups according to the grouping control instructions, meeting diverse grouping control requirements.

[0026] This grouping creation and control mechanism has strong flexibility and scalability. Users can dynamically create and adjust groups according to the actual application scenarios, and flexibly allocate output channels to different groups. Through the control instructions at the group level, independent control of the output channels of different groups can be achieved, improving the usability and maintainability of the system. At the same time, this mechanism also facilitates resource sharing and collaboration between groups, improving the comprehensive utilization efficiency of the system.

[0027] Compared with the prior art, the advantages of this application are as follows:

[0028] By cascading multiple IPG gateways with multiple interface types, a flexibly expandable signal scheduling matrix is constructed. When it is necessary to expand the input / output scale, only the number of IPG gateways needs to be increased, without changing the original system architecture, greatly improving the scalability and flexibility of the system.

[0029] The acquisition module supports obtaining video data of multiple sources and multiple formats such as SDI cameras, IP network videos, and device desktops. Through the encoding and encapsulation of the preprocessing module, heterogeneous data is converted into a unified format. Then, through the two-level scheduling processing of the signal scheduling matrix and the master control cross-point matrix, the efficient fusion of multi-source heterogeneous video data is finally achieved.

[0030] By introducing the master control cross-point matrix, secondary scheduling processing of the fused video is realized on the basis of the signal scheduling matrix. Using virtual cross-points, multiple matrix applications can be dynamically created for different application scenarios, such as testing, live broadcast, standby broadcast, etc. The switching control module can group the output channels according to the application requirements, realizing flexible adjustment of the output strategy.

[0031] The signal scheduling matrix adopts a distributed architecture with multiple cascaded IPG gateways, which can process multiple video channels in parallel, breaking through the processing capacity limitation of a single device. The master control cross-point matrix provides a flexible scheduling strategy, which can give full play to the utilization efficiency of the video link. The switching control module realizes the custom grouping of output channels, which can significantly reduce the number of switching operations compared with traditional single-channel switching. Brief Description of the Drawings

[0032] This application will be further described in the form of exemplary embodiments, and these exemplary embodiments will be described in detail through the drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where:

[0033] Figure 1It is an exemplary flowchart of a method for multi-source video fusion processing according to some embodiments of the present application;

[0034] Figure 2 It is an exemplary flowchart of multi-source video fusion processing according to some embodiments of the present application;

[0035] Figure 3 It is an exemplary flowchart of control module data processing according to some embodiments of the present application;

[0036] Figure 4 It is a schematic diagram of an IPG module according to some embodiments of the present application;

[0037] Figure 5 It is another schematic diagram of an IPG module according to some embodiments of the present application. Detailed implementation manners

[0038] The methods and systems provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0039] Figure 1 It is an exemplary flowchart of a method for multi-source video fusion processing according to some embodiments of the present application. The SDI camera resources are obtained through the SDI or CVBS acquisition unit, the external IP network video media resources are obtained through the network communication unit, and the device desktop media resources are obtained through the interface processing unit to obtain multi-source video data; the multi-source video data is preprocessed; the preprocessed multi-source video data is used as input, and multiple IPG gateways are cascaded to form a signal scheduling matrix, and the multi-source video data is subjected to the first scheduling and fusion processing; according to the output of the signal scheduling matrix, a total control cross-point matrix of 256x256 is set, and the total control cross-point matrix includes virtual cross-points; the signal processed by the signal scheduling matrix is used as the input signal and input into the total control cross-point matrix, and the input signal is subjected to secondary scheduling and processing by controlling the switching and scheduling of the virtual cross-points; in the total control cross-point matrix, different cross-point applications are created for each output channel according to different application scenarios, and the applications include testing, live broadcast, and standby broadcast; the output channels of the total control cross-point matrix are customarily hierarchically grouped through the switching control module, and according to different application scenarios, an output channel is set in multiple groups, and new groups are created according to the preset groups; the video data processed by the control module is transmitted to the display device for display through a wireless communication link, a satellite communication link, or a wired IP communication link.

[0040] Figure 2It is an exemplary flowchart of multi - video fusion processing shown in some embodiments of the present application. In this solution, based on the hybrid master control conversion and scheduling technology of SDI + HDMI + IP, the SDI signal and HDMI signal are first connected through the IP network. While ensuring low - latency signal transmission, it can ensure that each system can obtain high - quality signals and support all applications of the full - link acquisition, editing, and broadcasting. Secondly, the signals used by each system are determined by the application object, and the signal format can be selected according to different application requirements. Finally, the advantages of SDI - HDMI - IP for connecting various systems are also reflected in the fact that the signals as resources can be flexibly scheduled and managed, and the audio - visual processing capabilities can be distributed across the network, improving production capacity and management efficiency.

[0041] Figure 3 It is an exemplary flowchart of control module data processing shown in some embodiments of the present application. In this solution, an IPG module with 16 - channel SDI / HDMI inputs and 16 - channel SDI / HDMI outputs is constructed; inside each IPG module, a 16×16 cross - point matrix is integrated, which can independently complete the switching and scheduling of 16 signals; the IPG module supports converting SDI / HDMI signals into IP signals and transmitting them through the IP network.

[0042] Multiple IPG modules are cascaded and stacked according to the upstream - downstream relationship; the output of the upstream IPG module is connected to the input of the downstream IPG module; each upstream IPG module only outputs one signal to the downstream IPG module to ensure the number of input signals of the downstream IPG module. By cascading multiple IPG modules, a large - scale SDI / HDMI matrix is constructed; assuming that each IPG module has X SDI / HDMI input interfaces and Y SDI / HDMI output interfaces; N upstream IPG modules (N≤X) are used, and each module outputs one signal to the downstream IPG module; the input number of the downstream IPG module is N, and the output number is Y; the input scale of the entire matrix is N×X, and the output scale is Y, as Figure 4 、 Figure 5 shown.

[0043] Apply the constructed large-scale matrix to the broadcast channels; each upstream IPG module corresponds to one broadcast channel and provides the SDI / HDMI signals required for that channel; the Y outputs of the downstream IPG module correspond to the inputs of the broadcast channels to complete signal distribution and scheduling. If it is necessary to set up primary and backup SDI / HDMI channels, the number of IPG modules can be increased; set the number of IPG modules to 2(N + 1), where N modules are for the primary channel and 1 module is for the backup channel; the switching between the primary and backup channels can be achieved by controlling the outputs of the IPG modules. The routing control of the large-scale matrix is completed by being dispersed in multiple IPG modules; each IPG module is responsible for the switching and scheduling of its own 16×16 crosspoint matrix; through a unified control system, coordinate and manage the work of all IPG modules to achieve the routing control of the entire matrix. This solution uses the stacked combination of multiple IPG modules to construct a large-scale SDI / HDMI matrix. This matrix can provide N×X SDI / HDMI inputs and Y SDI / HDMI outputs to meet the signal scheduling requirements of the broadcast channels. At the same time, by setting up the primary and backup channels, the reliability and fault tolerance of the system are improved.

[0044] If there are multiple channels (for example, the number of channels is S), and there are primary and backup channels in both the master control and the broadcast, determine the configuration quantity of the IPG module according to the number of channels and the requirements of the primary and backup channels; for the case of S channels with primary and backup channels in both the master control and the broadcast, the number of IPG modules should be set to 2(N + S); where N represents the number of IPG modules on the master control side, and S represents the number of broadcast channels. On the master control side, configure N IPG modules with X SDI inputs; the X SDI inputs of each IPG module correspond to different signal sources, providing the signals required for master control scheduling; due to the consideration of primary and backup channels, the actual number of IPG modules configured on the master control side is 2N. On the broadcast side, configure one IPG module with Y SDI outputs for each channel; the IPG module of each channel provides Y signal outputs required for that channel; due to the consideration of primary and backup channels, the actual number of IPG modules configured for each channel is 2, and a total of 2S IPG modules need to be configured on the broadcast side. The 2N IPG modules on the master control side provide 2(N×X) SDI input signals for master control scheduling; through the physical cross-point matrix inside the IPG module, route and switch the SDI input signals on the master control side; the primary and backup IPG modules of each broadcast channel obtain the required Y signals from the IPG modules on the master control side to complete signal scheduling and transmission. On both the master control and the broadcast sides, each channel has independent primary and backup IPG modules; by controlling the output of the IPG module, the switching between the primary and backup channels can be realized; when the primary channel fails, it can be quickly switched to the backup channel to ensure the continuity and reliability of the signal. By increasing the number of IPG modules, the number of channels and the signal routing ability of the system can be easily expanded; if it is necessary to increase the broadcast channels, only the corresponding number of IPG modules needs to be added on the broadcast side; if it is necessary to expand the number of signal paths for master control scheduling, the number of IPG modules can be increased on the master control side. This solution uses IPG modules to build an SDI signal scheduling system that supports multiple channels and has primary and backup channels. The IPG modules on the master control side provide N×X SDI input signals, and the IPG modules on the broadcast side provide Y signal outputs for each channel. Through the setting of the primary and backup channels, the reliability and fault tolerance of the system are ensured.

[0045] Taking advantage of the multiple interface types of the IPG and the flexibility of virtual crosspoints, virtual crosspoints of different scales and functions can be created according to application requirements to achieve the mixed scheduling and broadcasting of SDI, HDMI, and IP signals. Interface configuration of the IPG: The IPG has SDI interfaces, HDMI interfaces, and IP interfaces, which can access and output different types of signals; by configuring the interfaces of the IPG, the input and output of SDI, HDMI, and IP signals can be achieved; the IP interface of the IPG can convert SDI and HDMI signals into IP signals to realize the fusion of different signal types. Creation of virtual crosspoints: Using the virtual crosspoint feature of the IPG, virtual crosspoints of different scales and functions can be created according to application requirements; the virtual crosspoints can be the same as physical crosspoints or the superposition and fusion of multiple physical crosspoints; through the control software, virtual crosspoints can be flexibly created and managed to achieve signal scheduling and switching. Creation of master control crosspoints: According to the requirements of the master control, a large-scale virtual crosspoint can be created, such as a 256×256 master control crosspoint; the master control crosspoint can access various signal sources, including SDI, HDMI, and IP signals; through the master control crosspoint, signal scheduling and switching within the global scope can be achieved to meet the requirements of the master control.

[0046] Cascading of signal processing and virtual crosspoints: The processed signal can be used as a new signal source and serve as the input signal source of the virtual crosspoint; by cascading multiple virtual crosspoints, complex signal processing and scheduling processes can be achieved; after each signal processing, the processed signal can be used as a new signal source and input into the next-level virtual crosspoint. Virtual crosspoints for broadcast channels: Independent virtual crosspoints are created for each broadcast channel, and different scales and functions are set according to the requirements of the channel; the virtual crosspoints of the broadcast channels can access the signal sources required by the channels, including SDI, HDMI, and IP signals; through the virtual crosspoints of the channels, signal scheduling and switching within the channels can be achieved to meet the requirements of broadcasting. Application of virtual crosspoints in scenarios: According to actual usage requirements, virtual crosspoint applications in different scenarios can be created; during signal testing, virtual crosspoints for test applications can be created for signal testing and verification; during signal live broadcast, virtual crosspoints for live broadcast applications can be created for the primary and backup signals to achieve the switching and scheduling of the primary and backup signals; in the SDI, HDMI, and IP hybrid broadcast domain, different backup broadcast scenarios can be created for the broadcast of important signals to ensure the reliability of the broadcast.

[0047] Combination of Physical Cross-Points and Virtual Cross-Points: The combined use of physical cross-points and virtual cross-points can give full play to the advantages of SDI and IP signals in their respective fields. Physical cross-points are suitable for stable and reliable SDI signal scheduling, while virtual cross-points are suitable for flexible IP signal scheduling. Through the integration of physical cross-points and virtual cross-points, the mixed use of SDI and IP can be achieved, providing a more flexible and customizable signal flow. By leveraging the IP input / output and virtual cross-point features of IPG and combining with control software, the switching and broadcasting in the IP domain are realized. By creating virtual cross-points of different scales and functions, the requirements of master control, broadcast channels, and different application scenarios can be met. At the same time, through the combination of physical cross-points and virtual cross-points, the mixed scheduling of SDI and IP signals can be achieved, providing a more flexible and reliable signal flow, and better serving safe broadcasting and definable signal processes.

[0048] This solution aims to achieve unified management of SDI, HDMI, and IP devices and signals, covering main functions such as device management, switching control, multi-view management, system monitoring, and integrated control, providing a converged master control management platform. Unified management of all devices in the system, including IP devices, IPG devices, HDMI devices, and SDI devices; defining and configuring the attributes and parameters of various devices, such as device type, interface type, IP address, etc.; grouping and classifying devices for convenient subsequent control and maintenance; defining signal sources and mapping the input and output ports of devices to logical signal sources for convenient signal scheduling and switching.

[0049] Fully customizable hierarchical grouping of destination channels. In the management platform, a function for grouping destination channels is provided, allowing users to create and manage different groups; each group can contain multiple destination channels, and one destination channel can also belong to multiple groups; groups can be hierarchically managed, supporting the creation of a multi-level group structure, such as hierarchical classification by region, function, etc.; users can customize the name, level, and included destination channels of the group according to actual needs. Copying groups from the default group: A default group containing all destination channels is provided; users can select destination channels from the default group to create new groups; one-key copying of the default group is supported to quickly create new groups and reduce repetitive operations; after copying the group, users can edit and customize the new group, adding or deleting destination channels.

[0050] Flexible switching control: In the management platform, an independent switching control interface is provided for each group; users can switch between different groups and switch and manage the destination channels within the current group; support for setting different switching strategies and priorities for each group according to application requirements; switching control can be achieved through manual operations or automation rules, providing flexible control methods.

[0051] Drag and Drop Switching between Signal Sources and Destination Channels: In the graphical interface of the management platform, a visual representation of signal sources and destination channels is provided. Signal sources can be displayed in the form of a list or icons, and destination channels can be displayed in the form of groups or grids. Users can perform a mouse drag-and-drop operation to drag a signal source onto a destination channel to achieve the switching of the signal source to the destination channel. One-to-one switching is supported, that is, switching a signal source to a single destination channel. One-to-many switching is supported, that is, switching a signal source to multiple destination channels simultaneously. The intuitive drag-and-drop operation simplifies the switching process between signal sources and destination channels and improves operation efficiency.

[0052] Multi-Panel Management: An intuitive multi-panel monitoring function is provided to achieve real-time monitoring of all video signal sources in the system. The "what you see is what you get" operation mode is supported, and operators can perform panel switching and management through the visual interface. Multi-panel monitoring can display all video signal sources in the system and support real-time switching. Through multi-panel management, the visibility and security of operations are improved, and the operation process is simplified.

[0053] System Monitoring: The running status of the system is monitored in real time, especially the usage status of signals and links. A signal source is specified in the signal source list, and its usage in the system is analyzed in real time. A usage link diagram of the signal source is listed to show which channels use the signal source and is updated in real time. A general protocol monitoring platform is established to support protocols such as SNMP, SSH, and WebSocket. Through the general protocol monitoring platform, custom data-level monitoring is achieved to obtain the status information of devices and signals.

[0054] Integrated Control: By combining device management and channel definition, protocol-related device management is centralized in a unified control platform. Signals and requirements are virtualized into operation objects, and operators only need to focus on how to perform operations on the objects. A custom mode setting function is provided to set functions such as device parameter modification, signal scheduling, panel layout switching, and backup data in the system into the mode. Through one-key execution of the mode, rapid configuration and operation of all functions in the system are achieved. Integrated control simplifies the operation process, improves operation efficiency, and reduces the risk of operation errors.

[0055] While the SDI+HDMI+IP hybrid architecture broadcast master control system described in this solution realizes the traditional SDI signal link scheduling function, it also utilizes the basic functions of IPG devices to complete the IPization of all SDI and HDMI signals in the system. Combined with the primary and standby core switches, a completely independent IP architecture application link can be achieved. This hybrid architecture broadcast master control system can replace the traditional SDI matrix and still retain the SDI signal link function.

[0056] For the purpose of smooth transition, the traditional SDI broadcast control channel can be used under this architecture to keep the SDI and HDMI signal broadcast control links clear and intuitive. In the main and standby link mode, the single device failure point has little impact on the system; the IPG device's own frame synchronization, up and down conversion, multi-screen and embedding (ST2110 audio and video are transmitted separately) and other functions can also be used to save a lot of complicated peripheral equipment. At the same time, since the signal in the system has been IP-based, the IP-based production and broadcasting process can be realized at a relatively low cost with the support of this system.

[0057] IPG has SDI interface, HDMI interface, and IP interface. It can use the IP input and output of IPG and the IPG virtual crosspoint features, combined with control software, to more easily complete the switching and broadcasting of IP domains. Virtual crosspoints have more flexibility. They can be consistent with physical crosspoints, or they can be superimposed and integrated with multiple physical crosspoints. It is also possible to create different virtual crosspoint scales according to application needs. For example, you can create a 256×256 master control crosspoint, or you can use each processed signal as a new signal source as the input signal source of the virtual crosspoint, so as to complete more signal scheduling applications; you can also create crosspoint applications of different scales for each broadcast channel; it may be based on actual use needs, such as creating a crosspoint for test applications during signal testing, creating a crosspoint for live broadcast applications for the main and standby signals during signal live broadcast, and creating different standby broadcast scenarios for important signal broadcasts in the SDI, HDMI and IP mixed broadcast domains. The combination of physical crosspoints and virtual crosspoints can give full play to the advantages of SDI and IP signals in their respective fields, and can also integrate the mixed use of SDI and IP to better serve safe broadcasting and definable signal processes.

Claims

1. A device for multi-video fusion processing, characterized in that: include: The acquisition module acquires multiple video data, including an SDI or CVBS acquisition unit, a network communication unit, and an interface processing unit, which are used to acquire SDI camera resources, external IP network video media resources, and device desktop media resources respectively; A preprocessing module preprocesses the collected multivariate video data; The control module performs fusion processing on the acquired multi-dimensional video data, including: The signal scheduling matrix, which consists of multiple cascade-connected IPG gateways, is used to perform the first scheduling and fusion processing of multi-dimensional video data; The master control crosspoint matrix sets virtual crosspoints according to the output of the signal scheduling matrix, and is used for secondary scheduling and processing of the signals processed by the signal scheduling matrix; The switching control module performs customized hierarchical grouping and switching control on the output channels of the master control crosspoint matrix; The transmission module transmits the data merged and processed by the control module to the display device through a wireless communication link, a satellite communication link or a wired IP communication link.

2. A method for multi-video fusion processing, characterized in that: include: Acquire SDI camera resources through SDI or CVBS acquisition unit, acquire external IP network video media resources through network communication unit, acquire device desktop media resources through interface processing unit, and obtain multi-dimensional video data; Preprocessing of multivariate video data; The pre-processed multi-dimensional video data is used as input, and multiple IPG gateways are cascaded to form a signal scheduling matrix to perform the first scheduling and fusion processing on the multi-dimensional video data; According to the output of the signal scheduling matrix, a 256x256 master control crosspoint matrix is ​​set, and the master control crosspoint matrix includes virtual crosspoints; the signal processed by the signal scheduling matrix is ​​used as an input signal and input into the master control crosspoint matrix, and the input signal is secondary scheduled and processed by controlling the switching and scheduling of the virtual crosspoints; In the master control crosspoint matrix, different crosspoint applications are created for each output channel according to different application scenarios, including testing, live broadcast and standby broadcast; The output channels of the master control crosspoint matrix are customized and grouped by switching control modules. According to different application scenarios, one output channel can be set in multiple groups, and new groups can be created based on preset groups. The video data after fusion and processing by the control module is transmitted to the display device for display via a wireless communication link, a satellite communication link or a wired IP communication link.

3. The method for multi-element video fusion processing according to claim 2, characterized in that: Get multivariate video data, including: The SDI signal or CVBS signal output by the SDI camera is collected by the SDI or CVBS collection unit to obtain SDI video data or CVBS video data; Establishing a communication connection with an external IP network through a network communication unit, receiving network video media data transmitted by the external IP network, and obtaining network video data; A connection is established between the interface processing unit and the video acquisition interface inside the device to acquire the device desktop media data and obtain desktop video data.

4. The method for multi-video fusion processing according to claim 3, characterized in that: Preprocess the multivariate video data, including: SDI video data, CVBS video data, network video data and desktop video data are encoded to obtain encoded multi-dimensional video data, wherein the audio data is encoded using the AAC encoding algorithm and the video data is encoded using the H.264 encoding algorithm; The encoded multi-video data is encapsulated by using a streaming media transmission protocol; the streaming media protocol includes at least one of the RTMP protocol, the HLS protocol and the HTTP-FLV protocol.

5. The method for multi-video fusion processing according to claim 4, characterized in that: Multiple IPG gateways are cascaded to form a signal scheduling matrix, including: Set N IPG modules with X SDI or HDMI input interfaces and Y output interfaces, where N is less than or equal to X, and Y is less than or equal to X; N IPG modules are cascaded to form a signal scheduling matrix with NxX input interfaces and NxY output interfaces; Input the multi-dimensional video data acquired by the SDI acquisition unit, the network communication unit and the interface processing unit into the input interface of the corresponding IPG module respectively; By controlling the IPG module, the input multi-dimensional video data is first scheduled and fused to obtain NxY fused video signals; The NxY fused video signals are output through the output interface of the cascaded IPG modules as the output signals of the signal scheduling matrix.

6. The method for multi-element video fusion processing according to claim 5, characterized in that: Multiple IPG gateways are cascaded to form a signal scheduling matrix, which also includes: Set the main and backup output channels of SDI or HDMI, and set the number of IPG modules to 2 (N+1), where 2N IPG modules are used as the main channels and 2 IPG modules are used as backup channels; By controlling the IPG module of the main channel and the IPG module of the backup channel, the output signal of the main channel is switched to the input interface of the IPG module of the backup channel to switch and back up the main and backup channels; The NxY output signals of the main channel IPG module and the Y output signals of the backup channel IPG module, a total of NxY+Y signals, are used as output signals of the signal scheduling matrix.

7. The method for multi-video fusion processing according to claim 6, characterized in that: Set up a 256x256 master control crosspoint matrix, including: Set up a 256x256 master control crosspoint matrix, which includes virtual crosspoints; Set S channels, and set each channel as a primary and backup channel. Set the number of IPG modules to 2 (N+S); 2N IPG modules are used for the primary channel, and 2S IPG modules are used for the backup channel; By controlling the virtual crosspoints in the master control crosspoint matrix, the input NxY+Y signals are secondary scheduled and processed to obtain 2SxY output signals as the main and standby channel signals of S channels; wherein the main and standby channels of each channel have Y signal outputs; The 2SxY output signals are output to the corresponding IPG modules respectively, wherein the SxY signals of the main channel are output to 2N main channel IPG modules, and the SxY signals of the backup channel are output to 2S backup channel IPG modules.

8. The method for multi-video fusion processing according to claim 7, characterized in that: Create different crosspoint applications for each output channel, including: According to different application scenarios, create different application scenarios for each output channel in the master control crosspoint matrix; Use the SDI interface, HDMI interface and IP interface of the IPG module to switch and schedule SDI signals, HDMI signals and IP signals; According to different application scenarios, virtual crosspoint matrices of different sizes are created by combining the crosspoints in the master control crosspoint matrix; The output signal of the master control cross-point matrix is ​​used as an input signal and input into the virtual cross-point matrix; According to different application scenarios, control is performed through a virtual crosspoint matrix; Through the control of the virtual cross-point matrix, the output signal of the master control cross-point matrix is ​​processed twice to obtain the output signal corresponding to the application scenario, and the output signal is output through the SDI interface, HDMI interface and IP interface of the IPG module; Among them: the virtual cross-point matrix of the test application scenario is used for signal testing and monitoring; the virtual cross-point matrix of the live broadcast application scenario is used for switching and scheduling of main and standby signals; the virtual cross-point matrix of the standby broadcast application scenario is used for backup of SDI, HDMI and IP signals.

9. The method for multi-video fusion processing according to claim 8, characterized in that: The output channels of the master control crosspoint matrix can be customized and grouped by switching control modules, including: The output channels of the master control crosspoint matrix are managed in groups, and the output channels are set according to different application scenarios; Multiple groups are preset, each group corresponds to a unique group number; multiple application scenarios are preset, each application scenario corresponds to a unique scenario number; Establish a mapping table between application scenarios and groups, and set the mapping relationship between output channels and groups under different application scenarios in the mapping table; Set the output channel configuration table, the mapping table of application scenarios and groups, set the corresponding relationship between output channel entries and group numbers in the output channel configuration table, and associate one output channel entry with multiple group numbers.

10. The method for multi-video fusion processing according to claim 9, characterized in that: Create new groups based on preset groups, including: Set up a group configuration table to store group configuration information, the configuration information includes group number and group name; According to the user's instruction to create a new group, according to the configuration information of the preset group in the group configuration table, a new group number and group name are created in the group configuration table to create a new group; In the output channel configuration table, associate the new group number with the output channel entry, and associate the output channel with the new group; Generate a group control instruction reflecting the group relationship of the output channels according to the group configuration table and the output channel configuration table; The master control crosspoint matrix controls the crosspoint switches according to the group control instructions to synchronously switch and control the audio and video data of the output channels belonging to the same group in the group control instructions, thereby realizing independent control of the output channels of different groups.

Images