Multi-link composite video transmission system and transmission method

Abstract

The present invention relates to a multi-link composite video transmission system, which comprises: a distributed system encoder and a distributed system decoder on a first transmission link, the distributed system encoder is used to encode a video signal into a first network stream signal, the distributed system decoder is used to decode it, and send the decoded video signal to a main video interface of a display device; a desktop screen capture client and a centralized system processor on a second transmission link, the desktop screen capture client is used to obtain a desktop image of the video signal source in real time, encode it into a second network stream signal, and the centralized system processor is used to decode it and send it to a backup video interface of a display device; an integrated management and control platform is used to bind multiple corresponding components on the first and second transmission links, and enable normal components when a node is abnormal; a display device, which can switch between the main and backup video interfaces. The present invention can improve the video transmission stability of the multi-link composite video transmission system and has scalability.

Description

Technical Field

[0001] The present invention relates to the technical field of video transmission, and in particular to a multi-link composite video transmission system and a transmission method. Background Art

[0002] In the prior art, users retrieve video signals through a video transmission system, wherein the video signals are encoded, transmitted over the network, decoded, and spliced ​​before reaching the display device. Due to the instability or failure of the actual network environment, various physical environments, or various physical nodes, the problem of video signals failing to reach the display device often occurs. Summary of the invention

[0003] In view of the above existing technical problems, the present application proposes a multi-link composite video transmission system and a transmission method thereof.

[0004] The multi-link composite video transmission system comprises:

[0005] The first transmission link includes a first distributed system encoder, a display and control network switch, and a distributed system decoder. The first distributed system encoder is connected to a video signal source, and the distributed system decoder is connected to a main video interface of a display device through a first video line. The first distributed system encoder is used to obtain a video signal from a video signal source and encode it, and encode it into a first network stream signal. The distributed system decoder is used to obtain and decode the first network stream signal through the display and control network switch, and send the decoded video signal to the main video interface of the display device.

[0006] The second transmission link includes a desktop screen capture client installed on the video signal source, a data network switch and a centralized system processor, the centralized system processor is connected to the backup video interface of the display device through a second video line; the desktop screen capture client is used to obtain the desktop image of the video signal source in real time and encode it into a second network stream signal; the centralized system processor is used to obtain the second network stream signal through the data network switch and decode it, and send the decoded video signal to the backup video interface of the display device;

[0007] An integrated management and control platform is connected to a data network switch and a display and control network switch through a core network switch, thereby being connected to a distributed system encoder, a distributed system decoder, a desktop screen capture client, and a centralized system processor network; the integrated management and control platform is used to bind the distributed system encoder and the desktop screen capture client as a first logical node, and to bind the distributed system decoder and the centralized system processor as a second logical node, and when the integrated management and control platform monitors that any logical node in the first transmission link is abnormal, enable the normal components on the second transmission link in the abnormal logical node, and keep using the normal components on the first transmission link on other normal logical nodes;

[0008] A display device having a main video interface and a backup video interface, wherein the display device can switch freely between the main video interface and the backup video interface, and when video signals are input to both the main video interface and the backup video interface, the video signal input to the main video interface is selected for display.

[0009] According to some embodiments of the present invention, the video transmission system also includes a backup signal source, which is connected to a second distributed system encoder via a video cable, and the second distributed system encoder is further connected to the display and control network switch via a network cable; a desktop screen capture client is installed on the backup signal source, and the desktop screen capture client captures its desktop image and encodes it into a network stream signal and transmits it to the data network switch; the integrated management and control platform is used to bind the signal source and the backup signal source as a third logical node; and the second distributed system encoder and the desktop screen capture client installed on the backup signal source are bound as a fourth logical node.

[0010] According to some embodiments of the present invention, the integrated management and control platform is further configured to synchronously transmit received user instructions to the first transmission link and the second transmission link.

[0011] According to some embodiments of the present invention, the integrated management and control platform monitors the performance parameters of each component in the first and second transmission links in real time, calculates the scores of the first and second transmission links according to the performance parameters, and selects the link with the higher score as the current transmission link, wherein the performance parameters of each component include: encoding delay (ED) and encoding quality (EQ) of the distributed system encoder; network bandwidth (NB), network delay (NL), and network packet loss rate (NPL) of the display and control network switch; decoding delay (DD) and decoding quality (DQ) of the distributed system decoder; screen capture delay (CD) and screen capture quality (CQ) of the desktop screen capture client; network bandwidth (DB), network delay (DL), and network packet loss rate (DPL) of the data network switch; decoding delay (CSDD) and decoding quality (CSDQ) of the centralized system processor;

[0012] First transmission link score (S1):

[0013]

[0014] Second transmission link score (S1):

[0015]

[0016] W EQ : Weight of coding quality; W DQ : The weight of decoding quality; W ED : Weight of coding delay; W NL : The weight of network delay; W NPL : The weight of the network packet loss rate; W DD : The weight of the decoding delay; W CQ : Weight of screen capture quality; W CD : Weight of screen capture delay; W DL : The weight of network delay; W DPL : The weight of the network packet loss rate; W CSDQ : The weight of decoding quality; W CSDD : Weight of decoding delay.

[0017] According to some embodiments of the present invention, when the integrated management and control platform monitors that any logical node in the first transmission link is abnormal, the first transmission link of the multi-link composite video transmission system is switched to the second transmission link.

[0018] According to some embodiments of the present invention, the delay of the second transmission link is greater than that of the first transmission link, and the integrated management and control platform is further used to calculate the number of image frames that need to be automatically removed according to the difference between the delay of the first transmission link and the delay of the second transmission link, and the frame rate, wherein the number of image frames = the difference x the frame rate, and the integrated management and control platform is further used to control the centralized system processor to remove the number of image frames before outputting the image externally; and / or,

[0019] The integrated management and control platform is also used to calculate the number of recent image frames that need to be temporarily stored in the first transmission link based on the difference between the delay of the first transmission link and the delay of the second transmission link, and the frame rate, where the number of image frames = the difference x frame rate, and the integrated management and control platform is also used to control the distributed decoder to temporarily store the number of image frames.

[0020] The present application also provides a multi-link composite video transmission method based on the multi-link composite video transmission system, which comprises:

[0021] The distributed system encoder of the first transmission link obtains the video signal of the video signal source and encodes it into a first network stream signal;

[0022] The distributed system decoder obtains the first network stream signal through the network and decodes it, and sends the decoded video signal to the main video interface of the display device for display;

[0023] The desktop screen capture client of the second transmission link obtains the desktop image of the video signal source in real time and encodes it into a second network stream signal;

[0024] The centralized system processor acquires the second network stream signal through the network and decodes it, and sends the decoded video signal to the backup video interface of the display device;

[0025] The integrated management and control platform monitors the first transmission link in real time. When it is monitored that any logical node in the first transmission link is abnormal, a normal component on the second transmission link in the abnormal logical node is enabled, and normal components on the first transmission link on other normal logical nodes are kept in use; and

[0026] The display device displays the video signal inputted from the standby video interface.

[0027] According to some embodiments of the present invention, the method includes, when it is monitored that any logical node in the first transmission link is abnormal, switching the first transmission link of the multi-link composite video transmission system to the second transmission link.

[0028] According to some embodiments of the present invention, when the integrated management and control platform monitors that the number of video windows in the centralized system processor is greater than a threshold, the integrated management and control platform instructs the distributed system encoder and / or desktop capture client to adopt a deep compression encoding mode when encoding.

[0029] According to some embodiments of the present invention, the method further includes the integrated management and control platform synchronously transmitting the received user instructions to the first transmission link and the second transmission link.

[0030] The present invention can improve the reliability of video transmission. Moreover, the present invention has flexible scalability at both the video signal source end and the display device end, and can achieve accurate design of system scale, thereby reducing system cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1a The centralized system hot backup system in the prior art is shown;

[0032] Figure 1b A distributed system hot backup system in the prior art is shown;

[0033] Figure 2 A topology diagram showing a multi-link composite video transmission system according to some embodiments of the present invention;

[0034] Figure 3 A flowchart showing a multi-link composite video transmission method according to some embodiments of the present invention;

[0035] Figure 4 A module diagram showing a comprehensive management device according to some embodiments of the present invention;

[0036] Figure 5 A schematic structural diagram of a comprehensive management device according to some embodiments of the present invention is shown. DETAILED DESCRIPTION

[0037] In this application, all references to implementation through a network are essentially intended to cover wired or wireless network connections implemented through necessary firmware or software such as switches and routers, and also cover wired or wireless network connections implemented through some server terminals or other computers as intermediaries. In order to simplify the description and highlight the invention, this application sometimes omits the description of routers / switches when describing network connections.

[0038] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined and referenced with each other.

[0039] Figure 2 A topology diagram of a multi-link composite video transmission system according to some embodiments of the present invention is shown. As shown in the figure, the system includes a first transmission link, a second transmission link, an integrated management and control platform, and a display device. Each component will be described in detail below. Signal sources 1000 and 1001 are two signal sources that play the same content, that is, the images on signal sources 1000 and 1001 are synchronized, for example, they may both be playing the news broadcast, and signal source 1000 is also referred to as a backup signal source in this application.

[0040] The first transmission link includes a distributed system encoder 101, a display control network switch 103, and a distributed system decoder 102. The distributed system encoder 101 is connected to the video signal source 10, and the distributed system decoder 102 is connected to the main video interface of the display device 401 through a first video line.

[0041] The distributed system encoder 101 is used to obtain the video signal of the video signal source 10 and encode it into a first network stream signal. The distributed system decoder 102 is used to obtain the first network stream signal through the display control network switch 103 and decode it, and send the decoded video signal to the main video interface of the display device 401.

[0042] The second transmission link includes a desktop screen capture client installed on the video signal source, a data network switch and a centralized system processor 202, and the centralized system processor 202 is connected to the backup video interface of the display device 401 through a second video line. The centralized system processor includes various video splicing display processing devices such as a splicing processor. The centralized system processor and the data network switch are connected through two network cables, one of which is used to transmit video signals and the other is used to transmit management instructions.

[0043] The desktop screen capture client is used to obtain the desktop image of the video signal source in real time and encode it into a second network stream signal; the centralized system processor 202 is used to obtain and decode the second network stream signal through the data network switch, and send the decoded video signal to the backup video interface of the display device 401.

[0044] The integrated management and control platform 301 is connected to the data network switch 203 and the display and control network switch 103 through the core network switch 402 , and is connected to the aforementioned distributed system encoder 101 , distributed system decoder 102 , desktop screen capture client and centralized system processor 202 .

[0045] The integrated management and control platform 301 is used to bind the distributed system encoder 101 and the desktop screen capture client as a first logical node, and to bind the distributed system decoder 102 and the centralized system processor 202 as a second logical node. When the integrated management and control platform 301 monitors that any logical node in the first transmission link is abnormal or fails, the normal components on the second transmission link in the abnormal logical node are enabled, and the normal components on the first transmission link on other normal logical nodes are kept in use (which will be described in detail below).

[0046] The integrated management and control platform is also used to synchronously transmit the control instruction or the received user instruction to the first transmission link and the second transmission link, so that the first transmission link and the second transmission link will make the same changes, thereby ensuring that the signals of the first transmission link and the second transmission link are consistent. For example, when a user inputs a call instruction and hopes to call the video signal on the video signal source 1 to the display device, the integrated management and control platform sends the user call instruction to the first transmission link and the second transmission link, so that the call action on the video signal source 1 is implemented simultaneously on the first transmission link and the second transmission link.

[0047] Through the above method, the first transmission link and the second transmission link always make the same change or response, thereby ensuring the consistency of the first and second transmission link signals, and when switching is required, enabling fast and seamless switching of display content on the display device, including displaying multiple video signal sources in the same windowing manner as the first transmission link.

[0048] The display device 401 has a main video interface and a backup video interface, and the display device can freely switch between the main video interface and the backup video interface, and when both the main video interface and the backup video interface have video signals input, the video signal input from the main video interface is selected for display. That is, the display device supports dual signal input, and when one signal is interrupted, the other signal automatically takes over and replaces.

[0049] According to some embodiments of the present invention, some functions in the above-mentioned display device can also be separated and implemented by another independent sending card. Specifically, the distributed system decoder can send the decoded video signal to the sending card, and the sending card then connects the video signal to the display device. In addition, the centralized system processor can also send the decoded video signal to the sending card, and the sending card then connects the video signal to the display device. The video signals transmitted by the above-mentioned two paths can be selected by setting the switching logic of the sending card to ensure that: when both paths have video signal input, the sending card selects the video signal in the first transmission link after identifying the video signal in the first transmission link, and further transmits it to the display device for display; and when one of the paths has no video signal input, the video signal of the other input is further transmitted to the display device for display.

[0050] After receiving the video signal, the sending card decodes it and converts it into a data format that can be recognized by the display device (such as an LED display), and then sends the data to the receiving card of the display device through the data transmission channel. After receiving the data, the receiving card parses it and controls the display of the display device to present an image. The sending card can have multiple input interfaces and can accept signals from multiple video signal sources.

[0051] According to some other embodiments of the present invention, the integrated management and control platform itself adopts a dual-machine hot standby mode, that is, the multi-link composite video transmission system also includes a standby integrated management and control platform as a hot backup of the integrated management and control platform, which is used to take over the work of the integrated management and control platform when the integrated management and control platform fails, thereby realizing full-link reliable operation of the entire multi-link composite video transmission system.

[0052] According to some embodiments of the present invention, when the integrated management and control platform monitors that the number of video windowing in the centralized system processor is greater than a threshold, the integrated management and control platform instructs the distributed system encoder and / or desktop capture client to adopt a deep compression encoding mode (including a deep compression mode with a greater compression amplitude) when encoding. Here, due to the limited processing power of the centralized system processor, too many windows and high-definition video splicing display will cause its capacity to decline and freeze. Therefore, in order to avoid these problems, the integrated management and control platform instructs the distributed system encoder and / or desktop capture client to deeply compress the image when encoding, thereby ensuring that the centralized system processor can easily cope with the splicing display output, and also saving transmission bandwidth.

[0053] Continue to refer Figure 2 , wherein the arrow part shows a schematic diagram of the node switching process according to some embodiments of the present invention. As mentioned above, when the integrated management and control platform monitors that any logical node in the first transmission link is abnormal, the normal components on the second transmission link in the abnormal logical node will be enabled, and the normal components on the first transmission link on other normal logical nodes will be kept in use.

[0054] The node abnormality may include a coding failure of the distributed system encoder, in which case the integrated management and control platform may automatically switch the video transmission line to the desktop screen capture client for further encoding, or vice versa, the node abnormality may also include a desktop screen capture client software disconnection, in which case the integrated management and control platform may automatically switch the video transmission line to the distributed system encoder. In the integrated management and control platform, the virtual node and the distributed system encoding node are bound to a logical node. In the present invention, binding to a logical node means that when one of them fails and disconnects, it will automatically switch to another path, so that the multi-link composite video transmission system can continue to operate normally.

[0055] When the integrated management and control platform detects that the distributed system encoder 101 of the first logical node is faulty, the normal components on the second transmission link in the abnormal logical node are enabled, that is, the desktop screen capture client is enabled, and the normal components on the first transmission link of other normal logical nodes are kept in use, such as keeping in use Figure 2 The solid arrow in the figure shows another possibility, that is, when the first transmission link returns to normal and the first logical node on the second transmission link has a problem, other normal components on the first transmission link can be considered, that is, the distributed system encoder 101 and the centralized system processor 202 in the figure are used.

[0056] The abnormal node may also include: a decoding failure of a distributed system decoder (such as a distributed system decoder signal being disconnected). In this case, the integrated management and control platform may automatically switch the video transmission line to a centralized system processor for further decoding.

[0057] According to other embodiments of the present invention, the integrated management and control platform can monitor the performance parameters of each component in the first and second transmission links in real time, calculate the scores of the first and second transmission links according to the performance parameters, and select the link with the higher score as the current transmission link, wherein the performance of each component includes: encoding delay (ED) and encoding quality (EQ) of the distributed system encoder; network bandwidth (NB), network delay (NL), and network packet loss rate (NPL) of the display and control network switch; decoding delay (DD) and decoding quality (DQ) of the distributed system decoder; screen capture delay (CD) and screen capture quality (CQ) of the desktop screen capture client; network bandwidth (DB), network delay (DL), and network packet loss rate (DPL) of the data network switch; decoding delay (CSDD) and decoding quality (CSDQ) of the centralized system processor.

[0058] First transmission link score (S1):

[0059]

[0060] Second transmission link score (S1):

[0061]

[0062] W EQ : Weight of coding quality; W DQ : The weight of decoding quality; W ED : Weight of coding delay; W NL : The weight of network delay; W NPL : The weight of the network packet loss rate; W DD : The weight of the decoding delay; W CQ : Weight of screen capture quality; W CD : Weight of screen capture delay; W DL : The weight of network delay; W DPL : The weight of the network packet loss rate; W CSDQ : The weight of decoding quality; W CSDD : Weight of decoding delay.

[0063] The present invention can improve the reliability of video transmission and realize accurate design of system scale, thereby reducing system cost.

[0064] According to other embodiments of the present invention, when the integrated management and control platform monitors that any logical node in the first transmission link is abnormal, the integrated management and control platform automatically switches the multi-link composite video transmission system to the second transmission link. As mentioned above, when the user inputs a call instruction and hopes to call the video signal on the video signal source 1 to the display device, the integrated management and control platform sends the user call instruction to the first transmission link and the second transmission link, so that the call action on the video signal source 1 is implemented simultaneously on the first transmission link and the second transmission link. Therefore, when the integrated management and control platform switches the multi-link composite video transmission system to the second transmission link, the second transmission link can immediately take over the first transmission link to correctly output the subsequent video content.

[0065] like Figure 2 As shown in , when the integrated management and control platform monitors that the distributed system encoder 101 of the first logical node or the first row transmission link where it is located has a fault, it will directly use Figure 2 The second transmission link where the second row transmission link is located, at this time, the display device can also automatically switch to the backup video interface to display the video signal input in the backup video interface. As mentioned above, the video signals transmitted by the above two paths can be selected by setting the switching logic of the sending card to ensure that: when both paths have video signal input, the sending card selects the video signal in the first transmission link after identifying the video signal in the first transmission link, and further transmits it to the display device for display; and when one of the paths has no video signal input, the video signal input from the other path is further transmitted to the display device for display.

[0066] According to some embodiments of the present invention, the entire multi-link composite video transmission system may be configured such that when a first transmission link is abnormal, the multi-link composite video transmission system will automatically enable a second transmission link.

[0067] Through the above embodiment, when the first transmission link is destroyed, the second transmission link will continue to maintain video transmission. Since the first transmission link is distributed (each signal source is connected to a distributed system encoder) and the second transmission link is centralized (all signals are connected to the centralized processing system and output to the display device), the two have completely different spatial distribution properties. Therefore, in the face of war, which usually chooses to activate centralized bombing or distributed bombing, the multi-link composite video transmission system of the present invention has a strong adaptability and can ensure that the video transmission of the entire multi-link composite video transmission system continues to the greatest extent.

[0068] According to some other embodiments of the present invention, the video transmission system may further include a backup signal source 1000. Similar to the connection relationship of the signal source 10, the signal source 1000 is connected to a distributed video encoder 1001 via a video cable, and the distributed encoder is further connected to the aforementioned display and control network switch 103 via a network cable; on the other hand, the signal source 1000 captures its desktop image through its desktop screen capture client, and encodes it into a network stream signal and transmits it to the switch 203. The above description of the signal source 10 is also applicable to the signal source 1000, that is, the signal source 1000 is also transmitted to the display device through two transmission links. The first transmission link includes a distributed system encoder 1001, a display control network switch 103 and a distributed system decoder 102. The distributed system encoder 1001 is connected to the video signal source 1000, and the distributed system decoder 102 is connected to the main video interface of the display device 401 through a first video line. The second transmission link includes a desktop screen capture client installed on the video signal source 1000, a data network switch 203 and a centralized system processor 202. The centralized system processor 202 is connected to the backup video interface of the display device 401 through a second video line. The centralized system processor includes various video splicing display processing devices such as a splicing processor. The first transmission link of signal source 1000 and signal source 10 shares the display control network switch 103, the distributed system decoder 102 and the sending card 403; the second transmission link of signal source 1000 and signal source 10 shares the data network switch 103, the centralized system processor 202 and the sending card 403.

[0069] In addition, the actions applied to the signal source 10 by the aforementioned integrated management and control center and other components are also applicable to the signal source 1000, that is, the integrated management and control platform binds the signal source and the backup signal source into a common logical node; and binds the distributed system encoder 1001 and the desktop screen capture client installed on the signal source 1000 into a common logical node. Therefore, when the signal source 10 fails, the signal source 1000 and its corresponding transmission link (including the link portion shared with the first transmission link and the second transmission link) will be immediately connected, thereby realizing the direct replacement of the signal source.

[0070] The above roughly classifies the signal source failures. The actual failures usually include the types listed in the following table. The integrated management and control platform can adopt the corresponding switching method when encountering the corresponding failure based on the correspondence between various failures and transmission link switching established in advance. The following system-level switching corresponds to the switching between the first transmission link and the second transmission link, the switching of the signal source channel corresponds to the switching between the screen capture client and the distributed system encoder, and the regional-level switching refers to directly abandoning the current Figure 2 The signal source and backup signal source are located in the area, and the signal sources in other areas are introduced.

[0071]

[0072] According to some embodiments of the present invention, when the integrated management and control platform switches the first transmission link to the second transmission link, it controls the video transmission on the second transmission link according to the difference in delay between the first transmission link and the second transmission link, so that the display image on the display device is naturally connected before and after the link switching.

[0073] Specifically, it is assumed that the delay of the second transmission link is greater than that of the first transmission link. Therefore, when the second transmission link is enabled, the user may have to repeatedly watch the video for a period of time. For example, in the first transmission link, the video image has been played to the 100th frame, but when switching to the second transmission link with a slower transmission speed, the video image may only proceed to the 10th frame. Therefore, the user must continue to repeatedly watch the image from the 10th frame to the 100th frame. To this end, the present invention proposes that the integrated management and control platform calculates the number of image frames that need to be automatically removed based on the difference between the delay of the first transmission link and the delay of the second transmission link, as well as the frame rate, and the number of image frames = the difference x frame rate.

[0074] According to some embodiments of the present invention, the integrated management and control platform is also used to control the centralized system processor to remove the number of image frames before outputting the image to the outside.

[0075] There is another possibility that the video image has been played to the 10th frame in the second transmission link, but when switching to the first transmission link with a faster transmission speed, the video image may have reached the 100th frame. Therefore, the user may have to experience the transition from the 10th frame to the 100th frame.

[0076] In this regard, the present application proposes to calculate the number of recent image frames that need to be temporarily stored in the first transmission link based on the difference between the delay of the first transmission link and the delay of the second transmission link, as well as the frame rate, wherein the number of image frames = the difference x frame rate, and when switching to the first transmission link with a faster video transmission speed, the image frames are first taken out from the temporary storage for playback.

[0077] According to some embodiments of the present invention, the integrated management and control platform is further used to control the distributed decoder to temporarily store the image frames of the number of image frames so that they can be retrieved and played at any time.

[0078] Figure 3 A flow chart of a multi-link composite video transmission method according to some embodiments of the present invention is shown.

[0079] As shown in the figure, the method includes:

[0080] S1. The distributed system encoder of the first transmission link obtains the video signal of the video signal source and encodes it, and encodes it into a first network stream signal;

[0081] S2. The distributed system decoder obtains the first network stream signal through the network and decodes it, and sends the decoded video signal to the main video interface of the display device for display;

[0082] S3. The desktop screen capture client of the second transmission link obtains the desktop image of the video signal source in real time and encodes it into a second network stream signal;

[0083] S4. The centralized system processor obtains the second network stream signal through the network and decodes it, and sends the decoded video signal to the standby video interface of the display device;

[0084] S5. In response to monitoring that any logical node in the first transmission link is abnormal, the integrated management and control platform will enable the normal components on the second transmission link in the abnormal logical node, and keep using the normal components on the first transmission link on other normal logical nodes; and

[0085] S6. The display device displays the video signal input into the backup video interface.

[0086] According to some embodiments of the present invention, when it is monitored that any logical node in the first transmission link is abnormal, the first transmission link of the multi-link composite video transmission system is switched to the second transmission link.

[0087] According to some embodiments of the present invention, when the integrated management and control platform monitors that the number of video windowing in the centralized system processor is greater than a threshold, the integrated management and control platform instructs the distributed system encoder and / or desktop capture client to adopt a deep compression encoding mode (including a compression method with a deeper compression amplitude than the original compression amplitude) when encoding. Here, due to the limited processing power of the centralized system processor, too many windowing and high-definition video splicing display will cause its capacity to decline and freeze. Therefore, in order to avoid these problems, the integrated management and control platform instructs the distributed system encoder and / or desktop capture client to perform deep compression on the image when encoding, thereby ensuring that the centralized system processor can easily cope with the splicing display output, and also saving transmission bandwidth.

[0088] According to some embodiments of the present invention, the integrated management and control platform adaptively instructs the distributed system encoder to select a deep compression encoding mode or a shallow compression encoding mode when encoding according to the network environment. Here, since video transmission consumes network bandwidth in particular, in this way, network bandwidth and video image quality can be flexibly taken into account.

[0089] In addition, the multi-link composite video transmission method also includes other process details described above around the various figures.

[0090] Figure 4 A module diagram of an integrated management device 4000 according to some embodiments of the present invention is shown.

[0091] As shown in the figure, the device includes: a monitoring module 4001, a judgment module 4002 and a switching module 4003.

[0092] The monitoring module 4001 is used to receive data sent by various components, which may include, for example, the online status, fault problems, names, device temperature, etc. of components such as distributed system encoders, desktop screen capture clients, distributed system decoders, switches, and centralized system processors.

[0093] The judgment module 4002 is used to judge the status of each transmission link based on the data. For example, when the temperature of a device on a transmission link is too high, it will be judged that the transmission link may be prone to failure, so link switching needs to be considered.

[0094] The switching module 4003 is used to switch the first transmission link of the multi-link composite video transmission system to the second transmission link according to the judgment result. For example, when any logical node in the first transmission link is monitored to be abnormal, the first transmission link of the multi-link composite video transmission system is switched to the second transmission link.

[0095] The integrated management device can be used as part of the management and control platform to implement the multi-link composite video transmission method or part thereof. Accordingly, according to some embodiments of the present invention, the integrated management device can also include more modules to implement more detailed processes in the multi-link composite video transmission method mentioned above.

[0096] Each module mentioned above may be a program module or may be a hardware component that realizes corresponding functions.

[0097] Figure 5 1 is a schematic structural diagram of a management control device 500 according to an embodiment of the present invention. As shown in the figure, the integrated management device 500 includes a processor 51, a memory 52 and a bus 53.

[0098] In some examples, the integrated management device 500 may also include multiple input devices 501, multiple input ports 502, multiple output ports 503, and multiple output devices 504. For simplicity, only one input device, one input port, one output port, and one output device are shown in the figure. Among them, the input port 502, the processor 51, the memory 52, and the output port 503 are interconnected through the bus 53, and the input device 501 and the output device 504 are connected to the bus 53 through the input port 502 and the output port 503 respectively, and then connected to other components of the integrated management device 500. It should be noted that the input port and the output port here can also be represented by an I / O interface.

[0099] Each input device 501 transmits each input information to the processor 51 through each input port 502; the processor 51 processes or responds to the input information based on the computer executable instructions stored in the memory 52 to generate output information, and temporarily or permanently stores the output information in the memory 52, and then transmits the output information to each output device 504 through each output port 504; the output device 504 outputs the output information to another terminal as needed. The input signal can be, for example, Figure 2 The status signals sent by the distributed system encoder 101 and the desktop screen capture client are shown. The process of the processor 51 processing or responding to each input information based on the computer executable instructions stored in the memory 52 to generate output information can be understood in conjunction with the aforementioned multi-link composite video transmission method.

[0100] According to actual needs, the above-mentioned memory may be various types of memory, such as flash memory, optical disk, magnetic disk or a combination of two or more of these; it may also include removable or non-removable media; it may be inside or outside the device; it may be a non-volatile solid-state memory, read-only memory, erasable PROM (EPROM) or a combination of two or more of these.

[0101] The present application also provides a computer-readable storage medium, which may include instructions. When the computer-readable storage medium is executed on a computer, the computer may execute the above-mentioned multi-link composite video transmission method or a part of the steps thereof.

[0102] The present invention can improve the reliability of video transmission. Moreover, the present invention has flexible scalability at both the video signal source end and the display device end, and can achieve accurate design of system scale, thereby reducing system cost.

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-link composite video transmission system, comprising: The first transmission link includes a first distributed system encoder, a display and control network switch, and a distributed system decoder. The first distributed system encoder is connected to a video signal source, and the distributed system decoder is connected to a main video interface of a display device through a first video line. The first distributed system encoder is used to obtain a video signal from a video signal source and encode it, and encode it into a first network stream signal. The distributed system decoder is used to obtain and decode the first network stream signal through the display and control network switch, and send the decoded video signal to the main video interface of the display device. The second transmission link includes a desktop screen capture client installed on the video signal source, a data network switch and a centralized system processor, and the centralized system processor is connected to the backup video interface of the display device through a second video line; The desktop screen capture client is used to obtain the desktop image of the video signal source in real time and encode it into a second network stream signal; the centralized system processor is used to obtain and decode the second network stream signal through the data network switch, and send the decoded video signal to the backup video interface of the display device; An integrated management and control platform is connected to a data network switch and a display and control network switch through a core network switch, thereby being connected to a distributed system encoder, a distributed system decoder, a desktop screen capture client, and a centralized system processor network; the integrated management and control platform is used to bind the distributed system encoder and the desktop screen capture client as a first logical node, and to bind the distributed system decoder and the centralized system processor as a second logical node, and when the integrated management and control platform monitors that any logical node in the first transmission link is abnormal, enable the normal components on the second transmission link in the abnormal logical node, and keep using the normal components on the first transmission link on other normal logical nodes; A display device having the main video interface and the backup video interface, wherein the display device can freely switch between the main video interface and the backup video interface, and when both the main video interface and the backup video interface have video signals input, the display device selects the video signal input from the main video interface for display; The delay of the second transmission link is greater than that of the first transmission link, the integrated management and control platform is further used to calculate the number of image frames that need to be automatically removed according to the difference between the delay of the first transmission link and the delay of the second transmission link, and the frame rate, wherein the number of image frames = the difference x the frame rate, and the integrated management and control platform is further used to control the centralized system processor to remove the number of image frames before outputting the image externally; and / or, The integrated management and control platform is also used to calculate the number of recent image frames that need to be temporarily stored in the first transmission link based on the difference between the delay of the first transmission link and the delay of the second transmission link, and the frame rate, where the number of image frames = the difference x frame rate, and the integrated management and control platform is also used to control the distributed decoder to temporarily store the number of image frames.

2. The multi-link composite video transmission system according to claim 1, wherein: The video transmission system also includes a backup signal source, which is connected to the second distributed system encoder via a video cable, and the second distributed system encoder is further connected to the display and control network switch via a network cable; a desktop screen capture client is installed on the backup signal source, and the desktop screen capture client captures its desktop image and encodes it into a network stream signal and transmits it to the data network switch; the integrated management and control platform is used to bind the signal source and the backup signal source as a third logical node; and the second distributed system encoder and the desktop screen capture client installed on the backup signal source are bound as a fourth logical node.

3. The multi-link composite video transmission system according to claim 1, wherein: The integrated management and control platform is also used to synchronously transmit the received user instructions to the first transmission link and the second transmission link.

4. According to the multi-link composite video transmission system of claim 1, the integrated management and control platform monitors the performance parameters of each component in the first and second transmission links in real time, calculates the scores of the first and second transmission links according to the performance parameters, and selects the link with the higher score as the current transmission link, wherein the performance parameters of each component include: Encoding delay (ED) and encoding quality (EQ) of the encoder of the distributed system; network bandwidth (NB), network delay (NL), and network packet loss rate (NPL) of the display and control network switch; decoding delay (DD) and decoding quality (DQ) of the decoder of the distributed system; screen capture delay (CD) and screen capture quality (CQ) of the desktop screen capture client; network bandwidth (DB), network delay (DL), and network packet loss rate (DPL) of the data network switch; decoding delay (CSDD) and decoding quality (CSDQ) of the processor of the centralized system; First transmission link score (S1): Second transmission link score (S1): W EQ : Weight of coding quality; W DQ : The weight of decoding quality; W ED : Weight of coding delay; W NL : The weight of network delay; W NPL : The weight of network packet loss rate; W DD : Weight of decoding delay; W CQ : Weight of screen capture quality; W CD : Weight of screen capture delay; W DL : The weight of network delay; W DPL : The weight of network packet loss rate; W CSDQ : The weight of decoding quality; W CSDD : Weight of decoding delay.

5. The multi-link composite video transmission system according to claim 3, wherein: When the integrated management and control platform monitors that any logical node in the first transmission link is abnormal, the first transmission link of the multi-link composite video transmission system is switched to the second transmission link.

6. A multi-link composite video transmission method based on the multi-link composite video transmission system according to any one of claims 1 to 5, comprising: The distributed system encoder of the first transmission link obtains the video signal of the video signal source and encodes it into a first network stream signal; The distributed system decoder obtains the first network stream signal through the network and decodes it, and sends the decoded video signal to the main video interface of the display device for display; The desktop screen capture client of the second transmission link obtains the desktop image of the video signal source in real time and encodes it into a second network stream signal; The centralized system processor acquires the second network stream signal through the network and decodes it, and sends the decoded video signal to the backup video interface of the display device; The integrated management and control platform monitors the first transmission link in real time. When it is monitored that any logical node in the first transmission link is abnormal, a normal component on the second transmission link in the abnormal logical node is enabled, and normal components on the first transmission link on other normal logical nodes are kept in use. as well as The display device freely switches between the main video interface and the backup video interface to display the video signal inputted into the main video interface or the backup video interface, and when the video signal is inputted into both the main video interface and the backup video interface, the video signal inputted into the main video interface is selected for display; Wherein, the delay of the second transmission link is greater than that of the first transmission link, and the integrated management and control platform further calculates the number of image frames that need to be automatically removed according to the difference between the delay of the first transmission link and the delay of the second transmission link, and the frame rate, wherein the number of image frames = the difference x the frame rate, and the integrated management and control platform further controls the centralized system processor to remove the image frames of the number before outputting the image externally; and / or, The integrated management and control platform also calculates the number of recent image frames that need to be temporarily stored in the first transmission link based on the difference between the delay of the first transmission link and the delay of the second transmission link, as well as the frame rate, where the number of image frames = the difference x frame rate, and the integrated management and control platform also controls the distributed decoder to temporarily store the number of image frames.

7. The multi-link composite video transmission method according to claim 6, comprising: when it is monitored that any logical node in the first transmission link is abnormal, switching the first transmission link of the multi-link composite video transmission system to the second transmission link.

8. The multi-link composite video transmission method according to claim 6, wherein: When the integrated management and control platform monitors that the number of video windows in the centralized system processor is greater than a threshold, the integrated management and control platform instructs the distributed system encoder and / or the desktop capture client to adopt a deep compression encoding mode when encoding.

9. The multi-link composite video transmission method according to claim 6 further includes the integrated management and control platform synchronously transmitting the received user instructions to the first transmission link and the second transmission link.

Images