Transmission system for multi-video-stream input convergence and single-screen display output

Abstract

The invention discloses a transmission system for multi-video-stream input convergence and single-screen display output. The system comprises a plurality of video source devices, a video processor and a display device, wherein the video source devices are used for sending video data through a multi-stream transmission interface; the video processor is used for receiving video data through the multi-stream transmission interface, converting the received video data into multiple pieces of single-stream video data, then preprocessing the multiple pieces of single-stream video data to form multiple pieces of single-stream video data with specified frame rates, and fusing the multiple pieces of single-stream video data with specified frame rates to form multiple pieces of single-stream video data with specified frame rates. Generating single-stream video data in a specified video format, and sending the single-stream video data in the specified video format through a video interface; the display device is used for receiving the single-stream video data through the video interface and controlling the single-stream video data to be displayed. According to the invention, the problem of chip pin resource limitation caused by a plurality of input video interfaces and the problem of cable complexity and entanglement caused by the plurality of input video interfaces can be avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of video transmission, and in particular to a transmission system for multi-video stream input, convergence and single-screen display output. Background Art

[0002] The method of multi-video fusion depends mainly on the specific application scenario and technical implementation. The following are some common methods of fusion: (1) Plane splicing and fusion Split the screen into multiple areas and play different videos simultaneously. Specific splicing methods include: a) Horizontal splicing One of the most common stitching methods, this method is suitable for scenes where multiple cameras are arranged horizontally. In this method, the system stitches the video streams from different cameras together horizontally to form a video image with a wider viewing angle.

[0003] b) Vertical splicing Similar to horizontal stitching, vertical stitching is suitable for scenarios where cameras are arranged vertically. In this way, the system can vertically stitch video streams from different cameras together to expand the height of the video image.

[0004] c) Grid stitching When you need to monitor multiple independent areas simultaneously, you can use grid stitching. In this way, multiple video images are combined in a grid format, with each grid representing an independent video source. This stitching method allows users to observe the situation in multiple areas at the same time.

[0005] d) Custom splicing In addition to the common splicing methods mentioned above, you can also customize splicing according to specific needs.

[0006] (2) Overlay and fusion Achieve image blending by adjusting layer transparency or blending algorithms (such as screen and overlay).

[0007] The existing technical solutions for multi-stream video signal processors are as follows: Each video source device uses an SST (Single-Stream Transport) interface for video data transmission, such as the DisplayPort (DP) interface in SST mode, the HDMI interface, the MIPI interface in SST mode, or the LVDS interface. Each interface transmits only one video stream. The video processor interfaces with a dedicated video interface receiver for each video source device, and the video interface receiver only needs to support SST single-stream transmission mode. The received video data is aggregated and processed within the video signal processor, including operations such as multi-view splicing or overlay, and output as a single-stream video data in the specified video format. The video interface transmitter then transmits this single-stream video data to the display device for display.

[0008] Disadvantages of existing technical solutions for multi-stream video processors: a) Each video source device is connected to the video processor via a cable. If the transmission is over long distances, the cables will become tangled and inconvenient for the design and organization of the entire system. b) Video processing chips need to provide multiple input video interfaces. The number of input video interfaces is limited by the number of chip pins, and generally only 2 to 4 input video interfaces are provided. Increasing the number of input video interfaces on a single chip will increase the cost of the chip. c) The number of input video interfaces of the video processing chip is smaller than that of the entire system, requiring multiple video processing chips to be cascaded, which increases the cost of the entire system. d) Current mainstream video source devices support MST multi-stream transmission interface output and cannot directly connect to single-stream video interface receivers in MST mode. They can only transmit single-stream video data in SST mode.

[0009] e) Current mainstream video source devices support VRR variable refresh rate video data processing technologies (such as FreeSync, G-SYNC, VESA AdaptiveSync, and other video processing technologies suitable for VRR), but video processing chips generally do not support VRR variable refresh rate video data processing technologies. Summary of the Invention

[0010] The purpose of the present invention is to provide a transmission system for multi-video stream input convergence and single-screen display output in response to the deficiencies in the prior art.

[0011] To achieve the above objectives, the present invention provides a transmission system for aggregating multiple video stream inputs and outputting them on a single screen, comprising: Several video source devices are used to send video data through a multi-stream transmission interface; a video processor, configured to receive video data sent by the video source device through a multi-stream transmission interface, convert the received video data into a plurality of single-stream video data, pre-process the plurality of single-stream video data to form a plurality of single-stream video data at a specified frame rate, fuse the plurality of single-stream video data at the specified frame rate to generate single-stream video data in a specified video format, and send the single-stream video data in the specified video format through a video interface; The display device is used to receive the single-stream video data sent by the video processor through a video interface and control the display.

[0012] Furthermore, the video processor includes: A first video interface receiver, configured to receive video data sent by the video source device through a multi-stream transmission interface, and convert the received video data into a plurality of single-stream video data; A video data preprocessor, configured to preprocess a plurality of single-stream video data to form a plurality of single-stream video data at a specified frame rate; A video signal processor, configured to process a plurality of single-stream video data of a specified frame rate to generate single-stream video data of a specified video format; The video interface transmitter is used to transmit the single-stream video data in the specified video format to the display device through the video interface.

[0013] Furthermore, the first video interface receiver includes: A receiving module, configured to receive video data sent by the video source device and perform serial-to-parallel conversion on the received video data to obtain physical layer parallel data; The physical layer decoding module is used to decode the physical layer parallel data according to the physical layer encoding method specified by the interface protocol to obtain the protocol layer data; The protocol layer parsing module is used to parse the protocol layer data according to the protocol layer encoding method specified by the interface protocol to obtain multi-stream data; A multi-stream data distribution module is used to distribute the multi-stream data according to the data placement rules specified by the interface protocol and output it as multiple single-stream data; The single-stream data generation module is used to generate single-stream video data from each single-stream data in a video format.

[0014] Furthermore, the video data preprocessor includes: a decompression module, configured to decompress the single-stream video data output by the first video interface receiver to obtain original video data of the application; The frame rate conversion module is used to convert the frame rate of the original video data into a specified frame rate.

[0015] Furthermore, the frame rate conversion module converts the frame rate of the original video data into the specified frame rate in the following manner: When the internal frame buffer stores output data and needs frame data, and the currently input frame data has not yet ended, the previous frame data is output repeatedly; when data is input to the internal frame buffer for storage, if all internal frame buffers are full, the input data will overwrite the frame data of the previous frame that has not yet been output in the internal frame buffer.

[0016] Furthermore, the video source device supports the Type-C / DP interface or MIPI interface in MST mode.

[0017] Furthermore, there are multiple video source devices, and they are respectively connected to the video interface converter via a single-stream transmission interface, and the video interface converter is connected to the video processor via a multi-stream transmission interface.

[0018] Furthermore, the video processor further includes a second video interface receiver, which is connected to another video source device via a single stream transmission interface and is connected to the video signal processor via a video data preprocessor.

[0019] Furthermore, the fusion processing method includes multi-image splicing fusion or multi-image synthesis fusion.

[0020] Beneficial effects: By using a multi-stream transmission interface to send video data, the present invention can directly connect to current mainstream video source devices that support MST multi-stream transmission interface output, avoiding the chip pin resource limitation problem caused by the number of multiple input video interfaces and the cumbersome cable entanglement problem caused by multiple input video interfaces; the video data preprocessor can meet the video compression transmission processing requirements, solve the limitation problem of the limited bandwidth of the MST interface, and better play the advantages of the MST interface in transmitting multi-stream video data; the video data preprocessor uses frame rate conversion technology to output video data at a fixed frame rate, which can effectively simplify the design of the video signal processor, reduce internal logic resources, and save costs and power consumption; the specific implementation scheme of the video signal processor can be selected according to actual conditions, so that the system has better system flexibility. At the same time, each stream supports VRR variable refresh rate video data processing technology (such as FreeSync, G-SYNC, VESA AdaptiveSync and other video processing technologies suitable for VRR), and can directly connect to current mainstream video source devices that support VRR processing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a principle block diagram of a transmission system for multi-video stream input, convergence, and single-screen display output according to an embodiment of the present invention; Figure 2 is a principle block diagram of a video processor according to an embodiment of the present invention; Figure 3 This is a system principle block diagram of Example 1 of the present invention; Figure 4 This is a system principle block diagram of embodiment 2 of the present invention; Figure 5 This is a system principle block diagram of embodiment 3 of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. These embodiments are implemented based on the technical solutions of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0023] like Figure 1 As shown, an embodiment of the present invention provides a transmission system for multi-video stream input convergence and single-screen display output, including several video source devices 1, a video processor 2 and a display device 3.

[0024] Among them, the video source device 1 is used to send video data through the multi-stream transmission interface (MST). The video processor 2 is used to receive the video data sent by the video source device 1 through the multi-stream transmission interface, and convert the received video data into multiple single-stream video data, and then pre-process the multiple single-stream video data to form multiple single-stream video data with specified frame rates, and perform fusion processing on the multiple single-stream video data with specified frame rates to generate single-stream video data with specified video formats, and send the single-stream video data with specified video formats through the video interface. The above-mentioned fusion processing includes operations such as multi-screen splicing and fusion or multi-screen overlay and fusion. The display device 3 is used to receive the single-stream video data sent by the video processor 2 through the video interface and control the display.

[0025] See also Figure 2 The video processor 2 of the embodiment of the present invention includes a first video interface receiver 21 , a video data preprocessor 22 , a video signal processor 23 and a video interface transmitter 24 .

[0026] The first video interface receiver 21 is configured to receive video data transmitted by the video source device 1 via a multi-stream transmission interface and convert the received video data into multiple single-stream video data. Specifically, the first video interface receiver 21 in this embodiment of the present invention includes a physical layer receiving module 211, a physical layer decoding module 212, a protocol layer parsing module 213, a multi-stream data distribution module 214, and a single-stream data generation module 215. The physical layer receiving module 211 is configured to receive video data transmitted by the video source device and perform serial-to-parallel conversion on the received video data to obtain physical layer parallel data. The physical layer decoding module 212 is configured to decode the physical layer parallel data according to the physical layer encoding method specified by the interface protocol to obtain protocol layer data. The protocol layer parsing module 213 is configured to parse the protocol layer data according to the protocol layer encoding method specified by the interface protocol to obtain multi-stream data. The multi-stream data distribution module 214 is configured to distribute the multi-stream data according to the data placement rules specified by the interface protocol, outputting the data as multiple single-stream data. The single-stream data generating module 215 is used to generate single-stream video data from each single-stream data according to a video format, and finally send the generated single-stream data to the video data preprocessor 22 .

[0027] The video data preprocessor 22 is used to preprocess multiple single-stream video data to form multiple single-stream video data at a specified frame rate. Specifically, the video data preprocessor 22 includes a decompression module 221 and a frame rate conversion module 222. The decompression module 221 is used to decompress the single-stream video data output by the first video interface receiver 21 to obtain the application's original video data. Taking the DSC compression algorithm as an example for transmitting video data, the specific process is as follows: 1) The video source device 1 actively obtains the support capability of the first video interface receiver 21 and determines whether the first video interface receiver 21 supports the DSC decompression capability; if supported, continues to obtain the capability range of supporting DSC decompression.

[0028] 2) The video source device 1 determines whether to use the DSC compression technology for data transmission based on the currently used interface bandwidth and the bandwidth required for the video data to be transmitted.

[0029] 3) When the video source device 1 compresses the video data using the DSC compression technology and then transmits it, the video source device 1 will send a PPS information packet in advance before sending each frame of compressed data, which includes the parameters required for DSC decompression.

[0030] 4) After receiving the PPS information packet, the first video interface receiver 21 parses the parameters required for DSC decompression and sends them to the decompression module 221 for DSC decompression.

[0031] The frame rate conversion module 222 is used to convert the VRR variable frame rate of the original video data to the specified frame rate. Specifically, the frame rate conversion module 222 uses the frame data stored in the internal frame buffer (storage size of 3 frames of data) to convert the VRR variable frame rate of the original video data to the specified frame rate. Its implementation principle is as follows: 1) There is a certain delay between internal frame buffer storage input and output. This means that the output is triggered only after the input is buffered for a certain period of time. This is to offset the dynamic offset of the VFront (vertical sync leading edge) in the VRR (variable refresh rate) data. Generally, a full frame of data can be buffered before output.

[0032] 2) When the dynamic offset of VFront exceeds the range tolerated by the input and output delay, the frame data is repeatedly displayed or discarded based on the current existing data: i) When the internal frame buffer stores output data and needs frame data, and the current input frame data has not yet ended, the previous frame data is repeatedly output; ii) When data is input to the internal frame buffer for storage, if all internal frame buffers are full, the input data overwrites the frame data of the previous frame that has not yet been output from the internal frame buffer.

[0033] At the same time, the frame rate conversion module 222 also supports the conversion of other fixed frame rate data into video data of a specified frame rate, including the conversion of video data exceeding the specified frame rate and video data below the specified frame rate.

[0034] The video signal processor 23 is used to process a plurality of single-stream video data of a specified frame rate to generate single-stream video data of a specified video format. The video interface transmitter 24 is used to transmit the single-stream video data of the specified video format to the display device 3 via the video interface. Example 1:

[0035] See also Figure 3, use the Type-C / DP MST interface as the input interface to connect to the video source device 1 that supports Type-C / DP MST, receive two streams of video data, and then send the video stream data to the video signal processor 23 for PBP (Picture By Picture) or PIP (Picture In Picture) function implementation, and output as single-stream video data in the specified video format. The single-stream video data is then sent to the display device 3 for display through the video interface transmitter 24. In this way, it is easy to achieve multiple uses of one screen without interfering with each other, which helps to facilitate efficient office work. In this case, the PBP / PIP function is implemented. In theory, as long as the requirements of the interface bandwidth and DSC compression algorithm are met, it can be configured into a suitable video format. And each stream supports VRR variable refresh rate video data processing technology (such as FreeSync, G-SYNC, VESA AdaptiveSync and other video processing technologies suitable for VRR). The following are common classic scenarios: 1) The video format of the two input video streams is 1080p (1920x1080), and the video format of the output video stream is 4K (3840x2160).

[0036] 2) The video format of the two input video streams is 2.5K (2560x1440), and the video format of the output video stream is 5K (5120x2880).

[0037] 3) The video format of the two input video streams is 4K (3840x2160), and the video format of the output video stream is 8K (7680x4320).

[0038] 4) The video format of the two input video streams is 8K (7680x4320), and the video format of the output video stream is 16K (15360x8640).

[0039] 5) The output video format can also be a custom resolution, such as 6K and other non-mainstream video formats. Example 2:

[0040] See also Figure 4Using the DP MST interface as the long-distance transmission interface, the upstream device can be considered a video source device system, consisting of N video source devices 1 and a video interface converter 4, which converts N video streams into a single DP MST interface data stream for transmission. The downstream device includes a video processor 2 and a display device 3. The first video interface receiver 21 of the video processor 2 interfaces with the DP MST interface of the video interface converter 4, receiving multiple streams of video data. This data is then sent to the video signal processor 23 for data fusion processing, including multi-view splicing and fusion, and output as a single stream of video data in a specified video format. This single stream of video data is then sent to the display device 3 via the video interface transmitter 24 for display. This allows for easy long-distance transmission using a single cable, simplifying the overall system design. Given the bandwidth of the DP MST interface, DSC compression technology can be used to compress and decompress video data for data transmission. Furthermore, each stream supports VRR variable refresh rate video data processing technologies (such as FreeSync, G-SYNC, VESA AdaptiveSync, and other VRR-compatible video processing technologies). Example 3:

[0041] See also Figure 5 In order to be compatible with more usage scenarios in actual product definition, the video processor 2 also includes a second video interface receiver, which is connected to another video source device through a single stream transmission interface, and is connected to the video signal processor 23 through a video data preprocessor 22. Figure 5 The second video interface receiver shown in the figure is an HDMI interface receiver 25, and the other video source device is an HDMI source device 5. Multiple video stream data are sent to the video signal processor 23 through the DP MST+HDMI mixed input interface for data fusion processing, including operations such as multi-screen splicing and fusion or multi-screen overlay fusion, and output as single-stream video data in a specified video format. The single-stream video data is then sent to the display device 3 for display. In view of the bandwidth of the DP MST interface and the HDMI interface, DSC compression technology can be used in the solution to compress and decompress the video data to achieve the purpose of data transmission. At the same time, each stream supports VRR variable refresh rate video data processing technology (such as FreeSync, G-SYNC, VESA AdaptiveSync and other video processing technologies suitable for VRR).

[0042] The above description is merely a preferred embodiment of the present invention. It should be noted that any other aspects not specifically described are considered prior art or common knowledge to those skilled in the art. Improvements and modifications may be made without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention.

Claims

1. A transmission system for multi-video stream input convergence and single-screen display output, characterized in that: include: Several video source devices are used to send video data through a multi-stream transmission interface; a video processor, configured to receive video data sent by the video source device through a multi-stream transmission interface, convert the received video data into a plurality of single-stream video data, pre-process the plurality of single-stream video data to form a plurality of single-stream video data at a specified frame rate, fuse the plurality of single-stream video data at the specified frame rate to generate single-stream video data in a specified video format, and send the single-stream video data in the specified video format through a video interface; The display device is used to receive the single-stream video data sent by the video processor through a video interface and control the display.

2. The transmission system for multi-video stream input convergence and single-screen display output according to claim 1, characterized in that: The video processor comprises: A first video interface receiver, configured to receive video data sent by the video source device through a multi-stream transmission interface, and convert the received video data into a plurality of single-stream video data; A video data preprocessor, configured to preprocess a plurality of single-stream video data to form a plurality of single-stream video data at a specified frame rate; A video signal processor, configured to process a plurality of single-stream video data of a specified frame rate to generate single-stream video data of a specified video format; The video interface transmitter is used to transmit the single-stream video data in the specified video format to the display device through the video interface.

3. The transmission system for multi-video stream input convergence and single-screen display output according to claim 2, characterized in that: The first video interface receiver comprises: A physical layer receiving module, configured to receive the video data sent by the video source device and perform serial-to-parallel conversion on the received video data to obtain physical layer parallel data; The physical layer decoding module is used to decode the physical layer parallel data according to the physical layer encoding method specified by the interface protocol to obtain the protocol layer data; The protocol layer parsing module is used to parse the protocol layer data according to the protocol layer encoding method specified by the interface protocol to obtain multi-stream data; A multi-stream data distribution module is used to distribute the multi-stream data according to the data placement rules specified by the interface protocol and output it as multiple single-stream data; The single-stream data generation module is used to generate single-stream video data from each single-stream data in a video format.

4. The transmission system for multi-video stream input convergence and single-screen display output according to claim 1, characterized in that: The video data preprocessor comprises: a decompression module, configured to decompress the single-stream video data output by the first video interface receiver to obtain original video data of the application; The frame rate conversion module is used to convert the frame rate of the original video data into a specified frame rate.

5. The transmission system for multi-video stream input convergence and single-screen display output according to claim 4, characterized in that: The frame rate conversion module converts the frame rate of the original video data into the specified frame rate in the following manner: When the internal frame buffer stores output data and needs frame data, and the currently input frame data has not yet ended, the previous frame data is output repeatedly; when data is input to the internal frame buffer for storage, if all internal frame buffers are full, the input data will overwrite the frame data of the previous frame that has not yet been output in the internal frame buffer.

6. The transmission system for multi-video stream input convergence and single-screen display output according to claim 1, characterized in that: The video source device supports the Type-C / DP interface or MIPI interface in MST mode.

7. The transmission system for multi-video stream input convergence and single-screen display output according to claim 1, characterized in that: There are multiple video source devices, and each of them is connected to a video interface converter via a single-stream transmission interface. The video interface converter is connected to a video processor via a multi-stream transmission interface.

8. The transmission system for multi-video stream input convergence and single-screen display output according to claim 2, characterized in that: The video processor further includes a second video interface receiver, which is connected to another video source device through a single stream transmission interface and is connected to the video signal processor through a video data preprocessor.

9. The transmission system for multi-video stream input convergence and single-screen display output according to claim 1, characterized in that: The fusion processing method includes multi-image splicing fusion or multi-image superposition fusion.

Images