Video stream transmission method, video stream echo method and related components

Abstract

The invention is suitable for the technical field of display screens, and provides a video stream transmission method, an echo method and related components. The video stream transmission method comprises the following steps: acquiring multiple paths of input video streams; performing refresh rate reduction processing on the multiple paths of input video streams to obtain corresponding multiple paths of refresh rate reduction video streams; synthesizing the multiple paths of refresh rate reduced video streams to obtain a to-be-transmitted video stream, N continuous frames of video pictures in the to-be-transmitted video stream being respectively from N paths of sub-video streams, one path of sub-video stream comprising a video picture corresponding to at least one path of refresh rate reduced video stream, and N being a positive integer greater than 1; and transmitting the video stream to be transmitted. According to the embodiment of the invention, the image resolution and the echo quantity during video stream transmission can be guaranteed in a limited transmission bandwidth, and the field echo demand is flexibly supported.

Description

Technical Field

[0001] The present application belongs to the technical field of display screens, and in particular, relates to a video stream transmission method, an echo method, and related components. Background Art

[0002] For large-scale video processing equipment, users usually need to watch the echo screen of the input source through the playback control software, so as to realize the visual operation, so that the users can intuitively view the display screen from the software end.

[0003] The number of input sources supported by current video processing equipment ranges from a few to dozens, and the number of input sources actually used in different sites is often different. For application scenarios with a large number of input sources, the internal and external hardware transmission bandwidth and the number of echoes of the video processing equipment are limited, making it difficult to guarantee the image resolution and number of echoes required during video stream transmission, resulting in low quality of the echo images received by the control software or the inability to flexibly support the echo requirements of the site, which is inconvenient for users' visual operations. Summary of the invention

[0004] The embodiments of the present application provide a video stream transmission method, an echo method, and related components, which can solve the problem in the related art that the echo image received by the control software is of low quality or cannot flexibly support the on-site echo requirements.

[0005] A first aspect of an embodiment of the present application provides a method for transmitting a video stream, which is applied to a video processing device. The method for transmitting a video stream comprises: obtaining multiple input video streams; performing refresh rate reduction processing on the multiple input video streams to obtain corresponding multiple reduced refresh rate video streams; performing synthesis processing on the multiple reduced refresh rate video streams to obtain a video stream to be transmitted, wherein N consecutive frames of video images in the video stream to be transmitted are respectively from N sub-video streams, wherein one sub-video stream includes a video image corresponding to at least one reduced refresh rate video stream, and N is a positive integer greater than 1; and transmitting the video stream to be transmitted.

[0006] In some embodiments of the present application, each of the multiple input video streams corresponds to an input interface; the synthesis processing of the multiple reduced refresh rate video streams to obtain the video stream to be transmitted includes: determining the target frame number of each of the input interfaces, the target frame number being the frame number of the sub-video stream to which the reduced refresh rate video stream corresponding to the input interface belongs in the video stream to be transmitted; for each of the input interfaces, according to the current frame number and the target frame number, synthesizing the video picture of the corresponding reduced refresh rate video stream to obtain the video stream to be transmitted.

[0007] In some embodiments of the present application, for each of the input interfaces, the video images of the corresponding reduced refresh rate video stream are synthesized according to the current frame number and the target frame number to obtain the video stream to be transmitted, including: for the input interface, when the current frame number is the corresponding target frame number, the video images of the corresponding reduced refresh rate video stream are synthesized, and the synthesized video stream is transmitted to the next input interface for synthesis processing until it is transmitted to the last input interface to obtain the video stream to be transmitted.

[0008] In some embodiments of the present application, for each of the input interfaces, the video images of the corresponding reduced refresh rate video stream are synthesized according to the current frame number and the target frame number to obtain the video stream to be transmitted, including: according to the target frame number, controlling each of the input interfaces to transmit the video image required by the current frame number in the reduced refresh rate video stream when receiving a frame synchronization signal; and synthesizing the video images transmitted by each of the input interfaces to obtain the video stream to be transmitted.

[0009] In some implementations of the present application, the transmitting the video stream to be transmitted includes: compressing the video stream to be transmitted, and transmitting the compressed video stream.

[0010] In some embodiments of the present application, compressing the video stream to be transmitted and transmitting the compressed video stream includes: restoring the video stream to be transmitted into the N sub-video streams; compressing the N sub-video streams respectively to obtain N compressed sub-video streams; and transmitting the N compressed sub-video streams.

[0011] A second aspect of an embodiment of the present application provides a method for echoing a video stream, which is applied to a terminal device, wherein the terminal device is configured with configuration software, the configuration software includes a display interface, the display interface includes at least one display window, and each display window corresponds to an input interface; the method for echoing the video stream includes: obtaining a video stream to be transmitted; determining an echo video stream for each input interface based on the video stream to be transmitted; and displaying the corresponding echo video stream in the display window.

[0012] A third aspect of an embodiment of the present application provides a video stream transmission device, which is configured in a video processing device, and the video stream transmission device includes: an acquisition unit, used to acquire multiple input video streams; a processing unit, used to reduce the refresh rate of the multiple input video streams to obtain corresponding multiple reduced refresh rate video streams; a synthesis unit, used to synthesize the multiple reduced refresh rate video streams to obtain a video stream to be transmitted, wherein N consecutive frames of video images in the video stream to be transmitted are respectively from N sub-video streams, each sub-video stream includes video images corresponding to at least one reduced refresh rate video stream, and N is a positive integer greater than 1; a transmission unit, used to transmit the video stream to be transmitted.

[0013] According to a fourth aspect of an embodiment of the present application, there is provided a device for displaying a video stream, which is configured on a terminal device. The terminal device is configured with configuration software, and the configuration software includes a display interface. The display interface includes at least one display window, and each display window corresponds to an input interface. The device for displaying a video stream includes: an acquisition unit, which is used to acquire a video stream to be transmitted; a determination unit, which is used to determine an echo video stream of each input interface according to the video stream to be transmitted; and an echo unit, which is used to display the corresponding echo video stream in a display window.

[0014] A fifth aspect of an embodiment of the present application provides a video processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the video stream transmission method described in the first aspect when executing the computer program.

[0015] A sixth aspect of an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the video stream echo method described in the second aspect when executing the computer program.

[0016] A seventh aspect of an embodiment of the present application provides a video processing system, comprising the video processing device as described in the fifth aspect, and a terminal device as described in the sixth aspect connected to the video processing device.

[0017] An eighth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video stream transmission method described in the first aspect are implemented, or when the computer program is executed by a processor, the steps of the video stream echo method described in the second aspect are implemented.

[0018] A seventh aspect of an embodiment of the present application provides a computer program product. When the computer program product runs on a video processing device, the video processing device executes the steps of the video stream transmission method described in the first aspect above; or, when the computer program product runs on a terminal device, the terminal device executes the steps of the video stream echo method described in the second aspect above.

[0019] In an implementation manner of the present application, multiple input video streams are obtained, refresh rate reduction processing is performed on the multiple input video streams to obtain corresponding multiple reduced refresh rate video streams, and the multiple reduced refresh rate video streams are synthesized to obtain a video stream to be transmitted, so as to transmit the video stream to be transmitted, wherein the continuous N frames of video images in the video stream to be transmitted come from N sub-video streams respectively, wherein one sub-video stream includes video images corresponding to at least one reduced refresh rate video stream, thereby being able to ensure the image resolution and the number of echoes during video stream transmission within a limited transmission bandwidth, making it convenient for users to view high-quality echo images, and being able to flexibly support on-site echo requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 It is a schematic diagram of an implementation flow of a video stream transmission method provided in an embodiment of the present application;

[0022] Figure 2 It is a schematic diagram of a video stream with a reduced refresh rate to form a sub-video stream provided by an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of the interval interpolation provided in an embodiment of the present application;

[0024] Figure 4 This is a first schematic diagram of a video processing device performing synthesis processing provided by an embodiment of the present application;

[0025] Figure 5 is a second schematic diagram of the video processing device provided in the embodiment of the present application performing synthesis processing;

[0026] Figure 6 It is a schematic diagram of a control chip provided in an embodiment of the present application for performing video stream transmission;

[0027] Figure 7 It is a schematic diagram of the implementation process of a method for echoing a video stream provided in an embodiment of the present application;

[0028] Figure 8 is a schematic diagram of a display interface of the configuration software provided in an embodiment of the present application;

[0029] Fig. 9 It is a structural schematic diagram of a video stream transmission device provided in an embodiment of the present application;

[0030] Fig.10 It is a structural schematic diagram of a video stream echoing device provided in an embodiment of the present application;

[0031] Fig.11 is a structural schematic diagram of a video processing device provided in an embodiment of the present application;

[0032] Fig.12 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0033] Fig.13 It is a structural diagram of a video processing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are protected by the present application.

[0035] The number of input sources supported by current video processing equipment ranges from a few to dozens, and the number of input sources actually used in different sites is often different. For application scenarios with a large number of input sources, the internal and external hardware transmission bandwidth of the video processing equipment is limited, making it difficult to guarantee the image resolution and number of echoes during video stream transmission, resulting in low quality of the echo images received by the playback control software or the inability to flexibly support the echo requirements of the site, which is inconvenient for users' visual operations.

[0036] In view of this, the present application proposes a video stream transmission method and an echo method, which can ensure the picture resolution and echo quantity requirements during video stream transmission within a limited transmission bandwidth through a synthesis method of frequency reduction and interval frame insertion, so as to facilitate users to watch high-quality echo images and flexibly support on-site echo requirements.

[0037] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0038] Figure 1A schematic diagram of the implementation flow of a video stream transmission method provided in an embodiment of the present application is shown. The method can be applied to a video processing device and is suitable for situations where flexible support for on-site echo requirements is required.

[0039] Among them, the above-mentioned video processing device can be a seamless switcher, a splicing processor or other intelligent devices with video processing and video transmission capabilities, and this application does not impose any restrictions on this.

[0040] Specifically, the above-mentioned video stream transmission method may include the following steps S101 to S104.

[0041] Step S101, obtaining multiple input video streams.

[0042] The input video stream refers to the video stream received by the video processing device.

[0043] In an implementation manner of the present application, the video processing device may obtain each of the multiple input interfaces to receive one input video stream respectively, wherein the multiple input interfaces are arranged on the video processing device or on at least one input card.

[0044] Specifically, in some embodiments of the present application, the video processing device may be a card-type device, in which multiple input cards may be inserted, and each input card may be configured with one or more input interfaces. The input card may be a chip with image processing capabilities, such as a field programmable gate array (FPGA) chip or a video processing IC. At this time, the video processing device may receive one input video stream through each input interface on the input card.

[0045] In other embodiments of the present application, the video processing device may also be a non-card device, in which case the video processing device may be configured with multiple input interfaces to receive one input video stream through each of the multiple input interfaces. The input interface may be a display port (DP), a high definition multimedia interface (HDMI), a digital visual interface (DVI), etc., and the present application does not limit this.

[0046] Step S102, performing refresh rate reduction processing on multiple input video streams to obtain corresponding multiple reduced refresh rate video streams.

[0047] Specifically, for a plurality of consecutive video frames in a single input video stream, one of the video frames may be retained and the remaining frames may be discarded to reduce the refresh rate of the input video stream.

[0048] After each input video stream is processed with a reduced refresh rate, a corresponding reduced refresh rate video stream can be obtained. The reduced refresh rate video stream is the video stream obtained after the reduced refresh rate processing.

[0049] Step S103, synthesizing multiple channels of reduced refresh rate video streams to obtain a video stream to be transmitted.

[0050] In an embodiment of the present application, the N consecutive video frames in the video stream to be transmitted are respectively from N sub-video streams, and one sub-video stream may include video frames corresponding to at least one reduced refresh rate video stream.

[0051] Specifically, the video pictures of one or more reduced refresh rate video streams can be combined to form the video pictures of one sub-video stream. In the case where the video pictures of one sub-video stream include the video pictures corresponding to one reduced refresh rate video stream, the one sub-video stream includes the video pictures corresponding to one reduced refresh rate video stream. In the case where the video pictures of one sub-video stream include the video pictures corresponding to multiple reduced refresh rate video streams, the one sub-video stream includes the corresponding video splicing pictures after the multiple reduced refresh rate video streams are spliced.

[0052] For example, Figure 2 As shown, a frame of video in each channel of reduced refresh rate video stream can be mapped to a different position of a frame of video in the sub-video stream, and spliced ​​to form a frame of video in the sub-video stream.

[0053] After obtaining N sub-video streams, interpolation processing can be performed on the video pictures of the N sub-video streams, and the video pictures of the N sub-video streams are sequentially used as a frame of video picture in the video stream to be transmitted to implement mapping processing.

[0054] For example, Figure 3As shown, in 4 4K×2K@15Hz sub-video streams. The first frame of the first sub-video stream can be used as the first frame of the video stream to be transmitted, the first frame of the second sub-video stream can be used as the second frame of the video stream to be transmitted, the first frame of the third sub-video stream can be used as the third frame of the video stream to be transmitted, and the first frame of the fourth sub-video stream can be used as the fourth frame of the video stream to be transmitted. So far, the interval interpolation process of N consecutive (N=4) frames is completed. Next, the second frame of the first sub-video stream can be used as the fifth frame of the video stream to be transmitted, and so on, a 4K×2K@60Hz video stream to be transmitted can be obtained.

[0055] Step S104: transmitting the video stream to be transmitted.

[0056] Specifically, in the implementation manner of the present application, the video stream to be transmitted may be transmitted inside the video processing device, or may be transmitted from the video processing device to an external terminal device, and the present application does not impose any limitation on this.

[0057] Taking the hardware transmission bandwidth supporting 4K×2K@60Hz video stream as an example, in the related technology, in order to transmit multiple 4K×2K@60Hz input video streams, if the device has 64 input sources, the mapping is performed according to 8x8 grids, and the mapping size allocated to each input source is only 512x270, that is, the picture resolution of each input video stream needs to be compressed to 512x270 for transmission. At this time, the echo screen of any input source seen by the user in the configuration software is 512x270, with poor clarity and low resolution. By adopting the method of this application, the picture resolution of each input source can be correspondingly improved by reducing the refresh rate. For example, when the refresh rate is reduced by four times, the echo screen of each input source can support a resolution of 4K×2K, and it also ensures that the echo screen of each input source can be transmitted.

[0058] In an implementation manner of the present application, multiple input video streams are obtained, refresh rate reduction processing is performed on the multiple input video streams to obtain corresponding multiple reduced refresh rate video streams, and the multiple reduced refresh rate video streams are synthesized to obtain a video stream to be transmitted, so as to transmit the video stream to be transmitted, wherein the continuous N frames of video images in the video stream to be transmitted come from N sub-video streams respectively, wherein one sub-video stream includes video images corresponding to at least one reduced refresh rate video stream, thereby being able to ensure the image resolution and the number of echoes during video stream transmission within a limited transmission bandwidth, thereby facilitating users to view high-quality echo images and flexibly supporting on-site echo requirements.

[0059] Moreover, this interpolation method does not require too much storage space, compared with inserting one video stream for a period of time and then inserting another video stream for a period of time.

[0060] In some embodiments of the present application, the video processing device may determine a preset target refresh rate, and reduce the refresh rate of each of the multiple input video streams to the target refresh rate to obtain corresponding multiple reduced refresh rate video streams.

[0061] Specifically, the user can set the required target refresh rate or reduced refresh rate multiple in the configuration software of the terminal device according to his or her needs for the picture resolution. The video processing device obtains the target refresh rate from the terminal device, or calculates the target refresh rate based on the reduced refresh rate multiple obtained from the terminal device. For example, for an input video stream with a refresh rate of 60Hz, the user can directly set the target refresh rate to 15Hz, or set the reduced refresh rate multiple to 4, and then the video processing device can determine that the target refresh rate is 15Hz, and reduce the 60Hz input video stream to 15Hz.

[0062] At this time, the number N of sub-video streams can be determined according to the target refresh rate and the refresh rate corresponding to the hardware transmission bandwidth.

[0063] Specifically, the refresh rate supported by the hardware transmission bandwidth can be divided by the target refresh rate to get the value of N, which is also equal to the refresh rate reduction multiple. For example, if the refresh rate supported by the hardware transmission bandwidth is 60Hz, after the input video stream is reduced to 15Hz, N=4 can be calculated.

[0064] Based on the value of N, the video processing device can perform synthesis processing of the video stream.

[0065] Specifically, the video processing device can determine the target frame number of each input interface. The target frame number is the frame number of the sub-video stream to which the reduced refresh rate video stream corresponding to the input interface belongs in the video stream to be transmitted. For each input interface, according to the current frame number and the target frame number, the video picture of the corresponding reduced refresh rate video stream can be synthesized to obtain the video stream to be transmitted.

[0066] Specifically, for any input interface, when the current frame number is the target frame number corresponding to itself, the video screen of the reduced refresh rate video stream can be synthesized to the screen position corresponding to the video stream to be transmitted to obtain the video stream to be transmitted.

[0067] by Figure 3Taking the video stream to be transmitted as an example, for the first sub-video stream, its frame numbers in the transmitted video stream are 1 and 5. If the first sub-video stream includes a reduced refresh rate video stream corresponding to the input video stream received by the upper input interface 1, the target frame numbers of the upper input interface 1 are 1 and 5. If the current frame number is 1, then, based on the current frame number and the target frame number, it can be determined whether input interface 1 needs to be mapped, and then the video screen of the reduced refresh rate video stream corresponding to input interface 1 is synthesized to map the first frame of the video stream to be transmitted. Similarly, each input interface performs synthesis processing, and after completion, a video stream to be transmitted can be formed.

[0068] In some implementations, please refer to Figure 4 In the process of video stream synthesis, the video processing device can be processed in the following manner. Specifically, in some embodiments of the present application, for each input interface, when the current frame number is the corresponding target frame number, the video screen of the corresponding reduced refresh rate video stream is synthesized, and the synthesized video stream is transmitted to the next input interface for synthesis processing until it is transmitted to the last input interface to obtain the video stream to be transmitted.

[0069] More specifically, the input interface can be set on the input card. For any input card, if the current frame number is the target frame number corresponding to its own input interface, the video screen of the corresponding reduced refresh rate video stream is synthesized and processed, and the synthesized video stream is transmitted to the next input card for synthesis processing until it is transmitted to the last input card to obtain the video stream to be transmitted.

[0070] by Figure 3 Take the video stream to be transmitted as an example, please refer to Figure 4 , for the first sub-video stream, its frame number in the transmitted video stream is 1, 5. If the first sub-video stream includes a reduced refresh rate video stream corresponding to the input video stream received by input interface 1, the target frame number of input interface 1 is 1, 5. If the current frame number is 1, then input interface 1 can determine that it needs to be mapped based on the current frame number and the target frame number, and then synthesize the video screen of the reduced refresh rate video stream corresponding to input interface 1, and then transmit it to input interface 2. If the target frame number of input interface 2 is 1, the synthesis process is also performed, otherwise it is directly transmitted to input interface 3, and so on, until it is transmitted to the last input interface i (i is a positive integer greater than 0). The last input interface chooses to perform synthesis processing or not according to its own target frame number. After completion, the first frame of the video stream to be transmitted can be obtained. Among them, mapping refers to using the video screen of the reduced refresh rate video stream as the video screen at the corresponding position on the video screen of the video stream to be transmitted, such as Figure 4In the example, input interface 1 can use the video screen of its own down-refresh rate video stream as the video screen at the two square positions in the upper left corner of the video screen of the video stream to be transmitted. Each frame of the video screen of the video stream to be transmitted can be processed in the same way to form the video stream to be transmitted.

[0071] It should be noted that, since one frame of the video stream to be transmitted comes from one sub-video stream, if the video frame of the sub-video stream is composed of video frames of multiple video streams with reduced refresh rates, the video frames of the multiple video streams with reduced refresh rates are respectively located at different positions of the video frame of the sub-video stream. Figure 4 As shown, the input interface needs to synthesize the video screen of the reduced refresh rate video stream to the screen position corresponding to the video stream to be transmitted.

[0072] In other embodiments of this application, please refer to Figure 5 The video processing device can also send a frame synchronization signal to each input interface. The frame synchronization signal is used to instruct the input interface to transmit the video picture required by the current frame number in the corresponding reduced refresh rate video stream. Accordingly, the video processing device can control each input interface to transmit the video picture required by the current frame number in the reduced refresh rate video stream when receiving the frame synchronization signal according to the target frame number, and then synthesize the video pictures transmitted by each input interface to obtain the video stream to be transmitted.

[0073] More specifically, the input interface can be set on the input card. For any input card, if a frame synchronization signal is received, the video picture required by the current frame number in the reduced refresh rate video stream of its own input interface can be transmitted, and the video processing device synthesizes the video pictures transmitted by each input interface to obtain the video stream to be transmitted.

[0074] by Figure 3 Take the video stream to be transmitted as an example, please refer to Figure 5 , if the current frame number is 1, and the first frame of the video to be transmitted includes the first frame of the video stream with a reduced refresh rate of input interface 1, the video processing device can send a frame synchronization signal to input interface 1 to instruct input interface 1 to transmit the first frame of the video stream with a reduced refresh rate. Based on the pictures provided by different input interfaces, the video processing device can perform synthesis processing in frame order, map the video pictures of each received reduced refresh rate video stream to the corresponding picture position, and form a video stream to be transmitted.

[0075] After obtaining the video stream to be transmitted, the video stream to be transmitted can be transmitted.

[0076] Specifically, in order to meet the transmission requirement, the video processing device may compress the video stream to be transmitted, and transmit the compressed video stream.

[0077] Specifically, the video processing device may include a control chip, and the video processing device may transmit the video stream to be transmitted to the control chip of the video processing device. Then, the control chip compresses the video stream to be transmitted and transmits the compressed video stream to the terminal device.

[0078] The control chip may be an ARM (Advanced RISC Machines) processing chip, an Application Specific Integrated Circuit (ASIC), or a Microcontroller Unit (MCU) chip, etc., for controlling the input and output of the video stream. The control chip and the terminal device may be connected via a network port. By compression, the compressed video stream may meet the transmission bandwidth limitation between the control chip and the network port of the terminal device.

[0079] Since compression coding is usually performed based on the continuity of the reference video, in the video stream to be transmitted, two adjacent frames of video images come from different sub-video streams and have no continuity with each other, resulting in poor compression and decompression effects of the back-end terminal device. Therefore, in some embodiments of the present application, the video processing device can restore the video stream to be transmitted into N sub-video streams, and then compress the N sub-video streams separately to obtain N compressed sub-video streams, and transmit the N compressed sub-video streams.

[0080] by Figure 3 Take the video stream to be transmitted as an example, Figure 6 As shown, the control chip needs to reversely split the video stream to be transmitted into 4 sub-video streams, and then compress and encode the four sub-video streams separately, and transmit the independent four-way compressed sub-video streams to the terminal device through the network.

[0081] It should be understood that the above-mentioned compression coding can adopt compression coding algorithms such as the highly compressed digital video codec standard algorithm H.264, the video compression standard algorithm H.265, and the shallow compression algorithm (such as VC2, JPEG2000, JPEG XS), and this application does not impose any restrictions on this.

[0082] Correspondingly, after receiving the compressed sub-video stream, the terminal device can map the echo screen.

[0083] Specifically, Figure 7 A schematic diagram of the implementation flow of a video stream echo method provided in an embodiment of the present application is shown. The method can be applied to a terminal device and is suitable for situations where flexible support for on-site echo requirements is required.

[0084] Among them, the above-mentioned terminal device can be a computer, a mobile phone or other smart device used to control the video processing device. For example, the above-mentioned terminal device can be a console, a host computer or other device, and this application does not impose any restrictions on this.

[0085] In an embodiment of the present application, the terminal device may be configured with configuration software, the configuration software includes a display interface, and the display interface includes at least one display window. Each display window corresponds to an input interface of the video processing device, and is used to display the reduced refresh rate video stream of the corresponding input interface.

[0086] Specifically, the above-mentioned video stream echoing method may include the following steps S701 to S703.

[0087] Step S701: Obtain the video stream to be transmitted.

[0088] The video stream to be transmitted may be input by a video processing device, and may refer to the aforementioned uncompressed video stream to be transmitted, or may refer to the aforementioned N-channel compressed sub-video streams. The video processing device may refer to the above description, and this application will not elaborate on this.

[0089] Step S702: Determine the echo video stream of each input interface according to the video stream to be transmitted.

[0090] The echo video stream of each input interface is also the reduced refresh rate video stream corresponding to each input interface. For a compressed sub-video stream, the compressed sub-video stream can be decompressed to obtain the corresponding sub-video stream. The video screen of the sub-video stream can be segmented to obtain the video screens of each reduced refresh rate video stream included in the sub-video stream, and then the echo video stream corresponding to each input interface can be obtained.

[0091] Step S703: display the corresponding echo video stream in the display window.

[0092] Specifically, each display window may correspond to an input interface of the video processing device. Accordingly, the terminal device may display the echo video stream of the corresponding input interface in the display window.

[0093] Please refer to Figure 8 In the display interface shown, the user can set the display window in the editing area of ​​the display interface and set the input interface corresponding to the display window. After obtaining the echo video stream of each input interface, if there is a display window corresponding to a certain input interface in the display interface, the echo video stream of the input interface can be loaded into the display window for display. It can be understood that at this time, the user can see the dynamic picture of the video in the display window.

[0094] Accordingly, the user can further perform editing operations in the editing area, such as adding a background color to the editing area, adding other pictures, etc., thereby realizing visual editing operations.

[0095] In the implementation manner of the present application, the user can see the echo video stream of the corresponding input interface in any display window, ensuring that the picture of each input source of the video processing device can be displayed in the display window, and can provide the user with a high-quality echo picture, facilitate the user's visual operation, and flexibly support on-site echo needs.

[0096] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders.

[0097] like Fig. 9 FIG. 1 is a schematic structural diagram of a video stream transmission device 900 provided in an embodiment of the present application. The video stream transmission device 900 is configured on a video processing device.

[0098] Specifically, the video stream transmission device 900 may include:

[0099] An acquisition unit 901 is used to acquire multiple input video streams;

[0100] The processing unit 902 is used to perform refresh rate reduction processing on the multiple input video streams to obtain corresponding multiple reduced refresh rate video streams;

[0101] A synthesis unit 903 is used to synthesize the multiple reduced refresh rate video streams to obtain a video stream to be transmitted, wherein the continuous N frames of video pictures in the video stream to be transmitted are respectively from N sub-video streams, wherein one sub-video stream includes a video picture corresponding to at least one reduced refresh rate video stream, and N is a positive integer greater than 1;

[0102] The transmission unit 904 is configured to transmit the video stream to be transmitted.

[0103] In some embodiments of the present application, each of the multiple input video streams corresponds to an input interface; the above-mentioned synthesis unit 903 can be specifically used to: determine the target frame number of each of the input interfaces, the target frame number being the frame number of the sub-video stream to which the reduced refresh rate video stream corresponding to the input interface belongs in the video stream to be transmitted; for each of the input interfaces, the video picture of the corresponding reduced refresh rate video stream is synthesized according to the current frame number and the target frame number to obtain the video stream to be transmitted.

[0104] In some embodiments of the present application, the above-mentioned synthesis unit 903 can be specifically used to: for each of the input interfaces, when the current frame number is the corresponding target frame number, perform synthesis processing on the video picture of the corresponding reduced refresh rate video stream, and transmit the synthesized video stream to the next input interface for synthesis processing until it is transmitted to the last input interface to obtain the video stream to be transmitted.

[0105] In some embodiments of the present application, the above-mentioned synthesis unit 903 can be specifically used to: control each of the input interfaces to transmit the video picture required by the current frame number in the reduced refresh rate video stream when receiving the frame synchronization signal according to the target frame number; and perform synthesis processing on the video pictures transmitted by each of the input interfaces to obtain the video stream to be transmitted.

[0106] In some embodiments of the present application, the above-mentioned synthesis unit 903 can be specifically used to: when the current frame number is the corresponding target frame number, synthesize the video picture of the reduced refresh rate video stream to the picture position corresponding to the video stream to be transmitted, so as to obtain the video stream to be transmitted.

[0107] In some embodiments of the present application, when the video picture of one sub-video stream includes the video picture corresponding to one of the reduced refresh rate video streams, the sub-video stream includes the video picture corresponding to the reduced refresh rate video stream; when the video picture of one sub-video stream includes the video pictures corresponding to multiple reduced refresh rate video streams, the sub-video stream includes the corresponding video spliced ​​picture after splicing the multiple reduced refresh rate video streams.

[0108] In some embodiments of the present application, the above-mentioned processing unit 902 can be specifically used to: determine a preset target refresh rate; reduce the refresh rate of each input video stream in the multiple input video streams to the target refresh rate, and obtain the corresponding multiple reduced refresh rate video streams.

[0109] In some implementations of the present application, the synthesis unit 903 may also be specifically configured to determine the number N of sub-video streams according to the target refresh rate and the refresh rate corresponding to the hardware transmission bandwidth.

[0110] In some implementations of the present application, the transmission unit 904 may be specifically configured to: compress the video stream to be transmitted, and transmit the compressed video stream.

[0111] In some embodiments of the present application, the transmission unit 904 may also be specifically used to: restore the video stream to be transmitted into the N sub-video streams; compress the N sub-video streams respectively to obtain N compressed sub-video streams; and transmit the N compressed sub-video streams.

[0112] It should be noted that, for the convenience and simplicity of description, the specific working process of the video stream transmission device 900 can be referred to Figures 1 to 8 The corresponding process of the method will not be repeated here.

[0113] like Fig.10 FIG. 1 is a schematic structural diagram of a video stream display device 1000 provided in an embodiment of the present application. The video stream display device 1000 is configured on a terminal device.

[0114] Specifically, the video stream transmission device 1000 may include:

[0115] An acquisition unit 1001 is used to acquire a video stream to be transmitted;

[0116] The determining unit 1002 is used to determine the echo video stream of each input interface according to the video stream to be transmitted;

[0117] The echo unit 1003 is used to display the corresponding echo video stream in a display window.

[0118] It should be noted that, for the convenience and simplicity of description, the specific working process of the video stream echoing device 1000 can be referred to Figures 1 to 8 The corresponding process of the method will not be repeated here.

[0119] like Fig.11 , which is a schematic diagram of a video processing device provided in an embodiment of the present application. Specifically, the video processing device 11 may include: a processor 110, a memory 111, and a computer program 112 stored in the memory 111 and executable on the processor 110, such as a video stream transmission program and a video stream echo program. When the processor 110 executes the computer program 112, the steps in the above-mentioned video stream transmission method embodiments are implemented, such as Figure 1 Alternatively, when the processor 110 executes the computer program 112, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Fig. 9 The functions of the acquisition unit 901, the processing unit 902, the synthesis unit 903 and the transmission unit 904 are shown.

[0120] The computer program may be divided into one or more modules / units, which are stored in the memory 111 and executed by the processor 110 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the video processing device.

[0121] For example, the computer program can be divided into: an acquisition unit, a processing unit, a synthesis unit and a transmission unit. The specific functions of each unit are as follows: an acquisition unit, used to acquire multiple input video streams; a processing unit, used to perform refresh rate reduction processing on the multiple input video streams to obtain corresponding multiple reduced refresh rate video streams; a synthesis unit, used to perform synthesis processing on the multiple reduced refresh rate video streams to obtain a video stream to be transmitted, wherein the continuous N frames of video images in the video stream to be transmitted are respectively from N sub-video streams, wherein one sub-video stream includes video images corresponding to at least one reduced refresh rate video stream, and N is a positive integer greater than 1; a transmission unit, used to transmit the video stream to be transmitted.

[0122] The video processing device may include, but is not limited to, a processor 110 and a memory 111. Those skilled in the art will appreciate that Fig.11 It is only an example of a video processing device and does not constitute a limitation of the video processing device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the video processing device may also include input and output devices, network access devices, buses, etc.

[0123] The processor 110 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0124] The memory 111 may be an internal storage unit of the video processing device, such as a hard disk or memory of the video processing device. The memory 111 may also be an external storage device of the video processing device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the video processing device. Further, the memory 111 may also include both an internal storage unit and an external storage device of the video processing device. The memory 111 is used to store the computer program and other programs and data required by the video processing device. The memory 111 may also be used to temporarily store data that has been output or is to be output.

[0125] It should be noted that, for the convenience and brevity of description, the structure of the video processing device 11 can also refer to the specific description of the structure in the method embodiment, which will not be repeated here.

[0126] like Fig.12 , which is a schematic diagram of a terminal device provided in an embodiment of the present application. Specifically, the terminal device 12 may include: a processor 120, a memory 121, and a computer program 122 stored in the memory 121 and executable on the processor 120, such as a video stream transmission program and a video stream echo program. When the processor 120 executes the computer program 122, the steps in the above-mentioned video stream echo method embodiments are implemented, such as Figure 7 Alternatively, when the processor 120 executes the computer program 122, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Fig.10 The functions of the acquisition unit 1001, the determination unit 1002 and the echo unit 1003 are shown.

[0127] For example, the computer program can be divided into: an acquisition unit, a determination unit and an echo unit. The specific functions of each unit are as follows: the acquisition unit is used to acquire the video stream to be transmitted; the determination unit is used to determine the echo video stream of each input interface according to the video stream to be transmitted; the echo unit is used to display the corresponding echo video stream in the display window.

[0128] The terminal device may include, but is not limited to, a processor 120 and a memory 121. Those skilled in the art will appreciate that Fig.12 It is only an example of a terminal device and does not constitute a limitation of the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access equipment, buses, etc.

[0129] The processor 120 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0130] The memory 121 may be an internal storage unit of the terminal device, such as a hard disk or memory of the terminal device. The memory 121 may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device. Further, the memory 121 may also include both an internal storage unit of the terminal device and an external storage device. The memory 121 is used to store the computer program and other programs and data required by the terminal device. The memory 121 may also be used to temporarily store data that has been output or is to be output.

[0131] It should be noted that, for the convenience and brevity of description, the structure of the above-mentioned terminal device 12 can also refer to the specific description of the structure in the method embodiment, and will not be repeated here.

[0132] like Fig.13 As shown, the present application also provides a video processing system 13, including a video processing device and a terminal device connected to the video processing device.

[0133] Among them, the specific functions and structures of the video processing device and the terminal device can be referred to the previous description, and this application will not elaborate on this.

[0134] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0135] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0136] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0137] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0140] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0141] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A video stream transmission method, characterized in that: Applied to a video processing device, the video stream transmission method includes: Get multiple input video streams; Performing refresh rate reduction processing on the multiple input video streams to obtain corresponding multiple reduced refresh rate video streams; Performing synthesis processing on the multiple reduced refresh rate video streams to obtain a video stream to be transmitted, wherein the continuous N frames of video pictures in the video stream to be transmitted are respectively from N sub-video streams, wherein one sub-video stream includes a video picture corresponding to at least one reduced refresh rate video stream, and N is a positive integer greater than 1; The video stream to be transmitted is transmitted.

2. The video stream transmission method according to claim 1, characterized in that: Each of the multiple input video streams corresponds to an input interface; The synthesizing process of the multiple channels of reduced refresh rate video streams to obtain a video stream to be transmitted includes: Determine a target frame number for each input interface, where the target frame number is a frame number of a sub-video stream to which the reduced refresh rate video stream corresponding to the input interface belongs in the video stream to be transmitted; For each of the input interfaces, the video images of the corresponding reduced refresh rate video stream are synthesized according to the current frame sequence number and the target frame sequence number to obtain the video stream to be transmitted.

3. The video stream transmission method according to claim 2, characterized in that: For each of the input interfaces, synthesizing the video picture of the corresponding reduced refresh rate video stream according to the current frame sequence number and the target frame sequence number to obtain the video stream to be transmitted, including: For each of the input interfaces, when the current frame number is the corresponding target frame number, the video picture of the corresponding reduced refresh rate video stream is synthesized, and the synthesized video stream is transmitted to the next input interface for synthesis processing until it is transmitted to the last input interface to obtain the video stream to be transmitted.

4. The video stream transmission method according to claim 2, characterized in that: For each of the input interfaces, synthesizing the video picture of the corresponding reduced refresh rate video stream according to the current frame sequence number and the target frame sequence number to obtain the video stream to be transmitted, including: According to the target frame sequence number, controlling each of the input interfaces to transmit the video picture required by the current frame sequence number in the reduced refresh rate video stream when receiving a frame synchronization signal; The video images transmitted by each of the input interfaces are synthesized to obtain the video stream to be transmitted.

5. The video stream transmission method according to claim 2, characterized in that: According to the current frame sequence number and the target frame sequence number, a video picture of the corresponding reduced refresh rate video stream is synthesized, including: In the case where the current frame number is the corresponding target frame number, the video picture of the reduced refresh rate video stream is synthesized to the picture position corresponding to the video stream to be transmitted, so as to obtain the video stream to be transmitted.

6. The video stream transmission method according to any one of claims 1 to 5, characterized in that: In the case where a video picture of one sub-video stream includes a video picture corresponding to one of the reduced refresh rate video streams, the one sub-video stream includes a video picture corresponding to one of the reduced refresh rate video streams; In the case where the video picture of one sub-video stream includes the video pictures corresponding to multiple channels of the reduced refresh rate video streams, the one sub-video stream includes the corresponding video splicing picture after the multiple channels of the reduced refresh rate video streams are spliced.

7. The video stream transmission method according to any one of claims 1 to 5, characterized in that: The step of performing refresh rate reduction processing on the multiple input video streams to obtain corresponding multiple reduced refresh rate video streams includes: Determine the preset target refresh rate; The refresh rate of each input video stream in the multiple input video streams is reduced to the target refresh rate to obtain the corresponding multiple reduced refresh rate video streams.

8. The video stream transmission method according to claim 7, characterized in that: Before synthesizing the multiple channels of reduced refresh rate video streams to obtain the video stream to be transmitted, the method further includes: The number N of sub-video streams is determined according to the target refresh rate and the refresh rate corresponding to the hardware transmission bandwidth.

9. The video stream transmission method according to any one of claims 1 to 5, characterized in that: The transmitting the video stream to be transmitted includes: The video stream to be transmitted is compressed, and the compressed video stream is transmitted.

10. The video stream transmission method according to claim 9, characterized in that: The step of compressing the video stream to be transmitted and transmitting the compressed video stream includes: Restoring the video stream to be transmitted into the N sub-video streams; Compressing the N sub-video streams respectively to obtain N compressed sub-video streams; The N compressed sub-video streams are transmitted.

11. A method for echoing a video stream, characterized in that: Applied to a terminal device, the terminal device is installed with configuration software, the configuration software includes a display interface, the display interface includes at least one display window, and each display window corresponds to an input interface; The video stream echoing method includes: Obtaining a video stream to be transmitted as described in any one of claims 1 to 10; Determine the echo video stream of each input interface according to the video stream to be transmitted; The corresponding echo video stream is displayed in the display window.

12. A video stream transmission device, characterized in that: Configured in a video processing device, the video stream transmission device includes: An acquisition unit, used for acquiring multiple input video streams; A processing unit, configured to perform refresh rate reduction processing on the multiple input video streams to obtain corresponding multiple reduced refresh rate video streams; A synthesis unit, configured to synthesize the multiple reduced refresh rate video streams to obtain a video stream to be transmitted, wherein the continuous N frames of video images in the video stream to be transmitted are respectively from N sub-video streams, each sub-video stream includes a video image corresponding to at least one reduced refresh rate video stream, and N is a positive integer greater than 1; A transmission unit is used to transmit the video stream to be transmitted.

13. A video stream echo device, characterized in that: Configured in a terminal device, the terminal device is configured with configuration software, the configuration software includes a display interface, the display interface includes at least one display window, and each display window corresponds to an input interface; The video stream echoing device comprises: An acquisition unit, used for acquiring the video stream to be transmitted as claimed in any one of claims 1 to 10; A determination unit, configured to determine an echo video stream of each input interface according to the video stream to be transmitted; The echo unit is used to display the corresponding echo video stream in the display window.

14. A video processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the video stream transmission method according to any one of claims 1 to 10 are implemented.

15. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the video stream transmission method according to claim 11 are implemented.

16. A video processing system, characterized in that: It comprises the video processing device as claimed in claim 14, and the terminal device as claimed in claim 15 connected to the video processing device.

17. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the video stream transmission method according to any one of claims 1 to 10 are implemented; or, when the computer program is executed by a processor, the steps of the video stream transmission method according to claim 11 are implemented.

Images