A method, device, display device and medium for displaying bullet screen after video redirection

Abstract

The present invention provides a method for displaying bullet screen after video redirection and related equipment. The method comprises: after receiving the original video stream and the system desktop image, the remote desktop client decodes the original video stream and renders it to the video layer, and decodes the system desktop image and renders it to the system main surface layer; the remote desktop client merges the video layer that renders the decoded original video stream and the system main surface layer that renders the decoded system desktop image to obtain a merged image, wherein the system main surface layer blending mode is modified to the source overlay mode; the remote desktop client sends the merged image to the display for output. Through the present invention, the control panel and the bullet screen are not blocked by the redirected video, ensuring that the display effect of the redirected video is consistent with the display effect when the video redirection is not turned on.

Description

Technical Field

[0001] The present invention relates to the technical field of desktop virtualization, and in particular to a bullet screen display method after video redirection and related equipment. Background Art

[0002] When watching HTML5 web videos in a virtual machine using a browser, in order to reduce server resource overhead, the web videos are generally redirected to the remote client for playback. When the web videos are redirected to the remote client for playback, a player window will be launched above the video area, but the player window will cover the control panel and bullet screen in the video area below.

[0003] The existing solution is to obtain the position and size of the control panel buttons and bullet screen in the browser and send them to the client, and then the client makes the corresponding position of the player window transparent, so that the control panel and bullet screen below the z-axis of the player window can be displayed through the player window. However, since the position of the bullet screen changes in real time, more positions of the player window will be made transparent. Therefore, although the bullet screen can be seen, the display effect is quite different from that when video redirection is not enabled, which seriously affects the user's viewing experience. Summary of the invention

[0004] The main purpose of the present invention is to provide a bullet screen display method and related equipment after video redirection, aiming to ensure that the display effect of the redirected video is consistent with the display effect when the video redirection is not turned on.

[0005] In a first aspect, the present invention provides a method for displaying a bullet screen after video redirection, the method for displaying a bullet screen after video redirection comprising:

[0006] After receiving the original video stream and the system desktop image, the remote desktop client decodes the original video stream and renders it to the video layer, and decodes the system desktop image and renders it to the system main surface layer, wherein the video layer is below the z-axis of the system main surface layer, and the system desktop image is obtained by merging the video canvas layer rendered with a black transparent frame, the bullet screen layer above the z-axis of the video canvas layer, the control panel layer above the z-axis of the video canvas layer, the webpage background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system;

[0007] The remote desktop client performs layer merging on the video layer that renders the decoded original video stream and the system main surface layer that renders the decoded system desktop image to obtain a merged image, wherein the system main surface layer blending mode is modified to a source overlay mode;

[0008] The remote desktop client sends the combined image to a display for output.

[0009] Optionally, after the remote desktop client receives the original video stream and the system desktop image, before the step of decoding the original video stream and rendering it to the video layer, and decoding the system desktop image and rendering it to the system main surface layer, the method includes:

[0010] The remote desktop client receives the original video stream sent by the modified video decoder, wherein the original video stream is not decoded.

[0011] Optionally, after the remote desktop client receives the original video stream and the system desktop image, before the step of decoding the original video stream and rendering it to the video layer, and decoding the system desktop image and rendering it to the system main surface layer, it also includes:

[0012] The modified video decoder program creates black transparent frames;

[0013] The web browser renders the black transparent frame to the video canvas layer;

[0014] The web browser renders the changed bullet screen rectangle to the bullet screen layer, and renders the changed control panel rectangle to the control panel layer, wherein the change includes a change in shape, position and / or color;

[0015] The web browser merges the video canvas layer, the bullet screen layer, the control panel layer, and the web page background layer, which render the black transparent frame, to obtain a synthesized image, wherein the blending mode of the video canvas layer is changed to the source color space mode;

[0016] The operating system merges the image synthesized by the web browser with the original desktop image of the operating system to obtain a system desktop image;

[0017] The remote desktop server encodes the system desktop image and sends it to the remote desktop client.

[0018] Optionally, the step of the remote desktop server encoding the system desktop image and sending it to the remote desktop client includes:

[0019] The remote desktop server encodes the RGB value and A value of the system desktop image respectively;

[0020] The remote desktop server sends the system desktop image after encoding the RGB value and the A value to the remote desktop client, wherein the A value represents the value of the Alpha channel.

[0021] Optionally, the step of decoding the system desktop image and rendering it to the system main surface layer includes:

[0022] The remote desktop client performs decoding operations on the RGB value and the A value of the system desktop image respectively;

[0023] The remote desktop client renders the system main surface layer according to the RGB value and A value of the system desktop image obtained after decoding, wherein the A value represents the value of the Alpha channel.

[0024] In a second aspect, the present invention further provides a barrage display device after video redirection, the barrage display device after video redirection comprising:

[0025] A first rendering module is used for, after the remote desktop client receives the original video stream and the system desktop image, decoding the original video stream and rendering it to the video layer, and decoding the system desktop image and rendering it to the system main surface layer, wherein the video layer is below the z-axis of the system main surface layer, and the system desktop image is obtained by merging a video canvas layer with a black transparent frame rendered, a bullet screen layer above the z-axis of the video canvas layer, a control panel layer above the z-axis of the video canvas layer, a web page background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system;

[0026] A first merging module is used for the remote desktop client to merge the video layer that renders the decoded original video stream and the system main surface layer that renders the decoded system desktop image to obtain a merged image, wherein the system main surface layer blending mode is changed to a source overlay mode;

[0027] The display module is used for the remote desktop client to send the combined image to the display for output.

[0028] Optionally, the first rendering module is further used to:

[0029] The remote desktop client performs decoding operations on the RGB value and the A value of the system desktop image respectively;

[0030] The remote desktop client renders the system main surface layer according to the RGB value and A value of the system desktop image obtained after decoding, wherein the A value represents the value of the Alpha channel.

[0031] Optionally, the barrage display device after video redirection further includes a first transmission module, which is used to:

[0032] The remote desktop client receives the original video stream sent by the modified video decoder, wherein the original video stream is not decoded.

[0033] In the third aspect, the present invention also provides a barrage display device after video redirection, wherein the barrage display device after video redirection includes a processor, a memory, and a barrage display program after video redirection stored in the memory and executable by the processor, wherein when the barrage display program after video redirection is executed by the processor, the steps of the barrage display method after video redirection as described above are implemented.

[0034] In a fourth aspect, the present invention also provides a readable storage medium, on which is stored a program for displaying barrages after video redirection. When the program for displaying barrages after video redirection is executed by a processor, the steps of the method for displaying barrages after video redirection as described above are implemented.

[0035] In the present invention, after receiving the original video stream and the system desktop image, the remote desktop client decodes the original video stream and renders it to the video layer, and decodes the system desktop image and renders it to the system main surface layer, wherein the video layer is below the z-axis of the system main surface layer, and the system desktop image is obtained by merging a video canvas layer that renders a black transparent frame, a bullet screen layer above the z-axis of the video canvas layer, a control panel layer above the z-axis of the video canvas layer, a web page background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system; the remote desktop client merges the video layer that renders the decoded original video stream and the system main surface layer that renders the decoded system desktop image to obtain a merged image; the remote desktop client sends the merged image to the display for output. Through the present invention, since the video layer that renders the decoded original video stream is merged below the z-axis of the system main surface layer that renders the decoded system desktop image, and the system desktop image is obtained by merging the video canvas layer that renders the black transparent frame, the bullet screen layer above the z-axis of the video canvas layer, the control panel layer above the z-axis of the video canvas layer, the web page background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system, and since the video canvas layer that renders the black transparent frame is transparent, it will not block the original video stream in the video layer below the z-axis of the system main surface layer. At the same time, the bullet screen layer and the control panel layer in the system main surface layer are above the original video stream in the video layer, so the redirected video will not block the bullet screen and the control panel, and there is no need for the remote desktop client to perform transparency processing at the corresponding position of the display window to display the bullet screen and the control panel, thereby ensuring that the display effect of the redirected video is consistent with the display effect when the video redirection is not turned on, and will not affect the user viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1A schematic diagram of the hardware structure of a bullet screen display device after video redirection involved in an embodiment of the present invention;

[0037] Figure 2 It is a flowchart of the first embodiment of the method for displaying bullet screen after video redirection of the present invention;

[0038] Figure 3 It is a flow chart of a second embodiment of the method for displaying bullet screen after video redirection of the present invention;

[0039] Figure 4 for Figure 3 Step S007 is a detailed flow chart;

[0040] Figure 5 This is a schematic diagram of the functional modules of an embodiment of a barrage display device after video redirection of the present invention.

[0041] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0042] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0043] In a first aspect, an embodiment of the present invention provides a barrage display device after video redirection, and the barrage display device after video redirection can be a personal computer (PC), a laptop computer, a server, or other device with data processing function.

[0044] Reference Figure 1 , Figure 1The hardware structure diagram of the bullet screen display device after video redirection involved in the embodiment of the present invention. In the embodiment of the present invention, the bullet screen display device after video redirection may include a processor 1001 (e.g., a central processing unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as wireless fidelity WIreless-FIdelity, WI-FI interface); the memory 1005 may be a high-speed random access memory (random access memory, RAM), or a stable memory (non-volatile memory), such as a disk storage, and the memory 1005 may optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art will understand that Figure 1 The hardware structure shown in the figure does not constitute a limitation of the present invention, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0045] Continue to refer to Figure 1 , Figure 1 The memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a bullet screen display program after video redirection. The processor 1001 may call the bullet screen display program after video redirection stored in the memory 1005, and execute the bullet screen display method after video redirection provided in an embodiment of the present invention.

[0046] In a second aspect, an embodiment of the present invention provides a method for displaying barrage after video redirection.

[0047] In one embodiment, referring to Figure 2 , Figure 2 FIG. 1 is a flow chart of the first embodiment of the method for displaying bullet screen after video redirection of the present invention. Figure 2 As shown, the method for displaying bullet screen after video redirection includes:

[0048] Step S10, after receiving the original video stream and the system desktop image, the remote desktop client decodes the original video stream and renders it to the video layer, and decodes the system desktop image and renders it to the system main surface layer, wherein the video layer is below the z-axis of the system main surface layer, and the system desktop image is obtained by merging the video canvas layer rendered with a black transparent frame, the bullet screen layer above the z-axis of the video canvas layer, the control panel layer above the z-axis of the video canvas layer, the webpage background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system;

[0049] In this embodiment, after the remote desktop client receives the original video stream sent by the modified video decoder and the system desktop image sent by the remote desktop server, the remote desktop client decodes the original video stream through the decoder and renders the decoded original video stream to the video layer of the remote desktop client. The remote desktop client decodes the system desktop image through the decoder and renders the decoded system desktop image to the system main surface layer of the remote desktop client. Among them, the video layer of the remote desktop client is below the z-axis of the system main surface layer, and the system desktop image is obtained by merging the video canvas layer with a black transparent frame rendered, the bullet screen layer above the z-axis of the video canvas layer, the control panel layer above the z-axis of the video canvas layer, the web page background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system. Rendering in computer graphics refers to the process of generating an image from a model using software.

[0050] Furthermore, in one embodiment, the step of decoding the system desktop image and rendering it to the system main surface layer includes:

[0051] The remote desktop client performs decoding operations on the RGB value and the A value of the system desktop image respectively;

[0052] The remote desktop client renders the system main surface layer according to the RGB value and A value of the system desktop image obtained after decoding, wherein the A value represents the value of the Alpha channel.

[0053] In this embodiment, RGBA is a color space representing Red, Green, Blue and Alpha channels. The RGB value represents a color space based on three basic colors: R value (Red), G value (Green), and B value (Blue). The A value is the value of the Alpha channel, ranging from 0 to 1. The Alpha channel is generally used as an opacity parameter. If the value of the Alpha channel of a pixel is 0, then the pixel is completely transparent (that is, invisible), and if the value of the Alpha channel is 1, then the pixel is completely opaque.

[0054] The remote desktop client decodes the RGB value and A value of the system desktop image through the decoder. It should be noted that one pixel is a color space. If the system desktop image has 1920*1080 pixels, the RGB value and A value of the 1920*1080 pixels of the system desktop image are decoded through the decoder.

[0055] The remote desktop client renders the system main surface layer according to the RGB value and A value of the system desktop image obtained after decoding. If the system desktop image has 1920*1080 pixels, the RGB value and A value of the corresponding 1920*1080 pixels in the system main surface layer are rendered according to the RGB value and A value of the 1920*1080 pixels. Specifically, if the RGB value and A value of the decoded 1920*1080 pixels are 000 and 0 respectively, the RGB value and A value of the corresponding 1920*1080 pixels in the system main surface layer are rendered as 000 and 0. Among them, the A value represents the value of the Alpha channel.

[0056] Step S20, the remote desktop client performs layer merging on the video layer for rendering the decoded original video stream and the system main surface layer for rendering the decoded system desktop image to obtain a merged image, wherein the system main surface layer blending mode is changed to the source overlay mode;

[0057] In this embodiment, after the remote desktop client renders the decoded original video stream to the video layer of the remote desktop client and renders the decoded system desktop image to the system main surface layer of the remote desktop client, the video layer that renders the decoded original video stream and the system main surface layer that renders the decoded system desktop image are merged through the alphablend function to obtain a merged image, wherein the video layer that renders the decoded original video stream is merged below the z-axis of the system main surface layer that renders the decoded system desktop image, and the system main surface layer blending mode is modified to the source overlay mode.

[0058] Among them, after changing the system main surface layer blending mode to source overlay mode, the R value of the merged image obtained corresponds to the formula r=s r +(1-sa)*d r , where r represents the R value of the merged image, s r represents the R value of the client's system main surface layer, sa represents the A value of the client's system main surface layer, d r Indicates the R value of the client's video layer;

[0059] The G value of the merged image corresponds to the formula g = s g+(1-sa)*d g , where g represents the G value of the merged image, s g represents the G value of the client's system main surface layer, sa represents the A value of the client's system main surface layer, d g Indicates the G value of the client's video layer;

[0060] The B value of the merged image corresponds to the formula b=s b +(1-sa)*d b , where b represents the B value of the merged image, s b represents the B value of the client's system main surface layer, sa represents the A value of the client's system main surface layer, d b Indicates the B value of the client's video layer;

[0061] The A value of the merged image corresponds to the formula a=sa+(1-sa)*d a , where a represents the A value of the merged image, sa represents the A value of the client's system main surface layer, and d a Indicates the A value of the client's video layer.

[0062] Step S30: The remote desktop client sends the merged image to a display for output.

[0063] In this embodiment, the remote desktop client sends the merged image to the display for output. It is easy to imagine that the video layer contains multiple frames of video images. After the video layer corresponding to each frame of video image is merged with the system main surface layer, a multiple-frame merged image is obtained. The merged image is sent to the display for output at a certain frequency, thereby completing the video redirection.

[0064] In this embodiment, after receiving the original video stream and the system desktop image, the remote desktop client decodes the original video stream and renders it to the video layer, and decodes the system desktop image and renders it to the system main surface layer, wherein the video layer is below the z-axis of the system main surface layer, and the system desktop image is obtained by merging the video canvas layer that renders a black transparent frame, the bullet screen layer above the z-axis of the video canvas layer, the control panel layer above the z-axis of the video canvas layer, the web page background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system; the remote desktop client merges the video layer that renders the decoded original video stream and the system main surface layer that renders the decoded system desktop image to obtain a merged image, wherein the system main surface layer blending mode is modified to the source overlay mode; the remote desktop client sends the merged image to the display for output. Through this embodiment, since the video layer that renders the decoded original video stream is merged below the z-axis of the system main surface layer that renders the decoded system desktop image, and the system desktop image is obtained by merging the video canvas layer that renders the black transparent frame, the bullet screen layer above the z-axis of the video canvas layer, the control panel layer above the z-axis of the video canvas layer, the web page background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system, and since the video canvas layer that renders the black transparent frame is transparent, it will not block the original video stream in the video layer below the z-axis of the system main surface layer. At the same time, the bullet screen layer and the control panel layer in the system main surface layer are above the original video stream in the video layer, so the redirected video will not block the bullet screen and the control panel, and there is no need for the remote desktop client to perform transparency processing at the corresponding position of the display window to display the bullet screen and the control panel, thereby ensuring that the display effect of the redirected video is consistent with the display effect when the video redirection is not turned on, and will not affect the user's viewing experience.

[0065] Furthermore, in one embodiment, before step S10, the following steps are included:

[0066] The remote desktop client receives the original video stream sent by the modified video decoder, wherein the original video stream is not decoded.

[0067] In this embodiment, the modified video decoder does not decode the original video stream, but sends the original video stream to the remote desktop client. Therefore, the original video stream sent by the modified video decoder received by the remote desktop client is an undecoded original video stream.

[0068] Further, in one embodiment, referring to Figure 3 , Figure 3 FIG. 2 is a flow chart of a second embodiment of a method for displaying bullet screen after video redirection according to the present invention. Figure 3As shown, before step S10, it also includes:

[0069] Step S001, the modified video decoder program creates a black transparent frame;

[0070] Step S002, the web browser renders the black transparent frame to the video canvas layer;

[0071] Step S003, the web browser renders the changed bullet screen rectangle to the bullet screen layer, and renders the changed control panel rectangle to the control panel layer, wherein the change includes a change in shape, position and / or color;

[0072] Step S004, the web browser merges the video canvas layer, the bullet screen layer, the control panel layer, and the web page background layer that render the black transparent frame to obtain a synthesized image, wherein the blending mode of the video canvas layer is changed to the source color space mode;

[0073] Step S005, the operating system merges the image synthesized by the web browser with the original desktop image of the operating system to obtain a system desktop image;

[0074] Step S006: the remote desktop server encodes the system desktop image and sends it to the remote desktop client.

[0075] In this embodiment, the modified video decoder program creates a black transparent frame, that is, creates an image with an RGBA value of 0000. The web browser renders the black transparent frame to the video canvas layer. Specifically, taking the web browser as a Chromium browser and the modified video decoder as an FFmpeg video decoder as an example, the modified FFmpeg video decoder creates a black transparent frame, and encapsulates the black transparent frame, the scaling operation instruction, the rotation operation instruction, and the cropping operation instruction into a texture rectangle, adds the texture rectangle to the RenderPass, and then adds the RenderPass to the composite frame (CompositorFrame), and then sends the composite frame to the surface aggregator (SurfaceAggregator), and the surface aggregator sends the RenderPass in the composite frame to the software rendering module. When the software rendering module (SoftwareRenderer) renders the texture rectangle in the RenderPass, the black transparent frame in the texture rectangle is scaled, rotated, and cropped according to the scaling operation instruction, the rotation operation instruction, and the cropping operation instruction, and then rendered to the video canvas layer. Among them, FFmpeg video decoder (FFmpegVideoDecoder), RenderPass, composite frame (CompositorFrame), surface aggregator (SurfaceAggregator) and software rendering module (SoftwareRenderer) are class names in the Chromium browser open source code.

[0076] And the web browser detects whether the shape, position and / or color of the barrage rectangle and the control panel rectangle have changed. If the shape, position and / or color of the barrage rectangle has changed, the changed barrage rectangle will be rendered to the barrage layer. If the shape, position and / or color of the control panel rectangle has changed, the changed control panel rectangle will be rendered to the control panel layer. Further, if the shape, position and / or color of the barrage rectangle has not changed, there is no need to render the barrage rectangle whose shape, position and / or color has not changed to the barrage layer. If the shape, position and / or color of the control panel rectangle has not changed, there is no need to render the control panel rectangle whose shape, position and / or color has not changed to the control panel layer. Among them, each barrage rectangle corresponds to one barrage layer or multiple barrage rectangles correspond to one barrage layer, and each control panel rectangle corresponds to one control panel layer or multiple control panel rectangles correspond to one control panel layer.

[0077] The web browser merges the video canvas layer that renders a black transparent frame, the bullet screen layer above the z-axis of the video canvas layer, the control panel layer above the z-axis of the video canvas layer, and the web page background layer below the z-axis of the video canvas layer from bottom to top through the alphablend function in the order of the z-axis coordinates of the layers to obtain a synthesized image, wherein the blending mode of the video canvas layer is modified to the source color space mode, so that the RGBA value of the image obtained after the video canvas layer and the web page background layer are merged is the same as the RGBA value of the video canvas layer; the video canvas layer is below the z-axis of the bullet screen layer and the control panel layer, the bullet screen layer is below the z-axis of the control panel layer or above the z-axis of the control panel layer, and the web page background layer is below the z-axis of the video canvas layer.

[0078] After obtaining the synthesized image, the operating system merges the synthesized image with the original desktop image of the operating system, so that the obtained system desktop image is the same as the web page video playback interface except for the video playback area when the video is not redirected.

[0079] The remote desktop server obtains the system desktop image in the operating system, and encodes the RGB value and A value of the system desktop image respectively, and then sends the system desktop image with the encoded RGB value and A value to the remote desktop client. Since the changed bullet screen rectangles will be rendered to the bullet screen layer, when sending the system desktop image obtained by merging the video canvas layer, the bullet screen layer, the control panel layer, and the web page background layer below the z-axis of the video canvas layer rendered with a black transparent frame to the remote desktop client, it only needs to be sent once, and the remote desktop client can obtain the shape, position and / or color of all the changed bullet screen rectangles, without the need to frequently send the bullet screen rectangles with changed shape, position and / or color to the remote desktop client, thereby avoiding a significant impact on the performance of the server.

[0080] Further, in one embodiment, referring to Figure 4 , Figure 4 for Figure 3 Step S007 in the figure is a detailed flow chart. Figure 4 As shown, step S006 includes:

[0081] Step S060, the remote desktop server performs encoding operations on the RGB value and the A value of the system desktop image respectively;

[0082] Step S061: The remote desktop server sends the system desktop image after encoding the RGB value and the A value to the remote desktop client, wherein the A value represents the value of the Alpha channel.

[0083] In this embodiment, the remote desktop server encodes the RGB value and the A value of the system desktop image through an encoder, thereby reducing the amount of data transmission. The remote desktop server then sends the system desktop image after encoding the RGB value and the A value to the remote desktop client, so that the remote desktop client can decode the system desktop image and render it to the system main surface layer, where the A value represents the value of the Alpha channel.

[0084] In a third aspect, an embodiment of the present invention further provides a barrage display device after video redirection.

[0085] In one embodiment, referring to Figure 5 , Figure 5 Schematic diagram of the functional modules of an embodiment of the bullet screen display device after video redirection of the present invention. Figure 5 As shown, the bullet screen display device after video redirection includes:

[0086] The first rendering module 10 is used for the remote desktop client to decode the original video stream and render it to the video layer after receiving the original video stream and the system desktop image, and to decode the system desktop image and render it to the system main surface layer after rendering, wherein the video layer is below the z-axis of the system main surface layer, and the system desktop image is obtained by merging the video canvas layer with a black transparent frame, the bullet screen layer above the z-axis of the video canvas layer, the control panel layer above the z-axis of the video canvas layer, the webpage background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system;

[0087] The first merging module 20 is used for the remote desktop client to merge the video layer that renders the decoded original video stream and the system main surface layer that renders the decoded system desktop image to obtain a merged image, wherein the system main surface layer blending mode is changed to a source overlay mode;

[0088] The display module 30 is used for the remote desktop client to send the combined image to the display for output.

[0089] Furthermore, in one embodiment, the barrage display device after video redirection further includes a first transmission module, which is used to:

[0090] The remote desktop client receives the original video stream sent by the modified video decoder, wherein the original video stream is not decoded.

[0091] Furthermore, in one embodiment, the barrage display device after video redirection further includes a creation module for:

[0092] The modified video decoder program creates black transparent frames;

[0093] The barrage display device after video redirection also includes a second rendering module, which is used to:

[0094] The web browser renders the black transparent frame to the video canvas layer;

[0095] The second rendering module is further used for: the web browser renders the changed bullet screen rectangle to the bullet screen layer, and renders the changed control panel rectangle to the control panel layer, wherein the change includes a change in shape, position and / or color;

[0096] The barrage display device after video redirection also includes a second merging module, which is used to:

[0097] The web browser merges the video canvas layer, the bullet screen layer, the control panel layer, and the web page background layer, which render the black transparent frame, to obtain a synthesized image, wherein the blending mode of the video canvas layer is changed to the source color space mode;

[0098] The second merging module is further used for: the operating system merges the image synthesized by the web browser and the original desktop image of the operating system to obtain a system desktop image;

[0099] The bullet screen display device after video redirection also includes a second transmission module, which is used to:

[0100] The remote desktop server encodes the system desktop image and sends it to the remote desktop client.

[0101] Furthermore, in one embodiment, the barrage display device after video redirection further includes an encoding module for:

[0102] The remote desktop server encodes the RGB value and A value of the system desktop image respectively;

[0103] The second transmission module is also used for: the remote desktop server sends the system desktop image after encoding the RGB value and the A value to the remote desktop client, wherein the A value represents the value of the Alpha channel.

[0104] Furthermore, in one embodiment, the bullet screen display device after video redirection further includes a decoding module for:

[0105] The remote desktop client performs decoding operations on the RGB value and the A value of the system desktop image respectively;

[0106] The first rendering module 10 is further used for: the remote desktop client renders the system main surface layer according to the RGB value and A value of the system desktop image obtained after decoding, wherein the A value represents the value of the Alpha channel.

[0107] Among them, the functional implementation of each module in the above-mentioned barrage display device after video redirection corresponds to the various steps in the above-mentioned barrage display method embodiment after video redirection, and its functions and implementation processes will not be repeated here one by one.

[0108] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium.

[0109] The readable storage medium of the present invention stores a bullet screen display program after video redirection, wherein when the bullet screen display program after video redirection is executed by a processor, the steps of the bullet screen display method after video redirection as described above are implemented.

[0110] Among them, the method implemented when the barrage display program after video redirection is executed can refer to the various embodiments of the barrage display method after video redirection of the present invention, and will not be repeated here.

[0111] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0112] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present invention.

[0114] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for displaying bullet screen after video redirection, characterized in that: The method for displaying bullet screen after video redirection includes: After receiving the original video stream and the system desktop image, the remote desktop client decodes the original video stream and renders it to the video layer, and decodes the system desktop image and renders it to the system main surface layer, wherein the video layer is below the z-axis of the system main surface layer, and the system desktop image is obtained by merging the video canvas layer rendered with a black transparent frame, the bullet screen layer above the z-axis of the video canvas layer, the control panel layer above the z-axis of the video canvas layer, the webpage background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system; The remote desktop client performs layer merging on the video layer that renders the decoded original video stream and the system main surface layer that renders the decoded system desktop image to obtain a merged image, wherein the system main surface layer blending mode is modified to a source overlay mode; The remote desktop client sends the merged image to a display for output; After the remote desktop client receives the original video stream and the system desktop image, before the steps of decoding the original video stream and rendering it to the video layer, and decoding the system desktop image and rendering it to the system main surface layer, the step further includes: The modified video decoder program creates black transparent frames; The web browser renders the black transparent frame to the video canvas layer; The web browser renders the changed bullet screen rectangle to the bullet screen layer, and renders the changed control panel rectangle to the control panel layer, wherein the change includes a change in shape, position and / or color; The web browser merges the video canvas layer, the bullet screen layer, the control panel layer, and the web page background layer, which render the black transparent frame, to obtain a synthesized image, wherein the blending mode of the video canvas layer is changed to the source color space mode; The operating system merges the image synthesized by the web browser with the original desktop image of the operating system to obtain a system desktop image; The remote desktop server encodes the system desktop image and sends it to the remote desktop client.

2. The method for displaying bullet screen after video redirection according to claim 1, characterized in that: After the remote desktop client receives the original video stream and the system desktop image, before the steps of decoding the original video stream and rendering it to the video layer, and decoding the system desktop image and rendering it to the system main surface layer, the method includes: The remote desktop client receives the original video stream sent by the modified video decoder, wherein the original video stream is not decoded.

3. The method for displaying bullet screen after video redirection according to claim 1, characterized in that: The step of encoding the system desktop image and sending it to the remote desktop client by the remote desktop server includes: The remote desktop server encodes the RGB value and A value of the system desktop image respectively; The remote desktop server sends the system desktop image after encoding the RGB value and the A value to the remote desktop client, wherein the A value represents the value of the Alpha channel.

4. The method for displaying bullet screen after video redirection according to claim 1, characterized in that: The step of decoding the system desktop image and rendering it to the system main surface layer comprises: The remote desktop client performs decoding operations on the RGB value and the A value of the system desktop image respectively; The remote desktop client renders the system main surface layer according to the RGB value and A value of the system desktop image obtained after decoding, wherein the A value represents the value of the Alpha channel.

5. A bullet screen display device after video redirection, characterized in that: The barrage display device after video redirection includes: A first rendering module is used for, after the remote desktop client receives the original video stream and the system desktop image, decoding the original video stream and rendering it to the video layer, and decoding the system desktop image and rendering it to the system main surface layer, wherein the video layer is below the z-axis of the system main surface layer, and the system desktop image is obtained by merging a video canvas layer with a black transparent frame rendered, a bullet screen layer above the z-axis of the video canvas layer, a control panel layer above the z-axis of the video canvas layer, a web page background layer below the z-axis of the video canvas layer, and the original desktop image of the operating system; A first merging module is used for the remote desktop client to merge the video layer that renders the decoded original video stream and the system main surface layer that renders the decoded system desktop image to obtain a merged image, wherein the system main surface layer blending mode is changed to a source overlay mode; A display module, used for the remote desktop client to send the merged image to a display for output; The first rendering module is used for the remote desktop client to decode the original video stream and render it to the video layer after receiving the original video stream and the system desktop image, and to decode and render the system desktop image to the system main surface layer before rendering, and further includes: The modified video decoder program creates black transparent frames; The web browser renders the black transparent frame to the video canvas layer; The web browser renders the changed bullet screen rectangle to the bullet screen layer, and renders the changed control panel rectangle to the control panel layer, wherein the change includes a change in shape, position and / or color; The web browser merges the video canvas layer, the bullet screen layer, the control panel layer, and the web page background layer, which render the black transparent frame, to obtain a synthesized image, wherein the blending mode of the video canvas layer is changed to the source color space mode; The operating system merges the image synthesized by the web browser with the original desktop image of the operating system to obtain a system desktop image; The remote desktop server encodes the system desktop image and sends it to the remote desktop client.

6. The device for displaying bullet screen after video redirection according to claim 5, characterized in that: The first rendering module is further used for: The remote desktop client performs decoding operations on the RGB value and the A value of the system desktop image respectively; The remote desktop client renders the system main surface layer according to the RGB value and A value of the system desktop image obtained after decoding, wherein the A value represents the value of the Alpha channel.

7. The device for displaying bullet screen after video redirection according to claim 5, characterized in that: The video redirected bullet screen display device further includes a first transmission module, which is used to: The remote desktop client receives the original video stream sent by the modified video decoder, wherein the original video stream is not decoded.

8. A bullet screen display device after video redirection, characterized in that: The device for displaying barrage after video redirection includes a processor, a memory, and a barrage display program for video redirection stored in the memory and executable by the processor, wherein when the barrage display program for video redirection is executed by the processor, the steps of the method for displaying barrage after video redirection as described in any one of claims 1 to 4 are implemented.

9. A readable storage medium, characterized in that: The readable storage medium stores a program for displaying barrage after video redirection, wherein when the program for displaying barrage after video redirection is executed by a processor, the steps of the method for displaying barrage after video redirection as described in any one of claims 1 to 4 are implemented.

Images