Video encoding method based on remote desktop protocol and its application
By monitoring network conditions and video stream data changes of the remote desktop protocol, and dynamically selecting the encoding mode, the problem of image quality and smoothness of the remote desktop protocol under different network conditions is solved, and a high-quality viewing experience is achieved under different network conditions.
Patent Information
- Application Number
- CN202210056778.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-18
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-01-18
AI Technical Summary
Existing remote desktop protocols suffer from stuttering when using lossless encoding in poor network conditions, while lossy encoding affects image quality, failing to provide a high-quality viewing experience under different network conditions.
The protocol server monitors the network status and total changes in video stream data within the channel, dynamically selects encoding modes, including lossless, high-quality, and low-quality encoding, and switches between different encoding modes based on network status and total data changes.
It provides high-quality video when network conditions are good, ensures smooth viewing when network conditions are average, and reduces network resource consumption when network conditions are poor, ensuring that users have a stable viewing experience under different network conditions.
Smart Images

Figure CN114390322B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of computers, and in particular to a video encoding method based on a remote desktop protocol and an application thereof. Background Art
[0002] The existing remote desktop protocol includes a client, a server, and corresponding display devices, display drivers, etc. The client runs on the user terminal device to provide a desktop environment for the user. After the remote desktop protocol is connected, multiple channels will be established, including a display channel, which controls the display on the remote desktop.
[0003] The video compression technology in the existing remote desktop protocol supports lossy encoding and lossless encoding. When lossless encoding is selected, a high-quality picture quality experience can be obtained, but it will cause lag when the network quality is poor. When lossy encoding is selected, a smoother user experience can be obtained when the network quality fluctuates, and it is less affected by network fluctuations, but the picture quality is lower, affecting the viewing effect.
[0004] Therefore, in view of the above technical problems, it is necessary to provide a video encoding method based on remote desktop protocol. Summary of the invention
[0005] The object of the present invention is to provide a video encoding method based on a remote desktop protocol, which can automatically select a corresponding encoding mode according to different network conditions to improve the user viewing experience.
[0006] To achieve the above purpose, the technical solution provided by the present invention is as follows:
[0007] In a first aspect, the present invention provides a video encoding method based on a remote desktop protocol, comprising:
[0008] The protocol server establishes a display channel connection with the client; the protocol server monitors the network status in the display channel and the total amount of data changes in the video stream in the display channel, and dynamically selects an encoding mode to encode the video stream according to the network status and the total amount of data changes.
[0009] In one or more embodiments, the protocol server monitors the network status in the display channel and the total amount of data changes in the video stream in the display channel, and dynamically selects an encoding mode to encode the video stream according to the network status and the total amount of data changes, including:
[0010] The protocol server periodically monitors the network status in the display channel and the total amount of data changes in the video stream in the display channel, wherein each monitoring period is defined as T n(n≥1), within the monitoring period T1, the protocol server selects one of the lossless coding mode, high-quality coding mode, and low-quality coding mode to encode the video stream; the monitoring period T m The video encoding mode in (m≥2) dynamically selects an encoding mode to encode the video stream according to the network status in the previous monitoring cycle and the total amount of data changes of the video stream in the display channel.
[0011] In one or more embodiments, when the monitoring period T n When the monitoring result is that the network condition is good, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 Select lossless coding mode or high-quality coding mode to encode the video stream within the monitoring period T n When the monitoring result is that the network condition is normal, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 Select a high-quality encoding mode or a low-quality encoding mode to encode the video stream within the monitoring period T n When the monitoring result is that the network condition is poor, the protocol server will n+1 A low-quality encoding mode is directly selected to encode the video stream.
[0012] In one or more embodiments, the total amount of data changes in the display channel is a single monitoring period T n The total length of the rectangular area of each frame data change in the internal display channel.
[0013] In one or more embodiments, the monitoring period T n The length of the rectangular area of each frame of data change is defined as s, and s is expressed by the following formula:
[0014] s=(right-left)×(bottom-top)×d
[0015] Among them, left and top are the coordinates of the upper left corner of each frame data; right and bottom are the coordinates of the lower right corner, and d is the color depth; the monitoring period T n Displays the total amount of data changes in the channel S n =s1+s2+s3+......+s p (p≥1).
[0016] In one or more embodiments, when the monitoring period T n When the monitoring result is that the network condition is good, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1Selecting a lossless encoding mode or a high-quality encoding mode to encode the video stream includes:
[0017] If the monitoring period T n The monitoring result is that the network condition is good, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is greater than the first threshold Q1, then in the next monitoring period T n+1 Select high-quality encoding mode within the monitoring period T n The monitoring result is that the network condition is good, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is less than the second threshold Q2, then in the next monitoring period T n+1 Select lossless coding mode within the monitoring period T n The monitoring result is that the network condition is good, and the monitoring period is T n The low-quality encoding mode or high-quality encoding mode is used within the monitoring period T n The total amount of data change S of the video stream in the display channel n If it is not less than the second threshold Q2, then in the next monitoring period T n+1 Switch to or maintain high-quality coding mode; wherein, Q1=P1×R, Q2=P2×R, R=(width×height) / resolution, and p×resolution×d>P1>P2>resolution×d; P1 is the preset threshold for switching from lossless coding mode to high-quality coding mode, P2 is the threshold for switching from high-quality coding mode to lossless coding mode, R is the threshold factor, resolution is the resolution of the video stream in the display channel, and width and height are the highest resolutions that the display screen can support.
[0018] In one or more embodiments, when the monitoring period T n When the monitoring result is that the network condition is normal, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 The method further comprises: selecting a low-quality encoding mode or a high-quality encoding mode to encode the video stream, comprising:
[0019] If the monitoring period T n The monitoring result is that the network status is normal, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is greater than the third threshold Q3, then in the next monitoring period T n+1 Select low-quality encoding mode within the monitoring period T n The monitoring result is that the network status is normal, and the monitoring period is T nThe total amount of data change S of the video stream in the display channel n When it is less than the fourth threshold value Q4, then in the next monitoring period T n+1 Select high-quality encoding mode within the monitoring period T n The monitoring result is that the network status is normal, and the monitoring period is T n The lossless coding mode or high-quality coding mode is used within the monitoring period T n The total amount of data change S of the video stream in the display channel n When it is not less than the fourth threshold value Q4, then in the next monitoring period T n+1 Switch to low-quality encoding mode; wherein, Q3=P3×R, Q4=P4×R, R=(width×height) / resolution, p×resolution×d>P3>P4>resolution×d; P3 is the preset threshold for switching from high-quality encoding mode to low-quality encoding mode, P4 is the threshold for switching from low-quality encoding mode to high-quality encoding mode, resolution is the resolution of the video stream in the display channel, R is the threshold factor, width and height are the highest resolutions that the display screen can support.
[0020] In one or more implementations, the evaluation indicators of the network status include network packet loss rate, round-trip delay and network jitter.
[0021] In one or more embodiments, the upper and lower limits of the network packet loss rate, round-trip delay, and network jitter are preset; when the monitoring period T n When the network packet loss rate, round-trip delay and network jitter monitored within the monitoring period are not higher than the lower limit, the network condition is judged to be good; when the monitoring period T n If the network packet loss rate, round-trip delay and network jitter monitored within the monitoring period are not lower than the upper limit, the network condition is judged to be poor; when the monitoring period T n When any evaluation index among the network packet loss rate, round-trip delay and network jitter monitored within the network is between the lower limit and the upper limit, the network condition is judged to be normal.
[0022] In a second aspect, the present invention provides a video encoding device based on a remote desktop protocol, comprising: a communication module, a monitoring module, a selection module and an encoding module; the communication module is used for a protocol server to establish a connection between a display channel and a client; the monitoring module is used for the protocol server to monitor the network status in the display channel and the total amount of data changes in the video stream in the display channel; the selection module is used for the protocol server to dynamically select an encoding mode according to the network status and the total amount of data changes in the video stream in the display channel; the encoding module is used to encode the video stream according to the encoding mode selected by the selection module.
[0023] In a third aspect, the present invention provides an electronic device comprising: at least one processor and a memory, wherein the memory stores instructions, and when the instructions are executed by the at least one processor, the at least one processor executes the video encoding method based on the remote desktop protocol as described above.
[0024] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the remote desktop protocol-based video encoding method as described above is implemented.
[0025] Compared with the prior art, the video encoding method based on the remote desktop protocol provided by the present invention can dynamically select the encoding mode for the video stream according to the current network condition and the total amount of data changes in the display channel. When the network condition is poor, the low-quality encoding mode of the high-compression image is automatically selected to minimize the occupation of network resources; when the network condition is general, it can dynamically switch between the low-quality encoding mode and the high-quality encoding mode according to the total amount of data changes in the display channel to ensure a smooth viewing experience for the user; when the network condition is good, it can dynamically switch between the lossless encoding mode and the high-quality encoding mode according to the total amount of data changes in the display channel, so that the user can obtain a viewing experience with higher image quality without network restrictions. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 is a flow chart of a video encoding method based on a remote desktop protocol according to an embodiment of the present invention;
[0027] Figure 2 It is a structural block diagram of a video encoding device based on a remote desktop protocol according to an embodiment of the present invention;
[0028] Figure 3 It is a structural block diagram of a computing device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0029] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.
[0030] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.
[0031] The flowchart of the video encoding method based on the remote desktop protocol in one embodiment of the present invention is as follows Figure 1 As shown, the method comprises the following steps:
[0032] S101: The protocol server establishes a display channel connection with the client.
[0033] Before performing video encoding based on the Remote Desktop Protocol, you need to deploy and prepare the Remote Desktop Protocol environment to establish a connection between the protocol server and the client display channel. The display channel is one of the multiple channels after the Remote Desktop Protocol is connected, and is mainly responsible for processing graphics commands and transmitting image and video data. The Remote Desktop Protocol handles different events by establishing different channels. When the protocol server and the protocol client are connected, the display channel is successfully created, so I will not go into details here.
[0034] The remote desktop protocol can be selected from existing remote desktop protocols according to actual needs and is not particularly limited here.
[0035] S102: The protocol server monitors the network status in the display channel and the total amount of data changes in the video stream in the display channel, and dynamically selects an encoding mode to encode the video stream according to the network status and the total amount of data changes in the display channel.
[0036] Step S102 specifically includes that the protocol server periodically monitors the network status in the display channel and the total amount of data changes in the video stream in the display channel. Each monitoring period is defined as T n (n≥1), within the monitoring period T1, the protocol server selects one of the lossless coding mode, high-quality coding mode, and low-quality coding mode to encode the video stream. Among them, the high-quality coding mode and the low-quality coding mode are both lossy coding, and the high-quality coding has higher image quality than the low-quality coding.
[0037] Monitoring period T m The video encoding mode within (m≥2) dynamically selects an encoding mode to encode the video stream according to the network status in the previous monitoring cycle and the total amount of data changes of the video stream in the display channel.
[0038] In an exemplary embodiment, the evaluation indicators of network status mainly include network packet loss rate, round-trip delay and network jitter, so the protocol server needs to monitor the evaluation indicators of the aforementioned network status. The current network status can be rated according to the evaluation indicators of the network status monitored by the protocol server. In actual applications, other indicators for evaluating the network status can also be added, and the specific indicator settings can be selected according to actual needs.
[0039] Specifically, the network conditions can be divided into three levels: good, general and poor according to the network packet loss rate, round-trip delay and network jitter monitored by the protocol server, and different encoding modes can be formulated according to different levels of network conditions.
[0040] Specifically, the upper and lower limits of network packet loss rate, round-trip delay and network jitter can be preset; when the monitoring period T n When the network packet loss rate, round-trip delay and network jitter monitored within the monitoring period are not higher than the lower limit, the network condition is judged to be good; when the monitoring period T n If the network packet loss rate, round-trip delay and network jitter monitored within the monitoring period are not lower than the upper limit, the network condition is judged to be poor; when the monitoring period T n When any evaluation index among the network packet loss rate, round-trip delay and network jitter monitored within the network is between the lower limit and the upper limit, the network condition is judged to be normal.
[0041] For example, when the network bandwidth is 50-100Gbp / s, the upper and lower limits of the network packet loss rate can be set to 0.1% and 0.01% respectively, the upper and lower limits of the round-trip delay can be set to 100ms and 30ms respectively, and the upper and lower limits of the network jitter can be set to 40ms and 10ms respectively. Different upper and lower limits can be preset based on different network bandwidths.
[0042] When the network packet loss rate monitored by the protocol server is less than or equal to 0.01%, and the round-trip delay is less than or equal to 30ms, and the network jitter is less than or equal to 10ms, the network condition is judged to be good. When the network packet loss rate monitored by the protocol server is greater than or equal to 0.1%, and the round-trip delay is greater than or equal to 100ms, and the network jitter is greater than or equal to 40ms, the network condition is judged to be poor. When the network packet loss rate monitored by the protocol server is between 0.01 and 0.1%, and / or the round-trip delay is between 30 and 100ms, and / or the network jitter is between 10 and 40ms, the network condition is judged to be poor.
[0043] In the present invention, the threshold of the evaluation index of the network status can be preset according to different actual use environments (such as the size of the overall bandwidth), and the threshold of the evaluation index of the network status can also be changed according to the change of the actual use environment. Therefore, it can be understood that the threshold of the evaluation index used to evaluate different levels of network status here can be changed according to the actual use situation.
[0044] In other embodiments, the network status may be divided into more or fewer levels, which can be selected according to actual needs. For example, when it comes to the selection of more coding modes based on network status, the network status may be divided into more levels. The thresholds of various indicators used to divide the network status levels may also be adjusted according to actual conditions, and are not particularly limited here. In this embodiment, the total amount of data changes in the display channel refers to the total amount of data changes in a single monitoring period T n The total length of the rectangular area of each frame data change in the internal display channel.
[0045] Assume monitoring period T n (The cycle time can be set according to actual needs, for example, it can be any value between 1 and 3 seconds), there are p frames of data in the display channel, and the length of the rectangular area of each frame of data change is defined as s, which can be expressed by the following formula:
[0046] s=(right-left)×(bottom-top)×d
[0047] Among them, left and top are the coordinates of the point in the upper left corner; right and bottom are the coordinates of the point in the lower right corner; d is the color depth.
[0048] The monitoring period is T n In the display channel, the length of the rectangular area of the first frame data change is s1, the length of the rectangular area of the second frame data change is s2, the length of the rectangular area of the third frame data change is s3... The length of the rectangular area of the pth frame data change is s p Then, in the monitoring period T n Displays the total amount of data change S of the p-frame data in the channel n =s1+s2+s3+......+s p .
[0049] In an exemplary embodiment, when the monitoring period T n When the monitoring result shows that the network condition is good, the protocol server will display the total amount of data changes of the video stream in the channel in the next monitoring period T n+1 The video stream is encoded by selecting a lossless encoding mode or a high-quality encoding mode. When the network condition is good, there are two encoding modes, lossless encoding mode and high-quality encoding mode, to choose from, so that users can get a high-quality video experience.
[0050] Specifically, if the monitoring period T n The monitoring result is that the network condition is good, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is greater than the first threshold Q1, then in the next monitoring period T n+1 Select high quality encoding mode.
[0051] If the monitoring period T n The monitoring result is that the network condition is good, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is less than the second threshold Q2, then in the next monitoring period T n+1 Select lossless encoding mode.
[0052] If the monitoring period T nThe monitoring result is that the network condition is good, and the monitoring period is T n The low-quality encoding mode or high-quality encoding mode is used within the monitoring period T n The total amount of data change S of the video stream in the display channel n If it is not less than the second threshold Q2, then in the next monitoring period T n+1 Switch or maintain high quality encoding mode.
[0053] Among them, Q1=P1×R, Q2=P2×R, R=(width×height) / resolution, and p×resolution×d>P1>P2>resolution×d; P1 is the preset threshold for switching from lossless coding mode to high-quality coding mode, P2 is the threshold for switching from high-quality coding mode to lossless coding mode, R is the threshold factor, resolution is the resolution of the video stream in the display channel, and width and height are the highest resolutions that the display screen can support. The first threshold Q1 and the second threshold Q2 can be set according to actual conditions, combined with actual network quality factors such as monitoring time, resolution and network downlink bandwidth. For example, the longer the monitoring period corresponding to the total amount of data changes in the display channel is set, the larger the corresponding number of frames, and the first threshold Q1 and the second threshold Q2 need to be adjusted accordingly; otherwise, the first threshold Q1 and the second threshold Q2 need to be adjusted accordingly. For example, the larger the resolution, the higher the first threshold Q1 and the second threshold Q2 need to be adjusted accordingly; otherwise, the first threshold Q1 and the second threshold Q2 need to be adjusted accordingly. For another example, the better the network condition, the higher the first threshold Q1 and the second threshold Q2 need to be adjusted accordingly; otherwise, the first threshold Q1 and the second threshold Q2 need to be adjusted lower accordingly.
[0054] In an exemplary embodiment, when the monitoring period T n When the monitoring result is that the network condition is normal, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 A high-quality encoding mode or a low-quality encoding mode is selected to encode the video stream. In the case of a normal network condition, there are two encoding modes, the high-quality encoding mode and the low-quality encoding mode, to choose from.
[0055] Specifically, if the monitoring period T n The monitoring result is that the network status is normal, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is greater than the third threshold Q3, then in the next monitoring period T n+1 Select low quality encoding mode.
[0056] If the monitoring period T n The monitoring result is that the network status is normal, and the monitoring period is T nThe total amount of data change S of the video stream in the display channel n When it is less than the fourth threshold value Q4, then in the next monitoring period T n+1 Select high quality encoding mode.
[0057] If the monitoring period T n The monitoring result is that the network status is normal, and the monitoring period is T n The lossless coding mode or high-quality coding mode is used within the monitoring period T n The total amount of data change S of the video stream in the display channel n When it is not less than the fourth threshold value Q4, then in the next monitoring period T n+1 Switch to low quality encoding mode.
[0058] Among them, Q3=P3×R, Q4=P4×R, R=(width×height) / resolution, p×resolution×d>P3>P4>resolution×d; P3 is the preset threshold for switching from high-quality encoding mode to low-quality encoding mode, P4 is the threshold for switching from low-quality encoding mode to high-quality encoding mode, resolution is the resolution of the video stream in the display channel, R is the threshold factor, width and height are the highest resolutions that the display screen can support.
[0059] The third threshold Q3 and the fourth threshold Q3 can be set according to the actual situation, combined with the actual network quality factors such as monitoring time, resolution and network downlink bandwidth. For example, the longer the time T corresponding to the total amount of data changes in the display channel is set, the larger the corresponding number of frames, and the third threshold Q3 and the fourth threshold Q4 need to be adjusted accordingly; conversely, the third threshold Q3 and the fourth threshold Q4 need to be adjusted accordingly. For another example, the larger the resolution, the third threshold Q3 and the fourth threshold Q4 need to be adjusted accordingly; conversely, the third threshold Q3 and the fourth threshold Q4 need to be adjusted accordingly. For another example, the better the network conditions, the third threshold Q3 and the fourth threshold Q4 need to be adjusted accordingly; conversely, the third threshold Q3 and the fourth threshold Q4 need to be adjusted accordingly.
[0060] In an exemplary embodiment, when the monitoring period T n If the monitoring result is that the network condition is poor, the protocol server will n+1 The low-quality encoding mode is directly selected to encode the video stream. In the case of poor network conditions, in order to ensure that users have a smooth viewing experience, a low-quality encoding mode can be selected to encode the video stream to adapt to the poor network conditions.
[0061] Specifically, the aforementioned encoding mode may be an encoding method based on encoding standards such as H.263, H.263+, H.263++, H.264 / AVC, H.265 / HEVC, H.266 / VVC, and MPEG4.
[0062] Among them, MPEG (Moving Picture Experts Group) is a working group established by ISO (International Organization for Standardization) to formulate standards for moving image compression coding. The MPEG standard formulated is an international standard, which consists of three parts: video, audio and system. MPEG4 is one of the network video image compression standards, characterized by high compression ratio and clear imaging. The MPEG4 video compression algorithm can provide an extremely high compression ratio, up to 200:1. More importantly, MPEG4 has little data loss while providing a high compression ratio.
[0063] H.263 is a draft standard of ITU (International Telecommunication Union) and is designed for low bit rate communication. But in fact, this standard can be used in a wide range of bit rates, not just low bit rates, and the standard has higher performance and error correction capabilities. H.263's motion compensation uses half-pixel precision, and some parts of the data stream hierarchy are optional in H.263, so that the codec can be configured to a lower data rate or better error correction capability. H.263 contains four negotiable options to improve performance. H.263 uses unlimited motion vectors and syntax-based arithmetic coding, and H.263 supports 5 resolutions.
[0064] The video encoding method based on the remote desktop protocol of the present invention is further described below in conjunction with a specific embodiment scenario.
[0065] Example 1
[0066] Deploy and prepare the remote desktop protocol environment, and set the first threshold Q1, the second threshold Q2, the third threshold Q3, and the fourth threshold Q4 according to the network environment, resolution, and other factors. Among them, the first threshold Q1 is greater than the second threshold Q2, and the third threshold Q3 is greater than the fourth threshold Q4. The upper and lower limits of the network packet loss rate are set to 0.1% and 0.01% respectively, the upper and lower limits of the round-trip delay are set to 100ms and 30ms respectively, and the upper and lower limits of the network jitter are set to 40ms and 10ms respectively.
[0067] The virtual machine is started, and the protocol server establishes a connection with the client (including the connection of the display channel). The protocol server periodically monitors the network status in the display channel and the total amount of data changes in the video stream in the display channel.
[0068] Assume monitoring period T n If the monitored network packet loss rate is less than or equal to 0.01%, the round-trip delay is less than or equal to 30ms, and the network jitter is less than or equal to 10ms, the network condition is judged to be good.
[0069] The protocol server monitors the period T n The total amount of data changes in the monitored display channel is S n . The total amount of data change in the display channel S n Compare with the first threshold Q1 and the second threshold Q2. There are the following situations:
[0070] If the total amount of data change S of the video stream in the display channel n is greater than the first threshold Q1, indicating that the monitoring period T n Total amount of internal data changes S n Larger, indicating monitoring period T n If the video stream transmission is unstable and does not meet the lossless coding mode, it is necessary to n+1 Select high quality encoding mode.
[0071] If the total amount of data change S of the video stream in the display channel n is less than the second threshold Q2, which means that the monitoring period T n Total amount of internal data changes S n Smaller, indicating monitoring period T n If the video stream transmission is relatively stable and meets the lossless coding mode, the next monitoring period T n+1 Select lossless encoding mode.
[0072] If the total amount of data change S of the video stream in the display channel n is not less than the second threshold Q2, and the monitoring period T n When the coding mode in the next monitoring period T is low-quality coding mode or high-quality coding mode, n+1 Switch or maintain high quality encoding mode.
[0073] In this embodiment, the dynamic selection of the video encoding mode is mainly described when the network condition is good.
[0074] Example 2
[0075] Deploy and prepare the remote desktop protocol environment, and set the first threshold Q1, the second threshold Q2, the third threshold Q3, and the fourth threshold Q4 according to the network environment, resolution, and other factors. Among them, the first threshold Q1 is greater than the second threshold Q2, and the third threshold Q3 is greater than the fourth threshold Q4. The upper and lower limits of the network packet loss rate are set to 0.1% and 0.01% respectively, the upper and lower limits of the round-trip delay are set to 100ms and 30ms respectively, and the upper and lower limits of the network jitter are set to 40ms and 10ms respectively.
[0076] The virtual machine is started, and the protocol server establishes a connection with the client (including the connection of the display channel). The protocol server periodically monitors the network status in the display channel and the total amount of data changes in the video stream in the display channel.
[0077] Assume monitoring period T n If the monitored network packet loss rate is between 0.01% and 0.1%, the round-trip delay is between 30 and 100ms, and the network jitter is between 10 and 40ms, the network status is judged to be normal.
[0078] The protocol server monitors the period T n The total amount of data changes in the monitored display channel is S n . The total amount of data change in the display channel S n Compare with the third threshold value Q3 and the fourth threshold value Q4. There are the following situations:
[0079] If the total amount of data change S of the video stream in the display channel n is greater than the third threshold Q3, indicating that the monitoring period T n Total amount of internal data changes S n Larger, indicating monitoring period T n If the video stream transmission is unstable and does not meet the high-quality encoding mode, then in the next monitoring period T n+1 Select low quality encoding mode.
[0080] If the total amount of data change S of the video stream in the display channel n is less than the fourth threshold Q4, which means that the monitoring period T n Total amount of internal data changes S n Smaller, indicating monitoring period T n If the video stream transmission is relatively stable and meets the high encoding mode, the next monitoring period T n+1 Select high encoding mode.
[0081] If the total amount of data change S of the video stream in the display channel n is not less than the second threshold Q2, and the monitoring period T nWhen the coding mode in the next monitoring period T is low-quality coding mode or high-quality coding mode, n+1 Switch or maintain high quality encoding mode.
[0082] In this embodiment, the dynamic selection of the video encoding mode is mainly described when the network condition is normal.
[0083] Example 3
[0084] Deploy and prepare the remote desktop protocol environment, and set the first threshold Q1, the second threshold Q2, the third threshold Q3, and the fourth threshold Q4 according to the network environment, resolution, and other factors. Among them, the first threshold Q1 is greater than the second threshold Q2, and the third threshold Q3 is greater than the fourth threshold Q4. The upper and lower limits of the network packet loss rate are set to 0.1% and 0.01% respectively, the upper and lower limits of the round-trip delay are set to 100ms and 30ms respectively, and the upper and lower limits of the network jitter are set to 40ms and 10ms respectively.
[0085] The virtual machine is started, and the protocol server establishes a connection with the client (including the connection of the display channel). The protocol server periodically monitors the network status in the display channel and the total amount of data changes in the video stream in the display channel.
[0086] Assume monitoring period T n If the monitored network packet loss rate is greater than 0.1%, the round-trip delay is greater than 100ms, and the network jitter is greater than 40ms, the network condition is judged to be poor. At this time, the network is relatively slow, and the protocol server will perform a network check in the next monitoring period T n+1 A low-quality encoding mode is directly selected to encode the video stream. The low-quality encoding mode can ensure that the user has a smooth viewing experience when the network condition is poor.
[0087] Based on the same inventive concept, the structural frame of the video encoding device 2 based on the remote desktop protocol in one embodiment of the present invention is as follows: Figure 2 As shown, the video encoding device based on the remote desktop protocol includes: a communication module 21, a monitoring module 22, a selection module 23 and an encoding module 24.
[0088] Among them, the communication module 21 is used for the protocol server to establish a connection between the display channel and the client. The monitoring module 22 is used for the protocol server to monitor the network status in the display channel and the total amount of data changes in the video stream in the display channel. The selection module 23 is used for the protocol server to dynamically select the encoding mode according to the network status and the total amount of data changes in the video stream in the display channel. The encoding module 24 is used to encode the video stream according to the encoding mode selected by the selection module.
[0089] The communication module 21 can realize the communication connection between the protocol server and the client. The communication module 21 can be a wireless communication module or a wired communication module.
[0090] like Figure 3 As shown, an embodiment of the present invention further provides a computing device 3, which includes at least one processor 31, a storage 32 (such as a non-volatile memory), a memory 33 and a communication interface 34, and the at least one processor 31, the storage 32, the memory 33 and the communication interface 34 are connected together via a bus 35. At least one processor 31 is used to call at least one program instruction stored or encoded in the memory 32, so that the at least one processor 31 performs various operations and functions of the video encoding method based on the remote desktop protocol described in various embodiments of this specification.
[0091] In the embodiments of the present specification, the computing device 3 may include, but is not limited to, a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile computing device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable computing device, a consumer electronic device, and the like.
[0092] An embodiment of the present invention further provides a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions are used to enable a computer to execute the above-mentioned video encoding method based on the remote desktop protocol.
[0093] The computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD), etc.).
[0094] In summary, the video encoding method based on the remote desktop protocol provided by the present invention can dynamically select the encoding mode for the video stream according to the current network condition and the total amount of data changes in the display channel. When the network condition is poor, the low-quality encoding mode of the high-compression image is automatically selected to minimize the occupation of network resources; when the network condition is general, it can dynamically switch between the low-quality encoding mode and the high-quality encoding mode according to the total amount of data changes in the display channel to ensure a smooth viewing experience for the user; when the network condition is good, it can dynamically switch between the lossless encoding mode and the high-quality encoding mode according to the total amount of data changes in the display channel, so that the user can obtain a viewing experience with higher image quality without network restrictions.
[0095] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0096] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0097] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0099] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.
Claims
1. A video encoding method based on remote desktop protocol, characterized in that: include: The protocol server establishes a display channel connection with the client; The protocol server periodically monitors the network status in the display channel and the total amount of data changes in the video stream in the display channel, wherein each monitoring period is defined as T n (n≥1), within the monitoring period T1, the protocol server selects one of the lossless encoding mode, high-quality encoding mode, and low-quality encoding mode to encode the video stream; the total amount of data change in the display channel is a single monitoring period T n The sum of the lengths of the rectangular areas of each frame of data change in the display channel, monitoring period T n The length of the rectangular area of each frame of data change is defined as s, s = (right-left) × (bottom-top) × d; left and top are the coordinates of the upper left corner of each frame of data; right and bottom are the coordinates of the lower right corner, d is the color depth; the monitoring period T n Displays the total amount of data changes in the channel S n =s1+s2+s3+......+s p (p≥1), where p is the monitoring period T n The number of data frames in the internal display channel; When the monitoring period T n When the monitoring result is that the network condition is good, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 Selecting a lossless encoding mode or a high-quality encoding mode to encode the video stream; When the monitoring period T n When the monitoring result is that the network condition is normal, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 Selecting a high-quality encoding mode or a low-quality encoding mode to encode the video stream; When the monitoring period T n When the monitoring result is that the network condition is poor, the protocol server will n+1 A low-quality encoding mode is directly selected to encode the video stream.
2. The video encoding method based on the remote desktop protocol as claimed in claim 1, characterized in that: When the monitoring period T n When the monitoring result is that the network condition is good, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 Selecting a lossless encoding mode or a high-quality encoding mode to encode the video stream includes: If the monitoring period T n The monitoring result is that the network condition is good, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is greater than the first threshold Q1, then in the next monitoring period T n+1 Select high quality encoding mode; If the monitoring period T n The monitoring result is that the network condition is good, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is less than the second threshold Q2, then in the next monitoring period T n+1 Select lossless encoding mode; If the monitoring period T n The monitoring result is that the network condition is good, and the monitoring period is T n The low-quality encoding mode or high-quality encoding mode is used within the monitoring period T n The total amount of data change S of the video stream in the display channel n If it is not less than the second threshold Q2, then in the next monitoring period T n+1 Switch or maintain high-quality encoding mode; Wherein, Q1=P1×R, Q2=P2×R, R=(width×height) / resolution, and p×resolution×d>P1>P2>resolution×d; P1 is the preset threshold for switching from lossless encoding mode to high-quality encoding mode, P2 is the threshold for switching from high-quality encoding mode to lossless encoding mode, R is the threshold factor, resolution is the resolution of the video stream in the display channel, and width and height are the highest resolutions that the display screen can support.
3. The video encoding method based on remote desktop protocol as claimed in claim 2, characterized in that: When the monitoring period T n When the monitoring result is that the network condition is normal, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 The method further comprises: selecting a low-quality encoding mode or a high-quality encoding mode to encode the video stream, comprising: If the monitoring period T n The monitoring result is that the network status is normal, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is greater than the third threshold Q3, then in the next monitoring period T n+1 Select low quality encoding mode; If the monitoring period T n The monitoring result is that the network status is normal, and the monitoring period is T n The total amount of data change S of the video stream in the display channel n When it is less than the fourth threshold value Q4, then in the next monitoring period T n+1 Select high quality encoding mode; If the monitoring period T n The monitoring result is that the network status is normal, and the monitoring period is T n The lossless coding mode or high-quality coding mode is used within the monitoring period T n The total amount of data change S of the video stream in the display channel n When it is not less than the fourth threshold value Q4, then in the next monitoring period T n+1 Switch to low-quality encoding mode; Wherein, Q3=P3×R, Q4=P4×R, R=(width×height) / resolution, p×resolution×d>P3>P4>resolution×d; P3 is the preset threshold for switching from high-quality encoding mode to low-quality encoding mode, P4 is the threshold for switching from low-quality encoding mode to high-quality encoding mode, resolution is the resolution of the video stream in the display channel, R is the threshold factor, width and height are the highest resolutions that the display screen can support.
4. The video encoding method based on the remote desktop protocol according to any one of claims 1 to 3, characterized in that: The evaluation indicators of the network status include network packet loss rate, round-trip delay and network jitter.
5. The video encoding method based on remote desktop protocol as claimed in claim 4, characterized in that: Preset the upper and lower limits of network packet loss rate, round-trip delay, and network jitter; When the monitoring period T n When the network packet loss rate, round-trip delay and network jitter monitored within the period are not higher than the lower limit, the network condition is judged to be good; When the monitoring period T n If the network packet loss rate, round-trip delay and network jitter monitored within the period are not lower than the upper limit, the network condition is judged to be poor; When the monitoring period T n When any evaluation index among the network packet loss rate, round-trip delay and network jitter monitored within the network is between the lower limit and the upper limit, the network condition is judged to be normal.
6. A video encoding device based on remote desktop protocol, characterized in that: include: Communication module, used for establishing a display channel connection between the protocol server and the client; The monitoring module is used for the protocol server to periodically monitor the network status in the display channel and the total amount of data changes in the video stream in the display channel, wherein each monitoring period is defined as T n (n≥1), within the monitoring period T1, the protocol server selects one of the lossless coding mode, high-quality coding mode, and low-quality coding mode to encode the video stream; The selection module is used for the protocol server to dynamically select the encoding mode according to the network status and the total amount of data changes in the video stream in the display channel; the total amount of data changes in the display channel is a single monitoring period T n The sum of the lengths of the rectangular areas of each frame of data change in the display channel, monitoring period T n The length of the rectangular area of each frame of data change is defined as s, s = (right-left) × (bottom-top) × d; left and top are the coordinates of the upper left corner of each frame of data; right and bottom are the coordinates of the lower right corner, d is the color depth; the monitoring period T n Displays the total amount of data changes in the channel S n =s1+s2+s3+......+s p (p≥1), where p is the number of data frames in the display channel within the monitoring period Tn; An encoding module, used for encoding the video stream according to the encoding mode selected by the selection module; When the monitoring period T n When the monitoring result is that the network condition is good, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 Selecting a lossless encoding mode or a high-quality encoding mode to encode the video stream; When the monitoring period T n When the monitoring result is that the network condition is normal, the protocol server will display the total amount of data changes of the video stream in the display channel in the next monitoring period T n+1 Selecting a high-quality encoding mode or a low-quality encoding mode to encode the video stream; When the monitoring period T n When the monitoring result is that the network condition is poor, the protocol server will n+1 A low-quality encoding mode is directly selected to encode the video stream.
7. An electronic device, characterized in that: include: at least one processor; as well as A memory storing instructions, which, when executed by the at least one processor, enables the at least one processor to execute the video encoding method based on the remote desktop protocol as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the video encoding method based on the remote desktop protocol according to any one of claims 1 to 5 is implemented.
