A high frame rate and high resolution remote desktop construction system

Abstract

The present invention relates to the technical field of remote desktop connection, and in particular to a high-frame-rate and high-resolution remote desktop construction system, comprising a collection module for collecting remote-end desktop image data, a first processing module for processing the collected desktop image data, a transmission module for transmitting data packets, a receiving module for receiving the transmitted data packets, a second processing module for completing data packet parsing, remote-end desktop image processing and desktop image push, a monitoring module for monitoring system operation information, and a management module for preliminarily determining whether the transmission of the data packet meets the preset standard according to the response time and determining whether the operation of the system meets the preset standard according to the packet loss rate of the data packet transmission within the preset time, thereby improving the stability of the high-frame-rate and high-resolution remote desktop and the smoothness of operation.

Description

Technical Field

[0001] The present invention relates to the technical field of remote desktop connection, and in particular to a remote desktop construction system with high frame rate and high resolution. Background Art

[0002] Remote desktop technology is a technology that allows users to connect to a remote computer through a network and access and control the remote computer desktop on a local computer. For industries such as film and television production, animation rendering, and 3D modeling, it is difficult for these industries to support their remote operations through existing technologies or tools because of the extremely high requirements for display devices.

[0003] The current mainstream remote desktop system has sufficient support for display resolution, but not enough support for display refresh rate. It can only meet simple remote desktop operations, and even the response of the mouse and keyboard is quite different from local operations. Supporting refresh rate means that the amount of data will increase, the difficulty of data compression will increase, and it also means that the difficulty of network transmission will increase. Based on these factors, the current remote desktop system is difficult to support high-resolution desktops while supporting high refresh rates, which brings great challenges. Summary of the invention

[0004] To this end, the present invention provides a high frame rate and high resolution remote desktop construction system to overcome the problems of poor stability and unsmooth operation of the prior art in constructing a high frame rate and high resolution remote desktop.

[0005] To achieve the above object, the present invention provides a high frame rate and high resolution remote desktop construction system, comprising:

[0006] A collection module is provided at a remote end to collect desktop image data at the remote end;

[0007] A first processing module, which is arranged at the remote end and connected to the acquisition module, is used to encode, packetize and compress the collected desktop image data to obtain a data packet to be transmitted;

[0008] A transmission module, which is arranged at the remote end and connected to the first processing module, and includes a plurality of transmission channels for transmitting data packets;

[0009] A receiving module, which is arranged at a local end and connected to the transmission module, for receiving the transmitted data packet;

[0010] A second processing module, which is arranged at the local end and connected to the receiving module, is used to successively complete the parsing of the data packet, the processing of the remote end desktop image and the pushing of the desktop image, wherein the image processing includes the rendering and scaling of the remote end desktop image;

[0011] A monitoring module, which is arranged at the local end and is respectively connected to the acquisition module, the first processing module, the transmission module and the second processing module, for detecting and obtaining the response time, the transmission rate of the transmission channel, the data packet size, the packet loss rate and the proportion of out-of-order data;

[0012] A management module is arranged at the local end and is respectively connected to the monitoring module, the acquisition module, the first processing module, the transmission module, the receiving module and the second processing module, and is used to preliminarily determine whether the transmission of the data packet meets the preset standard based on the response time, and to determine whether the operation of the system meets the preset standard based on the packet loss rate of the data packet transmission within the preset time.

[0013] Furthermore, when the management module preliminarily determines that the transmission of the data packet does not meet the preset standard based on the response duration, it determines the reason why the transmission does not meet the preset standard according to the transmission rate evaluation value, or expands the bandwidth of each transmission channel.

[0014] Furthermore, the response duration is the duration between the time when the first processing module receives the remote desktop image data and the time when the second processing module completes the processing of the remote desktop image.

[0015] Furthermore, the expansion amplitude of the bandwidth of each transmission channel is positively correlated with the response time difference, and the response time difference is the difference between the response time and a second preset response time.

[0016] Furthermore, the transmission rate evaluation value is set to S, and S is obtained by formula (1):

[0017]

[0018] In formula (1), μ is the evaluation coefficient, μ=0.68 is set, Li is the average transmission rate of the i-th transmission channel within the preset time length, i=1,2...n, n is the total number of transmission channels, and L0 is the average rate threshold.

[0019] Furthermore, the management module determines based on the transmission rate evaluation value that the reason why the transmission does not meet the preset standard is that the subpacketization of the desktop image data does not meet the standard and reduces the subpacketization length of the data subpacketization, or the compression of the desktop image data does not meet the standard and increases the compression ratio of the data compression.

[0020] Furthermore, the management module is provided with several adjustment methods for reducing the sub-packet length, and each adjustment method has a different adjustment range for the sub-packet length.

[0021] Furthermore, when the management module determines that the operation of the system does not meet the preset standard based on the packet loss rate of data packet transmission within the first preset time, it re-determines the level of system operation according to the proportion of disordered data after data packet analysis, or marks the system operation as level one operation.

[0022] Furthermore, the management module secondarily determines whether the system operation level is primary operation or secondary operation based on the proportion of disordered data after the data packet is parsed.

[0023] Furthermore, the management module determines that the reason why the system operation does not meet the preset standards is that the adjustment of the subpacket length does not meet the standards and corrects the subpacket length of the data subpacket based on the frequency and proportion of the operation of each level of systems within the second preset time period, or the first processing module is loaded and the capacity of the first processing module is expanded.

[0024] Compared with the prior art, the beneficial effect of the present invention lies in that: the present invention improves the stability and smooth operation of the remote desktop with high frame rate and high resolution by setting an acquisition module for acquiring remote desktop image data, a first processing module for processing the acquired desktop image data, a transmission module for transmitting data packets, a receiving module for receiving the transmitted data packets, a second processing module for completing data packet parsing, remote desktop image processing and desktop image push, a monitoring module for monitoring system operation information, and a management module for preliminarily determining whether the transmission of the data packet meets the preset standard based on the response time and determining whether the operation of the system meets the preset standard based on the packet loss rate of the data packet transmission within the preset time.

[0025] Furthermore, when the present invention determines that the transmission of a data packet does not meet the preset standard based on the response time, the reason why the transmission does not meet the preset standard is determined according to the transmission rate evaluation value, or the bandwidth of each of the transmission channels is expanded. The response time represents the delay in establishing the remote desktop control. Therefore, the data transmission rate of the established remote desktop can be accurately evaluated.

[0026] Furthermore, the present invention optimizes the data transmission rate by expanding the bandwidth of the transmission channel, and the present invention also accurately adjusts the expansion amplitude of the bandwidth according to the response time difference.

[0027] Furthermore, the present invention constructs a transmission rate evaluation value based on the transmission rate of the transmission module, thereby serving as a benchmark for evaluating transmission and accurately determining the reason why the transmission does not meet the preset standard.

[0028] Furthermore, the present invention provides several adjustment methods for the subpacket length, so as to accurately adjust the length of the data subpacket and optimize data transmission.

[0029] Furthermore, the present invention determines the operating status of the system within a period of time based on the packet loss rate and the proportion of disordered data. When the operation does not meet the standards, the operation level is marked. At the same time, the reason why the system does not operate up to the standards is accurately determined through the frequency and proportion of the operation level, thereby dynamically optimizing the operating parameters of the system and ensuring the stability of the remote desktop and smooth operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A structural block diagram of a high frame rate and high resolution remote desktop construction system according to an embodiment of the present invention;

[0031] Figure 2 A flowchart of an embodiment of the present invention for determining whether the transmission of a data packet meets a preset standard;

[0032] Figure 3 A flow chart of adjusting the sub-packet length according to an embodiment of the present invention;

[0033] Figure 4 The flowchart of the embodiment of the present invention is used to determine whether the operation of the system meets the preset standard. DETAILED DESCRIPTION

[0034] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] It should be pointed out that the data in this embodiment are obtained by comprehensive analysis and evaluation of the historical detection data of the three months before this operation and the corresponding historical operation results of the management module of the present invention. It can be understood by those skilled in the art that the determination method of the system of the present invention for the above-mentioned single parameter can be to select the value with the highest proportion as the preset standard parameter according to the data distribution, use weighted summation to use the obtained value as the preset standard parameter, substitute each historical data into a specific formula and use the value obtained by the formula as the preset standard parameter or other selection methods, as long as the system of the present invention can clearly define the different specific situations in the single determination process through the obtained values.

[0036] It should be pointed out that the high frame rate mentioned in the present invention refers to displaying content at a speed of 48 frames per second or above, and the high resolution mentioned in the present invention refers to a resolution of 3840x2160 pixels or above.

[0037] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0038] See also Figure 1, Figure 2 , Figure 3 as well as Figure 4 As shown, they are respectively a structural block diagram of a high frame rate and high resolution remote desktop construction system according to an embodiment of the present invention, a flowchart of determining whether the transmission of a data packet meets the preset standard according to an embodiment of the present invention, a flowchart of adjusting the subpacket length according to an embodiment of the present invention, and a flowchart of determining whether the operation of the system meets the preset standard according to an embodiment of the present invention.

[0039] The embodiment of the present invention provides a high frame rate and high resolution remote desktop construction system, including:

[0040] A collection module is provided at a remote end to collect desktop image data at the remote end;

[0041] A first processing module, which is arranged at the remote end and connected to the acquisition module, is used to encode, packetize and compress the collected desktop image data to obtain a data packet to be transmitted;

[0042] A transmission module, which is arranged at the remote end and connected to the first processing module, and includes a plurality of transmission channels for transmitting data packets;

[0043] A receiving module, which is arranged at a local end and connected to the transmission module, for receiving the transmitted data packet;

[0044] A second processing module, which is arranged at the local end and connected to the receiving module, is used to successively complete the parsing of the data packet, the processing of the remote end desktop image and the pushing of the desktop image, wherein the image processing includes the rendering and scaling of the remote end desktop image;

[0045] A monitoring module, which is arranged at the local end and is respectively connected to the acquisition module, the first processing module, the transmission module and the second processing module, for detecting and obtaining the response time, the transmission rate of the transmission channel, the data packet size, the packet loss rate and the proportion of out-of-order data;

[0046] A management module is arranged at the local end and is respectively connected to the monitoring module, the acquisition module, the first processing module, the transmission module, the receiving module and the second processing module, for preliminarily determining whether the transmission of the data packet meets the preset standard according to the response time, and determining whether the operation of the system meets the preset standard according to the packet loss rate of the data packet transmission within the preset time.

[0047] Specifically, the acquisition module acquires data from the remote desktop through DXGI. It can be understood that what is acquired is essentially the video memory data corresponding to the desktop. It can be understood that DXGI refers to DirectX Graphics Infrastructure, which is a graphics device interface provided by Microsoft.

[0048] Specifically, the transmission module sends the data to the receiving module at the local end through the UDP socket. It can be understood that the UDP socket is a network communication protocol, the full name of which is User Datagram Protocol Socket, which is used to transmit data packets in the network.

[0049] Specifically, the management module preliminarily determines whether the transmission of the data packet meets the preset standard based on the response time, wherein:

[0050] If the response time is less than the first preset response time of 8ms, the management module determines that the transmission of the data packet meets the preset standard, and determines whether the operation of the system meets the preset standard according to the packet loss rate of the data packet transmission within the preset time;

[0051] If the response time is greater than or equal to the first preset response time and less than the second preset response time of 35 ms, the management module determines that the transmission of the data packet does not meet the preset standard, and determines the reason why the transmission does not meet the preset standard according to the transmission rate evaluation value;

[0052] If the response time is greater than or equal to the second preset response time, the management module determines that the transmission of the data packet does not meet the preset standard and expands the bandwidth of each transmission channel.

[0053] Specifically, the response duration is the duration between the time when the first processing module receives the remote desktop image data and the time when the second processing module completes the processing of the remote desktop image.

[0054] Specifically, the expansion amplitude of the bandwidth of each transmission channel is positively correlated with the response time difference, and the response time difference is the difference between the response time and the second preset response time. It can be understood that the larger the response time difference is, the larger the expansion amplitude of the bandwidth of each transmission channel is.

[0055] Specifically, the transmission rate evaluation value is set to S, and S is obtained by formula (1):

[0056]

[0057] In formula (1), μ is the evaluation coefficient, set μ = 0.68, Li is the average transmission rate of the i-th transmission channel within the preset time length, i = 1, 2...n, n is the total number of transmission channels, L0 is the average rate threshold, set L0 = 6.90 MB / s.

[0058] Specifically, the management module determines the reason why the transmission does not meet the preset standard based on the transmission rate evaluation value, wherein:

[0059] If the transmission rate evaluation value is less than the preset transmission rate evaluation value of 0.85, the management module determines that the reason why the transmission does not meet the preset standard is that the subpacketization of the desktop image data does not meet the standard and reduces the subpacketization length of the data subpacketization;

[0060] If the transmission rate evaluation value is greater than or equal to a preset transmission rate evaluation value of 0.85, the management module determines that the reason why the transmission does not meet the preset standard is that the compression of the desktop image data does not meet the standard and increases the compression ratio of the data compression.

[0061] Specifically, the management module is provided with several adjustment methods for the sub-packet length, wherein:

[0062] The first adjustment method is that the management module uses the first preset adjustment coefficient 0.998 to determine the sub-packet length of the desktop image data in the next period in the form of a product; the first adjustment method is that the evaluation difference is less than the first preset evaluation difference 0.06;

[0063] The second adjustment method is that the management module uses the second preset adjustment coefficient 0.994 to determine the sub-packet length of the desktop image data in the next period in the form of a product; the second adjustment method is that the evaluation difference is greater than or equal to the first preset evaluation difference and less than the second preset evaluation difference 0.17;

[0064] The third adjustment method is that the management module uses the third preset adjustment coefficient 0.987 to determine the sub-packet length of the desktop image data in the next period in the form of a product; the third adjustment method is that the evaluation difference is greater than or equal to the second preset evaluation difference;

[0065] The evaluation difference value is a difference between the preset transmission rate evaluation value and the transmission rate evaluation value.

[0066] Specifically, the management module determines whether the operation of the system meets the preset standard based on the packet loss rate of the data packet transmission within the first preset time length of 300s, wherein:

[0067] If the packet loss rate is less than a first preset packet loss rate of 0.05%, the management module determines that the operation of the system meets the preset standard and completes the construction of the remote desktop according to the current parameters;

[0068] If the packet loss rate is greater than or equal to the first preset packet loss rate and less than the second preset packet loss rate of 0.14%, the management module determines that the operation of the system does not meet the preset standard, and re-determines the level of system operation within the time period according to the proportion of disordered data after data packet analysis;

[0069] If the packet loss rate is greater than or equal to the second preset packet loss rate, the management module determines that the operation of the system does not meet the preset standard, and marks the operation of the system within the time period as level one operation.

[0070] Specifically, the out-of-order data ratio is a ratio obtained by dividing the size of the out-of-order data by the size of the transmitted data.

[0071] Specifically, the management module secondary determines the system operation level as primary operation or secondary operation based on the proportion of out-of-order data after data packet analysis, wherein, under the condition that the proportion of out-of-order data is less than a preset proportion of 0.02%, the system operation level is secondary operation, and, under the condition that the proportion of out-of-order data is greater than or equal to the preset proportion, the system operation level is secondary operation.

[0072] Specifically, the management module determines the reason why the system operation does not meet the preset standard based on the frequency and proportion of the operation of each level of the system within the second preset time length of 1 hour, wherein:

[0073] If the frequency is greater than the preset frequency of 22 times / h and the first-level operation ratio is greater than the preset level ratio threshold of 0.54%, the management module determines that the reason why the system operation does not meet the preset standard is that the adjustment of the subpacket length does not meet the standard and corrects the subpacket length of the data subpacket;

[0074] If the frequency is greater than the preset frequency and the secondary operation ratio is greater than the preset level ratio threshold, the management module determines that the reason why the system operation does not meet the preset standard is the load of the first processing module and expands the capacity of the first processing module;

[0075] The frequency is the number of times the system's operation is marked as level one operation within the second preset time period;

[0076] The first-level operation proportion is the proportion of the first-level operation time to the second preset time, and the second-level operation proportion is the proportion of the second-level operation time to the second preset time.

[0077] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0078] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A high frame rate and high resolution remote desktop construction system, characterized in that: include: A collection module is provided at a remote end to collect desktop image data at the remote end; A first processing module, which is arranged at the remote end and connected to the acquisition module, is used to encode, packetize and compress the collected desktop image data to obtain a data packet to be transmitted; A transmission module, which is arranged at the remote end and connected to the first processing module, and includes a plurality of transmission channels for transmitting data packets; A receiving module, which is arranged at a local end and connected to the transmission module, for receiving the transmitted data packet; A second processing module, which is arranged at the local end and connected to the receiving module, is used to successively complete the parsing of the data packet, the processing of the remote end desktop image and the pushing of the desktop image, wherein the image processing includes the rendering and scaling of the remote end desktop image; A monitoring module, which is arranged at the local end and is respectively connected to the acquisition module, the first processing module, the transmission module and the second processing module, for detecting and obtaining the response time, the transmission rate of the transmission channel, the data packet size, the packet loss rate and the proportion of out-of-order data; A management module, which is arranged at the local end and is respectively connected to the monitoring module, the acquisition module, the first processing module, the transmission module, the receiving module and the second processing module, and is used to preliminarily determine whether the transmission of the data packet meets the preset standard according to the response time, and to determine whether the operation of the system meets the preset standard according to the packet loss rate of the data packet transmission within the preset time; When the management module preliminarily determines that the transmission of the data packet does not meet the preset standard based on the response time, the management module determines the reason why the transmission does not meet the preset standard according to the transmission rate evaluation value, or expands the bandwidth of each transmission channel; The transmission rate evaluation value is set to S, which is obtained by formula (1): In formula (1), μ is the evaluation coefficient, μ=0.68 is set, Li is the average transmission rate of the i-th transmission channel within the preset time length, i=1,2...n, n is the total number of transmission channels, and L0 is the average rate threshold; The management module determines based on the transmission rate evaluation value that the reason why the transmission does not meet the preset standard is that the subpackaging of the desktop image data does not meet the standard and reduces the subpackaging length of the data subpackaging, or the compression of the desktop image data does not meet the standard and increases the compression ratio of the data compression.

2. A high frame rate and high resolution remote desktop construction system according to claim 1, characterized in that: The response duration is the duration between the time when the first processing module receives the remote desktop image data and the time when the second processing module completes the processing of the remote desktop image.

3. A high frame rate and high resolution remote desktop construction system according to claim 2, characterized in that: The expansion amplitude of the bandwidth of each transmission channel is positively correlated with the response time difference, and the response time difference is the difference between the response time and a second preset response time.

4. A high frame rate and high resolution remote desktop construction system according to claim 1, characterized in that: The management module is provided with several adjustment methods for reducing the sub-packet length, and each adjustment method has a different adjustment range for the sub-packet length.

5. A high frame rate and high resolution remote desktop construction system according to claim 4, characterized in that: When the management module determines that the operation of the system does not meet the preset standard based on the packet loss rate of data packet transmission within the first preset time, it re-determines the level of system operation according to the proportion of disordered data after data packet analysis, or marks the system operation as level one operation.

6. A high frame rate and high resolution remote desktop construction system according to claim 5, characterized in that: The management module secondarily determines whether the system operation level is primary operation or secondary operation based on the proportion of disordered data after the data packet is parsed.

7. A high frame rate and high resolution remote desktop construction system according to claim 6, characterized in that: The management module determines that the system operation does not meet the preset standards based on the frequency and proportion of each level of system operation within the second preset time period, and the reason is that the adjustment of the subpacket length does not meet the standard and corrects the subpacket length of the data subpacket, or the first processing module is loaded and the capacity of the first processing module is expanded.

Images