Remote desktop data transmission method and device, electronic equipment and program product

Abstract

The application provides a remote desktop data transmission method and device, electronic equipment and program product, which can be used in the technical field of data transmission. The method comprises the following steps: in the case that there is a data transmission demand in a remote desktop session, acquiring a service data type corresponding to the to-be-transmitted data; if the service data type is a streaming media type, transmitting the to-be-transmitted data to a peer end based on a first user datagram protocol (UDP) transmission channel. According to the service data type corresponding to the to-be-transmitted data, the transmission channel is dynamically selected, in particular, the first UDP transmission channel is selected for transmission when the service data type is the streaming media type. Compared with the transmission control protocol (TCP) protocol, the UDP protocol has lower transmission delay, and the transmission efficiency of the streaming media data can be significantly improved.

Description

Technical Field

[0001] This application relates to the field of data transmission technology, and in particular to a remote desktop data transmission method, apparatus, electronic device, and program product. Background Technology

[0002] Remote desktop allows users to remotely access and control another computer from a client computer, enabling them to use its desktop environment, applications, and files for remote work, technical support, or management. It transmits data through specific protocols, ensuring real-time interaction and security. With the development of remote desktop technology, improving data transmission efficiency has become a pressing issue. Summary of the Invention

[0003] This application provides a remote desktop data transmission method, apparatus, electronic device, and program product to improve data transmission efficiency.

[0004] Firstly, this application provides a remote desktop data transmission method, including:

[0005] When there is a data transmission requirement in a remote desktop session, obtain the business data type corresponding to the data to be transmitted;

[0006] If the data type of the service is streaming media, the data to be transmitted is transmitted to the other end based on the first User Datagram Protocol (UDP) transmission channel.

[0007] In some embodiments, if the business data type is a streaming media type, it further includes:

[0008] If the data type of the service is the streaming media type, then control information related to the data to be transmitted is transmitted to the peer through the second UDP transmission channel; the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

[0009] In some embodiments, transmitting the data to be transmitted to the peer based on the first User Datagram Protocol (UDP) transport channel includes:

[0010] The data to be transmitted is encapsulated into a first data packet in Real-Time Transport Protocol (RTP) format; the first data packet contains streaming media data.

[0011] The first data packet is sent to the peer through the first UDP transmission channel;

[0012] The transmission of control information related to the data to be transmitted to the peer via the second UDP transmission channel includes:

[0013] Generate a second data packet in Real-Time Transport Control Protocol (RTCP) format; the second data packet contains control information related to the streaming media data;

[0014] The second data packet is encapsulated into a third data packet in UDP format and sent to the peer through the second UDP transmission channel.

[0015] In some embodiments, the business data type is one of a first real-time type or a second real-time type, wherein the first real-time type has higher real-time requirements than the second real-time type; further comprising:

[0016] If the data type of the service is the second real-time type, then the data to be transmitted is encapsulated into a fourth data packet in UDP format;

[0017] The fourth data packet is sent to the peer through the second UDP transmission channel; the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

[0018] In some embodiments, the business data type is one of a first real-time type or a second real-time type, wherein the first real-time type has higher real-time requirements than the second real-time type; further comprising:

[0019] If the data type of the service is the first real-time type, then the data to be transmitted is encapsulated into a fifth data packet in UDP format and the data to be transmitted is encapsulated into a sixth data packet in Transmission Control Protocol (TCP) format.

[0020] The fifth data packet is sent to the peer through the third UDP transmission channel, and the sixth data packet is sent to the peer through the TCP transmission channel; the reliability of the third UDP transmission channel is lower than that of the first UDP transmission channel.

[0021] In some embodiments, before obtaining the business data type corresponding to the data to be transmitted when there is a data transmission requirement in the remote desktop session, the method further includes:

[0022] When the remote desktop session is established, a UDP connection and a TCP connection are established with the peer respectively;

[0023] Based on the UDP connection, a first UDP transmission channel and a second UDP transmission channel are established respectively, and the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

[0024] In some embodiments, after transmitting the data to be transmitted to the peer based on the first User Datagram Protocol (UDP) transport channel, the method further includes:

[0025] If it is determined that data loss exists and the amount of lost data exceeds the data loss threshold corresponding to the first UDP transmission channel, then the data to be transmitted is retransmitted to the peer according to the number of retransmissions corresponding to the first UDP transmission channel.

[0026] Secondly, this application provides a remote desktop data transmission device, comprising:

[0027] The acquisition module is used to acquire the business data type corresponding to the data to be transmitted when there is a data transmission requirement in the remote desktop session;

[0028] The transmission module is used to transmit the data to be transmitted to the peer based on the first User Datagram Protocol (UDP) transmission channel if the data type of the service is streaming media.

[0029] Thirdly, this application provides an electronic device, including: a processor and a memory communicatively connected to the processor;

[0030] The memory stores computer-executed instructions;

[0031] The processor executes computer execution instructions stored in the memory to implement the remote desktop data transmission method as described in any of the first aspects.

[0032] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the remote desktop data transmission method described in any of the first aspects.

[0033] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the remote desktop data transmission method described in any of the first aspects.

[0034] The remote desktop data transmission method, apparatus, electronic device, and program products provided in this application dynamically select the transmission channel according to the business data type corresponding to the data to be transmitted. In particular, for streaming media types, the first UDP transmission channel is selected for transmission. Compared with the TCP protocol, the UDP protocol has lower transmission latency and can significantly improve the transmission efficiency of streaming media data. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0036] Figure 1 This is a schematic diagram of an application scenario provided in this application;

[0037] Figure 2 This is a flowchart illustrating a remote desktop data transmission method provided in this application;

[0038] Figure 3 This is a flowchart illustrating another remote desktop data transmission method provided in this application;

[0039] Figure 4 This is a schematic diagram of the structure of a remote desktop data transmission device provided in this application;

[0040] Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application.

[0041] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0042] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0043] Remote desktop allows users to remotely access and control another computer from a client computer, enabling them to use its desktop environment, applications, and files for remote work, technical support, or management. It transmits data through specific protocols, ensuring real-time interaction and security.

[0044] With the rapid development of cloud computing and virtualization technologies, remote desktop technology has become a key infrastructure connecting the internal and external worlds of enterprises, driving educational innovation, and enriching entertainment experiences. In enterprise environments, employees crave flexible work and seamless collaboration; in the field of distance education, there is an urgent need for instant knowledge transfer and interaction between teachers and students; and in gaming, players seek rapid response and immersive experiences. These diverse application scenarios have placed unprecedentedly high standards on the real-time performance, interactivity, and stability of remote desktop technology.

[0045] However, existing remote desktop technologies encounter significant technical bottlenecks when facing high-latency and unstable network environments. On the one hand, the traditional Transport Control Protocol (TCP) is known for its strong data reliability, ensuring the complete and error-free transmission of data. However, its complex connection establishment process and strict flow control mechanism often lead to significant transmission delays, making it difficult to meet the needs of applications with high real-time requirements. On the other hand, the User Datagram Protocol (UDP) is known for its low-latency characteristics, making it particularly suitable for scenarios with extremely high real-time requirements. However, it lacks mechanisms to guarantee data integrity and order, easily leading to data loss and disordered delivery, thus severely impacting the user experience.

[0046] Therefore, finding an effective balance between TCP and UDP to ensure reliable data transmission while achieving an extremely low-latency experience has become a key challenge for the development of remote desktop technology. This challenge urgently requires innovative solutions to adapt to the ever-changing network environment and user needs.

[0047] One approach in related technologies is to select either a TCP or UDP connection for transmission based on network conditions. Another approach is to establish multiple network connections, then divide the streaming media into multiple segments, and select the optimal target network connection for each segment based on the network conditions of each candidate network connection and the dependency of each segment. Yet another approach is to establish both UDP and TCP connections simultaneously, using UDP to encapsulate audio and video data with fixed-length headers, and transmitting audio and video data packets at an adaptive data transmission rate. Small data retransmissions and control information such as heartbeats are transmitted via TCP.

[0048] Based on this, this application proposes a remote desktop data transmission method, apparatus, electronic device, and program product, aiming to improve data transmission efficiency.

[0049] Figure 1 This is a schematic diagram illustrating one application scenario provided in this application. For example... Figure 1 As shown, the application scenario includes a first device 10 and a second device 20, and the first device 10 and the second device 20 are connected via a network.

[0050] Both the first device 10 and the second device 20 can be terminal devices or servers. For example, a terminal device can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these. For example, a server can be a standalone physical server, or a virtual machine running on a physical server using virtualization technology.

[0051] In one scenario, a first device 10 establishes a remote desktop session with a second device 20, and the local device transmits data to the peer device by executing the remote desktop data transmission method provided in this application. Both the first device 10 and the second device 20 can act as local devices to transmit data to the peer device. For example, if the first device 10 is the local device, then the second device 20 is the peer device; conversely, if the second device 20 is the local device, then the first device 10 is the peer device.

[0052] The remote desktop data transmission method provided in this application is executed by a remote desktop data transmission device, which is integrated into an electronic device, which can be the first device 10 or the second device 20 mentioned above.

[0053] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0054] Figure 2 This is a flowchart illustrating a remote desktop data transmission method provided in this application. Figure 2 As shown, the remote desktop data transmission method provided in this embodiment includes the following steps:

[0055] Step S101: If there is a data transmission requirement in the remote desktop session, obtain the business data type corresponding to the data to be transmitted.

[0056] In this embodiment, the local device establishes a remote desktop session with the remote device, thereby enabling data transmission within the remote desktop session. A remote desktop session refers to the process of accessing and controlling the desktop environment of one device (which can be considered a client) through a network connection. For example, the client device sends input (such as mouse and keyboard operations) commands to the server device, and the server device transmits desktop images and audio streams back to the client device, allowing the client device to display desktop content and enabling the user to remotely control the server device.

[0057] This embodiment is described from the perspective of sending data from the local device to the remote device. The process of sending data from the remote device to the local device is similar, and will not be described again in this application.

[0058] This application pre-classifies various business data types based on actual business needs: streaming media type, first real-time type, and second real-time type. When data transmission is required in a remote desktop session, the business data type corresponding to the data to be transmitted is first obtained, and then the data is transmitted to the other end based on that business data type.

[0059] The streaming media data can be audio stream data, video stream data, or audio-visual stream data; this application does not limit this.

[0060] The first real-time type has higher requirements for transmission real-time performance than the second real-time type. For ease of understanding, data of the first real-time type can be data with high real-time requirements, while data of the second real-time type can be data with some real-time requirements but lower requirements compared to the first real-time type. For example, data of the first real-time type can be keyboard commands, mouse commands, or control commands; data of the second real-time type can be clipboard data, files to be copied, or disk mapping data. For example, control commands can be system operation commands, application management commands, file management commands, system setting commands, or network configuration commands. For instance, system operation commands can be commands to restart the computer, shut down the computer, or log off the user; application management commands can be commands to start or close applications; file management commands can be commands to create, delete, or move files / folders, or copy or paste files / folders; system setting commands can be commands to change system settings or update the system or software; and network configuration commands can be commands to configure network connections, connect to or disconnect from the network.

[0061] Optionally, one way to obtain the business data type corresponding to the data to be transmitted is to mark the data during the data generation or preparation stage so that the data carries a type identifier. This type identifier is used to indicate the business data type corresponding to the data, so that the type identifier carried by the data to be transmitted can be directly read during transmission to determine its corresponding business data type. For example, audio and video data can be appended with a streaming media type identifier during encoding, while keyboard commands or mouse commands can be appended with a first real-time type identifier, and clipboard data can be appended with a second real-time type identifier. Optionally, the representation of the type identifier can be set according to actual needs, and this application does not limit it. For example, the streaming media type identifier is 1, the first real-time type identifier is 2, and the second real-time type identifier is 3.

[0062] Step S102: If the business data type is streaming media, then transmit the data to be transmitted to the other end based on the first UDP transmission channel.

[0063] This application pre-sets a corresponding preset transmission strategy for each type of business data, so that after obtaining the business data type corresponding to the data to be transmitted, it can transmit the data to be transmitted to the other end based on the preset transmission strategy corresponding to the business data type.

[0064] When the data to be transmitted corresponds to the streaming media data type, the data to be transmitted is streaming media data. Even if a small amount of data is lost during the transmission of streaming media data, it can be supplemented by the decoding algorithm during the decoding process at the other end. Therefore, a small amount of data loss has little impact on the transmission of streaming media data. Thus, streaming media data can be transmitted using the first UDP transmission channel, which reduces the burden of retransmission and ensures the low latency requirement of streaming media data.

[0065] In this embodiment, the transmission channel is dynamically selected based on the data type of the business data to be transmitted. In particular, for streaming media types, the first UDP transmission channel is selected for transmission. Compared with the TCP protocol, the UDP protocol has lower transmission latency, which can significantly improve the transmission efficiency of streaming media data.

[0066] In some embodiments, before step S101, the remote desktop data transmission method provided in this application further includes: establishing a UDP connection and a TCP connection with the peer respectively when the remote desktop session is established; establishing a first UDP transmission channel and a second UDP transmission channel respectively based on the UDP connection, wherein the reliability of the first UDP transmission channel is lower than the reliability of the second UDP transmission channel.

[0067] Optionally, both UDP and TCP connections can be established simultaneously via Interactive Connectivity Establishment (ICE). In this embodiment, UDP and TCP connections are established separately at the start of the remote desktop session, so that data can be transmitted using the established UDP and TCP connections when data transmission is required during the subsequent remote desktop session.

[0068] The first and second UDP transmission channels are encapsulated on top of a UDP connection, and therefore need to be established separately. For example, the first UDP transmission channel is encapsulated on top of a UDP connection based on Web Real-Time Communications (WebRTC), and the second UDP transmission channel is encapsulated on top of a UDP connection based on Quick UDP Internet Connections (QUIC).

[0069] Reliability refers to the ability to ensure data integrity and accuracy during data transmission. The first UDP transmission channel can be considered a semi-reliable UDP transmission channel, while the second UDP transmission channel can be considered a reliable UDP transmission channel.

[0070] Understandably, the second UDP transmission channel has higher requirements for data integrity, meaning that all data must reach the other end. If some data fails to arrive, multiple retransmissions are required. Optionally, the number of retransmissions for the second UDP transmission channel can be preset. Once data loss is confirmed, data will be retransmitted to the other end according to this preset number of retransmissions.

[0071] Understandably, the first UDP transmission channel has relatively low requirements for data integrity during transmission, meaning it allows for a small amount of data loss. Optionally, a threshold for the amount of data lost and the number of retransmissions can be preset for the first UDP transmission channel. If data loss is determined to exist and the amount of lost data exceeds the threshold, the data is retransmitted to the other end according to the specified number of retransmissions. Conversely, if the amount of lost data does not exceed the threshold, no retransmission is required.

[0072] Optionally, after receiving the data, the peer can determine whether the data is lost and send feedback information to the local end. This feedback information indicates whether data loss exists and the amount of data containing the lost data. Accordingly, after receiving the feedback information, if the feedback information indicates that data loss exists, the local end determines that data loss exists and extracts the amount of lost data from the feedback information; otherwise, if the feedback information indicates that no data loss exists, the local end determines that no data loss exists.

[0073] This embodiment improves transmission efficiency and resource utilization while ensuring data transmission quality through intelligent judgment and selective retransmission in the event of data loss. By setting a threshold for the amount of lost data, unnecessary retransmissions can be avoided when the amount of data loss is small and has little impact on the overall transmission, thereby saving bandwidth and computing resources. For data loss exceeding the threshold, retransmission ensures the integrity and reliability of important data, preventing transmission errors or data inconsistencies caused by data loss. Reducing the number of unnecessary retransmissions without affecting data integrity can improve overall transmission efficiency and reduce transmission latency.

[0074] Optionally, this application also provides a third UDP transmission channel, the reliability of which is lower than that of the first UDP transmission channel. The third UDP transmission channel refers to the transmission channel formed by the original UDP connection, and therefore does not require additional establishment.

[0075] Understandably, the third UDP transmission channel has lower requirements for data integrity when transmitting data and cannot guarantee whether data is lost. In other words, the third UDP transmission channel does not need to retransmit data.

[0076] From a real-time perspective, the first UDP transmission channel has higher real-time performance than the second UDP transmission channel, and the third UDP transmission channel has higher real-time performance than the first UDP transmission channel.

[0077] The data transmission process is described in detail below for each type of business data.

[0078] In some embodiments, when the data to be transmitted corresponds to a streaming media type, when transmitting the data to be transmitted to the peer via the first UDP transmission channel, the method further includes: transmitting control information related to the data to be transmitted to the peer via the second UDP transmission channel; the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

[0079] In practical applications, when transmitting streaming media data to the other end, related control information is also transmitted. The purpose of this control information is for the other end to determine whether streaming media data has been lost upon receipt and to provide feedback to the local end, allowing the local end to decide whether data retransmission is necessary. Control information related to streaming media data requires high reliability, and while the feedback information is small in size, it is crucial. Therefore, a second UDP transmission channel can be used to transmit both control and feedback information, ensuring reliability. This guarantees that neither control nor feedback information is lost during transmission, enabling the local end to optimize its transmission strategy based on the feedback information, thereby improving overall transmission efficiency.

[0080] As can be seen, this embodiment optimizes data transmission and improves reliability by using two independent UDP transmission channels to transmit streaming media data and related control information separately. Furthermore, the use of independent channels for streaming media data and control information avoids delays or blockages in streaming media data transmission caused by the transmission of control information.

[0081] Optionally, the data to be transmitted is transmitted to the peer via a first User Datagram Protocol (UDP) transmission channel, and control information related to the data to be transmitted is transmitted to the peer via a second UDP transmission channel. Specifically, this is implemented as follows:

[0082] The data to be transmitted is encapsulated into a first data packet in Real-time Transport Protocol (RTP) format, and a second data packet in Real-time Transport Control Protocol (RTCP) format is generated simultaneously. The first data packet is sent to the peer through the first UDP transmission channel. The second data packet is encapsulated into a third data packet in UDP format, and the third data packet is sent to the peer through the second UDP transmission channel.

[0083] The first data packet contains streaming media data, and the second data packet contains control information related to the streaming media data. The RTP protocol is designed for real-time data transmission, helping the receiving end reconstruct the data stream sequentially. Therefore, encapsulating the data to be transmitted into the first data packet in RTP format facilitates the transmission of streaming media data. The RTCP protocol is used to transmit control information related to the RTP stream. The UDP protocol itself has low overhead and high transmission efficiency, making it suitable for real-time data transmission. By further encapsulating the second data packet in RTCP format within a UDP packet, the advantages of UDP can be fully utilized to ensure the rapid transmission of control information.

[0084] In other embodiments, when the data to be transmitted corresponds to a streaming media type, the data to be transmitted and control information can also be transmitted through the first UDP transmission channel to further improve real-time performance, reduce latency, and ensure user experience. Optionally, the data to be transmitted is encapsulated into a first data packet in RTP format, and a second data packet in RTCP format is generated simultaneously; the first data packet and the second data packet are sent to the peer through the first UDP transmission channel.

[0085] In some embodiments, when the data type corresponding to the data to be transmitted is a first real-time type, the first real-time type has high real-time requirements, and the UDP protocol has high real-time performance; therefore, the data to be transmitted can be transmitted via the UDP protocol. Optionally, the method provided in this application further includes: if the data type corresponding to the data to be transmitted is a first real-time type, then encapsulating the data to be transmitted into a fifth data packet in UDP format; and sending the fifth data packet to the other end through a third UDP transmission channel. The third UDP transmission channel has higher real-time performance than the first UDP transmission channel and the second UDP transmission channel, thus better ensuring real-time performance.

[0086] Optionally, the reliability of the third UDP transmission channel is lower than that of the first and second UDP transmission channels. If the data type corresponding to the data to be transmitted is a first real-time type, and the data to be transmitted is of high importance, the data to be transmitted can be further transmitted in parallel via the TCP protocol to ensure reliability. Accordingly, the method provided in this application further includes: if the data type corresponding to the data to be transmitted is a first real-time type, then encapsulating the data to be transmitted into a fifth data packet in UDP format and encapsulating the data to be transmitted into a sixth data packet in TCP format; sending the fifth data packet to the peer through the third UDP transmission channel and sending the sixth data packet to the peer through the TCP transmission channel.

[0087] The above implementation transmits the first real-time data type through a third UDP transmission channel, reducing latency and improving the immediacy of interaction. Transmitting the first real-time data type through a TCP transmission channel ensures no data loss, and in some scenarios, TCP packets arrive at the other end faster than UDP packets. Parallel transmission reduces transmission latency and improves user experience.

[0088] In practical applications, when the data type corresponding to the data to be transmitted is of the first real-time type, if the data volume of the data to be transmitted is less than or equal to a preset data volume threshold, it indicates that the data volume is small. In this case, the transmission rate through the parallel transmission of the third UDP transmission channel and the TCP transmission channel is faster, and the transmission effect is less affected by network quality, resulting in better transmission performance. If the data volume of the data to be transmitted is greater than the preset data volume threshold, it indicates that the data volume is large. In this case, the transmission rate through the parallel transmission of the third UDP transmission channel and the TCP transmission channel is slower compared to transmitting small data volumes, and the transmission effect is more easily affected by network quality. Therefore, in this situation, the second UDP transmission channel can also be used to transmit the data to be transmitted. Optionally, the method provided in this application further includes: if the data type corresponding to the data to be transmitted is of the first real-time type, then the data to be transmitted is encapsulated into a seventh data packet in UDP format and sent to the other end through the second UDP transmission channel. Since the UDP protocol has high real-time performance, and the reliability of the second UDP transmission channel is higher than that of the first and third UDP transmission channels, this implementation method ensures both reliability and real-time performance. The preset data volume threshold can be set according to actual needs, and this application does not impose any restrictions on it.

[0089] In some embodiments, when the data to be transmitted corresponds to a second real-time type, such as clipboard data, files to be copied, or disk mapping data, this type of data has certain real-time requirements, a large data volume, and needs to ensure integrity and order. In this case, if the TCP protocol is used for transmission, it is very likely that the real-time requirements cannot be met. Therefore, the UDP protocol can be used for transmission to ensure real-time performance.

[0090] Optionally, the method provided in this application further includes: if the data type corresponding to the data to be transmitted is a second real-time type, then encapsulating the data to be transmitted into a fourth data packet in UDP format; and sending the fourth data packet to the other end through the second UDP transmission channel.

[0091] The above implementation method utilizes the reliability and real-time performance of the second UDP transmission channel, which can ensure the integrity and order of the data to be transmitted, and also has a fast transmission efficiency.

[0092] This application categorizes the data to be transmitted into various business data types based on actual business needs and configures corresponding transmission strategies for each type. For data to be transmitted, the corresponding business data type is first obtained, then the appropriate transmission strategy is selected based on that data type, and finally, the data is transmitted to the other end according to the transmission strategy. This achieves differentiated data processing and improves data transmission efficiency. This method dynamically selects the transmission strategy based on the business data type, providing greater flexibility and adaptability, thus achieving a balance between efficiency and reliability.

[0093] Figure 3 This is a flowchart illustrating another remote desktop data transmission method provided in this application. Based on any of the above embodiments, this embodiment relates to a remote desktop data transmission process, such as... Figure 3 As shown, the process includes: simultaneously establishing UDP and TCP connections via ICE, and establishing a semi-reliable UDP transmission channel and a reliable UDP transmission channel based on the UDP connection; determining the data type of the service corresponding to the data to be transmitted; specifically, it involves the following three cases:

[0094] In the first scenario, if the data to be transmitted is streaming media data (such as audio and video data), then RTP data packets are first encapsulated, and RTCP data packets are generated simultaneously. The RTP data packets are transmitted directly through a semi-reliable UDP transmission channel, while the RTCP data packets are first encapsulated into reliable UDP data packets and then transmitted through a reliable UDP transmission channel.

[0095] In the second scenario, if the data to be transmitted requires ensuring integrity, order, and a certain level of real-time performance, such as clipboard data, files to be copied, or disk mapping data, then it should be encapsulated into reliable UDP packets before being transmitted through a reliable UDP transmission channel.

[0096] In the third scenario, if the data to be transmitted is data with high real-time requirements and small data volume, such as keyboard commands, mouse commands, and control signaling, then UDP data packets and TCP data packets are first encapsulated separately, and then transmitted in parallel through unreliable UDP transmission channels and TCP transmission channels respectively.

[0097] This application optimizes the selection of transmission channels and the utilization of network resources by comprehensively using UDP and TCP connections, thereby improving the overall data transmission efficiency and enhancing the response speed and user experience of remote desktops.

[0098] Figure 4 This is a schematic diagram of the structure of a remote desktop data transmission device provided in this application, as shown below. Figure 4 As shown, in this embodiment, the remote desktop data transmission device 30 can be installed in an electronic device, and the remote desktop data transmission device 30 includes:

[0099] The acquisition module 301 is used to acquire the business data type corresponding to the data to be transmitted when there is a data transmission requirement in the remote desktop session;

[0100] The transmission module 302 is used to transmit the data to be transmitted to the other end based on the first User Datagram Protocol (UDP) transmission channel if the service data type is streaming media.

[0101] In some embodiments, the transmission module 302 is further configured to:

[0102] If the data type of the service is streaming media, then control information related to the data to be transmitted is transmitted to the peer through the second UDP transmission channel; the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

[0103] In some embodiments, when the transmission module 302 transmits data to be transmitted to the peer based on the first User Datagram Protocol (UDP) transmission channel, it is specifically used for:

[0104] The data to be transmitted is encapsulated into a first data packet in Real-Time Transport Protocol (RTP) format; the first data packet contains streaming media data.

[0105] The first data packet is sent to the other end through the first UDP transmission channel;

[0106] When transmitting control information related to the data to be transmitted to the other end via the second UDP transmission channel, the transmission module 302 is specifically used for:

[0107] Generate a second data packet in Real-Time Transport Control Protocol (RTCP) format; the second data packet contains control information related to the streaming media data;

[0108] The second data packet is encapsulated into a third data packet in UDP format and sent to the other end through the second UDP transmission channel.

[0109] In some embodiments, the service data type is either a first real-time type or a second real-time type, wherein the first real-time type has higher real-time requirements than the second real-time type; the transmission module 302 is further configured to:

[0110] If the data type of the business is the second real-time type, then the data to be transmitted will be encapsulated into a fourth data packet in UDP format;

[0111] The fourth data packet is sent to the other end through the second UDP transmission channel; the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

[0112] In some embodiments, the service data type is either a first real-time type or a second real-time type, wherein the first real-time type has higher real-time requirements than the second real-time type; the transmission module 302 is further configured to:

[0113] If the data type of the business is the first real-time type, then the data to be transmitted will be encapsulated into a fifth data packet in UDP format and a sixth data packet in Transmission Control Protocol (TCP) format.

[0114] The fifth data packet is sent to the other end through the third UDP transmission channel, and the sixth data packet is sent to the other end through the TCP transmission channel; the reliability of the third UDP transmission channel is lower than that of the first UDP transmission channel.

[0115] In some embodiments, the remote desktop data transmission device 30 further includes an establishment module for:

[0116] When a remote desktop session is established, UDP and TCP connections are established with the peer respectively.

[0117] A first UDP transmission channel and a second UDP transmission channel are established based on the UDP connection, and the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

[0118] In some embodiments, the transmission module 302 is further configured to, after transmitting the data to be transmitted to the peer based on the first User Datagram Protocol (UDP) transmission channel, if it is determined that there is data loss and the amount of lost data exceeds the data loss threshold corresponding to the first UDP transmission channel, retransmit the data to be transmitted to the peer according to the number of retransmissions corresponding to the first UDP transmission channel.

[0119] The remote desktop data transmission device 30 provided in this embodiment can execute the technical solution of the corresponding method embodiment. Its implementation principle and technical effect are similar to those of the corresponding method embodiment, and will not be described again here.

[0120] This application also provides an electronic device. Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application. For example... Figure 5 As shown, the electronic device 40 includes a processor 401 and a memory 402 communicatively connected to the processor 401.

[0121] The memory 402 stores computer-executable instructions; the processor 401 executes the computer-executable instructions stored in the memory 402 to implement the remote desktop data transmission method provided in this application.

[0122] In this embodiment, the memory 402 and the processor 401 are connected via a bus. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be categorized as an address bus, a data bus, a control bus, etc.

[0123] The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein. The various components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required.

[0124] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores computer-executable instructions that, when executed by a processor, are used to implement the remote desktop data transmission method provided in this application.

[0125] In an exemplary embodiment, a computer program product is also provided, including a computer program, which, when executed by a processor, is used to implement the remote desktop data transmission method provided in this application.

[0126] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0127] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0128] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0129] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0130] When an integrated unit / module is implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic component. Unless otherwise specified, memory can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as USB flash drives, random-access memory (RAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), enhanced dynamic random-access memory (EDRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), high-bandwidth memory (HBM), or hybrid memory cube (HMC) and other media capable of storing program code.

[0131] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause an electronic device to execute all or part of the steps of the methods of the various embodiments of this application.

[0132] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0133] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0134] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A remote desktop data transmission method, characterized in that, include: When there is a data transmission requirement in a remote desktop session, obtain the business data type corresponding to the data to be transmitted; If the data type of the service is streaming media, the data to be transmitted is transmitted to the other end based on the first User Datagram Protocol (UDP) transmission channel.

2. The method according to claim 1, characterized in that, The method further includes: If the data type of the service is the streaming media type, then control information related to the data to be transmitted is transmitted to the peer through the second UDP transmission channel; the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

3. The method according to claim 2, characterized in that, The transmission of the data to be transmitted to the peer via the first User Datagram Protocol (UDP) transmission channel includes: The data to be transmitted is encapsulated into a first data packet in Real-Time Transport Protocol (RTP) format; the first data packet contains streaming media data. The first data packet is sent to the peer through the first UDP transmission channel; The transmission of control information related to the data to be transmitted to the peer via the second UDP transmission channel includes: Generate a second data packet in Real-Time Transport Control Protocol (RTCP) format; the second data packet contains control information related to the streaming media data; The second data packet is encapsulated into a third data packet in UDP format and sent to the peer through the second UDP transmission channel.

4. The method according to claim 1, characterized in that, The business data type is either a first real-time type or a second real-time type, wherein the first real-time type has higher real-time requirements than the second real-time type; it also includes: If the data type of the service is the second real-time type, then the data to be transmitted is encapsulated into a fourth data packet in UDP format; The fourth data packet is sent to the peer through the second UDP transmission channel; the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

5. The method according to claim 1, characterized in that, The business data type is either a first real-time type or a second real-time type, wherein the first real-time type has higher real-time requirements than the second real-time type; it also includes: If the data type of the service is the first real-time type, then the data to be transmitted is encapsulated into a fifth data packet in UDP format and the data to be transmitted is encapsulated into a sixth data packet in Transmission Control Protocol (TCP) format. The fifth data packet is sent to the peer through the third UDP transmission channel, and the sixth data packet is sent to the peer through the TCP transmission channel; the reliability of the third UDP transmission channel is lower than that of the first UDP transmission channel.

6. The method according to any one of claims 1-5, characterized in that, Before obtaining the business data type corresponding to the data to be transmitted when there is a data transmission requirement in the remote desktop session, the method further includes: When the remote desktop session is established, a UDP connection and a TCP connection are established with the peer respectively; Based on the UDP connection, a first UDP transmission channel and a second UDP transmission channel are established respectively, and the reliability of the first UDP transmission channel is lower than that of the second UDP transmission channel.

7. The method according to claim 1, characterized in that, After transmitting the data to be transmitted to the peer via the first User Datagram Protocol (UDP) transmission channel, the method further includes: If it is determined that data loss exists and the amount of lost data exceeds the data loss threshold corresponding to the first UDP transmission channel, then the data to be transmitted is retransmitted to the peer according to the number of retransmissions corresponding to the first UDP transmission channel.

8. A remote desktop data transmission device, characterized in that, include: The acquisition module is used to acquire the business data type corresponding to the data to be transmitted when there is a data transmission requirement in the remote desktop session; The transmission module is used to transmit the data to be transmitted to the peer based on the first User Datagram Protocol (UDP) transmission channel if the data type of the service is streaming media.

9. An electronic device, characterized in that, include: A processor and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the remote desktop data transmission method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the remote desktop data transmission method as described in any one of claims 1 to 7.

Images