Information processing system, method, gateway and storage medium based on cloud desktop service

Abstract

The embodiment of the application provides an information processing system and method based on a cloud desktop service, a gateway and a storage medium. In the embodiment of the application, under the assistance of the gateway, multi-user collaborative operation of a cloud desktop server can be completed through only one TCP connection, thereby greatly reducing the number of TCP connections required to be established by the cloud desktop server. Meanwhile, the gateway also undertakes part of the functions of the cloud desktop server, thereby reducing the load pressure of the cloud desktop server, saving the computing power resources of the cloud desktop server, and guaranteeing the working performance of the cloud desktop server and the use experience of the cloud desktop service.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an information processing system and method based on cloud desktop service, a gateway and a storage medium. BACKGROUND

[0002] Cloud desktop (also known as desktop virtualization) refers to virtualizing the terminal system (also known as desktop) of a computer to achieve the security and flexibility of desktop use, so that users can access the cloud desktop server through various terminal devices at any time and anywhere.

[0003] With the continuous development of virtualization and cloud computing technology, the shunting capability of the cloud desktop server has been greatly improved, and the cloud desktop server supports multiple users to access through the client installed on their respective terminal devices. In actual application, when multiple users collaboratively operate the resources provided by the cloud desktop server, the cloud desktop server needs to establish multiple TCP (Transmission Control Protocol) connections with multiple clients, combine the control streams uploaded by multiple clients, and perform various operations, which consumes a large amount of computing resources and affects the use experience of the cloud desktop service. SUMMARY

[0004] The present application provides an information processing system and method based on cloud desktop service, a gateway and a storage medium, which saves the computing resources of the cloud desktop server and ensures the working performance of the cloud desktop server and the use experience of the cloud desktop service.

[0005] The present application provides an information processing system based on cloud desktop service, comprising: a plurality of clients, a gateway cluster and a cloud desktop server; each of the plurality of clients establishes a communication connection with a first gateway, the first gateway establishes a Transmission Control Protocol (TCP) connection with the cloud desktop server, and the first gateway is one of the gateway cluster; the first gateway receives and combines at least two first uplink control streams synchronously sent by at least two clients to obtain a second uplink control stream after combination, sends the second uplink control stream to the cloud desktop server through the TCP connection, receives first downlink streaming data returned by the cloud desktop server, and sends the first downlink streaming data to at least two clients respectively; the cloud desktop server generates first downlink streaming data in response to the second uplink control stream, and sends the first downlink streaming data to the first gateway through the TCP connection.

[0006] The embodiment of the present application further provides an information processing method, which is applied to a first gateway in a gateway cluster, the first gateway establishes a communication connection with each client in a plurality of clients, and the first gateway establishes a transmission control protocol (TCP) connection with a cloud desktop server; the method comprises the following steps: receiving at least two first uplink control streams synchronously sent by at least two clients; merging the at least two first uplink control streams synchronously sent by the at least two clients to obtain a second uplink control stream after merging; sending the second uplink control stream to the cloud desktop server through the TCP connection and receiving first downlink streaming data returned by the cloud desktop server; and sending the first downlink streaming data to the at least two clients respectively.

[0007] The embodiment of the present application further provides an information processing device, which comprises: a receiving module, configured to receive at least two first uplink control streams synchronously sent by at least two clients; a merging module, configured to merge the at least two first uplink control streams synchronously sent by the at least two clients to obtain a second uplink control stream after merging; a sending module, configured to send the second uplink control stream to a cloud desktop server through a TCP connection; a receiving module, configured to receive first downlink streaming data returned by the cloud desktop server; and a sending module, configured to send the first downlink streaming data to the at least two clients respectively.

[0008] The embodiment of the present application further provides a gateway, which comprises: a memory and a processor; the memory is configured to store a computer program; and the processor is coupled to the memory and is configured to execute the computer program to perform the steps in the information processing method.

[0009] The embodiment of the present application further provides a computer readable storage medium storing a computer program, when the computer program is executed by a processor, the processor can implement the steps in the information processing method.

[0010] In the embodiment of the present application, with the assistance of the gateway, the multi-user collaborative operation of the cloud desktop server can be completed through only one TCP connection, which greatly reduces the number of TCP connections required to be established by the cloud desktop server. Meanwhile, the gateway also undertakes part of the functions of the cloud desktop server, reduces the load pressure of the cloud desktop server, saves the computing power resources of the cloud desktop server, and guarantees the working performance of the cloud desktop server and the use experience of the cloud desktop service. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0012] Figure 1 FIG. 1 is a structural schematic diagram of an information processing system based on a cloud desktop service provided by the embodiment of the present application;

[0013] Figure 2 A signaling interaction diagram of an information processing method provided in an embodiment of the present application;

[0014] Figure 3 A flowchart of an information processing method provided in an embodiment of the present application;

[0015] Figure 4 A structural schematic diagram of an information processing apparatus provided in an embodiment of the present application;

[0016] Figure 5 A structural schematic diagram of a gateway provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below with reference to the embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0018] In the embodiments of the present application, “at least one” means one or more, and “multiple” means two or more. “And / or” describes the access relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, wherein A and B can be singular or plural. In the textual description of the present application, the character “ / ” generally represents that the associated objects before and after it are in an “or” relationship. In addition, in the embodiments of the present application, “first”, “second”, “third”, “fourth”, “fifth” and “sixth” are only used to distinguish the contents of different objects, and have no other special meanings.

[0019] In actual application, when multiple users cooperatively operate the cloud desktop server, the cloud desktop server needs to establish multiple TCP (Transmission Control Protocol) connections with multiple clients. The multiple users may simultaneously initiate control operations on the cloud desktop resources, and the multiple clients simultaneously send control streams triggered by the response control operations to the cloud desktop server through the respective corresponding TCP connections. The cloud desktop server merges the multiple control streams simultaneously initiated by the multiple clients, and determines the latest cloud desktop picture and / or audio data of the cloud desktop server in response to the merged control streams, encodes the latest cloud desktop picture and / or audio data into streaming data, and sends the streaming data to the corresponding clients through the corresponding TCP connections, so that the terminal device where the client is located displays the latest cloud desktop picture and / or plays the latest audio data based on the received streaming data. Obviously, when multiple users cooperatively operate the cloud desktop server, the cloud desktop server needs to establish multiple TCP connections with multiple clients, merge the control streams uploaded by the multiple clients, and perform multiple operations, which consumes more computing resources and easily causes the cloud desktop server to have insufficient computing resources to respond to the control streams, thereby affecting the working performance of the cloud desktop server and the use experience of the cloud desktop service.

[0020] Therefore, an embodiment of the present application provides an information processing system, method, gateway and storage medium based on a cloud desktop service. In the embodiment of the present application, under the assistance of the gateway, multiple users cooperatively operate the cloud desktop server through only one TCP connection, which greatly reduces the number of TCP connections that the cloud desktop server needs to establish. At the same time, the gateway also undertakes part of the functions of the cloud desktop server, thereby reducing the load pressure of the cloud desktop server, saving the computing resources of the cloud desktop server, and ensuring the working performance of the cloud desktop server and the use experience of the cloud desktop service.

[0021] Figure 1 An embodiment of the present application provides a structural schematic diagram of an information processing system based on a cloud desktop service. Referring to Figure 1 The system can include multiple clients, a gateway cluster and a cloud desktop server.

[0022] In this embodiment, the client is installed on a terminal device of a user, and the terminal device is, for example, a mobile phone, a tablet computer, a desktop computer, a wearable smart device, a smart home device, etc. The terminal device is mainly responsible for displaying and / or playing the streaming data provided by the cloud desktop server, and receiving the control operation triggered by the user on the resource data provided by the cloud desktop server. In actual application, the user can input the keyboard and mouse operation on the resource data provided by the cloud desktop server to the terminal device through a keyboard or a mouse, etc. The client in the terminal device responds to the keyboard and mouse operation, generates a corresponding control flow, and provides the control flow externally. The keyboard and mouse operation includes mouse operation, keyboard operation and mouse and keyboard combined operation. Different keyboard and mouse operations perform different controls, and which control corresponds to the keyboard and mouse operation can be flexibly set as needed. For example, the user presses the left mouse button, indicating that the user clicks the cloud desktop picture currently displayed by the terminal device; the user presses the right mouse button, indicating that the user long-presses the cloud desktop picture currently displayed by the terminal device; the user scrolls the mouse, indicating that the user scrolls the cloud desktop picture on the terminal device; and the user simultaneously operates the mouse and the keyboard, which can zoom in or zoom out the cloud desktop picture currently displayed by the terminal device.

[0023] In this embodiment, the client can be connected to the gateway cluster through a wired network or a wireless network. For example, the wired network can include coaxial cable, twisted pair and optical fiber, etc., and the wireless network can be a 2G network, a 3G network, a 4G network or a 5G network, a wireless fidelity (WIFI) network, etc. Further, in actual application, the client and the gateway cluster can establish a desktop streaming communication connection based on a streaming media protocol. The streaming media protocol includes, but is not limited to, RTP (Real-time Transport Protocol), RTCP (Real-time Transport Control Protocol), RTSP (Real Time Streaming Protocol), MMS, HLS, HTTP progressive streaming, ASP (Adaptive Streaming Protocol). ASP is a new type of end-to-cloud collaborative protocol for application flow, which provides end-to-cloud integrated protocol support and provides low-latency high-quality real-time interactive experience for terminal users.

[0024] The number of gateways in the gateway cluster is not limited, for example, it can be one or more. Further, the gateway cluster can be horizontally expanded, that is, new gateways can be added to the gateway cluster to expand the cluster size of the gateway cluster, and the number of gateways in the gateway cluster can be reduced to reduce the cluster size of the gateway cluster.

[0025] In this embodiment, in addition to the general gateway functions such as security verification, load balancing, stream forwarding, and flow control, the gateway also has extended functions, for example, undertaking some of the functions of the cloud desktop server. For instance, the gateway can replace the cloud desktop server in performing control flow merging or stream data replication, etc. Because the gateway undertakes some of the functions of the cloud desktop server, it reduces the load pressure on the cloud desktop server, saves its computing resources, and thus allows the cloud desktop server to have sufficient computing resources to respond to control flows, ensuring the working performance of the cloud desktop server and the user experience of the cloud desktop service.

[0026] In this embodiment, the cloud desktop server provides cloud desktop services, primarily responsible for tasks such as cloud computing and data storage. The cloud desktop server can provide various resource data, such as text files, audio data, and video data, which can be shared with users. The cloud desktop server can be, for example, an ARM architecture server or an x86 architecture server; there are no restrictions on which.

[0027] In this embodiment, any gateway in the gateway cluster only needs to establish one TCP (Transmission Control Protocol) connection with the cloud desktop server to achieve data interaction between the cloud desktop server and multiple clients. Compared to the existing method of establishing multiple TCP connections between the cloud desktop server and multiple clients, this greatly reduces the number of TCP connections required by the cloud desktop server, alleviating the load pressure on the cloud desktop server and saving computing resources. Of course, gateways can also establish multiple TCP connections with the cloud desktop server; there is no restriction on this.

[0028] Specifically, in scenarios involving multi-user collaborative operation of a cloud desktop server, multiple users simultaneously initiate keyboard and / or mouse operations. The client responds to these operations by generating a control flow. For ease of understanding and distinction, the control flow generated by the client is referred to as the first uplink control flow. Multiple clients simultaneously upload multiple first uplink control flows to the gateway cluster. See also... Figure 1As shown in ①, any gateway in the gateway cluster merges multiple first uplink control flows to obtain a second uplink control flow. It's worth noting that the control flow merging operation can reduce the number of control flows, combining multiple control flows into a single control flow. Furthermore, if multiple control flows belong to the same type, one control flow of the same type can be selected and retained as needed, while other control flows of the same type are deleted. For example, if multiple user-initiated keyboard and mouse operations all indicate zooming in on the cloud desktop displayed on the terminal device, then during the control flow merging process, only one control flow related to zooming in needs to be retained. As another example, if multiple user-initiated keyboard and mouse operations indicate zooming in, scrolling in, or zooming out, since these keyboard and mouse operations correspond to different types of control flows, they can all be retained and combined into a single control flow during the control flow merging process. Of course, the control flow merging rules can be flexibly set as needed and are not restricted.

[0029] See Figure 1 As shown in ②, the gateway uploads the second uplink control flow to the cloud desktop server through a TCP channel established with the cloud desktop server via a single TCP connection. See also Figure 1 As shown in ③, the cloud desktop server responds to the second uplink control flow and generates the first downlink stream data. The first downlink stream data can be understood as the stream data generated by the cloud desktop server that needs to be sent to the gateway.

[0030] See Figure 1 As shown in ④, the gateway receives the first downlink stream data sent by the cloud desktop server via a TCP channel. See also... Figure 1 As shown in ⑤, the gateway sends the first downlink stream data to multiple clients respectively, thus completing the entire task of supporting multiple users to collaboratively operate the cloud desktop server.

[0031] It is worth noting that, Figure 1 The number of clients, gateways, and cloud desktop servers mentioned is merely an example and is not limited.

[0032] For a specific example, in a multi-person collaborative work scenario or online education scenario, the cloud desktop resource owner first logs into the cloud desktop server and invites multiple collaborators to participate in collaborative operations. At this time, the cloud desktop resource owner and multiple collaborators can see the same cloud desktop screen on their respective terminal device display screens.

[0033] In actual application, only one collaborator may trigger the keyboard and mouse operation at the same time, the client generates the control flow in response to the keyboard and mouse operation, and sends the control flow to the cloud desktop server through the TCP channel via the gateway, the cloud desktop server renders the next moment of the cloud desktop picture in response to the control flow, and encapsulates the next moment of the cloud desktop picture into the downstream streaming data and sends the downstream streaming data to the gateway through the TCP channel, and the gateway sends the downstream streaming data to each client, so that the cloud desktop resource owner and the multiple collaborators can see the same cloud desktop picture on the display screen of the terminal device.

[0034] In actual application, only one collaborator may trigger the keyboard and mouse operation at the same time, the client generates the control flow in response to the keyboard and mouse operation, and sends the control flow to the cloud desktop server through the TCP channel via the gateway, the cloud desktop server renders the next moment of the cloud desktop picture in response to the control flow, and encapsulates the next moment of the cloud desktop picture into the downstream streaming data and sends the downstream streaming data to the gateway through the TCP channel, and the gateway sends the downstream streaming data to each client, so that the cloud desktop resource owner and the multiple collaborators can see the same cloud desktop picture on the display screen of the terminal device.

[0035] The technical solutions provided by the embodiments of the present application are described in detail below with reference to the drawings.

[0036] Figure 2 A signaling interaction diagram of an information processing method provided by the embodiments of the present application is provided. Referring to Figure 2 The method can include the following steps:

[0037] 201. The first gateway receives at least two first uplink control flows synchronously sent by at least two clients.

[0038] 202. The first gateway merges the at least two first uplink control flows synchronously sent by the at least two clients to obtain a second uplink control flow after merging.

[0039] 203. The first gateway sends the second uplink control flow to the cloud desktop server through a TCP connection.

[0040] 204. The cloud desktop server generates first downstream streaming data in response to the second uplink control flow.

[0041] 205. The cloud desktop server receives the first downstream streaming data returned by the cloud desktop server through the TCP connection.

[0042] 206. The first gateway sends the first downstream streaming data to the at least two clients respectively.

[0043] Specifically, each of the plurality of clients establishes a communication connection with a first gateway, the first gateway establishes a TCP connection with the cloud desktop server, and the first gateway is any gateway in the gateway cluster.

[0044] If at least two users synchronously operate the keyboard and / or mouse to initiate the keyboard and mouse operation, each of the at least two clients generates a first uplink control stream in response to the corresponding keyboard and mouse operation, and then synchronously uploads the first uplink control stream generated by each client to the gateway cluster. The first gateway in the gateway cluster receives the at least two first uplink control streams. Then, the first gateway merges the multiple first uplink control streams into one uplink control stream, and for the convenience of understanding and distinguishing, the merged uplink control stream is referred to as a second uplink control stream. The first gateway sends the second uplink control stream to the cloud desktop server through the TCP connection. The cloud desktop server generates first downlink streaming data in response to the second uplink control stream. Specifically, the cloud desktop server can render the latest cloud desktop picture and / or obtain the latest audio data in response to the second uplink control stream, and encapsulate the latest cloud desktop picture and / or audio data into the first downlink streaming data. The cloud desktop server receives the first downlink streaming data returned by the cloud desktop server through the TCP connection. The first gateway sends the first downlink streaming data to the at least two clients respectively. When the first downlink streaming data is downloaded to the client, the terminal device where the client is located can display the latest cloud desktop picture and / or play the latest audio data.

[0045] In actual application, if only one user synchronously operates the keyboard and / or mouse to initiate the keyboard and mouse operation at the same time, the corresponding client generates a first uplink control stream in response to the corresponding keyboard and mouse operation, and the first gateway in the gateway cluster uploads the first uplink control stream to the cloud desktop server. The cloud desktop server generates first downlink streaming data in response to the first uplink control stream. The cloud desktop server receives the first downlink streaming data returned by the cloud desktop server through the TCP connection. The first gateway sends the first downlink streaming data to the at least two clients respectively.

[0046] As an example, when the first gateway sends the first downlink streaming data to the at least two clients respectively, it is specifically used for: data replication of the first downlink streaming data to obtain at least two first downlink streaming data; and sending the at least two first downlink streaming data to the at least two clients respectively. Specifically, after data replication to obtain multiple first downlink streaming data, one first downlink streaming data is sent to one client, and each client is allocated one downlink streaming data.

[0047] The performance of the terminal device where the client is located can be different, and thus the display effect of the first downlink stream data on different terminal devices is different. For example, the hardware configuration of some terminal devices is good, and the cloud desktop picture is displayed smoothly on the terminal device. The hardware configuration of some terminal devices is poor, and the cloud desktop picture is displayed less smoothly on the terminal device. In addition, the channel state of the transmission channel between different clients and the first gateway is different, which also causes the output effect of the first downlink stream data on different terminal devices to be different.

[0048] Based on the above, further optionally, when the first gateway sends at least two first downlink stream data to at least two clients, specifically for each client in the at least two clients, the target resolution that the client can support is determined according to the device type of the terminal device where the client is located and / or the channel state of the transmission channel between the client and the first gateway; in the case that the current resolution of the first downlink stream data is not the target resolution, the first downlink stream data is re-encoded to obtain second downlink stream data with the target resolution; the second downlink stream data is sent to the corresponding client through the transmission channel. In the case that the current resolution of the first downlink stream data is the target resolution, the first downlink stream data is sent to the corresponding client through the transmission channel.

[0049] Specifically, before the gateway distributes the first downlink stream data to the client, it will determine whether the target resolution that the client can support is adapted to the resolution of the first downlink stream data, and in the case of non-adaptation, the first downlink stream data is re-encoded to obtain second downlink stream data adapted to the target resolution that the client can support, and thus the adaptive coding mode is achieved to send stream data with different resolutions to different clients, which improves the smoothness of client rendering and ensures the display effect of downlink stream data on terminal devices.

[0050] In practical applications, the resolutions corresponding to different device types can be determined in advance, and a mapping relationship between the device type and the resolution is established; and / or, the resolutions corresponding to different channel states are established in advance, and a mapping relationship between the channel state and the resolution is established. The channel state includes a strong network state reflecting a good communication signal, a weak network state reflecting a good communication signal, and a no network state reflecting no communication signal.

[0051] When the target resolution that the client can support is determined based on the device type and the channel state, the resolution determined based on the mapping relationship between the device type and the resolution and the resolution determined based on the mapping relationship between the channel state and the resolution are selected as the target resolution that the client can support.

[0052] In this embodiment, the first downlink stream data can be re-encoded by using a video encoding technology. For more information about the video encoding technology, please refer to the related art.

[0053] With the passage of time, there can be new clients establishing communication connections with the gateway cluster, and there can also be clients disconnecting from the gateway cluster. Expanding the cluster size of the gateway cluster can effectively cope with the demand for multi-path collaborative operation on the cloud desktop server, and reducing the cluster size of the gateway cluster can reduce resource consumption. Based on this, the gateway cluster can be horizontally expanded. Optionally, the second gateway can control the cluster size of the gateway cluster according to the number of clients, and the second gateway is any gateway in the gateway cluster. As an example, when the number of clients is greater than or equal to the first number, the gateway in the gateway cluster that is in an unavailable state is controlled to change to an available state, or a new gateway is prompted to be added to the gateway cluster to expand the cluster size of the gateway cluster; when the number of clients is less than the second number, the gateway in the gateway cluster that is in an available state is controlled to change to an unavailable state to reduce the cluster size of the gateway cluster; wherein the second number is less than the first number.

[0054] It is worth noting that the first number is the maximum number of clients that can be allowed to communicate with the gateway cluster under the concurrent performance constraint of the gateway cluster, and in addition, the second number is the minimum number of clients that can be allowed to communicate with the gateway cluster under the concurrent performance constraint of the gateway cluster.

[0055] The technical scheme provided by the embodiments of the present application, with the assistance of the gateway, can complete the multi-user collaborative operation of the cloud desktop server through only one TCP connection, greatly reducing the number of TCP connections required to be established by the cloud desktop server. At the same time, the gateway also undertakes part of the functions of the cloud desktop server, reduces the load pressure of the cloud desktop server, saves the computing power resources of the cloud desktop server, and guarantees the working performance of the cloud desktop server and the use experience of the cloud desktop service.

[0056] Figure 3 A flowchart of an information processing method provided by the embodiments of the present application. The execution subject of the method is an information processing device, which can be composed of hardware and / or software, and can generally be integrated in a first gateway in the gateway cluster, and the first gateway is any gateway in the gateway cluster, the first gateway establishes a communication connection with each client in the plurality of clients, and the first gateway establishes a transmission control protocol (TCP) connection with the cloud desktop server, see Figure 3 , the method can include the following steps:

[0057] 301、receive at least two first uplink control streams synchronously sent by at least two clients;

[0058] 302、merge the at least two first uplink control streams sent by the at least two clients synchronously to obtain a merged second uplink control stream;

[0059] 303、send the second uplink control stream to the cloud desktop server through the TCP connection and receive the first downlink stream data returned by the cloud desktop server;

[0060] 304、send the first downlink stream data to the at least two clients respectively.

[0061] Further optionally, sending the first downlink stream data to the at least two clients respectively comprises: duplicating the first downlink stream data to obtain at least two first downlink stream data; and sending the at least two first downlink stream data to the at least two clients respectively.

[0062] Further optionally, sending the at least two first downlink stream data to the at least two clients respectively comprises: for each of the at least two clients, determining a target resolution that the client can support according to a device type of a terminal device where the client is located and / or a channel state of a transmission channel between the client and the first gateway; in a case where a current resolution of the first downlink stream data is not the target resolution, re-encoding the first downlink stream data to obtain second downlink stream data with the target resolution; and sending the second downlink stream data with the target resolution to the corresponding client through the transmission channel.

[0063] Further optionally, in a case where the current resolution of the first downlink stream data is the target resolution, sending the first downlink stream data to the corresponding client through the transmission channel.

[0064] Further optionally, controlling the cluster size of the gateway cluster according to the number of clients.

[0065] Further optionally, controlling the cluster size of the gateway cluster according to the number of clients comprises: in a case where the number of clients is greater than or equal to a first number, controlling a part of the gateways in the gateway cluster that are in an unavailable state to change to an available state, or prompting to add new gateways in the gateway cluster to expand the cluster size of the gateway cluster; in a case where the number of clients is less than a second number, controlling a part of the gateways in the gateway cluster that are in the available state to change to the unavailable state to reduce the cluster size of the gateway cluster; wherein the second number is less than the first number.

[0066] Regarding Figure 3 The specific implementation manners of each step in the method shown can be referred to the related content of the foregoing embodiments, which will not be described here again.

[0067] The technical scheme provided by the embodiments of the present application can be completed by only one TCP connection with the assistance of the gateway, thereby greatly reducing the number of TCP connections required to be established by the cloud desktop server. Meanwhile, the gateway also undertakes part of the functions of the cloud desktop server, thereby reducing the load pressure of the cloud desktop server, saving the computing power resources of the cloud desktop server, and guaranteeing the working performance of the cloud desktop server and the use experience of the cloud desktop service.

[0068] Figure 4 A structural schematic diagram of an information processing device provided by the embodiments of the present application is provided. The device can be composed of hardware and / or software, and can generally be integrated in a first gateway in a gateway cluster. The first gateway is any gateway in the gateway cluster, the first gateway establishes a communication connection with each of a plurality of clients, and the first gateway establishes a transmission control protocol (TCP) connection with a cloud desktop server.

[0069] Referring to Figure 4 The device can include:

[0070] The receiving module 41 is configured to receive at least two first uplink control streams synchronously transmitted by at least two clients.

[0071] The merging module 42 is configured to merge the at least two first uplink control streams synchronously transmitted by the at least two clients to obtain a second uplink control stream after merging.

[0072] The sending module 43 is configured to send the second uplink control stream to the cloud desktop server through the TCP connection.

[0073] The receiving module 41 is configured to receive first downlink streaming data returned by the cloud desktop server.

[0074] The sending module 43 is configured to send the first downlink streaming data to the at least two clients respectively.

[0075] Further, when the sending module sends the first downlink streaming data to the at least two clients respectively, the sending module is specifically configured to: perform data replication on the first downlink streaming data to obtain at least two first downlink streaming data; and send the at least two first downlink streaming data to the at least two clients respectively.

[0076] Further optionally, the sending module is configured to send the at least two first downlink stream data to the at least two clients respectively, and specifically configured to: for each of the at least two clients, determine a target resolution supported by the client according to a device type of a terminal device where the client is located and / or a channel state of a transmission channel between the client and the first gateway; in a case where a current resolution of the first downlink stream data is not the target resolution, re-encode the first downlink stream data to obtain second downlink stream data having the target resolution; and send the second downlink stream data to the corresponding client through the transmission channel.

[0077] Further optionally, the sending module is further configured to: in a case where the current resolution of the first downlink stream data is the target resolution, send the first downlink stream data to the corresponding client through the transmission channel.

[0078] Further optionally, the apparatus further includes a control module configured to control a cluster size of the gateway cluster according to the number of clients.

[0079] Further optionally, the control module is configured to control the cluster size of the gateway cluster according to the number of clients, and specifically configured to: in a case where the number of clients is greater than or equal to a first number, control a gateway in the gateway cluster that is in an unavailable state to change to an available state, or prompt to add a new gateway in the gateway cluster to expand the cluster size of the gateway cluster; in a case where the number of clients is less than a second number, control a gateway in the gateway cluster that is in the available state to change to the unavailable state to reduce the cluster size of the gateway cluster; wherein the second number is less than the first number.

[0080] Figure 4 The apparatus shown can perform Figure 3 The method of the embodiments shown, the implementation principles and technical effects will not be described again. For the Figure 4 The specific manners in which each module, unit of the apparatus performs operations have been described in detail in the related system embodiments or signaling interaction embodiments, and will not be described in detail here.

[0081] It should be noted that the execution subject of each step of the method provided by the above embodiments can be the same device, or the method can also be executed by different devices as the execution subject. For example, the execution subject of steps 301 to 304 can be device A; for another example, the execution subject of steps 301 and 302 can be device A, and the execution subject of steps 303 and 304 can be device B; and so on.

[0082] In addition, in some of the processes described in this specification, including the processes described in the description and the accompanying drawings, multiple operations are described in a particular, sequential order. However, it should be understood that unless otherwise specifically stated in this specification, the order of the operations can be changed, and that the operations can be performed in parallel rather than sequentially. Also, it should be understood that unless otherwise specifically stated in this specification, the operations described in this specification can be combined or divided into sub-operations, and that the operations can be performed in an order different from the order described in this specification. Further, it should be understood that the operations described in this specification can be performed by more than one component or module.

[0083] Figure 5 A structure schematic diagram of a gateway provided by an embodiment of the present application is shown in FIG. 1. As shown in the figure, the gateway comprises a memory 51 and a processor 52. Figure 5

[0084] The memory 51 is configured to store computer programs and can be configured to store other various data to support operations on the computing platform. Examples of the data include instructions of any application program or method for operating on the computing platform, contact data, phonebook data, messages, pictures, videos, etc.

[0085] The memory 51 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0086] The processor 52 is coupled to the memory 51 and is configured to execute the computer programs in the memory 51 to receive at least two first uplink control streams synchronously transmitted by at least two clients, combine the at least two first uplink control streams synchronously transmitted by the at least two clients to obtain a second uplink control stream, send the second uplink control stream to a cloud desktop server through a TCP connection and receive first downlink streaming data returned by the cloud desktop server, and send the first downlink streaming data to the at least two clients respectively.

[0087] Further, as shown in FIG. 2, the gateway comprises a memory 21 and a processor 22.​Figure 5 As shown, the gateway further includes a communication component 53, a display 54, a power supply component 55, an audio component 56, and other components. Figure 5 The partial components are only schematically given, and it is not meant that the gateway only includes the components shown. Figure 5 The components shown. In addition, Figure 5 The components in the dashed box are optional components, rather than mandatory components, and can be determined according to the product form of the gateway. The gateway of the embodiment can be implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, or an IOT (Internet of Things) device, or can be implemented as a server device such as a conventional server, a cloud server, or a server array. If the gateway of the embodiment is implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, etc., the gateway can include the components in the dashed box. Figure 5 If the gateway of the embodiment is implemented as a server device such as a conventional server, a cloud server, or a server array, the gateway can not include the components in the dashed box. Figure 5 If the gateway of the embodiment is implemented as a server device such as a conventional server, a cloud server, or a server array, the gateway can not include the components in the dashed box.

[0088] The detailed implementation process of the processor performing each action can refer to the related description in the foregoing method embodiment or device embodiment, and will not be described here.

[0089] Correspondingly, the embodiment of the application further provides a computer readable storage medium storing a computer program, and the computer program is executed to implement each step that can be executed by the gateway in the foregoing method embodiment.

[0090] Correspondingly, the embodiment of the application further provides a computer program product, including a computer program / instruction, and when the computer program / instruction is executed by a processor, the processor can implement each step that can be executed by the gateway in the foregoing method embodiment.

[0091] The communication component is configured to facilitate wired or wireless communication between a device on which the communication component is installed and other devices. The device on which the communication component is installed can access a wireless network based on a communication standard, such as WiFi, a 2G, 3G, 4G / LTE, 5G, or the like cellular communication network, or a combination thereof. In an example embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, The Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0092] The display includes a screen, which can include a Liquid Crystal Display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect a duration and a pressure related to a touch or a slide operation.

[0093] The power supply component provides power to various components of the device on which the power supply component is installed. The power supply component can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device on which the power supply component is installed.

[0094] The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) that is configured to receive an external audio signal when the device on which the audio component is installed is in a particular mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in memory or transmitted via the communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0095] Those skilled in the art will appreciate that embodiments of the application can be readily used as a method, a system or a computer program product. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0096] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0097] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0099] In one typical configuration, the computing device includes one or more processors (central processing units, CPUs), input / output interfaces, network interfaces, and memory.

[0100] Memory can include non-persistent memory in computer-readable media, Random Access Memory (RAM), and / or non-volatile memory, such as Read Only Memory (ROM) or flash memory, etc. Memory is an example of computer-readable media.

[0101] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, Phase Change RAM (PRAM), Static Random-Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0102] It should also be noted that the terms "comprising", "comprising" or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0103] The above merely provides an example of the present application, and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall fall into the scope of claims of the present application.

Claims

1. An information processing system based on cloud desktop services, characterized in that, The system, applicable to scenarios supporting collaborative operation of a cloud desktop server by multiple users, includes: multiple clients, a gateway cluster, and a cloud desktop server; each client establishes a communication connection with a first gateway, the first gateway establishes a Transmission Control Protocol (TCP) connection with the cloud desktop server, the first gateway is one of the gateways in the gateway cluster, and the first gateway establishes a unique TCP channel with the cloud desktop server based on the TCP connection; the gateway replaces the cloud desktop server in performing control flow merging and streaming data replication; The first gateway is configured to receive and merge at least two first uplink control flows synchronously sent by at least two clients to obtain a merged second uplink control flow. Each client generates a first uplink control flow in response to keyboard and mouse operations. The merged second uplink control flow is a single control flow. The gateway sends the second uplink control flow to the cloud desktop server and receives first downlink stream data returned by the cloud desktop server through the TCP connection. It also copies the first downlink stream data to obtain at least two first downlink stream data. The at least two first downlink stream data are then sent to the at least two clients respectively. Each client is allocated one first downlink stream data. The cloud desktop server is configured to generate the first downlink streaming data in response to the second uplink control flow, and to send the first downlink streaming data to the first gateway via the TCP connection.

2. The system according to claim 1, characterized in that, When the first gateway sends the at least two first downlink stream data to the at least two clients respectively, it is specifically used for: For each of the at least two clients, the target resolution that the client can support is determined based on the device type of the terminal device where the client is located and / or the channel state of the transmission channel between the client and the first gateway; If the current resolution of the first downlink streaming data is not the target resolution, the first downlink streaming data is re-encoded to obtain a second downlink streaming data with the target resolution. The second downlink streaming data is sent to the corresponding client through the transmission channel.

3. The system according to claim 2, characterized in that, The first gateway is also used for: When the current resolution of the first downlink stream data is the target resolution, the first downlink stream data is sent to the corresponding client through the transmission channel.

4. The system according to claim 1, characterized in that, The second gateway is used to control the cluster size of the gateway cluster based on the number of clients. The second gateway is one of the gateways in the gateway cluster.

5. The system according to claim 4, characterized in that, The second gateway is specifically used to: when the number of clients is greater than or equal to the first number, control some gateways in the gateway cluster that are in an unavailable state to become available, or prompt the addition of a new gateway in the gateway cluster to expand the cluster size of the gateway cluster. When the number of clients is less than the second number, some of the gateways in the gateway cluster that are in an available state are controlled to become unavailable in order to reduce the cluster size of the gateway cluster; wherein, the second number is less than the first number.

6. An information processing method, characterized in that, It is applied to scenarios that support multiple users to collaboratively operate a cloud desktop server. It is applied to the first gateway in a gateway cluster. The first gateway establishes a communication connection with each of the multiple clients and establishes a Transmission Control Protocol (TCP) connection with the cloud desktop server. The first gateway establishes a unique TCP channel based on the TCP connection with the cloud desktop server. The first gateway replaces the cloud desktop server in performing control flow merging and streaming data replication; The method includes: Receive at least two first uplink control flows synchronously sent by at least two clients; At least two first uplink control flows synchronously sent by at least two clients are merged to obtain a merged second uplink control flow, wherein the merged second uplink control flow is a single control flow; each client generates a first uplink control flow in response to keyboard and mouse operations; The second uplink control flow is sent to the cloud desktop server through the TCP connection, and the first downlink stream data returned by the cloud desktop server is received. The first downlink stream data is copied to obtain at least two sets of the first downlink stream data. The at least two first downlink streaming data are respectively sent to the at least two clients.

7. The method according to claim 6, characterized in that, Sending the at least two first downlink stream data to the at least two clients respectively includes: For each of the at least two clients, the target resolution that the client can support is determined based on the device type of the terminal device where the client is located and / or the channel state of the transmission channel between the client and the first gateway; If the current resolution of the first downlink streaming data is not the target resolution, the first downlink streaming data is re-encoded to obtain a second downlink streaming data with the target resolution. The second downlink streaming data is sent to the corresponding client through the transmission channel.

8. The method according to claim 6, characterized in that, Also includes: The cluster size of the gateway cluster is controlled based on the number of clients.

9. The method according to claim 8, characterized in that, The cluster size of the gateway cluster is controlled based on the number of clients, including: When the number of clients is greater than or equal to a first number, control some of the gateways in the gateway cluster that are in an unavailable state to become available, or prompt the addition of a new gateway to the gateway cluster to expand the cluster size. When the number of clients is less than the second number, some of the gateways in the gateway cluster that are in an available state are controlled to become unavailable in order to reduce the cluster size of the gateway cluster; wherein, the second number is less than the first number.

10. An information processing device, characterized in that, This device is designed for scenarios supporting collaborative operation of a cloud desktop server by multiple users. It includes a first gateway configured in a gateway cluster. The first gateway establishes a communication connection with each of the multiple clients, and establishes a Transmission Control Protocol (TCP) connection with the cloud desktop server. The first gateway establishes a unique TCP channel with the cloud desktop server based on this TCP connection. The gateway replaces the cloud desktop server in performing control flow merging and streaming data replication. The device comprises: The receiving module is used to receive at least two first uplink control flows synchronously sent by at least two clients; The merging module is used to merge at least two first uplink control flows synchronously sent by at least two clients to obtain a merged second uplink control flow, wherein the merged second uplink control flow is a single control flow; each client generates a first uplink control flow in response to keyboard and mouse operations; The sending module is used to send the second uplink control flow to the cloud desktop server via a TCP connection; The receiving module is used to receive the first downlink streaming data returned by the cloud desktop server; The sending module is configured to copy the first downlink stream data to obtain at least two first downlink stream data; send the at least two first downlink stream data to the at least two clients respectively; and allocate one first downlink stream data to each client.

11. A gateway, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor is coupled to the memory for executing the computer program to perform the steps of the method according to any one of claims 6-9.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 6-9.

Images