Window processing system, method and device

Abstract

The embodiments of the present specification provide a window processing system, method and device, wherein the window processing system includes: a client and a cloud server, wherein the cloud server is communicatively connected to the client; the client is configured to send a window processing request for the cloud application of the cloud desktop to the cloud server based on a window processing instruction submitted by a user for the cloud application; the cloud server is configured to intercept a currently running window screen of the cloud application based on the received window processing request, and return the window screen to the client as a virtual window; the client is configured to respond to the window processing instruction and display the received virtual window according to a pre-created window processing style layout.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and more particularly to a window processing system. One or more embodiments of this specification also relate to a window processing method, a window processing device, a computing device, and a computer-readable storage medium. Background Art

[0002] Window processing technology refers to the technology of displaying multiple active windows on the terminal screen at the same time, and users can use different applications in multiple active windows. During window processing, users can not only see the applications displayed in multiple active windows on the terminal screen at the same time, but also interact with multiple applications separately by switching between multiple active windows.

[0003] Traditional multi-windows are real multiple active windows. In the cloud desktop scenario, the client needs to establish a multi-channel video stream connection with the cloud server to realize multi-channel video stream transmission from the cloud server to the client to achieve multi-window display on the client. However, establishing a multi-channel video stream link requires a lot of client resources, which can easily affect the client's business processing efficiency. Therefore, an effective method is urgently needed to solve this problem. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a window processing system. One or more embodiments of this specification also relate to a window processing method, a window processing device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a window processing system is provided, including:

[0006] A client and a cloud server, wherein the cloud server is in communication connection with the client;

[0007] The client is configured to send a window processing request of the cloud application to the cloud server based on a window processing instruction submitted by a user for a cloud application of a cloud desktop;

[0008] The cloud server is configured to intercept a currently running window screen of the cloud application based on the received window processing request, and return the window screen as a virtual window to the client;

[0009] The client is configured to respond to the window processing instruction and display the received virtual window according to a pre-created window processing style layout.

[0010] Optionally, the cloud server is further configured to set identification information for the virtual window to establish a mapping relationship between the identification information and the virtual window, and return the mapping relationship to the client.

[0011] Optionally, the client is further configured to send a window running screen rendering request to the cloud server according to the window running screen rendering instruction submitted by the user for the target virtual window, wherein the window running screen rendering request carries identification information of the target virtual window;

[0012] The cloud server is also configured to receive the window running screen rendering request, determine the target cloud application corresponding to the target virtual window according to the identification information, render the to-be-rendered screen content corresponding to the target cloud application, and transmit the rendered output window running screen to the client.

[0013] Optionally, the client is further configured to send a window processing request of the cloud application to the cloud server based on a window processing instruction submitted by the user for the cloud application of the cloud desktop, wherein the window processing request carries the number of foreground application windows and / or background application windows and identification information of each window;

[0014] The cloud server is further configured to receive and intercept the running window screen of the foreground application window and / or the background application window based on the window processing request, and return the running window screen as a virtual window to the client.

[0015] Optionally, the client is further configured to receive the virtual window in response to the window processing instruction, adjust the position and / or size of the virtual window according to a pre-created window processing style layout, and display according to the adjustment result.

[0016] Optionally, the client is further configured to send a window activation request to the cloud server according to an activation instruction submitted by the user for the target virtual window, wherein the window activation request carries identification information of the target virtual window.

[0017] Optionally, the cloud server is further configured to receive the activation request, determine the target cloud application corresponding to the target virtual window according to the identification information, obtain the to-be-output video stream corresponding to the target cloud application, and transmit the to-be-output video stream to the client;

[0018] The client is also configured to receive the video stream to be output, and display the video stream to be output as the real window display screen of the target cloud application in the foreground.

[0019] Optionally, the cloud server is further configured to encode the window running screen and transmit the generated encoded data to the client.

[0020] Optionally, the cloud server is further configured to encode the video stream to be output and transmit the generated encoded data to the client.

[0021] Optionally, the client is further configured to receive and decode the encoded data, obtain the window running screen and display it in the foreground.

[0022] Optionally, the video stream to be output includes picture data and / or audio data corresponding to the current running window of the target cloud application;

[0023] The cloud server is further configured to encode the picture data according to a picture encoding function, and / or to encode the audio data using an audio encoding function.

[0024] Optionally, the client is further configured to send a window switching request to the cloud server based on a window switching instruction submitted by a user;

[0025] The cloud server is further configured to receive the window switching request, intercept the window running screen of the foreground application window, obtain the to-be-output video stream corresponding to the target virtual window to be switched in the background application window, and transmit the to-be-output video stream to the client;

[0026] The client is also configured to receive the video stream to be output, and display the video stream to be output as the real window display screen of the target cloud application in the foreground.

[0027] Optionally, the client is further configured to send a multi-window processing request for the at least two cloud applications of the cloud desktop to the cloud server based on a multi-window processing instruction submitted by the user for the at least two cloud applications of the cloud desktop;

[0028] The cloud server is configured to intercept at least two currently running window images of the at least two cloud applications based on the received multi-window processing request, and return the at least two window images as virtual windows to the client;

[0029] The client is configured to respond to the window processing instruction and display the received virtual window according to a pre-created multi-window processing style layout.

[0030] Optionally, the client is further configured to send a window processing request for a single cloud application of the cloud desktop to the cloud server based on a window processing instruction submitted by the user for the single cloud application of the cloud desktop;

[0031] The cloud server is configured to intercept a currently running window screen of the single cloud application based on the received window processing request, and return the window screen as a virtual window to the client;

[0032] The client is configured to respond to the window processing instruction and display the received virtual window according to a preset single window processing style layout.

[0033] According to a second aspect of an embodiment of this specification, a window processing method is provided, which is applied to a cloud server, including:

[0034] Receive a window processing request submitted by a client to a cloud application of a cloud desktop;

[0035] intercepting a currently running window screen of the cloud application based on the window processing request;

[0036] The window images are returned to the client as virtual windows respectively.

[0037] Optionally, the method further includes:

[0038] Identification information is set for the virtual window to establish a mapping relationship between the identification information and the virtual window, and the mapping relationship is returned to the client.

[0039] Optionally, intercepting the currently running window screen of the cloud application based on the window processing request includes:

[0040] The window images of the foreground application window and / or the background application window are intercepted based on the window processing request, and the window images are returned to the client as virtual windows respectively.

[0041] Optionally, the method further includes:

[0042] Receiving a window activation request submitted by a client for a target virtual window;

[0043] Determining a target cloud application corresponding to the target virtual window according to the identification information of the target virtual window carried in the window activation request;

[0044] Obtain the to-be-output video stream corresponding to the target cloud application, and transmit the to-be-output video stream to the client.

[0045] Optionally, transmitting the to-be-outputted video stream to the client comprises:

[0046] The video stream to be output is encoded, and the generated encoded data is transmitted to the client.

[0047] Optionally, the video stream to be output includes picture data and / or audio data corresponding to the current running window of the target cloud application;

[0048] Correspondingly, the transmitting the to-be-output video stream to the client includes:

[0049] The picture data is encoded according to a picture encoding function, and / or the audio data is encoded using an audio encoding function.

[0050] Optionally, the receiving a window processing request submitted by the client to the cloud application of the cloud desktop includes:

[0051] Receiving a multi-window processing request submitted by a client for at least two cloud applications of a cloud desktop;

[0052] Accordingly, intercepting the currently running window screen of the cloud application based on the window processing request includes:

[0053] At least two currently running window screens of the at least two cloud applications are intercepted based on the multi-window processing request.

[0054] Optionally, the receiving a window processing request submitted by the client to the cloud application of the cloud desktop includes:

[0055] Receive a window processing request submitted by a client for a single cloud application of a cloud desktop;

[0056] Accordingly, intercepting the currently running window screen of the cloud application based on the window processing request includes:

[0057] A currently running window screen of the single cloud application is captured based on the window processing request.

[0058] According to a third aspect of an embodiment of this specification, a window processing device is provided, including:

[0059] A receiving module is configured to receive a window processing request submitted by a client to a cloud application of a cloud desktop;

[0060] An interception module, configured to intercept a currently running window screen of the cloud application based on the window processing request;

[0061] The returning module is configured to return the window images to the client as virtual windows respectively.

[0062] According to a fourth aspect of the embodiments of this specification, another window processing method is provided, which is applied to a client and includes:

[0063] Based on the window processing instruction submitted by the user for the cloud application of the cloud desktop, sending a window processing request of the cloud application to the cloud server;

[0064] receiving a currently running window screen of the cloud application intercepted by the cloud server based on the window processing request;

[0065] In response to the window processing instruction, the window images are respectively used as virtual windows, and the virtual windows are displayed according to a pre-created window processing style layout.

[0066] Optionally, the method further includes:

[0067] According to the activation instruction submitted by the user for the target virtual window, a window activation request is sent to the cloud server, wherein the window activation request carries identification information of the target virtual window.

[0068] Optionally, the method further includes:

[0069] receiving a video stream to be output returned by the cloud server, wherein the video stream to be output corresponds to a target cloud application, the target cloud application corresponds to the target virtual window, and the target virtual window is determined according to the identification information;

[0070] The video stream to be output is displayed as the foreground of the real window display screen of the target cloud application.

[0071] Optionally, the method further includes:

[0072] Receiving the encoded data returned by the cloud server, the encoded data being generated by the cloud server encoding the video stream to be output;

[0073] The encoded data is decoded to obtain the window running screen and display it in the foreground.

[0074] Optionally, the sending a window processing request of the cloud application to the cloud server based on the window processing instruction submitted by the user for the cloud application of the cloud desktop includes:

[0075] Based on the multi-window processing instruction submitted by the user for at least two cloud applications of the cloud desktop, sending a multi-window processing request for the at least two cloud applications to the cloud server;

[0076] Correspondingly, the receiving the currently running window screen of the cloud application intercepted by the cloud server based on the window processing request includes:

[0077] Receive at least two currently running window images of the at least two cloud applications intercepted by the cloud server based on the multi-window processing request.

[0078] Optionally, the sending a window processing request of the cloud application to the cloud server based on the window processing instruction submitted by the user for the cloud application of the cloud desktop includes:

[0079] Based on the window processing instruction submitted by the user for the single cloud application of the cloud desktop, sending a window processing request of the single cloud application to the cloud server;

[0080] Correspondingly, the receiving the currently running window screen of the cloud application intercepted by the cloud server based on the window processing request includes:

[0081] A currently running window screen of the single cloud application is received, which is intercepted by the cloud server based on the window processing request.

[0082] According to a fifth aspect of the embodiments of this specification, another window processing device is provided, including:

[0083] A sending module, configured to send a window processing request of the cloud application to the cloud server based on a window processing instruction submitted by a user for the cloud application of the cloud desktop;

[0084] a receiving module configured to receive a currently running window screen of the cloud application intercepted by the cloud server based on the window processing request;

[0085] The display module is configured to respond to the window processing instruction, use the window images as virtual windows respectively, and display the virtual windows according to the pre-created window processing style layout.

[0086] According to a sixth aspect of an embodiment of this specification, a computing device is provided, including:

[0087] Memory and processor;

[0088] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the window processing method.

[0089] According to a seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the window processing method are implemented.

[0090] In one embodiment of the present specification, after receiving a window processing request submitted by a client, the cloud server captures the running window screen of the cloud application running in the cloud desktop, and returns the running window screen to the client as a virtual window. The client performs window processing by displaying the running window screen, that is, the client uses the window screenshot of the cloud application returned by the cloud server as a virtual window for static display of multiple windows, thereby avoiding establishing multiple video streams between the client and the cloud server, reducing the resource usage of the client, and realizing window processing through a single video stream to meet the user's window processing needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 is a schematic diagram of a window processing system provided by an embodiment of the present specification;

[0092] Figure 2 is a processing flow chart of a window processing method provided by an embodiment of this specification;

[0093] Figure 3 is a schematic diagram of a window processing device provided by an embodiment of this specification;

[0094] Figure 4 is a processing flow chart of another window processing method provided by an embodiment of this specification;

[0095] Figure 5 is a schematic diagram of another window processing device provided by an embodiment of the present specification;

[0096] Figure 6 is an interactive schematic diagram of a window processing method provided by an embodiment of this specification;

[0097] Figure 7 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0098] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0099] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0100] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0101] First, the terms involved in one or more embodiments of this specification are explained.

[0102] Cloud desktop: A desktop system that actually runs on the server (cloud) side, and the terminal only displays its images and sounds.

[0103] Zero terminal: A terminal product without CPU, memory, or hard disk operating system. It is a hardware product used to display cloud desktops and is the carrier of the service (cloud) end.

[0104] Multi-window: Application windows are displayed in a tiled manner according to certain rules.

[0105] Cloud application: an application running on the server (cloud).

[0106] Cloud Server (Elastic Compute Service, ECS): also known as cloud host, is a host product in the cloud computing service system. It is a simple, efficient, secure, reliable, and elastically scalable computing service.

[0107] In this specification, a window processing system is provided. This specification also relates to a window processing method, a window processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0108] Figure 1 A schematic diagram of a window processing system provided according to an embodiment of the present specification is shown, comprising: a client 102 and a cloud server 104, wherein the cloud server 104 is in communication connection with the client 102;

[0109] The client 102 is configured to send a window processing request of the cloud application to the cloud server 104 based on a window processing instruction submitted by a user for a cloud application of a cloud desktop;

[0110] The cloud server 104 is configured to intercept the currently running window screen of the cloud application based on the received window processing request, and return the window screen as a virtual window to the client 102;

[0111] The client 102 is configured to respond to the window processing instruction and display the received virtual window according to a pre-created window processing style layout.

[0112] In the embodiments of this specification, the number of clients 102 is not limited, and generally there are multiple clients. Client 102 can be any device with communication capabilities and screen display capabilities, for example, a smart phone, a tablet computer, a desktop computer, a wearable device or a game console. Compared with a physical server, a cloud server 104 is actually a service platform deployed in the cloud that can provide comprehensive business capabilities to various Internet users.

[0113] Cloud applications refer to applications that run in the cloud based on cloud computing technology. Cloud applications run on cloud servers 104. Clients 102 do not need to download, install, or consider the configuration of client 102. The application can be run through the network. In actual applications, the client 102 can also be a zero terminal with a display. The zero terminal serves as a carrier for the service (cloud) end and can display the current running window of the cloud application through the display.

[0114] A cloud application can be an independent application program. Users can use the client 102 to communicate with the cloud server 104 to operate the cloud application. The cloud application can also be subordinate to an application platform. Users can log in to the cloud application platform by entering their username and password through the client 102 and select a cloud application from the many applications on the cloud application platform to run. The cloud application includes but is not limited to various application products such as games, news, social networking, and life that run on the cloud.

[0115] When a user sends a request to run a target cloud application to the cloud server 104 through the client 102 , the cloud server 104 may transmit the window parameter information of the running window of the target cloud application to the client for display in the form of a video stream.

[0116] Since the number of cloud applications that can be run simultaneously in the cloud server 104 is not limited, therefore, in the case where only a single cloud application is running, in the embodiment of this specification, the client 102 is further configured to send a window processing request of the single cloud application to the cloud server based on the window processing instruction submitted by the user for the single cloud application of the cloud desktop;

[0117] The cloud server 104 is configured to intercept the currently running window screen of the single cloud application based on the received window processing request, and return the window screen as a virtual window to the client 102;

[0118] The client 102 is configured to respond to the window processing instruction and display the received virtual window according to a preset single window processing style layout.

[0119] Alternatively, when multiple cloud applications are running simultaneously in the cloud server 104, in the embodiment of the specification, the client 102 is further configured to send a multi-window processing request of the at least two cloud applications to the cloud server 104 based on a multi-window processing instruction submitted by the user for the at least two cloud applications of the cloud desktop;

[0120] The cloud server 104 is configured to intercept at least two currently running window images of the at least two cloud applications based on the received multi-window processing request, and return the at least two window images as virtual windows to the client 102;

[0121] The client 102 is configured to respond to the window processing instruction and display the received virtual window according to a pre-created multi-window processing style layout.

[0122] When there are running cloud applications on the cloud desktop (the number of running cloud applications can be one or more), the user can send a window processing instruction to the client 102, so that the client 102 sends a window processing request to the cloud server 104. Since each running cloud application in the cloud server 104 corresponds to a running window, after the cloud server 104 receives the window processing request, if the window processing request is a window processing request for a single cloud application, the currently running window screen of the single running cloud application is captured; if the window processing request is a window processing request for multiple cloud applications, the currently running multiple window screens of the multiple running cloud applications are captured, that is, screenshots are taken of the windows of one or more running cloud applications, and the screenshot window screen is returned to the client 102 as a virtual window.

[0123] The window processing instruction may be a window display instruction, the window processing request may be a window display request, and the window processing style layout may be a window display style layout.

[0124] In addition, if the client 102 sends a window display request for a single cloud application, the cloud server 104 takes a screenshot of the currently running window screen of the single running cloud application, and returns the single screenshot to the client 102; if the client 102 sends a multi-window display request for multiple cloud applications, the cloud server 104 takes screenshots of the currently running window screens of the multiple running cloud applications, and obtains multiple window screens of the multiple cloud applications. The cloud server 104 can return all the multiple window screens obtained by the screenshot as virtual windows to the client 102, or return part of the multiple window screens obtained by the screenshot as virtual windows to the client 102. The specific number of virtual windows returned can be determined according to the actual needs of the user and is not limited here.

[0125] After receiving the screenshot, the client 102 displays the screenshot according to the window processing style layout to respond to the user's window processing instruction.

[0126] In actual applications, after the cloud server 104 captures the multiple window images of multiple cloud applications currently running, the multiple window images can be transmitted to the client 102 through a single video stream, or through multiple video streams. In order to save the resource occupancy rate of the client, the embodiment of this specification preferably uses a single video stream for transmission.

[0127] Furthermore, the client 102 may pre-create different window display style layouts for different numbers of virtual windows. Therefore, when receiving different numbers of virtual windows, they can all be displayed using the window display style layout corresponding to the number of virtual windows; or, the client 102 may pre-create a window display style layout, in which the number of virtual windows that can be displayed is as large as possible. When the client 102 receives different numbers of virtual windows, the virtual windows can be displayed according to the window display style layout.

[0128] When creating a window display style layout, you can set different window display sizes for different windows, such as Figure 1 As shown, in the set window display style layout, windows 1 to 4 have larger window display sizes than windows 5 and 6. In actual applications, the virtual window display position of the target cloud application can be set at a fixed position in the window display style layout, and a fixed window display size can be set. For example, for a payment cloud application, its virtual window can be set at Figure 1The position of window 1 in the middle, and its window display size is relatively large compared to other windows.

[0129] by Figure 1 Taking the multi-window display style layout shown in the figure as an example, if the window display style layout is the window display style layout corresponding to the case where the number of virtual windows is 6, when the client 102 receives 6 virtual windows returned by the cloud server 104, the received virtual windows are displayed according to the window display style layout; if the window display style layout is a universal display style layout that is independent of the number of received virtual windows, when the client 102 receives 1, 2, ... or 6 virtual windows, they can all be displayed through the window display style layout. If 1 virtual window is received, it can be displayed in Figure 1 If you receive 2, 3, ... or 6 virtual windows, you can also display them in any window from 1 to 6 in the Figure 1 Any 2, 3, ... or 6 windows from windows 1 to 6 can be displayed.

[0130] The cloud server returns the captured running window screen of the cloud application running in the cloud desktop to the client as a virtual window. The client performs window processing by displaying the virtual window (screenshot), thereby avoiding the establishment of multiple video streams between the client and the cloud server, reducing the resource usage of the client, and realizing window processing through a single video stream to meet the user's window processing needs.

[0131] Further, the client 102 is further configured to send a window processing request of the cloud application to the cloud server 104 based on the window processing instruction submitted by the user for the cloud application of the cloud desktop, wherein the window processing request carries the number of foreground application windows and / or background application windows and identification information of each window;

[0132] The cloud server 104 is further configured to receive and intercept the running window screen of the foreground application window and / or the background application window based on the window processing request, and return the running window screen to the client 102 as a virtual window.

[0133] Specifically, when there are multiple running cloud applications in the cloud server 104, if the display mode of the client 102 is single-window display, the client 102 can only display the running window of any running cloud application through a single window, that is, there is only one foreground application window, and the running windows of other running cloud applications can only serve as background application windows. Therefore, when the user sends a window processing instruction to the client 102, the window processing instruction carries the number of foreground application windows and background application windows and the identification information of each window.

[0134] After receiving the window processing instruction sent by the user, the client 102 sends a window processing request to the cloud server 104 according to the number of foreground application windows and background application windows carried in the instruction and the identification information of each window.

[0135] Similarly, after receiving the window processing request, the cloud server 104 captures the running window screens of multiple running cloud applications, where the running windows of the multiple cloud applications include a foreground application window and a background application window, that is, the foreground application window and the background application window are captured, and the screenshot results are returned to the client 102 as a virtual window.

[0136] After receiving the virtual window, the client 102 displays the virtual window according to the window processing style layout to respond to the user's window processing instruction.

[0137] In a specific implementation, after receiving the virtual window returned by the cloud server 104, the client 102 may adjust the position and / or size of the virtual window according to the pre-created window processing style layout, and display it according to the adjustment result.

[0138] The cloud server returns the captured window images of the foreground application window and the background application window running in the cloud desktop to the client as virtual windows. The client performs window processing by displaying the virtual windows (screenshots), thereby avoiding the establishment of multiple video streams between the client and the cloud server and reducing the resource usage of the client. Window processing can be achieved through a single video stream to meet the user's window processing needs.

[0139] In a specific implementation, the cloud server 104 is further configured to set identification information for the virtual window to establish a mapping relationship between the identification information and the virtual window, and return the mapping relationship to the client 102 .

[0140] Specifically, after the cloud server 104 captures the currently running window screen of the cloud application, before returning the window screen as a virtual window to the client 102, it is necessary to set identification information for the virtual window to establish a mapping relationship between the identification information and the virtual window, and return the mapping relationship and the virtual window to the client, wherein different identification information is used to point to a unique virtual window.

[0141] In addition, after the client 102 receives the mapping relationship and performs window processing on the virtual window, the user can submit a virtual window activation instruction for any target virtual window in the window processing screen;

[0142] Accordingly, the client 102 is further configured to send a window activation request to the cloud server 104 according to the activation instruction submitted by the user for the target virtual window, wherein the window activation request carries the identification information of the target virtual window;

[0143] The cloud server 104 is further configured to receive the activation request, determine the target cloud application corresponding to the target virtual window according to the identification information, obtain the to-be-output video stream corresponding to the target cloud application, and transmit the to-be-output video stream to the client 102;

[0144] The client 102 is further configured to receive the video stream to be output, and display the video stream to be output as the real window display screen of the target cloud application in the foreground.

[0145] Specifically, after the client 102 receives the window activation instruction submitted by the user for the target virtual window, it can determine the identification information of the target virtual window according to the mapping relationship between the identification information returned by the cloud server 104 and the virtual window, and send a window activation request for the target virtual window to the cloud server 104 based on the identification information of the target virtual window, so that the cloud server 104 responds to the window activation request, determines the target virtual window according to the identification information, and determines the target cloud application corresponding to the target virtual window, so as to transmit the to-be-output video stream of the target cloud application to the client 102. The client 102 receives the to-be-output video stream, and displays the to-be-output video stream as the foreground as the real window display screen of the target cloud application, thereby activating the target virtual window.

[0146] Since the window activation request submitted by the client 102 carries the identification information of the target virtual window, after receiving the window activation request, the cloud server 104 can determine the target virtual window that has a mapping relationship with the identification information and the target cloud application corresponding to the target virtual window based on the mapping relationship between the identification information and the virtual window, so as to obtain the video stream to be output corresponding to the target cloud application, and transmit the video stream to be output to the client 102.

[0147] In addition, when the user submits a window activation instruction to the client for the target virtual window, the window parameters such as the window size and window display position of the real window of the target cloud application can be added to the window activation instruction to achieve a flexible display effect for the real window.

[0148] Different from the current window processing technology that dynamically displays the real running windows of each application, the embodiment of this specification realizes the static display of multiple windows by displaying a screenshot (virtual window) of the running window of the cloud application. When the user chooses to activate the static virtual window, the cloud server 104 transmits the video stream to the client 102 through a single video stream to activate the corresponding real window, thereby avoiding the establishment of multiple video streams, which is beneficial to reducing the resource occupation of the client. Window processing can be realized through a single video stream to meet the user's window processing needs;

[0149] Through window processing, the client uses the screenshots of the cloud server to simulate the real window for the user, and displays the screenshots in a tiled manner for easy browsing by the user. When the user selects one of the screenshots, the cloud server activates the corresponding real window for the user, which is conducive to improving the user experience.

[0150] In a specific implementation, the cloud server 104 is further configured to encode the video stream to be output, and transmit the generated encoded data to the client 102;

[0151] The client 102 is further configured to receive and decode the encoded data, obtain the window running screen and display it in the foreground.

[0152] Specifically, after obtaining the video stream to be output, the cloud server 104 can transmit the video stream to be output to the client 102. In actual applications, the cloud server 104 can encode the video stream to be output, and transmit the encoded data obtained by encoding to the client 102. The client 102 decodes the encoded data and configures a corresponding display window according to the decoding result, so as to display the window running screen obtained by decoding in the foreground through the display window.

[0153] If the target cloud application is a video playback application, the video stream to be output may include picture data and / or audio data corresponding to the current running window of the target cloud application. The cloud server 104 encodes the video stream to be output, that is, encodes the picture data according to a picture encoding function to obtain picture encoding data, and / or encodes the audio data using an audio encoding function to obtain audio encoding data, and encapsulates the picture encoding data and audio encoding data, and transmits the encapsulated data to the client 102 via the network.

[0154] In addition, as mentioned above, the cloud applications include but are not limited to various application products such as games, news, social, and life that run on the cloud. Since different products have different requirements for image quality, different bit rate encoding standards can be used to encode the video streams to be output corresponding to the current running windows of different cloud applications. For example, for video or interactive application products such as games, a higher bit rate encoding technology is required to encode the video stream, while for text or non-interactive application products such as news, a lower bit rate encoding technology can be used to encode the video stream. In actual applications, video stream encoding can be performed using video encoding standards such as H264 and H265, thereby achieving the goal of meeting the image quality requirements of different cloud applications while reducing the amount of data transmitted, thereby improving data transmission efficiency.

[0155] After the cloud server 104 captures the running window screen of the target application, due to the huge amount of data, it is encoded and compressed before network transmission, which is beneficial to improving data transmission efficiency and user experience quality.

[0156] In the embodiment of the present specification, the client 102 is further configured to send a window switching request to the cloud server 104 based on the window switching instruction submitted by the user;

[0157] The cloud server 104 is further configured to receive the window switching request, intercept the window running screen of the foreground application window, obtain the to-be-output video stream corresponding to the target virtual window to be switched in the background application window, and transmit the to-be-output video stream to the client 102;

[0158] The client 102 is further configured to receive the video stream to be output, and display the video stream to be output as the real window display screen of the target cloud application in the foreground.

[0159] Specifically, since the cloud server 104 can run multiple cloud applications at the same time, when the cloud desktop runs multiple (at least two) cloud applications at the same time, if the display mode of the client 102 is single-window display, the client 102 can only display the running window of any running cloud application through a single window, that is, there is only one foreground application window, and the running windows of other running cloud applications can only serve as background application windows. Therefore, when the user submits a window switching instruction to the client 102, the client 102 sends a window switching request to the cloud server 104. The window switching request may carry the identification information of the currently running window and the identification information of the target virtual window to be switched.

[0160] After receiving the window switching request, the cloud server 104 needs to capture the window running screen of the foreground application window before the window switching, and return the capture result to the client 102 for storage. The purpose is to save the display screen of the foreground application window before the switching, so that when the foreground application window before the switching is switched to the foreground again, it can start displaying from the display screen corresponding to the capture result.

[0161] After capturing the window running screen of the foreground application window, obtain the video stream to be output corresponding to the target virtual window to be switched in the background application window, and transmit the video stream to be output to the client 102; or, encode the video stream to be output, and transmit the obtained encoded data to the client 102, the client 102 decodes the encoded data, and displays the decoding result as the real window display screen of the target cloud application in the foreground.

[0162] Similarly, after the cloud server 104 captures the running window screen of the target application to be switched, due to the huge amount of data, it is encoded and compressed before network transmission, which is beneficial to improving data transmission efficiency and user experience quality.

[0163] In a specific implementation, the client 102 is further configured to send a window running screen rendering request to the cloud server 104 according to the window running screen rendering instruction submitted by the user for the target virtual window, wherein the window running screen rendering request carries the identification information of the target virtual window;

[0164] The cloud server 104 is also configured to receive the window running screen rendering request, determine the target cloud application corresponding to the target virtual window according to the identification information, render the to-be-rendered screen content corresponding to the target cloud application, and transmit the rendered output window running screen to the client 102.

[0165] Specifically, the aforementioned cloud server 104 obtains the to-be-output video stream corresponding to the target virtual window to be switched in the background application window, and transmits the to-be-output video stream to the client 102. It can be understood that the cloud server 104 activates the dynamic real window of the target cloud application for the client, which is suitable for video or interactive application products such as games. If the target cloud application is a static non-interactive application product, the user sends a cloud application activation request to the cloud server 104 through the client 102, which can be understood as the user sends a window running screen rendering instruction of the target virtual window to the cloud server 104 through the client 102. The cloud server 104 renders the to-be-rendered screen content corresponding to the target cloud application, and after sending the rendered output window running screen to the client 102, the client 102 displays the activated static window of the target cloud application to the user.

[0166] In addition, the cloud server 104 renders the to-be-rendered screen content corresponding to the target cloud application. After obtaining the rendered output window running screen, the window running screen can be encoded and the encoded data can be sent to the client 102. The client 102 decodes the encoded data and displays the decoding result as the real window display screen of the target cloud application in the foreground.

[0167] Different activation methods are used for different types of cloud applications, that is, by transmitting video streams or single images to the client 102, while meeting the image quality requirements of different cloud applications, the data transmission volume is reduced, which is conducive to improving data transmission efficiency.

[0168] In one embodiment of the present specification, after receiving a window processing request submitted by the client 102, the cloud server 104 captures the running window screen of the cloud application running in the cloud desktop, and returns the running window screen to the client 102 as a virtual window. The client 102 performs window processing by displaying the running window screen, that is, the client 102 uses the window screenshot of the cloud application returned by the cloud server 104 as a virtual window for static display of multiple windows, thereby avoiding the establishment of multiple video streams between the client 102 and the cloud server 104, reducing the resource usage of the client, and realizing window processing through a single video stream to meet the user's window processing needs.

[0169] In the case where the window processing request is a multi-window display request, after receiving the multi-window display request submitted by the client 102, the cloud server 104 captures the running window screens of at least two cloud applications running in the cloud desktop, and returns the running window screens as virtual windows to the client 102, so that the client 102 can perform multi-window display by displaying the virtual windows (screenshots), thereby avoiding the establishment of multiple video streams between the client 102 and the cloud server 104, reducing the resource usage of the client 102, and realizing multi-window display through a single video stream to meet the user's multi-window display needs. Through the multi-window display method, the client 102 uses the screenshots of the cloud server 104 to simulate real windows for users, and displays the screenshots in a tiled manner for easy browsing by users. When the user selects one of the screenshots, the cloud server 104 activates the corresponding real window for the user, which is conducive to improving the user experience.

[0170] Figure 2 A processing flow chart of a window processing method provided according to an embodiment of the present specification is shown, including steps 202 to 206.

[0171] Step 202: Receive a window processing request submitted by a client to a cloud application of a cloud desktop.

[0172] Step 204: intercepting the currently running window screen of the cloud application based on the window processing request.

[0173] Step 206: Return the window image as a virtual window to the client.

[0174] A window processing method provided in an embodiment of the present specification is applied to a cloud server. A cloud server is actually a service platform deployed in the cloud that can provide comprehensive business capabilities to various Internet users.

[0175] Cloud applications refer to applications that run in the cloud based on cloud computing technology. Cloud applications run on cloud servers. Clients do not need to download, install, or consider client configuration. They can run the application simply through the network. In actual applications, the client can also be a zero terminal with a display. The zero terminal serves as a carrier for the service (cloud) end and can display the current running window of the cloud application through the display.

[0176] A cloud application can be an independent application program. Users can operate the cloud application by using a client to communicate with a cloud server. A cloud application can also be subordinate to an application platform. Users can log in to the cloud application platform by entering their username and password through the client and select a cloud application from the many applications on the cloud application platform to run. The cloud application includes but is not limited to various application products such as games, news, social networking, and life that run in the cloud.

[0177] When a user sends a request to run a target cloud application to a cloud server through a client, the cloud server can transmit the window parameter information of the running window of the target cloud application to the client for display in the form of a video stream. Therefore, when there are running cloud applications on the cloud desktop (there can be one or more running cloud applications), the user can send a window processing instruction to the client, so that the client sends a window processing request to the cloud server. After receiving the window processing request, the cloud server captures the window images of the multiple running cloud applications, that is, takes screenshots of the windows of the multiple running cloud applications, and returns the screenshots to the client as virtual windows. After receiving the screenshots, the client displays the screenshots according to the window processing style layout to respond to the user's window processing instruction.

[0178] Optionally, the method further includes:

[0179] Identification information is set for the virtual window to establish a mapping relationship between the identification information and the virtual window, and the mapping relationship is returned to the client.

[0180] Optionally, intercepting the currently running window screen of the cloud application based on the window processing request includes:

[0181] The window screen of the foreground application window and / or the background application window is intercepted based on the window processing request, and the window screen is returned to the client as a virtual window.

[0182] Optionally, the method further includes:

[0183] Receiving a window activation request submitted by a client for a target virtual window;

[0184] Determining a target cloud application corresponding to the target virtual window according to the identification information of the target virtual window carried in the window activation request;

[0185] Obtain the to-be-output video stream corresponding to the target cloud application, and transmit the to-be-output video stream to the client.

[0186] Optionally, transmitting the to-be-outputted video stream to the client comprises:

[0187] The video stream to be output is encoded, and the generated encoded data is transmitted to the client.

[0188] Optionally, the video stream to be output includes picture data and / or audio data corresponding to the current running window of the target cloud application;

[0189] Correspondingly, the transmitting the to-be-output video stream to the client includes:

[0190] The picture data is encoded according to a picture encoding function, and / or the audio data is encoded using an audio encoding function.

[0191] Optionally, the receiving a window processing request submitted by the client to the cloud application of the cloud desktop includes:

[0192] Receiving a multi-window processing request submitted by a client for at least two cloud applications of a cloud desktop;

[0193] Accordingly, intercepting the currently running window screen of the cloud application based on the window processing request includes:

[0194] At least two currently running window screens of the at least two cloud applications are intercepted based on the multi-window processing request.

[0195] Optionally, the receiving a window processing request submitted by the client to the cloud application of the cloud desktop includes:

[0196] Receive a window processing request submitted by a client for a single cloud application of a cloud desktop;

[0197] Accordingly, intercepting the currently running window screen of the cloud application based on the window processing request includes:

[0198] A currently running window screen of the single cloud application is captured based on the window processing request.

[0199] In one embodiment of the present specification, after receiving a window processing request (window display request) submitted by a client, the cloud server captures the running window screen of the cloud application running in the cloud desktop, and returns the running window screen to the client as a virtual window, so that the client performs window processing by displaying the virtual window (screenshot), thereby avoiding establishing multiple video streams between the client and the cloud server, reducing the resource usage of the client, and realizing window processing through a single video stream to meet the user's window processing needs.

[0200] The above is a schematic scheme of a window processing method applied to a cloud server in this embodiment. It should be noted that the technical scheme of the window processing method and the technical scheme of the above-mentioned window processing system belong to the same concept, and the details not described in detail in the technical scheme of the window processing method can be referred to the description of the technical scheme of the above-mentioned window processing system.

[0201] Corresponding to the above method embodiment, this specification also provides a window processing device embodiment, Figure 3 FIG. 1 is a schematic diagram of a window processing device provided by an embodiment of the present specification. Figure 3 As shown, the device comprises:

[0202] The receiving module 302 is configured to receive a window processing request submitted by the client to the cloud application of the cloud desktop;

[0203] The interception module 304 is configured to intercept the currently running window screen of the cloud application based on the window processing request;

[0204] The returning module 306 is configured to return the window image as a virtual window to the client.

[0205] Optionally, the device further comprises:

[0206] The identification information setting module is configured to set identification information for the virtual window to establish a mapping relationship between the identification information and the virtual window, and return the mapping relationship to the client.

[0207] Optionally, the interception module 304 includes:

[0208] The interception submodule is configured to intercept the window images of the foreground application window and the background application window based on the window processing request, and return the window images to the client as virtual windows.

[0209] Optionally, the device further comprises:

[0210] A window activation request receiving module is configured to receive a window activation request submitted by a client for a target virtual window;

[0211] a target cloud application determination module, configured to determine a target cloud application corresponding to the target virtual window according to the identification information of the target virtual window carried in the window activation request;

[0212] The video stream transmission module is configured to obtain the to-be-output video stream corresponding to the target cloud application and transmit the to-be-output video stream to the client.

[0213] Optionally, the video stream transmission module includes:

[0214] The first encoding submodule is configured to encode the video stream to be output and transmit the generated encoded data to the client.

[0215] Optionally, the video stream to be output includes picture data and / or audio data corresponding to the current running window of the target cloud application;

[0216] Correspondingly, the video stream transmission module includes:

[0217] The second encoding submodule is configured to encode the picture data according to a picture encoding function and / or to encode the audio data using an audio encoding function.

[0218] Optionally, the receiving module 302 includes:

[0219] A first receiving submodule is configured to receive a multi-window processing request submitted by a client for at least two cloud applications of a cloud desktop;

[0220] Accordingly, the interception module 304 includes:

[0221] The first interception submodule is configured to intercept at least two currently running window screens of the at least two cloud applications based on the multi-window processing request.

[0222] Optionally, the receiving module 302 includes:

[0223] A second receiving submodule is configured to receive a window processing request submitted by a client for a single cloud application of a cloud desktop;

[0224] Accordingly, the interception module 304 includes:

[0225] The second interception submodule is configured to intercept the currently running window screen of the single cloud application based on the window processing request.

[0226] The above is a schematic scheme of a window processing device of this embodiment. It should be noted that the technical scheme of the window processing device and the technical scheme of the above-mentioned window processing method belong to the same concept, and the details not described in detail in the technical scheme of the window processing device can be referred to the description of the technical scheme of the above-mentioned window processing method.

[0227] Figure 4 A processing flow chart of another window processing method provided according to an embodiment of the present specification is shown, including steps 402 to 406.

[0228] Step 402: Based on the window processing instruction submitted by the user for the cloud application of the cloud desktop, a window processing request of the cloud application is sent to the cloud server.

[0229] Step 402: Receive the currently running window screen of the cloud application captured by the cloud server based on the window processing request.

[0230] Step 406: In response to the window processing instruction, the window image is used as a virtual window, and the virtual window is displayed according to a pre-created window processing style layout.

[0231] A window processing method provided in an embodiment of the present specification is applied to a client, and the client 102 can be any device with communication capabilities and screen display capabilities, for example, it can be a smart phone, a tablet computer, a desktop computer, a wearable device or a game console.

[0232] Compared with physical servers, cloud servers are actually a service platform deployed in the cloud that can provide comprehensive business capabilities to various Internet users.

[0233] Cloud applications refer to applications that run in the cloud based on cloud computing technology. Cloud applications run on cloud servers. Clients do not need to download, install, or consider client configuration. They can run the application simply through the network. In actual applications, the client can also be a zero terminal with a display. The zero terminal serves as a carrier for the service (cloud) end and can display the current running window of the cloud application through the display.

[0234] A cloud application can be an independent application program. Users can operate the cloud application by using a client to communicate with a cloud server. A cloud application can also be subordinate to an application platform. Users can log in to the cloud application platform by entering their username and password through the client and select a cloud application from the many applications on the cloud application platform to run. The cloud application includes but is not limited to various application products such as games, news, social networking, and life that run in the cloud.

[0235] When a user sends a request to run a target cloud application to a cloud server through a client, the cloud server can transmit the window parameter information of the running window of the target cloud application to the client for display in the form of a video stream. Therefore, when there are running cloud applications on the cloud desktop (the number of running cloud applications can be one or more), the user can send a window processing instruction to the client, so that the client sends a window processing request to the cloud server. After receiving the window processing request, the cloud server captures the window images of the running multiple cloud applications, that is, takes screenshots of the windows of the running multiple cloud applications, and returns the screenshots to the client as virtual windows. After receiving the screenshots, the client displays the screenshots according to the window processing style layout to respond to the user's window processing instruction.

[0236] Optionally, the method further includes:

[0237] According to the activation instruction submitted by the user for the target virtual window, a window activation request is sent to the cloud server, wherein the window activation request carries identification information of the target virtual window.

[0238] Optionally, the method further includes:

[0239] receiving a video stream to be output returned by the cloud server, wherein the video stream to be output corresponds to a target cloud application, the target cloud application corresponds to the target virtual window, and the target virtual window is determined according to the identification information;

[0240] The video stream to be output is displayed as the foreground of the real window display screen of the target cloud application.

[0241] Optionally, the method further includes:

[0242] Receiving the encoded data returned by the cloud server, the encoded data being generated by the cloud server encoding the video stream to be output;

[0243] The encoded data is decoded to obtain the window running screen and display it in the foreground.

[0244] Optionally, the sending a window processing request of the cloud application to the cloud server based on the window processing instruction submitted by the user for the cloud application of the cloud desktop includes:

[0245] Based on the multi-window processing instruction submitted by the user for at least two cloud applications of the cloud desktop, sending a multi-window processing request for the at least two cloud applications to the cloud server;

[0246] Correspondingly, the receiving the currently running window screen of the cloud application intercepted by the cloud server based on the window processing request includes:

[0247] Receive at least two currently running window images of the at least two cloud applications intercepted by the cloud server based on the multi-window processing request.

[0248] Optionally, the sending a window processing request of the cloud application to the cloud server based on the window processing instruction submitted by the user for the cloud application of the cloud desktop includes:

[0249] Based on the window processing instruction submitted by the user for the single cloud application of the cloud desktop, sending a window processing request of the single cloud application to the cloud server;

[0250] Correspondingly, the receiving the currently running window screen of the cloud application intercepted by the cloud server based on the window processing request includes:

[0251] A currently running window screen of the single cloud application is received, which is intercepted by the cloud server based on the window processing request.

[0252] In one embodiment of the present specification, after receiving a window processing request (window display request) submitted by a client, the cloud server captures the running window screen of the cloud application running in the cloud desktop, and returns the running window screen to the client as a virtual window, so that the client performs window processing (window display) by displaying the virtual window (screenshot), thereby avoiding establishing multiple video streams between the client and the cloud server, reducing the resource usage of the client, and realizing window processing through a single video stream to meet the user's window processing needs.

[0253] The above is a schematic scheme of a window processing method applied to a client of this embodiment. It should be noted that the technical scheme of the window processing method and the technical scheme of the above-mentioned window processing system belong to the same concept, and the details not described in detail in the technical scheme of the window processing method can be referred to the description of the technical scheme of the above-mentioned window processing system.

[0254] Corresponding to the above method embodiment, this specification also provides another window processing device embodiment, Figure 5FIG. 2 is a schematic diagram of another window processing device provided by an embodiment of the present specification. Figure 5 As shown, the device comprises:

[0255] The sending module 502 is configured to send a window processing request of the cloud application to the cloud server based on the window processing instruction submitted by the user for the cloud application of the cloud desktop;

[0256] The receiving module 504 is configured to receive a currently running window screen of the cloud application intercepted by the cloud server based on the window processing request;

[0257] The display module 506 is configured to respond to the window processing instruction, use the window image as a virtual window, and display the virtual window according to a pre-created window processing style layout.

[0258] Optionally, the device further comprises:

[0259] The window activation request sending module is configured to send a window activation request to the cloud server according to an activation instruction submitted by the user for the target virtual window, wherein the window activation request carries identification information of the target virtual window.

[0260] Optionally, the device further comprises:

[0261] A video stream receiving module is configured to receive a to-be-output video stream returned by the cloud server, wherein the to-be-output video stream corresponds to a target cloud application, the target cloud application corresponds to the target virtual window, and the target virtual window is determined according to the identification information;

[0262] The display module is configured to display the to-be-output video stream as the real window display screen of the target cloud application in the foreground.

[0263] Optionally, the device further comprises:

[0264] A coded data receiving module, configured to receive coded data returned by the cloud server, wherein the coded data is generated by the cloud server encoding the video stream to be output;

[0265] The decoding module is configured to decode the encoded data, obtain the window running screen and display it in the foreground.

[0266] Optionally, the sending module 502 includes:

[0267] A first sending submodule is configured to send a multi-window processing request for at least two cloud applications of the cloud desktop to the cloud server based on a multi-window processing instruction submitted by the user for the at least two cloud applications of the cloud desktop;

[0268] Accordingly, the receiving module 504 includes:

[0269] The first receiving submodule is configured to receive at least two currently running window images of the at least two cloud applications intercepted by the cloud server based on the multi-window processing request.

[0270] Optionally, the sending module 502 includes:

[0271] A second sending submodule is configured to send a window processing request of a single cloud application of the cloud desktop to the cloud server based on a window processing instruction submitted by the user for the single cloud application of the cloud desktop;

[0272] Accordingly, the receiving module 504 includes:

[0273] The second receiving submodule is configured to receive the currently running window screen of the single cloud application intercepted by the cloud server based on the window processing request.

[0274] The above is a schematic scheme of another window processing device of this embodiment. It should be noted that the technical scheme of the window processing device and the technical scheme of the above another window processing method belong to the same concept, and the details of the technical scheme of the window processing device that are not described in detail can all be referred to the description of the technical scheme of the above another window processing method.

[0275] The following combination Figure 6 , taking the application of the window processing method provided in this specification in an interactive scenario as an example, the window processing method is further described. Figure 6 An interactive schematic diagram of a window processing method provided by an embodiment of the present specification is shown, and the specific steps include step 602 to step 620.

[0276] Step 602: The client receives a multi-window display instruction from the user.

[0277] Specifically, the multi-window display instruction is submitted by the user for at least two cloud applications of the cloud desktop.

[0278] Step 604: The client sends a multi-window display request for the at least two cloud applications to the cloud server.

[0279] Step 606: The cloud server assigns ID numbers to at least two running windows of the at least two cloud applications.

[0280] The cloud server establishes a mapping relationship between the at least two running windows and the ID numbers.

[0281] Step 608: The cloud server captures the window images of the at least two running windows.

[0282] Step 610: The cloud server returns the ID number and the window image to the client.

[0283] Specifically, the cloud server returns the window image as a virtual window to the client, and returns the mapping relationship between the running window and the ID number to the client.

[0284] Step 612: The client displays the received virtual window according to a pre-created multi-window display style layout.

[0285] Step 614: the client receives an activation instruction submitted by the user for the target virtual window.

[0286] Step 616: The client sends a window activation request for the target virtual window to the cloud server.

[0287] Specifically, the window activation request carries identification information of the target virtual window.

[0288] Step 618: The cloud server activates the target virtual window and displays the activated real window of the target cloud application in the foreground.

[0289] Specifically, the cloud server determines the target cloud application corresponding to the target virtual window according to the identification information, and obtains the to-be-output video stream corresponding to the real window of the target cloud application.

[0290] Step 620: The cloud server transmits the to-be-output video stream corresponding to the real window of the target cloud application to the client.

[0291] Step 622: The client displays the to-be-output video stream as the real window display screen of the target cloud application in the foreground.

[0292] In one embodiment of the present specification, after receiving a multi-window display request submitted by a client, the cloud server captures the running window screens of at least two cloud applications running in the cloud desktop, and returns the running window screens to the client as virtual windows, so that the client can perform multi-window display by displaying the virtual windows (screenshots), thereby avoiding the establishment of multiple video streams between the client and the cloud server and reducing the resource usage of the client. Multi-window display can be achieved through a single video stream to meet the user's multi-window display needs. Through multi-window display, the client uses the screenshots of the cloud server to simulate real windows for users, and displays the screenshots in a tiled manner for easy browsing by users. When the user selects one of the screenshots, the cloud server activates the corresponding real window for the user, which is conducive to improving the user experience.

[0293] Figure 7The block diagram of a computing device 700 according to an embodiment of the present specification is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0294] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0295] In one embodiment of the present specification, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0296] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 700 may also be a mobile or stationary server.

[0297] The memory 710 is used to store computer executable instructions, and the processor 720 is used to execute the following computer executable instructions to implement the steps of the window processing method.

[0298] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above window processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above window processing method.

[0299] An embodiment of the present specification further provides a computer-readable storage medium storing computer instructions, which are used to implement the steps of the window processing method when executed by a processor.

[0300] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above window processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above window processing method.

[0301] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0302] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0303] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0304] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0305] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A window processing system, include: A client and a cloud server, wherein the cloud server is in communication connection with the client; The client is configured to send a window processing request of the cloud application to the cloud server based on a window processing instruction submitted by a user for a cloud application of a cloud desktop; The cloud server is configured to intercept a currently running window screen of the cloud application based on the received window processing request, and return the window screen as a virtual window to the client through a single-channel video stream; The client is configured to display the received virtual window in accordance with a pre-created window processing style layout in response to the window processing instruction; The client is further configured to send a window processing request of the cloud application to the cloud server based on a window processing instruction submitted by the user for the cloud application of the cloud desktop, wherein the window processing request carries the number of foreground application windows and / or background application windows and identification information of each window; The cloud server is further configured to receive and intercept the running window screen of the foreground application window and / or the background application window based on the window processing request, and return the running window screen as a virtual window to the client.

2. According to the system of claim 1, the cloud server is further configured to set identification information for the virtual window to establish a mapping relationship between the identification information and the virtual window, and return the mapping relationship to the client.

3. According to the system of claim 2, the client is further configured to send a window running screen rendering request to the cloud server according to the window running screen rendering instruction submitted by the user for the target virtual window, wherein the window running screen rendering request carries the identification information of the target virtual window; The cloud server is also configured to receive the window running screen rendering request, determine the target cloud application corresponding to the target virtual window according to the identification information, render the to-be-rendered screen content corresponding to the target cloud application, and transmit the rendered output window running screen to the client.

4. According to the system of claim 1, the client is further configured to receive the virtual window in response to the window processing instruction, adjust the position and / or size of the virtual window according to a pre-created window processing style layout, and display it according to the adjustment result.

5. According to the system of claim 2, the client is further configured to send a window activation request to the cloud server according to an activation instruction submitted by the user for the target virtual window, and the window activation request carries identification information of the target virtual window.

6. The system according to claim 5, wherein the cloud server is further configured to receive the activation request, determine the target cloud application corresponding to the target virtual window according to the identification information, obtain the to-be-output video stream corresponding to the target cloud application, and transmit the to-be-output video stream to the client; The client is also configured to receive the video stream to be output, and display the video stream to be output as the real window display screen of the target cloud application in the foreground.

7. According to the system of claim 3, the cloud server is further configured to encode the window running screen and transmit the generated encoded data to the client.

8. According to the system of claim 6, the cloud server is further configured to encode the video stream to be output and transmit the generated encoded data to the client.

9. According to the system of claim 7 or 8, the client is further configured to receive and decode the encoded data, obtain the window running screen and display it in the foreground.

10. The system according to claim 6, wherein the video stream to be output includes picture data and / or audio data corresponding to the current running window of the target cloud application; The cloud server is further configured to encode the picture data according to a picture encoding function, and / or to encode the audio data using an audio encoding function.

11. The system according to claim 1, wherein the client is further configured to send a window switching request to the cloud server based on a window switching instruction submitted by a user; The cloud server is further configured to receive the window switching request, intercept the window running screen of the foreground application window, obtain the to-be-output video stream corresponding to the target virtual window to be switched in the background application window, and transmit the to-be-output video stream to the client; The client is also configured to receive the video stream to be output, and display the video stream to be output as the real window display screen of the target cloud application in the foreground.

12. The system according to claim 1, wherein the client is further configured to send a multi-window processing request for the at least two cloud applications of the cloud desktop to the cloud server based on a multi-window processing instruction submitted by the user for the at least two cloud applications of the cloud desktop; The cloud server is configured to intercept at least two currently running window images of the at least two cloud applications based on the received multi-window processing request, and return the at least two window images as virtual windows to the client; The client is configured to respond to the window processing instruction and display the received virtual window according to a pre-created multi-window processing style layout.

13. The system according to claim 1, wherein the client is further configured to send a window processing request for a single cloud application of a cloud desktop to the cloud server based on a window processing instruction submitted by a user for the single cloud application; The cloud server is configured to intercept a currently running window screen of the single cloud application based on the received window processing request, and return the window screen to the client via a single video stream as a virtual window; The client is configured to respond to the window processing instruction and display the received virtual window according to a preset single window processing style layout.

14. A window processing method, applied to a cloud server, include: Receiving a window processing request submitted by a client to a cloud application of a cloud desktop, wherein the window processing request is sent based on a window processing instruction submitted by a user to a cloud application of a cloud desktop, and the window processing request carries the number of foreground application windows and / or background application windows and identification information of each window; Intercepting the running window screen of the foreground application window and / or the background application window based on the window processing request; The running window images are respectively returned to the client as virtual windows through a single-channel video stream.

15. The method according to claim 14, further comprising: include: Identification information is set for the virtual window to establish a mapping relationship between the identification information and the virtual window, and the mapping relationship is returned to the client.

16. The method according to claim 15, wherein the window screen of the currently running cloud application is intercepted based on the window processing request. include: The window images of the foreground application window and / or the background application window are intercepted based on the window processing request, and the window images are returned to the client as virtual windows through a single-channel video stream.

17. The method according to claim 15, further comprising: include: Receiving a window activation request submitted by a client for a target virtual window; Determining a target cloud application corresponding to the target virtual window according to the identification information of the target virtual window carried in the window activation request; Obtain the to-be-output video stream corresponding to the target cloud application, and transmit the to-be-output video stream to the client.

18. The method according to claim 17, wherein the video stream to be output is transmitted to the client. include: The video stream to be output is encoded, and the generated encoded data is transmitted to the client.

19. The method according to claim 18, wherein the video stream to be output includes picture data and / or audio data corresponding to the current running window of the target cloud application; Correspondingly, the video stream to be output is transmitted to the client. include: The picture data is encoded according to a picture encoding function, and / or the audio data is encoded using an audio encoding function.

20. The method according to claim 14, wherein the receiving a window processing request submitted by a client to a cloud application of a cloud desktop, include: Receiving a multi-window processing request submitted by a client for at least two cloud applications of a cloud desktop; Accordingly, intercepting the currently running window screen of the cloud application based on the window processing request includes: At least two currently running window screens of the at least two cloud applications are intercepted based on the multi-window processing request.

21. The method according to claim 14, wherein the receiving the window processing request submitted by the client to the cloud application of the cloud desktop, include: Receive a window processing request submitted by a client for a single cloud application of a cloud desktop; Accordingly, intercepting the currently running window screen of the cloud application based on the window processing request includes: A currently running window screen of the single cloud application is captured based on the window processing request.

22. A window processing device, applied to a cloud server, include: A receiving module, configured to receive a window processing request submitted by a client for a cloud application of a cloud desktop, wherein the window processing request is sent based on a window processing instruction submitted by a user for a cloud application of a cloud desktop, and the window processing request carries the number of foreground application windows and / or background application windows and identification information of each window; An interception module, configured to intercept the running window screen of the foreground application window and / or the background application window based on the window processing request; The return module is configured to return the running window images to the client as virtual windows through a single video stream.

23. A window processing method, applied to a client, include: Based on the window processing instruction submitted by the user for the cloud application of the cloud desktop, a window processing request of the cloud application is sent to the cloud server, wherein the window processing request carries the number of foreground application windows and / or background application windows and identification information of each window; Receiving a running window screen of a foreground application window and / or a background application window intercepted by the cloud server based on the window processing request, wherein the window screen is transmitted to the client via a single-channel video stream; In response to the window processing instruction, the window images are respectively used as virtual windows, and the virtual windows are displayed according to a pre-created window processing style layout.

24. The method according to claim 23, further comprising: include: According to the activation instruction submitted by the user for the target virtual window, a window activation request is sent to the cloud server, wherein the window activation request carries identification information of the target virtual window.

25. The method according to claim 24, further comprising: include: receiving a video stream to be output returned by the cloud server, wherein the video stream to be output corresponds to a target cloud application, the target cloud application corresponds to the target virtual window, and the target virtual window is determined according to the identification information; The video stream to be output is displayed as the foreground of the real window display screen of the target cloud application.

26. The method according to claim 25, further comprising: include: Receiving the encoded data returned by the cloud server, the encoded data being generated by the cloud server encoding the video stream to be output; The encoded data is decoded to obtain the window running screen and display it in the foreground.

27. The method according to claim 23, wherein the window processing request of the cloud application is sent to the cloud server based on the window processing instruction submitted by the user for the cloud application of the cloud desktop, include: Based on the multi-window processing instruction submitted by the user for at least two cloud applications of the cloud desktop, sending a multi-window processing request for the at least two cloud applications to the cloud server; Correspondingly, the receiving the currently running window screen of the cloud application intercepted by the cloud server based on the window processing request includes: Receive at least two currently running window images of the at least two cloud applications intercepted by the cloud server based on the multi-window processing request.

28. The method according to claim 23, wherein the window processing request of the cloud application is sent to the cloud server based on the window processing instruction submitted by the user for the cloud application of the cloud desktop, include: Based on the window processing instruction submitted by the user for the single cloud application of the cloud desktop, sending a window processing request of the single cloud application to the cloud server; Correspondingly, the receiving the currently running window screen of the cloud application intercepted by the cloud server based on the window processing request includes: A currently running window screen of the single cloud application is received, which is intercepted by the cloud server based on the window processing request.

29. A window processing device, applied to a client, include: A sending module, configured to send a window processing request of the cloud application to the cloud server based on a window processing instruction submitted by a user for the cloud application of the cloud desktop, wherein the window processing request carries the number of foreground application windows and / or background application windows and identification information of each window; A receiving module, configured to receive a running window screen of a foreground application window and / or a background application window intercepted by the cloud server based on the window processing request, wherein the window screen is transmitted to the client via a single-channel video stream; The display module is configured to respond to the window processing instruction, use the window images as virtual windows respectively, and display the virtual windows according to the pre-created window processing style layout.

30. A computing device, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the method described in any one of claims 14 to 21 or 23 to 28.

31. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the method described in any one of claims 14 to 21 or 23 to 28.

Images