Cloud application display method, device, storage medium, program and system

Abstract

The embodiment of the invention provides a cloud application display method and device, a storage medium, a program and a system, and the method comprises the steps that a cloud application client sends the layout information of a local first desktop to a cloud application server, and the cloud application server sets a second desktop and an available area of the second desktop which are the same as the first desktop in size and the same as the first desktop in position and size, and generates a virtual taskbar in the second desktop. When the cloud application determines to enter the full-screen display state, the first video window is displayed in the second desktop in a full-screen mode, and the virtual taskbar is shielded by the first video window. And the cloud application server sends the layout information of the first video window and the first video picture to the cloud application client, so that the cloud application client displays the first video window and the first video picture in the first desktop in a full-screen manner according to the layout information of the first video window, thereby realizing accurate full-screen display of the cloud application at the cloud application client side.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a cloud application display method, device, storage medium, program and system. Background Art

[0002] With the continuous emergence of various cloud applications such as cloud gaming and cloud rendering, users can use these cloud applications for office work, entertainment, and other activities at a lower cost and in a more flexible and convenient manner. They do not need to purchase expensive terminal devices with greater computing power (i.e., host devices) or install various required applications on local terminal devices, especially those that consume a lot of computing power. They can use these cloud applications by simply using the corresponding cloud services provided by cloud service providers. Cloud service providers have abundant hardware resources and can use virtualization technologies such as virtualization to create a large number of virtual machines (i.e., cloud servers) on which they can deploy various cloud services for users to use.

[0003] For example, to use cloud gaming, a user needs to install the corresponding cloud application client on their local terminal device and the corresponding cloud application server installed on the cloud server, which also runs the cloud application. Users can remotely access the cloud application through the transmission link between the cloud application client and the cloud application server to use the cloud application.

[0004] Both the cloud server and the user's local terminal device contain an operating system, and the operating systems on both sides may be the same or different. Each operating system includes a desktop application that displays the corresponding desktop. The desktop layout of different operating systems may differ. For example, some operating systems have a taskbar and menu bar located in one location on the desktop, while other operating systems have a taskbar located in another location on the desktop. Generally, cloud applications are required to support cloud application clients running any operating system and connect to cloud application servers running any operating system. Furthermore, cloud applications support application window streaming. In this scenario, the cloud application server transmits the cloud application interface content to the cloud application client via a video stream for display. The cloud application video window (including its size, position, and layout, as well as the video image within the video window) must be displayed intact on the cloud application client. To achieve a better user experience, the desktop layout (including size and available area) of the operating system on the cloud server and the operating system on the terminal device must be synchronized. Different synchronization methods will affect the user experience of the video window in different display states. In particular, when the video window needs to be displayed full screen, the cloud application client may not provide an accurate and good user experience. Summary of the Invention

[0005] Embodiments of the present invention provide a cloud application display method, device, storage medium, program, and system to ensure that users obtain a good full-screen experience when using cloud applications.

[0006] In a first aspect, an embodiment of the present invention provides a cloud application display method, which is applied to a cloud application server. The method includes:

[0007] Receiving layout information of a first desktop sent by a cloud application client, where the first desktop corresponds to the cloud application client, and the layout information of the first desktop includes a size of the first desktop and a position and size of an available area in the first desktop;

[0008] Setting the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and setting the same available area in the second desktop according to the position and size of the available area in the first desktop;

[0009] generating a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, wherein the virtual taskbar is used to occupy the non-available area in the second desktop;

[0010] In response to the cloud application determining that the first video window is displayed in full screen in the second desktop when entering a full-screen display state, the first video screen contained in the first video window and the layout information of the first video window are sent to the cloud application client, so that the cloud application client displays the first video window and the first video screen in full screen in the first desktop according to the layout information of the first video window, wherein the layout information of the first video window indicates entering a full-screen display state, and the virtual taskbar is blocked by the first video window.

[0011] In a second aspect, an embodiment of the present invention provides a cloud application display device, which is applied to a cloud application server, and the device includes:

[0012] A first receiving module is configured to receive layout information of a first desktop sent by a cloud application client, where the first desktop corresponds to the cloud application client, and the layout information of the first desktop includes a size of the first desktop and a position and size of an available area in the first desktop;

[0013] a desktop setting module, configured to set the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and to set the same available area in the second desktop based on the position and size of the available area in the first desktop; and to generate a virtual taskbar in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop, the virtual taskbar being used to occupy the non-available area in the second desktop;

[0014] The first sending module is used to display the first video window in full screen in the second desktop in response to the cloud application determining to enter a full-screen display state, and send the first video picture contained in the first video window and the layout information of the first video window to the cloud application client, so that the cloud application client displays the first video window and the first video picture in full screen in the first desktop according to the layout information of the first video window, wherein the layout information of the first video window indicates entering a full-screen display state, and the virtual taskbar is blocked by the first video window.

[0015] In a third aspect, an embodiment of the present invention provides a cloud application display method, applied to a cloud application client, the method comprising:

[0016] In response to a user's launch operation of a cloud application, layout information of a local first desktop is sent to a cloud application server, where the layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop; the layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop based on the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop, where the virtual taskbar is used to occupy the non-available area in the second desktop;

[0017] receiving a first video screen and layout information of the first video window sent by the cloud application server, wherein the first video window includes the first video screen, and the layout information of the first video window indicates entering a full-screen display state, wherein the layout information of the first video window is obtained by the cloud application server in response to the cloud application displaying the first video window in full screen on the second desktop, and the virtual taskbar is obscured by the first video window;

[0018] The first video window and the first video picture are displayed in full screen on the first desktop according to the layout information of the first video window.

[0019] In a fourth aspect, an embodiment of the present invention provides a cloud application display device, applied to a cloud application client, the device comprising:

[0020] a second sending module, configured to send, in response to a user's operation of launching a cloud application, layout information of a local first desktop to a cloud application server, wherein the layout information of the first desktop includes the size of the first desktop and the position and size of an available area in the first desktop; the layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, to set the same available area in the second desktop according to the position and size of the available area in the first desktop, and to generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, wherein the virtual taskbar is used to occupy the non-available area in the second desktop;

[0021] a second receiving module, configured to receive a first video screen and layout information of the first video window sent by the cloud application server, wherein the first video window contains the first video screen, and the layout information of the first video window indicates entering a full-screen display state, wherein the layout information of the first video window is obtained by the cloud application server in response to the cloud application displaying the first video window in full screen on the second desktop, and the virtual taskbar is obscured by the first video window;

[0022] A video display module is used to display the first video window and the first video picture in full screen on the first desktop according to the layout information of the first video window.

[0023] In a fifth aspect, an embodiment of the present invention provides a cloud application display method, which is applied to a cloud application server, and the method includes:

[0024] Receiving layout information of a first desktop sent by a cloud application client, where the first desktop corresponds to the cloud application client, and the layout information of the first desktop includes a size of the first desktop and a position and size of an available area in the first desktop;

[0025] Setting the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and setting the same available area in the second desktop according to the position and size of the available area in the first desktop;

[0026] generating a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, wherein the virtual taskbar is used to occupy the non-available area in the second desktop;

[0027] In response to the cloud application displaying the video window in the second desktop, the video screen contained in the video window and the layout information of the video window are sent to the cloud application client, so that the cloud application client displays the video window and the video screen in the first desktop according to the layout information of the video window.

[0028] In a sixth aspect, an embodiment of the present invention provides a cloud application display method, applied to a cloud application client, the method comprising:

[0029] In response to a user's launch operation of a cloud application, layout information of a local first desktop is sent to a cloud application server, where the layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop; the layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop based on the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop, where the virtual taskbar is used to occupy the non-available area in the second desktop;

[0030] Receiving a video screen and layout information of a video window sent by the cloud application server, wherein the video window includes the video screen;

[0031] The video window and the video screen are displayed on the first desktop according to the layout information of the video window.

[0032] In the seventh aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein, executable code is stored on the memory, and when the executable code is executed by the processor, the processor can at least implement the cloud application display method described in the above aspects.

[0033] In an eighth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the cloud application display method described in the above aspects.

[0034] In a ninth aspect, an embodiment of the present invention provides a computer program product, which includes a computer program, and when the computer program product is executed by a processor of an electronic device, enables the processor to at least implement the cloud application display method described in the above aspects.

[0035] In a tenth aspect, an embodiment of the present invention provides a cloud application display system, comprising: a cloud application and a cloud application server located in a cloud server, and a cloud application client located in a terminal device;

[0036] The cloud application client is configured to send, in response to a user's operation of starting a cloud application, layout information of a local first desktop to a cloud application server, wherein the layout information of the first desktop includes the size of the first desktop and the position and size of an available area in the first desktop; and receive layout information of a first video screen and a first video window sent by the cloud application server, and display the first video window and the first video screen in full screen on the first desktop according to the layout information of the first video window, wherein the layout information of the first video window indicates entering a full-screen display state, and the first video window contains the first video screen;

[0037] The cloud application server is configured to set the size of a second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and to set the same available area in the second desktop according to the position and size of the available area in the first desktop, and to generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, wherein the virtual taskbar is used to occupy the non-available area in the second desktop; and in response to the cloud application displaying the first video window in full screen in the second desktop, sending the layout information of the first video window and the first video picture to the cloud application client, wherein the virtual taskbar is obscured by the first video window;

[0038] The cloud application is used to display the first video window in full screen on the second desktop when it is determined to enter the full screen display state.

[0039] The cloud application data transmission system provided by an embodiment of the present invention includes a cloud application and a cloud application server located in a cloud server, and a cloud application client located in a terminal device. When a user launches the cloud application, the cloud application client sends the layout information of the local first desktop to the cloud application server, so that the cloud application server synchronizes the layout of the local second desktop with the first desktop based on the layout information of the first desktop. Specifically, the layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The cloud application server sets the size of the second desktop to be the same as the size of the first desktop, thereby maintaining the same size between the cloud server desktop and the terminal device desktop. Furthermore, the cloud application server sets the same available area in the second desktop based on the position and size of the available area in the first desktop. Furthermore, based on the size of the second desktop and the position and size of the available area in the second desktop, a virtual taskbar is generated in the second desktop to occupy the non-available area in the second desktop. In fact, the position and size of the virtual taskbar in the second desktop are consistent with the position and size of the non-available area in the first desktop. This completes the synchronization of the desktops of the cloud application server and the cloud application client, aligning the coordinate systems of the two desktops.

[0040] Afterwards, based on the capture of the full-screen display operation triggered by the user, the cloud application determines that it has entered the full-screen display state and displays the first video window in full screen on the second desktop. At this time, the virtual taskbar is blocked by the first video window, that is, the virtual taskbar is in a hidden state. After the cloud application server perceives that the virtual taskbar is in a hidden state, it determines that the cloud application has entered the full-screen display state and sends the first video image contained in the first video window and the layout information of the first video window to the cloud application client. The layout information of the first video window indicates that the full-screen display state has been entered. The cloud application client displays the first video window and the first video image in full screen on the first desktop based on the layout information of the first video window.

[0041] In the above solution, based on the setting of the virtual taskbar on the cloud application server, the cloud application server can sense that the cloud application has entered the full-screen display state, and the full-screen display state is independently controlled by the cloud application, giving users a full-screen experience that is completely consistent with using cloud applications and local applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1A schematic diagram of a desktop layout of a traditional cloud application server and a cloud application client provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of a cloud application display system provided by an embodiment of the present invention;

[0045] Figure 3 A flowchart of a full-screen display method for cloud applications provided by an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of a new desktop layout of a cloud application server and a cloud application client provided by an embodiment of the present invention;

[0047] Figure 5 A flowchart of a full-screen display method for cloud applications provided by an embodiment of the present invention;

[0048] Figure 6 A flowchart of a full-screen display method for cloud applications provided by an embodiment of the present invention;

[0049] Figure 7 A schematic diagram of a full-screen display method for cloud applications provided by an embodiment of the present invention;

[0050] Figure 8 A schematic diagram of the structure of a full-screen display device for cloud applications provided by an embodiment of the present invention;

[0051] Figure 9 A schematic diagram of the structure of a full-screen display device for cloud applications provided by an embodiment of the present invention;

[0052] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0054] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0055] The following describes some embodiments of the present invention in detail with reference to the accompanying drawings. The following embodiments and features thereof may be combined with one another unless they conflict with each other. Furthermore, the sequence of steps in the following method embodiments is provided for illustrative purposes only and is not intended to be a strict limitation.

[0056] First, some terms involved in the embodiments of the present invention are introduced.

[0057] Cloud applications: Virtualized cloud applications, that is, applications running on cloud servers, such as cloud gaming, cloud rendering, and other applications.

[0058] Cloud server: The cloud-side environment where cloud applications run. The cloud server contains the cloud-side operating system and integrates the cloud application server.

[0059] Cloud application server: Both the cloud application server and the cloud application run in the cloud server. The cloud application server is responsible for streaming the video window of the cloud application and injecting the operation behavior information input by the user from the cloud application client into the cloud-side operating system or cloud application.

[0060] Cloud application client: This runs on the user's local terminal device, which includes the client-side operating system. The cloud application client is responsible for displaying the video in the cloud application's streamed video window and collecting user input and sending it to the cloud application server.

[0061] Coordinate system: The system corresponding to the coordinates of the mouse or the video window of the cloud application on the desktop of the cloud application client side or the desktop of the cloud application server side.

[0062] Available area: The desktop includes areas such as the taskbar and menu bar, which are not available for cloud applications. Excluding the functional areas occupied by the operating system, the remaining area on the desktop is the available area where video windows can be displayed.

[0063] Virtual taskbar: It only occupies the corresponding area of ​​the desktop on the cloud application server side, so that cloud applications that are not displayed in full screen cannot use this area. It does not need to include UI screens like a real taskbar and provide various functions such as displaying application icons and various function controls.

[0064] Full-screen display: This means that the video window fills the entire desktop (that is, the entire screen), blocking non-usable areas such as the taskbar and menu bar on the desktop.

[0065] Maximized display: The video window fills the available area of ​​the desktop, but does not block non-available areas such as the taskbar and menu bar.

[0066] Figure 1 A schematic diagram of a desktop layout of a traditional cloud application server and a cloud application client provided by an embodiment of the present invention is shown as follows: Figure 1 As shown, assuming that the cloud server where the cloud application server is located has an operating system 1 installed, and the terminal device where the cloud application client is located has an operating system 2 installed, the layout of the desktop corresponding to operating system 1 (for ease of description, called the server desktop) and the desktop corresponding to operating system 2 (for ease of description, called the client desktop) is as follows Figure 1 As shown in: it is assumed that the server desktop includes a taskbar A located in the lower part of the desktop, and the client desktop includes a taskbar B and a menu bar C located in the left part and the upper part of the desktop respectively.

[0067] Unlike the virtual taskbar described above, Taskbar A, Taskbar B, and Menu Bar C here are all real functional areas within the corresponding desktop. These taskbars and menu bars can contain various functional controls for the user, including but not limited to time, calendar, search, browser, and other functional controls. Taskbar A, Taskbar B, and Menu Bar C are all non-usable areas of the corresponding desktop, meaning they are not occupied when the application is not in full-screen mode.

[0068] In cloud application scenarios, both local cloud application clients and cloud application servers in the cloud support multiple operating systems. For example, local cloud application clients include Windows, Linux, Android, and iOS clients. Similarly, cloud application servers can be divided into Windows, Linux, Android, and iOS servers. In cloud application scenarios, cloud application clients that support any operating system are generally required to connect to cloud application servers running any operating system.

[0069] Cloud applications support application window streaming. Streaming refers to the processing of video streams, so cloud applications stream the application screen into a video screen and display it in a video window. In this scenario, the cloud application server encodes the cloud application video screen using a streaming protocol and sends it to the cloud application client for decoding and display. The cloud application video window size, position, and other layout information, as well as the video screen in the video window, will be displayed intact on the cloud application client. For a better user experience, you need to synchronize the desktop layout of the server and client desktops. The desktop layout information includes the desktop size and the location and size of the available area within it. The desktop size can be described by resolution and dots per inch (dpi).

[0070] If the desktop layout is not synchronized, for example, Figure 1 As shown in , the cloud application client and the cloud application server each have different desktop layouts. So it is conceivable that if the cloud application displays a video window at a certain location in the available area of ​​the server desktop, when the cloud application server sends the layout information of the video window (the location and size of the video window) to the cloud application client, the cloud application client needs to locate the same location coordinates on the local client desktop based on the location of the video window to determine the video window of the same size. Figure 1 As shown in , the video window's positioning on the client desktop may overlap taskbar B on the left. However, for the cloud application client, the video window can only overlap non-usable areas such as taskbar B and menu bar C when the cloud application server notifies the cloud application to enter full-screen display. If the cloud application client displays the video window over taskbar B, this will violate normal display logic and cause an error. If the video window is displayed without overlapping taskbar B, the user on the client desktop will only see part of the video window because the other part is obscured by taskbar B, resulting in a poor user experience.

[0071] Therefore, in cloud application scenarios, it is necessary to synchronize the layout of the server desktop and the client desktop.

[0072] As mentioned above, desktop layouts vary across operating systems. For example, Linux has both a menu bar and a taskbar, while Windows has only a taskbar. Different desktop layouts primarily affect the location and size of the available area on the desktop, so a method is needed to synchronize the available areas.

[0073] One solution is to use the size of the client desktop's available area as the server desktop's size. This means the server desktop does not include non-available areas such as the taskbar and menu bar. This ensures synchronization between the server desktop's available area and the client desktop's available area, aligning the coordinate systems. However, this approach presents a problem: the cloud application server cannot distinguish between maximized and full-screen displays of the cloud application, and therefore must default to maximized display. Specifically, when a cloud application maximizes or displays a video window in full screen, since the video window now occupies the entire server desktop, the cloud application server cannot distinguish whether the video window is maximized or full-screen, and must simply assume it is maximized. After the video window is streamed and encoded and transmitted to the cloud application client, the decoded display results in the video window being displayed maximized on the client desktop, meaning the taskbar / menu bar on the client desktop still exists. That is to say, even if the user triggers the full-screen display operation of the cloud application through the cloud application client, the final display result of the cloud application client is: the video window of the cloud application is displayed in a maximized manner, which is contrary to the user's intention and leads to a poor user experience.

[0074] For some full-screen applications, such as full-screen games and full-screen video playback, if the cloud application client side is only maximized, it will bring about a disconnect between the cloud application usage experience and the experience of using the same application locally. Users may be confused because they have clearly enabled the full-screen display function provided by the cloud application (for example, clicking the full-screen display button provided in the video window of the cloud application), but the full-screen display effect is not achieved.

[0075] It is understandable that, under the aforementioned synchronization method between the client desktop and the server desktop, when the cloud application displays a video window on the server desktop in a non-full-screen manner (for example, maximized, or displaying a video window smaller than the server desktop size on the server desktop), the cloud application server transmits the current video window layout information to the cloud application client. When the cloud application client displays the video window accordingly, it can obtain a video window display result consistent with that of the cloud application server. Therefore, under the aforementioned synchronization method between the client desktop and the server desktop, only the full-screen display of the cloud application may be problematic.

[0076] In view of this, in an embodiment of the present invention, a virtual taskbar is created in the server desktop to occupy the non-available area, so that the cloud application server can perceive the cloud application entering the full-screen display state, and the full-screen display state is controlled independently by the cloud application, giving users a full-screen experience that is completely consistent with using cloud applications and local applications.

[0077] The following describes a data transmission system and a full-screen display method for cloud applications provided by embodiments of the present invention.

[0078] Figure 2 A schematic diagram of a cloud application display system provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the system includes: a cloud application and a cloud application server located in a cloud server, and a cloud application client located in a terminal device.

[0079] A cloud server is a virtual machine in the cloud. Cloud applications and cloud application servers can run on the cloud server.

[0080] In an embodiment of the present invention, the cloud application client and the cloud application server are both service programs that provide services for the cloud application. The services provided are mainly to stream the application screen of the cloud application into a video screen: encoding it into a video stream to be displayed in the video window of the cloud application, and transmitting the video window containing the corresponding video screen to the cloud application client so that the video window can be decoded and displayed by the cloud application client.

[0081] In real-world applications, data transmission between cloud application servers and cloud application clients must conform to a predefined streaming protocol. The data transmitted from the cloud application server to the cloud application client primarily consists of the video window containing the video image. The data transmitted from the cloud application client to the cloud application server primarily contains user input, such as a single click or double click at a specific coordinate location.

[0082] Cloud applications are applications running on cloud servers, such as gaming applications, audio and video applications, etc. Cloud applications provide some video window manipulation functions, such as full-screen display, maximized display, minimized display, etc.

[0083] In fact, if Figure 2 As shown, the cloud server also includes a cloud operating system, which has a corresponding desktop Z1, and the terminal device also includes a user-side operating system, which has a corresponding desktop Z2.

[0084] Based on the above display system, combined with Figure 3 Embodiments such as these introduce full-screen display methods for cloud applications.

[0085] Figure 3 A flowchart of a full-screen display method for cloud applications provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the method may include the following steps:

[0086] 301. In response to a user's start-up operation on a cloud application, the cloud application client sends layout information of a first desktop to a cloud application server.

[0087] 302. The cloud application server sets a second desktop of the same size as the first desktop, and sets the same available area in the second desktop according to the position and size of the available area in the first desktop.

[0088] 303. The cloud application server generates a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop.

[0089] 304. The cloud application determines to enter a full-screen display state and displays the first video window in full screen on the second desktop.

[0090] 305. The cloud application server sends the first video image and the layout information of the first video window to the cloud application client. The layout information of the first video window indicates entering a full-screen display state.

[0091] 306. The cloud application client displays the first video window and the first video image in full screen on the first desktop according to the layout information of the first video window.

[0092] In actual applications, the user can first log in to the cloud application client on the terminal device, thereby triggering the establishment of a communication connection between the cloud application client and the cloud application server. After that, the user can click the cloud application icon to trigger the cloud application to start the cloud application. At this time, the cloud application client will send the layout information of the first desktop corresponding to the local operating system to the cloud application server, so that the cloud application server can synchronize the desktop.

[0093] Among them, the layout information of the first desktop includes the size of the first desktop, the position and size of the available area in the first desktop. The cloud application server sets the size of the local second desktop to be the same as the size of the first desktop, and sets the same available area in the second desktop according to the position and size of the available area in the first desktop. In addition, the cloud application server generates a virtual taskbar in the second desktop that occupies the non-available area in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop. In this way, the synchronization of the available area of ​​the second desktop with the available area of ​​the first desktop is achieved, that is, the synchronization of the desktop layout of the cloud application server and the cloud application client is achieved.

[0094] For ease of understanding, combined Figure 4 This shows the synchronization results of the desktop layout between the cloud application server and the cloud application client. Figure 4 As shown in , assume that the size of the first desktop is: W*H, where the first desktop contains a taskbar X and a menu bar Y located in location area L1 and location area L2. The taskbar X and menu bar Y are the actual taskbar and menu bar on the first desktop. The remaining area of ​​the first desktop, excluding the area where the taskbar X and menu bar Y are located, is the available area Z.

[0095] After the cloud application client sends the layout information of the first desktop to the cloud application server, the cloud application server sets the size of the local second desktop to be consistent with the size of the first desktop: W*H, and locates the same location area in the second desktop as the available zone Z of the second desktop based on the position and size of the available zone Z.

[0096] It should be noted that since the original second desktop corresponding to the operating system of the cloud server where the cloud application server resides may include a real taskbar, menu bar, and other non-usable areas, in this embodiment of the present invention, the cloud server operating system needs to first delete the taskbar and menu bar, for example, by ending the display thread of the taskbar and menu bar, so that the original second desktop no longer contains the taskbar, menu bar, and other non-usable areas. Afterwards, the layout of the second desktop is reset according to the layout of the first desktop.

[0097] In addition to setting up the aforementioned availability zone Z on the second desktop, the cloud application server also determines the number and location of virtual taskbars based on the size of the second desktop and the location and size of the availability zone on the second desktop. In practice, the virtual taskbars set up on the second desktop correspond to the taskbar X and menu bar Y included in the first desktop, and are used to set up non-availability zone placeholders of the same size and location on the second desktop.

[0098] like Figure 4 As shown in , the cloud application server finds that there are remaining location areas above and to the left of the available area Z of the second desktop. Then it can be determined that the number of virtual taskbars to be generated is two, and the positions correspond to these two free location areas: virtual taskbar a and virtual taskbar b.

[0099] In actual applications, since the virtual taskbar only serves as a placeholder, the virtual taskbar a and the virtual taskbar b can be implemented as transparent layers, or UI screens of a certain set style, which do not need to include various functional controls such as taskbar X and menu bar Y.

[0100] It should be noted that since the above-mentioned layout needs to be modified on the second desktop of the operating system in the cloud server, the operating system needs to support the function of modifying the desktop layout, such as opening relevant interfaces for setting the desktop layout for the cloud application server to set the above-mentioned layout.

[0101] like Figure 4 As shown in , after the cloud application server reconfigures the layout of the second desktop, it has the same desktop layout as the first desktop corresponding to the cloud application client: the same desktop size, the same location and size of the usable area, and the same location and size of the non-usable area. This achieves synchronization between the second desktop and the first desktop, that is, alignment of the coordinate systems of the two desktops.

[0102] After completing the synchronization of the above desktops, the cloud application server can start the cloud application and notify the cloud application of the layout information of the second desktop so that the cloud application displays the application screen according to the layout of the second desktop.

[0103] Here we assume that the default startup mode of the cloud application is the full-screen display startup mode. Then, when the cloud application server starts the cloud application, the cloud application enters the full-screen display state, and the cloud application displays the application window containing the current application screen in full-screen display on the second desktop: the first video window. When the cloud application server perceives that the cloud application has entered the full-screen display state, that is, when it perceives that the virtual taskbar is blocked (i.e., hidden), on the one hand, it performs streaming processing of the application screen: the application screen in the first video window is encoded into the first video screen, and the first video screen is sent to the cloud application client. On the other hand, it is also necessary to determine the layout information of the first video window, so as to send the layout information of the first video window to the cloud application client together, wherein the layout information of the first video window indicates that the cloud application has entered the full-screen display state. Optionally, the layout information of the first video window may only include the information that "the cloud application has entered the full-screen display state", or may also include the size and position information corresponding to the first video window in the second desktop.

[0104] In actual applications, optionally, the cloud application server can register a monitoring event for the virtual taskbar in the operating system of the cloud server to monitor whether the virtual taskbar is blocked, that is, whether it is in a hidden state. When the cloud application displays the first video window in full-screen display mode, the first video window will block the virtual taskbar, making it hidden. After the operating system of the cloud server detects that the virtual taskbar is in a hidden state, it will notify the cloud application server, so that the cloud application server perceives that the virtual taskbar is blocked (that is, in a hidden state), and thus determines that the cloud application enters a full-screen display state.

[0105] After receiving the first video image and the layout information of the first video window, the cloud application client decodes the first video image from the first video window and then displays the first video window and the first video image contained therein in full screen on the first desktop based on the layout information of the first video window. At this time, the first video window obscures non-usable areas of the first desktop, such as the taskbar and menu bar, creating an effect where the first video window fills the entire first desktop.

[0106] In the above solution, users can use the full-screen capabilities of the cloud application itself (for example, start it in full-screen display mode at startup), without changing the user's operating habits, and the user experience is better. During the operation of the cloud application, the layout of the second desktop on the cloud application server side is always consistent with the layout of the first desktop on the cloud application client side. Then, for a certain application, it will achieve completely consistent display effects when running as a cloud application in the cloud server and as a local application on the user terminal. Moreover, when the corresponding cloud application enters the full-screen display state, the virtual taskbar set in the second desktop can allow the cloud application server to perceive that the cloud application has entered the full-screen display state, so that the cloud application client can be accurately notified of the full-screen display state, allowing the user to obtain an accurate full-screen display experience.

[0107] Figure 5 A flowchart of a full-screen display method for cloud applications provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the method may include the following steps:

[0108] 501. In response to a user's start-up operation on a cloud application, the cloud application client sends layout information of a first desktop to a cloud application server.

[0109] 502. The cloud application server sets a second desktop with the same size as the first desktop, and sets the same available area in the second desktop according to the position and size of the available area in the first desktop.

[0110] 503. The cloud application server generates a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop.

[0111] 504. The cloud application determines to enter a full-screen display state and displays the first video window in full screen on the second desktop.

[0112] 505. The cloud application server sends the first video image and the layout information of the first video window to the cloud application client. The layout information of the first video window indicates entering a full-screen display state.

[0113] 506. The cloud application client displays the first video window and the first video image in full screen on the first desktop according to the layout information of the first video window.

[0114] 507. The cloud application determines to exit the full-screen display state and displays the third video window in the available area of ​​the second desktop.

[0115] 508. The cloud application server sends the third video image and the layout information of the third video window to the cloud application client. The layout information of the third video window indicates exiting the full-screen display state.

[0116] 509. The cloud application client displays the third video window and the third video image in the available area of ​​the first desktop according to the layout information of the third video window.

[0117] In this embodiment, steps 501-506 may refer to the description in the aforementioned embodiment and will not be repeated here.

[0118] This embodiment only emphasizes that when the cloud application exits the full-screen display state, the video window will be adjusted to a third video window that is smaller than or equal to the available area size of the second desktop, and the application screen that currently needs to be displayed will be displayed in the third video window. At this time, the virtual taskbar in the second desktop will not be blocked by the third video window, that is, the virtual taskbar is in an unhidden state.

[0119] When the cloud application server detects that the virtual taskbar is not blocked, it can determine that the cloud application has exited the full-screen display state, thereby determining the layout information corresponding to the third video window in the second desktop at this time (such as the size and position of the third video window), encoding the application screen in the third video window to obtain the third video screen, and transmitting the third video screen and the layout information of the third video window to the cloud application client. The cloud application client decodes the third video screen to obtain the third video screen, determines the same third video window in the available area of ​​the first desktop based on the layout information of the third video window, and displays the third video screen in the third video window.

[0120] The cloud application server's method of sensing that the virtual taskbar is not hidden is similar to the method described above for sensing that it is hidden, and will not be further elaborated. Furthermore, in actual applications, the user can trigger an operation to exit full-screen display within the cloud application client based on the full-screen function provided by the cloud application itself, so that the cloud application determines to exit full-screen display based on this operation.

[0121] Figure 6 A flowchart of a full-screen display method for cloud applications provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the method may include the following steps:

[0122] 601. In response to a user's start-up operation on a cloud application, the cloud application client sends layout information of a first desktop to a cloud application server.

[0123] 602. The cloud application server sets the size of the second desktop to be the same as that of the first desktop, and sets the same available area in the second desktop according to the position and size of the available area in the first desktop.

[0124] 603. The cloud application server generates a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop.

[0125] 604. The cloud application displays the second video window in the available area of ​​the second desktop.

[0126] As mentioned above, after the cloud application server sets the layout of the second desktop, it can start the cloud application. At this time, the cloud application determines a second video window in the available area of ​​the second desktop based on the position of the available area of ​​the second desktop, and displays the picture to be displayed after startup in the second video window. At this time, the virtual taskbar in the second desktop is not blocked by the second video window. The position and size of the second video window in the available area of ​​the second desktop can be determined by the cloud application according to the default configuration information, such as being displayed in the center area of ​​the available area with a size that occupies half of the available area.

[0127] 605. The cloud application server sends the second video screen and the layout information of the second video window to the cloud application client.

[0128] After the cloud application displays the second video window in the available area of ​​the second desktop and displays its application screen in the second video window, the cloud application server captures the display event, determines the layout information corresponding to the second video window in the available area of ​​the second desktop, encodes the application screen into the second video screen, and sends the second video screen and the layout information of the second video window to the cloud application client.

[0129] 606. The cloud application client displays the second video window and the second video image in the available area of ​​the first desktop according to the layout information of the second video window.

[0130] The cloud application client locates the same second video window in the available area of ​​the first desktop according to the layout information of the second video window, decodes the second video image, and displays it in the second video window.

[0131] 607. The cloud application client sends the first user operation behavior information to the cloud application server. The first user operation behavior information corresponds to the operation behavior of starting the full-screen display function provided by the cloud application itself in the second video window.

[0132] After the cloud application client displays the second video window, the user can use some of the display functions provided by the cloud application, including but not limited to full-screen display. For example, if the function list of the second video window includes a full-screen display function, the user clicking this function triggers the first user operation behavior information. Alternatively, if the cloud application also provides a function that allows double-clicking the video window area to trigger full-screen display, the first user operation behavior information can be a description of the user's double-clicking the display area of ​​the second video window.

[0133] It should be noted that the cloud application client does not identify what function the user's current operation behavior is operating. Therefore, the above-mentioned first user operation behavior information only describes the location coordinates and behavior actions (such as click, double-click) corresponding to the user's operation behavior on the first desktop.

[0134] 608. The cloud application server provides the first user's operation behavior information to the cloud application.

[0135] The cloud application server also does not have the ability to identify which function provided by the cloud application the first user's operation behavior information is used to operate, and only provides the first user's operation behavior information to the cloud application for identification.

[0136] In actual applications, the cloud application server can optionally send the first user operation behavior information directly to the cloud application. Alternatively, the cloud application server can inject the first user operation behavior information into the operating system of the cloud server so that the information is sent to the cloud application through the operating system. That is, the cloud application captures the first user operation behavior information through the operating system.

[0137] 609. When the cloud application determines that the first user operation behavior information matches the full-screen display startup behavior, the second video window is adjusted to the first video window, and the first video window is displayed in full screen on the second desktop.

[0138] For example, the first user operation behavior information describes that the user performed a click action at the (x, y) position coordinates of the first desktop. The cloud application determines that the position coordinates match the "full screen display" function button, and the click action matches the user's set action to start the full-screen display function. Then the cloud application determines to enter the full-screen display state, adjusts the previous second video window to the first video window, and displays the first video window in full screen on the second desktop, that is, adjusts the video window to the first video window that covers the second desktop.

[0139] 610. The cloud application server sends the first video image and the layout information of the first video window to the cloud application client, and the layout information of the first video window indicates entering a full-screen display state.

[0140] 611. The cloud application client displays the first video window and the first video image in full screen on the first desktop according to the layout information of the first video window.

[0141] 612. The cloud application client sends second user operation behavior information to the cloud application server. The second user operation behavior information corresponds to an operation behavior of exiting the full-screen display function provided by the cloud application itself in the first video window.

[0142] Similar to the above-mentioned first user operation behavior information, when the user triggers the operation behavior of exiting the full-screen display in the current full-screen display of the first video window (such as double-clicking the area of ​​the first video window, clicking the function button to exit the full-screen display), the cloud application client collects the corresponding second user operation behavior information (behavioral action and operation location coordinates).

[0143] 613. The cloud application server provides the second user's operation behavior information to the cloud application.

[0144] 614. When the cloud application determines that the second user operation behavior information matches the exit full-screen display behavior, the cloud application adjusts the first video window to a third video window to be displayed in the available area of ​​the second desktop.

[0145] Similarly, when the cloud application identifies that the second user's operation behavior information matches the full-screen display exit behavior, it determines to exit the full-screen display state and resizes the previous first video window into a smaller third video window to be displayed in the available area of ​​the second desktop. At this time, the virtual taskbar is not obscured by the third video window. The size and position of the third video window can be the default settings.

[0146] 615. The cloud application server sends the layout information of the third video screen and the third video window to the cloud application client.

[0147] The layout information of the third video window indicates exiting the full screen display state. In fact, the layout information of the third video window may include layout information such as the position and size of the third video window, and optionally, may further include an indicator for exiting the full screen display state.

[0148] 616. The cloud application client displays the third video window and the third video image in the available area of ​​the first desktop according to the layout information of the third video window.

[0149] In summary, the embodiments of the present invention allow users to utilize the cloud application's inherent full-screen capabilities (e.g., full-screen display at startup, a full-screen display function button provided by the cloud application, and double-clicking the video window area to enable full-screen display) without requiring users to change their full-screen operating habits, resulting in a better user experience. By setting up a virtual taskbar on the second desktop, the cloud application server can sense that the cloud application has entered a full-screen display state, thereby accurately notifying the cloud application client of this full-screen display state, allowing users to obtain an accurate full-screen display experience. This also aligns with the concept of the cloud application server controlling the cloud application client, eliminating the need for significant changes to the cloud application client, making the solution simpler to implement.

[0150] Figure 7 A schematic diagram of a full-screen display method for cloud applications provided by an embodiment of the present invention is shown in FIG. Figure 7As shown, in actual applications, the data transmission process of cloud applications may involve the following processes: synchronization of the desktop layout between the server and the client, streaming of application windows from the server to the client, transmission of user input from the client to the server, launching of a new full-screen application / entering of an old application into a full-screen display state, and exit of the application from the full-screen display state.

[0151] The desktop layout synchronization process between the server and the client refers to the desktop layout synchronization process between the cloud application server and the cloud application client. As mentioned above, this process includes: after the user opens the cloud application, the cloud application client collects the layout information of the client desktop (the first desktop mentioned above) and sends the layout information of the client desktop to the cloud application server. The cloud application server sets the size and available area of ​​the server desktop (the second desktop mentioned above) according to the size and available area of ​​the client desktop. The cloud application server creates a virtual taskbar and then starts the cloud application.

[0152] The process of streaming application windows from the server to the client refers to the process of transmitting video images from the cloud application server to the cloud application client. The process includes: the cloud application server captures the layout information (including window position, size, etc.) of the video window displayed by the cloud application on the server desktop, and sends the video window layout information to the cloud application client. The cloud application client displays the video window at the corresponding position on the client desktop based on the video window layout information (it can create a new identical video window at this position or move the original video window to this position, and synchronize the window size). The cloud application server encodes the image in the video window through the streaming protocol and sends the video image to the cloud application client so that the cloud application client displays the video image in the video window.

[0153] The process of transmitting user input from the client to the server refers to the process of the cloud application client sending the user input operation behavior information to the cloud application server. This process includes: the user performs an operation on the client desktop (such as clicking, moving the video window, etc. through a mouse, touch screen, etc.), and the cloud application client collects the operation behavior information (operation location, action) and sends it to the cloud application server. The cloud application server generates a corresponding operation event in the cloud operating system. The cloud application captures the operation event through the cloud operating system to update the layout (position, size) of the video window on the server desktop.

[0154] The process of starting a new full-screen application / the old application entering the full-screen display state refers to the processing process after the cloud application enters the full-screen display state. The process includes: the cloud application enters the full-screen display state (for example, the cloud application is started in full-screen display mode, or the user clicks the full-screen display function button provided by the cloud application on the client desktop or triggers a double-click operation to start the full-screen display function). The cloud application server perceives that the virtual taskbar is hidden, and notifies the cloud application client that the cloud application enters the full-screen display state. The cloud application client displays the video window in full screen on the client desktop. The detailed execution process of this process (such as the transmission of the video screen, etc.) can refer to the relevant description in the aforementioned embodiment, and only some key processes are illustrated here.

[0155] The process of an application exiting the full-screen display state refers to the processing process after the cloud application exits the full-screen display state. This process includes: the cloud application exits the full-screen display state, the cloud application server perceives that the virtual taskbar changes from a hidden state to a displayed state, and notifies the cloud application client that the cloud application exits the full-screen display state, thereby causing the cloud application client to exit the full-screen display of the video window. The detailed execution process of this process (such as the transmission of the video screen, etc.) can be referred to the relevant description in the aforementioned embodiment. Only some key processes are illustrated here.

[0156] In an optional embodiment, the present invention further provides a cloud application display method, which is applied to a cloud application server and includes the following steps:

[0157] Receiving layout information of a first desktop sent by a cloud application client, where the first desktop corresponds to the cloud application client, and the layout information of the first desktop includes a size of the first desktop and a position and size of an available area in the first desktop;

[0158] Set the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and set the same available area in the second desktop according to the position and size of the available area in the first desktop;

[0159] generating a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, wherein the virtual taskbar is used to occupy the non-available area in the second desktop;

[0160] In response to the cloud application displaying the video window in the second desktop, the video screen contained in the video window and the layout information of the video window are sent to the cloud application client, so that the cloud application client displays the video window and the video screen in the first desktop according to the layout information of the video window.

[0161] In an optional embodiment, the present invention further provides a cloud application display method, which is applied to a cloud application client and includes the following steps:

[0162] In response to a user's launch operation of a cloud application, the layout information of the local first desktop is sent to the cloud application server, where the layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop based on the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop, where the virtual taskbar is used to occupy the non-available area in the second desktop.

[0163] Receive the video image and layout information of the video window sent by the cloud application server, where the video window contains the video image;

[0164] The video window and the video screen are displayed on the first desktop according to the layout information of the video window.

[0165] From this, it can be seen that after the desktop synchronization of the cloud application server and the cloud application client is achieved based on the setting of the above-mentioned virtual taskbar, the video window of any size displayed on the second desktop by the cloud application server (whether in maximized display state, full-screen display state or other display state that only occupies the available area) and the video screen therein can be displayed consistently on the cloud application client.

[0166] The following describes in detail one or more embodiments of the cloud application full-screen display device of the present invention. Those skilled in the art will understand that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0167] Figure 8 A schematic diagram of the structure of a cloud application full-screen display device provided by an embodiment of the present invention, wherein the cloud application full-screen display device is applied to a cloud application server, such as Figure 8 As shown, the device includes: a first receiving module 11, a desktop setting module 12, and a first sending module 13.

[0168] The first receiving module 11 is used to receive layout information of a first desktop sent by a cloud application client, where the first desktop corresponds to the cloud application client and the layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop.

[0169] The desktop setting module 12 is used to set the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and to set the same available area in the second desktop according to the position and size of the available area in the first desktop; and to generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, wherein the virtual taskbar is used to occupy the non-available area in the second desktop.

[0170] The first sending module 13 is used to display the first video window in full screen in the second desktop in response to the cloud application determining to enter the full-screen display state, and send the first video picture contained in the first video window and the layout information of the first video window to the cloud application client, so that the cloud application client displays the first video window and the first video picture in full screen in the first desktop according to the layout information of the first video window, wherein the layout information of the first video window indicates entering the full-screen display state, and the virtual taskbar is blocked by the first video window.

[0171] Figure 8 The device shown can execute the steps executed by the cloud application server in the aforementioned embodiment. The detailed execution process and technical effects can be found in the description of the aforementioned embodiment and will not be repeated here.

[0172] Figure 9 A schematic diagram of the structure of a cloud application full-screen display device provided by an embodiment of the present invention, wherein the cloud application full-screen display device is applied to a cloud application client, such as Figure 9 As shown, the device includes: a second sending module 21, a second receiving module 22, and a video display module 23.

[0173] The second sending module 21 is used to send the layout information of the local first desktop to the cloud application server in response to the user's startup operation of the cloud application, where the layout information of the first desktop includes the size of the first desktop, the position and size of the available area in the first desktop; the layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop according to the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, where the virtual taskbar is used to occupy the non-available area in the second desktop.

[0174] The second receiving module 22 is used to receive the first video screen and the layout information of the first video window sent by the cloud application server, the first video window contains the first video screen, and the layout information of the first video window indicates entering the full-screen display state, wherein the layout information of the first video window is obtained by the cloud application server in response to the cloud application displaying the first video window in full screen on the second desktop, and the virtual taskbar is blocked by the first video window.

[0175] The video display module 23 is configured to display the first video window and the first video image in full screen on the first desktop according to the layout information of the first video window.

[0176] Figure 9 The device shown can execute the steps executed by the cloud application client in the aforementioned embodiment. The detailed execution process and technical effects can be found in the description of the aforementioned embodiment and will not be repeated here.

[0177] In one possible design, the above Figure 8 or Figure 9 The structure of the device shown can be realized as an electronic device. Figure 10 As shown, the electronic device may include: a processor 31, a memory 32, and a communication interface 33. The memory 32 stores executable code, and when the executable code is executed by the processor 31, the processor 31 can at least implement the cloud application full-screen display method provided in the above embodiment.

[0178] In addition, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the cloud application full-screen display method provided in the aforementioned embodiment.

[0179] The device embodiments described above are merely illustrative, wherein the network elements described as separate components may or may not be physically separate. Some or all of these modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art can understand and implement these embodiments without inventive effort.

[0180] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A cloud application display method, characterized in that: Applied to a cloud application server, the method includes: Receiving layout information of a first desktop sent by a cloud application client, where the first desktop corresponds to the cloud application client, and the layout information of the first desktop includes a size of the first desktop and a position and size of an available area in the first desktop; Setting the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and setting the same available area in the second desktop according to the position and size of the available area in the first desktop; generating a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, wherein the virtual taskbar is used to occupy the non-available area in the second desktop; In response to the cloud application determining that the first video window is displayed in full screen in the second desktop when entering a full-screen display state, the first video screen contained in the first video window and the layout information of the first video window are sent to the cloud application client, so that the cloud application client displays the first video window and the first video screen in full screen in the first desktop according to the layout information of the first video window, wherein the layout information of the first video window indicates entering a full-screen display state, and the virtual taskbar is blocked by the first video window.

2. The method according to claim 1, characterized in that The method further comprises: In response to the cloud application displaying the second video window in the available area of ​​the second desktop, the second video screen contained in the second video window and the layout information of the second video window are sent to the cloud application client, so that the cloud application client displays the second video window and the second video screen in the available area of ​​the first desktop according to the layout information of the second video window, wherein the virtual taskbar is not blocked by the second video window, and the second video window is displayed earlier than the first video window.

3. The method according to claim 2, characterized in that The method further comprises: Receiving first user operation behavior information sent by the cloud application client, where the first user operation behavior information corresponds to an operation behavior of starting a full-screen display function provided by the cloud application itself in the second video window; The first user operation behavior information is provided to the cloud application so that when the cloud application determines that the first user operation behavior information matches the full-screen display startup behavior, the second video window is adjusted to the first video window and the first video window is displayed in full screen in the second desktop.

4. The method according to claim 1, wherein The cloud application is started in a full-screen display mode.

5. The method according to claim 1, characterized in that The method further comprises: Receiving second user operation behavior information sent by the cloud application client, where the second user operation behavior information corresponds to an operation behavior of exiting the full-screen display function provided by the cloud application itself in the first video window; providing the second user operation behavior information to the cloud application, so that when the cloud application determines that the second user operation behavior information matches an exit full-screen display behavior, the cloud application adjusts the first video window to a third video window to be displayed in an available area of ​​the second desktop, wherein the virtual taskbar is not blocked by the third video window; The third video screen contained in the third video window and the layout information of the third video window are sent to the cloud application client, so that the cloud application client displays the third video window and the third video screen in the available area of ​​the first desktop according to the layout information of the third video window, wherein the layout information of the third video window indicates exiting the full-screen display state.

6. A cloud application display method, characterized in that: Applied to a cloud application client, the method includes: In response to a user's launch operation of a cloud application, layout information of a local first desktop is sent to a cloud application server, where the layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop; the layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop based on the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop, where the virtual taskbar is used to occupy the non-available area in the second desktop; receiving a first video screen and layout information of a first video window sent by the cloud application server, wherein the first video window includes the first video screen, and the layout information of the first video window indicates entering a full-screen display state, wherein the layout information of the first video window is obtained by the cloud application server in response to the cloud application displaying the first video window in full screen on the second desktop, and the virtual taskbar is obscured by the first video window; The first video window and the first video picture are displayed in full screen on the first desktop according to the layout information of the first video window.

7. The method according to claim 6, characterized in that The method further comprises: Receiving a second video screen and layout information of a second video window sent by the cloud application server, where the second video window includes the second video screen, wherein the layout information of the second video window is obtained by the cloud application server in response to the cloud application displaying the second video window in the available area of ​​the second desktop, the virtual taskbar is not blocked by the second video window, and the second video window is displayed earlier than the first video window; The second video window and the second video picture are displayed in the available area of ​​the first desktop according to the layout information of the second video window.

8. The method according to claim 7, characterized in that The method further comprises: Collecting first user operation behavior information triggered by the user in the second video window, where the first user operation behavior information corresponds to an operation behavior of starting a full-screen display function provided by the cloud application itself in the second video window; The first user operation behavior information is sent to the cloud application server, so that the cloud application server provides the first user operation behavior information to the cloud application, so that when the cloud application determines that the first user operation behavior information matches the full-screen display startup behavior, the second video window is adjusted to the first video window, and the first video window is displayed in full screen in the second desktop.

9. The method according to claim 6, characterized in that The method further comprises: Collecting second user operation behavior information triggered by the user in the first video window, where the second user operation behavior information corresponds to an operation behavior of exiting the full-screen display function provided by the cloud application itself in the first video window; sending the second user operation behavior information to the cloud application server, so that the cloud application server provides the second user operation behavior information to the cloud application, so that when the cloud application determines that the second user operation behavior information matches the exit full-screen display behavior, the first video window is adjusted to a third video window to be displayed in the available area of ​​the second desktop, wherein the virtual taskbar is not blocked by the third video window; Receiving the third video screen and layout information of the third video window sent by the cloud application server, wherein the layout information of the third video window indicates exiting the full-screen display state, and the third video window contains the third video screen; The third video window and the third video picture are displayed in the available area of ​​the first desktop according to the layout information of the third video window.

10. A cloud application display method, characterized in that: Applied to a cloud application server, the method includes: Receiving layout information of a first desktop sent by a cloud application client, where the first desktop corresponds to the cloud application client, and the layout information of the first desktop includes a size of the first desktop and a position and size of an available area in the first desktop; Setting the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and setting the same available area in the second desktop according to the position and size of the available area in the first desktop; generating a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, wherein the virtual taskbar is used to occupy the non-available area in the second desktop; In response to the cloud application displaying the video window in the second desktop, the video screen contained in the video window and the layout information of the video window are sent to the cloud application client, so that the cloud application client displays the video window and the video screen in the first desktop according to the layout information of the video window.

11. A cloud application display method, characterized in that: Applied to a cloud application client, the method includes: In response to a user's launch operation of a cloud application, layout information of a local first desktop is sent to a cloud application server, where the layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop; the layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop based on the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop, where the virtual taskbar is used to occupy the non-available area in the second desktop; Receiving a video screen and layout information of a video window sent by the cloud application server, wherein the video window includes the video screen; The video window and the video screen are displayed on the first desktop according to the layout information of the video window.

12. A cloud application display system, characterized in that: include: Cloud applications and cloud application servers located in cloud servers, and cloud application clients located in terminal devices; The cloud application client is configured to send, in response to a user's operation of starting a cloud application, layout information of a local first desktop to a cloud application server, wherein the layout information of the first desktop includes the size of the first desktop and the position and size of an available area in the first desktop; and receive layout information of a first video screen and a first video window sent by the cloud application server, and display the first video window and the first video screen in full screen on the first desktop according to the layout information of the first video window, wherein the layout information of the first video window indicates entering a full-screen display state, and the first video window contains the first video screen; The cloud application server is configured to set the size of a second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and to set the same available area in the second desktop according to the position and size of the available area in the first desktop, and to generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, wherein the virtual taskbar is used to occupy the non-available area in the second desktop; and in response to the cloud application displaying the first video window in full screen in the second desktop, sending the layout information of the first video window and the first video picture to the cloud application client, wherein the virtual taskbar is obscured by the first video window; The cloud application is used to display the first video window in full screen on the second desktop when it is determined to enter the full screen display state.

13. The system according to claim 12, characterized in that: The cloud application is further configured to display a second video window in an available area of ​​the second desktop, wherein the second video window is displayed earlier than the second video window, and the virtual taskbar is not blocked by the second video window; The cloud application server is further configured to send the second video screen and layout information of the second video window to the cloud application client, wherein the second video window includes the second video screen; The cloud application client is further configured to display the second video window and the second video screen in the available area of ​​the first desktop according to the layout information of the second video window.

14. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the cloud application display method as described in any one of claims 1 to 11.

15. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the cloud application display method according to any one of claims 1 to 11.

16. A computer program product, characterized in that include: A computer program, when executed by a processor of an electronic device, causes the processor to execute the cloud application display method according to any one of claims 1 to 11.

Images