Cloud application display method, device, storage medium, program, and system

Abstract

The embodiment of the application provides a cloud application display method, device, storage medium, program and system, which comprises the following steps: a cloud application client sends layout information of a local first desktop to a cloud application server; the cloud application server sets a second desktop and an available area of the second desktop, wherein the second desktop has the same size as the first desktop, and the available area has the same size and position as the first desktop; a virtual taskbar is generated in the second desktop; when the cloud application determines to enter a full-screen display state, the cloud application displays a first video window in the second desktop in full screen, and the virtual taskbar is blocked by the first video window; the cloud application server sends layout information and a first video picture of the first video window to the cloud application client, so that the cloud application client displays the first video window and the first video picture in full screen in the first desktop according to the layout information of the first video window, thereby realizing accurate full-screen display of the cloud application on the cloud application client side.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and in particular to a cloud application display method, device, storage medium, program, and system. Background Technology

[0002] With the emergence of various cloud applications such as cloud gaming and cloud rendering, users can utilize these applications for work, entertainment, and other activities at a lower cost and in a more flexible and convenient manner. They no longer need to purchase expensive, high-computing-power terminal devices (i.e., host devices) or install various applications on their local devices, especially those applications that consume significant computing power. They can simply use the corresponding cloud services provided by the cloud service provider. Cloud service providers possess abundant hardware resources and, through technologies such as virtualization, can create numerous virtual machines (i.e., cloud servers) on which various cloud services can be deployed for user access.

[0003] Taking cloud gaming as an example, to use this cloud application, users need to install the corresponding cloud application client on their local terminal device, and install the corresponding cloud application server on the cloud server, where the cloud application also runs. Users can remotely access the cloud application through the transmission link between the cloud application client and the cloud application server to use it.

[0004] Both cloud servers and users' local terminal devices contain operating systems, which may be the same or different. Each operating system comes with its own desktop applications for displaying the desktop, and the layout of these desktops may differ between operating systems. For example, some operating systems may have a taskbar and menu bar in a specific location, while another operating system may have a taskbar in a different location. Generally, cloud applications are required to support connections between cloud application clients and cloud application servers running on any operating system. Furthermore, cloud applications support streaming applications into a window format. In this scenario, the cloud application server transmits the application's interface content as a video stream to the cloud application client for display. The video window (including its size, position, layout, and the video content) needs to be displayed exactly as it appears on the cloud application client. To achieve a better user experience, it's necessary to synchronize the desktop layout (including size and availability) between the operating systems on the cloud server and the terminal device. Different synchronization methods will affect the user experience of the video window in different display states, especially when the video window needs to be displayed in full screen, which may result in an inaccurate and unsatisfactory user experience on the cloud application client. Summary of the Invention

[0005] This invention provides a cloud application display method, device, storage medium, program, and system to ensure that users have a good full-screen experience when using cloud applications.

[0006] In a first aspect, embodiments of the present invention provide a cloud application display method, applied to a cloud application server, the method comprising:

[0007] Receive layout information of a first desktop sent by a cloud application client. The first desktop corresponds to the cloud application client. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop.

[0008] The size of the second desktop corresponding to the cloud application server is set to be the same as that of the first desktop, and the same availability zone is set in the second desktop according to the position and size of the availability zone in the first desktop;

[0009] Based on the size of the second desktop and the position and size of the available area in the second desktop, a virtual taskbar is generated in the second desktop, and the virtual taskbar is used to occupy the non-available area in the second desktop;

[0010] In response to the cloud application determining that it has entered full-screen display mode, the first video window is displayed in full-screen mode on the second desktop. The first video frame contained in the first video window and the layout information of the first video window are sent to the cloud application client, so that the cloud application client displays the first video window and the first video frame in full-screen mode on the first desktop according to the layout information of the first video window. The layout information of the first video window indicates that it has entered full-screen display mode, and the virtual taskbar is obscured by the first video window.

[0011] Secondly, embodiments of the present invention provide a cloud application display device, applied to a cloud application server, the device comprising:

[0012] The first receiving module is used to receive the layout information of the first desktop sent by the cloud application client. The first desktop corresponds to the cloud application client. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop.

[0013] The desktop settings module is used to set the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and to set the same available area in the second desktop according to the position and size of the available area in the first desktop; and to generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, the virtual taskbar being used to occupy the non-available area in the second desktop;

[0014] The first sending module is configured to, in response to the cloud application determining that it has entered a full-screen display state, display the first video window in full-screen mode on the second desktop, and send the first video frame contained in the first video window and the layout information of the first video window to the cloud application client, so that the cloud application client displays the first video window and the first video frame in full-screen mode on the first desktop according to the layout information of the first video window, wherein the layout information of the first video window indicates that it has entered a full-screen display state, and the virtual taskbar is obscured by the first video window.

[0015] Thirdly, embodiments of the present invention provide a cloud application display method, applied to a cloud application client, the method comprising:

[0016] In response to a user's launch operation of a cloud application, the layout information of the local first desktop is sent to the cloud application server. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop according to the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop. The virtual taskbar is used to occupy the non-available area in the second desktop.

[0017] The system receives a first video frame and layout information of the first video window sent by the cloud application server. The first video window contains the first video frame, and the layout information of the first video window indicates that the system is in full-screen display mode. The layout information of the first video window is obtained by the cloud application server in response to the cloud application displaying the first video window in full-screen mode on the second desktop. The virtual taskbar is obscured by the first video window.

[0018] Based on the layout information of the first video window, the first video window and the first video screen are displayed in full screen on the first desktop.

[0019] Fourthly, embodiments of the present invention provide a cloud application display device, applied to a cloud application client, the device comprising:

[0020] The second sending module is used to respond to the user's launch operation of the cloud application by sending the layout information of the local first desktop to the cloud application server. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop according to the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop. The virtual taskbar is used to occupy the non-available area in the second desktop.

[0021] The second receiving module is used to receive the first video frame and the layout information of the first video window sent by the cloud application server. The first video window contains the first video frame, and the layout information of the first video window indicates that it has entered the full-screen display state. The layout information of the first video window is obtained by the cloud application server in response to the cloud application displaying the first video window in full screen on the second desktop. The virtual taskbar is obscured by the first video window.

[0022] The video display module is used to display the first video window and the first video frame in full screen on the first desktop according to the layout information of the first video window.

[0023] Fifthly, embodiments of the present invention provide a cloud application display method, applied to a cloud application server, the method comprising:

[0024] Receive layout information of a first desktop sent by a cloud application client. The first desktop corresponds to the cloud application client. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop.

[0025] The size of the second desktop corresponding to the cloud application server is set to be the same as that of the first desktop, and the same availability zone is set in the second desktop according to the position and size of the availability zone in the first desktop;

[0026] Based on the size of the second desktop and the position and size of the available area in the second desktop, a virtual taskbar is generated in the second desktop, and the virtual taskbar is used to occupy the non-available area in the second desktop;

[0027] In response to the cloud application displaying a video window on the second desktop, the video frame contained in the video window and the layout information of the video window are sent to the cloud application client, so that the cloud application client displays the video window and the video frame on the first desktop according to the layout information of the video window.

[0028] Sixthly, embodiments of the present invention provide a cloud application display method, applied to a cloud application client, the method comprising:

[0029] In response to a user's launch operation of a cloud application, the layout information of the local first desktop is sent to the cloud application server. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop according to the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop. The virtual taskbar is used to occupy the non-available area in the second desktop.

[0030] Receive video footage and video window layout information sent by the cloud application server, wherein the video window contains the video footage;

[0031] The video window and the video frame are displayed on the first desktop according to the layout information of the video window.

[0032] In a seventh aspect, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a communication interface; wherein, the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the cloud application display method described in the above aspects.

[0033] Eighthly, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the cloud application display method described in the above aspects.

[0034] Ninthly, embodiments of the present invention provide a computer program product comprising a computer program that, when executed by a processor of an electronic device, enables the processor to at least implement the cloud application display method described in the foregoing aspects.

[0035] In a tenth aspect, embodiments of the present invention provide a cloud application display system, comprising: a cloud application and a cloud application server located in a cloud server, and a cloud application client located in a terminal device;

[0036] The cloud application client is used to respond to the user's launch operation of the cloud application by sending the layout information of the local first desktop to the cloud application server. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. It also receives the layout information of the first video screen and the first video window sent by the cloud application server, and displays the first video window and the first video screen in full screen on the first desktop according to the layout information of the first video window. The layout information of the first video window indicates that it has entered the full screen display state, and the first video window contains the first video screen.

[0037] The cloud application server is configured to set the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and to set the same available area in the second desktop according to the position and size of the available area in the first desktop. Based on the size of the second desktop and the position and size of the available area in the second desktop, a virtual taskbar is generated in the second desktop, the virtual taskbar being used to occupy a non-available area in the second desktop. In response to the cloud application displaying the first video window in full screen on the second desktop, the server sends the layout information of the first video window and the first video frame to the cloud application client, wherein the virtual taskbar is obscured by the first video window.

[0038] The cloud application is used to display the first video window in full-screen mode on the second desktop when it is determined that the full-screen display state has been entered.

[0039] The cloud application data transmission system provided in this embodiment of the invention includes a cloud application and a cloud application server located on a cloud server, and a cloud application client located on a terminal device. When a user launches a cloud application, the cloud application client sends the layout information of its local first desktop to the cloud application server, enabling the cloud application server to synchronize the layout of its local second desktop with the first desktop based on the layout information of the first desktop. Specifically, the layout information of the first desktop includes the size of the first desktop and the position and size of the available area within the first desktop. The cloud application server sets the size of the second desktop to be the same as the size of the first desktop, that is, to keep the desktop size of the cloud server consistent with the desktop size of the terminal device. Furthermore, it sets the same available area in the second desktop based on the position and size of the available area in the first desktop, and generates a virtual taskbar in the second desktop to occupy the non-available area in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop. In fact, the position and size of this virtual taskbar in the second desktop are consistent with the position and size of the non-available area in the first desktop. This completes the synchronization of the desktops of the cloud application server and the cloud application client, aligning the coordinate systems of the two desktops.

[0040] Subsequently, based on the capture of the user's full-screen display action, the cloud application determines that it has entered full-screen display mode and displays the first video window in full-screen mode on the second desktop. At this time, the virtual taskbar is obscured by the first video window, meaning it is in a hidden state. After the cloud application server detects that the virtual taskbar is hidden, it determines that the cloud application has entered full-screen display mode and sends the first video frame contained in the first video window and the layout information of the first video window to the cloud application client. This layout information of the first video window indicates that it has entered full-screen display mode. The cloud application client then displays the first video window and the first video frame in full-screen mode on the first desktop according to the layout information of the first video window.

[0041] In the above solution, based on the virtual taskbar settings of the cloud application server, the cloud application server can sense when the cloud application enters full-screen display mode, and the full-screen display mode is autonomously controlled by the cloud application, giving users a full-screen experience that is completely consistent with using cloud applications and local applications. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1This is a schematic diagram of a traditional desktop layout for a cloud application server and a cloud application client, provided as an embodiment of the present invention.

[0044] Figure 2 This is a schematic diagram of a cloud application display system provided in an embodiment of the present invention;

[0045] Figure 3 A flowchart illustrating a method for full-screen display of cloud applications provided in an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of a new desktop layout for a cloud application server and a cloud application client provided in an embodiment of the present invention;

[0047] Figure 5 A flowchart illustrating a method for full-screen display of cloud applications provided in an embodiment of the present invention;

[0048] Figure 6 A flowchart illustrating a method for full-screen display of cloud applications provided in an embodiment of the present invention;

[0049] Figure 7 This is an application diagram illustrating a cloud application full-screen display method provided in an embodiment of the present invention;

[0050] Figure 8 This is a schematic diagram of the structure of a cloud application full-screen display device provided in an embodiment of the present invention;

[0051] Figure 9 This is a schematic diagram of the structure of a cloud application full-screen display device provided in an embodiment of the present invention;

[0052] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0055] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features can be combined with each other. Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0056] First, some terms involved in the embodiments of this invention will be introduced.

[0057] Cloud applications: Virtualized cloud applications, that is, applications that run on cloud servers, such as cloud gaming and cloud rendering.

[0058] Cloud server: The cloud-side environment in which cloud applications run. A cloud server contains a cloud-side operating system and integrates cloud application servers.

[0059] Cloud application server: Both the cloud application server and the cloud application run on the cloud server. The cloud application server is responsible for streaming the video window of the cloud application and injecting the operation behavior information of the user input from the cloud application client into the cloud-side operating system or cloud application.

[0060] Cloud application client: Runs on the user's local terminal device, which contains the operating system on the device side. The cloud application client is responsible for displaying the video frame in the streamed video window of the cloud application and collecting user input operation information and sending it to the cloud application server.

[0061] Coordinate system: The system corresponding to the coordinates of the mouse or the video window of the cloud application on the desktop of the cloud application client or the desktop of the cloud application server.

[0062] Available area: The desktop includes areas such as the taskbar and menu bar, which are not available to cloud applications. Apart from the functional areas occupied by the operating system, the remaining part of the desktop is the available area, which can be used to display video windows.

[0063] Virtual taskbar: It only occupies the corresponding area of ​​the desktop on the cloud application server side, so that cloud applications that are not displayed in full screen cannot use this area. Unlike a real taskbar, it does not need to include UI screens and provide various functions such as displaying application icons and various functional controls.

[0064] Full-screen display: This means that the video window fills the entire desktop (i.e., the entire screen), which will block the taskbar, menu bar, and other non-usable areas of the desktop.

[0065] Maximize display: This means that the video window fills the available area of ​​the desktop, but does not obscure the non-available areas such as the taskbar and menu bar.

[0066] Figure 1 This is a schematic diagram of a traditional desktop layout for a cloud application server and a cloud application client, provided as an embodiment of the present invention. Figure 1 As shown, assuming the cloud application server has operating system 1 installed on it, and the cloud application client has operating system 2 installed on it, the layout of the desktop corresponding to operating system 1 (referred to as the server desktop for ease of description) and the desktop corresponding to operating system 2 (referred to as the client desktop for ease of description) is as follows: Figure 1 As shown in the diagram: Assume that the server desktop includes a taskbar A located in the lower part of the desktop, and the client desktop includes a taskbar B located in the left part of the desktop and a menu bar C located in the upper part of the desktop.

[0067] Unlike the virtual taskbars mentioned above, taskbar A, taskbar B, and menu bar C here are actual functional areas on the corresponding desktop. These taskbars and menu bars can contain various functional controls for user use, including but not limited to: time, calendar, search, and browser controls. Taskbar A, taskbar B, and menu bar C are all non-usable areas of the corresponding desktop; that is, these areas are not occupied when the application is not in full-screen mode.

[0068] In cloud application scenarios, both local cloud application clients and cloud application servers support multiple operating systems. For example, local cloud application clients include Windows, Linux, Android, and iOS clients; similarly, cloud application servers can be categorized as Windows servers, Linux servers, Android servers, and iOS servers. Generally, cloud application clients supporting any operating system are required to connect to cloud application servers operating on any operating system.

[0069] Cloud applications support streaming via application windows. Streaming refers to the processing of video streams, so cloud applications stream the application screen as video and display it in a video window. In this scenario, the cloud application server encodes the video screen using a streaming protocol and sends it to the cloud application client for decoding and display. The size, position, and other layout information of the cloud application's video window, as well as the video screen itself, are reflected in the cloud application client. For a better user experience, the desktop layout of the server and client desktops needs to be synchronized. Desktop layout information includes the desktop size and the position and size of the available area. Desktop size can be described by resolution and pixels per inch (dpi).

[0070] If the desktop layout is not synchronized, for example, if... Figure 1 As shown, the cloud application client and cloud application server each have different desktop layouts. Therefore, imagine that if the cloud application displays a video window in a certain location on the server's desktop, and the cloud application server sends the video window's layout information (position and size) to the cloud application client, the cloud application client needs to locate the same coordinates on its local desktop based on the video window's position to determine a video window of the same size. For example... Figure 1 As shown in the image, the video window's position on the client's desktop might currently cover the taskbar B on the left. However, for cloud application clients, the video window can only cover non-usable areas like the taskbar B and menu bar C when the cloud application server instructs it to enter full-screen mode. Therefore, if the cloud application client were to display the video window covering the taskbar B at this point, it would violate normal display logic and cause an error. If the video window were displayed without covering the taskbar B, the user would only see a portion of the video image on the client's desktop because the other part would be obscured by the taskbar B, resulting in a poor user experience.

[0071] Therefore, in cloud application scenarios, it is necessary to synchronize the layout of the server-side desktop and the client-side desktop.

[0072] However, as mentioned above, different operating systems have different desktop layouts. For example, Linux systems have both a menu bar and a taskbar, while Windows systems only have a taskbar and no menu bar. These different desktop layouts primarily affect the position and size of the available area on the desktop, so a way to synchronize the available area is needed.

[0073] One solution is to use the size of the available area on the client's desktop as the size of the server's desktop. The available area of ​​the server's desktop is the same as the size of the server's desktop itself, meaning the server's desktop does not include non-available areas such as the taskbar and menu bar. This ensures that the available areas of the server's desktop and the client's desktop are synchronized, i.e., their coordinate systems are aligned. However, this approach has a problem: the cloud application server cannot distinguish between maximized and full-screen displays, so it can only default to maximized display. In other words, when the cloud application maximizes or displays a video window in full-screen mode, since the video window occupies the entire server's desktop, the cloud application server cannot distinguish whether the video window is maximized or full-screen; it can only assume it is maximized. Therefore, after the video window is streamed and encoded to the cloud application client, the decoded display shows the video window as maximized on the client's desktop, meaning the client's taskbar / menu bar still exists. In other words, even if a user triggers the action of displaying the cloud application in full screen through the cloud application client, the final display result of the cloud application client is that the video window of the cloud application is displayed in a maximized manner, which contradicts the user's intention and results in a poor user experience.

[0074] For some full-screen applications, such as full-screen games and full-screen video playback, if the cloud application client only maximizes the display, it will lead to a disconnect between the cloud application user experience and the experience of using the same application locally. Users may be confused because they have already enabled the full-screen display function provided by the cloud application (for example, by clicking the full-screen display button provided in the video window of the cloud application), but have not achieved the full-screen usage effect.

[0075] Understandably, with the aforementioned client-server desktop synchronization method, when the cloud application displays a video window on the server desktop in a non-full-screen manner (e.g., maximized display, or displaying a video window smaller than the server desktop size), the cloud application server transmits the layout information of the video window to the cloud application client. Based on this information, the cloud application client can achieve the same video window display result as the cloud application server. Therefore, with the aforementioned client-server desktop synchronization method, the only issue arises with full-screen display of the cloud application.

[0076] Therefore, in this embodiment of the invention, a virtual taskbar is created on the server desktop to occupy non-availability zones, enabling the cloud application server to sense that the cloud application has entered full-screen display mode, allowing the full-screen display mode to be autonomously controlled by the cloud application, and providing users with a full-screen experience that is completely consistent with using cloud applications and local applications.

[0077] The data transmission system and full-screen display method for cloud applications provided in the embodiments of the present invention will be described below.

[0078] Figure 2 This is a schematic diagram of a cloud application display system provided in an embodiment of the present invention, such as... Figure 2 As shown, the system includes: a cloud application and a cloud application server located on a cloud server, and a cloud application client located on a terminal device.

[0079] A cloud server is a virtual machine in the cloud. Cloud applications and cloud application servers can run on this cloud server.

[0080] In this embodiment of the invention, both the cloud application client and the cloud application server are service programs that provide services for the cloud application. The main service they provide is to stream the application screen of the cloud application: encode it into a video stream to be displayed in the video window of the cloud application, and transmit the video window containing the corresponding video screen to the cloud application client so that the cloud application client can decode and display the video window.

[0081] In practical applications, data transmission between cloud application servers and cloud application clients must conform to a defined streaming protocol. The data transmitted from the cloud application server to the cloud application client mainly consists of a video window containing the video feed, while the data transmitted from the cloud application client to the cloud application server mainly consists of user input information, such as a single click or double click at a specific coordinate.

[0082] Cloud applications are applications that run on cloud servers, such as games or audio / video applications. Cloud applications typically provide features for video windows, such as full-screen mode, maximized mode, minimized mode, and so on.

[0083] In fact, such as Figure 2 As shown, the cloud server also includes a cloud operating system, which has a corresponding desktop Z1, and the terminal device also includes a user-end operating system, which has a corresponding desktop Z2.

[0084] Based on the above display system, combined with Figure 3 Examples of cloud application full-screen display methods are described.

[0085] Figure 3 A flowchart of a cloud application full-screen display method provided in an embodiment of the present invention is shown below. Figure 3 As shown, the method may include the following steps:

[0086] 301. In response to the user's launch operation of the cloud application, the cloud application client sends the layout information of the first desktop to the cloud application server.

[0087] 302. Set up a second desktop with the same size as the first desktop on the cloud application server, and set up the same availability zone on the second desktop according to the position and size of the availability zone in the first desktop.

[0088] 303. The cloud application server generates a virtual taskbar in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop.

[0089] 304. The cloud application has entered full-screen display mode, and the first video window will be displayed in full-screen mode on the second desktop.

[0090] 305. The cloud application server sends the first video frame and the layout information of the first video window to the cloud application client. The layout information of the first video window indicates that the full-screen display state has been entered.

[0091] 306. The cloud application client displays the first video window and the first video screen in full screen on the first desktop according to the layout information of the first video window.

[0092] In practical applications, users can first log in to the cloud application client on their terminal device, thereby triggering the establishment of a communication connection between the cloud application client and the cloud application server. Then, users can click the cloud application icon to launch the application. At this point, the cloud application client will send the layout information of the primary desktop corresponding to the local operating system to the cloud application server, enabling the server to perform desktop synchronization.

[0093] The layout information of the first desktop includes its dimensions, the position and size of the available area within it. The cloud application server sets the size of the local second desktop to be the same as the first desktop, and sets the same available area in the second desktop based on the position and size of the available area in the first desktop. Furthermore, based on the size of the second desktop and the position and size of the available area in the second desktop, the cloud application server generates a virtual taskbar in the second desktop to occupy the non-available area. This achieves synchronization between the available areas of the second desktop and the first desktop, that is, synchronization of the desktop layout between the cloud application server and the cloud application client.

[0094] For ease of understanding, combined with Figure 4 This illustrates the synchronization result of the desktop layout between the cloud application server and the cloud application client. For example... Figure 4 As shown in the diagram, assume the size of the first desktop is W*H, where a taskbar X and a menu bar Y are located in position areas L1 and L2, respectively. Taskbar X and menu bar Y are actual taskbars and menu bars that exist on the first desktop. The remaining area on the first desktop, excluding the areas containing taskbar X and menu bar Y, is the usable area Z.

[0095] After the cloud application client sends the layout information of the first desktop to the cloud application server, the cloud application server sets the size of the local second desktop to be the same as that of the first desktop: W*H, and locates the same location area in the second desktop as the availability area Z of the second desktop based on the position and size of the availability area Z.

[0096] It should be noted that, since the original second desktop corresponding to the operating system of the cloud server hosting the cloud application server may include non-usable areas such as the actual taskbar and menu bar, in this embodiment of the invention, the cloud server's operating system needs to first delete the taskbar and menu bar, for example, by ending the display thread of the taskbar and menu bar, so that the original second desktop no longer contains non-usable areas such as the taskbar and menu bar. Afterwards, the layout of the second desktop is reset according to the layout of the first desktop.

[0097] In addition to configuring availability zone Z on the second desktop as described above, the cloud application server also determines the number and location of virtual taskbars to be set based on the size of the second desktop and the location and size of its availability zones. In fact, the virtual taskbars set on the second desktop correspond to the taskbar X and menu bar Y contained in the first desktop, and are used to set placeholders of the same size for non-availability zones in the same locations on the second desktop.

[0098] like Figure 4 As shown, the cloud application server finds that there are remaining space areas above and to the left of the available area Z of the second desktop. Therefore, it can be determined that there are two virtual taskbars that need to be generated, and the positions correspond to these two free space areas: virtual taskbar a and virtual taskbar b.

[0099] In practical applications, since the virtual taskbar only serves as a placeholder, virtual taskbar a and virtual taskbar b can be implemented as transparent layers or UI screens with certain set styles, without needing to contain the various functional controls contained in taskbar X and menu bar Y.

[0100] It should be noted that since the above layout needs to be modified for the second desktop of the operating system in the cloud server, the operating system needs to support the function of modifying the desktop layout, such as opening up relevant interfaces for setting the desktop layout so that the cloud application server can set the above layout.

[0101] like Figure 4 As shown, after the cloud application server reconfigures the layout of the second desktop, the second desktop and the first desktop corresponding to the cloud application client have the same desktop layout: the same desktop size, the same availability zone location and size, and the same non-availability zone location and size. This achieves synchronization between the second desktop and the first desktop, that is, alignment of the coordinate systems of the two desktops.

[0102] After the desktop synchronization is completed, the cloud application server can start the cloud application and notify the cloud application of the layout information of the second desktop, so that the cloud application can display the application screen according to the layout of the second desktop.

[0103] Here, we assume that the default launch method for cloud applications is full-screen display. When the cloud application server launches the application, it immediately enters full-screen mode. The application window containing the current application screen—the first video window—will be displayed in full-screen mode on the second desktop. When the cloud application server detects that the application has entered full-screen mode, i.e., when it detects that the virtual taskbar is obscured (hidden), it performs two actions: firstly, it streams the application screen by encoding the application screen within the first video window into a first video frame, which is then sent to the cloud application client. Secondly, it needs to determine the layout information of the first video window and send this layout information to the cloud application client as well. The layout information of the first video window indicates that the cloud application has entered full-screen mode. Optionally, the layout information of the first video window may only contain the information that "the cloud application has entered full-screen mode," or it may include the size and position information of the first video window on the second desktop.

[0104] In practical applications, optionally, the cloud application server can register a listener event for the virtual taskbar in the cloud server's operating system to monitor whether the virtual taskbar is obscured, i.e., whether it is hidden. When the cloud application displays the first video window in full-screen mode, the first video window will obscure the virtual taskbar, making it hidden. After the cloud server's operating system detects that the virtual taskbar is hidden, it will notify the cloud application server, thus the cloud application server perceives that the virtual taskbar is obscured (i.e., hidden), and determines that the cloud application has entered full-screen mode.

[0105] After receiving the first video frame and the layout information of the first video window, the cloud application client decodes the first video frame in the first video window and then displays the first video window and the first video frame contained therein in full screen on the first desktop according to the layout information of the first video window. At this time, the first video window will cover the taskbar, menu bar and other unusable areas of the first desktop, presenting a full-screen display effect where the first video window fills the entire first desktop.

[0106] The above solution allows users to utilize the full-screen capabilities of cloud applications (such as launching in full-screen mode) without altering user habits, resulting in a superior user experience. During cloud application operation, the layout of the second desktop on the cloud application server side remains consistent with the layout of the first desktop on the cloud application client side. Therefore, for a given application, running as a cloud application on the cloud server will achieve a completely consistent display effect whether it is running as a local application on the user's terminal. Furthermore, when a cloud application enters full-screen mode, the virtual taskbar set in the second desktop allows the cloud application server to detect this state and accurately notify the cloud application client, ensuring a consistent full-screen experience for the user.

[0107] Figure 5 A flowchart of a cloud application full-screen display method provided in an embodiment of the present invention is shown below. Figure 5 As shown, the method may include the following steps:

[0108] 501. In response to the user's launch operation of the cloud application, the cloud application client sends the layout information of the first desktop to the cloud application server.

[0109] 502. Set up a second desktop with the same size as the first desktop on the cloud application server, and set up the same availability zone on the second desktop according to the position and size of the availability zone in the first desktop.

[0110] 503. The cloud application server generates a virtual taskbar in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop.

[0111] 504. The cloud application has entered full-screen display mode, and the first video window will be displayed in full-screen mode on the second desktop.

[0112] 505. The cloud application server sends the first video frame and the layout information of the first video window to the cloud application client. The layout information of the first video window indicates that the full-screen display state has been entered.

[0113] 506. The cloud application client displays the first video window and the first video screen in full screen on the first desktop according to the layout information of the first video window.

[0114] 507. The cloud application confirms that it is exiting full-screen mode and will display the third video window in the available area of ​​the second desktop.

[0115] 508. The cloud application server sends the third video screen and the layout information of the third video window to the cloud application client. The layout information of the third video window indicates exiting the full-screen display state.

[0116] 509. The cloud application client displays the third video window and the third video screen in the available area of ​​the first desktop based on the layout information of the third video window.

[0117] In this embodiment, steps 501-506 can be referred to the description in the previous embodiment, and will not be repeated here.

[0118] This embodiment only emphasizes that when the cloud application exits the full-screen display state, the video window will be adjusted to a third video window that is smaller than or equal to the available area size of the second desktop, and the application screen that needs to be displayed will be displayed in the third video window. At this time, the virtual taskbar in the second desktop will not be obscured by the third video window, that is, the virtual taskbar is in an unhidden state.

[0119] When the cloud application server detects that the virtual taskbar is not obscured, it can determine that the cloud application has exited full-screen mode. It then determines the layout information of the third video window on the second desktop (such as the size and position of the third video window), encodes the application screen in the third video window to obtain the third video image, and transmits the third video image and the layout information of the third video window to the cloud application client. The cloud application client decodes the third video image, identifies the same third video window in the available area of ​​the first desktop based on the layout information of the third video window, and displays the third video image in the third video window.

[0120] The method by which the cloud application server detects that the virtual taskbar is not hidden is based on the same principle as the method described above for detecting that it is hidden, and will not be repeated here. Additionally, in practical applications, users can trigger an action to exit full-screen display within the cloud application client, using the full-screen function provided by the cloud application itself. This action then prompts the cloud application to determine whether to exit full-screen mode.

[0121] Figure 6 A flowchart of a cloud application full-screen display method provided in an embodiment of the present invention is shown below. Figure 6 As shown, the method may include the following steps:

[0122] 601. In response to the user's launch operation of the cloud application, the cloud application client sends the layout information of the first desktop to the cloud application server.

[0123] 602. The cloud application server sets the size of the second desktop to be the same as that of the first desktop, and sets the same availability zone in the second desktop according to the position and size of the availability zone in the first desktop.

[0124] 603. The cloud application server generates a virtual taskbar in the second desktop based on the size of the second desktop and the position and size of the available area in the second desktop.

[0125] 604. The cloud application displays the second video window in the available area of ​​the second desktop.

[0126] As mentioned above, after the cloud application server sets up the layout of the second desktop, it can launch the cloud application. At this point, the cloud application determines a second video window within the available area of ​​the second desktop based on the available area's location, and displays the screen to be shown after launch in this second video window. At this time, the virtual taskbar on the second desktop is not obscured by the second video window. The position and size of this second video window within the available area of ​​the second desktop can be determined by the cloud application based on default configuration information, such as displaying it in the center of the available area, occupying half its size.

[0127] 605. The cloud application server sends the layout information of the second video screen and the second video window to the cloud application client.

[0128] After the cloud application displays the second video window in the available area of ​​the second desktop and displays its application screen in the second video window, the cloud application server captures the display event, determines the layout information of the second video window in the available area of ​​the second desktop, encodes the application screen as the second video screen, and sends the second video screen and the layout information of the second video window to the cloud application client.

[0129] 606. The cloud application client displays the second video window and the second video screen in the available area of ​​the first desktop according to the layout information of the second video window.

[0130] Based on the layout information of the second video window, the cloud application client locates the same second video window in the available area of ​​the first desktop, decodes the second video image, and displays it in the second video window.

[0131] 607. The cloud application client sends the first user operation behavior information to the cloud application server. The first user operation behavior information corresponds to the operation behavior of launching the full-screen display function provided by the cloud application itself in the second video window.

[0132] After the second video window has been displayed on the cloud application client, the user can use some display functions provided by the cloud application itself, including but not limited to full-screen display. For example, the function list of the second video window may include a full-screen display function item, and clicking this function item will trigger the first user operation behavior information mentioned above. Alternatively, assuming the cloud application also provides a function to double-click a video window area to trigger full-screen display, then the first user operation behavior information mentioned above could be a description of the user's action of double-clicking the display area of ​​the second video window.

[0133] It should be noted that the cloud application client does not recognize which function the user's current operation is performing. Therefore, the first user operation behavior information mentioned above only describes the location coordinates of the user's operation behavior on the first desktop and the action (such as clicking or double-clicking).

[0134] 608. The cloud application server will provide the first user operation behavior information to the cloud application.

[0135] The cloud application server also does not have the ability to identify what function provided by the cloud application the first user's operation behavior information is being used for; it simply provides the first user's operation behavior information to the cloud application for identification.

[0136] In practical applications, optionally, the cloud application server can directly send the first user operation behavior information to the cloud application. Alternatively, optionally, the cloud application server can inject the first user operation behavior information into the operating system of the cloud server, so that it can send it to the cloud application through the operating system, that is, the cloud application captures the first user operation behavior information through the operating system.

[0137] 609. When the cloud application determines that the first user's operation behavior information matches the start full-screen display behavior, it adjusts the second video window to the first video window and displays the first video window in full-screen mode on the second desktop.

[0138] For example, the first user operation behavior information describes that the user performed a click action at the (x,y) position coordinates on the first desktop. The cloud application determines that the position coordinates match the "full-screen display" function button, and the click action matches the user's set action to start the full-screen display function. Then the cloud application determines to enter the full-screen display state, adjusts the previous second video window to the first video window, and displays the first video window in full-screen mode on the second desktop, that is, adjusts the video window to fill the first video window on the second desktop.

[0139] 610. The cloud application server sends the first video frame and the layout information of the first video window to the cloud application client. The layout information of the first video window indicates that the full-screen display state has been entered.

[0140] 611. The cloud application client displays the first video window and the first video screen in full screen on the first desktop according to the layout information of the first video window.

[0141] 612. The cloud application client sends the second user operation behavior information to the cloud application server. The second user operation behavior information corresponds to the operation behavior of exiting the full-screen display function provided by the cloud application itself in the first video window.

[0142] Similar to the first user operation behavior information mentioned above, when a user triggers the operation of exiting full-screen display in the currently full-screen first video window (such as double-clicking an area of ​​the first video window or clicking the function button to exit full-screen display), the cloud application client collects the corresponding second user operation behavior information (behavior and position coordinates of the operation).

[0143] 613. The cloud application server provides the second user's operation behavior information to the cloud application.

[0144] 614. When the cloud application determines that the second user's operation behavior information matches the exit full-screen display behavior, it adjusts the first video window to the third video window to be displayed in the available area of ​​the second desktop.

[0145] Similar to the above, when the cloud application identifies a match between the second user's operation behavior and the exit from full-screen display, it determines to exit full-screen display and adjusts the previous first video window into a smaller third video window to be displayed in the available area of ​​the second desktop. At this time, the virtual taskbar is not obscured by the third video window. The size and position of the third video window can be the default settings.

[0146] 615. The cloud application server sends the layout information of the third video screen and the third video window to the cloud application client.

[0147] The layout information of the third video window indicates whether to exit full-screen mode. In fact, the layout information of the third video window may include its position, size, and other layout details; optionally, it may further include an indicator to exit full-screen mode.

[0148] 616. The cloud application client displays the third video window and the third video screen in the available area of ​​the first desktop based on the layout information of the third video window.

[0149] In summary, this embodiment of the invention allows users to utilize the full-screen capabilities of cloud applications (e.g., full-screen display upon startup, full-screen display function buttons provided by the cloud application, and double-clicking a video window area to enable full-screen display) without requiring users to change their full-screen operation habits, resulting in a better user experience. The virtual taskbar set in the second desktop allows the cloud application server to detect when the cloud application enters full-screen mode, thus accurately notifying the cloud application client of this full-screen state. This provides users with a precise full-screen display experience and aligns with the concept of cloud application server controlling cloud application client. The cloud application client does not require significant changes, making the solution simpler to implement.

[0150] Figure 7 This is an application illustration of a cloud application full-screen display method provided in an embodiment of the present invention, such as... Figure 7As shown, in practical applications, the data transmission process of cloud applications may involve the following processes: synchronization of desktop layout between the server and the client, streaming of application windows from the server to the client, transmission of user input from the client to the server, startup of new full-screen applications / entry of old applications into full-screen display state, and exit of applications from full-screen display state.

[0151] The desktop layout synchronization process between the server and client refers to the process of synchronizing the desktop layout between the cloud application server and the cloud application client. As mentioned above, this process includes: after the user opens the cloud application, the cloud application client collects the layout information of the client's desktop (the first desktop mentioned above) and sends the layout information to the cloud application server. The cloud application server sets the size and availability area of ​​the server's desktop (the second desktop mentioned above) according to the size and availability area of ​​the client's desktop. The cloud application server creates a virtual taskbar, and then launches the cloud application.

[0152] The process of streaming an application window from the server to the client refers to the transmission of video footage from the cloud application server to the cloud application client. This process includes: the cloud application server capturing the layout information (including window position and size) of the video window displayed on the server's desktop and sending this information to the cloud application client. The cloud application client then displays the video window at the corresponding position on its desktop based on this layout information (it can create a new identical video window at that position or move the existing video window there, synchronizing the window size). Finally, the cloud application server encodes the video stream in the video window using a streaming protocol and sends the video footage to the cloud application client, enabling the client to display the video footage in its video window.

[0153] The process of transmitting user input from the client to the server refers to the process by which the cloud application client sends user input operation behavior information to the cloud application server. This process includes: the user performing an operation on the client desktop (such as clicking or moving a video window via mouse or touch screen); the cloud application client collecting the operation behavior information (operation location, action) and sending it to the cloud application server; the cloud application server generating a corresponding operation event in its cloud operating system; and the cloud application capturing this operation event through the cloud operating system to update the layout (position, size) of the video window on the server desktop.

[0154] The process of launching a new full-screen application / entering full-screen mode for an existing application refers to the processing after a cloud application enters full-screen mode. This process includes: the cloud application entering full-screen mode (e.g., the cloud application is launched in full-screen mode, or the user clicks the full-screen function button provided by the cloud application on the client desktop, or triggers a double-click operation to start full-screen mode). The cloud application server detects that the virtual taskbar is hidden and notifies the cloud application client that the cloud application has entered full-screen mode. The cloud application client then displays the video window in full-screen mode on the client desktop. Detailed execution of this process (e.g., video transmission) can be found in the descriptions in the aforementioned embodiments; only some key processes are illustrated here.

[0155] The process of an application exiting full-screen mode refers to the handling process after a cloud application exits full-screen mode. This process includes: the cloud application server detecting that the virtual taskbar has changed from a hidden state to a displayed state, notifying the cloud application client that the cloud application has exited full-screen mode, thereby causing the cloud application client to exit the full-screen display of the video window. Detailed execution of this process (such as video frame transmission) can be found in the descriptions in the aforementioned embodiments; only some key processes are illustrated here.

[0156] In an optional embodiment, the present invention also provides a cloud application display method, applied to a cloud application server, comprising the following steps:

[0157] Receive layout information of the first desktop sent by the cloud application client. The first desktop corresponds to the cloud application client. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop.

[0158] Set the size of the second desktop corresponding to the cloud application server to be the same as that of the first desktop, and set the same availability zone in the second desktop according to the position and size of the availability zone in the first desktop;

[0159] Based on the size of the second desktop and the position and size of the available area in the second desktop, a virtual taskbar is generated on the second desktop. The virtual taskbar is used to occupy the non-available area in the second desktop.

[0160] In response to the cloud application displaying the video window on the second desktop, the video frame contained in the video window and the layout information of the video window are sent to the cloud application client, so that the cloud application client displays the video window and the video frame on the first desktop according to the layout information of the video window.

[0161] In an optional embodiment, the present invention also provides a cloud application display method, applied to a cloud application client, comprising the following steps:

[0162] In response to the user's launch operation of the cloud application, the layout information of the local first desktop is sent to the cloud application server. The layout information of the first desktop includes the size of the first desktop, the position and size of the available area in the first desktop. The layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as that of the first desktop, set the same available area in the second desktop according to the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop. The virtual taskbar is used to occupy the non-available area in the second desktop.

[0163] Receive video feeds and video window layout information sent by the cloud application server. The video window contains the video feed.

[0164] The video window and video feed are displayed on the first desktop based on the layout information of the video window.

[0165] Therefore, after synchronizing the desktops of the cloud application server and the cloud application client based on the above virtual taskbar settings, any video window of any size displayed on the second desktop (whether in maximized display mode, full-screen display mode, or other display modes that only occupy the available area) and the video screen within it can be displayed consistently on the cloud application client.

[0166] The following describes in detail one or more embodiments of the cloud application full-screen display device of the present invention. Those skilled in the art will understand that these devices can all be configured using commercially available hardware components through the steps taught in this solution.

[0167] Figure 8 This is a schematic diagram of a cloud application full-screen display device provided in an embodiment of the present invention. This cloud application full-screen display device is applied to a cloud application server, such as... Figure 8 As shown, the device includes: a first receiving module 11, a desktop setting module 12, and a first sending module 13.

[0168] The first receiving module 11 is used to receive layout information of a first desktop sent by the cloud application client. The first desktop corresponds to the cloud application client. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop.

[0169] The desktop settings module 12 is used to set the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and to set the same available area in the second desktop according to the position and size of the available area in the first desktop; and to generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop, the virtual taskbar being used to occupy the non-available area in the second desktop.

[0170] The first sending module 13 is configured to, in response to the cloud application determining that it has entered a full-screen display state, display the first video window in full-screen mode on the second desktop, and send the first video frame contained in the first video window and the layout information of the first video window to the cloud application client, so that the cloud application client displays the first video window and the first video frame in full-screen mode on the first desktop according to the layout information of the first video window, wherein the layout information of the first video window indicates that it has entered a full-screen display state, and the virtual taskbar is obscured by the first video window.

[0171] Figure 8 The device shown can execute the steps performed by the cloud application server in the foregoing embodiments. For detailed execution process and technical effects, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0172] Figure 9 This is a schematic diagram of a cloud application full-screen display device provided in an embodiment of the present invention. This cloud application full-screen display device is applied to a cloud application client, such as... Figure 9 As shown, the device includes: a second transmitting module 21, a second receiving module 22, and a video display module 23.

[0173] The second sending module 21 is used to respond to the user's launch operation of the cloud application by sending the layout information of the local first desktop to the cloud application server. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop according to the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop. The virtual taskbar is used to occupy the non-available area in the second desktop.

[0174] The second receiving module 22 is used to receive the first video frame and the layout information of the first video window sent by the cloud application server. The first video window contains the first video frame, and the layout information of the first video window indicates that it has entered the full-screen display state. The layout information of the first video window is obtained by the cloud application server in response to the cloud application displaying the first video window in full screen on the second desktop. The virtual taskbar is obscured by the first video window.

[0175] The video display module 23 is used to display the first video window and the first video screen in full screen on the first desktop according to the layout information of the first video window.

[0176] Figure 9 The device shown can execute the steps performed by the cloud application client in the foregoing embodiments. For detailed execution process and technical effects, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0177] In one possible design, the above Figure 8 or Figure 9 The structure of the device shown can be implemented as an electronic device. For example... Figure 10 As shown, the electronic device may include: a processor 31, a memory 32, and a communication interface 33. The memory 32 stores executable code, which, when executed by the processor 31, enables the processor 31 to at least implement the cloud application full-screen display method provided in the foregoing embodiments.

[0178] In addition, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the cloud application full-screen display method provided in the foregoing embodiments.

[0179] The device embodiments described above are merely illustrative. The network elements described as separate components may or may not be physically separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0180] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cloud application display method, characterized in that, Applied to cloud application servers, the method includes: Receive layout information of a first desktop sent by a cloud application client. The first desktop corresponds to the cloud application client. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The size of the second desktop corresponding to the cloud application server is set to be the same as that of the first desktop, and the same availability zone is set in the second desktop according to the position and size of the availability zone in the first desktop; Based on the size of the second desktop and the position and size of the available area in the second desktop, a virtual taskbar is generated in the second desktop, and the virtual taskbar is used to occupy the non-available area in the second desktop; In response to the cloud application determining that it has entered full-screen display mode, the first video window is displayed in full-screen mode on the second desktop. The first video frame contained in the first video window and the layout information of the first video window are sent to the cloud application client, so that the cloud application client displays the first video window and the first video frame in full-screen mode on the first desktop according to the layout information of the first video window. The layout information of the first video window indicates that it has entered full-screen display mode, and the virtual taskbar is obscured by the first video window.

2. The method according to claim 1, characterized in that, The method further includes: In response to the cloud application displaying the second video window in the available area of ​​the second desktop, the second video frame contained in the second video window and the layout information of the second video window are sent to the cloud application client, so that the cloud application client displays the second video window and the second video frame in the available area of ​​the first desktop according to the layout information of the second video window, wherein the virtual taskbar is not obscured by the second video window, and the second video window is displayed earlier than the first video window.

3. The method according to claim 2, characterized in that, The method further includes: Receive first user operation behavior information sent by the cloud application client, the first user operation behavior information corresponding to the operation behavior of launching the full-screen display function provided by the cloud application itself in the second video window; The first user operation behavior information is provided to the cloud application so that when the cloud application determines that the first user operation behavior information matches the start full-screen display behavior, it adjusts the second video window to the first video window and displays the first video window in full-screen mode on the second desktop.

4. The method according to claim 1, characterized in that, The cloud application is launched in full-screen mode.

5. The method according to claim 1, characterized in that, The method further includes: Receive second user operation behavior information sent by the cloud application client, the second user operation behavior information corresponding to the operation behavior of exiting the full-screen display function provided by the cloud application itself in the first video window; The second user operation behavior information is provided to the cloud application so that when the cloud application determines that the second user operation behavior information matches the exit full-screen display behavior, the first video window is adjusted to a third video window to be displayed in the available area of ​​the second desktop, wherein the virtual taskbar is not obscured by the third video window. The third video frame contained in the third video window and the layout information of the third video window are sent to the cloud application client, so that the cloud application client displays the third video window and the third video frame in the available area of ​​the first desktop according to the layout information of the third video window, wherein the layout information of the third video window indicates exiting the full-screen display state.

6. A cloud application display method, characterized in that, Applied to cloud application clients, the method includes: In response to a user's launch operation of a cloud application, the layout information of the local first desktop is sent to the cloud application server. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop according to the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop. The virtual taskbar is used to occupy the non-available area in the second desktop. The system receives a first video frame and layout information of a first video window sent by the cloud application server. The first video window contains the first video frame, and the layout information of the first video window indicates that the system is in full-screen display mode. The layout information of the first video window is obtained by the cloud application server in response to the cloud application displaying the first video window in full-screen mode on the second desktop. The virtual taskbar is obscured by the first video window. Based on the layout information of the first video window, the first video window and the first video screen are displayed in full screen on the first desktop.

7. The method according to claim 6, characterized in that, The method further includes: The system receives a second video frame and layout information of a second video window sent by the cloud application server. The second video window contains the second video frame. The layout information of the second video window is obtained by the cloud application server in response to the cloud application displaying the second video window in the available area of ​​the second desktop. The virtual taskbar is not obscured by the second video window. The second video window is displayed earlier than the first video window. Based on the layout information of the second video window, the second video window and the second video frame are displayed in the available area of ​​the first desktop.

8. The method according to claim 7, characterized in that, The method further includes: Collect information on the first user operation behavior triggered by the user in the second video window. The first user operation behavior information corresponds to the operation behavior of launching the full-screen display function provided by the cloud application itself in the second video window. The first user operation behavior information is sent to the cloud application server so that the cloud application server provides the first user operation behavior information to the cloud application. When the cloud application determines that the first user operation behavior information matches the start full-screen display behavior, it adjusts the second video window to the first video window and displays the first video window in full-screen mode on the second desktop.

9. The method according to claim 6, characterized in that, The method further includes: Collect information on second user operation behavior triggered by the user in the first video window. The second user operation behavior information corresponds to the operation behavior of exiting the full-screen display function provided by the cloud application itself in the first video window. The second user operation behavior information is sent to the cloud application server so that the cloud application server provides the second user operation behavior information to the cloud application. When the cloud application determines that the second user operation behavior information matches the exit full-screen display behavior, it adjusts the first video window to a third video window to be displayed in the available area of ​​the second desktop, wherein the virtual taskbar is not obscured by the third video window. The system receives the third video frame and the layout information of the third video window sent by the cloud application server, wherein the layout information of the third video window indicates exiting the full-screen display state, and the third video window contains the third video frame. The third video window and the third video image are displayed in the available area of ​​the first desktop according to the layout information of the third video window.

10. A cloud application display method, characterized in that, Applied to cloud application servers, the method includes: Receive layout information of a first desktop sent by a cloud application client. The first desktop corresponds to the cloud application client. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The size of the second desktop corresponding to the cloud application server is set to be the same as that of the first desktop, and the same availability zone is set in the second desktop according to the position and size of the availability zone in the first desktop; Based on the size of the second desktop and the position and size of the available area in the second desktop, a virtual taskbar is generated in the second desktop, and the virtual taskbar is used to occupy the non-available area in the second desktop; In response to the cloud application displaying a video window on the second desktop, the video frame contained in the video window and the layout information of the video window are sent to the cloud application client, so that the cloud application client displays the video window and the video frame on the first desktop according to the layout information of the video window.

11. A cloud application display method, characterized in that, Applied to cloud application clients, the method includes: In response to a user's launch operation of a cloud application, the layout information of the local first desktop is sent to the cloud application server. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. The layout information of the first desktop is used to enable the cloud application server to set the size of the second desktop to be the same as the size of the first desktop, set the same available area in the second desktop according to the position and size of the available area in the first desktop, and generate a virtual taskbar in the second desktop according to the size of the second desktop and the position and size of the available area in the second desktop. The virtual taskbar is used to occupy the non-available area in the second desktop. Receive video footage and video window layout information sent by the cloud application server, wherein the video window contains the video footage; The video window and the video frame are displayed on the first desktop according to the layout information of the video window.

12. A cloud application display system, characterized in that, include: Cloud applications and cloud application servers located on cloud servers, and cloud application clients located on terminal devices; The cloud application client is used to respond to the user's launch operation of the cloud application by sending the layout information of the local first desktop to the cloud application server. The layout information of the first desktop includes the size of the first desktop and the position and size of the available area in the first desktop. It also receives the layout information of the first video screen and the first video window sent by the cloud application server, and displays the first video window and the first video screen in full screen on the first desktop according to the layout information of the first video window. The layout information of the first video window indicates that it has entered the full screen display state, and the first video window contains the first video screen. The cloud application server is configured to set the size of the second desktop corresponding to the cloud application server to be the same as the size of the first desktop, and to set the same available area in the second desktop according to the position and size of the available area in the first desktop. Based on the size of the second desktop and the position and size of the available area in the second desktop, a virtual taskbar is generated in the second desktop, the virtual taskbar being used to occupy a non-available area in the second desktop. In response to the cloud application displaying the first video window in full screen on the second desktop, the server sends the layout information of the first video window and the first video frame to the cloud application client, wherein the virtual taskbar is obscured by the first video window. The cloud application is used to display the first video window in full-screen mode on the second desktop when it is determined that the full-screen display state has been entered.

13. The system according to claim 12, characterized in that: The cloud application is also used to display the second video window in the available area of ​​the second desktop, wherein the second video window is displayed earlier than the second video window, and the virtual taskbar is not obscured by the second video window; The cloud application server is also used to send the second video frame and the layout information of the second video window to the cloud application client, wherein the second video window contains the second video frame; The cloud application client is also used to display the second video window and the second video frame in the available area of ​​the first desktop according to the layout information of the second video window.

14. An electronic device, characterized in that, include: The system includes a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor performs the cloud application display method as described in any one of claims 1 to 11.

15. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the cloud application display method as described in any one of claims 1 to 11.

16. A computer program product, characterized in that, include: A computer program, when executed by a processor of an electronic device, causes the processor to perform the cloud application display method as described in any one of claims 1 to 11.

Images