Cloud desktop display method, cloud desktop server, computer readable medium and computer program product

Abstract

The invention provides a cloud desktop display method, which comprises the following steps of: in a first stage, controlling a first display adapter to provide display data to a sending end of a virtual host through a display only driver (DOD); the first stage is a non-running state of an operating system of the virtual host, and the sending end is a port used for sending the display data to a terminal; in the second stage, a second display adapter is controlled through an indirect display driver (IDD) to generate display data, the display data are transmitted to the first display adapter, and the first display adapter provides the display data to the sending end; the second stage is a running state of an operating system of the virtual host. The invention further provides a cloud desktop server, a computer readable medium and a computer program product.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of cloud desktop display, in particular to a cloud desktop display method, a cloud desktop server, a computer readable medium, and a computer program product. BACKGROUND

[0002] In cloud desktop display, if a display only driver (DOD) is used, the rendering frame rate is low and the viewing experience is not good; if an indirect display driver (IDD) is used, the picture is missing during startup and shutdown, and the operation that needs to be performed during startup and shutdown cannot be completed. SUMMARY

[0003] The present disclosure provides a cloud desktop display method, a cloud desktop server, a computer readable medium, and a computer program product.

[0004] In a first aspect, the present disclosure provides a cloud desktop display method, which includes:

[0005] In a first stage, a display only driver (DOD) is used to control a first display adapter to provide display data to a sending end of a virtual host; the first stage is a state in which an operating system of the virtual host is not running, and the sending end is a port used to send the display data to a terminal;

[0006] In a second stage, an indirect display driver (IDD) is used to control a second display adapter to generate display data and transmit the display data to the first display adapter, and the first display adapter provides the display data to the sending end; the second stage is a state in which the operating system of the virtual host is running.

[0007] In a second aspect, the present disclosure provides a cloud desktop server, which includes a memory and a processor; the memory stores a computer program that can be executed by the processor, and the computer program is executed by the processor to implement any of the cloud desktop display methods of the present disclosure.

[0008] In a third aspect, the present disclosure provides a computer readable medium, which stores a computer program; the computer program is executed by a processor to implement any of the cloud desktop display methods of the present disclosure.

[0009] In a fourth aspect, the present disclosure provides a computer program product, which includes a computer program; the computer program is executed by a processor to implement any of the cloud desktop display methods of the present disclosure.

[0010] In the first stage, the display data is provided to the sending end using the DOD, so the display can be performed when starting, shutting down, etc., there is no picture missing, and no operation cannot be completed, and the starting, shutting down, etc. time is very short, although the rendering frame rate is low, but the impact on the viewing experience is not big; and in the second stage, the display data is transmitted to the first display adapter of the DOD using the IDD, and then provided to the sending end, so that the rendering frame rate is high and the viewing experience is good in most time; and the embodiment of the present disclosure can be realized only through the driver layer, and does not depend on the upper layer application, and will not affect the stability and robustness of the system. BRIEF DESCRIPTION OF DRAWINGS

[0011] In the drawings of the embodiments of the present disclosure:

[0012] Figure 1 A flow chart of a cloud desktop display method provided by the embodiments of the present disclosure;

[0013] Figure 2 A flow chart of another cloud desktop display method provided by the embodiments of the present disclosure;

[0014] Figure 3 A schematic diagram of a system architecture of another cloud desktop display method provided by the embodiments of the present disclosure;

[0015] Figure 4 A schematic diagram of the correspondence between the IDD display and the QXL graphics card in another cloud desktop display method provided by the embodiments of the present disclosure;

[0016] Figure 5 A composition block diagram of a cloud desktop server provided by the embodiments of the present disclosure;

[0017] Figure 6 A composition block diagram of a computer readable medium provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the cloud desktop display method, cloud desktop server, computer readable medium, computer program product provided by the embodiments of the present disclosure are described in detail below with reference to the drawings.

[0019] The embodiments shown will be described in more detail below with reference to the drawings, but the embodiments shown can be embodied in different forms, and the present disclosure should not be interpreted as being limited to the embodiments described below. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete, and to enable those skilled in the art to fully understand the scope of the present disclosure.

[0020] The accompanying drawings are used to provide a further understanding of embodiments of the present disclosure, and constitute a part of the specification, and together with the detailed embodiments serve to explain the present disclosure, and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent from the detailed embodiments described below taken in conjunction with the accompanying drawings.

[0021] The present disclosure can be described with reference to plan views and / or cross-sectional views by virtue of the idealized schematic illustrations of the present disclosure. Thus, the example illustrations can be modified according to manufacturing techniques and / or tolerances.

[0022] The embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict, if not conflicting.

[0023] The terms used in the present disclosure are only used to describe specific embodiments, and are not intended to limit the present disclosure. As used in the present disclosure, the term "and / or" includes any and all combinations of one or more of the associated listed items. As used in the present disclosure, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used in the present disclosure, the terms "comprises," "comprising," "includes," "including," "contains," "containing," "has," "having," "consists," "consisting," and "consists of" specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0024] Unless otherwise defined, all terms used in the present disclosure, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in a commonly used dictionary should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined in the present disclosure.

[0025] The present disclosure is not limited to the embodiments shown in the drawings, but includes modifications of configurations formed based on manufacturing processes. Therefore, the regions exemplified in the drawings have a schematic property, and the shape of the regions shown in the drawings exemplifies a specific shape of a region of an element, but is not intended to be restrictive.

[0026] In cloud desktop technology, a virtual host (VM, Virtual Machine) is formed by the hardware of a cloud desktop server (physical machine), and an image of a virtual host desktop is remotely sent to a terminal (such as a computer, a mobile phone, etc.) of a user for display, so that the user can see the desktop of the virtual host on the terminal and perform corresponding operations, which is equivalent to remotely using the virtual host.

[0027] Since the virtual host does not actually connect the display of the terminal, it is necessary to send display data (data for the display to display a picture, which can include coordinates, gray scale, color, etc. of each pixel in an image) to the terminal in some way.

[0028] In some related technologies, the virtual host can generate display data in a display only driver (DOD) mode, which is provided to a sending end of the virtual host, such as a port of a simple protocol for independent computing environment (SPICE) (SPICE port), and then the sending end sends the display data to the terminal for display. However, since the DOD directly uses original rendering display data (such as system rendering images) for display, its performance is poor, and the rendering frame rate is low (such as the refresh rate is usually lower than 60 FPS), so the user's viewing experience is not good.

[0029] In another related technology, the virtual host can also generate display data in an indirect display driver (IDD) mode to improve the rendering frame rate. However, the IDD cannot directly provide the display data to the sending end (i.e. the IDD is not applicable to the SPICE protocol), but has to send the display data to the terminal through a network (such as the Internet) according to a direct connection protocol when the operating system is running, so that the display data cannot be sent to the terminal when the virtual host is started or shut down, and the user cannot see the picture when the virtual host is started or shut down, resulting in picture loss, and some operations that need to be performed when starting or shutting down cannot be completed.

[0030] In a first aspect, the embodiments of the present disclosure provide a cloud desktop display method.

[0031] The embodiments of the present disclosure are used to implement the display of the cloud desktop, specifically executed by a cloud desktop server, to generate display data (such as coordinates, gray scale, color, etc. of each pixel in an image) of the cloud desktop and send it to the terminal for displaying the cloud desktop, so that the user can see and operate the cloud desktop on the terminal.

[0032] Referring to Figure 1 The cloud desktop display method of the embodiments of the present disclosure includes:

[0033] S101, in the first stage, the DOD controls the first display adapter to provide display data to the sending end of the virtual host.

[0034] The first stage is a state in which the operating system of the virtual host is not running, and the sending end is a port for sending display data to the terminal.

[0035] In some embodiments, the first phase includes a startup phase and / or a shutdown phase for the virtual host.

[0036] S102. In the second stage, the second display adapter is controlled by IDD to generate display data and transmit the display data to the first display adapter. The first display adapter then provides the display data to the sending end.

[0037] The second stage refers to the running state of the virtual host's operating system.

[0038] In this embodiment of the disclosure, the state in which the virtual host is powered on but the operating system (such as the Windows operating system) is not fully loaded is the "first stage", which includes the startup stage and shutdown stage of the virtual host; while the state in which the virtual host is powered on and the operating system is fully loaded and running is the "second stage", that is, the stage of normal operation.

[0039] Reference Figure 3 In the first stage, the first display adapter (GPU, such as a QXL graphics card with the SPICE protocol) is started through DOD, which enables it to generate display data and provide the display data to the virtual host's sending end (such as the SPICE end). The sending end can then transmit the display data to the terminal via the network (such as the Internet) so that the terminal can display the cloud desktop's screen in the first stage (such as the startup screen, shutdown screen, etc.).

[0040] Reference Figure 3 In the second stage, the second display adapter (i.e., the second graphics card) is controlled by IDD to render display data and output the display data to the first display adapter. The first display adapter provides the display data to the sending end of the virtual host, so that the sending end transmits the display data to the terminal for the terminal to display the cloud desktop screen in the second stage (such as the screen during normal operation).

[0041] In this embodiment, the DOD is used to provide display data to the sending end in the first stage, so display can still be performed during startup, shutdown, etc., without any missing images or inability to complete operations. The startup and shutdown times are very short, and although the rendering frame rate is low, it has little impact on the viewing experience. In the second stage, the IDD is used to transmit display data to the first display adapter of the DOD and then provide it to the sending end, thereby ensuring a high rendering frame rate and a good viewing experience most of the time. Moreover, this embodiment can be implemented only through the driver layer and does not depend on upper-layer applications, so it will not affect the stability and robustness of the system.

[0042] In some embodiments, the second display adapter includes a physical display adapter and / or a virtual display adapter.

[0043] As one embodiment of this disclosure, in the second stage, since IDD is used for driving, the second display adapter used can be either a virtual display adapter (i.e., a second virtual graphics card) or a physical display adapter (physical graphics card). Compared to virtual graphics cards such as QXL graphics cards, physical graphics cards generally have stronger performance. Therefore, this embodiment of the disclosure can not only make full use of the performance of the physical graphics card through IDD to improve the viewing experience, but is also applicable to cloud desktops (such as the SPICE protocol).

[0044] In some embodiments, transmitting display data to the first display adapter (S102) includes:

[0045] S1021A and IDD transmit display data to the first display adapter.

[0046] or,

[0047] S1021 B. Transmit the display data generated by IDD to the first display adapter via the transmission application.

[0048] As one embodiment of this disclosure, in the second stage, the IDD can directly transmit the display data to the DOD within the driver layer; or, as another embodiment of this disclosure, in the second stage, the display data generated by the IDD can be transmitted to the DOD through other applications (transmission applications).

[0049] In some embodiments, the first display adapter provides display data to the sending end (S102) including:

[0050] S1022, The first display adapter transmits display data to the sending end via video memory.

[0051] Reference Figure 3 As one embodiment of this disclosure, the first display adapter (such as a QXL graphics card) may transmit display data to the sending end (such as a SPICE end) of the virtual host via video memory (i.e., the display storage of DOD, i.e., DOD GPU Memory).

[0052] In some embodiments, controlling the first display adapter to provide display data to the sending end of the virtual host via DOD (S101) includes:

[0053] S1011. Transmit raw rendering display data to the first display adapter.

[0054] S1012, The first display adapter provides the raw rendering display data to the sending end.

[0055] As one embodiment of this disclosure, when the DOD is displayed in the first stage, it may input raw rendering display data (such as system rendered image) to the first display adapter (first graphics card), and the first display adapter directly transmits the raw rendering display data as display data to the sending end (such as SPICE end).

[0056] For example, a first virtual monitor (such as a QXL monitor) can be created when the virtual host is powered on, and the first display adapter uses the first virtual monitor as output to generate display data (raw rendered display data).

[0057] In some embodiments, controlling the second display adapter to generate display data via IDD (S102) includes:

[0058] S1023, The second display adapter generates display data using the second virtual display as the output terminal.

[0059] The second virtual display corresponds one-to-one with the first display adapter.

[0060] As one embodiment of this disclosure, in the second stage, referring to Figure 4 It can be a second display adapter (second graphics card) using a second virtual display (such as an IDD display) as the output end, to generate display data based on the original rendered display data (such as system rendered images), and transmit the display data to the first display adapter (such as a QXL graphics card), and then the first display adapter transmits the display data to the sending end (such as a SPICE end).

[0061] Moreover, the second virtual display (such as an IDD display) used at this time corresponds one-to-one with the first display adapter (such as a QXL graphics card). That is, the display data generated by each second virtual display will be transmitted to a corresponding first display adapter, thereby ensuring that the number of physical displays (such as the terminal's display) that can be supported remains unchanged in the first and second stages.

[0062] In some embodiments, the first phase includes the startup phase of the virtual host.

[0063] The startup phase also includes:

[0064] S10191. Activate the first display adapter and the corresponding first virtual display.

[0065] The second phase also includes:

[0066] S10291, DOD records the identifier of the first display adapter and creates a system event pointing to the first display adapter.

[0067] S10292. In response to initialization completion, IDD opens a system event based on the identifier of the first display adapter.

[0068] S10293. In response to a system event being triggered, DOD deletes the first virtual display, and IDD creates a second virtual display that corresponds one-to-one with the first display adapter.

[0069] As one embodiment of this disclosure, the change in driving method can be achieved through events: during the startup phase of the virtual host, the first display adapter (such as a QXL graphics card) and the first virtual display (such as a QXL monitor) are automatically activated; after the operating system starts running, the DOD records the identifiers of the first display adapter and the corresponding first virtual display (such as recording the ID in the registry) and creates a system event pointing to the first display adapter; thus, when the IDD initialization is completed, it can open the system event according to the above identifier, that is, the system event is triggered; and because the system event is triggered, the DOD can delete its first virtual display, and the IDD creates a second virtual display corresponding one-to-one with the first display adapter and begins to generate display data.

[0070] It should be understood that when the first stage also includes a shutdown phase, the driving method can change in a similar way. For example, after IDD is shut down, DOD can recreate the first display adapter and start generating display data.

[0071] Example 1:

[0072] The following is an exemplary description of a cloud desktop display method according to an embodiment of this disclosure, referring to... Figure 2 The method may include the following steps:

[0073] A101. When the virtualization host (virtual host) starts up (startup phase, first phase), the QXL graphics card (first display adapter) and QXL monitor (first virtual monitor) are automatically activated.

[0074] A102. During startup, the QXL graphics card acquires the system-rendered image (raw rendering display data) and sends it as image information (display data, which may include the coordinates, grayscale, color, etc. of each pixel in the image) to the SPICE terminal (sender) of the virtualization host. The SPICE terminal then sends the image information to the user's client (terminal) for display.

[0075] A103. After the Windows system (operating system) starts normally (second stage), QXL (DOD) completes initialization and writes the QXL graphics card's LUID and device ID (identifier of the first display adapter) into the registry.

[0076] A104. QXL creates a Windows named event (system time) named qx l_idd_event_x, where x represents the ID of the QXL device.

[0077] A105, IDD begins initialization. After successful initialization, the qx l_idd_event_x event is opened based on the LUID and device ID in the registry.

[0078] A1061. If the opening is successful, an event is triggered to add an IDD monitor (second virtual monitor) that corresponds one-to-one with the QXL graphics card, and record the correspondence between the IDD monitor and LUID.

[0079] After receiving the notification that the event has been triggered, A1062 and QXL remove their own QXL monitors, so that the IDD graphics card connects to the IDD monitors and acts as the display output device of the virtualization host.

[0080] A107 and IDD control the physical graphics card (physical display adapter, second display adapter) to receive the system-rendered image, render it to obtain image information, and send the image information to the QXL graphics card through D3DKMTEscape according to the above correspondence between the IDD display and LUID.

[0081] After receiving the image information, the A108 and QXL graphics cards transmit the image information through the video memory to the SPICE terminal of the virtualization host, which then sends the image information to the client for display.

[0082] Secondly, referring to Figure 5 This disclosure provides a cloud desktop server, which includes a memory and a processor; the memory stores a computer program that can be executed by the processor, and when the computer program is executed by the processor, it implements any of the cloud desktop display methods of this disclosure.

[0083] Thirdly, referring to Figure 6 This disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements any of the cloud desktop display methods of this disclosure.

[0084] Fourthly, embodiments of this disclosure provide a computer program product, which includes a computer program that, when executed by a processor, implements any of the cloud desktop display methods of embodiments of this disclosure.

[0085] Among them, the processor is a device with data processing capabilities, including but not limited to the central processing unit (CPU); the memory is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read-write interface) is connected between the processor and the memory, enabling information exchange between the memory and the processor, including but not limited to the data bus (Bus).

[0086] Those skilled in the art will understand that all or some of the steps, systems, and devices disclosed above, as functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0087] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be executed by several physical components working together.

[0088] Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit (CPU), digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technique for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH) or other disk storage; read-only optical disc (CD-ROM), digital versatile disc (DVD) or other optical disc storage; magnetic cartridges, magnetic tapes, disk storage or other magnetic storage; and any other media that can be used to store desired information and can be accessed by a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0089] This disclosure has disclosed exemplary embodiments, and although specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for displaying a cloud desktop, comprising: In the first phase, the display data is provided to the sending end of the virtual host by controlling the first display adapter through the display-only driver DOD. The first stage is a state in which the operating system of the virtual host is not running, and the sending end is a port used to send the display data to the terminal; In the second stage, the second display adapter is controlled by the indirect display driver (IDD) to generate display data, and the display data is transmitted to the first display adapter. The first display adapter then provides the display data to the sending end. The second stage refers to the running state of the operating system of the virtual host.

2. The method according to claim 1, wherein, The first stage includes the startup stage and / or shutdown stage of the virtual host.

3. The method according to claim 1, wherein, The second display adapter includes a physical display adapter and / or a virtual display adapter.

4. The method according to claim 1, wherein, Transferring the display data to the first display adapter includes: IDD transferring the display data to the first display adapter, or, the display data generated by IDD being transferred to the first display adapter via a transmission application; The first display adapter provides the display data to the sending end by: the first display adapter transmitting the display data to the sending end through video memory.

5. The method according to any one of claims 1 to 4, wherein, The DOD controls the first display adapter to provide display data to the virtual host's sending end, including: Transmit raw rendering display data to the first display adapter; The first display adapter provides the raw rendering display data to the sending end.

6. The method according to any one of claims 1 to 4, wherein, The step of controlling the second display adapter to generate display data via IDD includes: The second display adapter generates the display data using the second virtual display as the output terminal; the second virtual display corresponds one-to-one with the first display adapter.

7. The method according to claim 6, wherein, The first stage includes the startup stage of the virtual host; The startup phase also includes: activating the first display adapter and the corresponding first virtual display; In the second stage, the method further includes: DOD recording the identifier of the first display adapter and creating a system event pointing to the first display adapter; in response to initialization completion, IDD opening the system event according to the identifier of the first display adapter; in response to the system event being triggered, DOD deleting the first virtual display and IDD creating the second virtual display corresponding one-to-one with the first display adapter.

8. A cloud desktop server, comprising a memory and a processor; the memory storing a computer program executable by the processor, wherein the computer program, when executed by the processor, implements the cloud desktop display method according to any one of claims 1 to 7.

9. A computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method for displaying a cloud desktop according to any one of claims 1 to 7.

10. A computer program product comprising a computer program that, when executed by a processor, implements the cloud desktop display method of any one of claims 1 to 7.

Images