Cloud desktop display method, cloud desktop server, computer readable medium, and computer program product

Abstract

The present disclosure provides a cloud desktop display method, comprising: in a first stage, a display only driver (DOD) controls a first display adapter to provide display data to a sending end of a virtual host, wherein the first stage is a state in which an operating system of the virtual host is not running, and the sending end is a port used for sending the display data to a terminal; and in a second stage, an indirect display driver (IDD) controls a second display adapter to generate display data, and transmits the display data to the first display adapter, and the first display adapter provides the display data to the sending end, wherein the second stage is a state in which the operating system of the virtual host is running. The present disclosure further provides a cloud desktop server, a computer readable medium, and a computer program product.

Description

Method for cloud desktop display, cloud desktop server, computer readable medium, computer program product

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application CN 202411092522.9, entitled "Method for cloud desktop display, cloud desktop server, computer readable medium, computer program product", filed on August 7, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to the technical field of cloud desktop display, and in particular to a method for cloud desktop display, a cloud desktop server, a computer readable medium, and a computer program product. BACKGROUND

[0004] In cloud desktop display, if a display only driver (DOD) is used, the rendering frame rate is low, and the viewing experience is not good; if an indirect display driver (IDD) is used, the picture is missing during startup and shutdown, and the operations performed during startup and shutdown cannot be completed. SUMMARY

[0005] The present disclosure provides a method for cloud desktop display, a cloud desktop server, a computer readable medium, and a computer program product.

[0006] The present disclosure provides a method for cloud desktop display, comprising: in a first stage, controlling a first display adapter to provide display data to a sending end of a virtual host through a display only driver (DOD), wherein the first stage is a state in which an operating system of the virtual host is not running, and the sending end is a port for sending the display data to a terminal; in a second stage, controlling a second display adapter to generate display data and transmit the display data to the first display adapter through an indirect display driver (IDD), and the first display adapter provides the display data to the sending end, wherein the second stage is a state in which the operating system of the virtual host is running.

[0007] The present disclosure also provides a cloud desktop server, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method for cloud desktop display according to the embodiments of the present disclosure when the computer program is executed by the processor.

[0008] The present disclosure also provides a computer readable medium having a computer program stored thereon, wherein the processor implements the method for cloud desktop display according to the embodiments of the present disclosure when the computer program is executed by the processor.

[0009] The embodiment of the present disclosure further provides a computer program product comprising a computer program, wherein the computer program is executed by a processor, and the processor implements the method for cloud desktop display according to the embodiment of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0010] In the drawings of the embodiment of the present disclosure:

[0011] Fig. 1 is a flow chart of the method for cloud desktop display provided by the embodiment of the present disclosure;

[0012] Fig. 2 is another flow chart of the method for cloud desktop display provided by the embodiment of the present disclosure;

[0013] Fig. 3 is a schematic diagram of the system architecture of the method for cloud desktop display provided by the embodiment of the present disclosure;

[0014] Fig. 4 is a schematic diagram of the correspondence between the IDD display and the QXL graphics card in the method for cloud desktop display provided by the embodiment of the present disclosure;

[0015] Fig. 5 is a block diagram of the composition of the cloud desktop server provided by the embodiment of the present disclosure;

[0016] Fig. 6 is a block diagram of the composition of the computer readable medium provided by the embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the method for cloud desktop display, the cloud desktop server, the computer readable medium, the computer program product provided by the embodiment of the present disclosure are described in detail below with reference to the drawings.

[0018] The embodiments shown will be described in greater detail in connection with the drawings, which can be embodied in different forms and the present disclosure should not be construed as being limited to the embodiments set forth below. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0019] The drawings of the embodiment of the present disclosure are used to provide further understanding of the embodiment of the present disclosure and form part of the specification, which, together with the detailed embodiments, is used to explain the present disclosure and does not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by referring to the detailed embodiments described below with reference to the drawings.

[0020] The present disclosure can be described with reference to plan views and / or sectional views by means of ideal schematic diagrams of the present disclosure. Therefore, the example diagrams can be modified according to manufacturing techniques and / or tolerances.

[0021] In the case of no conflict, each embodiment of the disclosure and each feature in the embodiment can be combined with each other.

[0022] The terms used in the disclosure are only used to describe specific embodiments, and are not intended to limit the disclosure. As used in the disclosure, the term "and / or" includes any and all combinations of one or more of the associated listed items. As used in the disclosure, the singular forms "a" and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. As used in the disclosure, the terms "comprise", "consist of", designate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0023] Unless otherwise defined, all terms used in the disclosure, including technical and scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the disclosure, and should not be interpreted in an idealized or overly formal sense unless the disclosure explicitly so defines.

[0024] The disclosure is not limited to the embodiments shown in the drawings, but includes modifications of the configuration formed based on the manufacturing process. Therefore, the regions exemplified in the drawings have a schematic property, and the shape of the regions shown in the drawings exemplifies the specific shape of the region of the element, but is not intended to be restrictive.

[0025] In cloud desktop technology, a virtual host (VM, Virtual Machine) is formed by the hardware of a cloud desktop server (physical machine), and an image of the virtual host desktop is remotely sent to a terminal (such as a computer, a mobile phone, etc.) of a user for display, so that the user can see the desktop of the virtual host on the terminal and perform corresponding operations, which is equivalent to remotely using the virtual host.

[0026] Since the virtual host is not actually connected to the display of the user terminal, it is necessary to send display data (data used for the display to display a picture, which can include the coordinates, gray scale, color, etc. of each pixel in the image) to the terminal in a specific manner.

[0027] In some related technologies, a virtual host can generate display data in a display only driver (DOD) mode, and the display data is provided to a sending end of the virtual host, such as a port of a simple protocol for independent computing environment (SPICE port), and then the sending end sends the display data to a terminal for display. However, since the DOD directly uses original rendering display data (for example, a system rendering image) for display, the performance of the DOD is poor, and the rendering frame rate is low (for example, the refresh rate is usually lower than 60 FPS), so that the user's viewing experience is poor.

[0028] In some other related technologies, a virtual host can generate display data in an indirect display driver (IDD) mode to improve the rendering frame rate. However, the IDD cannot directly provide the display data to the sending end (that is, the IDD is not applicable to the SPICE protocol), but has to send the display data to the terminal through a network (for example, the Internet) according to a direct connection protocol when the operating system is running, so that the display data cannot be sent to the terminal when the virtual host is started or shut down, and the user cannot see the picture when the virtual host is started or shut down, which causes picture loss, and operations that need to be performed by the user when the virtual host is started or shut down cannot be completed.

[0029] Embodiments of the present disclosure provide a cloud desktop display method.

[0030] Embodiments of the present disclosure are used to implement the display of a cloud desktop, and are executed by a cloud desktop server to generate display data (for example, coordinates, gray scale, color, etc. of each pixel in an image) of the cloud desktop and send the display data to a terminal, so that the terminal displays the cloud desktop, and the user can see and operate the cloud desktop on the terminal.

[0031] Referring to FIG. 1, the cloud desktop display method according to an embodiment of the present disclosure includes the following steps S101-S102.

[0032] In step S101, in a first stage, a first display adapter is controlled by a DOD to provide display data to a sending end of a virtual host.

[0033] The first stage is a state in which an operating system of the virtual host is not running, and the sending end is a port for sending display data to a terminal.

[0034] In some embodiments, the first stage includes a starting stage and / or a shutting down stage of the virtual host.

[0035] In step S102, in the second stage, the second display adapter is controlled by the IDD to generate display data, the display data is transmitted to the first display adapter, and the first display adapter provides the display data to the sending end.

[0036] The second stage is a state in which the operating system of the virtual host runs.

[0037] In the embodiments of the present disclosure, a stage after the virtual host is powered on and before the operating system (for example, Windows) is completely loaded is a "first stage", for example, the first stage can include a startup stage, a shutdown stage, and the like of the virtual host; a stage after the virtual host is powered on and after the operating system is completely loaded is a "second stage", that is, a normal running stage. The operating system) is completely loaded. The first stage can include a startup stage, a shutdown stage, and the like of the virtual host; a stage after the virtual host is powered on and after the operating system is completely loaded is a "second stage", that is, a normal running stage.

[0038] Referring to FIG. 3, in the first stage, the first display adapter (GPU, for example, a QXL graphics card of a SPICE protocol) is started by the DOD, display data is generated, and the display data is provided to the sending end (for example, a SPICE end) of the virtual host, so that the sending end can transmit the display data to the terminal through a network (for example, the Internet) to display a picture (for example, a startup picture, a shutdown picture, and the like) of the cloud desktop in the first stage.

[0039] Referring to FIG. 3, in the second stage, the second display adapter (that is, a second graphics card) is controlled by the IDD to render display data, and the display data is output to the first display adapter, and the first display adapter provides the display data to the sending end of the virtual host, so that the sending end transmits the display data to the terminal to display a picture (for example, a picture in a normal running state) of the cloud desktop in the second stage.

[0040] In the embodiments of the present disclosure, the DOD is used to provide the display data to the sending end in the first stage, so that display can be performed when starting, shutting down, and the like, there is no picture loss, and there is no case in which the operation cannot be completed. Since the starting, shutting down, and the like are relatively short in time, although the rendering frame rate is low, the viewing experience is not greatly affected. In the second stage, the IDD is used to transmit the display data to the first display adapter of the DOD, and then the first display adapter provides the display data to the sending end, so that the rendering frame rate is high and the viewing experience is good in most of the time in the normal running state. On the other hand, the embodiments of the present disclosure can be implemented only through a driver layer, and do not depend on an upper layer application, so that the stability and robustness of the system are not affected.

[0041] In some embodiments, the second display adapter includes a physical display adapter and / or a virtual display adapter.

[0042] As an embodiment of the present disclosure, in the second stage, the second display adapter used can be a virtual display adapter (i.e., a second virtual graphics card) or a physical display adapter (a physical graphics card) due to the driving by IDD. As compared with a virtual graphics card such as a QXL graphics card, a physical graphics card usually has stronger performance, so that the embodiment of the present disclosure can not only improve the viewing experience by fully utilizing the performance of the physical graphics card through IDD, but also be applicable to a cloud desktop (e.g., a SPICE protocol).

[0043] In some embodiments, the step of transmitting the display data to the first display adapter (i.e., step S102) includes the following steps S1021A or S1021B.

[0044] In step S1021A, the IDD transmits the display data to the first display adapter.

[0045] In step S1021B, the display data generated by the IDD is transmitted to the first display adapter by a transmission application.

[0046] As an embodiment of the present disclosure, in the second stage, the IDD can directly transmit the display data to the DOD in the driving layer; or, as another embodiment of the present disclosure, in the second stage, the display data generated by the IDD can be transmitted to the DOD by another application (i.e., a transmission application).

[0047] In some embodiments, the step of providing the display data by the first display adapter to the sending end (i.e., step S102) includes the following step S1022.

[0048] In step S1022, the first display adapter transmits the display data to the sending end by a video memory.

[0049] Referring to FIG. 3, as an embodiment of the present disclosure, the first display adapter (e.g., a QXL graphics card) can transmit the display data to the sending end (e.g., a SPICE end) of the virtual host by a video memory (i.e., a display memory of the DOD, such as a DOD GPU Memory).

[0050] In some embodiments, the step of controlling the first display adapter to provide the display data to the sending end of the virtual host by the DOD (i.e., step S101) includes the following steps S1011 and S1012.

[0051] In step S1011, the original rendering display data is transmitted to the first display adapter.

[0052] In step S1012, the first display adapter provides the original rendering display data to the sending end.

[0053] As an embodiment of the present disclosure, when the DOD displays in the first stage, the original rendering display data (for example, system rendering image) can be input to the first display adapter (first graphics card), and the first display adapter directly transmits the original rendering display data as display data to the sending end (for example, SPICE end).

[0054] For example, the first virtual display (for example, QXL display) can be created when the virtual host is started, and the first display adapter generates display data (that is, original rendering display data) with the first virtual display as the output end.

[0055] In some embodiments, the step S102 of controlling the second display adapter to generate display data by the IDD includes the following step S1023.

[0056] In step S1023, the second display adapter generates display data with the second virtual display as the output end.

[0057] The second virtual display corresponds to the first display adapter one by one.

[0058] As an embodiment of the present disclosure, in the second stage, referring to FIG. 4, the second display adapter (second graphics card) can generate display data according to the original rendering display data (for example, system rendering image) with the second virtual display (for example, IDD display) as the output end, and transmit the display data to the first display adapter (for example, QXL graphics card), and then the first display adapter transmits the display data to the sending end (for example, SPICE end).

[0059] The second virtual display (for example, IDD display) corresponds to the first display adapter (for example, QXL graphics card) one by one, that is, the display data generated by each second virtual display is transmitted to a corresponding first display adapter, so that the number of physical displays (for example, the display of the user terminal) that can be supported in the first stage and the second stage remains unchanged.

[0060] In some embodiments, the first stage includes the start-up stage of the virtual host.

[0061] In the start-up stage, the following step S10191 is further included.

[0062] In step S10191, the first display adapter and the corresponding first virtual display are activated.

[0063] In the second stage, the following steps S10291 to S10293 are further included.

[0064] In step S10291, the DOD records the identifier of the first display adapter, and creates a system event pointing to the first display adapter.

[0065] In step S10292, in response to the initialization being completed, the IDD opens a system event according to the identification of the first display adapter.

[0066] In step S10293, in response to the system event being triggered, the DOD deletes the first virtual display, and the IDD creates a second virtual display corresponding to each first display adapter.

[0067] As one of the embodiments of the present disclosure, the transformation of the driving mode can be realized by means of an event. In the start-up phase of the virtual host, the first display adapter (for example, the QXL graphics card) and the first virtual display (for example, the QXL display) are automatically activated; after the operating system starts running, the DOD records the identification of the first display adapter and the identification of the corresponding first virtual display (for example, records the identification in the registry), and creates a system event pointing to the first display adapter; when the initialization of the IDD is completed, the system event can be opened according to the identification, that is, the system event is triggered; in response to the system event being triggered, the DOD can delete the first virtual display, and the IDD can create a second virtual display corresponding to each first display adapter and start generating display data.

[0068] It should be understood that the first phase can also include a shutdown phase, and the driving mode can also be changed in a similar manner, for example, after the IDD is closed, the DOD can re-create the first virtual display and start generating display data.

[0069] The method for cloud desktop display according to the embodiment of the present disclosure will be described below. Referring to FIG. 2, the method for cloud desktop display according to the embodiment of the present disclosure can include the following steps A101 to A108.

[0070] In step A101, when the virtualization host (virtual host) is started (start-up phase, first phase), the QXL graphics card (first display adapter) and the QXL display (first virtual display) are automatically activated.

[0071] In step A102, during the start-up process, the QXL graphics card obtains a system rendering image (original rendering display data) and sends the image information (display data, which can include the coordinates, grayscale, color, etc. of each pixel in the image) to the SPICE end (sending end) of the virtualization host, which sends the image information to the client (terminal) of the user for display.

[0072] In step A103, After the system (operating system) is normally started (second phase), the QXL (DOD) completes the initialization, and writes the LUID and device ID (identification of the first display adapter) of the QXL graphics card into the registry.

[0073] At step A104, the QXL creates a named event (a system event) named qxl_idd_event_x, x representing the ID of the QXL device.

[0074] At step A105, the IDD initializes, and after initialization is successful, opens the qxl_idd_event_x event according to the LUID in the registry, the device ID.

[0075] At step A1061, if the event is successfully opened, the event is triggered, each IDD display (second virtual display) corresponding to each QXL graphics card is added, and the correspondence between the IDD display and the LUID is recorded.

[0076] At step A1062, after the QXL receives the notification that the event is triggered, the QXL display is deleted, so that the QXL graphics card connects the IDD display and serves as a display output device of the virtualization host.

[0077] At step A107, the IDD controls the physical graphics card (physical display adapter, second display adapter) to receive a system rendering image, performs rendering to obtain image information, and according to the correspondence between the IDD display and the LUID, sends the image information to the QXL graphics card, for example, through a D3DKMTEscape call.

[0078] At step A108, after the QXL graphics card receives the image information, the image information is transmitted to the SPICE end of the virtualization host through the video memory, and the SPICE end sends the image information to the client for display.

[0079] Referring to FIG. 5, the embodiment of the present disclosure further provides a cloud desktop server, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method for cloud desktop display according to the embodiments of the present disclosure when the computer program is executed.

[0080] Referring to FIG. 6, the embodiment of the present disclosure further provides a computer readable medium, which stores a computer program, and the processor implements the method for cloud desktop display according to the embodiments of the present disclosure when the computer program is executed.

[0081] The embodiment of the present disclosure further provides a computer program product, which comprises a computer program, and the processor implements the method for cloud desktop display according to the embodiments of the present disclosure when the computer program is executed.

[0082] The processor is a device with data processing capability, including but not limited to a central processing unit (CPU) and the like; the memory is a device with data storage capability, including but not limited to a random access memory (RAM, more specifically SDRAM, DDR, etc.), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory (FLASH); the I / O interface (read-write interface) is connected between the processor and the memory, and can realize information interaction between the memory and the processor, including but not limited to a data bus (Bus) and the like.

[0083] Those of ordinary skill in the art can understand that all or some of the functional modules / units in the above disclosed steps, systems and devices can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0084] In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation.

[0085] Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit (CPU), a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH) or other magnetic disk storage; compact discs (CD-ROM), digital versatile discs (DVD) or other optical disk storage; magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage; any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically includes computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery medium.

[0086] The present disclosure has disclosed example embodiments, and while specific terminology has been employed, it is merely in the service of a general descriptive purpose and should not be construed as limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with a particular embodiment can be used in conjunction with other embodiments unless otherwise explicitly stated. As such, those skilled in the art will appreciate that various changes can be made in form and detail without departing from the scope of the disclosure as set forth in the appended claims.

Claims

1. A method for cloud desktop display, comprising: in a first stage, controlling a first display adapter to provide display data to a sending end of a virtual host by display on demand (DOD), wherein the first stage is a state in which an operating system of the virtual host is not running, and the sending end is a port for sending the display data to a terminal; in a second stage, controlling a second display adapter to generate display data by indirect display driver (IDD), and transmitting the display data to the first display adapter, wherein the first display adapter provides the display data to the sending end, and wherein the second stage is a state in which the operating system of the virtual host is running. 2.The method of claim 1, wherein the first stage comprises a start-up stage and / or a shutdown stage of the virtual host. 3.The method of claim 1, wherein the second display adapter comprises a physical display adapter and / or a virtual display adapter.

4. The method of claim 1, wherein, transmitting the display data to the first display adapter comprises: transmitting the display data to the first display adapter by IDD, or transmitting the display data generated by IDD to the first display adapter by a transmission application, providing the display data to the sending end by the first display adapter comprises: transmitting the display data to the sending end by a video memory of the first display adapter.

5. The method according to any one of claims 1 to 4, wherein, controlling the first display adapter to provide display data to the sending end of the virtual host by DOD comprises: transmitting raw rendering display data to the first display adapter; providing the raw rendering display data to the sending end by the first display adapter.

6. The method of any one of claims 1 to 4, wherein, controlling the second display adapter to generate display data by IDD comprises: generating the display data by the second display adapter with a second virtual display as an output, wherein the second virtual display corresponds to the first display adapter one-to-one.

7. The method of claim 6, wherein, the first stage comprises a start-up stage of the virtual host, in the start-up stage, the method further comprises: activating the first display adapter and a corresponding first virtual display, in the second stage, the method further comprises: recording an identifier of the first display adapter by DOD, and creating a system event pointing to the first display adapter; opening the system event according to the identifier of the first display adapter by IDD in response to initialization completion; in response to the system event being triggered, deleting the first virtual display by DOD, and creating the second virtual display corresponding to the first display adapter by IDD.

8. A cloud desktop server comprising a memory and a processor, wherein, The memory stores a computer program, and the computer program is executed by the processor to implement the method for cloud desktop display according to any one of claims 1 to 7. 9.A computer readable medium having stored thereon a computer program, wherein the computer program is executed by a processor to implement the method for cloud desktop display according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, wherein, The computer program is executed by the processor to implement the method for cloud desktop display according to any one of claims 1 to 7.

Images