An image processing method and device for a virtual desktop, a server and a storage medium

Abstract

The embodiment of the application discloses a virtual desktop image processing method, which comprises the following steps: after receiving an operation instruction representing that the display size of a virtual desktop changes at a client, determining the current display size of the virtual desktop; based on the alignment step length of a hardware encoder of a server and the current display size, performing expansion processing on original image data of the virtual desktop to obtain processed image data; the alignment step length represents the number of bytes of data read by the hardware encoder each time; reading the processed image data by the hardware encoder, and encoding the original image data in the processed image data to obtain desktop encoding data; and sending the desktop encoding data to the client. The embodiment of the application also discloses a virtual desktop image processing device, a server and a storage medium.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an image processing method, apparatus, server, and storage medium for a virtual desktop. Background Technology

[0002] In a desktop cloud system, when data is transmitted between the server and client cloud desktops, the encoder in the Simple Protocol for Independent Computing Environments (SPICE) encodes the image to be displayed on the cloud desktop to improve the transmission rate of the desktop cloud system. However, because the size of the client cloud desktop can be arbitrarily scalable, the size of the encoded image changes accordingly. Since the encoder reads information from the image in fixed byte counts, when the size of the cloud desktop changes, the encoder can no longer read information in fixed byte counts and must read it word by word. This reduces the encoder's information reading efficiency and increases the overall encoding time. Summary of the Invention

[0003] To address the aforementioned technical problems, this application aims to provide an image processing method, apparatus, server, and storage medium for virtual desktops. This solves the problem in related technologies where, after the size of the cloud desktop changes, the encoder cannot read information from the screen in fixed byte counts and can only read it word by word, resulting in reduced encoder efficiency and increased overall encoding time.

[0004] The technical solution of this application is implemented as follows:

[0005] An image processing method for a virtual desktop, the method comprising:

[0006] Upon receiving an operation instruction indicating a change in the display size of the virtual desktop on the client, the current display size of the virtual desktop is determined;

[0007] Based on the alignment step size of the server-side hardware encoder and the current display size, the original image data of the virtual desktop is augmented to obtain processed image data; the alignment step size represents the number of bytes read by the hardware encoder each time.

[0008] The hardware encoder reads the processed image data and encodes the original image data in the processed image data to obtain desktop encoded data.

[0009] The desktop encoded data is sent to the client.

[0010] In the above scheme, the change in display size includes:

[0011] The virtual desktop changes from full-screen mode to windowed mode; or,

[0012] The virtual desktop changes from the first window state to the second window state; or,

[0013] The virtual desktop switches from the first display screen to the second display screen.

[0014] In the above scheme, determining the current display size of the virtual desktop includes:

[0015] The current display size is determined based on the coordinates of the virtual desktop displayed on the client's screen sent by the client and the display size of the virtual desktop before it changed;

[0016] Accordingly, based on the alignment step size of the server-side hardware encoder and the current display size, the original image data of the virtual desktop is augmented to obtain processed image data, including:

[0017] Based on the current display size and the alignment step, the virtual width and virtual height of the processed image data are obtained;

[0018] The original image data is augmented based on the virtual width and virtual height to obtain the processed image data.

[0019] In the above scheme, obtaining the virtual width and virtual height of the processed image data based on the current display size and the alignment step size includes:

[0020] Determine the alignment step size of the hardware encoder;

[0021] The actual width and actual height of the virtual desktop are processed based on the alignment step to obtain the virtual height and the virtual height; the current display size includes the actual width and actual height.

[0022] In the above scheme, the step of processing the actual width and actual height of the virtual desktop based on the alignment step to obtain the virtual height includes:

[0023] Determine the difference between the alignment step size and the actual width and the actual height;

[0024] The actual width and actual height are expanded based on the difference to obtain the virtual width and virtual height; the virtual width and virtual height are integer multiples of the alignment step size.

[0025] In the above scheme, the step of augmenting the original image data based on the virtual height and the virtual height to obtain processed image data includes:

[0026] The original image data is augmented based on the virtual height and the virtual width to obtain augmented image data.

[0027] The expanded image data is filled in according to the target character to obtain the processed image data.

[0028] In the above scheme, the step of reading the processed image data through the hardware encoder and encoding the original image data in the processed image data to obtain desktop encoded data includes:

[0029] The processed image data is read by the hardware encoder according to the alignment step size.

[0030] The original image data is encoded using the hardware encoder to obtain the desktop encoded data.

[0031] An image processing apparatus for a virtual desktop, the apparatus comprising:

[0032] The processing unit is configured to determine the current display size of the virtual desktop after receiving an operation instruction indicating a change in the display size of the virtual desktop on the client.

[0033] The processing unit is further configured to perform augmentation processing on the original image data of the virtual desktop based on the alignment step size of the server-side hardware encoder and the current display size to obtain processed image data; the alignment step size represents the number of bytes read by the hardware encoder each time.

[0034] The processing unit is also configured to read the processed image data through the hardware encoder and encode the original image data in the processed image data to obtain desktop encoded data;

[0035] A sending unit is used to send the desktop encoded data to the client.

[0036] A server, the server comprising: a processor, a memory, and a communication bus;

[0037] The communication bus is used to realize the communication connection between the processor and the memory;

[0038] The processor is used to execute the image processing program of the virtual desktop in the memory to implement the steps of the above-described image processing method for the virtual desktop.

[0039] A storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the above-described virtual desktop image processing method.

[0040] The image processing method, apparatus, server, and storage medium for virtual desktops provided in the embodiments of this application can, upon receiving an operation instruction indicating a change in the display size of the virtual desktop on the client, determine the current display size of the virtual desktop. Then, based on the alignment step size of the server's hardware encoder and the current display size, it performs augmentation processing on the original image data of the virtual desktop to obtain processed image data. The processed image data is then read through the hardware encoder, and the original image data within the processed image data is encoded to obtain desktop encoded data, which is then sent to the client. Thus, since the processed image data is augmented based on the alignment step size of the hardware encoder on the original image data of the virtual desktop... The image data obtained after processing is matched with the alignment step size of the hardware encoder. Therefore, the hardware encoder can read the processed image data according to the alignment step size, without any bytes that need to be read separately. The data reading rate is high, and the encoding only encodes the original image data in the processed image data, without adding unnecessary operations. This reduces the processing time, improves the data encoding efficiency, and thus improves the overall performance of the virtual desktop. It solves the problem in related technologies where, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to a fixed number of bytes and can only read it word by word, which reduces the efficiency of the encoder in reading information and increases the overall encoding time. Attached Figure Description

[0041] Figure 1 A flowchart illustrating an image processing method for a virtual desktop provided in an embodiment of this application;

[0042] Figures 2(a) and 2(b) are schematic diagrams showing the change in the display size of the virtual desktop in an image processing method for a virtual desktop provided in an embodiment of this application;

[0043] Figure 3 A flowchart illustrating another image processing method for a virtual desktop provided in an embodiment of this application;

[0044] Figure 4 A flowchart illustrating another virtual desktop image processing method provided in this application embodiment;

[0045] Figure 5 A flowchart illustrating an image processing method for a virtual desktop, provided as another embodiment of this application;

[0046] Figure 6 A schematic diagram of the structure of an image processing device for a virtual desktop provided in an embodiment of this application;

[0047] Figure 7This is a schematic diagram of a server structure provided in an embodiment of this application. Detailed Implementation

[0048] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0049] It should be understood that the phrases "embodiments of this application" or "foreign embodiments" throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "embodiments of this application" or "in the foreign embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0050] Unless otherwise specified, any step in the embodiments of this application performed by the electronic device may be executed by the processor of the electronic device. It is also worth noting that the embodiments of this application do not limit the order in which the electronic device performs the following steps. Furthermore, the methods used to process data in different embodiments may be the same or different methods. It should also be noted that any step in the embodiments of this application can be executed independently by the electronic device; that is, when the electronic device performs any step in the following embodiments, it may not depend on the execution of other steps.

[0051] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0052] This application provides an image processing method for a virtual desktop, which can be applied to a server. (Refer to...) Figure 1 As shown, the method includes the following steps:

[0053] Step 101: After receiving the operation instruction indicating that the display size of the virtual desktop on the client has changed, determine the current display size of the virtual desktop.

[0054] Virtual desktops are desktops that are virtualized on the server and can be accessed by clients.

[0055] In this embodiment, the virtual desktop can be a desktop virtualized on the server and accessible through the client; a virtual desktop is also known as a cloud desktop. Operation commands are used to indicate changes in the display size of the virtual desktop on the client. The current display size of the virtual desktop can be the changed display size of the virtual desktop. In one feasible implementation, when the virtual desktop changes from full-screen to windowed mode, it is determined that the display size of the virtual desktop on the client has changed. At this time, the current display size of the virtual desktop is the display size in the windowed mode, where windowed mode means that the virtual desktop is in a non-full-screen state. When the virtual desktop changes from a first windowed state to a second windowed state, it is determined that the display size of the virtual desktop on the client has changed. At this time, the current display size of the virtual desktop is the display size in the second windowed state, where the display size in the first windowed state is different from the display size in the second windowed state, that is, the display size in the first windowed state may be larger or smaller than the display size in the second windowed state. When the virtual desktop switches from a first display screen to a second display screen, it is determined that the display size of the virtual desktop on the client has changed. At this time, the current display size of the virtual desktop is the display size in the second display screen, where the screen size of the first display screen is different from the screen size of the second display screen. For example, as shown in Figures 2(a) and 2(b), the virtual desktop is in the first window state in Figure 2(a) and in the second window state in Figure 2(b). Obviously, the display size in the first window state shown in Figure 2(a) is different from the display size in the second window state shown in Figure 2(b). If the virtual desktop changes from the first window state shown in Figure 2(a) to the second window state shown in Figure 2(b), then it is determined that the display size of the virtual desktop on the client has changed.

[0056] In one feasible approach, the client's external devices, such as mouse and keyboard, can send an operation command indicating a change in the virtual desktop's display size to the server. Upon receiving the operation command from the client, the server determines the current display size of the virtual desktop based on the command.

[0057] Step 102: Based on the alignment step size of the server-side hardware encoder and the current display size, the original image data of the virtual desktop is augmented to obtain the processed image data.

[0058] The alignment step size represents the number of bytes of data read by the hardware encoder each time.

[0059] In this embodiment, the original image data can be the image data displayed on the client after the virtual desktop has changed. For example, the original image data can be the content displayed in the window when the virtual desktop is in the second window state in Figure 2(b). The processed image data is the image data after processing the original image data based on the alignment step size of the hardware encoder and the current display size. At this time, the processed image data is the image data that matches the alignment step size of the hardware encoder. That is, the hardware encoder can read the processed image data in one go. Since the hardware encoder reads the image data according to the alignment step size, in order to prevent the hardware encoder from not being able to read the original image data in one go, it is necessary to expand the original image data based on the alignment step size of the hardware encoder and the current display size so that the processed image data can be read by the hardware encoder in one go, so that there is no situation where data needs to be read word by word. The hardware encoder is used to process the raw image data. Further, it reads the image data displayed on the client side of the virtual desktop, compresses the image data, and transmits it to the client. By compressing the image data, the communication traffic of the cloud desktop system is reduced, and the data transmission rate is improved. In one feasible implementation, the hardware encoder can be an Advanced Video Coding (H.264) protocol or a High Efficiency Video Coding (H.265) protocol. Of course, the hardware encoder can also be other encoders, and this application embodiment does not limit this. The alignment step size of the hardware encoder represents the number of bytes of data read sequentially. This alignment step size can be preset, generally set to a multiple of 2, and can be set to 32 or other values ​​according to actual business needs. This application embodiment does not limit this. Typically, once the alignment step size of the hardware encoder is set, it will not be changed arbitrarily; that is, it is fixed.

[0060] In this embodiment, the original image data is expanded based on the alignment step size of the hardware encoder and the current display size to obtain processed image data whose data size matches the alignment step size. This is so that even if the display size of the virtual desktop changes, the changed image data (i.e., the original image data) can still be read according to the alignment step size of the hardware encoder, without any bytes that need to be read separately. This improves the data reading rate, thereby improving the encoding rate and thus enhancing the overall performance of the cloud desktop system.

[0061] In one feasible implementation, the alignment step size of the hardware encoder can be determined, and then the original image data can be augmented according to the alignment step size and the current display size to obtain processed image data whose data size matches the alignment step size.

[0062] Step 103: Read the processed image data through a hardware encoder and encode the original image data in the processed image data to obtain desktop encoded data.

[0063] In this embodiment, the desktop encoded data is the encoded original image data within the processed image data. That is, when using a hardware encoder for encoding, only the original image data within the processed image data is encoded, not the processed image data itself. The hardware encoder reads the processed image data and encodes the original image data within it. In essence, although processed image data is read, only the original image data is encoded. Since the size of the processed image data matches the alignment step size of the hardware encoder, the hardware encoder can read the processed image data according to the alignment step size, without encountering bytes that need to be read separately, resulting in a high data reading rate. During encoding, the original image data is encoded normally, without encoding any other data. Therefore, the encoding operation of the original image data is guaranteed, and the resulting desktop encoded data is only the encoded information of the original image data, without generating unnecessary operations that could affect the desktop cloud system. This improves data encoding efficiency and enhances the overall performance of the cloud desktop system.

[0064] In one feasible approach, the processed image data can be read by a hardware encoder according to the alignment step size, and the original image data in the processed image data can be encoded to obtain desktop encoded data.

[0065] In other embodiments of this application, an encoding acceleration card can be set in the server. This encoding acceleration card can be a chip specifically used for encoding data. In this way, the steps of reading the processed image data by the hardware encoder and encoding the original image data in the processed image data to obtain desktop encoded data can be performed on the encoding acceleration card, which can further improve the encoding efficiency and reduce the resource consumption of the cloud desktop system.

[0066] Step 104: Send the desktop encoding data to the client.

[0067] In this embodiment of the application, after the desktop encoded data is sent to the client, the client can display the original image data on the target desktop based on the desktop encoded data; in one possible way, after receiving the desktop encoded data, the client can decode the desktop encoded data to obtain the original image data, and then display the original image data on the client's virtual desktop.

[0068] In this embodiment, in related technologies, when the display size of the virtual desktop changes, the display content corresponding to the changed virtual desktop does not match the step size of the target encoder, resulting in bytes that need to be read separately and a low information reading rate. However, the image processing method for the virtual desktop in this embodiment reads processed image data that matches the alignment step size of the hardware encoder. Therefore, the processed image data can be read completely by aligning the step size, and there are no bytes that need to be read separately. The data reading rate is higher than that in related technologies. Furthermore, the processing only processes the original image data in the processed image data, so there are no unnecessary operations that would affect the desktop cloud system. This reduces the processing time, improves information encoding efficiency, and thus improves the overall performance of the cloud desktop system.

[0069] The image processing method for virtual desktops provided in this application provides image data that is obtained by expanding the original image data of the virtual desktop based on the alignment step size of the hardware encoder. Therefore, the size of the processed image data matches the alignment step size of the hardware encoder. Thus, the hardware encoder can read the processed image data according to the alignment step size, without any bytes that need to be read separately. The data reading rate is high, and the encoding only encodes the original image data in the processed image data, without adding unnecessary operations. This reduces the processing time, improves data encoding efficiency, and thus improves the overall performance of the virtual desktop. It solves the problem in related technologies where, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to a fixed number of bytes and can only read it word by word, resulting in reduced encoder reading efficiency and increased overall encoding time.

[0070] Based on the foregoing embodiments, this application provides an image processing method for a virtual desktop, referring to... Figure 3 As shown, the method includes the following steps:

[0071] Step 201: After receiving the operation instruction indicating that the display size of the virtual desktop on the client has changed, the server determines the current display size based on the coordinates of the virtual desktop on the client's screen sent by the client and the display size of the virtual desktop before the change.

[0072] The changes in display size include: the virtual desktop changing from full-screen mode to windowed mode; or, the virtual desktop changing from a first window mode to a second window mode; or, the virtual desktop switching from a first display screen to a second display screen.

[0073] In this embodiment, the operation instruction can carry the coordinates of the virtual desktop displayed on the client's screen. This allows the coordinates of the virtual desktop on the client to be determined, and then the current display size can be determined based on the display coordinates and the virtual desktop's original display size. The current display size is the display size of the virtual desktop after the change. For example, the original display size can be the display size of the virtual desktop in the first window state as shown in Figure 2(a), the display coordinates can be the coordinates of the virtual desktop in the second window state as shown in Figure 2(b), and the current display size can be the display size of the virtual desktop in the second window state as shown in Figure 2(b).

[0074] In one feasible approach, the display size of the virtual desktop before the change can be obtained from the server's own memory. Then, based on the display coordinates carried in the operation instructions and the display size before the change, the change of the virtual desktop can be determined, thereby determining the display size of the virtual desktop after the change (i.e., the current display size).

[0075] Step 202: The server obtains the virtual width and virtual height of the processed image data based on the current display size and alignment step.

[0076] In this embodiment, the virtual width and virtual height of the processed image data are obtained by processing the current display size based on the alignment step. In one possible approach, the virtual width and virtual height of the processed image data can be obtained by expanding the current display size based on the alignment step.

[0077] Step 202 can be achieved through the following steps:

[0078] Step 202a: The server determines the alignment step size of the hardware encoder.

[0079] The alignment step size represents the number of bytes of data read by the hardware encoder each time.

[0080] In this embodiment of the application, the step size of the hardware encoder can be preset, so that the server can perform subsequent processing based on the alignment step size of the hardware encoder.

[0081] Step 202b: The server processes the actual width and height of the virtual desktop based on the alignment step size to obtain the virtual width and virtual height.

[0082] The current display size includes the actual width and the actual height.

[0083] In this embodiment, since the hardware encoder reads data according to the alignment step, the display size of the original image data needs to be adjusted to match the alignment step to facilitate subsequent image data reading. The actual width and actual height in the current display size can be the width and height of the virtual desktop in the second window state as shown in Figure 2(b), while the virtual width and virtual height are the width and height that match the alignment step obtained after processing the actual width and actual height.

[0084] In one feasible approach, the actual width and actual height can be mapped separately based on the alignment step to obtain virtual width and virtual height that match the alignment step.

[0085] In one feasible approach, if the alignment step is 32, the actual width is 63, and the actual height is 31, then the virtual height obtained by mapping the actual height based on the alignment step is 32, and the virtual width obtained by mapping the actual width based on the alignment step is 64. That is to say, the virtual width and virtual height adjusted based on the alignment step must be divisible by 32, at which point both the virtual height and the virtual height match the alignment step.

[0086] Step 203: The server expands the original image data according to the virtual width and virtual height to obtain the processed image data.

[0087] In this embodiment, the original image data is expanded according to the virtual width and virtual height in order to make the processed image data match the alignment step size, so that when the data is read by the hardware encoder, it can be read in one go, and there will be no bytes that need to be read separately, thereby improving the data reading efficiency.

[0088] In this embodiment, the display size corresponding to the actual width and actual height (i.e., the current display size) can be referred to as the first display size, and the virtual width and virtual height can be referred to as the second display size. Since the second display size is the display size after expanding the first display size based on the alignment step, the second display size is larger than the first display size. At this time, it is also necessary to expand the original image data with the first display size to the processed image data with the second display size. Since the second display size matches the alignment step, the information size of the obtained processed image data matches the alignment step.

[0089] In one feasible approach, the content in the original image data can be filled in according to the virtual width and virtual height to obtain the processed image data.

[0090] Step 204: The server reads the processed image data according to the alignment step size using a hardware encoder.

[0091] In this embodiment, since the processed image data matches the alignment step size of the hardware encoder, the processed image data can be read completely by the hardware encoder according to the alignment step size, and there are no bytes that need to be read separately, thus improving the data reading speed.

[0092] Step 205: The server encodes the original image data using a hardware encoder to obtain desktop encoded data.

[0093] In this embodiment of the application, a hardware encoder can be used to encode only the original image data in the processed image data, thereby performing the operation of encoding the original image data without adding the operation of encoding the filling content. Therefore, no extra operations will be generated that will affect the desktop cloud system.

[0094] Step 206: The server sends the desktop encoding data to the client.

[0095] In this embodiment, the first display size is adjusted based on the alignment step size of the hardware encoder to obtain a second display size that matches the step size. Then, the first display information is expanded based on the second display size to obtain processed image data whose data size matches the alignment step size. Thus, during encoding, the processed image data is read by the hardware encoder according to the alignment step size, which can improve the data reading rate. The hardware encoder only encodes the original image data in the processed image data and does not add operations to encode the supplementary content. Therefore, no extra operations will be generated that will affect the desktop cloud system, thereby improving the data encoding efficiency and the overall performance of the cloud desktop system.

[0096] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0097] The image processing method for virtual desktops provided in this application provides image data that is obtained by expanding the original image data of the virtual desktop based on the alignment step size of the hardware encoder. Therefore, the size of the processed image data matches the alignment step size of the hardware encoder. Thus, the hardware encoder can read the processed image data according to the alignment step size, without any bytes that need to be read separately. The data reading rate is high, and the encoding only encodes the original image data in the processed image data, without adding unnecessary operations. This reduces the processing time, improves data encoding efficiency, and thus improves the overall performance of the virtual desktop. It solves the problem in related technologies where, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to a fixed number of bytes and can only read it word by word, resulting in reduced encoder reading efficiency and increased overall encoding time.

[0098] Based on the foregoing embodiments, this application provides an image processing method for a virtual desktop, referring to... Figure 4 As shown, the method includes the following steps:

[0099] Step 301: After receiving the operation instruction indicating that the display size of the virtual desktop on the client has changed, the server determines the current display size based on the coordinates of the virtual desktop on the client's screen sent by the client and the display size of the virtual desktop before the change.

[0100] The changes in display size include: the virtual desktop changing from full-screen mode to windowed mode; or, the virtual desktop changing from a first window mode to a second window mode; or, the virtual desktop switching from a first display screen to a second display screen.

[0101] In this embodiment of the application, the server and the client can communicate through a target transmission protocol, which includes, but is not limited to, SPICE protocol, Remote Desktop Protocol (RDP), Personal Computer over Internet Protocol (PCoIP), Independent Computing Architecture (ICA) protocol, High Definition Experience (HDX) protocol, and Remote Frame Buffer (RFB) protocol.

[0102] In one feasible approach, the display coordinates carried in the operation command can be obtained through the target transmission protocol (SPICE protocol), and then the current display size can be obtained based on the display coordinates and the display size before the change.

[0103] Step 302: The server determines the alignment step size of the hardware encoder.

[0104] The step size represents the number of bytes of data read by the target encoder each time.

[0105] Step 303: The server determines the difference between the alignment step size and the actual width and actual height.

[0106] The current display size includes the actual width and the actual height.

[0107] In this embodiment, the second display size (i.e., virtual width and virtual height) is matched with the alignment step, that is, the second display size can be divided by the alignment step, that is, the relationship between the second display size and the alignment step is a multiple relationship. Therefore, the difference between the alignment step and the first display size (i.e., actual width and actual height) can be determined based on the multiple relationship. In one feasible way, if the alignment step is 32, the actual width of the virtual desktop is 63, and the actual height in the virtual current display ruler is 31, then the difference between the alignment step and the actual width and actual height is 1.

[0108] Step 304: The server expands the actual width and actual height based on the differences to obtain the virtual width and virtual height.

[0109] In this embodiment, the actual width of the first display size and its corresponding difference can be calculated, and the actual height of the first display size and its corresponding difference can be calculated to obtain the virtual height and virtual width under the second display size; in one possible way, the actual width and its corresponding difference can be added together, and the actual height and its corresponding difference can be added together to obtain the virtual width and virtual height (i.e., the second display size).

[0110] Step 305: The server expands the original image data according to the virtual width and virtual height to obtain the expanded image data.

[0111] In this embodiment of the application, the original image data is expanded based on the virtual width and virtual height, that is, the display size of the original image data is expanded to a second display size (i.e., virtual width and virtual height).

[0112] Step 306: The server fills in the expanded image data based on the target characters to obtain the processed image data.

[0113] In this embodiment, the target character can be a character that does not affect the display content of the first display information. Thus, the content filled in the expanded first display information based on the target character will not affect subsequent encoding. In one feasible approach, the target character can be 0. In this case, the expanded portion of the expanded image data is filled with 0s to obtain the processed image data.

[0114] In this embodiment, since the processed image data is stored in the server's memory, and the data is stored in multiples of the alignment step size, the processed image data is memory aligned during storage, and there is no free memory.

[0115] Step 307: The server reads the processed image data according to the alignment step size using a hardware encoder.

[0116] In this embodiment of the application, the hardware encoder can be invoked through the target transmission protocol to read the processed image data according to the alignment step size. That is, the H265 encoder can be invoked through the SPICE protocol to read the processed image data according to the alignment step size.

[0117] Step 308: The server encodes the original image data using a hardware encoder to obtain desktop encoded data.

[0118] In this embodiment, the hardware encoder is invoked via the target transmission protocol to read the processed image data into the encoding accelerator card according to the alignment step size, so as to perform the encoding operation on the encoding accelerator card to encode the original image data in the processed image data to obtain desktop encoded data.

[0119] Step 309: The server sends the desktop encoding data to the client.

[0120] In this embodiment, the expanded image data is filled with the target character to obtain the processed image data. The filled content does not affect the original image data, so the hardware encoder can encode only the original image data in the processed image data without adding the operation of encoding the filled content. Therefore, no extra operation will be generated that will affect the desktop cloud system, thereby improving the data encoding efficiency and improving the overall performance of the cloud desktop system.

[0121] like Figure 5 As shown, users can arbitrarily resize the cloud desktop via the client. The client sends an instruction to the server indicating a change in the cloud desktop's size. Upon receiving this instruction, the server obtains the actual width and height of the cloud desktop via the SPICE protocol. Then, it adjusts the actual width and height of the cloud desktop according to the alignment step size of the H.265 encoder to obtain the aligned virtual height and virtual height. Next, it expands the original image data based on the virtual height and virtual height to obtain processed image data. The H.265 encoder reads the processed image data according to the alignment step size and encodes the original image data within the processed image data to obtain desktop encoded data. This desktop encoded data is then sent to the client, enabling the client to display the content corresponding to the original image data on the cloud desktop based on the desktop encoded data.

[0122] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0123] The image processing method for virtual desktops provided in this application provides image data that is obtained by expanding the original image data of the virtual desktop based on the alignment step size of the hardware encoder. Therefore, the size of the processed image data matches the alignment step size of the hardware encoder. Thus, the hardware encoder can read the processed image data according to the alignment step size, without any bytes that need to be read separately. The data reading rate is high, and the encoding only encodes the original image data in the processed image data, without adding unnecessary operations. This reduces the processing time, improves data encoding efficiency, and thus improves the overall performance of the virtual desktop. It solves the problem in related technologies where, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to a fixed number of bytes and can only read it word by word, resulting in reduced encoder reading efficiency and increased overall encoding time.

[0124] Based on the foregoing embodiments, this application provides an image processing apparatus for a virtual desktop, which can be applied to... Figure 1 and 3 In the image processing method for virtual desktops provided in embodiments ~4, refer to Figure 6 As shown, the image processing device 4 of the virtual desktop may include:

[0125] The processing unit 41 is used to determine the current display size of the virtual desktop after receiving an operation instruction indicating that the display size of the virtual desktop on the client has changed;

[0126] The processing unit 41 is also used to expand the original image data of the virtual desktop based on the alignment step size of the server-side hardware encoder and the current display size to obtain the processed image data; the alignment step size represents the number of bytes of data read by the hardware encoder each time.

[0127] The processing unit 41 is also used to read the processed image data through a hardware encoder and encode the original image data in the processed image data to obtain desktop encoded data.

[0128] The sending unit 42 is used to send desktop encoded data to the client.

[0129] In other embodiments of this application, the display size changes, including:

[0130] The virtual desktop changes from full-screen mode to windowed mode; or,

[0131] The virtual desktop changes from the first window state to the second window state; or,

[0132] The virtual desktop switches from the first display screen to the second display screen.

[0133] In other embodiments of this application, the processing unit 41 is further configured to perform the following steps:

[0134] The current display size is determined based on the coordinates of the virtual desktop sent by the client on the client's screen and the display size of the virtual desktop before it changed.

[0135] Accordingly, the processing unit 41 is also used to implement the following steps:

[0136] Based on the current display size and alignment step, the virtual width and virtual height of the processed image data are obtained;

[0137] The original image data is augmented based on the virtual width and virtual height to obtain the processed image data.

[0138] In other embodiments of this application, the processing unit 41 is further configured to perform the following steps:

[0139] Determine the alignment step size of the hardware encoder;

[0140] The actual width and height of the virtual desktop are processed based on the alignment step to obtain the virtual height and virtual height; the current display size includes the actual width and actual height.

[0141] In other embodiments of this application, the processing unit 41 is further configured to perform the following steps:

[0142] Determine the difference between the alignment step size and the actual width and actual height;

[0143] The actual width and actual height are expanded based on the difference to obtain virtual width and virtual height; the virtual width and virtual height are integer multiples of the alignment step size.

[0144] In other embodiments of this application, the processing unit 41 is further configured to perform the following steps:

[0145] The original image data is augmented based on the virtual height and virtual width to obtain the augmented image data.

[0146] The expanded image data is filled in according to the target character to obtain the processed image data.

[0147] In other embodiments of this application, the processing unit 41 is further configured to perform the following steps:

[0148] The processed image data is read using a hardware encoder according to the alignment step size;

[0149] The original image data is encoded using a hardware encoder to obtain desktop encoded data.

[0150] It should be noted that a detailed explanation of the steps performed by each unit can be found in [reference needed]. Figure 1 and 3 The implementation process of the virtual desktop image processing method provided in the embodiments corresponding to ~4 will not be described in detail here.

[0151] The image processing apparatus for virtual desktops provided in this application provides image data that is obtained by expanding the original image data of the virtual desktop based on the alignment step size of the hardware encoder. Therefore, the size of the processed image data matches the alignment step size of the hardware encoder. Thus, the hardware encoder can read the processed image data according to the alignment step size, without any bytes that need to be read separately. The data reading rate is high, and during encoding, only the original image data in the processed image data is encoded, without adding any extra operations. This reduces the processing time, improves the data encoding efficiency, and thus improves the overall performance of the virtual desktop. It solves the problem in related technologies where, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to a fixed number of bytes and can only read it word by word, resulting in reduced encoder reading efficiency and increased overall encoding time.

[0152] Based on the foregoing embodiments, embodiments of this application provide a server that can be applied to... Figure 1 and 3 In the image processing method for virtual desktops provided in embodiments ~4, refer to Figure 7 As shown, the server 5 may include: a processor 51, a memory 52, and a communication bus 53, wherein:

[0153] Communication bus 53 is used to realize the communication connection between processor 51 and memory 52;

[0154] Processor 51 is used to execute the image processing program of the virtual desktop in memory 52 to perform the following steps:

[0155] Upon receiving an operation command indicating a change in the display size of the virtual desktop on the client, determine the current display size of the virtual desktop;

[0156] Based on the alignment step size of the server-side hardware encoder and the current display size, the original image data of the virtual desktop is augmented to obtain the processed image data; the alignment step size represents the number of bytes read by the hardware encoder each time.

[0157] The processed image data is read by a hardware encoder, and the original image data in the processed image data is encoded to obtain desktop encoded data.

[0158] Send desktop-encoded data to the client.

[0159] In other embodiments of this application, the display size changes, including:

[0160] The virtual desktop changes from full-screen mode to windowed mode; or,

[0161] The virtual desktop changes from the first window state to the second window state; or,

[0162] The virtual desktop switches from the first display screen to the second display screen.

[0163] In other embodiments of this application, processor 51 is used to execute an image processing program for the virtual desktop in memory 52 to determine the current display size of the virtual desktop in order to perform the following steps:

[0164] The current display size is determined based on the coordinates of the virtual desktop sent by the client on the client's screen and the display size of the virtual desktop before it changed.

[0165] Accordingly, processor 51 executes the image processing program for the virtual desktop in memory 52, based on the alignment step size of the server-side hardware encoder and the current display size, to augment the original image data of the virtual desktop and obtain processed image data, in order to perform the following steps:

[0166] Based on the current display size and alignment step, the virtual width and virtual height of the processed image data are obtained;

[0167] The original image data is augmented based on the virtual width and virtual height to obtain the processed image data.

[0168] In other embodiments of this application, the processor 51 is used to execute the image processing program of the virtual desktop in the memory 52 to obtain the virtual width and virtual height of the processed image data based on the current display size and alignment step, in order to implement the following steps:

[0169] Determine the alignment step size of the hardware encoder;

[0170] The actual width and height of the virtual desktop are processed based on the alignment step to obtain the virtual height and virtual height; the current display size includes the actual width and actual height.

[0171] In other embodiments of this application, the processor 51 is used to execute an image processing program for the virtual desktop in the memory 52 to process the actual width and actual height of the virtual desktop based on an alignment step size, to obtain a virtual height and a virtual depth, in order to perform the following steps:

[0172] Determine the difference between the alignment step size and the actual width and actual height;

[0173] The actual width and actual height are expanded based on the difference to obtain virtual width and virtual height; the virtual width and virtual height are integer multiples of the alignment step size.

[0174] In other embodiments of this application, the processor 51 is used to execute the image processing program of the virtual desktop in the memory 52 to augment the original image data according to the virtual height and virtual height, so as to obtain the processed image data and implement the following steps:

[0175] The original image data is augmented based on the virtual height and virtual width to obtain the augmented image data.

[0176] The expanded image data is filled in according to the target character to obtain the processed image data.

[0177] In other embodiments of this application, the processor 51 is used to execute the image processing program of the virtual desktop in the memory 52, read the processed image data through a hardware encoder, and encode the original image data in the processed image data to obtain desktop encoded data, in order to implement the following steps:

[0178] The processed image data is read using a hardware encoder according to the alignment step size;

[0179] The original image data is encoded using a hardware encoder to obtain desktop encoded data.

[0180] It should be noted that a detailed description of the steps performed by the processor can be found in [reference needed]. Figure 1 and 3 The implementation process of the virtual desktop image processing method provided in the embodiments corresponding to ~4 will not be described in detail here.

[0181] The server provided in this application provides a method for processing image data. Since the processed image data is obtained by expanding the original image data of the virtual desktop based on the alignment step size of the hardware encoder, the data size of the processed image data matches the alignment step size of the hardware encoder. Therefore, the hardware encoder can read the processed image data according to the alignment step size, and there are no bytes that need to be read separately. The data reading rate is high, and the encoding only encodes the original image data in the processed image data, without adding unnecessary operations. This reduces the processing time, improves the data encoding efficiency, and thus improves the overall performance of the virtual desktop. This solves the problem in related technologies where, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to a fixed number of bytes and can only read it word by word, resulting in reduced encoder reading efficiency and increased overall encoding time.

[0182] Based on the foregoing embodiments, embodiments of this application provide a storage medium that stores one or more programs, which can be executed by one or more processors to achieve... Figure 1 and 3 The steps of the image processing method for virtual desktops provided in embodiments ~4.

[0183] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0184] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0185] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0186] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0187] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. An image processing method for a virtual desktop, characterized in that, The method is applied to the server side and includes: Upon receiving an operation instruction indicating a change in the display size of the virtual desktop on the client, the current display size of the virtual desktop is determined; Based on the alignment step size of the server-side hardware encoder and the current display size, the original image data of the virtual desktop is expanded to obtain processed image data; the alignment step size represents the number of bytes read by the hardware encoder each time; the processed image data is obtained by expanding the display size of the original image data. The hardware encoder reads the processed image data and encodes the original image data in the processed image data to obtain desktop encoded data. The desktop encoded data is sent to the client.

2. The method according to claim 1, characterized in that, The change in display size includes: The virtual desktop changes from full-screen mode to windowed mode; or, The virtual desktop changes from the first window state to the second window state; or, The virtual desktop switches from the first display screen to the second display screen.

3. The method according to claim 1, characterized in that, Determining the current display size of the virtual desktop includes: The current display size is determined based on the coordinates of the virtual desktop displayed on the client's screen sent by the client and the display size of the virtual desktop before it changed; Accordingly, based on the alignment step size of the server-side hardware encoder and the current display size, the original image data of the virtual desktop is augmented to obtain processed image data, including: Based on the current display size and the alignment step, the virtual width and virtual height of the processed image data are obtained; The original image data is augmented based on the virtual width and virtual height to obtain the processed image data.

4. The method according to claim 3, characterized in that, The process of obtaining the virtual width and virtual height of the processed image data based on the current display size and the alignment step includes: Determine the alignment step size of the hardware encoder; The actual width and actual height of the virtual desktop are processed based on the alignment step to obtain the virtual height and the virtual height; the current display size includes the actual width and actual height.

5. The method according to claim 4, characterized in that, The process of processing the actual width and actual height of the virtual desktop based on the alignment step size to obtain the virtual height includes: Determine the difference between the alignment step size and the actual width and the actual height; The actual width and actual height are expanded based on the difference to obtain the virtual width and virtual height; the virtual width and virtual height are integer multiples of the alignment step size.

6. The method according to claim 3, characterized in that, The step of augmenting the original image data based on the virtual width and the virtual height to obtain processed image data includes: The original image data is augmented based on the virtual height and the virtual width to obtain augmented image data. The expanded image data is filled in according to the target character to obtain the processed image data.

7. The method according to any one of claims 1-6, characterized in that, The step of reading the processed image data through the hardware encoder and encoding the original image data in the processed image data to obtain desktop encoded data includes: The processed image data is read by the hardware encoder according to the alignment step size; The original image data is encoded using the hardware encoder to obtain the desktop encoded data.

8. An image processing device for a virtual desktop, characterized in that, The device includes: The processing unit is configured to determine the current display size of the virtual desktop after receiving an operation instruction indicating a change in the display size of the virtual desktop on the client. The processing unit is further configured to expand the original image data of the virtual desktop based on the alignment step size of the server-side hardware encoder and the current display size to obtain processed image data; the alignment step size represents the number of bytes read by the hardware encoder each time; the processed image data is obtained by expanding the display size of the original image data. The processing unit is also configured to read the processed image data through the hardware encoder and encode the original image data in the processed image data to obtain desktop encoded data; A sending unit is used to send the desktop encoded data to the client.

9. A server-side component, characterized in that, The server includes: a processor, a memory, and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the image processing program of the virtual desktop in the memory to implement the steps of the image processing method of the virtual desktop as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the image processing method for the virtual desktop as described in any one of claims 1 to 7.

Images