Image data processing method and cloud desktop and zero terminal

Abstract

The disclosure provides an image data processing method, a cloud desktop and a zero terminal. The method comprises the following steps: instructing a cloud application host and sending collected cloud application picture data and cloud application window information to the zero terminal, and sending the cloud application window information to the cloud desktop; receiving the cloud application window information sent by the cloud application host, creating a cloud application shadow window corresponding to the cloud application window information according to the cloud application window information, and obtaining cloud application shadow window information according to the cloud application shadow window; collecting local window information and local picture data, and sending the local window information, the local picture data and the cloud application shadow window information to the zero terminal; instructing the zero terminal to replace the cloud application shadow window information with the cloud application window information, calculating region information of an area in the cloud application picture data which is not covered by the local picture data according to the cloud application window information and the local window information, and performing transparent display on a corresponding position of the local picture data according to the region information.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to an image data processing method, a cloud desktop, and a zero-terminal. Background Technology

[0002] Traditional cloud application usage methods based on cloud desktops, such as Figure 1 As shown, the user connects to their office desktop via a terminal, which then connects to the cloud application host. After acquiring the cloud application image, rendering and window composition are performed on the office desktop, such as overlaying and masking the cloud application with local applications opened on the desktop. Finally, an image of the office desktop is captured, which now includes the cloud application image, and displayed on the terminal. The cloud application image undergoes two encoding / decoding processes: one from the cloud application host to the office host, and another after rendering and composition on the office host. Only then can the cloud application image be displayed on the terminal. The user experience of the cloud application screen depends entirely on the quality of the office host screen, and the latency increases due to the two encoding / decoding processes.

[0003] To address the above issues, optimized solutions include: Figure 2 As shown, this can effectively solve the problems of double encoding / decoding and performance dependency. However, the optimized solution has the following problems: In Display, borderless image compositing can lead to a time lag between the position of the shadow window and the DESKTOP window during compositing and the actual position of the app on the cloud. If Display completely uses the positional relationship calculated by DESKTOP for setting transparent areas, the ultimately exposed lower-layer app area may be out of sync with the actual position, resulting in a certain lag and thus displaying part of the app's desktop screen. Summary of the Invention

[0004] The purpose of this disclosure is to overcome the shortcomings of the prior art and provide an image data processing method, a cloud desktop, and a zero terminal. This image data processing method can solve the problem in the existing optimization solutions where the zero terminal only displays a part of the desktop screen where the APP is located because the lower-level APP area displayed on the zero terminal is not synchronized with the actual location.

[0005] According to a first aspect of the present disclosure, an image data processing method is provided, applied to a cloud desktop, the method comprising: The cloud application host is instructed to collect cloud application screen data and cloud application window information, and send the cloud application screen data and cloud application window information to the zero terminal and send the cloud application window information to the cloud desktop. Receive cloud application window information sent by the cloud application host, create a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtain the cloud application shadow window information according to the cloud application shadow window. Collect local window information and local screen data, and package the local window information, local screen data and cloud application shadow window information and send them to the zero terminal; The terminal is instructed to replace the cloud application shadow window information with the cloud application window information, calculate the area information in the cloud application screen data that is not covered by the local screen data based on the cloud application window information and the local window information, and display the corresponding position of the local screen data transparently based on the area information.

[0006] In one embodiment, before instructing the cloud application host to collect cloud application screen data and cloud application window information, the method further includes: The cloud application host connection information is forwarded to the zero terminal, so that the zero terminal can connect to the cloud application host according to the cloud application host connection information.

[0007] In one embodiment, before forwarding the cloud application host connection information to the zero terminal, the method further includes: Receive connection requests sent by zero terminals; A connection is established between the cloud desktop and the zero terminal based on the connection request sent by the zero terminal.

[0008] In one embodiment, before instructing the cloud application host to collect cloud application screen data and cloud application window information, the method further includes: Send a notification message to the cloud application host; wherein the notification message carries cloud desktop information; The notification information is used to instruct the cloud application host to open the corresponding application and to set the resolution of the cloud application host to be consistent with the resolution of the cloud desktop according to the cloud desktop information.

[0009] According to a second aspect of the present disclosure, an image data processing method is provided, applied to a zero-terminal device, the method comprising: Receive cloud application screen data and cloud application window information sent by the cloud application host; Receive local window information, local screen data, and cloud application shadow window information sent by the cloud desktop; wherein, the cloud application shadow window information is: the cloud desktop receives cloud application window information sent by the cloud application host, and creates a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtains the cloud application shadow window. The cloud application window information is used to replace the cloud application shadow window information, and the area information in the cloud application screen data that is not covered by the local screen data is calculated based on the cloud application window information and the local window information. The corresponding position of the local screen data is then transparently displayed based on the area information.

[0010] In one embodiment, before receiving the cloud application screen data and cloud application window information sent by the cloud application host, the method further includes: Receive cloud application host connection information forwarded by the cloud desktop, and connect to the cloud application host according to the cloud application host connection information.

[0011] In one embodiment, before receiving the cloud application host connection information forwarded by the cloud desktop, the method further includes: A connection request is sent to the cloud desktop so that the cloud desktop establishes a connection between the cloud desktop and the zero terminal based on the connection request.

[0012] According to a third aspect of the embodiments of this disclosure, a cloud desktop is provided, the cloud desktop comprising: a first instruction module, a creation module, a sending module, and a second instruction module; wherein... The first instruction module is used to instruct the cloud application host to collect cloud application screen data and cloud application window information, and to send the cloud application screen data and cloud application window information to the zero terminal and the cloud application window information to the cloud desktop. The creation module is used to receive cloud application window information sent by the cloud application host, create a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtain the cloud application shadow window information according to the cloud application shadow window. The sending module is used to collect local window information and local screen data, and package the local window information, local screen data and cloud application shadow window information and send them to the zero terminal. The second instruction module is used to instruct the zero terminal to replace the cloud application shadow window information with the cloud application window information, calculate the area information in the cloud application screen data that is not covered by the local screen data based on the cloud application window information and the local window information, and display the corresponding position of the local screen data transparently based on the area information.

[0013] In one embodiment, the cloud desktop further includes: a first receiving module and an establishing module; wherein, The first receiving module is used to receive connection requests sent by the zero terminal; The connection establishment module is used to establish a connection between the cloud desktop and the zero terminal based on the connection request sent by the zero terminal.

[0014] According to a fourth aspect of the present disclosure, a zero-terminal is provided, the zero-terminal comprising: a second receiving module, a third receiving module, and a processing module; wherein... The second receiving module is used to receive cloud application screen data and cloud application window information sent by the cloud application host; The third receiving module is used to receive local window information, local screen data and cloud application shadow window information sent by the cloud desktop; wherein, the cloud application shadow window information is: the cloud desktop receives cloud application window information sent by the cloud application host, and creates a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtains it according to the cloud application shadow window. The processing module is used to replace the cloud application shadow window information with the cloud application window information, calculate the area information in the cloud application screen data that is not covered by the local screen data based on the cloud application window information and the local window information, and display the corresponding position of the local screen data transparently based on the area information.

[0015] The image data processing method provided in this disclosure improves real-time performance and eliminates the location synchronization problem. This solves the problem in existing optimized solutions where the lower-level APP area displayed on the zero terminal is not synchronized with the actual location, resulting in the zero terminal only displaying a portion of the APP's desktop. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0017] Figure 1 This is a diagram illustrating the traditional way of using cloud applications based on cloud desktops.

[0018] Figure 2 This is a schematic diagram of the image data processing system in an existing optimized solution.

[0019] Figure 3 This is a flowchart illustrating an image data processing method provided in an embodiment of the present disclosure.

[0020] Figure 4 This is a flowchart illustrating an image data processing method provided in an embodiment of the present disclosure.

[0021] Figure 5This is an architecture diagram of a cloud desktop provided in an embodiment of the present disclosure.

[0022] Figure 6 This is an architecture diagram of a cloud desktop provided in an embodiment of the present disclosure.

[0023] Figure 7 This is a diagram illustrating a zero-terminal architecture provided in an embodiment of this disclosure. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0025] Figure 3 This is a schematic flowchart illustrating an image data processing method provided in an embodiment of this disclosure. Figure 3 As shown, this method is applied to cloud desktops, and the method includes: Step 301: Instruct the cloud application host to collect cloud application screen data and cloud application window information, and send the cloud application screen data and cloud application window information to the zero terminal, and send the cloud application window information to the cloud desktop. In this step, the cloud application host simultaneously collects complete cloud application screen data (which is actually the complete desktop of the cloud application host after the application is opened) and cloud application window information, and sends the cloud application window information to the cloud desktop. At the same time, the cloud application screen data and cloud application window information are sent to the zero terminal.

[0026] It should be noted that this embodiment does not limit the number of cloud applications; only the specific applications need to be clearly labeled.

[0027] In one embodiment, before instructing the cloud application host to collect cloud application screen data and cloud application window information, the method further includes: Send a notification message to the cloud application host; wherein the notification message carries cloud desktop information; The notification information is used to instruct the cloud application host to open the corresponding application and to set the resolution of the cloud application host to be consistent with the resolution of the cloud desktop according to the cloud desktop information.

[0028] In this embodiment, when the window where the cloud application host is located is launched, the cloud application window is set to be exactly the same as the cloud desktop window according to the cloud desktop resolution information carried in the notification information sent by the cloud desktop. In this way, even if there is a window asynchrony between the cloud application and the cloud desktop composite image, the exposed cloud application desktop area is consistent with the cloud desktop itself, which can also achieve a seamless connection effect. Thus, the user can not perceive this brief asynchrony at all.

[0029] Step 302: Receive cloud application window information sent by the cloud application host, create a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtain the cloud application shadow window information according to the cloud application shadow window. Step 303: Collect local window information and local screen data, and package the local window information, local screen data and cloud application shadow window information and send them to the zero terminal; In this step, the cloud desktop collects the position and layer data of the local window and the shadow window corresponding to the cloud application, without performing window segmentation and occlusion calculations, and packages this data together with the image data and sends it to the zero terminal.

[0030] Step 304: Instruct the zero terminal to replace the cloud application shadow window information with the cloud application window information, and calculate the area information in the cloud application screen data that is not covered by the local screen data based on the cloud application window information and the local window information, and display the corresponding position of the local screen data transparently based on the area information.

[0031] In this step, the zero terminal receives a set of window position and layer data from the cloud desktop, including information about the cloud application's shadow window; simultaneously, it receives the actual cloud application window position from the cloud application host. At this point, the zero terminal only needs to replace the shadow window position information in its own image, leaving the layers unchanged, and calculate the window overlay position to determine the area exposed after the cloud application is obscured by the local application. Finally, by setting the corresponding position on the zero terminal image to transparent, the underlying cloud application screen data is revealed, thus enabling the parallel and simultaneous display of two layers.

[0032] Optionally, before instructing the cloud application host to collect cloud application screen data and cloud application window information, the method further includes: The cloud application host connection information is forwarded to the zero terminal, so that the zero terminal can connect to the cloud application host according to the cloud application host connection information.

[0033] In this embodiment, when a user opens the cloud application icon, the cloud desktop will forward the relevant connection information of the cloud application host to the zero terminal. When the zero terminal receives the connection information of the cloud application host, it connects to the cloud application host to obtain the cloud application screen. In fact, it obtains the entire screen of the zero terminal host, not just the application window screen. At this time, the zero terminal has two complete screen information.

[0034] Optionally, before forwarding the cloud application host connection information to the zero terminal, the method further includes: Receive connection requests sent by zero terminals; A connection is established between the cloud desktop and the zero terminal based on the connection request sent by the zero terminal.

[0035] Figure 4 This is a schematic flowchart illustrating an image data processing method provided in an embodiment of this disclosure. Figure 4 As shown, this method is applied to zero terminals, and the method includes: Step 401: Receive cloud application screen data and cloud application window information sent by the cloud application host; Step 402: Receive local window information, local screen data and cloud application shadow window information sent by the cloud desktop; wherein, the cloud application shadow window information is: the cloud desktop receives the cloud application window information sent by the cloud application host, and creates a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtains the cloud application shadow window. Step 403: Replace the cloud application shadow window information with the cloud application window information, calculate the area information in the cloud application screen data that is not covered by the local screen data based on the cloud application window information and the local window information, and display the corresponding position of the local screen data transparently based on the area information.

[0036] Optionally, before receiving the cloud application host connection information sent by the cloud desktop, the method further includes: A connection request is sent to the cloud desktop so that the cloud desktop establishes a connection between the cloud desktop and the zero terminal based on the connection request.

[0037] The workflow of an image data processing system disclosed herein will be described in detail below.

[0038] Step 1. DESKTOP collects the position and layer data of the shadow window of the local window and the cloud application. Instead of performing window segmentation and occlusion calculations, this data is packaged together with the image data and sent to DISPLAY. Step 2. Originally, the APP would send application window position data to the DESKTOP and cloud application image data to the DISPLAY; now, the APP will also attach the window position data sent to the DESKTOP in the image data sent to the DISPLAY. There is no limit to the number of applications, as long as it is clearly labeled which application it is.

[0039] Step 3. The Display receives a set of window position and layer data sent by the Desktop, including information related to the cloud application's shadow window; simultaneously, it receives the actual cloud application window position sent by the app. At this point, simply replace the shadow window position information in the Desktop, leaving the layers unchanged, and then calculate the window overlay position in the Display to determine the area exposed after the cloud application is obscured by the local application (B). Then, set the transparency of the corresponding position in the Display image to reveal the underlying cloud application screen data. Both layers are sent and displayed in parallel.

[0040] Step 4. When the APP window starts, set the desktop scene to be exactly the same as the DESKTOP. Even if the cloud application and the cloud desktop composite image are out of sync, the exposed cloud application desktop area will be consistent with the cloud desktop itself. This can also achieve a seamless effect, so that users will not be able to perceive this brief asynchrony.

[0041] The image data processing method provided in this disclosure improves real-time performance and eliminates the location synchronization problem. This solves the problem in existing optimized solutions where the lower-level APP area displayed on the zero terminal is not synchronized with the actual location, resulting in the zero terminal only displaying a portion of the APP's desktop.

[0042] Figure 5 This is an architecture diagram of a cloud desktop provided as an embodiment of this disclosure. Figure 5As shown, the cloud desktop includes: a first instruction module 501, a creation module 502, a sending module 503, and a second instruction module 504; wherein, the first instruction module 501 is used to instruct the cloud application host to collect cloud application screen data and cloud application window information, and send the cloud application screen data and cloud application window information to the zero terminal, and send the cloud application window information to the cloud desktop; the creation module 502 is used to receive the cloud application window information sent by the cloud application host, and create a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and according to the cloud application window information... The application shadow window obtains the cloud application shadow window information; the sending module 503 is used to collect local window information and local screen data, and package the local window information, local screen data and cloud application shadow window information and send them to the zero terminal; the second instruction module 504 is used to instruct the zero terminal to replace the cloud application shadow window information with the cloud application window information, and calculate the area information in the cloud application screen data that is not covered by the local screen data according to the cloud application window information and the local window information, and make the corresponding position of the local screen data transparent according to the area information.

[0043] Figure 6 This is an architecture diagram of a cloud desktop provided as an embodiment of this disclosure. Figure 6 As shown, the cloud desktop includes: a first instruction module 601, a creation module 602, a sending module 603, a second instruction module 604, a first receiving module 605, and an establishment module 606; wherein, the first receiving module 605 is used to receive a connection request sent by a zero terminal; the establishment module 606 is used to establish a connection between the cloud desktop and the zero terminal according to the connection request sent by the zero terminal.

[0044] Figure 7 This is a diagram illustrating a zero-terminal architecture provided in an embodiment of this disclosure. Figure 7As shown, the zero terminal includes: a second receiving module 701, a third receiving module 702, and a processing module 703; wherein, the second receiving module 701 is used to receive cloud application screen data and cloud application window information sent by the cloud application host; the third receiving module 702 is used to receive local window information, local screen data, and cloud application shadow window information sent by the cloud desktop; wherein, the cloud application shadow window information is obtained by the cloud desktop receiving cloud application window information sent by the cloud application host, creating a cloud application shadow window corresponding to the cloud application window based on the cloud application window information, and obtaining information based on the cloud application shadow window; the processing module 703 is used to replace the cloud application shadow window information with the cloud application window information, calculate the area information in the cloud application screen data that is not covered by the local screen data based on the cloud application window information and the local window information, and transparently display the corresponding position of the local screen data based on the area information.

[0045] It should be noted that in this embodiment, DESKTOP refers to a cloud desktop, DISPLAY to a zero-terminal device, and APP to a cloud application host.

[0046] Although preferred embodiments of this disclosure have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this disclosure.

[0047] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. An image data processing method, characterized in that, Applied to cloud desktops, the method includes: The cloud application host is instructed to collect cloud application screen data and cloud application window information, and send the cloud application screen data and cloud application window information to the zero terminal and send the cloud application window information to the cloud desktop. Receive cloud application window information sent by the cloud application host, create a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtain the cloud application shadow window information according to the cloud application shadow window. Collect local window information and local screen data, and package the local window information, local screen data and cloud application shadow window information and send them to the zero terminal; The terminal is instructed to replace the cloud application shadow window information with the cloud application window information, calculate the area information in the cloud application screen data that is not covered by the local screen data based on the cloud application window information and the local window information, and display the corresponding position of the local screen data transparently based on the area information.

2. The method according to claim 1, characterized in that, Before instructing the cloud application host to collect cloud application screen data and cloud application window information, the method further includes: The cloud application host connection information is forwarded to the zero terminal, so that the zero terminal can connect to the cloud application host according to the cloud application host connection information.

3. The method according to claim 2, characterized in that, Before forwarding the cloud application host connection information to the zero terminal, the method further includes: Receive connection requests sent by zero terminals; A connection is established between the cloud desktop and the zero terminal based on the connection request sent by the zero terminal.

4. The method according to claim 1, characterized in that, Before instructing the cloud application host to collect cloud application screen data and cloud application window information, the method further includes: Send a notification message to the cloud application host; wherein the notification message carries cloud desktop information; The notification information is used to instruct the cloud application host to open the corresponding application and to set the resolution of the cloud application host to be consistent with the resolution of the cloud desktop according to the cloud desktop information.

5. An image data processing method, characterized in that, Applied to zero terminals, the method includes: Receive cloud application screen data and cloud application window information sent by the cloud application host; Receive local window information, local screen data, and cloud application shadow window information sent by the cloud desktop; wherein, the cloud application shadow window information is: the cloud desktop receives cloud application window information sent by the cloud application host, and creates a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtains the cloud application shadow window. The cloud application window information is used to replace the cloud application shadow window information, and the area information in the cloud application screen data that is not covered by the local screen data is calculated based on the cloud application window information and the local window information. The corresponding position of the local screen data is then transparently displayed based on the area information.

6. The method according to claim 5, characterized in that, Before receiving the cloud application screen data and cloud application window information sent by the cloud application host, the method further includes: Receive cloud application host connection information forwarded by the cloud desktop, and connect to the cloud application host according to the cloud application host connection information.

7. The method according to claim 5, characterized in that, Before receiving the cloud application host connection information forwarded by the cloud desktop, the method further includes: A connection request is sent to the cloud desktop so that the cloud desktop establishes a connection between the cloud desktop and the zero terminal based on the connection request.

8. A cloud desktop, characterized in that, The cloud desktop includes: a first instruction module, a creation module, a sending module, and a second instruction module; wherein... The first instruction module is used to instruct the cloud application host to collect cloud application screen data and cloud application window information, and to send the cloud application screen data and cloud application window information to the zero terminal and the cloud application window information to the cloud desktop. The creation module is used to receive cloud application window information sent by the cloud application host, create a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtain the cloud application shadow window information according to the cloud application shadow window. The sending module is used to collect local window information and local screen data, and package the local window information, local screen data and cloud application shadow window information and send them to the zero terminal. The second instruction module is used to instruct the zero terminal to replace the cloud application shadow window information with the cloud application window information, calculate the area information in the cloud application screen data that is not covered by the local screen data based on the cloud application window information and the local window information, and display the corresponding position of the local screen data transparently based on the area information.

9. The cloud desktop according to claim 8, characterized in that, The cloud desktop further includes: a first receiving module and an establishing module; wherein... The first receiving module is used to receive connection requests sent by the zero terminal; The connection establishment module is used to establish a connection between the cloud desktop and the zero terminal based on the connection request sent by the zero terminal.

10. A zero-terminal, characterized in that, The zero-terminal includes: a second receiving module, a third receiving module, and a processing module; wherein... The second receiving module is used to receive cloud application screen data and cloud application window information sent by the cloud application host; The third receiving module is used to receive local window information, local screen data and cloud application shadow window information sent by the cloud desktop; wherein, the cloud application shadow window information is: the cloud desktop receives cloud application window information sent by the cloud application host, and creates a cloud application shadow window corresponding to the cloud application window according to the cloud application window information, and obtains it according to the cloud application shadow window. The processing module is used to replace the cloud application shadow window information with the cloud application window information, calculate the area information in the cloud application screen data that is not covered by the local screen data based on the cloud application window information and the local window information, and display the corresponding position of the local screen data transparently based on the area information.

Images