Application virtualization method and device, cloud server, zero terminal and storage medium

Abstract

The invention provides an application virtualization method and device, a cloud server, a zero terminal and a storage medium, relates to the field of data processing, and can improve the problem of poor user experience of application virtualization in a scene of dragging an application window. According to the specific technical scheme, the method comprises the steps that a current frame desktop image is collected, and coordinate position data of a target application window at the previous moment and the next moment when the current frame desktop image is collected are recorded; obtaining position data of the target application window based on the recorded coordinate position data, and generating target application window information corresponding to the current frame desktop image based on the position data of the target application window; integrating the current frame desktop image and the target application window information into a minimum coding unit for coding to obtain an application code stream; and sending the application code stream to the zero terminal. Through synchronous data acquisition, data merging coding and transmission, the zero terminal can synchronize image rendering display and mask processing, ghost shadows can be eliminated fundamentally, and user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to an application virtualization method and device, a cloud server, a zero terminal, and a storage medium. Background Art

[0002] Currently, application virtualization technology (abbreviated as application virtualization) is a virtualization technology based on cloud computing. It transfers user applications from local computers to cloud servers for operation and management, which can improve the mobility, centralized management, security, and compatibility of applications, thereby improving user work efficiency and portability.

[0003] However, when the user drags the application window, there will be afterimages near the edge of the window. Existing technologies cannot completely eliminate the afterimage content near the edge of the window, and the visual experience is not significantly improved. It may also increase hardware costs and other costs. Therefore, further analysis and solutions are needed to improve the user experience. Summary of the invention

[0004] The embodiments of the present disclosure provide an application virtualization method and device, a cloud server, a zero terminal, and a storage medium, which can improve the problem of poor user experience of application virtualization in the scenario of dragging application windows. The technical solution is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an application virtualization method, which is applied to a cloud server, and the method comprises: acquiring a current frame desktop image, and recording coordinate position data of a target application window at a moment before and a moment after the current frame desktop image is acquired; obtaining position data of the target application window based on the recorded coordinate position data, and generating target application window information corresponding to the current frame desktop image based on the position data of the target application window; integrating the current frame desktop image and the target application window information into a minimum coding unit for encoding to obtain an application code stream; and sending the application code stream to a zero terminal.

[0006] Based on the above solution, through synchronous data collection, data merging encoding and transmission, the zero terminal can synchronize image rendering display and mask processing, ensuring that the desktop image data and application window information data are synchronously processed in all key links of collection, encoding, transmission, rendering display and mask processing. The purpose is to achieve synchronous movement of the mask window and the application window in the desktop image at the same speed and in the same direction, eliminate the position offset between the two, fundamentally eliminate the residual image, and then solve the problem that the application virtualization experience is not as good as running the local application in the scenario of dragging the application window, and improve the user experience.

[0007] In some embodiments, obtaining the position data of the target application window based on the recorded coordinate position data includes: obtaining the position data of the target application window by calculating the arithmetic mean of the coordinate position data at a moment before the current frame desktop image is collected and the coordinate position data at a moment after the current frame desktop image is collected.

[0008] In some embodiments, the method further includes: establishing a session connection with the zero terminal; and launching an application.

[0009] According to the second aspect of an embodiment of the present disclosure, there is provided an application virtualization method, which is applied to a zero terminal, and the method includes: receiving an application code stream from a cloud server; decoding the application code stream to obtain a current frame desktop image and target application window information corresponding to the current frame desktop image; rendering and displaying the current frame image, and at the same time, performing masking processing based on the target application window information.

[0010] Based on the above scheme, the desktop image data and application window information data that have been collected, merged and encoded and transmitted are processed synchronously through synchronous image rendering and mask processing, ensuring that the desktop image data and application window information data are processed synchronously in all key links of collection, encoding, transmission, rendering, display and mask processing. The purpose is to achieve synchronous movement of the mask window and the application window in the desktop image at the same speed and in the same direction, eliminate the position offset between the two, and fundamentally eliminate the residual image, thereby solving the problem that the application virtualization experience is not as good as running the local application in the scenario of dragging the application window, and improving the user experience.

[0011] In some embodiments, the current frame image is rendered and displayed, and at the same time, mask processing is performed based on the target application window information, including: encapsulating the current frame image and the target application window information into a display data unit; unpacking a display data unit to obtain the current frame image and the target application window information; rendering and display processing is performed on the current frame image, and at the same time, mask processing is performed based on the target application window information.

[0012] In some embodiments, the method further includes: establishing a session connection with a cloud server.

[0013] According to a third aspect of an embodiment of the present disclosure, an application virtualization device is provided, comprising a memory and a processor. The memory stores a program. When the program is executed in the processor, the processor is used to execute the method of the first aspect and any embodiment of the first aspect.

[0014] Based on the above devices, through synchronous data acquisition, data merging encoding and transmission, the zero terminal can synchronize image rendering display and mask processing, ensuring that the desktop image data and application window information data are synchronously processed in all key links of acquisition, encoding, transmission, rendering display and mask processing. The purpose is to achieve synchronous movement of the mask window and the application window in the desktop image at the same speed and in the same direction, eliminate the position offset between the two, fundamentally eliminate the residual image, and then solve the problem that the application virtualization experience is not as good as running the local application in the scenario of dragging the application window, and improve the user experience.

[0015] According to a fourth aspect of the embodiments of the present disclosure, an application virtualization device is provided, comprising a memory and a processor. The memory stores a program. When the program is executed in the processor, the processor is used to execute the method of the second aspect and any embodiment of the second aspect.

[0016] Based on the above device, the desktop image data and application window information data that have been collected, merged and encoded and transmitted through synchronous image rendering and mask processing are ensured to be processed synchronously in all key links of collection, encoding, transmission, rendering, display and mask processing. The purpose is to achieve synchronous movement of the mask window and the application window in the desktop image at the same speed and in the same direction, eliminate the position offset between the two, fundamentally eliminate the residual image, and then solve the problem that the application virtualization experience is not as good as running the local application in the scenario of dragging the application window, and improve the user experience.

[0017] According to a fifth aspect of an embodiment of the present disclosure, a cloud server is provided, comprising: a recording module, used to capture a current frame desktop image, and record coordinate position data of a target application window at a moment before and a moment after the current frame desktop image is captured; a generating module, used to obtain position data of the target application window based on the recorded coordinate position data, and generate target application window information corresponding to the current frame desktop image based on the position data of the target application window; an encoding module, used to integrate the current frame desktop image and the target application window information into a minimum encoding unit for encoding to obtain an application code stream; and a sending module, used to send the application code stream to a zero terminal.

[0018] In some embodiments, the recording module specifically includes: a calculation submodule, which is used to obtain the position data of the target application window by calculating the arithmetic mean of the coordinate position data of the previous moment before the current frame desktop image is collected and the coordinate position data of the next moment after the current frame desktop image is collected.

[0019] In some embodiments, the cloud server further includes: an establishment module for establishing a session connection with the zero terminal; and a startup module for starting an application.

[0020] According to the sixth aspect of an embodiment of the present disclosure, a zero terminal is provided, comprising: a receiving module for receiving an application code stream from a cloud server; a decoding module for decoding the application code stream to obtain a current frame desktop image and target application window information corresponding to the current frame desktop image; a processing module for rendering and displaying the current frame image, and at the same time, performing mask processing based on the target application window information.

[0021] In some embodiments, the processing module specifically includes: an encapsulation sub-module, used to encapsulate the current frame image and the target application window information into a display data unit; an unpacking sub-module, used to unpack a display data unit to obtain the current frame image and the target application window information; a processing sub-module, used to render and display the current frame image, and at the same time, perform mask processing based on the target application window information.

[0022] In some embodiments, the zero terminal further includes: an establishment module for establishing a session connection with a cloud server.

[0023] According to the seventh aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, characterized in that the computer program storage medium has program instructions, and when the program instructions are executed by a processor, the processor executes the method of the first aspect and any embodiment of the first aspect, or the method of the second aspect and any embodiment of the second aspect.

[0024] According to the eighth aspect of the embodiments of the present disclosure, a chip system is provided, characterized in that the chip system includes at least one processor, and when the program instructions are executed in the at least one processor, the at least one processor executes the method of the first aspect and any embodiment of the first aspect, or the method of the second aspect and any embodiment of the second aspect.

[0025] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0027] Figure 1 is a schematic diagram of an overall system framework 100 provided by an embodiment of the present disclosure;

[0028] Figure 2 is a schematic diagram of an application virtualization method 200 provided in an embodiment of the present disclosure;

[0029] Figure 3 is a schematic diagram of a packaging method 300 provided by an embodiment of the present disclosure;

[0030] Figure 4 is a flow chart of an application virtualization method 400 provided in an embodiment of the present disclosure;

[0031] Figure 5 is a structural diagram of an application virtualization device 500 provided in an embodiment of the present disclosure;

[0032] Figure 6 is a structural diagram of an application virtualization device 600 provided in an embodiment of the present disclosure;

[0033] Figure 7 is a structural diagram of a cloud server 700 provided in an embodiment of the present disclosure;

[0034] Figure 8 is another structural diagram of a cloud server 700 provided in an embodiment of the present disclosure;

[0035] Fig. 9 is another structural diagram of a cloud server 700 provided in an embodiment of the present disclosure;

[0036] Fig.10 is a structural diagram of a zero terminal 800 provided in an embodiment of the present disclosure;

[0037] Fig.11 is another structural diagram of a zero terminal 800 provided in an embodiment of the present disclosure;

[0038] Fig.12 It is another structural diagram of a zero terminal 800 provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0040] Currently, application virtualization technology (abbreviated as application virtualization) is a virtualization technology based on cloud computing, which transfers user applications from local computers to cloud servers for operation and management. Users can access the same application from any terminal device (such as a personal computer, laptop, tablet or smartphone) through a network connection, and enjoy a similar experience as opening the application on a local computer. Application virtualization technology can improve the mobility, centralized management, security, and compatibility of applications, thereby improving user efficiency and portability. It has been widely used in corporate office, education and training, medical care and other fields. Application virtualization technology can only guarantee a good user experience when sufficient hardware resources such as servers and networks are invested. Therefore, with less hardware cost investment, how to improve user experience and enhance product competitiveness by improving technology is a key research direction in the industry.

[0041] Application virtualization technology places the user's application on a central server, and then presents the application to the terminal user's device through image acquisition, image encoding, network transmission and rendering display. This method allows users to access the same desktop environment and applications on different terminal devices, thereby improving the user's portability and accessibility. However, while improving the user's portability and accessibility, it also introduces a new problem. For example, when the user drags the application window, there is an afterimage near the edge of the window. The afterimage content is part of the desktop background on the server side. The size of the afterimage area depends on factors such as the speed at which the user drags the window, server performance, and the degree of network transmission jitter. If the drag speed is slightly faster, the server performance is slightly worse, or the network transmission jitter is slightly larger, the afterimage area will become larger. In this case, the user experience of the application virtualization product is far less good than that of running a local application, which greatly weakens the competitiveness of the application virtualization product.

[0042] In the prior art, by setting the desktop background of the cloud desktop to a solid color picture, the user only sees a solid color afterimage near the edge of the window when dragging the application window. This method improves the visual effect to a certain extent, and is simple and easy to implement, so it is widely used, but people are still not satisfied with the effect. Further, people have proposed new technologies. For example, by optimizing the acquisition, encoding and transmission performance, it is expected to solve the problem of poor synchronization between the cloud desktop image and the application window information, thereby reducing the size of the afterimage area near the edge of the window, and achieving the purpose of improving the user experience. However, without increasing the hardware cost and providing hardware performance, the improvement in user experience is limited due to the small space for optimization of the acquisition, encoding and transmission performance. If the hardware cost investment is increased, the market competitiveness of the product will be weakened to a certain extent.

[0043] In summary, none of them can completely eliminate the residual image content near the edge of the window, the visual experience is not significantly improved, and it may also increase hardware costs and other costs. Therefore, further analysis and solutions are needed to improve the user experience.

[0044] In view of this, the present disclosure provides an application virtualization method and device, a cloud server, a zero terminal and a storage medium, which can improve the above problems and enhance the user experience.

[0045] First, the terms involved in the method provided in the embodiment of the present disclosure are explained.

[0046] H.264: The full name is Advanced Video Coding (AVC), or MPEG-4 Part 10. H.264 is a video compression standard jointly developed by the International Telecommunication Union (ITU) and the International Organization for Standardization (ISO). It is a widely used video coding format and is widely used in digital video broadcasting, Internet video transmission, Blu-ray discs, video conferencing and other fields.

[0047] Supplemental enhancement information (SEI): It is an additional information defined by the H.264 and H.265 standards, which is used to pass additional metadata during the video encoding and decoding process, such as timestamps, scene changes, image quality parameters, etc. SEI does not directly participate in video compression and decompression, but provides an extension mechanism for passing supplementary information to the decoder to improve video quality, add functionality, or provide other related data. SEI data is encapsulated in H.264 and H.265 bitstreams and transmitted through NALU.

[0048] Network abstraction layer unit (NALU): It is a basic unit in video coding standards such as H.264 and H.265, used to encapsulate and transmit video data. NALU contains the data after video compression encoding and can be transmitted, stored and decoded independently.

[0049] In order to better understand the method provided by the embodiment of the present disclosure, the system framework involved in the method is first introduced.

[0050] Figure 1 1 is a schematic diagram of a system overall framework 100 provided by an embodiment of the present disclosure. Figure 1 As shown in the figure, the overall framework of the system includes a cloud server subsystem and a terminal subsystem. The modules included in each of the above subsystems are introduced in detail below:

[0051] 1. Cloud server subsystem

[0052] The rendering module is responsible for cloud server image rendering and generating desktop images, which include various application windows and the contents displayed in the windows; the acquisition module is responsible for acquiring desktop image data and application window information data from the graphics card or the operating system of the cloud server subsystem; the encoding module is responsible for compressing and encoding the desktop image data and application window information data, and its input is the output of the acquisition module, and its output is the input of the transmission module; the transmission module is responsible for sending the encoded cloud desktop image data and application window information data, and receiving reverse control data; the reverse control module is responsible for receiving keyboard, mouse events and coordinate information sent by the terminal subsystem; the session module is responsible for session management such as session establishment, maintenance and disconnection between the cloud server subsystem and the terminal subsystem.

[0053] 2. Terminal subsystem

[0054] The session module is responsible for session management such as session establishment, maintenance and disconnection between the terminal subsystem and the cloud server subsystem; the transmission module is responsible for receiving desktop image data, application window information data and cursor data sent by the cloud server subsystem, and sending the reverse control data of the terminal subsystem to the cloud server subsystem; the decoding module is responsible for decoding the desktop image data and application window information data received by the transmission module, and its output is the input of the display module; the display module includes the one responsible for receiving the decoded desktop image data and rendering to generate the desktop image data, and performing window mask processing in combination with the application window information data, among which the rendering submodule is responsible for rendering to generate the desktop image, and the mask submodule is responsible for window mask processing; the reverse control module is responsible for collecting the keyboard, mouse events and coordinate information of the terminal subsystem.

[0055] It should be understood that the following method provided in this embodiment is applicable to Figure 1 The overall system framework shown in the figure, as well as the introduction of each module in the framework, the cloud server is suitable for Figure 1 The cloud server subsystem in the zero terminal is suitable for Figure 1 The terminal subsystem in.

[0056] The present disclosure provides an application virtualization method 200, such as Figure 2 As shown, the application virtualization method 200 includes the following steps:

[0057] S210, capturing a current frame desktop image, and recording coordinate position data of a target application window at a moment before and a moment after capturing the current frame desktop image.

[0058] For example, the cloud server's acquisition module can continuously acquire desktop image data from the graphics card or the cloud server's operating system to obtain the current frame desktop image. At the same time, the coordinate position data of the target application window before and after the acquisition of the current frame desktop image is recorded.

[0059] Optionally, before step S210, the method 200 may further include:

[0060] S201, the cloud server establishes a session connection with the zero terminal.

[0061] For example, a session connection may be established through the cloud server and the respective session modules of the zero terminal.

[0062] S202, the cloud server starts the application.

[0063] S220, obtaining position data of the target application window based on the recorded coordinate position data, and generating target application window information corresponding to the current frame desktop image based on the position data of the target application window.

[0064] Furthermore, obtaining the position data of the target application window based on the recorded coordinate position data can be achieved in the following manner:

[0065] The position data of the target application window is obtained by calculating the arithmetic mean of the coordinate position data at a moment before the current frame desktop image is collected and the coordinate position data at a moment after the current frame desktop image is collected.

[0066] For example, the position data of the target application window can be obtained by calculating the arithmetic mean of the coordinate position data at the moment before the current frame desktop image is collected and the coordinate position data at the moment after the current frame desktop image is collected. Then, the target application window information corresponding to the current frame desktop image is generated based on the position data of the target application window.

[0067] It should be understood that the window involved in this embodiment includes the window itself and all its derived sub-windows, and the target application window information includes the position, size, status and other information data of the window itself and its derived sub-windows.

[0068] S230, integrating the current frame desktop image and the target application window information into a minimum coding unit for encoding to obtain an application code stream.

[0069] For example, the current frame desktop image and the target application window information can be integrated into a minimum coding unit for encoding through the encoding module of the cloud server, and the encoding output application code stream. For the currently widely used H.264 encoding, considering compatibility, the target application window information can be encoded into the SEI field of H.264, and the same is true for H.265 and other encoding standards.

[0070] S240, the cloud server sends the application code stream to the null terminal. Correspondingly, the null terminal receives the application code stream from the cloud server.

[0071] For example, the application code stream can be sent to the null terminal through the transmission module of the cloud server. Correspondingly, the transmission module of the null terminal receives the application code stream from the cloud server.

[0072] Furthermore, the transmission module of the null terminal sends the application code stream to the parsing module of the null terminal.

[0073] S250, decoding the application code stream to obtain a current frame desktop image and target application window information corresponding to the current frame desktop image.

[0074] For example, the application code stream may be decoded by a decoding module of the zero terminal to obtain a current frame desktop image and target application window information corresponding to the current frame desktop image.

[0075] S260, rendering and displaying the current frame image, and performing masking based on the target application window information.

[0076] Specifically, the above steps can be implemented in the following ways:

[0077] Encapsulate the current frame image and the target application window information into a display data unit;

[0078] Unpack a display data unit to obtain the current frame image and target application window information;

[0079] The current frame image is rendered and displayed, and at the same time, mask processing is performed based on the target application window information.

[0080] For example, the current frame image and the target application window information may be encapsulated into a display data unit through a decoding module of the null terminal, and then the display data unit is sent to a display module of the null terminal.

[0081] It should be understood that when the current frame image and target application window information are continuously obtained and encoded to obtain the application code stream and transmitted, the zero terminal obtains multiple display data units through decoding and encapsulation. At this time, each display data unit is sent to the display module of the zero terminal in sequence.

[0082] Further, Figure 3 FIG. 3 is a schematic diagram of a packaging method 300 provided in an embodiment of the present disclosure. Figure 3 As shown, the encapsulation principle and encapsulation data structure of the current frame image and the target application window information applicable to the embodiment of the present application can be obtained.

[0083] Specifically, by Figure 3 It can be seen that the display data unit (i.e., the application code stream) contains desktop image data (i.e., the current frame image data) and application window data (i.e., the target application window information). Among them, the desktop image data includes image width, image height, image channel, and image data; the application window data includes window status, number of windows, and window list. Furthermore, the window list includes window position x, window position y, window width, and window height. It can be seen from the above scheme that the position data of the target application window (i.e., the window list) is obtained by calculating the arithmetic mean of the coordinate position data of the previous moment before the current frame desktop image is collected and the coordinate position data of the next moment after the current frame desktop image is collected, and the window status and number of windows are obtained through the operating system of the cloud server.

[0084] Then, after receiving the display data unit, the display module of the zero terminal can unpack the display data unit and obtain the current frame image and the target application window information. In addition, two aspects of work are completed in sequence and simultaneously: first, the current frame image is sent to the rendering submodule of the display module of the zero terminal; second, the target application window information is sent to the mask submodule of the display module of the zero terminal.

[0085] It should be understood that when the null terminal obtains multiple display data through decoding and encapsulation, the display module of the null terminal receives the multiple display data units in sequence and decapsulates the multiple display data units in sequence.

[0086] Finally, after receiving the current frame image, the rendering submodule renders and displays the current frame image; at the same time, after receiving the target application window information, the mask submodule masks the target application window information according to the window information.

[0087] As an example, Figure 4 It is a flowchart of an application virtualization method 400 provided in an embodiment of the present disclosure, which is applicable to the above-mentioned embodiment.

[0088] Based on the above scheme, the method provided by the embodiment of the present disclosure ensures that the desktop image data and the application window information data are synchronously processed in all key links of acquisition, encoding, transmission, rendering, display and mask processing through synchronous data acquisition, data merging encoding and transmission, synchronous image rendering and display and mask processing. The purpose is to achieve synchronous movement of the mask window and the application window in the desktop image at the same speed and in the same direction, eliminate the position offset between the two, fundamentally eliminate the residual image, and then solve the problem that the application virtualization experience is not as good as running the local application in the scenario of dragging the application window, and improve the user experience.

[0089] Based on the above Figure 2 The application virtualization method described in the corresponding embodiment is as follows: an embodiment of the device of the present disclosure, which can be used to execute the embodiment of the method of the present disclosure.

[0090] The present disclosure provides an application virtualization device 500, which is applied to a cloud server, such as Figure 5 The application virtualization device 500 includes: a memory 501 and a processor 502 .

[0091] The memory 501 is used to store programs.

[0092] When the program is executed in the processor 502 , the processor 502 is used to execute the above-mentioned application virtualization method.

[0093] Processor 502 is used to: capture the current frame desktop image, and record the coordinate position data of the target application window at the moment before and after the current frame desktop image is captured; obtain the position data of the target application window based on the recorded coordinate position data, and generate the target application window information corresponding to the current frame desktop image based on the position data of the target application window; integrate the current frame desktop image and the target application window information into a minimum coding unit for encoding to obtain an application code stream; and send the application code stream to the zero terminal.

[0094] Optionally, the processor 502 is specifically configured to obtain the position data of the target application window by calculating the arithmetic mean of the coordinate position data at a moment before the current frame desktop image is acquired and the coordinate position data at a moment after the current frame desktop image is acquired.

[0095] Optionally, the processor 502 is further configured to establish a session connection with the zero terminal and start an application.

[0096] The application virtualization device provided by the embodiment of the present disclosure enables the zero terminal to synchronize image rendering display and mask processing through synchronous data acquisition, data merging encoding and transmission, ensuring that the desktop image data and the application window information data are synchronously processed in all key links of acquisition, encoding, transmission, rendering display and mask processing. The purpose is to achieve synchronous movement of the mask window and the application window in the desktop image at the same speed and in the same direction, eliminate the position offset between the two, fundamentally eliminate the residual image, and then solve the problem that the application virtualization experience is not as good as running the local application in the scenario of dragging the application window, and improve the user experience.

[0097] The present disclosure provides an application virtualization device 600, which is applied to a zero terminal, such as Figure 6 The application virtualization device 600 includes: a memory 601 and a processor 602 .

[0098] The memory 601 is used to store programs.

[0099] When the program is executed in the processor 602 , the processor 602 is used to execute the above application virtualization method.

[0100] Processor 602 is used to: receive an application code stream from a cloud server; decode the application code stream to obtain a current frame desktop image and target application window information corresponding to the current frame desktop image; render and display the current frame image, and at the same time, perform masking based on the target application window information.

[0101] Optionally, processor 602 is specifically used to encapsulate the current frame image and target application window information into a display data unit; unpack a display data unit to obtain the current frame image and target application window information; render and display the current frame image, and at the same time, perform mask processing based on the target application window information.

[0102] Optionally, the processor 602 is further configured to establish a session connection with a cloud server.

[0103] The application virtualization device provided by the embodiment of the present disclosure ensures that the desktop image data and the application window information data are synchronously processed in all key links of acquisition, encoding, transmission, rendering, display and mask processing by synchronous image rendering and display and mask processing of the desktop image data and the application window information data after synchronous data acquisition, data merging encoding and transmission. The purpose is to achieve synchronous movement of the mask window and the application window in the desktop image at the same speed and in the same direction, eliminate the position offset between the two, fundamentally eliminate the residual image, and then solve the problem that the application virtualization experience is not as good as running the local application in the scenario of dragging the application window, and improve the user experience.

[0104] Based on the above Figure 2The application virtualization method described in the corresponding embodiment is as follows: an embodiment of the device of the present disclosure, which can be used to execute the embodiment of the method of the present disclosure.

[0105] Based on the above Figure 2 Corresponding to the application virtualization method described in the embodiment, the embodiment of the present disclosure also provides a cloud server 700, such as Figure 7 shown.

[0106] The cloud server 700 includes:

[0107] The recording module 710 is used to collect the desktop image of the current frame and record the coordinate position data of the target application window at the moment before and after the collection of the desktop image of the current frame;

[0108] A generating module 720, configured to obtain the position data of the target application window based on the recorded coordinate position data, and generate the target application window information corresponding to the current frame desktop image based on the position data of the target application window;

[0109] The encoding module 730 is used to integrate the current frame desktop image and the target application window information into a minimum encoding unit for encoding to obtain an application code stream;

[0110] The sending module 740 is used to send the application code stream to the zero terminal.

[0111] Alternatively, if Figure 8 As shown, the recording module 710 specifically includes:

[0112] The calculation submodule 7101 is used to obtain the position data of the target application window by calculating the arithmetic mean of the coordinate position data at a moment before the current frame desktop image is collected and the coordinate position data at a moment after the current frame desktop image is collected.

[0113] Alternatively, if Fig. 9 As shown, the cloud server 700 also includes:

[0114] Establishing module 750, used to establish a session connection with the null terminal;

[0115] The startup module 760 is used to start the application program.

[0116] Based on the above Figure 2 Corresponding to the application virtualization method described in the embodiment, the embodiment of the present disclosure also provides a zero terminal 800, such as Fig.10 shown.

[0117] Zero Terminal 800 includes:

[0118] The receiving module 810 is used to receive the application code stream from the cloud server;

[0119] The decoding module 820 is used to decode the application code stream to obtain the current frame desktop image and the target application window information corresponding to the current frame desktop image;

[0120] The processing module 830 is used to render and display the current frame image, and at the same time, perform mask processing based on the target application window information.

[0121] Alternatively, if Fig.11 As shown, the processing module 830 specifically includes:

[0122] The encapsulation submodule 8301 is used to encapsulate the current frame image and the target application window information into a display data unit;

[0123] The unpacking submodule 8302 is used to unpack a display data unit to obtain the current frame image and the target application window information;

[0124] The processing submodule 8303 is used to render and display the current frame image, and at the same time, perform mask processing based on the target application window information.

[0125] Alternatively, if Fig.12 As shown, the zero terminal 800 also includes:

[0126] Establishing module 840, used to establish a session connection with the cloud server.

[0127] Based on the above Figure 2 In accordance with the application virtualization method described in the embodiment, the embodiment of the present disclosure further provides a computer-readable storage medium, for example, a non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The storage medium stores computer instructions for executing the above Figure 2 The application virtualization method applied to the cloud server or to the zero terminal described in the corresponding embodiment will not be repeated here.

[0128] Based on the above Figure 2 Corresponding to the application virtualization method described in the embodiment, the embodiment of the present disclosure also provides a chip system, the chip system includes at least one processor, when the program instruction is executed in the at least one processor, the at least one processor executes the above Figure 2 The application virtualization method applied to the cloud server or to the zero terminal described in the corresponding embodiment will not be repeated here.

[0129] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0130] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An application virtualization method, applied to a cloud server, characterized in that: The method comprises: Capturing a current frame desktop image, and recording coordinate position data of a target application window at a moment before and a moment after the current frame desktop image is captured; Obtaining the position data of the target application window based on the recorded coordinate position data, and generating the target application window information corresponding to the current frame desktop image based on the position data of the target application window; Integrate the current frame desktop image and the target application window information into a minimum coding unit for encoding to obtain an application code stream; The application code stream is sent to the null terminal.

2. The method according to claim 1, characterized in that The obtaining the position data of the target application window based on the recorded coordinate position data includes: The position data of the target application window is obtained by calculating the arithmetic mean of the coordinate position data at a moment before the current frame desktop image is collected and the coordinate position data at a moment after the current frame desktop image is collected.

3. The method according to claim 1, characterized in that The method further comprises: Establishing a session connection with the null terminal; Start the application.

4. An application virtualization method, applied to a zero terminal, characterized in that: The method comprises: Receive application code stream from cloud server; Decoding the application code stream to obtain a current frame desktop image and target application window information corresponding to the current frame desktop image; The current frame image is rendered and displayed, and at the same time, masked based on the target application window information.

5. The method according to claim 4, characterized in that The rendering and display processing of the current frame image and the mask processing based on the target application window information include: Encapsulating the current frame image and the target application window information into a display data unit; Unpacking the one display data unit to obtain the current frame image and the target application window information; The rendering and display processing is performed on the current frame image, and at the same time, the mask processing is performed based on the target application window information.

6. The method according to claim 4, characterized in that The method further comprises: A session connection is established with the cloud server.

7. A cloud server, characterized in that: include: A recording module, used to collect the current frame desktop image, and record the coordinate position data of the target application window at a moment before and a moment after collecting the current frame desktop image; A generating module, configured to obtain the position data of the target application window based on the recorded coordinate position data, and generate the target application window information corresponding to the current frame desktop image based on the position data of the target application window; An encoding module, used for integrating the current frame desktop image and the target application window information into a minimum encoding unit for encoding to obtain an application code stream; The sending module is used to send the application code stream to the null terminal.

8. A zero terminal, characterized in that: include: A receiving module, used for receiving application code streams from a cloud server; A decoding module, used for decoding the application code stream to obtain a current frame desktop image and target application window information corresponding to the current frame desktop image; The processing module is used to render and display the current frame image, and at the same time, perform mask processing based on the target application window information.

9. An application virtualization device, characterized in that: including memory and processor; The memory has a stored program; When the program is executed in the processor, the processor is configured to execute the method according to any one of claims 1 to 3 or the method according to any one of claims 4 to 6.

10. A computer-readable storage medium, characterized in that: The computer program storage medium has program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 3 or the method according to any one of claims 4 to 6.

Images