Display method, device, server, terminal and storage medium of cloud application picture

Abstract

The application discloses a cloud application picture display method and device, a server, a terminal and a storage medium, and relates to the field of cloud applications. The method comprises the following steps: running a cloud application program, and acquiring a cloud application picture in a cloud application program running process; based on a picture segmentation strategy corresponding to the cloud application program, the cloud application picture is segmented into at least two sub-pictures; the at least two sub-pictures are parallel encoded to obtain at least two sub-encoding data blocks corresponding to the cloud application picture; the at least two sub-encoding data blocks are sent to a proxy server in parallel, so that the proxy server sends the at least two sub-encoding data blocks to a terminal, the at least two sub-encoding data blocks are decoded by the terminal, and the at least two sub-pictures obtained by decoding are spliced to obtain the cloud application picture. In the case of fully utilizing hardware resources, the parallel degree of the encoding and transmission process is improved by segmenting the cloud application picture, and the effect of reducing the display delay of the cloud application picture on the terminal side is achieved.

Description

Technical Field

[0001] This application relates to the field of cloud applications, and in particular to a method, apparatus, server, terminal and storage medium for displaying cloud application screens. Background Technology

[0002] Cloud applications are online application technologies based on cloud computing. In cloud application scenarios, the application runs on a cloud server, and the cloud server transmits the rendered application screen as a video stream to the terminal for display.

[0003] In related technologies, cloud servers are typically equipped with high-performance graphics processing units (GPUs) to improve the encoding speed of cloud application screens, thereby reducing the display latency of cloud application screens on the terminal side.

[0004] However, since hardware performance has its limits, simply improving hardware performance has limited effect on reducing the display latency of cloud applications, resulting in a large display latency for cloud applications that cannot achieve a performance close to that of local applications. Summary of the Invention

[0005] This application provides a method, apparatus, server, terminal, and storage medium for displaying cloud application screens, which can reduce the display latency of cloud application screens. The technical solution is as follows:

[0006] On one hand, embodiments of this application provide a method for displaying a cloud application screen, the method being used on a cloud application server, the method comprising:

[0007] Run the cloud application and obtain the cloud application screen during the running process;

[0008] Based on the screen segmentation strategy corresponding to the cloud application, the cloud application screen is divided into at least two sub-screens;

[0009] At least two of the sub-screens are encoded in parallel to obtain at least two sub-encoded data blocks corresponding to the cloud application screen;

[0010] At least two of the sub-encoded data blocks are sent in parallel to the proxy server, so that the proxy server sends the at least two of the sub-encoded data blocks to the terminal, the terminal decodes the at least two of the sub-screens, and splices the decoded at least two sub-screens to obtain the cloud application screen.

[0011] On the other hand, embodiments of this application provide a method for displaying a cloud application screen, the method being used on a terminal, the method comprising:

[0012] The system receives at least two sub-encoded data blocks sent by the proxy server. The sub-encoded data blocks are obtained by the cloud application server encoding at least two sub-screens in parallel. The at least two sub-screens are obtained by segmenting the cloud application screen based on the screen segmentation strategy corresponding to the cloud application.

[0013] Decode at least two of the sub-coded data blocks to obtain at least two sub-pictures;

[0014] At least two of the sub-screens are stitched together to obtain the cloud application screen, which is then displayed.

[0015] On the other hand, embodiments of this application provide a display device for a cloud application screen, the device comprising:

[0016] The screen acquisition module is used to run the cloud application and acquire the cloud application screen during the running process of the cloud application;

[0017] The screen segmentation module is used to segment the cloud application screen into at least two sub-screens based on the screen segmentation strategy corresponding to the cloud application.

[0018] An encoding module is used to encode at least two sub-screens in parallel to obtain at least two sub-encoded data blocks corresponding to the cloud application screen;

[0019] The data sending module is used to send at least two of the sub-encoded data blocks to the proxy server in parallel, so that the proxy server sends the at least two sub-encoded data blocks to the terminal, and the terminal decodes the at least two sub-screens obtained by decoding to obtain the cloud application screen.

[0020] On the other hand, embodiments of this application provide a display device for a cloud application screen, the device comprising:

[0021] The data receiving module is used to receive at least two sub-encoded data blocks sent by the proxy server. The sub-encoded data blocks are obtained by the cloud application server encoding at least two sub-screens in parallel. The at least two sub-screens are obtained by segmenting the cloud application screen based on the screen segmentation strategy corresponding to the cloud application.

[0022] A decoding module is used to decode at least two of the sub-encoded data blocks to obtain at least two sub-pictures;

[0023] The display module is used to stitch together at least two of the sub-screens to obtain the cloud application screen and display it.

[0024] On the other hand, embodiments of this application provide a server, which includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the cloud application screen display method on the cloud server side as described above.

[0025] On the other hand, embodiments of this application provide a terminal, which includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the method for displaying cloud application screens on the terminal side as described above.

[0026] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the cloud application screen display method as described above.

[0027] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the cloud application screen display method provided in the various optional implementations of the above aspects.

[0028] In this embodiment, during the operation of a cloud application, the cloud application server employs a screen segmentation strategy corresponding to the cloud application to divide the cloud application screen into several sub-screens. These sub-screens are then encoded in parallel, and the resulting sub-encoded data blocks are sent in parallel to the proxy server. The proxy server then sends these sub-encoded data blocks to the terminal, where the terminal decodes and reassembles them to reconstruct the cloud application screen. By utilizing the solution provided in this embodiment, and by segmenting the cloud application screen and performing parallel encoding and transmission while fully leveraging hardware resources, the computational load when encoding a single sub-screen can be reduced, the parallelism of the encoding and transmission process can be improved, and the encoding and transmission speed of a single frame of the cloud application screen can be accelerated, thereby reducing the display latency of the cloud application screen on the terminal side. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1This is a schematic diagram illustrating the principle of cloud application screen display process in an exemplary embodiment of this application;

[0031] Figure 2 A schematic diagram of an implementation environment provided by an exemplary embodiment of this application is shown;

[0032] Figure 3 A flowchart illustrating a method for displaying a cloud application screen according to an exemplary embodiment of this application is shown;

[0033] Figure 4 A flowchart illustrating a method for displaying a cloud application screen according to another exemplary embodiment of this application is shown;

[0034] Figure 5 This is a schematic diagram illustrating the segmentation method of the main region and edge region in an illustrative embodiment of this application;

[0035] Figure 6 This is a schematic diagram illustrating an exemplary embodiment of the cloud application screen segmentation encoding process.

[0036] Figure 7 This is a schematic diagram illustrating an exemplary embodiment of the sub-coded data block transmission process of this application;

[0037] Figure 8 This is a schematic diagram illustrating another exemplary embodiment of the cloud application screen segmentation encoding process.

[0038] Figure 9 A flowchart illustrating a method for displaying a cloud application screen according to another exemplary embodiment of this application is shown;

[0039] Figure 10 This is a schematic diagram illustrating an exemplary embodiment of the decoding and splicing process of sub-data encoded blocks in this application;

[0040] Figure 11 A block diagram of a display device for a cloud application screen provided in an exemplary embodiment of this application is shown;

[0041] Figure 12 A block diagram of a display device for a cloud application screen provided in another exemplary embodiment of this application is shown;

[0042] Figure 13 This invention provides a schematic diagram of the structure of a server according to an exemplary embodiment of the present application.

[0043] Figure 14 A structural block diagram of a terminal provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0045] In related technologies, when a cloud application server provides cloud application services to a terminal, after rendering the cloud application screen, the cloud application server encodes the screen frame by frame to obtain video frames, and transmits the video frames to the terminal in the form of a video stream. Correspondingly, the terminal decodes the video frames in the video stream frame by frame to reconstruct the cloud application screen.

[0046] However, when using the above method, when the resolution of the cloud application screen is high (such as 4K resolution), the encoding time required to encode the complete cloud application screen frame by frame is long. Furthermore, the terminal side needs to receive the complete video frame before it can decode, resulting in a significant delay between the cloud application server side completing the rendering of the cloud application screen and the terminal side displaying the cloud application screen.

[0047] In the technical solutions provided in the embodiments of this application, such as Figure 1 As shown, after rendering the cloud application screen 111, the cloud application server 11 does not directly encode the cloud application screen 111. Instead, based on a screen segmentation strategy, it first divides the cloud application screen 111 into several sub-screens 112, and then uses parallel encoding to encode these sub-screens 112 in parallel, resulting in several sub-encoded data blocks 113. Further, the cloud application server 11 uses parallel transmission to transmit these sub-encoded data blocks 113 to the proxy server 12, which then transmits them to the terminal 13. Upon receiving each sub-encoded data block 113, the terminal 13 can decode it to obtain the corresponding sub-screen 112. When all the sub-encoded data blocks 113 corresponding to the same cloud application screen 111 have been decoded, the terminal 13 can then stitch the sub-screens 112 together to reconstruct the cloud application screen 111.

[0048] By dividing the cloud application screen into several sub-screens and encoding these sub-screens in parallel, the computational load of the encoding process is reduced while the parallelism of the encoding process is increased. Compared to directly encoding a single frame of the cloud application screen, this helps reduce the encoding time on the cloud application server side. Furthermore, using parallel transmission to transmit sub-encoded data blocks to the terminal helps reduce transmission latency, and the terminal can decode each sub-encoded data block immediately upon receiving it, without waiting for the complete video frame to be received. This reduces the computational load of the decoding process while increasing its parallelism, further reducing decoding time on the terminal side. Therefore, the solution provided in this application can reduce the time spent in the encoding, transmission, and decoding stages, thereby shortening the latency between the cloud application server side completing the rendering of the cloud application screen and the terminal side displaying the cloud application screen, which is particularly significant for high-resolution cloud applications.

[0049] Figure 2 A schematic diagram of an implementation environment provided by an exemplary embodiment of this application is shown. This implementation environment may include: a cloud application server 210, a proxy server 220, and a terminal 230.

[0050] The cloud application server 210 has cloud applications installed and running. It can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. This application does not impose any restrictions on these services.

[0051] In one possible implementation, the cloud application server 210 provides a sandbox virtualization mechanism, running applications that originally ran on the terminal on the server, and providing powerful computing power to the applications with the help of a high-performance CPU and GPU.

[0052] In addition, the cloud application server 210 also has the function of encoding the cloud application screen. After the cloud application screen is drawn, the cloud application server 210 encodes the cloud application screen to obtain encoded data blocks. In this embodiment of the application, before encoding the cloud application screen, it is first necessary to divide the cloud application screen into several sub-screens, so as to encode each sub-screen in parallel to obtain several sub-encoded data blocks.

[0053] like Figure 2As shown, the cloud application server 210 is equipped with N CPUs, N GPUs, and memory. During the execution of the cloud application, the application process runs in an application sandbox, with the CPUs and GPUs providing computing power support. After the application process completes the rendering of the cloud application screen, it divides the screen into several sub-screens based on a screen segmentation strategy. These sub-screens are then encoded in parallel by an encoding process to obtain several sub-encoded data blocks. The screen segmentation strategy can be stored in a blockchain.

[0054] In some embodiments, the cloud application running on the cloud application server 210 is a cloud game. Cloud gaming, also known as game on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data processing capabilities to run high-quality games. In a cloud gaming scenario, the game does not reside on the player's gaming terminal but runs on a cloud gaming server. The cloud gaming server renders the game scene as a video and audio stream, which is then transmitted to the player's gaming terminal via the network. The player's gaming terminal does not need powerful graphics processing and data processing capabilities; it only needs basic streaming media playback capabilities and the ability to acquire player input commands and send them to the cloud gaming server.

[0055] The cloud application server 210 is connected to the proxy server 220 via a wired or wireless network. Optionally, a dedicated line connection is established between the cloud application server 210 and the proxy server 220 to ensure the stability of data transmission between the two and reduce transmission latency.

[0056] Proxy server 220 is deployed on an edge node close to (e.g., within 200km) the terminal 230 to provide high-bandwidth, low-latency, and jitter-resistant network transmission services. In this embodiment, proxy server 220 acts as a relay node between cloud application service 11 and terminal 13, receiving sub-encoded data blocks transmitted in parallel by cloud application server 210 and forwarding the sub-encoded data blocks to terminal 230.

[0057] In some embodiments, the proxy server 220 uses technologies such as multi-queue network cards and zero-copy to achieve low latency during data forwarding; and the proxy server 220 dynamically adjusts the bit rate and frame rate when transmitting data to the terminal 230 by detecting the network status of the terminal 230, thereby optimizing the user experience of cloud applications on the terminal side.

[0058] The proxy server 220 is connected to the terminal 230 via a wired or wireless network.

[0059] Terminal 230 is an electronic device running a cloud application client, which may be a smartphone, tablet, or personal computer, etc. In some embodiments, different cloud applications correspond to the same cloud application client, that is, users can use different cloud applications through the cloud application client. For example, users can use game applications, instant messaging applications, office applications, etc. through the cloud application client, but this embodiment does not limit this.

[0060] In this embodiment of the application, after the terminal 230 receives the sub-encoded data block sent by the proxy server 220, the cloud application client performs software decoding on the sub-encoded data block and / or calls hardware resources to perform hardware decoding on the sub-encoded data block to obtain the sub-screen corresponding to the sub-encoded data block, thereby splicing together several sub-screens belonging to the same cloud application screen to restore the cloud application screen before segmentation and display it.

[0061] During the operation of a cloud application, users can control the cloud application through touch operations on the physical display or external input devices (such as a mouse or keyboard).

[0062] In one possible scenario, when the cloud application is a game application, after the terminal 230 receives the control operation triggered by the user, it generates a control command and sends the control command to the cloud application server 210 through the proxy server 220. The cloud application server 210 then controls the virtual objects in the game application based on the control command.

[0063] It should be noted that, Figure 2 This explanation only takes the example of a cloud application server providing cloud application services to a single terminal. In practical applications, a cloud application server can provide cloud application services to multiple terminals simultaneously, which will not be elaborated upon in this embodiment.

[0064] Figure 3 A flowchart illustrating a method for displaying a cloud application screen according to an exemplary embodiment of this application is shown. This embodiment applies the method to... Figure 2 Taking the cloud application server 210 shown as an example, the method includes the following steps.

[0065] Step 301: Run the cloud application and obtain the cloud application screen during the running process.

[0066] The cloud application can be a game application, an instant messaging application, a social application, an office application, etc. This application embodiment does not limit the specific type of cloud application.

[0067] In one possible implementation, the cloud application server runs the cloud application in an application sandbox and obtains the cloud application screen drawn by the cloud application.

[0068] Step 302: Based on the screen segmentation strategy corresponding to the cloud application, the cloud application screen is divided into at least two sub-screens.

[0069] In this embodiment of the application, before encoding the cloud application screen, the cloud application server first needs to segment the cloud application screen to obtain at least two sub-screens. The different sub-screens are non-overlapping rectangular regions.

[0070] The screen segmentation strategy is used to indicate how the cloud application screen is segmented. Since different cloud applications have different screen characteristics, before performing screen segmentation, the cloud application server determines the screen segmentation strategy corresponding to the cloud application and then adopts the segmentation method indicated by the screen segmentation strategy to divide the cloud application screen into at least two sub-screens.

[0071] Optionally, each sub-screen can be the same size, or each sub-screen can be different sizes.

[0072] In one possible implementation, the same cloud application corresponds to a single screen segmentation strategy, that is, during the operation of the cloud application, the cloud application server uses a unified screen segmentation method to segment the cloud application screen. Alternatively, the same cloud application corresponds to at least two screen segmentation strategies, that is, under different operating scenarios of the cloud application, the cloud application server uses different screen segmentation methods to segment the cloud application screen.

[0073] Step 303: Perform parallel encoding on at least two sub-screens to obtain at least two sub-encoded data blocks corresponding to the cloud application screen.

[0074] For each segmented sub-screen, the cloud application server performs parallel encoding on each sub-screen to obtain the corresponding sub-encoded data block for each sub-screen. The cloud application server can encode the sub-screen into sub-encoded data blocks in H264 or H265 format; this embodiment does not limit the specific encoding format.

[0075] Since the segmented sub-screens are part of the cloud application screen, encoding a single sub-screen is faster than encoding the entire cloud application screen (because the encoding computation is smaller). Furthermore, since the encoding of several sub-screens is performed in parallel, the speed of encoding at least two sub-screens in parallel (depending on the slowest encoding speed in the parallel encoding process) is faster than the speed of encoding the entire cloud application screen.

[0076] In one possible implementation, the cloud application server performs parallel encoding of sub-frames using the GPU, or parallel encoding of sub-frames using both the GPU and CPU. Hardware encoding of sub-frames using the GPU is faster than software encoding of sub-frames using the CPU.

[0077] Step 304: Send at least two sub-encoded data blocks in parallel to the proxy server so that the proxy server can send the at least two sub-encoded data blocks to the terminal. The terminal can then decode the at least two sub-screens and stitch them together to obtain the cloud application screen.

[0078] In one possible implementation, at least two communication connections are established between the cloud application server and the proxy server. In order to reduce the transmission latency of sub-coded data blocks, the cloud application server adopts a parallel transmission method to transmit at least two sub-coded data blocks corresponding to the same cloud application screen to the proxy server in parallel, instead of transmitting the sub-coded data blocks to the proxy server in a serial manner.

[0079] In an illustrative example, the cloud application server divides the cloud application screen into four sub-screens and encodes the four sub-screens in parallel to obtain four sub-encoded data blocks, which are then sent to the proxy server in parallel through four communication connections.

[0080] Furthermore, the proxy server forwards the received sub-encoded data blocks to the terminal. Optionally, during the operation of the cloud application, the proxy server maintains multiple connections with the terminal, and the proxy server forwards the sub-encoded data blocks to the terminal in parallel through these multiple connections.

[0081] In some embodiments, the proxy server can use network acceleration technology to transmit sub-coded data blocks to the terminal in parallel through multiple network nodes, thereby improving the transmission speed of sub-coded data blocks in complex network environments and avoiding transmission delays caused by congestion of a single network node.

[0082] After receiving the sub-encoded data block, the terminal decodes the sub-encoded data block to restore the sub-screen before encoding, and splices together at least two sub-screens corresponding to the same cloud application screen to restore the cloud application screen before splitting.

[0083] Compared to the case where the terminal needs to receive the complete encoded data block before decoding when transmitting a complete cloud application screen (data reception and decoding are performed serially), in this embodiment, since different sub-encoded data blocks are independent of each other, the terminal can decode each sub-encoded data block it receives (i.e., data reception and decoding can be performed in parallel), without having to wait for all sub-encoded data blocks corresponding to the same cloud application screen to be received, thus improving the decoding speed on the terminal side.

[0084] In summary, in this embodiment, during the operation of a cloud application, the cloud application server employs a screen segmentation strategy corresponding to the cloud application to divide the cloud application screen into several sub-screens. These sub-screens are then encoded in parallel, and the resulting sub-encoded data blocks are sent in parallel to the proxy server. The proxy server then sends these sub-encoded data blocks to the terminal, where the terminal decodes and reassembles them to reconstruct the cloud application screen. By utilizing the solution provided in this embodiment, and by segmenting the cloud application screen and performing parallel encoding and transmission, the computational load when encoding a single sub-screen can be reduced, the parallelism of the encoding and transmission process can be improved, and the encoding and transmission speed of a single frame of the cloud application screen can be accelerated, thereby reducing the display latency of the cloud application screen on the terminal side.

[0085] During the operation of a cloud application, not all areas of the application screen are areas of user focus. For example, in a game application, the area of ​​user focus is the central part of the game screen. Therefore, when segmenting the cloud application screen, the screen can be divided into a main screen and an edge screen based on the user's level of focus on different areas. Different encoding and transmission strategies can then be used for the main screen and the edge screen to ensure the quality of the screen within the area of ​​user focus. An exemplary embodiment will be used below for illustration.

[0086] Figure 4 A flowchart illustrating a method for displaying a cloud application screen according to another exemplary embodiment of this application is shown. This embodiment applies the method to... Figure 1 Taking the cloud application server 210 shown as an example, the method includes the following steps.

[0087] Step 401: Run the cloud application and obtain the cloud application screen during the running process.

[0088] The implementation method of this step can be referred to step 301, and will not be repeated here in this embodiment.

[0089] Step 402: Obtain the main area information of the main area and the edge area information of the edge area in the cloud application screen from the screen segmentation strategy corresponding to the cloud application.

[0090] In one possible implementation, developers pre-determine the main and edge regions of the cloud application screen based on the user's level of attention to different areas of the screen, thereby generating a screen segmentation strategy that includes information on both the main and edge regions. Specifically, the user's attention to the main region is higher than their attention to the edge regions. Correspondingly, when a cloud application execution command is received, the cloud application server retrieves the screen segmentation strategy corresponding to that cloud application.

[0091] The cloud application screen includes at least one main area and at least one edge area. Optionally, the size of the main area is larger than the size of any edge area.

[0092] In one possible implementation, when dividing the cloud application screen into regions, the sizes of the main region and the edge regions are related to the magnitude of screen changes in the cloud application screen. Specifically, the greater the magnitude of screen changes in the cloud application screen, the larger the size of the main region and the smaller the size of the edge region; conversely, the smaller the magnitude of screen changes in the cloud application screen, the smaller the size of the main region and the larger the size of the edge region.

[0093] For example, in the case of game applications, the screen changes of role-playing games (RPGs) are relatively small, so the size of the edge area is larger (e.g., the main area accounts for 60% of the area and the edge area accounts for 40%), while the screen changes of racing and shooting games are larger, so the size of the edge area is smaller (e.g., the main area accounts for 90% of the area and the edge area accounts for 10%).

[0094] Because different applications have different interface layouts and operation methods, the areas that users focus on in different application screens also differ. Accordingly, different cloud applications have different ways of dividing the cloud application screen.

[0095] Indicative, such as Figure 5 As shown, the cloud application screen of cloud application A is divided into one main area and two edge areas, with the main area located in the upper left corner of the screen; the cloud application screen of cloud application B is divided into one main area and two edge areas, with the main area located in the lower right corner of the screen; the cloud application screen of cloud application C is divided into one main area and four edge areas, with the main area located in the center of the screen and the four edge areas surrounding the main area. It should be noted that... Figure 5 The zoning method shown is for illustrative purposes only and does not limit the specific zoning method in actual application.

[0096] Optionally, the main area information is used to indicate the location and size of the main area in the cloud application screen, and the edge area information is used to indicate the location and size of the edge area in the cloud application screen.

[0097] In some embodiments, the position of a region in the image can be represented by the coordinates of the top-left corner vertex of the region, and the size of the region can be represented by its width and height.

[0098] In an illustrative example, such as Figure 6As shown, the size of the cloud application screen 61 is 1920×1080, and the top left corner of the cloud application screen 61 is the origin of the coordinate system. The corresponding screen segmentation strategy includes: the main area information of the main area is "(0,0), 1600×980", the edge area information of the first edge area is "(1600,0), 320×980", and the edge area information of the second edge area is "(0,980), 1920×100".

[0099] Step 403: Based on the main area information and the edge area information, the cloud application screen is divided into at least one main screen and at least one edge screen, wherein the main screen is more important than the edge screen.

[0100] Based on the acquired main area and edge area information, the cloud application server divides the cloud application screen into a main screen and edge screens. Since the main screen is the screen that the user focuses on, it is of higher importance than the edge screens during encoding and transmission.

[0101] Indicative, such as Figure 6 As shown, the cloud application server divides the cloud application screen 61 into a main screen 62, a first edge screen 63, and a second edge screen 64.

[0102] Step 404: Encode the main screen using the target GPU that renders the cloud application screen to obtain the main screen encoded data block corresponding to the main screen.

[0103] Since the main screen is usually large, in order to avoid delays caused by data copying, in this embodiment of the application, the cloud application server encodes the main screen through the target GPU that renders the cloud application screen to obtain the main screen encoded data block.

[0104] Indicative, such as Figure 6 As shown, since the cloud application screen 61 is rendered by GPU1, after the cloud application server completes the screen segmentation, it uses GPU1 to encode the main screen 62 to obtain the main screen encoded data block 65.

[0105] Step 405: Copy the edge image to another GPU other than the target GPU, and encode the edge image through the other GPU to obtain the edge image encoded data block corresponding to the edge image.

[0106] Cloud application servers are equipped with multiple GPUs. Therefore, when the target GPU is used to encode the main screen, the cloud application server can use other GPUs to encode the edge screen in parallel, thereby improving the parallelism of the encoding process.

[0107] Before using other GPUs to encode the edge images in parallel, the cloud application server first needs to copy the edge images from the target GPU to other GPUs. Since the size of the edge images is usually small, the latency caused by data copying is manageable.

[0108] In one possible implementation, the cloud application server copies the edge image data from the target GPU to other GPUs via a bus (such as a PCIe bus, nvadia bus, or nvlink bus).

[0109] To avoid impacting application rendering when using other GPUs for encoding, in one possible implementation, the cloud application server obtains the working status of other GPUs besides the target GPU, including idle and busy states. When an idle GPU exists, the cloud application server copies the edge image to the idle GPU and encodes the edge image using the idle GPU to obtain the corresponding edge image encoded data block. Optionally, the idle GPU is a GPU with a utilization rate below a utilization threshold (e.g., 40%).

[0110] In one possible application scenario, when a large number of terminals are using cloud applications, there may not be any idle GPUs in the cloud application server. In order to ensure the normal encoding of edge images, in one possible implementation, when there are no idle GPUs, the cloud application server copies the edge images to memory and encodes the edge images through the CPU to obtain the edge image encoding data block corresponding to the edge images.

[0111] Although hardware encoding via GPU is faster than software encoding via CPU, the overall encoding speed is still faster than serial encoding of multiple screens due to parallel encoding of multiple screens.

[0112] Indicative, such as Figure 6 As shown, while the cloud application server encodes the main screen 62 through GPU1, it also copies the first edge screen 63 to GPU2 and the second edge screen 64 to GPU3, thereby performing parallel encoding through GPU2 and GPU3 to obtain the first edge screen encoding data block 66 and the second edge screen encoding data block 67.

[0113] Step 406: Send the main screen encoded data block and the edge screen encoded data block to the proxy server in parallel.

[0114] Furthermore, the cloud application server sends the encoded main screen data blocks and edge screen data blocks obtained from the encoding to the proxy server in parallel.

[0115] In one possible implementation, in order to ensure the accuracy of sub-screen stitching on the terminal side, before the cloud application server sends the sub-encoded data blocks to the proxy server, it needs to add a timestamp and screen position identifier to each sub-encoded data block. At least two sub-encoded data blocks corresponding to the same cloud application screen have the same timestamp, and the screen position identifier is used to indicate the position of the sub-screen in the cloud application screen.

[0116] Optionally, the screen location identifier can be represented by a sub-screen identifier corresponding to the sub-screen. Based on this sub-screen identifier and the screen segmentation strategy corresponding to the cloud application, the terminal can determine the position of the sub-screen obtained by decoding the sub-encoded data block within the cloud application screen. Of course, in other possible implementations, the screen location identifier can also be represented by main area information and edge area information; this embodiment does not limit this.

[0117] Indicative, such as Figure 6 As shown, the cloud application server sets the same timestamp for the main screen encoding data block 65, the first edge screen encoding data block 66, and the second edge screen encoding data block 67, and sets the screen position identifier "1" for the main screen encoding data block 65, the screen position identifier "2" for the first edge screen encoding data block 66, and the screen position identifier "3" for the second edge screen encoding data block 67.

[0118] In one possible implementation, when the proxy server forwards the main screen encoded data block and the edge encoded data block to the terminal, it prioritizes sending the main screen encoded data block to the terminal through the connection with the lowest latency and the lowest congestion, and uses other connections to send the edge screen encoded data block to the terminal in parallel, that is, it prioritizes ensuring the transmission quality of the main screen encoded data block.

[0119] Step 407: Receive control commands sent by the terminal through the proxy server. These control commands are used to control the cloud application.

[0120] When a user needs to control a cloud application, they can operate it on the cloud application screen through the terminal's touch screen or an external input device. Correspondingly, after receiving the control operation, the terminal generates a control command based on the operation location and operation type, and forwards the control command to the cloud application server through a proxy server, whereby the cloud application server responds to the control command.

[0121] For example, when a user clicks on a control in a cloud application screen via a touchscreen, the terminal sends a control command to the cloud application server containing the click coordinates and the operation type "click". After receiving the control command, the cloud application server simulates the user's click operation in the cloud application screen.

[0122] Step 408: Based on the control area indicated by the control command, update the operation frequency of different areas within the cloud application screen.

[0123] Typically, users operate within the area of ​​interest on the screen. Therefore, to improve the accuracy of the division between the main area and the edge area, in one possible implementation, the cloud application server performs statistics on the control area indicated by the control command, and then updates the operation frequency of different areas within the cloud application screen based on the statistical results.

[0124] Optionally, the cloud application server analyzes the control commands sent by different terminals, determines the common control area of ​​different terminals, and updates the operation frequency of the common control area to improve the applicability of the screen segmentation strategy after subsequent updates.

[0125] Step 409: Based on the operation frequency, update the main area information and edge area information in the screen segmentation strategy corresponding to the cloud application.

[0126] In one possible implementation, the cloud application server sorts the regions in the cloud application screen according to descending order of operation frequency, thereby determining the rectangular area formed by the first k regions as the main region and the regions outside the main region as edge regions, and then updating the main region information and edge region information. Subsequently, the screen is segmented based on the updated main region information and edge region information.

[0127] In an illustrative example, the primary region information before the update is "(0,0), 1600×980", and the primary region information after the update is "(0,0), 1600×1000".

[0128] In this embodiment, the cloud application server divides the cloud application screen into main screen and edge screens of different importance based on main area information and edge area information. During the encoding process, the target GPU for rendering the cloud application screen is used to encode the main screen, while other idle GPUs are used to encode the edge screen in parallel. This reduces the impact of data copying on the encoding speed during parallel encoding and further improves the encoding speed of the cloud application screen.

[0129] Furthermore, in this embodiment, the cloud application server updates the operation frequency in different areas of the cloud application screen based on the received control commands, determines the areas that the user focuses on when using the cloud application, and then updates the main area information and edge area information in the screen segmentation strategy, which helps to improve the accuracy of subsequent screen segmentation.

[0130] In the above embodiments, since the main screen is more important than the edge screen, and users' attention is usually focused on the main screen, the cloud application server needs to prioritize the encoding quality of the main screen when encoding, while the encoding quality requirements for the edge screen can be reduced.

[0131] In one possible implementation, when encoding the edge image, the cloud application server uses at least two encoding methods and other GPUs to encode the edge image to obtain at least two edge image encoded data blocks. Different encoding methods correspond to different compression ratios, and the higher the compression ratio, the smaller the encoded data block. Consequently, the quality of the sub-image restored after decoding is worse.

[0132] Optionally, the cloud application server may prioritize sending high-compression edge image encoded data blocks to the proxy server.

[0133] Indicative, such as Figure 7 As shown, for the segmented first edge image and second edge image, the cloud application server uses two encoding methods to perform redundant encoding on both, resulting in the first edge image encoding block 71 and the second edge image encoding block 72 corresponding to the first edge image (the compression ratio of the second edge image encoding block 72 is higher than that of the first edge image encoding block 71), and the third edge image encoding block 73 and the fourth edge image encoding block 74 corresponding to the second edge image (the compression ratio of the fourth edge image encoding block 74 is higher than that of the third edge image encoding block 73).

[0134] Furthermore, when sending sub-encoded data blocks to the proxy server, the cloud application server sends main screen encoded data blocks and at least two types of edge screen encoded data blocks to the proxy server in parallel. This allows the proxy server to determine the edge screen encoded data blocks to send to the terminal based on the network status between the proxy server and the terminal. The compression rate of the edge screen encoded data blocks sent to the terminal is negatively correlated with the network status; that is, the compression rate of edge screen encoded data blocks sent with good network status is lower than that of edge screen encoded data blocks sent with poor network status.

[0135] Indicative, such as Figure 7As shown, when the network status between the proxy server and the terminal is good, the proxy server sends the main screen encoded data block 70, the low-compression first edge screen encoded data block 71, and the third edge screen encoded data block 73 to the terminal in parallel, while discarding the high-compression second edge screen encoded data block 72 and the fourth edge screen encoded data block 74, thereby improving the display quality of the edge screen. However, when the network status between the proxy server and the terminal is poor, the proxy server sends the main screen encoded data block 70, the high-compression second edge screen encoded data block 72, and the fourth edge screen encoded data block 74 to the terminal in parallel, while discarding the low-compression first edge screen encoded data block 71 and the third edge screen encoded data block 73, prioritizing the normal display of the edge screen rather than its display quality.

[0136] It should be noted that when the proxy server sends sub-encoded data blocks to the terminal, it needs to inform the terminal of the encoding method used for the sub-encoded data blocks so that the terminal can use the corresponding decoding method to decode them.

[0137] In some embodiments, if the network condition deteriorates further, the proxy server can discard the edge image encoded data block corresponding to the edge image while ensuring the normal transmission of the main image encoded data block. When the terminal receives only the main image encoded data block, it can use a pre-trained neural network model (such as a generative adversarial network) to generate a complete cloud application screen based on the decoded main image, or enlarge the main image to match the size of the cloud application screen, or reuse the previously decoded edge image to prioritize the smooth display of the cloud application screen.

[0138] In this embodiment, the cloud application server generates edge image encoded data blocks with different compression ratios using different encoding methods for the same edge image, and redundantly sends edge image encoded data blocks with different compression ratios to the proxy server. The proxy server selects to send one type of edge image encoded data block according to the network status, ensuring the image quality of the edge image when the network status is good, and ensuring the normal display of the edge image when the network status is poor.

[0139] In the above embodiments, since the main screen and edge screens have different levels of importance, different encoding and transmission strategies are used for the main screen and edge screens. However, not all applications are suitable for this division of the main screen and edge screens. For example, for applications with large screen changes, dividing the cloud application screen into main screen and edge screens and discarding some encoded data blocks corresponding to the edge screens during transmission based on their importance will affect the overall visual experience of the application screen on the terminal side.

[0140] Therefore, in another possible implementation, the cloud application server obtains proportional segmentation information from the screen segmentation strategy, and based on the proportional segmentation information, divides the cloud application screen into at least two sub-screens of equal importance.

[0141] The proportional segmentation information indicates how the cloud application screen is divided in terms of width and height. For example, when the proportional segmentation information is 2×2, it means that the cloud application screen needs to be divided into 4 sub-screens, each with a width of 1 / 2 the width of the cloud application screen and a height of 1 / 2 the height of the cloud application screen; when the proportional segmentation information is 3×4, it means that the cloud application screen needs to be divided into 12 sub-screens, each with a width of 1 / 3 the width of the cloud application screen and a height of 1 / 4 the height of the cloud application screen.

[0142] Optionally, for the segmented sub-pictures, the cloud application server can encode one of the sub-pictures using the target GPU and copy the remaining sub-pictures to other idle GPUs, thus enabling parallel encoding of the remaining sub-pictures using the idle GPUs. If no idle CPUs are available, the cloud application server can copy the remaining sub-pictures to memory, allowing software decoding of the remaining sub-pictures using the CPU.

[0143] In an illustrative example, such as Figure 8 As shown, after the cloud application server renders a 1920×1080 cloud application screen using GPU1, it divides the cloud application screen 81 into four 960×540 sub-screens based on the proportional segmentation information (2×2) in the screen segmentation strategy. These are the first sub-screen 82, the second sub-screen 83, the third sub-screen 84, and the fourth sub-screen 85. Further, the cloud application server encodes the first sub-screen 82 using GPU1, and copies the second sub-screen 83, the third sub-screen 84, and the fourth sub-screen 85 to the idle GPUs 2, 3, and 4, respectively. This allows the four GPUs to encode the four sub-screens in parallel, resulting in the first sub-encoded data block 86, the second sub-encoded data block 87, the third sub-encoded data block 88, and the fourth sub-encoded data block 89.

[0144] The above embodiments describe the encoding, transmission, and decoding processes of cloud application screens under two screen segmentation strategies: main / edge screen segmentation and proportional segmentation. Regarding the determination of the screen segmentation strategy, in one possible implementation, the cloud application server determines the screen segmentation strategy based on the application identifier of the cloud application.

[0145] Optionally, the cloud application server stores the correspondence between application identifiers and screen segmentation strategies. Upon receiving a running instruction from a cloud application, the cloud application server obtains the application identifier of the cloud application and then retrieves the corresponding screen segmentation strategy based on that application identifier. This screen segmentation strategy includes a first screen segmentation strategy that instructs the division of the main screen and edge screens, and a second screen segmentation strategy that instructs proportional division.

[0146] In an illustrative example, when running an RPG game, the cloud application server uses a first screen segmentation strategy to divide the game screen into a main screen and edge screens; when running a racing game, the cloud application server uses a second screen segmentation strategy to divide the game screen proportionally.

[0147] In practical applications, it has been found that the same cloud application may exhibit different visual characteristics under different operating scenarios. For example, for game applications, the visual changes are relatively small in non-team battle scenarios, but significantly larger in team battle scenarios. Using a single screen segmentation strategy during cloud application operation may affect the terminal-side display. For instance, if main / edge screen segmentation is performed in both non-team battle and team battle scenarios, discarding sub-encoded data blocks corresponding to edge screens under poor network conditions will affect the normal display of team battle screens.

[0148] Therefore, in another possible implementation, the cloud application server dynamically determines the screen segmentation strategy based on the application identifier of the cloud application and the current running scenario.

[0149] In some embodiments, when two screen segmentation strategies are determined based on the application identifier of the cloud application, the cloud application server monitors the current running scenario of the cloud application in real time, and dynamically determines the screen segmentation strategy to be adopted based on the current running scenario. The adopted screen segmentation strategy is related to the magnitude of screen changes under the current running scenario.

[0150] Optionally, when the cloud application corresponds to the first screen segmentation strategy and the second screen segmentation strategy, and the current running scenario is the first running scenario, the cloud application server determines the screen segmentation strategy as the first screen segmentation strategy.

[0151] When a cloud application corresponds to a first screen segmentation strategy and a second screen segmentation strategy, and the current running scenario is the second running scenario, the cloud application server determines the screen segmentation strategy to be the second screen segmentation strategy, wherein the screen change range in the second running scenario is greater than the screen change range in the first running scenario.

[0152] Optionally, the cloud application server can determine the current operating scenario by comparing the changes in cloud application screens within a preset time period. For example, if the changes in cloud application screens within the preset time period are less than a threshold, it is determined to be in a first operating scenario; if the changes in cloud application screens within the preset time period are greater than the threshold, it is determined to be in a second operating scenario. Of course, in other possible implementations, the cloud application server can also obtain scenario information from the application process of the cloud application to determine the current operating scenario, and this application embodiment does not limit this.

[0153] In an illustrative example, during the running of an RPG game, in non-team battle scenarios, the cloud application server divides the game screen into a main screen and peripheral screens, while in team battle scenarios (where the screen changes significantly), the cloud application server divides the game screen proportionally.

[0154] It should be noted that when switching screen segmentation strategies, the cloud application server needs to ensure that the screens in the same group of pictures (GOP) use the same screen segmentation strategy to ensure the normal encoding and decoding of cloud application screens.

[0155] In this embodiment, the cloud application server dynamically adjusts the screen segmentation strategy of the cloud application based on the cloud application and the current operating scenario, so that the screen segmentation method adopted is consistent with the actual operating scenario. This avoids the problem that a single screen segmentation strategy cannot be applied to some operating scenarios, resulting in poor display quality of the cloud application screen in some operating scenarios, and helps to improve the display quality of the cloud application screen in different operating scenarios.

[0156] Figure 9 A flowchart illustrating a method for displaying a cloud application screen according to an exemplary embodiment of this application is shown. This embodiment applies the method to... Figure 2 Taking terminal 230 as an example, the method includes the following steps.

[0157] Step 901: Receive at least two sub-encoded data blocks sent by the proxy server. The sub-encoded data blocks are obtained by the cloud application server encoding at least two sub-screens in parallel. The at least two sub-screens are obtained by segmenting the cloud application screen based on the screen segmentation strategy corresponding to the cloud application.

[0158] In one possible implementation, the terminal receives at least two sub-coded data blocks sent by the proxy server in parallel through at least two connections with the proxy server, wherein the at least two sub-coded data blocks may be received simultaneously or sequentially.

[0159] Step 902: Decode at least two sub-coded data blocks to obtain at least two sub-pictures.

[0160] Since the at least two sub-coded data blocks corresponding to the same cloud application screen are independent of each other, the terminal does not need to wait for all the sub-coded data blocks corresponding to the same cloud application screen to be received before decoding. Instead, it decodes each sub-coded data block as soon as it is received to obtain the sub-screen corresponding to the sub-coded data block, thereby improving the parallelism between the data block reception and decoding process and improving the decoding efficiency on the terminal side.

[0161] The terminal performs hardware or software decoding on the sub-encoded data blocks.

[0162] In one possible implementation, when the cloud application server divides the cloud application screen into a main screen and edge screens based on the importance of the screen, the main screen is typically larger than the edge screen, therefore the main screen encoded data block is larger than the edge screen encoded data block. To improve the decoding speed of the main screen encoded data block, the terminal decodes the main screen encoded data block using the GPU and the edge screen encoded data block using the CPU to obtain the main screen and edge screen. Further, the terminal stitches the decoded main screen and edge screen together to obtain the cloud application screen. Notably, hardware decoding using the GPU is faster than software decoding using the CPU.

[0163] Optionally, when using the CPU to decode the edge image encoded data block, the terminal first copies the edge image data block to memory, and then uses the CPU to perform software decoding on the edge image data block to obtain the corresponding edge image.

[0164] Indicative, such as Figure 10 As shown, after receiving the main screen encoded data block 1001, the first edge screen encoded data block 1002, and the second edge screen encoded data block 1003, the terminal performs hardware decoding of the main screen encoded data block 1001 through the GPU, and software decoding of the first edge screen encoded data block 1002 and the second edge screen encoded data block 1003 through CPU1 and CPU2 respectively, to obtain the main screen 1004, the first edge screen 1005, and the second edge screen 1006. Further, the terminal stitches together the main screen 1004, the first edge screen 1005, and the second edge screen 1006 to reconstruct a 1920×1080 cloud application screen 1007.

[0165] Optionally, when the cloud application server encodes the edge image using at least two encoding methods and redundantly sends at least two sub-encoded data blocks with different compression rates to the proxy server, and the proxy server selects to send one of the sub-encoded data blocks to the terminal based on the network status between the proxy server and the terminal, the terminal needs to use the decoding method corresponding to the encoding method to decode the sub-encoded data block to ensure the normal execution of the decoding process.

[0166] Furthermore, when the network condition between the proxy server and the terminal deteriorates further, the proxy server may discard the sub-coded data blocks corresponding to the edge screen and only transmit the sub-coded data blocks corresponding to the main screen. Optionally, if the sub-coded data blocks corresponding to the edge screen are not received within a preset time period, and only the sub-coded data blocks corresponding to the main screen are received, the terminal will generate a complete cloud application screen based on the decoded main screen using a pre-trained neural network model (such as a generative adversarial network), or enlarge the main screen to make the size of the main screen consistent with the size of the cloud application screen, or reuse the previously decoded edge screen, thereby ensuring the smooth display of the cloud application screen.

[0167] In other possible implementations, when the cloud application server divides the cloud application screen into several sub-screens of equal importance, the terminal uses the GPU to perform hardware decoding of the sub-encoded data block corresponding to one of the sub-screens, and uses the CPU to perform software decoding of the sub-encoded data blocks corresponding to the remaining sub-screens, thereby improving the parallelism of the decoding process on the terminal side.

[0168] Step 903: At least two sub-screens are spliced ​​together to obtain the cloud application screen and display it.

[0169] When all sub-encoded data blocks corresponding to the same cloud application screen have been decoded, the terminal splices the decoded sub-screens together to restore the cloud application screen before splitting and displays it.

[0170] In one possible implementation, a timestamp and a screen location identifier are added to the sub-encoded data block. The terminal then determines the sub-screen corresponding to the same cloud application screen based on the timestamp and the position of each sub-screen in the cloud application screen based on the screen location identifier, thereby improving the accuracy of the cloud application screen splicing process.

[0171] In summary, in this embodiment, during the operation of a cloud application, the cloud application server employs a screen segmentation strategy corresponding to the cloud application to divide the cloud application screen into several sub-screens. These sub-screens are then encoded in parallel, and the resulting sub-encoded data blocks are sent in parallel to the proxy server. The proxy server then sends these sub-encoded data blocks to the terminal, where the terminal decodes and reassembles them to reconstruct the cloud application screen. By utilizing the solution provided in this embodiment, and by segmenting the cloud application screen and performing parallel encoding and transmission, the computational load when encoding a single sub-screen can be reduced, the parallelism of the encoding and transmission process can be improved, and the encoding and transmission speed of a single frame of the cloud application screen can be accelerated, thereby reducing the display latency of the cloud application screen on the terminal side.

[0172] Figure 11 A block diagram of a display device for a cloud application screen provided in an exemplary embodiment of this application is shown. The device includes:

[0173] The screen acquisition module 1101 is used to run a cloud application and acquire the cloud application screen during the running process of the cloud application;

[0174] The screen segmentation module 1102 is used to segment the cloud application screen into at least two sub-screens based on the screen segmentation strategy corresponding to the cloud application.

[0175] Encoding module 1103 is used to encode at least two sub-screens in parallel to obtain at least two sub-encoded data blocks corresponding to the cloud application screen;

[0176] The data sending module 1104 is used to send at least two of the sub-encoded data blocks to the proxy server in parallel, so that the proxy server sends at least two of the sub-encoded data blocks to the terminal, and the terminal decodes the at least two sub-screens obtained by decoding to obtain the cloud application screen.

[0177] Optionally, the screen segmentation module 1102 includes:

[0178] The first information acquisition unit is used to acquire the main region information of the main region and the edge region information of the edge region in the cloud application screen from the screen segmentation strategy.

[0179] The first segmentation unit is used to segment the cloud application screen into at least one main screen and at least one edge screen based on the main area information and the edge area information, wherein the importance of the main screen is higher than that of the edge screen.

[0180] Optionally, the encoding module 1103 includes:

[0181] The first encoding unit is used to encode the main screen through the target GPU that renders the cloud application screen to obtain the main screen encoding data block corresponding to the main screen;

[0182] The second encoding unit is used to copy the edge image to another GPU other than the target GPU, and encode the edge image through the other GPU to obtain the edge image encoded data block corresponding to the edge image.

[0183] Optionally, the second encoding unit is used for:

[0184] Obtain the working status of other GPUs besides the target GPU, including idle status and busy status;

[0185] In response to the existence of an idle GPU, the edge image is copied to the idle GPU, and the edge image is encoded by the idle GPU to obtain the edge image encoded data block corresponding to the edge image;

[0186] The encoding module 1103 further includes:

[0187] The third encoding unit is used to copy the edge image to memory in response to the absence of the idle GPU, and encode the edge image by the CPU to obtain the edge image encoded data block corresponding to the edge image.

[0188] Optionally, the second encoding unit is further configured to:

[0189] At least two encoding methods are used to encode the edge image using other GPUs to obtain at least two types of edge image encoded data blocks, wherein different encoding methods correspond to different compression rates;

[0190] The data transmission module 1104 is used for:

[0191] The proxy server sends the main screen encoded data block and at least two types of edge screen encoded data blocks in parallel, so that the proxy server determines the edge screen encoded data block to send to the terminal based on the network status between the proxy server and the terminal, wherein the compression rate of the edge screen encoded data block sent to the terminal is negatively correlated with the network status.

[0192] Optionally, the device further includes:

[0193] The instruction receiving module is used to receive control instructions sent by the terminal through the proxy server, the control instructions being used to control the cloud application;

[0194] The frequency update module is used to update the operation frequency of different areas within the cloud application screen based on the control area indicated by the control command.

[0195] The strategy update module is used to update the main region information and the edge region information in the screen segmentation strategy corresponding to the cloud application based on the operation frequency.

[0196] Optionally, the screen segmentation module 1102 further includes:

[0197] The second information acquisition unit is used to acquire proportional segmentation information from the screen segmentation strategy;

[0198] The second segmentation unit is used to divide the cloud application screen into at least two sub-screens proportionally based on the proportional segmentation information, wherein the different sub-screens have the same degree of importance.

[0199] Optionally, the device further includes:

[0200] The first strategy determination module is used to determine the screen segmentation strategy based on the application identifier of the cloud application.

[0201] or,

[0202] The second strategy determination module is used to determine the screen segmentation strategy based on the application identifier of the cloud application and the current running scenario.

[0203] The screen segmentation strategy includes a first screen segmentation strategy and a second screen segmentation strategy. The first screen segmentation strategy indicates that the main screen and the edge screen are segmented, and the main screen is of higher importance than the edge screen. The second screen segmentation strategy indicates that the screen is segmented proportionally.

[0204] Optionally, the second strategy determination module includes:

[0205] The first strategy determination unit is configured to determine the screen segmentation strategy as the first screen segmentation strategy in response to the cloud application corresponding to the first screen segmentation strategy and the second screen segmentation strategy, and the current running scenario being the first running scenario.

[0206] The second strategy determination unit is configured to determine the screen segmentation strategy as the second screen segmentation strategy in response to the cloud application corresponding to the first screen segmentation strategy and the second screen segmentation strategy, and the current running scenario being the second running scenario, wherein the screen change range in the second running scenario is greater than the screen change range in the first running scenario.

[0207] Optionally, the device further includes:

[0208] An add module is used to add timestamps and screen position identifiers to each of the sub-encoded data blocks, wherein at least two of the sub-encoded data blocks corresponding to the same cloud application screen have the same timestamp, and the screen position identifier is used to indicate the position of the sub-screen in the cloud application screen.

[0209] In summary, in this embodiment, during the operation of a cloud application, the cloud application server employs a screen segmentation strategy corresponding to the cloud application to divide the cloud application screen into several sub-screens. These sub-screens are then encoded in parallel, and the resulting sub-encoded data blocks are sent in parallel to the proxy server. The proxy server then sends these sub-encoded data blocks to the terminal, where the terminal decodes and reassembles them to reconstruct the cloud application screen. By utilizing the solution provided in this embodiment, and by segmenting the cloud application screen and performing parallel encoding and transmission, the computational load when encoding a single sub-screen can be reduced, the parallelism of the encoding and transmission process can be improved, and the encoding and transmission speed of a single frame of the cloud application screen can be accelerated, thereby reducing the display latency of the cloud application screen on the terminal side.

[0210] Figure 12 A block diagram of a display device for a cloud application screen provided in another exemplary embodiment of this application is shown. The device includes:

[0211] The data receiving module 1201 is used to receive at least two sub-encoded data blocks sent by the proxy server. The sub-encoded data blocks are obtained by the cloud application server encoding at least two sub-screens in parallel. The at least two sub-screens are obtained by dividing the cloud application screen based on the screen segmentation strategy corresponding to the cloud application.

[0212] Decoding module 1202 is used to decode at least two of the sub-coded data blocks to obtain at least two sub-pictures;

[0213] Display module 1203 is used to stitch together at least two of the sub-screens to obtain the cloud application screen and display it.

[0214] Optionally, the decoding module 1202 is used for:

[0215] The main screen encoded data block is decoded by the GPU, and the edge screen encoded data block is decoded by the CPU to obtain the main screen and the edge screen. The main screen is of higher importance than the edge screen.

[0216] The display module 1203 is used for:

[0217] The main screen and the edge screen are stitched together to obtain the cloud application screen, which is then displayed.

[0218] Optionally, the data receiving module 1201 is used for:

[0219] At least two sub-encoded data blocks sent by the proxy server are received in parallel through at least two connections with the proxy server.

[0220] In summary, in this embodiment, during the operation of a cloud application, the cloud application server employs a screen segmentation strategy corresponding to the cloud application to divide the cloud application screen into several sub-screens. These sub-screens are then encoded in parallel, and the resulting sub-encoded data blocks are sent in parallel to the proxy server. The proxy server then sends these sub-encoded data blocks to the terminal, where the terminal decodes and reassembles them to reconstruct the cloud application screen. By utilizing the solution provided in this embodiment, and by segmenting the cloud application screen and performing parallel encoding and transmission, the computational load when encoding a single sub-screen can be reduced, the parallelism of the encoding and transmission process can be improved, and the encoding and transmission speed of a single frame of the cloud application screen can be accelerated, thereby reducing the display latency of the cloud application screen on the terminal side.

[0221] It should be noted that the apparatus provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their implementation process can be found in the method embodiments, which will not be repeated here.

[0222] Please refer to Figure 13 This illustration shows a schematic diagram of a server provided in an exemplary embodiment of this application. Specifically, the server 1300 includes a Central Processing Unit (CPU) 1301, a system memory 1304 including a random access memory 1302 and a read-only memory 1303, and a system bus 1305 connecting the system memory 1304 and the CPU 1301. The server 1300 may also include a basic input / output system (I / O system) 1306 to facilitate information transfer between various devices within the computer, and a mass storage device 1307 for storing the operating system 1313, application programs 1314, and other program modules 1315.

[0223] In some embodiments, the basic input / output system 1306 may include a display 1308 for displaying information and an input device 1309 for user input of information, such as a mouse or keyboard. Both the display 1308 and the input device 1309 are connected to the central processing unit 1301 via an input / output controller 1310 connected to the system bus 1305. The basic input / output system 1306 may also include the input / output controller 1310 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1310 also provides output to a display screen, printer, or other types of output devices.

[0224] The mass storage device 1307 is connected to the central processing unit 1301 via a mass storage controller (not shown) connected to the system bus 1305. The mass storage device 1307 and its associated computer-readable media provide non-volatile storage for the server 1300. That is, the mass storage device 1307 may include computer-readable media (not shown) such as a hard disk or drive.

[0225] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include random access memory (RAM), read-only memory (ROM), flash memory or other solid-state storage technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1304 and mass storage device 1307 described above can be collectively referred to as memory.

[0226] The memory stores one or more programs, which are configured to be executed by one or more central processing units 1301. The one or more programs contain instructions for implementing the methods described above, and the central processing unit 1301 executes the one or more programs to implement the steps on the cloud application server side in the various method embodiments described above.

[0227] According to various embodiments of this application, the server 1300 can also be connected to a remote computer on a network, such as the Internet. That is, the server 1300 can be connected to the network 1312 via the network interface unit 1311 connected to the system bus 1305, or the network interface unit 1311 can be used to connect to other types of networks or remote computer systems (not shown).

[0228] The memory further includes one or more programs stored in the memory, and the one or more programs include steps executed by the cloud application server in the method provided in the embodiments of this application.

[0229] Please refer to Figure 14 This diagram illustrates a structural block diagram of a terminal provided in an exemplary embodiment of this application. The terminal 1400 can be a portable mobile terminal, such as a smartphone, tablet computer, Moving Picture Experts Group Audio Layer III (MP3) player, or Moving Picture Experts Group Audio Layer IV (MP4) player. The terminal 1400 may also be referred to as a user device, portable terminal, or other names.

[0230] Typically, terminal 1400 includes a processor 1401 and a memory 1402.

[0231] Processor 1401 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1401 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1401 may also include a main processor and a coprocessor. The main processor, also known as a central processing unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1401 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1401 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0232] Memory 1402 may include one or more computer-readable storage media, which may be tangible and non-transitory. Memory 1402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 1402 is used to store at least one instruction, which is executed by processor 1401 to implement the method provided in the embodiments of this application.

[0233] In some embodiments, the terminal 1400 may also optionally include: a peripheral device interface 1403 and at least one peripheral device. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1404, a touch display screen 1405, a camera assembly 1406, an audio circuit 1407, a positioning assembly 1408, and a power supply 1409.

[0234] Peripheral interface 1403 can be used to connect at least one input / output (I / O) related peripheral device to processor 1401 and memory 1402. In some embodiments, processor 1401, memory 1402 and peripheral interface 1403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1401, memory 1402 and peripheral interface 1403 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0235] The radio frequency (RF) circuit 1404 is used to receive and transmit radio frequency (RF) signals, also known as electromagnetic signals. The RF circuit 1404 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1404 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1404 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or Wireless Fidelity (WiFi) networks. In some embodiments, the RF circuit 1404 may also include circuitry related to Near Field Communication (NFC), which is not limited in this application.

[0236] The touch display screen 1405 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, and any combination thereof. The touch display screen 1405 also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to the processor 1401 for processing. The touch display screen 1405 is used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one touch display screen 1405, positioned on the front panel of the terminal 1400; in other embodiments, there may be at least two touch display screens, respectively positioned on different surfaces of the terminal 1400 or in a folded design; in still other embodiments, the touch display screen 1405 may be a flexible display screen, positioned on a curved or folded surface of the terminal 1400. Furthermore, the touch display screen 1405 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The touch display screen 1405 may be made of materials such as Liquid Crystal Display (LCD) or Organic Light-Emitting Diode (OLED).

[0237] The camera assembly 1406 is used to acquire images or videos. Optionally, the camera assembly 1406 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is used for video calls or selfies, and the rear-facing camera is used for taking photos or videos. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, and a wide-angle camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, and panoramic shooting and virtual reality (VR) shooting by fusion of the main camera and the wide-angle camera. In some embodiments, the camera assembly 1406 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash is a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0238] Audio circuitry 1407 provides an audio interface between the user and terminal 1400. Audio circuitry 1407 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to processor 1401 for processing, or input to radio frequency circuitry 1404 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of terminal 1400. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from processor 1401 or radio frequency circuitry 1404 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, audio circuitry 1407 may also include a headphone jack.

[0239] Positioning component 1408 is used to locate the current geographic location of terminal 1400 in order to enable navigation or location-based service (LBS). Positioning component 1408 can be a positioning component based on the U.S. Global Positioning System (GPS), China's BeiDou system, or Russia's Galileo system.

[0240] Power supply 1409 is used to power the various components in terminal 1400. Power supply 1409 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1409 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0241] In some embodiments, the terminal 1400 further includes one or more sensors 1410. The one or more sensors 1410 include, but are not limited to: an accelerometer 1411, a gyroscope 1412, a pressure sensor 1413, a fingerprint sensor 1414, an optical sensor 1415, and a proximity sensor 1416.

[0242] Accelerometer 1411 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal 1400. For example, accelerometer 1411 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1401 can control touchscreen 1405 to display the user interface in landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1411. Accelerometer 1411 can also be used for games or for acquiring user motion data.

[0243] The gyroscope sensor 1412 can detect the orientation and rotation angle of the terminal 1400. The gyroscope sensor 1412 can work in conjunction with the accelerometer sensor 1411 to collect the user's 3D movements on the terminal 1400. Based on the data collected by the gyroscope sensor 1412, the processor 1401 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0244] The pressure sensor 1413 can be disposed on the side bezel of the terminal 1400 and / or on the lower layer of the touch display screen 1405. When the pressure sensor 1413 is disposed on the side bezel of the terminal 1400, it can detect the user's grip signal on the terminal 1400 and perform left / right hand recognition or quick operation based on the grip signal. When the pressure sensor 1413 is disposed on the lower layer of the touch display screen 1405, it can control the operable controls on the UI interface based on the user's pressure operation on the touch display screen 1405. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0245] The fingerprint sensor 1414 is used to collect a user's fingerprint to identify the user's identity. When the user's identity is identified as trusted, the processor 1401 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 1414 can be located on the front, back, or side of the terminal 1400. When the terminal 1400 has a physical button or manufacturer logo, the fingerprint sensor 1414 can be integrated with the physical button or manufacturer logo.

[0246] An optical sensor 1415 is used to collect ambient light intensity. In one embodiment, the processor 1401 can control the display brightness of the touch screen 1405 based on the ambient light intensity collected by the optical sensor 1415. Specifically, when the ambient light intensity is high, the display brightness of the touch screen 1405 is increased; when the ambient light intensity is low, the display brightness of the touch screen 1405 is decreased. In another embodiment, the processor 1401 can also dynamically adjust the shooting parameters of the camera assembly 1406 based on the ambient light intensity collected by the optical sensor 1415.

[0247] The proximity sensor 1416, also known as a distance sensor, is typically located on the front of the terminal 1400. The proximity sensor 1416 is used to detect the distance between the user and the front of the terminal 1400. In one embodiment, when the proximity sensor 1416 detects that the distance between the user and the front of the terminal 1400 is gradually decreasing, the processor 1401 controls the touchscreen display 1405 to switch from a screen-on state to a screen-off state; when the proximity sensor 1416 detects that the distance between the user and the front of the terminal 1400 is gradually increasing, the processor 1401 controls the touchscreen display 1405 to switch from a screen-off state to a screen-on state.

[0248] Those skilled in the art will understand that Figure 14 The structure shown does not constitute a limitation on terminal 1400 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0249] This application also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the cloud application screen display method described in any of the above embodiments.

[0250] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the cloud application screen display method described in the above embodiments.

[0251] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0252] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A display method of a cloud application screen, characterized by, The method is used for cloud application servers, and the method includes: Run the cloud application and obtain the cloud application screen during the running process; The screen segmentation strategy is determined based on the application identifier of the cloud application and the current running scenario. The screen segmentation strategy is related to the change range of the screen under the current running scenario. The screen segmentation strategy includes a first screen segmentation strategy and a second screen segmentation strategy. The first screen segmentation strategy indicates that the main screen and the edge screen are segmented. The main screen is more important than the edge screen. The second screen segmentation strategy indicates that the screen is segmented proportionally and the sub-screens obtained by proportional segmentation are of equal importance. Based on the screen segmentation strategy corresponding to the cloud application, the cloud application screen is divided into at least two sub-screens; At least two of the sub-screens are encoded in parallel to obtain at least two sub-encoded data blocks corresponding to the cloud application screen; At least two sub-encoded data blocks are sent in parallel to the proxy server, so that the proxy server sends the at least two sub-encoded data blocks to the terminal, the terminal decodes the at least two sub-screens, and splices the decoded at least two sub-screens to obtain the cloud application screen. If the first screen segmentation strategy is adopted, the proxy server is also used to discard some sub-encoded data blocks corresponding to sub-screens during transmission based on the importance of the sub-screens.

2. The method of claim 1, wherein, The screen segmentation strategy based on the cloud application divides the cloud application screen into at least two sub-screens, including: From the screen segmentation strategy, obtain the main area information of the main area and the edge area information of the edge area in the cloud application screen; Based on the main area information and the edge area information, the cloud application screen is divided into at least one main screen and at least one edge screen, wherein the main screen is more important than the edge screen.

3. The method of claim 2, wherein, The parallel encoding of at least two sub-screens to obtain at least two sub-encoded data blocks corresponding to the cloud application screen includes: The main screen is encoded by the target GPU that renders the cloud application screen to obtain the main screen encoded data block corresponding to the main screen; The edge image is copied to another GPU other than the target GPU, and the edge image is encoded by the other GPU to obtain the edge image encoded data block corresponding to the edge image.

4. The method according to claim 3, characterized in that, The step of copying the edge image to another GPU other than the target GPU and encoding the edge image using another CPU to obtain the edge image encoded data block corresponding to the edge image includes: Obtain the working status of other GPUs besides the target GPU, including idle status and busy status; In response to the existence of an idle GPU, the edge image is copied to the idle GPU, and the edge image is encoded by the idle GPU to obtain the edge image encoded data block corresponding to the edge image; The step of performing parallel encoding on at least two sub-screens to obtain at least two sub-encoded data blocks corresponding to the cloud application screen further includes: In response to the absence of the idle GPU, the edge image is copied to memory, and the edge image is encoded by the CPU to obtain the edge image encoded data block corresponding to the edge image.

5. The method according to claim 3, characterized in that, The step of encoding the edge image using other GPUs to obtain the edge image encoded data block corresponding to the edge image includes: At least two encoding methods are used to encode the edge image using other GPUs to obtain at least two types of edge image encoded data blocks, wherein different encoding methods correspond to different compression rates; Sending at least two of the sub-encoded data blocks to the proxy server in parallel includes: The proxy server sends the main screen encoded data block and at least two types of edge screen encoded data blocks in parallel, so that the proxy server determines the edge screen encoded data block to send to the terminal based on the network status between the proxy server and the terminal, wherein the compression rate of the edge screen encoded data block sent to the terminal is negatively correlated with the network status.

6. The method according to claim 1, characterized in that, The screen segmentation strategy based on the cloud application divides the cloud application screen into at least two sub-screens, including: Obtain proportional segmentation information from the aforementioned image segmentation strategy; Based on the proportional segmentation information, the cloud application screen is proportionally divided into at least two sub-screens, wherein the different sub-screens have the same level of importance.

7. The method according to any one of claims 1 to 6, characterized in that, The process of determining the screen segmentation strategy based on the application identifier of the cloud application and the current running scenario includes: In response to the cloud application corresponding to the first screen segmentation strategy and the second screen segmentation strategy, and the current running scenario being the first running scenario, the screen segmentation strategy is determined to be the first screen segmentation strategy; In response to the cloud application corresponding to the first screen segmentation strategy and the second screen segmentation strategy, and the current running scenario being the second running scenario, the screen segmentation strategy is determined to be the second screen segmentation strategy, wherein the screen change range in the second running scenario is greater than the screen change range in the first running scenario.

8. A method for displaying a cloud application screen, characterized in that, The method is used in a terminal, and the method includes: The proxy server receives at least two sub-encoded data blocks sent by a proxy server. These sub-encoded data blocks are obtained by the cloud application server encoding at least two sub-screens in parallel. The at least two sub-screens are obtained by segmenting the cloud application screen based on a screen segmentation strategy corresponding to the cloud application. The screen segmentation strategy is determined by the cloud application server based on the application identifier of the cloud application and the current running scenario. The screen segmentation strategy is related to the magnitude of screen changes under the current running scenario. The screen segmentation strategy includes a first screen segmentation strategy and a second screen segmentation strategy. The first screen segmentation strategy indicates segmentation into main screen and edge screen segments, with the main screen having a higher importance than the edge screen. The second screen segmentation strategy indicates proportional segmentation, and the sub-screens obtained from proportional segmentation have the same importance. If the first screen segmentation strategy is used, the proxy server is also used to discard some sub-encoded data blocks corresponding to the sub-screens during transmission based on the importance of the sub-screens. Decode at least two of the sub-coded data blocks to obtain at least two sub-pictures; At least two of the sub-screens are stitched together to obtain the cloud application screen, which is then displayed.

9. The method according to claim 8, characterized in that, Decoding at least two of the sub-coded data blocks to obtain at least two sub-pictures includes: The main screen encoded data block is decoded by the GPU, and the edge screen encoded data block is decoded by the CPU to obtain the main screen and the edge screen. The main screen is of higher importance than the edge screen. The step of stitching together at least two of the sub-screens to obtain and display the cloud application screen includes: The main screen and the edge screen are stitched together to obtain the cloud application screen, which is then displayed.

10. A display device for a cloud application screen, characterized in that, The device includes: The screen acquisition module is used to run the cloud application and acquire the cloud application screen during the running process of the cloud application; The second strategy determination module is used to determine a screen segmentation strategy based on the application identifier of the cloud application and the current running scenario. The screen segmentation strategy adopted is related to the change range of the screen under the current running scenario. The screen segmentation strategy includes a first screen segmentation strategy and a second screen segmentation strategy. The first screen segmentation strategy indicates that the main screen and the edge screen are segmented, and the main screen is more important than the edge screen. The second screen segmentation strategy indicates that the screen is segmented proportionally, and the sub-screens obtained by proportional segmentation are of equal importance. The screen segmentation module is used to segment the cloud application screen into at least two sub-screens based on the screen segmentation strategy corresponding to the cloud application. An encoding module is used to encode at least two sub-screens in parallel to obtain at least two sub-encoded data blocks corresponding to the cloud application screen; The data sending module is used to send at least two of the sub-encoded data blocks in parallel to the proxy server, so that the proxy server sends at least two of the sub-encoded data blocks to the terminal, and the terminal decodes the at least two sub-screens obtained by decoding to obtain the cloud application screen. If the first screen segmentation strategy is adopted, the proxy server is also used to discard some of the sub-encoded data blocks corresponding to the sub-screens during transmission based on the importance of the sub-screens.

11. A display device for a cloud application screen, characterized in that, The device includes: The data receiving module is used to receive at least two sub-encoded data blocks sent by the proxy server. These sub-encoded data blocks are obtained by the cloud application server through parallel encoding of at least two sub-screens. The at least two sub-screens are obtained by segmenting the cloud application screen based on a screen segmentation strategy corresponding to the cloud application. The screen segmentation strategy is determined by the cloud application server based on the application identifier of the cloud application and the current running scenario. The adopted screen segmentation strategy is related to the magnitude of screen changes under the current running scenario. The screen segmentation strategy includes a first screen segmentation strategy and a second screen segmentation strategy. The first screen segmentation strategy indicates segmentation of the main screen and edge screens, with the main screen having a higher importance than the edge screens. The second screen segmentation strategy indicates proportional segmentation, and the sub-screens obtained from proportional segmentation have the same importance. If the first screen segmentation strategy is adopted, the proxy server is also used to discard some sub-encoded data blocks corresponding to the sub-screens during transmission based on the importance of the sub-screens. A decoding module is used to decode at least two of the sub-encoded data blocks to obtain at least two sub-pictures; The display module is used to stitch together at least two of the sub-screens to obtain the cloud application screen and display it.

12. A server, characterized in that, The server includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement the method for displaying a cloud application screen as described in any one of claims 1 to 7.

13. A terminal, characterized in that, The terminal includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement the method for displaying a cloud application screen as described in any one of claims 8 to 9.

14. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the method for displaying a cloud application screen as described in any one of claims 1 to 7, or to implement the method for displaying a cloud application screen as described in any one of claims 8 to 9.

Images