A cloud phone remote interface display fluency optimization method

Abstract

The application relates to a cloud mobile phone remote interface display fluency optimization method, which comprises the following steps: extending a region prefetch mechanism; extending a remote server end region screenshot and coding; managing a local App memory buffer; after the local App receives coding data returned by the remote server, the coding data is stored in a local memory buffer; and rendering the local memory buffer during sliding; when a user performs a sliding operation, corresponding image data is directly extracted from the local memory buffer, rendering is performed through a graphics rendering library, and a new request does not need to be sent to the remote server. The application provides a cloud mobile phone remote interface display fluency optimization method, through optimization of a cache strategy and a request mechanism, the application reduces the delay and lag during up and down sliding in the cloud mobile phone remote interface function used by a customer, and improves the interactive fluency. Compared with a traditional on-demand request mode, the application reduces the number of network requests, and significantly improves the user experience on a low-power consumption platform.

Description

Technical Field

[0001] This invention relates to the fields of cloud computing and mobile communication technology, and in particular to a method for optimizing the smoothness of remote interface display on cloud phones. Background Technology

[0002] With the continuous evolution of cloud computing technology and the significant enhancement of network communication capabilities, cloud deployment has become the core architecture of modern applications and services. Cloud phones, as a key application in this architecture, enable users to remotely access and operate smart terminal resources via the internet, effectively freeing them from dependence on local high-performance hardware. This technology not only significantly reduces the hardware specifications required of local devices but also greatly enhances the flexibility and scalability of mobile computing, demonstrating broad application value in diverse scenarios such as low-power mobile terminals, remote enterprise environments, and cloud gaming. However, in practical applications, the display and interaction mechanisms of remote application interfaces on local devices still face significant challenges.

[0003] The current mainstream implementation uses an on-demand request mechanism based on the HTTP protocol. This means that the local application sends a request to a remote server when the user interacts with the interface. This request only contains the width and height parameters of the currently visible area to obtain the corresponding interface image data. In static display scenarios, this method can maintain a basically usable user experience. However, in dynamic interactive scenarios, especially when the user performs interface swiping operations, its inherent defects become glaringly apparent. During swiping, the user needs to frequently trigger new requests to obtain continuous frame data. This process involves multiple stages, including network transmission, image encoding and decoding, and screen rendering. Each stage introduces significant latency: network transmission is constrained by round-trip time fluctuations and bandwidth limitations; the server needs to perform screenshot and encoding operations in real time, consuming a large amount of computing resources; and the client faces timing pressures in decoding and rendering. These delays accumulate and amplify during rapid swiping, leading to frequent screen stuttering and lag, severely disrupting the continuity and naturalness of user interaction.

[0004] In-depth analysis reveals that each swipe request forces the server to regenerate the complete frame data and transmit it to the client. This not only causes a non-linear increase in server computing load but also excessively consumes network bandwidth resources. For client devices, the continuous network requests, data decoding, and rendering processes consume scarce memory and processor resources. Especially on low-power mobile devices, their limited hardware performance cannot support high-frequency data exchange tasks. Furthermore, traditional mechanisms lack adaptability to dynamic environments: when network conditions deteriorate (such as high latency or unstable connections) or when users perform complex swipe operations, the interface response speed drops sharply, smoothness is significantly reduced, and the user experience deteriorates accordingly. Some existing attempts to improve decoding performance by increasing the local device's CPU operating frequency are not feasible in typical cloud phone application scenarios. Terminal devices generally adopt low-power designs, and forcibly increasing the main frequency will lead to problems such as a surge in power consumption, shortened battery life, and device overheating. Therefore, this solution is not feasible in resource-constrained environments. In summary, there is an urgent need for an innovative data transmission and interface rendering strategy that can deeply integrate user interaction behavior characteristics, dynamic changes in network status, server load balancing, and the performance limits of mobile devices, to fundamentally alleviate screen lag issues and improve the real-time responsiveness and visual smoothness of dynamic content display. Summary of the Invention

[0005] Therefore, the purpose of this invention is to provide a method for optimizing the smoothness of remote interface display on cloud phones, which has the advantages of improving the smoothness of remote interface display on cloud phones and reducing lag during swiping operations.

[0006] This invention provides a method for optimizing the smoothness of remote interface display on cloud phones, comprising the following steps: S1. Extended area prefetching mechanism: When the local App starts or requests a new interface, the HTTP request sent to the remote server contains not only the width and height of the current visible area, but also an extended height, along with the relative position information of the current visible area. S2. Remote server-side extended area screenshot and encoding: After receiving the HTTP request, the remote server performs a screenshot operation on the current application interface based on the width and height information of the visible area, the extended height information and the position information in the request, ensuring that the screenshot range covers the extended height area, and encodes the captured image into an encoded data stream suitable for network transmission. S3. Local App Memory Buffer Management: After receiving encoded data returned by the remote server, the local App stores it in the local memory buffer. S4. Local memory buffer rendering during swiping: When the user performs a swipe operation, the local App listens to touch events and swipe distance to calculate the relative position of the new visible area in the entire interface, and extracts the corresponding image data directly from the local memory buffer based on this information, and renders it through the graphics rendering library without sending a new request to the remote server.

[0007] In one embodiment of the present invention, the following steps are also included: S5, Dynamic Buffer Refresh and Update: S5.1 When the user finishes swiping and releases their finger, the local app will send a new request to the remote server based on the latest location information POS to obtain new extended height area data and asynchronously update the contents of the local memory buffer. S5.2 During continuous swiping by the user, the local App monitors in real time whether the swipe exceeds the current buffer range. Once the swipe exceeds the range, it immediately sends a new request to the remote server to obtain new extended height area data. After the remote server returns the new extended height area data, it updates the local memory buffer in a timely manner to ensure that subsequent swipe operations can obtain the updated content from the local memory buffer for rendering.

[0008] In one embodiment of the present invention, the extended height covers a range that extends upward and downward by N times the screen height along the sliding direction, with the center line of the current visible area as a reference, where N > 1.

[0009] In one embodiment of the present invention, step S1 includes: S1.1, Precise location information calculation mechanism: The local app obtains the current finger position on the screen through a unique identifier on the interface. When a touch event occurs, the application calculates the relative position of the current visible area in the entire application interface based on the screen coordinates of the touch point, combined with the interface layout information and screen resolution. It then maps the position of the touch point to the overall layout of the interface to ensure that the position information can be accurately obtained and reflect the interface position corresponding to the touch event.

[0010] In one embodiment of the present invention, step S2 includes: a remote server capturing the area required by the current application interface through an image processing library, wherein the width of the screenshot is W, the height of the screenshot is 3H, and 3H is an extension of 1.5H upwards and downwards from the center line position of the currently visible area along the sliding direction.

[0011] In one embodiment of the present invention, step S2 further includes: conversion of the internal data format of the request parameters on the remote server side, that is, the remote server converts the parameter values ​​in the request header into an internal data format suitable for processing by the remote server.

[0012] In one embodiment of the present invention, the local App creates a local memory buffer in memory for storing and reading image data, the size of which is calculated based on the width W and height 3H of the image data.

[0013] In one embodiment of the present invention, step S5.2, local buffer rendering during sliding, includes: During the swipe operation, the local app listens to the user's touch events and swipe distance to determine how to extract data from the local memory buffer for rendering. Based on this information, the system calculates the position information of the current visible area relative to the overall interface, as well as the top, left, W, and H of the next frame image. Here, top and left represent the starting position for reading data from the local memory buffer, while W and H represent the width and height of the data area to be read.

[0014] In one embodiment of the present invention, step S5.2 further includes: the refresh operation of the local memory buffer is usually performed asynchronously, that is, the display content on the screen is updated immediately after the new image data is loaded.

[0015] In one embodiment of the present invention, step S4 includes: the system is able to locate the correct memory address in the local memory buffer, thereby extracting the corresponding image data and rendering the data onto the screen through a graphics rendering library.

[0016] The method for optimizing the smoothness of remote interface display on cloud phones described above has the following advantages compared to the prior art: By using an extended region prefetching mechanism, remote server-side extended region screenshotting and encoding, local App memory buffer management, and local memory buffer rendering during swiping, image data is directly extracted from the local memory buffer for rendering when the user swipes, avoiding frequent requests to the remote server. This solves the interface lag problem caused by swiping operations in existing technologies and has the advantages of improving display smoothness and reducing lag. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the method for optimizing the smoothness of the remote interface display of cloud phones according to the present invention; Figure 2This is a schematic diagram of the interface rendering and sliding operation of the present invention; Figure 3 This is a schematic diagram of the data processing flow during the local App swiping operation of the present invention. Detailed Implementation

[0019] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0020] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] It should be noted that the following description covers various aspects of embodiments within the scope of the appended claims. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0022] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0023] Additionally, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that practice can be carried out without these specific details.

[0024] Existing cloud phone remote interfaces frequently request new frame data from remote servers during user swiping operations. This introduces delays in multiple stages, including network transmission, image encoding / decoding, and screen rendering, resulting in noticeable screen stuttering and severely impacting the user experience. Furthermore, each swipe request causes the server to regenerate new frame data and transmit it to the client, increasing the server's computational load and consuming significant network bandwidth. For the client, the continuous network requests and decoding / rendering processes consume considerable computing resources, especially on low-power devices with limited processing power, making it difficult to support high-frequency data exchange and rendering. This leads to sluggish remote interface response and reduced smoothness.

[0025] To address this issue, this application proposes a method for optimizing the smoothness of remote interface display on cloud phones. When the local app launches or requests a new interface, an extended region prefetching mechanism is employed. The HTTP request sent to the remote server includes an extended height and the relative position information of the currently visible area. Upon receiving the request, the remote server takes a screenshot of the application interface covering the extended height area based on the information in the request, and encodes the captured image into an encoded data stream suitable for network transmission. The local app receives the encoded data returned by the remote server and stores it in its local memory buffer. When the user performs a swipe operation, the local app listens for touch events and swipe distances to calculate the relative position of the new visible area within the entire interface. Based on this information, it directly extracts the corresponding image data from the local memory buffer and renders it using a graphics rendering library, thus eliminating the need to send a new request to the remote server.

[0026] Reference Figure 1 As shown, the specific steps include the following: S1. Extended area prefetching mechanism: When the local App starts or requests a new interface, the HTTP request sent to the remote server contains not only the width and height of the current visible area, but also an extended height, along with the relative position information of the current visible area. S2. Remote Server-Side Extended Area Screenshot and Encoding: After receiving an HTTP request, the remote server takes a screenshot of the current application interface based on the width and height information of the visible area, the extended height information, and the position information in the request. This ensures the screenshot covers the extended height area, and the captured image is encoded into a data stream suitable for network transmission. Alternatively, the remote server can maintain a virtual application interface canvas. Upon receiving a request, the server captures a rectangular area from this canvas based on the size and position information in the request. The extended height can be interpreted as adding a portion of height at the top and bottom of the visible area. After capture, the image data can be directly compressed into single-frame data in video encoding formats such as H.264 or VP9 to suit network transmission.

[0027] S3. Local App Memory Buffer Management: After receiving encoded data from the remote server, the local app stores it in a local memory buffer. This memory area can be a simple byte array or a circular buffer. Data can be stored in the order it is received, or indexed according to its relative position within the entire interface. Alternatively, the local app can create a data structure, such as a hash table or linked list, to manage the received image data blocks. Each data block can contain the image data itself and its corresponding interface area information. When data arrives, it is decompressed and stored in this data structure for quick subsequent lookup and access.

[0028] S4. Local Memory Buffer Rendering During Swiping: When the user swipes, the local app listens for touch events and swipe distance to calculate the relative position of the new visible area within the entire interface. Based on this information, it directly extracts the corresponding image data from the local memory buffer and renders it using the graphics rendering library, without sending new requests to a remote server. One implementation involves the local app maintaining a virtual scrollbar or an internal coordinate system to track user swipes. When the user swipes, the app updates its internal view offset based on the swipe event. Then, the app selects image data blocks corresponding to the current screen display area from the local memory buffer based on this offset. These data blocks are directly loaded into video memory and rendered through the graphics rendering pipeline to achieve smooth scrolling of the interface.

[0029] This application effectively reduces the frequency of requests to the remote server by the cloud phone's remote interface during user swiping operations by extending the region prefetching and the local memory buffer rendering mechanism, significantly reducing network transmission latency and server computing load. This avoids screen stuttering and improves the real-time performance and smoothness of dynamic content display, especially in low-power devices and environments with poor network conditions, providing a superior user experience.

[0030] In the above-described embodiments of this application, an extended region prefetching mechanism is proposed to reduce the number of network requests and improve the smoothness of scrolling. However, in its implementation, when the user scrolls beyond the prefetching range or when the scrolling ends, the contents of the local memory buffer may become outdated, causing subsequent scrolling operations to wait for new data requests, thereby introducing latency and affecting the user experience.

[0031] To address this, this application further proposes a method for optimizing the smoothness of remote interface display on cloud phones, including the following steps: S5, dynamic buffer refresh and update. This step aims to solve the problem that the content of the local memory buffer may become outdated due to the user ending the swipe or exceeding the prefetch range, ensuring that the local App always has the latest interface data in dynamic interaction scenarios, thereby maintaining a smooth user experience. Its core lies in actively or passively triggering the update of the buffer content based on user behavior (ending the swipe or continuous swipe exceeding the range). Specifically, a state machine can be set in the local App to determine whether a buffer update needs to be triggered based on the end state of the user's touch event or a threshold of the swipe distance; alternatively, a timer or background thread can be maintained in the local App to periodically check the matching degree between the current visible area and the buffer content, and initiate the update process when a mismatch is detected.

[0032] Specifically, for step S5, dynamic buffer refresh and update, refer to... Figure 3 As shown, it specifically includes: S5.1 When the user finishes swiping and releases their finger, the local app will send a new request to the remote server based on the latest location information POS to obtain new extended height area data and asynchronously update the contents of the local memory buffer. This sub-step focuses on updating the buffer after the user stops swiping. When the user completes a swipe and stops touching the screen, the native app needs to obtain the latest data around the currently visible area to prepare for the next possible interaction. The asynchronous update mechanism avoids blocking the user interface while waiting for a server response. For example, the native app can trigger an event handler upon receiving a touch event `ACTION_UP` or `ACTION_CANCEL`. This handler encapsulates the latest position information (POS) of the currently visible area and sends a data request to a remote server via protocols such as HTTP or WebSocket. Alternatively, when the swipe ends, the native app can place the update request in a request queue, with a separate network request module responsible for sending requests in order and processing server responses, ensuring the main UI thread is not blocked.

[0033] S5.2 During continuous swiping by the user, the local App monitors in real time whether the swipe exceeds the current buffer range. Once the swipe exceeds the range, it immediately sends a new request to the remote server to obtain new extended height area data. After the remote server returns the new extended height area data, it updates the local memory buffer in a timely manner to ensure that subsequent swipe operations can obtain the updated content from the local memory buffer for rendering.

[0034] This sub-step aims to address the issue that the current prefetch buffer may be insufficient to cover new visible areas when a user swipes rapidly and continuously. By monitoring in real-time and making immediate requests, it ensures that the buffer content is dynamically and proactively updated during the user's swipe, thus avoiding stuttering due to insufficient data. For example, the local app can maintain a swipe distance counter or a prediction model to predict the upcoming visible area based on the user's swipe speed and direction. When the predicted area exceeds the coverage of the current local memory buffer, a new data request is immediately triggered; alternatively, before each rendering, the local app checks whether the current visible area is completely contained within the local memory buffer. If not, the missing area is calculated, and a request is immediately sent to a remote server to retrieve data for that missing area, while continuing rendering using the existing data in the buffer, updating only after the new data is returned.

[0035] Through the above technical solution, this application effectively solves the problem of interface lag caused by outdated content in the local memory buffer when the user finishes swiping or continuously swipes beyond the prefetch range in the remote interface display of a cloud phone. Specifically, when the user finishes swiping and releases their finger, the local app can send a new request to the remote server based on the latest location information (POS) to obtain and asynchronously update the content of the local memory buffer. This ensures that the buffer is ready with the latest interface data in a timely manner after the user stops swiping, avoiding delays in subsequent operations due to outdated data. At the same time, during continuous swiping, the local app monitors in real time whether the swipe exceeds the current buffer range. Once it detects an exceedance, it immediately sends a new request to the remote server to obtain and update the local memory buffer in a timely manner. This allows the buffer content to be dynamically and proactively replenished in fast swiping scenarios, ensuring that the user can always obtain the latest image data from the local buffer for rendering during continuous swiping. This significantly improves the display smoothness of the remote interface and the user interaction experience, and effectively reduces latency caused by frequent waiting for network requests.

[0036] In the above-described embodiments of this application, it is further proposed that the extended height covers a range that extends upward and downward by N times the screen height along the sliding direction, with the current visible area center line as a reference, where N > 1.

[0037] Specifically, "the extended height covers the area based on the center line of the current visible region" means that when determining the prefetch area, the system does not simply start extending from one edge of the current visible region, but uses the vertical center line of the current visible region as the axis of symmetry. For example, the system can obtain the top and bottom coordinates of the current visible region and calculate the vertical position of its center line, using this as the axis of symmetry for the extended area; or, the local app can determine the geometric center of the current visible region based on the current screen resolution and interface layout information, and calculate the extended height based on this center line. This center line-based setting ensures that the prefetch area is evenly distributed above and below the user's current focus point, avoiding potential blind spots caused by prefetching biased in one direction.

[0038] "Expanding the range upwards and downwards by N times the screen height along the swipe direction" refers to the actual size and direction of the pre-fetched area. In scenarios where the user might swipe up or down, the system pre-loads content above and below the currently visible area by N times the screen height. For example, when a local app sends a request, it calculates the upward and downward expansion ranges by NH based on the actual screen height H and includes this information in the HTTP request; alternatively, after receiving the request, the remote server determines the start and end positions of the screenshot based on the screen height information and the N value included in the request, thus covering the area above and below by N times the screen height. This bidirectional expansion strategy aims to pre-cover the area the user might swipe, thereby reducing the frequency of sending new requests to the remote server during swipes.

[0039] Here, N > 1 is a constraint on the expansion factor N. This constraint ensures that the prefetch area is at least greater than the height of one screen in each direction. For example, the system defaults to a minimum value of 1.5 for N to ensure that the prefetch area is large enough, for example, expanding upwards and downwards by 1.5 times the screen height each; or, the value of N can be dynamically adjusted according to network conditions, device performance, or user habits, but always remain greater than 1. For example, when network conditions are good, N can be set to 2 or 3 to provide a larger prefetch range. By setting N to be greater than 1, a sufficiently large buffer area can be provided for the user to cope with fast or large-amplitude swiping operations, significantly reducing screen stuttering caused by insufficient prefetching.

[0040] Through the above technical solution, this application clearly defines the specific range of the extended height, solving the problem of unclear prefetching area. By extending bidirectionally based on the center line of the current visible area and ensuring the extended range is greater than one screen height, the local app can extract sufficient image data from the local memory buffer for rendering when the user scrolls, thus significantly reducing the number of new requests sent to the remote server. This not only effectively avoids the problem of screen stuttering caused by insufficient prefetching area but also avoids the server's computational load and network bandwidth waste caused by excessive prefetching. This solution, by optimizing the prefetching mechanism, improves the display smoothness of the cloud phone's remote interface in dynamic scrolling scenarios, providing users with a smoother and more responsive interactive experience.

[0041] In the above-described embodiments of this application, an extended region prefetching mechanism is proposed to optimize display smoothness. However, in its implementation, inaccurate location information calculation may lead to inaccurate server screenshot range, increasing network latency and rendering errors.

[0042] In response, this application further proposes a method for optimizing the smoothness of remote interface display on cloud phones, wherein step S1 specifically includes: S1.1, Precise location information calculation mechanism: The local app obtains the current finger position on the screen through a unique identifier on the interface. When a touch event occurs, the application calculates the relative position of the current visible area in the entire application interface based on the screen coordinates of the touch point, combined with the interface layout information and screen resolution. It then maps the position of the touch point to the overall layout of the interface to ensure that the position information can be accurately obtained and reflect the interface position corresponding to the touch event.

[0043] Specifically, a native app obtains the current finger's position on the screen through unique identifiers on the interface. These unique identifiers are unique markers used to identify interface elements or specific areas, such as control IDs, DOM element IDs, and Accessibility IDs. Using these identifiers, the native app can accurately locate the screen area touched by the user's finger, ensuring the accuracy of the positional information. For example, the coordinates or bounding box information of a specific UI element can be obtained through APIs provided by UI automation testing frameworks (such as Appium and Selenium), and then matched with the coordinates of the touch event; alternatively, developers can pre-define unique identifiers for key interactive areas when designing the app. When the native app receives a touch event, it compares the coordinates of the touch point with the areas associated with these identifiers to determine the finger's position.

[0044] When a touch event occurs, the application calculates the relative position of the currently visible area within the entire application interface based on the touch point's screen coordinates, combined with interface layout information and screen resolution. The touch point's screen coordinates are the raw pixel coordinates reported by the operating system or touchscreen hardware. Interface layout information refers to data such as the arrangement, size, and hierarchy of UI elements defined within the app. Screen resolution is the device's physical pixel size. By combining this information, the application can convert the raw screen coordinates into logical coordinates, or relative position, within the entire scrollable or scalable application interface.

[0045] Simultaneously, the touch point's position is mapped to the overall layout of the interface. Mapping to the overall layout means aligning local (touch point) information with the global (entire application interface) structure and coordinate system. This ensures that even if the interface is scrollable or scalable, the touch point's position is accurately understood as a specific point within the entire potential, complete application content. For example, after calculating the relative position of the current visible area, the application associates the touch point's coordinates with that relative position. If the top-left corner of the visible area has coordinates (X, Y) in the overall layout, and the touch point's coordinates on the screen are (x, y), then the touch point's coordinates in the overall layout are (X + x_offset, Y + y_offset), where x_offset and y_offset are the offsets of the touch point relative to the top-left corner of the visible area. Alternatively, the application can maintain a coordinate system for a virtual canvas or content area, upon which the layout of all UI elements is based. When a touch event occurs, the screen coordinates are first converted to points in this virtual canvas coordinate system, thus achieving mapping with the overall layout. The above mechanism ensures that location information can be accurately obtained, reflecting the interface location corresponding to the touch event.

[0046] Through the above technical solution, this application introduces a precise location information calculation mechanism, enabling the local App to provide highly accurate relative location information of the current visible area when sending HTTP requests to the remote server. This solves the problem of inaccurate screenshot range caused by inaccurate location information calculation in traditional methods. After receiving the precise location information, the remote server can more accurately locate and capture the data actually needed by the user when performing the extended area screenshot and encoding in step S2, avoiding the problem of inaccurate screenshot range caused by position deviation. By reducing unnecessary screenshot and encoding work, the computational load of the server and the amount of data transmitted over the network are effectively reduced, thereby reducing network latency. At the same time, due to the accuracy of the screenshot range, when the local App performs local memory buffer rendering during sliding in step S4, it can extract image data that better matches the user's expectations from the local memory buffer, significantly reducing rendering errors and screen stuttering, and improving the smoothness of the remote interface display of the cloud phone. This precision also provides a solid foundation for subsequent dynamic buffer refresh and update (as described in some of the above embodiments), ensuring that a high-quality user experience can be maintained even in fast sliding or complex interaction scenarios.

[0047] In the above-described embodiments of this application, a remote server-side extended area screenshot and encoding method is proposed to cover the extended height area and generate image data suitable for transmission. However, in its implementation, the definition of the screenshot range lacks specific size specifications, which may lead to inaccurate or redundant image capture areas, resulting in excessive data transmission, waste of server resources, and low efficiency of local buffer management, thereby affecting the rendering smoothness and response speed during sliding operations.

[0048] In response, this application further proposes that when expanding the screenshot and encoding on the remote server side, the remote server uses an image processing library to capture the area required by the current application interface, referring to... Figure 2 As shown, the width of the screenshot is W, and the height of the screenshot is 3H. 3H is the distance that extends 1.5H upwards and downwards from the center line of the currently visible area along the sliding direction.

[0049] Specifically, the remote server uses an image processing library to capture the area required by the current application interface. The remote server is responsible for performing the actual screenshot operation, while the image processing library is a software toolset that implements this function. Its role is to provide efficient and accurate image capture capabilities, ensuring that the required pixel data can be extracted from the remote application interface according to specified parameters. For example, an operating system-level screenshot API can be used, combined with an image processing library for subsequent cropping, scaling, and other processing; alternatively, specific APIs provided by cloud platforms or virtualization environments can be used. These APIs can directly access the virtual machine's graphics output buffer, thereby efficiently acquiring image data of the application interface without affecting the virtual machine's performance.

[0050] The screenshot width is denoted by W. W represents the screenshot width, which is usually consistent with the width of the current visible area. Its purpose is to ensure that the captured image matches the display ratio of the local device in the horizontal direction, avoiding horizontal stretching or compression during local rendering, thus maintaining the original layout and visual effect of the interface. After receiving the HTTP request from the local app, the remote server can directly obtain the width value of the current visible area from the request parameters and use it as the width parameter for the screenshot operation; alternatively, the remote server can dynamically determine a standard width W based on the local app's device type or preset display configuration, or adaptively adjust the screenshot width to fit the content based on the layout characteristics of the remote application itself.

[0051] The screenshot height is 3H. 3H represents the total height of the screenshot, where H typically refers to the height of the current visible area. This height value is expanded, making it much larger than the height of a single visible area. Its purpose is to pre-capture potential display content above and below the current visible area (or in the forward / backward direction of the swipe), providing sufficient local buffer data for user swipe operations and reducing the need to frequently request new data from the server due to swipes. The remote server can calculate 3H as the total screenshot height based on the current visible area height H provided in the local app's request; alternatively, the remote server can dynamically adjust this expansion factor based on the characteristics of the current application (e.g., whether it's a long list, document, etc.) or the user's historical swipe behavior patterns, thereby determining a more optimized screenshot height to balance data volume and prefetching effectiveness.

[0052] Furthermore, 3H represents an extension of 1.5H upwards and downwards from the center line of the current visible area along the swipe direction. This further precisely defines the distribution of the 3H height: extending 1.5H upwards and downwards from the center line of the current visible area. This symmetrical, center-line-based extension ensures that pre-loaded image data can be found in the local buffer regardless of the swipe direction, whether the user swipes upwards or downwards. Its purpose is to optimize buffer space utilization and ensure a smooth experience during bidirectional swipes, avoiding buffering issues that might occur with unidirectional expansion. The remote server first determines the vertical coordinates of the center line of the current visible area within the entire application interface. Then, using this center line as a reference, it calculates the starting Y-coordinate (center line coordinates minus 1.5H) upwards and the ending Y-coordinate (center line coordinates plus 1.5H) downwards, thus determining the vertical range of the screenshot. Alternatively, the remote server can dynamically adjust the vertical expansion ratio based on the user's current swipe trend to more accurately match the user's swipe intentions and further optimize the buffer hit rate.

[0053] The above technical solution clearly defines the specific parameters of the screenshot size, solving the problem of inaccurate screenshot range and thus optimizing the efficiency of image data acquisition and transmission. The precise screenshot width W is consistent with the width of the visible area, avoiding horizontal stretching or compression of the image during local rendering. The screenshot height is 3H, and based on the center line of the current visible area, it extends upwards and downwards by 1.5H each along the sliding direction, ensuring sufficient and symmetrical prefetched data to cover the potential visible area when the user slides in both directions, significantly reducing additional network requests triggered by sliding beyond the buffer range. This precise screenshot range avoids capturing redundant image data, effectively reducing the amount of data transmitted over the network, lowering network bandwidth usage and the encoding processing load on the remote server. Simultaneously, the data received by the local app is precisely cropped, allowing the local memory buffer to store and manage image data more efficiently, avoiding the storage of unnecessary pixel information. Because sufficient and precise image data is pre-stored in the local buffer, the local app can directly extract and render it from the buffer when the user slides, significantly reducing the impact of network latency and server response time on the smoothness of the slide and improving the user experience. In particular, when combined with the dynamic buffer refresh and update mechanism, the precise 3H screenshot range allows the system to rely on local buffer data for a longer period of time during continuous swiping, reducing emergency requests triggered by exceeding the buffer range and further improving the smoothness and responsiveness of continuous swiping.

[0054] Furthermore, S2 also includes: internal data format conversion of remote server request parameters, that is, the remote server converts the parameter values ​​in the request header into an internal data format suitable for remote server processing. This internal data format conversion refers to the remote server parsing and processing the parameter values ​​carried in the request header after receiving the HTTP request, converting them from an external transmission format into a data structure or type that the server's internal system can directly and efficiently recognize and manipulate. This conversion process aims to eliminate parsing errors, data loss, or additional processing overhead that may occur due to incompatibility between the external data format and the server's internal processing logic. Specifically, this conversion can be implemented in various ways. For example, the remote server can pre-set a set of parameter mapping rules to parse and convert string type parameters (such as width, height, extended height, etc.) in the request header into integer or floating-point variables within the server, and perform necessary range checks. Another implementation method is to integrate a specific data parsing library or framework on the server side. This library can automatically identify and convert the parameters in the request header according to a predefined protocol or data model, encapsulating them into internal server objects for direct use by subsequent screenshot and encoding modules. These mechanisms ensure the accuracy and availability of parameters, laying the foundation for subsequent image processing.

[0055] Through the above technical solution, the remote server can efficiently and accurately process the request parameters sent by the client. Since the request parameters are converted into an internal format best suited for server processing before entering the core processing logic, additional parsing, verification, or conversion steps due to format mismatches are avoided, thus significantly reducing server-side processing latency. This not only improves the response speed and accuracy of screenshot and encoding operations in step S2 but also reduces the server's computational load, ensuring smooth data flow. Consequently, the overall smoothness of the cloud phone remote interface display is further optimized; users can receive updated interface images faster during swiping operations, effectively alleviating lag and improving the interactive experience.

[0056] In the above-described embodiments of this application, a remote server is proposed to capture image data with a width of W and a height of 3H for transmission to a local App for storage and rendering. However, in its implementation, if the size of the local memory buffer is not set according to the actual size of the image data, it may lead to wasted or insufficient memory resources, affecting the storage efficiency and rendering performance of the image data.

[0057] In response, this application further proposes that the local App create a local memory buffer in memory for storing and reading image data, the size of which is calculated based on the width W and height 3H of the image data.

[0058] Specifically, this local memory buffer is a dedicated storage area allocated by the local app within the device's memory. Its main function is to temporarily store image data received from a remote server. By storing image data locally, frequent requests to the remote server during user actions such as swiping can be significantly reduced, thereby lowering network latency and server load. One implementation approach is for the local app to utilize the operating system's dynamic memory allocation mechanism to create and adjust the buffer size as needed at runtime. Another approach is for the local app to pre-allocate a fixed-size memory area as a buffer at application startup and manage and reuse it in subsequent operations. Furthermore, for scenarios requiring the processing of large amounts of image data, the local app can also consider using memory-mapped file technology to directly map file content into the process's virtual address space, thereby achieving efficient data access.

[0059] Meanwhile, the size of the local memory buffer is calculated based on the width W and height 3H of the image data. This feature clarifies how the capacity of the local memory buffer is determined; its size is not arbitrarily set, but precisely calculated based on the actual dimensions (width W and height 3H) of the image data captured and transmitted by the remote server. This precise calculation ensures that the buffer can accommodate the complete image data while avoiding unnecessary memory consumption.

[0060] Through the above technical solution, the local app can accurately create and manage a local memory buffer based on the actual dimensions (width W and height 3H) of the image data transmitted from the remote server. This precise memory allocation mechanism effectively avoids wasted memory resources due to an excessively large buffer, especially for low-power devices, optimizing the utilization of their limited memory resources. Simultaneously, it avoids the problem of insufficient image data storage due to an excessively small buffer, ensuring that the extended area image data received from the remote server can be stored completely and efficiently locally. When the user performs a swipe operation, the local app can directly extract the required image data from this appropriately sized local memory buffer for rendering, without frequently sending new requests to the remote server, thus significantly reducing network latency and server load. This not only improves the storage efficiency and access speed of image data, but more importantly, it ensures the continuity and smoothness of the interface content when the user swipes quickly, greatly improving the user interaction experience of the cloud phone's remote interface and reducing the occurrence of lag.

[0061] Furthermore, in S5.2, the local buffer rendering during sliding includes: During the swipe operation, the local app listens to the user's touch events and swipe distance to determine how to extract data from the local memory buffer for rendering. Based on this information, the system calculates the position information of the current visible area relative to the overall interface, as well as the top, left, W, and H of the next frame image. Here, top and left represent the starting position for reading data from the local memory buffer, while W and H represent the width and height of the data area to be read.

[0062] Specifically, local apps can acquire real-time user interaction behavior on the screen by listening to touch events and swipe distances. This can be achieved through the standard touch event callback mechanism provided by the operating system. One implementation method is to utilize a more advanced gesture recognition framework. This framework can parse the raw touch event sequence, identify specific gestures such as scrolling and dragging, and provide more abstract swipe distance and direction information. In this way, the system can respond promptly to the user's operational intentions, providing accurate input for subsequent data extraction and rendering decisions.

[0063] Based on detected touch events and swipe distances, the local app needs to intelligently determine which image data to extract from the local memory buffer for rendering. This can be done by predicting what the user will see based on the swipe direction and speed, combined with the current position of the visible area, and pre-locating or loading the corresponding image blocks from the buffer.

[0064] To accurately retrieve data from the local memory buffer, the system needs to calculate the starting position (top, left) of the next frame image within the buffer, as well as its width (W) and height (H). The calculation of top and left is typically based on the position of the current visible area relative to the overall interface, combined with the storage layout of the image data in the local memory buffer. For example, if the buffer stores a large image, top and left will be the pixel coordinates of the area within that large image that needs to be rendered. W and H are usually matched to the device's screen resolution or the target rendering size, or slightly larger than the screen size to provide a smooth scrolling experience. In some cases, W and H can also be dynamically adjusted based on factors such as rendering performance and network bandwidth to optimize data transfer and rendering efficiency. The top and left parameters explicitly specify the precise starting point from which image data is read from the local memory buffer. If the local memory buffer is a contiguous block of image data, top and left can be converted to linear byte offsets within that block, allowing direct access to the target pixel data. Alternatively, if the buffer uses block storage (e.g., dividing a large image into multiple small tiles), top and left are used to locate the specific image tile containing the required data and calculate the offset within that tile. This precise starting position positioning ensures correct alignment and extraction of image data, avoiding unnecessary calculations and data redundancy. Meanwhile, the W and H parameters define the size of the image data area to be read from the local memory buffer. These dimensions typically correspond to the screen width and height of the current device to ensure that the rendered image fully covers the visible area. In some optimized scenarios, W and H may be slightly larger than the screen size to provide additional buffer space, thereby reducing edge whitespace or loading latency during rapid scrolling. For example, an area slightly wider and taller than the screen can be pre-fetched. Furthermore, W and H can also be dynamically adjusted based on the current device's performance, network conditions, or user settings to achieve a balance between display quality and rendering efficiency.

[0065] Through the above technical solution, this application can significantly improve the display smoothness of the remote interface of the cloud phone during swiping operations. When the user performs a swiping operation, the local App can accurately capture the user's interaction intent by listening to the user's touch events and swiping distance in real time. Based on this real-time interaction information, the system can efficiently calculate the precise position of the current visible area in the entire interface, and further determine the specific reading start position (top, left) and reading range (W, H) of the next frame image in the local memory buffer. This precise calculation and positioning mechanism allows the local App to quickly and accurately extract the image data required for the current visible area directly from the extended area image data pre-stored in the local memory buffer, and render it through the graphics rendering library. Compared with the traditional on-demand request method, this solution avoids sending a new request to the remote server every time you swipe, thereby greatly reducing network latency and server load. In particular, by combining the above-mentioned extended area prefetching mechanism and dynamic buffer refresh and update mechanism, this solution can make full use of the locally cached extended area data during continuous user swiping, and perform precise data extraction and rendering on this basis. Even during rapid scrolling, because the data is already local, the system does not need to wait for a response from a remote server, thus eliminating stuttering caused by network transmission, image encoding / decoding, and other processes. This optimization not only improves the user's interactive experience in dynamic scenes, making interface scrolling smoother and response faster, but also reduces the consumption of computing resources on both the client and server sides, making it particularly suitable for low-power devices and environments with poor network conditions.

[0066] Furthermore, S5.2 also includes: the refresh operation of the local memory buffer is usually performed asynchronously, that is, the display content on the screen is updated immediately after the new image data is loaded.

[0067] Specifically, refreshing the local memory buffer is typically asynchronous, meaning that data loading and buffer update tasks do not block the main thread or user interface thread. This allows these tasks to run independently in the background, while the main thread continues processing user input, rendering existing interfaces, or performing other tasks. There are various ways to implement asynchronous operations. For example, multithreading can be used to distribute data loading and buffer update tasks to one or more worker threads, with the main thread receiving notifications of task completion via message queues or callback mechanisms. Alternatively, coroutines or event loops provided by modern programming languages ​​and frameworks can be used to achieve non-blocking execution of tasks without creating new threads, thus releasing CPU resources while waiting for I / O operations (such as network data reception). Furthermore, updating the screen display immediately after new image data is loaded means that once the asynchronous loading and refresh tasks are complete, and the new image data is ready and stored in the local memory buffer, the system quickly triggers a screen redraw operation to render the latest data in the buffer to the screen.

[0068] Through the above technical solution, this application effectively solves the problem of interface stuttering and response delay that may occur during the scrolling process of a cloud phone remote interface due to the synchronous execution of buffer refresh operations. Specifically, the refresh operation of the local memory buffer is designed to be asynchronous, allowing data loading and buffer update tasks to be executed independently in the background without blocking the main rendering thread. This means that even if network data transmission or image processing takes some time, the user interface can still remain responsive, continuing to process user touch events or render existing interface content, thereby avoiding interface freezing or stuttering. Once new image data is loaded and successfully updated to the local memory buffer, the system immediately triggers the update display of the screen content. This instant update mechanism ensures that users can quickly see the latest interface content, significantly reducing the delay between data readiness and visual presentation. Combined with the aforementioned local memory buffer rendering mechanism during scrolling, asynchronous refresh further enhances the smoothness of the user experience during rapid scrolling, making the interaction of the cloud phone remote interface smoother and more natural, and greatly improving user satisfaction.

[0069] In the above embodiments of this application, a method is proposed to extract image data directly from the local memory buffer for rendering during sliding operations in order to reduce network requests and improve smoothness. However, in its implementation, the system may not be able to accurately locate the correct memory address in the local memory buffer, resulting in image data extraction errors or rendering delays.

[0070] In response, this application further proposes that the system can locate the correct memory address in the local memory buffer, thereby extracting the corresponding image data and rendering the data onto the screen through a graphics rendering library.

[0071] The system's ability to locate the correct memory address in the local memory buffer means that after the local app receives encoded data returned from the remote server and stores it in the local memory buffer, when the user performs a swipe operation, the system can accurately locate the storage location of the image data corresponding to the new visible area in the local memory buffer based on the calculated relative position of the new visible area within the entire interface. Specifically, one implementation involves the system maintaining a metadata structure that records the mapping relationship between the logical coordinates of each image data block in the buffer and its actual memory address. When a specific area needs to be rendered, the system queries this metadata structure and combines it with the coordinate information of the current visible area to calculate the starting address and length of the target image data in the buffer. Extracting the corresponding image data means that after the system successfully locates the correct memory address in the local memory buffer, it can read the image data block pointed to by that address from memory for subsequent rendering operations. Finally, rendering the data onto the screen using a graphics rendering library means that the local app uses a dedicated graphics processing software library to complete the final display of the image data. The graphics rendering library can efficiently process image data and convert it into pixels visible on the screen. For example, a local app can call low-level graphics APIs such as OpenGL ES, Vulkan, or DirectX to upload the extracted image data as a texture to the graphics processor (GPU), and then process and draw it through the shader program.

[0072] Through the above technical solution, this application effectively solves the problem that during swiping operations, the system may be unable to accurately locate the correct memory address in the local memory buffer, leading to image data extraction errors or rendering delays. Specifically, the system can accurately locate the correct memory address in the local memory buffer, ensuring that during user swiping, the local App can accurately obtain image data corresponding to the currently visible area, avoiding data misalignment or display abnormalities caused by address errors. Based on this, the corresponding image data is extracted efficiently, reducing data preparation time. Finally, the data is rendered onto the screen through a graphics rendering library, utilizing its optimized graphics processing capabilities to ensure that the image data is presented to the user quickly and smoothly. These synergistic mechanisms enable the local App to stably and efficiently retrieve and display images from the local memory buffer during user swiping, significantly improving the display smoothness and user interaction experience of the cloud phone remote interface in dynamic scenarios, while reducing dependence on network bandwidth and remote server computing resources.

[0073] For example: Suppose user A is remotely accessing a news reading application via a cloud phone. The application's interface is quite long, requiring frequent swiping to view the entire content.

[0074] This technical solution optimizes this process through the following steps: S1, Extended Region Prefetching Mechanism: When user A launches a news reading application or requests a new news list interface on a local device, the local app sends an HTTP request to a remote server. Unlike existing technologies that only include the width and height of the currently visible area, this request includes not only the width W and height H of the current visible area but also an extended height. This extended height covers a range extending upwards and downwards by 1.5 times the screen height along the swipe direction, with the center line of the current visible area as the reference point, resulting in a total prefetched height of 3H. Simultaneously, the local app uses a precise location information calculation mechanism—for example, obtaining the coordinates of the current touch point on the screen through a unique identifier on the interface—combined with interface layout information and screen resolution, to calculate the precise relative position of the current visible area within the entire application interface. This location information is then included in the HTTP request and sent to the remote server.

[0075] S2. Screenshot and encoding of the extended area on the remote server: After receiving an HTTP request from the local app, the remote server takes a screenshot of the current application interface based on the width (W) and height (H) of the currently visible area, the extended height (e.g., a total height of 3H), and the precise location information contained in the request. The server uses an image processing library to capture the required area, ensuring that the screenshot covers the current visible area plus an extended height of 1.5H above and below it; that is, the screenshot's width is W and its height is 3H. Before taking the screenshot, the remote server also converts the parameter values ​​in the request header into a data format suitable for its internal processing. After capturing the image, the server encodes it, converting it into an encoded data stream suitable for network transmission.

[0076] S3, Local App Memory Buffer Management: After receiving the encoded data stream returned by the remote server, the local app stores it in a local memory buffer. The local app creates a local memory buffer in memory for storing and retrieving image data. The size of this buffer is calculated based on the width W and height 3H of the image data to ensure that the prefetched extended area image data can be completely stored.

[0077] S4. Local memory buffer rendering during scrolling: When user A begins swiping on the news reading interface, the local app continuously monitors the user's touch events and swipe distance. Based on this information, the local app can calculate in real time the relative position of the new visible area within the entire interface, as well as the top, left, W, and H parameters of the next frame image. Top and left represent the starting position for reading data from the local memory buffer, while W and H represent the width and height of the data area to be read. At this point, the local app does not need to send new requests to the remote server; instead, it directly locates the correct memory address in the local memory buffer based on the calculated position information, extracts the corresponding image data, and renders the data onto the screen using the graphics rendering library.

[0078] Unlike existing technologies that require requesting new frame data from the server for each swipe, this solution directly retrieves data from the local memory buffer for rendering when the user swipes within the prefetched range. This significantly reduces network transmission and server processing latency, thereby improving the smoothness of the interface display.

[0079] S5, Dynamic Buffer Refresh and Update: To further optimize the user experience and adapt to more complex swiping scenarios, this solution also includes a dynamic buffer refresh and update mechanism: S5.1 When the user finishes swiping and releases their finger: The local app sends a new request to the remote server based on the user's latest position (POS) when the swipe stops, retrieving data for a new extended height region centered on that new position. After the remote server returns the new data, the local app asynchronously updates its local memory buffer. This asynchronous refresh ensures that the buffer is updated promptly after the user stops swiping, preparing for the next swipe without blocking the current screen display.

[0080] S5.2 During continuous swiping by the user: The local app continuously monitors whether the user's swipe exceeds the prefetch range of the current local memory buffer (i.e., a height of 3H). Once it detects that the swipe has exceeded the current buffer range, the local app immediately sends a new request to the remote server to obtain new extended height region data. After the remote server returns the new extended height region data, the local app promptly updates the local memory buffer. During the swipe operation, the local app listens to the user's touch events and swipe distance to determine how to extract data from the local memory buffer for rendering. Based on this information, the system calculates the position of the current visible area relative to the overall interface, as well as the top, left, W, and H of the next frame image. This ensures that even during rapid or large swipes, updated content can be retrieved from the local memory buffer for rendering, avoiding stuttering caused by insufficient buffer data.

[0081] Through the above mechanism, this technical solution can effectively reduce the frequency of requests to the remote server, reduce network transmission latency, and reduce the computing load on the server and client when the user performs a swipe operation, thereby significantly improving the smoothness of the remote interface display of the cloud phone, especially in dynamic swipe scenarios, where the user experience is significantly improved.

[0082] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for optimizing the smoothness of remote interface display on cloud phones, characterized in that, Includes the following steps: S1. Extended area prefetching mechanism: When the local App starts or requests a new interface, the HTTP request sent to the remote server contains not only the width and height of the current visible area, but also an extended height, along with the relative position information of the current visible area. S2. Remote server-side extended area screenshot and encoding: After receiving the HTTP request, the remote server performs a screenshot operation on the current application interface based on the width and height information of the visible area, the extended height information and the position information in the request, ensuring that the screenshot range covers the extended height area, and encodes the captured image into an encoded data stream suitable for network transmission. S3. Local App Memory Buffer Management: After receiving encoded data returned by the remote server, the local App stores it in the local memory buffer. S4. Local memory buffer rendering during swiping: When the user performs a swipe operation, the local App listens to touch events and swipe distance to calculate the relative position of the new visible area in the entire interface, and extracts the corresponding image data directly from the local memory buffer based on this information, and renders it through the graphics rendering library without sending a new request to the remote server.

2. The method for optimizing the smoothness of remote interface display on cloud phones according to claim 1, characterized in that: It also includes the following steps: S5, Dynamic Buffer Refresh and Update: S5.1 When the user finishes swiping and releases their finger, the local app will send a new request to the remote server based on the latest location information POS to obtain new extended height area data and asynchronously update the contents of the local memory buffer. S5.2 During continuous swiping by the user, the local App monitors in real time whether the swipe exceeds the current buffer range. Once the swipe exceeds the range, it immediately sends a new request to the remote server to obtain new extended height area data. After the remote server returns the new extended height area data, it updates the local memory buffer in a timely manner to ensure that subsequent swipe operations can obtain the updated content from the local memory buffer for rendering.

3. The method for optimizing the smoothness of remote interface display on cloud phones according to claim 1, characterized in that: The extended height covers a range that extends upwards and downwards by N times the screen height, based on the center line of the current visible area, along the sliding direction, where N > 1.

4. The method for optimizing the smoothness of the remote interface display of a cloud phone according to any one of claims 1-3, characterized in that: Step S1 includes: S1.1, Precise location information calculation mechanism: The local app obtains the current finger position on the screen through a unique identifier on the interface. When a touch event occurs, the application calculates the relative position of the current visible area in the entire application interface based on the screen coordinates of the touch point, combined with the interface layout information and screen resolution. It then maps the position of the touch point to the overall layout of the interface to ensure that the position information can be accurately obtained and reflect the interface position corresponding to the touch event.

5. The method for optimizing the smoothness of the remote interface display of cloud phones according to any one of claims 1-3, characterized in that: S2 includes: a remote server capturing the area required by the current application interface through an image processing library, wherein the width of the screenshot is W, the height of the screenshot is 3H, and 3H is 1.5H extended upwards and downwards from the center line of the current visible area along the sliding direction.

6. The method for optimizing the smoothness of remote interface display on cloud phones according to claim 2, characterized in that: S2 further includes: conversion of the internal data format of the request parameters on the remote server side, that is, the remote server converts the parameter values ​​in the request header into an internal data format suitable for processing by the remote server.

7. The method for optimizing the smoothness of the remote interface display of a cloud phone according to any one of claims 5, characterized in that: The local App creates a local memory buffer in memory for storing and retrieving image data. The size of the local memory buffer is calculated based on the width W and height 3H of the image data.

8. The method for optimizing the smoothness of remote interface display in cloud phones according to claim 2, characterized in that: In S5.2, local buffer rendering during sliding includes: During the swipe operation, the local app listens to the user's touch events and swipe distance to determine how to extract data from the local memory buffer for rendering. Based on this information, the system calculates the position information of the current visible area relative to the overall interface, as well as the top, left, W, and H of the next frame image. Here, top and left represent the starting position for reading data from the local memory buffer, while W and H represent the width and height of the data area to be read.

9. The method for optimizing the smoothness of remote interface display in cloud phones according to claim 8, characterized in that: S5.2 also includes: the refresh operation of the local memory buffer is usually performed asynchronously, that is, the display content on the screen is updated immediately after the new image data is loaded.

10. The method for optimizing the smoothness of remote interface display in cloud phones according to claim 1, characterized in that: S4 includes: the system is able to locate the correct memory address in the local memory buffer, thereby extracting the corresponding image data and rendering the data onto the screen through the graphics rendering library.

Images