A cloud mobile phone-based remote real-time streaming screenshot method
By creating a memory file device and a virtual driver module on the cloud phone host, the raw pixel data stream is directly transmitted, solving the problems of efficiency and real-time performance in cloud phone screenshots and realizing an efficient and low-latency screenshot method.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUNAN XIAOSUAN TECH INFORMATION CO LTD
- Filing Date
- 2026-02-09
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies for taking screenshots in cloud phones have poor efficiency and real-time performance, especially when capturing local changes on the screen at low latency and high frequency or capturing instantaneous images accurately on demand, resulting in redundant operations and resource consumption issues.
Create a memory file device on the host machine of the cloud phone instance and configure the virtual file device driver module. Trigger the screenshot module through read operations to capture raw pixel data and transmit it directly to the remote client in the form of byte stream, avoiding the generation of intermediate files and network transmission overhead.
It improves screenshot efficiency, reduces resource consumption and latency, and achieves a simpler architecture and flexible encoding format selection, meeting the needs of high-frequency and low-latency screen capture.
Smart Images
Figure CN122111552A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of data processing technology, and in particular to a remote real-time streaming screenshot method based on cloud phones. Background Technology
[0002] Currently, in order to acquire screen images from a cloud phone and transmit them to a remote client, the following solutions are commonly used in existing technologies: 1. Scheduled Screenshot and Public Storage Solution: Within the cloud phone instance, the system's screenshot function is invoked at fixed time intervals (e.g., several times per second) to generate screenshot image files. These image files are then uploaded to a public file storage service (e.g., an object storage server). When a client needs to obtain the latest screen view, it downloads the corresponding image file from this public storage service. This solution generates a large amount of redundant screenshots and network transmissions, and relies on external storage services, resulting in latency and additional costs.
[0003] 2. Internal Generation and Temporary Transmission Solution: When a client requests a screen image, the cloud phone instance immediately invokes the screenshot function to generate an image in a local temporary storage area (such as memory or a temporary file). Then, the entire image file is sent to the client via a specific network channel (such as a custom TCP / UDP connection). This solution generates images on demand, avoiding redundant screenshots. However, each request involves the complete generation, storage, and transmission of the image file, resulting in processing and transmission overhead when handling high-frequency, small-change screen requests.
[0004] Therefore, existing technical solutions suffer from the following problems: they rely on external public storage services, resulting in complex and costly architectures; or they require repeatedly generating complete image files within the cloud phone, consuming computing and memory resources and generating unnecessary overhead for complete file transfers. Especially in scenarios requiring low-latency, high-frequency acquisition of local screen changes or precise on-demand capture of a specific moment's image, existing solutions are inefficient.
[0005] It is evident that there is an urgent need for a remote, real-time streaming screen capture method based on cloud phones that offers high efficiency and real-time performance. Summary of the Invention
[0006] In view of this, the present disclosure provides a remote real-time streaming screen capture method based on cloud phones, which at least partially solves the problems of poor screenshot efficiency and real-time performance in the prior art.
[0007] This disclosure provides a method for remote real-time streaming screen capture based on a cloud phone, including: Step 1: Create a memory file device that is logically bound to the cloud phone instance on the host machine where the cloud phone instance is located, and configure the virtual file device driver module inside the cloud phone instance; Step 2: When the agent program on the host machine receives a screen data request from the remote client, it performs a read operation on the memory file device. Step 3: When the read operation is intercepted by the virtual file device driver module, the virtual file device driver module controls the screenshot module in the cloud phone instance to capture the raw pixel data of the current frame screen. Step 4: The virtual file device driver module writes the raw pixel data directly into the buffer of the memory file device in the form of a byte stream; Step 5: The agent reads the byte stream from the memory file device, encodes the byte stream, and transmits it to the remote client over the network.
[0008] According to one specific implementation of this disclosure, the memory file device is a specific file based on tmpfs or ramfs, or a character device created by a kernel module.
[0009] According to a specific implementation of an embodiment of this disclosure, step 3 specifically includes: After intercepting the read operation, the virtual file device driver module notifies the screenshot module to call the operating system's underlying graphics interface to obtain the raw pixel data of the current frame of the screen.
[0010] According to one specific implementation of this disclosure, the original pixel data is in the format of RGB or YUV buffer data.
[0011] According to a specific implementation of an embodiment of this disclosure, step 5 specifically includes: The proxy program encodes the byte stream into a preset format and transmits it to the remote client over the network according to the needs of the remote client or network conditions. The preset format includes at least one of H.264 frames, JPEG images, or WebP streams.
[0012] According to a specific implementation of an embodiment of this disclosure, the step of transmitting data over a network to a remote client includes: The proxy program transmits the encoded data stream to the remote client in real time via the WebSocket or QUIC protocol.
[0013] According to one specific implementation of the present disclosure, a read operation corresponds to triggering a screen capture and acquiring a frame or a data stream.
[0014] The remote real-time streaming screen capture scheme based on cloud phones in this embodiment includes: Step 1, creating a memory file device logically bound to the cloud phone instance on the host machine where the cloud phone instance is located, and configuring a virtual file device driver module inside the cloud phone instance; Step 2, when the agent program on the host machine receives a screen data request from a remote client, performing a read operation on the memory file device; Step 3, when the read operation is intercepted by the virtual file device driver module, the virtual file device driver module controls the screen capture module in the cloud phone instance to capture the raw pixel data of the current frame screen; Step 4, the virtual file device driver module directly writes the raw pixel data into the buffer of the memory file device in the form of a byte stream; Step 5, the agent program streams the byte stream from the memory file device, encodes the byte stream, and transmits it to the remote client over the network.
[0015] The beneficial effects of the embodiments disclosed herein are as follows: 1. Lower resource consumption: It eliminates the step of generating complete image files (encoding and writing to storage) inside the cloud phone, reducing the CPU and I / O consumption inside the cloud phone instance, especially under high frame rate requests; 2. Higher transmission efficiency: Data is transmitted directly to the host agent through memory in the form of raw pixel stream, avoiding the overhead of transmitting image files as a whole within the internal network or between processes, and achieving a lower latency data path; 3. Simplified architecture and more thorough on-demand capability: Completely eliminates dependence on any form of intermediate file (whether remote object storage or local temporary files), achieving the "shortest path" from screen buffer to network stream. Screenshot actions are strictly driven by client requests, with no redundant operations; 4. Enhanced flexibility: The host agent program can flexibly select the encoding format and parameters of the raw pixel stream according to network conditions and client capabilities, without requiring changes to the cloud phone instance, thus achieving better separation of the control plane and the data plane. Attached Figure Description
[0016] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 A flowchart illustrating a remote real-time streaming screen capture method based on a cloud phone, provided in an embodiment of this disclosure; Figure 2 The system architecture diagram corresponding to a remote real-time streaming screen capture method based on a cloud phone provided in this embodiment of the disclosure; Figure 3 This is a core architecture interaction diagram of a remote real-time streaming screen capture method based on a cloud phone, provided in an embodiment of this disclosure. Detailed Implementation
[0018] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.
[0019] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.
[0020] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.
[0021] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The illustrations only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0022] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.
[0023] This disclosure provides a remote real-time streaming screenshot method based on cloud phones, which can be applied to the remote screenshot process in cloud phone scenarios.
[0024] See Figure 1 This is a flowchart illustrating a remote real-time streaming screen capture method based on a cloud phone, provided in an embodiment of this disclosure. Figure 1 , Figure 2 and Figure 3 As shown, the method mainly includes the following steps: Step 1: Create a memory file device that is logically bound to the cloud phone instance on the host machine where the cloud phone instance is located, and configure the virtual file device driver module inside the cloud phone instance; In practice, the creation and association process of the memory file device is as follows: When the cloud phone instance starts, a management service or the cloud phone instance's own driver module creates a special memory file device (e.g., a specific file based on tmpfs or ramfs, or a character device created through a kernel module) on its host machine. This device is logically bound to the cloud phone instance. Simultaneously, a "virtual file device" module is installed and runs within the cloud phone instance.
[0025] Step 2: When the agent program on the host machine receives a screen data request from the remote client, it performs a read operation on the memory file device. In practice, the access triggering and data capture process is as follows: The "agent program" on the host machine acts as the receiving end of the client request. When the agent program receives a screen data request from the remote client, it no longer sends complex screenshot instructions to the cloud phone, but directly "opens" and "reads" the aforementioned associated memory file device.
[0026] Step 3: When the read operation is intercepted by the virtual file device driver module, the virtual file device driver module controls the screenshot module in the cloud phone instance to capture the raw pixel data of the current frame screen. In practice, when the read operation is intercepted by the "virtual file device driver" module inside the cloud phone instance, the driver module immediately notifies the "screenshot module" within the cloud phone operating system to trigger a screen content capture. The screenshot module calls the underlying interface of the operating system (such as the SurfaceFlinger related interface of Android) to obtain the raw pixel data of the current frame of the screen (such as a buffer in RGB or YUV format).
[0027] Step 4: The virtual file device driver module writes the raw pixel data directly into the buffer of the memory file device in the form of a byte stream; In practice, the process of streaming data filling and transmission is as follows: The acquired raw pixel data is not encoded into image formats such as JPEG / PNG within the cloud phone. Instead, the "virtual file device driver" module writes this raw data directly into the buffer of the corresponding "memory file device" on the host machine in the form of a continuous byte stream. The "agent program" on the host machine receives these continuously written byte streams in real time during its read system call.
[0028] Step 5: The agent reads the byte stream from the memory file device, encodes the byte stream, and transmits it to the remote client over the network.
[0029] In practice, the data encoding and forwarding process is as follows: The proxy program encodes the raw pixel stream read from the memory file device according to the client's requirements (e.g., encoding as an H.264 frame, JPEG image, or WebP stream). This encoding process can be completed within the proxy program using host machine resources. The encoded data stream is then transmitted in real-time to the remote client via a network (such as WebSocket, QUIC, etc.). The client can then decode and display the data.
[0030] Furthermore, the entire data flow chain is "on-demand fetching" and "synchronous." The agent's read operation is the driving force behind the entire process. One read call corresponds to triggering a screen capture and acquiring one frame (or segment) of data stream. By controlling the frequency of reads, precise control over the screenshot frame rate can be achieved. When there are no client requests, there are no screenshot operations or resource consumption.
[0031] The remote real-time streaming screen capture method based on cloud phones provided in this embodiment creates a virtual "memory file device" at the host level of the cloud phone instance and exposes it to the agent program on the host machine. When the agent program (representing the client) accesses the device file, it triggers the cloud phone to perform an on-demand screen content capture and directly and continuously writes the captured raw image data to the memory file device. The agent program then reads the data from the device file in a streaming manner and forwards it to the client. There is no need to generate intermediate image files or use public storage, which improves the efficiency and real-time performance of screenshots and reduces transmission overhead.
[0032] It should be understood that the various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof.
[0033] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.
Claims
1. A method for remote real-time streaming screen capture based on cloud phones, characterized in that, include: Step 1: Create a memory file device that is logically bound to the cloud phone instance on the host machine where the cloud phone instance is located, and configure the virtual file device driver module inside the cloud phone instance; Step 2: When the agent program on the host machine receives a screen data request from the remote client, it performs a read operation on the memory file device. Step 3: When the read operation is intercepted by the virtual file device driver module, the virtual file device driver module controls the screenshot module in the cloud phone instance to capture the raw pixel data of the current frame screen. Step 4: The virtual file device driver module writes the raw pixel data directly into the buffer of the memory file device in the form of a byte stream; Step 5: The agent reads the byte stream from the memory file device, encodes the byte stream, and transmits it to the remote client over the network.
2. The method according to claim 1, characterized in that, The memory file device is a specific file based on tmpfs or ramfs, or a character device created by a kernel module.
3. The method according to claim 1, characterized in that, Step 3 specifically includes: After intercepting the read operation, the virtual file device driver module notifies the screenshot module to call the operating system's underlying graphics interface to obtain the raw pixel data of the current frame of the screen.
4. The method according to claim 3, characterized in that, The original pixel data is in the format of RGB or YUV buffer data.
5. The method according to claim 1, characterized in that, Step 5 specifically includes: The agent program encodes the byte stream into a preset format and transmits it to the remote client over the network according to the needs of the remote client or network conditions. The preset format includes at least one of H.264 frames, JPEG images, or WebP streams.
6. The method according to claim 1, characterized in that, The step of transmitting the data to the remote client via the network includes: The proxy program transmits the encoded data stream to the remote client in real time via the WebSocket or QUIC protocol.
7. The method according to claim 1, characterized in that, Each read operation corresponds to triggering a screen capture and acquiring a frame or a data stream.