Method for transmitting screen image, and electronic device and computer-readable medium

Abstract

Provided in the embodiments of the present application are a method for transmitting a screen image, and an electronic device and a computer-readable medium. The method comprises: a first terminal acquiring a first image frame and a second image frame, wherein the first image frame and the second image frame are adjacent image frames in screen content which is displayed by means of the first terminal, and the display time of the first image frame is earlier than the display time of the second image frame; determining that an image change region in which the second image frame changes relative to the first image frame satisfies a small-region change condition; and sending to a second terminal first image data in the second image frame, which first image data corresponds to the image change region.

Description

Screen image transmission method, electronic device and computer readable medium

[0001] Cross-references

[0002] The present invention claims priority to a Chinese patent application filed with the Patent Office of China on June 26, 2023, with application number 2023107622692 and invention name “Screen image transmission method, electronic device and computer-readable medium”. The entire contents of the application are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of video coding and decoding technology, and in particular to a screen image transmission method, electronic device and computer-readable medium. Background Art

[0004] Transmitting computer screen images (also known as screen content) is essential for applications such as video conferencing, remote online learning, live streaming, and remote computer monitoring. For example, related technologies employ image transmission schemes that capture screen data as video data and encode it for transmission, or encode the screen as a static image for transmission.

[0005] However, in actual applications, the aforementioned technical solutions adopted in related technologies still have problems such as large amount of transmitted data and high resource occupancy rate.

[0006] Summary of the Invention

[0007] The purpose of the embodiments of the present application is to provide a screen image transmission method, electronic device and computer-readable medium, which can solve the problems existing in related technologies such as large transmission data volume and high resource occupancy rate.

[0008] To solve the above technical problems, the embodiments of the present application are implemented through the following aspects.

[0009] In a first aspect, an embodiment of the present application provides a method for transmitting a screen image, which is applied to a first terminal, and the method includes: acquiring a first frame image and a second frame image, wherein the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image; determining that an image change area of ​​the second frame image that has changed relative to the first frame image satisfies a small area change condition; and sending first image data corresponding to the image change area in the second frame image to the second terminal.

[0010] In the second aspect, an embodiment of the present application provides a method for transmitting a screen image, which is applied to a second terminal, and the method includes: receiving first image data sent by a first terminal; determining a target position of the first image data in a target frame image displayed by the second terminal; copying the first image data to the target position in the target frame image and displaying it; wherein, the first image data is determined based on an image change area in which the second frame image changes relative to the first frame image, the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image.

[0011] In a third aspect, an embodiment of the present application provides an electronic device comprising: a memory, a processor, and computer-executable instructions stored on the memory and executable on the processor, wherein the computer-executable instructions, when executed by the processor, implement the steps of executing the method described in the first aspect or the second aspect.

[0012] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0014] FIG1 shows one of the flow charts of the screen image transmission method provided in an embodiment of the present application.

[0015] FIG2 shows a second flow chart of the screen image transmission method provided in an embodiment of the present application.

[0016] FIG3 shows a third flow chart of the screen image transmission method provided in an embodiment of the present application.

[0017] FIG4 shows one of the structural schematic diagrams of the screen image transmission device provided in an embodiment of the present application.

[0018] FIG5 shows a second structural schematic diagram of the screen image transmission device provided in an embodiment of the present application.

[0019] FIG6 is a schematic diagram of the hardware structure of an electronic device that executes the screen image transmission method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0021] For the transmission of screen images (or screen content, desktop images, screen pictures, etc.) in cloud desktops, considering factors such as the encoding / decoding capabilities of the terminal and network bandwidth, if each captured frame is encoded, it will inevitably lead to problems such as large transmission data volume and high resource utilization. In this regard, the present application provides a screen image transmission method, electronic device, and computer-readable medium, which can add a small area change scene judgment for continuous small area changes after capturing each frame of the image. If the conditions of the small area change scene are met, the captured current frame image is not discarded, but the entire picture is not encoded. Instead, the image is transmitted based on the change area of ​​the previous and next frames. This can significantly reduce the data transmission volume and resource utilization.

[0022] In some embodiments, in addition to the aforementioned cloud desktop scenario, the screen image transmission solution provided by the embodiments of the present application is also applicable to real-time encoding systems containing screen content, such as cloud desktops with interactive operations, video conferencing, live webcasts, computer remote monitoring, screen sharing, and other scenarios that have high real-time requirements and certain requirements for compression rate. It is particularly suitable for any type of small area or small area graphics rendering scenario, such as some drawing software with self-drawing mouse features (such as AutoCAD, etc.). The self-drawing mouse can be understood as when the program is enabled, the application will replace the cursor shape configured by the operating system with the cursor shape used by itself.

[0023] Based on this, the technical solution provided by this application is introduced below in combination with the accompanying drawings and embodiments, as follows.

[0024] FIG1 illustrates a flow chart of a method 100 for transmitting a screen image according to an embodiment of the present application. The method 100 may be executed by a first terminal (e.g., a mobile phone, a computer, a television, etc.), for example, by software and / or hardware installed in the first terminal. As shown in FIG1 , the method 100 may include the following steps.

[0025] S110, acquiring a first frame image and a second frame image.

[0026] The first terminal (also referred to as an encoding terminal) may utilize, but is limited to, a desktop software development kit (SDK) to acquire or capture the first and second frames of image data. In this embodiment, the first and second frames of image data are adjacent frames of image data within the screen content displayed by the first terminal, and the first frame of image data is displayed earlier than the second frame of image data. For example, the second frame of image data is the current frame of image data acquired by the first terminal, and the first frame of image data is the previous frame of image data.

[0027] Among them, when the first terminal obtains the first frame image and the second frame image, if a virtual machine is configured on the first terminal, then the first terminal can respond to input based on an interactive device (such as a microphone, keyboard, mouse, wearable device, etc.) through the virtual machine, adjust the screen content displayed by the first terminal according to the input, and then obtain relevant information of adjacent frame images of the screen content displayed by the first terminal through the virtual machine. The relevant information of the adjacent frame images includes but is not limited to the first frame image and the second frame image.

[0028] S120: Determine whether an image change region of the second frame image that has changed relative to the first frame image satisfies a small region change condition.

[0029] The small area change condition can be configured according to actual needs. For example, in this embodiment, the small area change condition can include at least one of the following (a)-(b).

[0030] (a) The proportion of the image change area in the first frame image or the second frame image is less than a first threshold.

[0031] The first threshold can be designed according to actual needs, such as 1%, 2%, etc. Based on this, assuming that the first threshold is 1%, if the first terminal determines that the image change area of ​​the second frame image relative to the first frame image accounts for less than 1% of the first frame image or the second frame image, it can be determined that the image change area of ​​the second frame image relative to the first frame image meets the small area change condition; otherwise, it does not meet the small area change condition.

[0032] (b) The dispersion of the image change region in the first frame image or the second frame image is less than a second threshold value. There are multiple ways to determine the dispersion. For example, assuming that the desktop image is divided into five equal parts horizontally and the second threshold value is 25%, then if the image change region is dispersed in a single desktop image block after the division (i.e., the dispersion is 20%), it can be determined that the dispersion of the image change region in the first frame image or the second frame image is less than the second threshold value. If the image change region is dispersed in each desktop image block after the division (i.e., the dispersion is 100%), it can be determined that the dispersion of the image change region in the first frame image or the second frame image is greater than the second threshold value.

[0033] Among them, when determining the aforementioned image change area, the granularity can be based on the macroblocks included in the screen image. For example, the first terminal can determine whether the image change area meets the small area change condition based on information such as the macroblock position and / or the number of macroblocks in the image change area, and the number of macroblocks included in the image change area accounts for the proportion of the number of macroblocks included in the desktop image (i.e., the first frame image, the second frame image), etc.

[0034] For example, assuming that the first frame image includes 100 macroblocks, the image change area includes 1 macroblock, and the first threshold is 2%, then it can be determined that the dispersion (1%) of the image change area in the first frame image is less than the second threshold.

[0035] In one implementation, the shape of the aforementioned image change region can be of any type. That is, the technical solution provided by this application is applicable to any type of small area scene recognition, thereby improving the applicability and flexibility of the technical solution of this application.

[0036] The present application uses the aforementioned S120 to achieve the comparison of continuous frame images to determine the area where the image changes, which can avoid the problem of recognizing the mouse shape drawn by the application as screen image data, effectively improve the accuracy of screen image recognition, and avoid the problem of more obvious lag and larger delay caused by the refresh rate of the self-drawn mouse in the screen being consistent with the refresh rate of the screen image due to misrecognition.

[0037] S130: Send the first image data corresponding to the image change area in the second frame image to the second terminal.

[0038] There are many ways for the first terminal to send the first image data. For example, the first terminal can send the first image data to the second terminal, or send the first image data to the second terminal through a virtual machine based on a desktop protocol, etc., which are not limited here.

[0039] Correspondingly, when the second terminal (which can be called the decoding end) receives the first image data sent by the first terminal, it can determine the target position of the first image data in the target frame image displayed by the second terminal (corresponding to the second frame image on the first terminal side), and copy the first image data to the target position in the target frame image and display it, that is, overwrite the first image data to the corresponding target position of the target frame image (or overwrite the corresponding position of the current target frame image cache, etc.) to achieve refresh or update of the target frame image.

[0040] In this embodiment, during the screen image transmission process, the first terminal obtains the first frame image and the second frame image, and when the image change area of ​​the second frame image that has changed relative to the first frame image meets the small area change condition, the first image data corresponding to the image change area in the second frame image is sent to the second terminal. Therefore, by only transmitting the changed image data, the amount of transmitted data and resource occupancy can be greatly reduced, and the transmission efficiency of the image data and the refresh efficiency of the screen image can be improved.

[0041] In this case, in one possible implementation method, considering that the shallow compression method itself can be supported by desktop protocols such as SPICE, and only the current frame image information is required, therefore, when the first terminal sends the first image data corresponding to the image change area in the second frame image to the second terminal, it can also use shallow compression to compress the first image data. The implementation process is shown in Figure 2, and the content is as follows.

[0042] S210: Extract first image data corresponding to the image change area from the second frame image.

[0043] In an optional implementation, the first terminal may extract the first image data (e.g., RGB data) corresponding to the image change region from the second frame of image in various ways. For example, assuming that the desktop image (e.g., the first frame of image and the second frame of image) includes multiple macroblocks, and each macroblock has corresponding index information, the first terminal may extract the first image data corresponding to the image change region from the second frame of image based on the index information of the macroblocks included in the image change region, thereby separating the first image data from the desktop image data.

[0044] S220: Encode the first image data according to a shallow compression algorithm to obtain second image data.

[0045] The shallow compression algorithm may include, but is not limited to, a JPEG image compression algorithm and / or a quic graphic compression algorithm.

[0046] S230: Send the second image data to the second terminal.

[0047] Correspondingly, when the second terminal receives the second image data sent by the first terminal, it can decode the second image data according to the corresponding shallow compression algorithm to obtain the first image data, and copy the first image data to the target position in the target frame image and display it. Therefore, by utilizing the shallow compression algorithm to decompress image data only by relying on the characteristics of the current frame image information, the present application can effectively reduce the encoding / decoding delay during image data transmission, improve the encoding / decoding speed of small area change areas, and make the refresh rate of the change area significantly higher than the original desktop frame rate, thereby achieving the purpose of improving the smoothness of interactive scene use.

[0048] In one possible implementation, when the first terminal sends the first image data corresponding to the image change area in the second frame image to the second terminal, the second image data, the index information of the macroblocks included in the image change area, and the algorithm type data corresponding to the shallow compression algorithm can be compressed into a target data stream (also called an instruction stream) with a specific format (the data stream in the specific format is different from the video stream), and then the target data stream is added to the instruction queue to wait for being sent to the second terminal.

[0049] Correspondingly, when the second terminal receives the target data stream sent by the first terminal, if it is determined that the format of the target data stream is a specific format (i.e., an instruction stream format), then the target data stream can be parsed according to the data stream parsing method of the specific format to obtain the second image data, the type data of the compression algorithm, and the index information of the macroblock, and then the second image data is decompressed according to the shallow compression algorithm corresponding to the type data of the compression algorithm to obtain the first image data, and the target position of the first image data in the target frame image is determined according to the index information of the macroblock, so as to copy the first image data to the target position in the target frame image and display it. Therefore, the present application can further improve the encoding and decoding speed of small area change areas by encoding and decompressing image data based on the shallow compression algorithm combined with the instruction stream method, thereby achieving the purpose of improving the smoothness of interactive scene use.

[0050] In addition, when the first terminal transmits the second image data, the index information of the macroblocks included in the image change area, the algorithm type corresponding to the shallow compression algorithm and the second image data are compressed into a target data stream for transmission, so that the second terminal can timely and accurately obtain the shallow compression algorithm and the location information of the first image data, thereby ensuring the accuracy of the encoding and decoding results and the reliability of the screen display.

[0051] In this embodiment, in addition to using a shallow compression algorithm and instruction stream to transmit image data when determining that the image change area of ​​the second frame image relative to the first frame image meets the small area change condition, the first terminal may also use other methods other than the aforementioned shallow compression algorithm and instruction stream (such as deep compression algorithms (such as H.264 / H.265 / AV1, etc., which require reference to the relationship between previous and next frames to achieve a further compression rate of video encoding algorithms), video stream transmission methods, etc.) to transmit image data when the first terminal determines that the image change area of ​​the second frame image relative to the first frame image does not meet the small area change condition. In this way, the flexibility and reliability of the first terminal in image transmission are improved.

[0052] The aforementioned technical solution provided in the embodiment of the present application, on the one hand, combines the information of the desktop capture process itself, does not introduce new information collection and processing procedures, and can ensure the applicability of the solution of the present application; on the other hand, it also effectively combines the existing protocol's support for instruction streams and the characteristics of shallow compression, thereby improving the smoothness of desktop use in small area change scenarios.

[0053] FIG3 shows a flow chart of a method 300 for transmitting a screen image according to an embodiment of the present application. The method 300 may be executed by a second terminal, such as by software and / or hardware installed in the second terminal. As shown in FIG3 , the method 300 may include but is not limited to the following steps.

[0054] S310: Receive first image data sent by a first terminal.

[0055] S320: Determine a target position of the first image data in a target frame image displayed on the second terminal.

[0056] S330: Copy the first image data to a target position in the target frame image and display the copy.

[0057] Among them, the first image data is determined based on the image change area of ​​the second frame image relative to the first frame image, the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image.

[0058] In an optional implementation, the receiving the second image data sent by the first terminal includes: receiving the second image data sent by the first terminal; and decoding the second image data according to a shallow compression algorithm to obtain the first image data.

[0059] In an optional implementation, the receiving of the second image data sent by the first terminal also includes: receiving a target data stream sent by the first terminal; determining that the format of the target data stream is a specific format; parsing the target data stream according to the data stream parsing method of the specific format to obtain the second image data, type data of the compression algorithm, and index information of the macroblock, wherein the type data of the compression algorithm is used to determine the shallow compression algorithm, and the index information of the macroblock is used to determine the target position.

[0060] It can be understood that since the various implementation methods of the screen image transmission method provided in method embodiment 300 have the same or corresponding technical features as those of the aforementioned method embodiment 100, the implementation process of the various implementation methods of the screen image transmission method provided in method embodiment 300 can refer to the relevant description in the aforementioned method embodiment 100, and achieve the same or corresponding technical effects, which will not be repeated here.

[0061] Figure 4 shows a structural schematic diagram of a screen image transmission device 400 provided in an embodiment of the present application, wherein the device 400 includes: an image acquisition module 410, configured to acquire a first frame image and a second frame image, wherein the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image; a small area recognition module 420, configured to determine whether an image change area of ​​the second frame image that has changed relative to the first frame image satisfies a small area change condition; and a data sending module 430, configured to send first image data corresponding to the image change area in the second frame image to the second terminal.

[0062] In an optional implementation, the small area change condition includes at least one of the following: the proportion of the image change area in the first frame image or the second frame image is less than a first threshold; the dispersion of the image change area in the first frame image or the second frame image is less than a second threshold.

[0063] In an optional implementation, the data sending module 430 sends the first image data corresponding to the image change area in the second frame image to the second terminal, including: extracting the first image data corresponding to the image change area from the second frame image; encoding the first image data according to a shallow compression algorithm to obtain second image data; and sending the second image data to the second terminal.

[0064] In an optional implementation, the shallow compression algorithm includes a JPEG image compression algorithm and / or a quic graphic compression algorithm.

[0065] In an optional implementation, the data sending module 430 extracts the first image data corresponding to the image change area from the second frame image, including: extracting the first image data corresponding to the image change area from the second frame image based on the index information of the macroblocks included in the image change area; wherein the first frame image and the second frame image both include multiple macroblocks.

[0066] In an optional implementation, the data sending module 430 sends the first image data corresponding to the image change area in the second frame image to the second terminal, and also includes: compressing the second image data, the index information of the macroblocks included in the image change area, and the algorithm type data corresponding to the shallow compression algorithm into a target data stream with a specific format, and the data stream in a specific format is different from the video stream; adding the target data stream to the instruction queue to wait for sending to the second terminal.

[0067] In an optional implementation, the image acquisition module 410 acquires a first frame image and a second frame image, including: responding to input based on an interactive device through a virtual machine, and adjusting the screen content displayed by the first terminal according to the input; acquiring relevant information of adjacent frame images of the screen content displayed by the first terminal through the virtual machine, the relevant information of the adjacent frame images including the first frame image and the second frame image.

[0068] Figure 5 shows a structural schematic diagram of a screen image transmission device 500 provided in an embodiment of the present application, wherein the device 500 includes: a data receiving module 510, for receiving first image data sent by a first terminal; a position determination module 520, for determining a target position of the first image data in a target frame image displayed by the second terminal; and a display module 530, for copying the first image data to a target position in the target frame image and displaying the same; wherein the first image data is determined based on an image change area in which the second frame image changes relative to the first frame image, and the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image.

[0069] In an optional implementation, the data receiving module 510 receives the second image data sent by the first terminal, including: receiving the second image data sent by the first terminal; decoding the second image data according to a shallow compression algorithm to obtain the first image data.

[0070] In an optional implementation, the data receiving module 510 receives the second image data sent by the first terminal, and also includes: receiving the target data stream sent by the first terminal; determining that the format of the target data stream is a specific format; parsing the target data stream according to the data stream parsing method of the specific format to obtain the second image data, the type data of the compression algorithm, and the index information of the macroblock, wherein the type data of the compression algorithm is used to determine the shallow compression algorithm, and the index information of the macroblock is used to determine the target position.

[0071] The screen image transmission devices 400 and 500 provided in the embodiments of the present application can execute the various methods described in the above method embodiments and realize the functions and beneficial effects of the various methods described in the above method embodiments, which will not be repeated here.

[0072] FIG6 shows a schematic diagram of the hardware structure of an electronic device that implements an embodiment of the present application. Referring to the figure, at the hardware level, the electronic device includes a processor, and in some embodiments, may also include an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0073] The processor, network interface, and memory can be interconnected via an internal bus, such as an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. These buses can be classified as address buses, data buses, and control buses. For ease of illustration, the figure uses only one bidirectional arrow, but this does not imply that there is only one bus or only one type of bus.

[0074] The memory is used to store the program. In an optional implementation, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0075] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, logically forming a device for locating a specific user. The processor executes the program stored in the memory and is specifically configured to perform the methods disclosed in the embodiments shown in Figures 1-3 and to achieve the functions and benefits of the various methods described in the preceding method embodiments, which will not be further described here.

[0076] The methods disclosed in the embodiments shown in Figures 1 to 3 of the present application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor or software instructions. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0077] The electronic device can also execute the methods described in the above method embodiments and realize the functions and beneficial effects of the methods described in the above method embodiments, which will not be described in detail here.

[0078] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0079] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes the method disclosed in the embodiments shown in Figures 1 to 3 and realizes the functions and beneficial effects of the various methods described in the previous method embodiments, which will not be repeated here.

[0080] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0081] An embodiment of the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the following process is implemented: the method disclosed in the embodiment shown in Figures 1-2 and the functions and beneficial effects of the various methods described in the previous method embodiments are implemented, which will not be repeated here.

[0082] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0083] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. In an optional implementation, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0084] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0085] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0086] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

Claims

1. A method for transmitting a screen image, characterized in that: Applied to a first terminal, the method includes: Acquire a first frame image and a second frame image, where the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image; Determining that an image change region of the second frame image that changes relative to the first frame image satisfies a small region change condition; The first image data corresponding to the image change area in the second frame image is sent to the second terminal.

2. The method according to claim 1, characterized in that The small area change condition includes at least one of the following: The proportion of the image change area in the first frame image or the second frame image is less than a first threshold; The dispersion degree of the image change region in the first frame image or the second frame image is less than a second threshold.

3. The method according to claim 1 or 2, characterized in that The sending the first image data corresponding to the image change area in the second frame image to the second terminal includes: Extracting first image data corresponding to the image change area from the second frame image; Encoding the first image data according to a shallow compression algorithm to obtain second image data; The second image data is sent to the second terminal.

4. The method according to claim 3, characterized in that The shallow compression algorithm includes a JPEG image compression algorithm and / or a quic graphic compression algorithm.

5. The method according to claim 3, characterized in that The extracting the first image data corresponding to the image change area from the second frame image includes: Extracting first image data corresponding to the image change area from the second frame image according to index information of the macroblocks included in the image change area; Wherein, the first frame image and the second frame image both include a plurality of the macroblocks.

6. The method according to claim 3, characterized in that The sending the first image data corresponding to the image change area in the second frame image to the second terminal also includes: compressing the second image data, the index information of the macroblocks included in the image change area, and the algorithm type data corresponding to the shallow compression algorithm into a target data stream having a specific format, wherein the data stream in the specific format is different from the video stream; The target data stream is added to an instruction queue to wait for being sent to the second terminal.

7. The method according to any one of claims 1 to 6, characterized in that A virtual machine is deployed in the first terminal, and obtaining the first frame image and the second frame image includes: Responding to input based on an interactive device through the virtual machine, and adjusting screen content displayed by the first terminal according to the input; Relevant information about adjacent frame images of screen content displayed by the first terminal is obtained through the virtual machine, where the relevant information about adjacent frame images includes the first frame image and the second frame image.

8. A method for transmitting a screen image, characterized in that: Applied to the second terminal, the method includes: Receiving first image data sent by a first terminal; Determining a target position of the first image data in a target frame image displayed by the second terminal; Copying the first image data to a target position in the target frame image and displaying the image; The first image data is determined according to an image change region in which the second frame image changes relative to the first frame image, the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than In the second frame image.

9. The method according to claim 8, characterized in that The receiving the second image data sent by the first terminal includes: receiving second image data sent by the first terminal; The second image data is decoded according to a shallow compression algorithm to obtain the first image data.

10. The method according to claim 9, characterized in that The receiving the second image data sent by the first terminal also includes: receiving a target data stream sent by the first terminal; Determining that the format of the target data stream is a specific format; The target data stream is parsed according to the data stream parsing method of the specific format to obtain the second image data, type data of the compression algorithm, and index information of the macroblock, wherein the type data of the compression algorithm is used to determine the shallow compression algorithm, and the index information of the macroblock is used to determine the target position.

11. An electronic device, characterized in that: include: processor; as well as A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the steps of the method of any one of claims 1 to 10.

12. A computer readable medium, characterized in that The computer-readable medium stores one or more programs, which, when executed by an electronic device including a plurality of application programs, enable the electronic device to perform the steps of the method according to any one of claims 1 to 10.

Images