Screen image transmission method, electronic equipment and computer readable medium

Abstract

The embodiment of the invention provides a screen image transmission method, electronic equipment and a computer readable medium, and the method comprises the steps that a first terminal obtains a first frame image and a second frame image, the first frame image and the second frame image are adjacent frame images in screen content displayed by the first terminal, the display time of the first frame image is earlier than that of the second frame image; determining that an image change region of the second frame image, which is changed relative to the first frame image, meets a small region change condition; and sending first image data corresponding to the image change area in the second frame image to a second terminal.

Description

Technical Field

[0001] The present application relates to the field of video coding and decoding technology, and in particular to a screen image transmission method, electronic device and computer-readable medium. Background Art

[0002] In the fields of video conferencing, remote online teaching, online live broadcasting, computer remote monitoring, etc., the transmission of screen images (also called screen content) of computer screens is essential. For example, the image transmission schemes adopted by the related technologies include collecting screen data as video data and encoding it before transmission, or treating the screen as a static picture and encoding it before transmission.

[0003] However, in actual applications, the aforementioned technical solutions adopted in related technologies still have problems such as large amount of transmitted data and high resource occupancy rate. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a screen image transmission method, an electronic device and a computer-readable medium, which can solve the problems existing in the related art such as large transmission data volume and high resource occupancy rate.

[0005] To solve the above technical problems, the embodiments of the present application are implemented through the following aspects.

[0006] In a first aspect, an embodiment of the present application provides a method for transmitting a screen image, which is applied to a first terminal, and the method includes: acquiring a first frame image and a second frame image, wherein the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image; determining that an image change area of ​​the second frame image that has changed relative to the first frame image satisfies a small area change condition; and sending first image data corresponding to the image change area in the second frame image to a second terminal.

[0007] In a second aspect, an embodiment of the present application provides a method for transmitting a screen image, which is applied to a second terminal, and the method includes: receiving first image data sent by a first terminal; determining a target position of the first image data in a target frame image displayed by the second terminal; copying the first image data to the target position in the target frame image and displaying it; wherein the first image data is determined based on an image change area in which the second frame image changes relative to the first frame image, the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and computer executable instructions stored in the memory and executable on the processor, wherein the computer executable instructions, when executed by the processor, implement the steps of executing the method described in the first aspect or the second aspect.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0010] In an embodiment of the present application, the first terminal obtains a first frame image and a second frame image, and when the image change area of ​​the second frame image relative to the first frame image satisfies a small area change condition, the first image data corresponding to the image change area in the second frame image is sent to the second terminal. Thus, by transmitting only the changed image data, the amount of transmitted data and resource occupancy can be greatly reduced, thereby improving the transmission efficiency of image data and the refresh efficiency of the screen image. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0012] Figure 1 One of the flow charts of the screen image transmission method provided in the embodiment of the present application is shown.

[0013] Figure 2 A second flow chart of the screen image transmission method provided in an embodiment of the present application is shown.

[0014] Figure 3 A third flow chart of the screen image transmission method provided in an embodiment of the present application is shown.

[0015] Figure 4 One of the structural schematic diagrams of the screen image transmission device provided in an embodiment of the present application is shown.

[0016] Figure 5 A second structural schematic diagram of the screen image transmission device provided in an embodiment of the present application is shown.

[0017] Figure 6 A schematic diagram of the hardware structure of an electronic device for executing the screen image transmission method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0019] For the transmission of screen images (or screen contents, desktop images, screen pictures, etc.) in cloud desktops, considering the encoding / decoding capabilities of the terminal, network bandwidth and other factors, if each captured frame of the image is encoded, it will inevitably lead to problems such as large amount of transmission data and high resource occupancy rate. In this regard, the present application provides a screen image transmission method, electronic device and computer-readable medium, which can add small area change scene judgment of continuous changes in small areas after capturing each frame of the image. If the conditions of the small area change scene are met, the captured current frame image is not discarded, but the entire picture is not encoded. Instead, the image is transmitted based on the change area of ​​the previous and next frames. Therefore, the data transmission amount and resource occupancy rate can be greatly reduced.

[0020] Of course, in addition to the aforementioned cloud desktop scenarios, the screen image transmission scheme provided in the embodiments of the present application is also applicable to real-time encoding systems that include screen content, such as cloud desktops that include interactive operations, video conferencing, live webcasts, computer remote monitoring, screen sharing, and other scenarios that have high real-time requirements and certain requirements for compression rates. It is especially suitable for any type of small area or small area graphics rendering scenarios, such as some drawing software with self-drawing mouse features (such as AutoCAD, etc.). The self-drawing mouse can be understood as when the program is enabled, the application will replace the cursor shape configured by the operating system with the cursor shape used by itself, etc.

[0021] Based on this, the technical solution provided by the present application is introduced below in combination with the accompanying drawings and embodiments, as follows.

[0022] Figure 1 A schematic flow chart of a screen image transmission method 100 provided in an embodiment of the present application is shown. The method 100 can be executed by a first terminal (such as a mobile phone, a computer, a television, etc.), such as by software and / or hardware installed in the first terminal. Figure 1 As shown, the method 100 may include the following steps.

[0023] S110, acquiring a first frame image and a second frame image.

[0024] The first terminal (also referred to as the encoding end) may adopt but is limited to desktop software development kit capture (SDK) to acquire the first frame image and the second frame image. In this embodiment, the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image, such as the second frame image is the current frame image acquired by the first terminal, and the first frame image is the previous frame image of the current frame image.

[0025] It is worth noting that when the first terminal obtains the first frame image and the second frame image, if a virtual machine is configured on the first terminal, then the first terminal can respond to input based on an interactive device (such as a microphone, keyboard, mouse, wearable device, etc.) through the virtual machine, adjust the screen content displayed by the first terminal according to the input, and then obtain relevant information of adjacent frame images of the screen content displayed by the first terminal through the virtual machine. The relevant information of the adjacent frame images includes but is not limited to the first frame image and the second frame image.

[0026] S120: Determine whether an image change region of the second frame image that is changed relative to the first frame image satisfies a small region change condition.

[0027] The small area change condition may be configured according to actual needs. For example, in this embodiment, the small area change condition may include at least one of the following (a)-(b).

[0028] (a) The proportion of the image change area in the first frame image or the second frame image is less than a first threshold.

[0029] The first threshold value can be designed according to actual needs, such as 1%, 2%, etc. Based on this, assuming that the first threshold value is 1%, if the first terminal determines that the image change area of ​​the second frame image relative to the first frame image accounts for less than 1% in the first frame image or the second frame image, it can be determined that the image change area of ​​the second frame image relative to the first frame image meets the small area change condition, otherwise it does not meet the condition.

[0030] (b) The dispersion of the image change region in the first frame image or the second frame image is less than a second threshold value. There are multiple ways to determine the dispersion. For example, assuming that the desktop image is divided into five equal parts horizontally and the second threshold value is 25%, if the image change region is dispersed in a single desktop image block after the division (i.e., the dispersion is 20%), it can be determined that the dispersion of the image change region in the first frame image or the second frame image is less than the second threshold value. If the image change region is dispersed in each desktop image block after the division (i.e., the dispersion is 100%), it can be determined that the dispersion of the image change region in the first frame image or the second frame image is greater than the second threshold value.

[0031] It is worth noting that when determining the aforementioned image change area, the granularity can be determined based on the macroblocks included in the screen image. For example, the first terminal can determine whether the image change area meets the small area change condition based on information such as the macroblock position and / or the number of macroblocks in the image change area, and the number of macroblocks included in the image change area accounts for the ratio of the number of macroblocks included in the desktop image (i.e., the first frame image, the second frame image), etc.

[0032] For example, assuming that the first frame image includes 100 macroblocks, the image change area includes 1 macroblock, and the first threshold is 2%, then it can be determined that the dispersion (1%) of the image change area in the first frame image is less than the second threshold.

[0033] In one implementation, the shape of the aforementioned image change area can be of any type, that is, the technical solution provided by the present application is applicable to any type of small area scene recognition, thereby improving the applicability and flexibility of the technical solution of the present application.

[0034] It is worth noting that the present application realizes the comparison of continuous frame images to determine the image change area through the aforementioned S120, which can avoid the problem of recognizing the mouse shape drawn by the application as screen image data, effectively improves the accuracy of screen image recognition, and avoids the problem of more obvious stuttering and larger delays caused by the refresh rate of the self-drawn mouse in the screen being consistent with the refresh rate of the screen image due to misrecognition.

[0035] S130: Send the first image data corresponding to the image change area in the second frame image to the second terminal.

[0036] There are many ways for the first terminal to send the first image data. For example, the first terminal can send the first image data to the second terminal, or send the first image data to the second terminal through a virtual machine based on a desktop protocol, etc., which are not limited here.

[0037] Correspondingly, when the second terminal (which may be called the decoding end) receives the first image data sent by the first terminal, it can determine the target position of the first image data in the target frame image displayed by the second terminal (corresponding to the second frame image on the first terminal side), and copy the first image data to the target position in the target frame image and display it, that is, overwrite the first image data to the corresponding target position of the target frame image (or overwrite to the corresponding position of the current target frame image cache, etc.), so as to refresh or update the target frame image.

[0038] In this embodiment, during the screen image transmission process, the first terminal obtains the first frame image and the second frame image, and when the image change area of ​​the second frame image relative to the first frame image satisfies the small area change condition, the first image data corresponding to the image change area in the second frame image is sent to the second terminal. Thus, by transmitting only the changed image data, the amount of transmitted data and resource occupancy rate can be greatly reduced, and the transmission efficiency of image data and the refresh efficiency of the screen image can be improved.

[0039] In this case, in a possible implementation, considering that the shallow compression method itself can be supported by desktop protocols such as SPICE, and only the current frame image information is needed, the first terminal can also use the shallow compression method to compress the first image data when sending the first image data corresponding to the image change area in the second frame image to the second terminal. The implementation process is as follows: Figure 2 As shown, the content is as follows.

[0040] S210: extracting first image data corresponding to the image change area from the second frame image.

[0041] Optionally, the first terminal may extract the first image data (such as RGB data, etc.) corresponding to the image change region from the second frame image in a variety of ways. For example, assuming that the desktop image (such as the first frame image and the second frame image) includes multiple macroblocks, and each macroblock corresponds to index information, then the first terminal may extract the first image data corresponding to the image change region from the second frame image based on the index information of the macroblocks included in the image change region, so as to separate it from the desktop image data.

[0042] S220: Encode the first image data according to a shallow compression algorithm to obtain second image data.

[0043] The shallow compression algorithm may include but is not limited to a JPEG image compression algorithm and / or a quic graphic compression algorithm.

[0044] S230: Send the second image data to the second terminal.

[0045] Correspondingly, when the second terminal receives the second image data sent by the first terminal, it can decode the second image data according to the corresponding shallow compression algorithm to obtain the first image data, and copy the first image data to the target position in the target frame image and display it. Therefore, by utilizing the shallow compression algorithm to decompress image data only by relying on the characteristics of the current frame image information, the present application can well reduce the encoding / decoding delay during image data transmission, improve the encoding / decoding speed of small area change areas, and make the refresh rate of the change area much higher than the original desktop frame rate, thereby achieving the purpose of improving the smoothness of interactive scene use.

[0046] In one possible implementation, when the first terminal sends the first image data corresponding to the image change area in the second frame image to the second terminal, the second image data, the index information of the macroblocks included in the image change area, and the algorithm type data corresponding to the shallow compression algorithm can be compressed into a target data stream (also referred to as an instruction stream) having a specific format (the data stream in the specific format is different from the video stream), and then the target data stream is added to the instruction queue to wait for being sent to the second terminal.

[0047] Correspondingly, when the second terminal receives the target data stream sent by the first terminal, if it is determined that the format of the target data stream is a specific format (i.e., an instruction stream format), then the target data stream can be parsed according to the data stream parsing method of the specific format to obtain the second image data, the type data of the compression algorithm, and the index information of the macroblock, and then the second image data is decompressed according to the shallow compression algorithm corresponding to the type data of the compression algorithm to obtain the first image data, and the target position of the first image data in the target frame image is determined according to the index information of the macroblock, so as to copy the first image data to the target position in the target frame image and display it. Therefore, the present application can further improve the encoding and decoding speed of small area change areas by combining the instruction stream method to encode / decompress image data on the basis of the shallow compression algorithm, thereby achieving the purpose of improving the fluency of interactive scene use.

[0048] Furthermore, when the first terminal transmits the second image data, the index information of the macroblocks included in the image change area, the algorithm type corresponding to the shallow compression algorithm and the second image data are compressed together into a target data stream for transmission, so that the second terminal can timely and accurately obtain the shallow compression algorithm and the location information of the first image data, thereby ensuring the accuracy of the encoding and decoding results and the reliability of the screen display.

[0049] It is worth noting that in addition to using a shallow compression algorithm and instruction stream to transmit image data when it is determined that the image change area of ​​the second frame image relative to the first frame image meets the small area change condition, when the first terminal determines that the image change area of ​​the second frame image relative to the first frame image does not meet the small area change condition, the first terminal may also use other methods (deep compression algorithms (such as H.264 / H.265 / AV1, etc., which require reference to the relationship between previous and subsequent frames to achieve a further compression rate video encoding algorithm), video stream transmission method) other than the aforementioned shallow compression algorithm and instruction stream to transmit image data, thereby improving the flexibility and reliability of the first terminal in image transmission.

[0050] The aforementioned technical solution provided in the embodiment of the present application, on the one hand, combines the information of the desktop capture process itself, does not introduce new information collection and processing procedures, and can ensure the applicability of the solution of the present application; on the other hand, it also effectively combines the support of the existing protocol for instruction streams and the characteristics of shallow compression, thereby improving the smoothness of desktop use in small area change scenarios.

[0051] Figure 3 A schematic flow chart of a screen image transmission method 300 provided in an embodiment of the present application is shown. The method 300 may be executed by a second terminal, such as software and / or hardware installed in the second terminal. Figure 3 As shown, the method 300 may include but is not limited to the following steps.

[0052] S310, receiving first image data sent by a first terminal.

[0053] S320: Determine a target position of the first image data in a target frame image displayed by the second terminal.

[0054] S330: Copy the first image data to a target position in the target frame image and display it.

[0055] Among them, the first image data is determined based on the image change area of ​​the second frame image relative to the first frame image, the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image.

[0056] Optionally, the receiving the second image data sent by the first terminal includes: receiving the second image data sent by the first terminal;

[0057] The second image data is decoded according to a shallow compression algorithm to obtain the first image data.

[0058] Optionally, the receiving of the second image data sent by the first terminal also includes: receiving a target data stream sent by the first terminal; determining that the format of the target data stream is a specific format; parsing the target data stream according to the data stream parsing method of the specific format to obtain the second image data, type data of the compression algorithm, and index information of the macroblock, wherein the type data of the compression algorithm is used to determine the shallow compression algorithm, and the index information of the macroblock is used to determine the target position.

[0059] It can be understood that since the various implementation methods of the screen image transmission method provided in method embodiment 300 have the same or corresponding technical features as those of the aforementioned method embodiment 100, the implementation process of the various implementation methods of the screen image transmission method provided in method embodiment 300 can refer to the relevant description in the aforementioned method embodiment 100, and achieve the same or corresponding technical effects, which will not be repeated here.

[0060] Figure 4 A structural schematic diagram of a screen image transmission device 400 provided in an embodiment of the present application is shown, and the device 400 includes: an image acquisition module 410, used to acquire a first frame image and a second frame image, wherein the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image, a small area recognition module 420, used to determine whether an image change area of ​​the second frame image that has changed relative to the first frame image satisfies a small area change condition; and a data sending module 430, used to send first image data corresponding to the image change area in the second frame image to the second terminal.

[0061] Optionally, the small area change condition includes at least one of the following: the proportion of the image change area in the first frame image or the second frame image is less than a first threshold; the dispersion of the image change area in the first frame image or the second frame image is less than a second threshold.

[0062] Optionally, the data sending module 430 sends the first image data corresponding to the image change area in the second frame image to a second terminal, including: extracting the first image data corresponding to the image change area from the second frame image; encoding the first image data according to a shallow compression algorithm to obtain second image data; and sending the second image data to the second terminal.

[0063] Optionally, the shallow compression algorithm includes a JPEG image compression algorithm and / or a quic graphic compression algorithm.

[0064] Optionally, the data sending module 430 extracts the first image data corresponding to the image change area from the second frame image, including: extracting the first image data corresponding to the image change area from the second frame image according to the index information of the macroblocks included in the image change area; wherein the first frame image and the second frame image both include multiple macroblocks.

[0065] Optionally, the data sending module 430 sends the first image data corresponding to the image change area in the second frame image to the second terminal, and also includes: compressing the second image data, the index information of the macroblocks included in the image change area, and the algorithm type data corresponding to the shallow compression algorithm into a target data stream with a specific format, wherein the data stream in a specific format is different from the video stream; and adding the target data stream to an instruction queue to wait for sending to the second terminal.

[0066] Optionally, the image acquisition module 410 acquires a first frame image and a second frame image, including: responding to input based on an interactive device through a virtual machine, and adjusting the screen content displayed by the first terminal according to the input; acquiring relevant information of adjacent frame images of the screen content displayed by the first terminal through the virtual machine, the relevant information of the adjacent frame images including the first frame image and the second frame image.

[0067] Figure 5 A structural schematic diagram of a screen image transmission device 500 provided in an embodiment of the present application is shown, and the device 500 includes: a data receiving module 510, used to receive first image data sent by a first terminal; a position determination module 520, used to determine a target position of the first image data in a target frame image displayed by a second terminal; a display module 530, used to copy the first image data to the target position in the target frame image and display it; wherein the first image data is determined based on an image change area in which the second frame image changes relative to the first frame image, the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image.

[0068] Optionally, the data receiving module 510 receives the second image data sent by the first terminal, including: receiving the second image data sent by the first terminal; decoding the second image data according to a shallow compression algorithm to obtain the first image data.

[0069] Optionally, the data receiving module 510 receives the second image data sent by the first terminal, and also includes: receiving a target data stream sent by the first terminal; determining that the format of the target data stream is a specific format; parsing the target data stream according to the data stream parsing method of the specific format to obtain the second image data, type data of the compression algorithm, and index information of the macroblock, wherein the type data of the compression algorithm is used to determine the shallow compression algorithm, and the index information of the macroblock is used to determine the target position.

[0070] The screen image transmission devices 400 and 500 provided in the embodiments of the present application can execute the methods described in the foregoing method embodiments and realize the functions and beneficial effects of the methods described in the foregoing method embodiments, which will not be repeated here.

[0071] Figure 6 A schematic diagram of the hardware structure of an electronic device that implements the embodiment of the present application is shown. Referring to the figure, at the hardware level, the electronic device includes a processor, and optionally, an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device may also include hardware required for other services.

[0072] The processor, the network interface and the memory may be interconnected through an internal bus, which may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0073] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0074] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a device for locating the specified user at the logical level. The processor executes the program stored in the memory and is specifically used to execute: Figure 1-Figure 3The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0075] The above application Figure 1-Figure 3 The method disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0076] The electronic device can also execute the methods described in the foregoing method embodiments, and realize the functions and beneficial effects of the methods described in the foregoing method embodiments, which will not be repeated here.

[0077] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0078] The embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable medium stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes Figure 1-Figure 3The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0079] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0080] Furthermore, an embodiment of the present application also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the following process is implemented: Figure 1-Figure 2 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0081] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0082] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0083] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0084] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0085] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A method for transmitting a screen image, characterized in that: Applied to a first terminal, the method includes: Acquire a first frame image and a second frame image, where the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image; Determining that an image change region of the second frame image that changes relative to the first frame image satisfies a small region change condition; The first image data corresponding to the image change area in the second frame image is sent to the second terminal.

2. The method according to claim 1, characterized in that The small area change condition includes at least one of the following: The proportion of the image change area in the first frame image or the second frame image is less than a first threshold; The dispersion degree of the image change region in the first frame image or the second frame image is less than a second threshold.

3. The method according to claim 1 or 2, characterized in that The sending the first image data corresponding to the image change area in the second frame image to the second terminal includes: Extracting first image data corresponding to the image change area from the second frame image; Encoding the first image data according to a shallow compression algorithm to obtain second image data; The second image data is sent to the second terminal.

4. The method according to claim 3, characterized in that The shallow compression algorithm includes a JPEG image compression algorithm and / or a quic graphic compression algorithm.

5. The method according to claim 3, characterized in that The extracting the first image data corresponding to the image change area from the second frame image includes: Extracting first image data corresponding to the image change area from the second frame image according to index information of the macroblocks included in the image change area; Wherein, the first frame image and the second frame image both include a plurality of the macroblocks.

6. The method according to claim 3, characterized in that The sending the first image data corresponding to the image change area in the second frame image to the second terminal also includes: compressing the second image data, the index information of the macroblocks included in the image change area, and the algorithm type data corresponding to the shallow compression algorithm into a target data stream having a specific format, wherein the data stream in the specific format is different from the video stream; The target data stream is added to an instruction queue to wait for being sent to the second terminal.

7. The method according to any one of claims 1 to 6, characterized in that A virtual machine is deployed in the first terminal, and obtaining the first frame image and the second frame image includes: Responding to input based on an interactive device through the virtual machine, and adjusting screen content displayed by the first terminal according to the input; Relevant information about adjacent frame images of screen content displayed by the first terminal is obtained through the virtual machine, where the relevant information about adjacent frame images includes the first frame image and the second frame image.

8. A method for transmitting a screen image, characterized in that: Applied to the second terminal, the method includes: Receiving first image data sent by a first terminal; Determining a target position of the first image data in a target frame image displayed by the second terminal; Copying the first image data to a target position in the target frame image and displaying the image; Among them, the first image data is determined based on the image change area of ​​the second frame image relative to the first frame image, the first frame image and the second frame image are adjacent frame images in the screen content displayed by the first terminal, and the display time of the first frame image is earlier than that of the second frame image.

9. The method according to claim 8, characterized in that The receiving the second image data sent by the first terminal includes: receiving second image data sent by the first terminal; The second image data is decoded according to a shallow compression algorithm to obtain the first image data.

10. The method according to claim 9, characterized in that The receiving the second image data sent by the first terminal also includes: receiving a target data stream sent by the first terminal; Determining that the format of the target data stream is a specific format; The target data stream is parsed according to the data stream parsing method of the specific format to obtain the second image data, type data of the compression algorithm, and index information of the macroblock, wherein the type data of the compression algorithm is used to determine the shallow compression algorithm, and the index information of the macroblock is used to determine the target position.

11. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the steps of the method of any one of claims 1 to 10.

12. A computer-readable medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute the steps of the method according to any one of claims 1 to 10.

Images