Real-time screen sharing

Abstract

In embodiments of the subject matter described herein, a method and apparatus for real-time screen sharing are presented. During screen sharing between two devices, if a certain predetermined condition is met between the image that the first device has encoded and the image that the second device has decoded, the first device suspends processing (e.g., capturing, encoding, or sending) of the image. If the suspension time of the suspension of capturing reaches a certain length, a parameter associated with the image compression ratio is adjusted. After the first device resumes processing of the image, the new image captured on the first device is encoded with the adjusted parameter. According to embodiments of the subject matter described herein, the transmission bitrate during screen sharing is controlled according to the suspension time of the suspension of processing of the image without the need to estimate or determine the specific network bandwidth or the codec speed. Embodiments of the subject matter described herein can reduce the transmission delay of screen sharing, thereby effectively guaranteeing the user experience during screen sharing.

Description

[0001] This application is a divisional application of Chinese invention patent application filed on June 20, 2017, with application number 201710471755.3 and invention title "Real-time Screen Sharing". Technical Field

[0002] Embodiments of this disclosure relate to methods and apparatus for real-time screen sharing. Background Technology

[0003] Documents, videos, and other digital content are increasingly being shared via the Internet or other networks. Screen sharing is a common sharing method where one device (called the "sender") displays its screen interface over a network to another device (called the "receiver") within an allowable time delay. The receiver has an interface synchronized with the sender to view the shared content. Screen sharing can be used in many scenarios, such as remote desktops (e.g., collaborative work, remote slideshow presentations), video conferencing, cloud-based applications (e.g., cloud gaming), and more.

[0004] Typically, screen sharing is achieved using video-based sharing schemes. This involves capturing video image frames at the sending end and then transmitting these frames to the receiving end to be combined into a video. In video-based sharing schemes, the sending end captures screen content and uses a video encoder (e.g., H.264 / AVC, H.265 / HEVC, VP9, ​​AVS, etc.) to encode the captured content before sending the encoded content to the receiving end. The receiving end decodes the received encoded content and renders it on its own screen. During screen sharing, the content of the sending end's screen is graphically transmitted to the receiving end. In some scenarios, the user at the receiving end can also interact with the sending end's screen to control the sending end's screen. Summary of the Invention

[0005] In embodiments of the subject matter described herein, a method and apparatus for real-time screen sharing are proposed. During screen sharing between two devices, if a predetermined condition is met between an image encoded by a first device and an image decoded by a second device, the first device pauses image processing, for example, pausing image capture or pausing encoding and transmission operations after image capture. If the pause time for capture reaches a certain length, parameters associated with the image compression ratio (such as quantization parameters) are adjusted. After the first device resumes image processing, the newly captured image on the first device is encoded using the adjusted parameters. According to embodiments of the subject matter described herein, the transmission bitrate during screen sharing is controlled based on the pause time for image capture, without needing to estimate or determine specific network bandwidth or encoding / decoding speeds. Therefore, embodiments of the subject matter described herein can reduce transmission latency during screen sharing, thereby effectively ensuring a good user experience during screen sharing.

[0006] The summary section is provided for the purpose of presenting the chosen concepts in a simplified form, which will be further described in the detailed embodiments below. The summary section is not intended to identify key or principal features of the subject matter described herein, nor is it intended to limit the scope of the subject matter described herein. Attached Figure Description

[0007] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0008] Figure 1 A block diagram of a computing device / server in which one or more embodiments of the subject matter described herein may be implemented is shown;

[0009] Figure 2 A schematic diagram illustrates an example environment in which one or more embodiments of the subject matter described herein may be implemented;

[0010] Figure 3 A flowchart illustrating a method for real-time screen sharing according to an embodiment of the subject matter described herein is shown;

[0011] Figure 4 A flowchart illustrating a method for controlling screen sharing according to an embodiment of the subject matter described herein is shown;

[0012] Figure 5 A flowchart illustrating a method for adjusting quantization parameters associated with an image compression ratio, according to an embodiment of the subject matter described herein;

[0013] Figure 6A flowchart illustrating a screen sharing method based on multiple codecs, according to embodiments of the subject matter described herein, is shown; and

[0014] Figures 7A-7D An example graphical user interface (GUI) for sharing a screen is shown, according to an embodiment of the subject matter described herein. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0017] Low latency is a fundamental requirement during real-time screen sharing. Traditionally, to achieve low latency, the amount of data transmitted can be reduced by decreasing the quality of captured image frames, resulting in degraded image quality; or the amount of data transmitted can be reduced by decreasing the number of image frames captured per second (i.e., reducing the frame rate), resulting in poor image continuity and severely impacting user experience. An improvement over this traditional approach is to dynamically adjust the bitrate during screen sharing, which dynamically adjusts the transmission bitrate by measuring the latency of each image (i.e., the time it takes for an image to be transmitted from one device to another). This allows for dynamic adjustment of the transmission bitrate. However, this improvement only considers the latency of a single image, making it susceptible to transient network conditions. Furthermore, it does not consider the decoding capabilities of the receiving end; however, poor decoding capabilities at the receiving end can cause it to fail to display the shared screen in a timely manner. Therefore, this improvement cannot accurately and effectively reduce latency while guaranteeing improved image quality.

[0018] Therefore, the subject matter described herein proposes a method and apparatus for real-time screen sharing. During screen sharing between two devices, if certain predetermined conditions are met between an image encoded by the first device and an image decoded by the second device, the first device pauses image processing (e.g., capturing, encoding, or transmitting). If the pause time for capturing reaches a certain length, parameters associated with the image compression ratio (e.g., quantization parameters) are adjusted. After the first device resumes image processing, the newly captured image on the first device is encoded using the adjusted parameters. According to embodiments of the subject matter described herein, the transmission bitrate during screen sharing is controlled based on the pause time for image processing, without needing to estimate or determine specific network bandwidth or encoding / decoding speeds. Furthermore, embodiments of the subject matter described herein not only reduce transmission latency during screen sharing but also ensure the quality of the screen shared image, thereby improving the user experience.

[0019] Furthermore, according to embodiments of the subject matter described herein, by dynamically adjusting the quantization parameter values ​​used to determine the compression ratio and transmission bitrate of an image using factors such as pause time, delay time, and the number of continuous images, the transmission bitrate can be controlled more accurately and in real-time. Additionally, embodiments of the subject matter described herein effectively reduce the amount of data to be transmitted by selecting a suitable codec to encode partially altered images. Furthermore, in cases with small areas of variation, using multiple codecs to encode each small area of ​​variation separately further reduces the amount of data to be transmitted.

[0020] The following is for reference Figure 1 Figure 7 illustrates the basic principles of the topic described in this paper and several example implementations. Figure 1 A block diagram of a computing device / server 100 in which one or more embodiments of the subject matter described herein may be implemented is shown. It should be understood that... Figure 1 The computing device / server 100 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0021] like Figure 1 As shown, the computing device / server 100 is in the form of a general-purpose computing device. Components of the computing device / server 100 may include, but are not limited to, one or more processors or processing units 110, memory 120, storage devices 130, one or more communication units 140, one or more input devices 150, and one or more output devices 160. The processing unit 110 may be a physical or virtual processor and is capable of performing various processes based on the information stored in memory 120. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capabilities of the computing device / server 100.

[0022] The computing device / server 100 typically includes multiple computer storage media. Such media can be any available media accessible to the computing device / server 100, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 120 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 130 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data and accessible within the computing device / server 100.

[0023] The computing device / server 100 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 1 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 120 may include modules 125, such as one or more program modules configured to perform the methods or functions of the various embodiments described herein.

[0024] The communication unit 140 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of the computing device / server 100 can be implemented as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device / server 100 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0025] Input device 150 can be one or more various input devices, such as a mouse, keyboard, trackball, etc. Output device 160 can be one or more output devices, such as a monitor, speaker, printer, etc. The computing device / server 100 can also communicate as needed with one or more external devices (not shown) via communication unit 140. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with the computing device / server 100, or with any device (e.g., network card, modem, etc.) that enables the computing device / server 100 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interfaces (not shown).

[0026] Figure 2 A schematic diagram of an example environment 200 in which one or more embodiments of the subject matter described herein may be implemented is shown. Figure 2 As shown, environment 200 includes device 210 (also referred to as "first device") and device 220 (also referred to as "second device"), at least one of which can be used for reference. Figure 1 The computing device / server 100 is described. In practice, devices 210 and 220 can be any device with networking, display, and processing capabilities, such as fixed devices (e.g., desktop computers, servers, projectors, etc.) or mobile devices (e.g., laptops, smartphones, tablets, etc.). Figure 2 As shown, devices 210 and 220 can communicate with each other via network 230. Network 230 can be any wired and / or wireless network. Optionally, the network can include, but is not limited to, the Internet, local area network, wide area network, metropolitan area network, virtual private network (VPN) network, wireless communication network, etc.

[0027] exist Figure 2 In the example shown, device 210 can be the sharing end (i.e., the sender) during screen sharing, and it may include a display module 212, a capture module 214, an encoding module 216, and a transmission module 218. The display module 212 can display a graphical user interface (GUI) on the screen, such as GUI 213. The capture module 214 can capture the screen during screen sharing, for example, at a predetermined number of frames per second. The number of frames captured per second (i.e., the frame rate) can be dynamically set according to the application scenario. For example, for scenarios with high realism and real-time requirements (such as cloud gaming), tens of frames per second can be captured, while for ordinary scenarios (such as slideshow sharing), only a few frames per second may be captured. The encoding module 216 can encode the captured image according to encoding parameters (such as quantization parameters associated with the image compression ratio). Generally, the larger the encoded data, the higher the image quality. The transmission module 218 is used to send the encoded image over a network, for example, to device 220.

[0028] It should be understood that, although Figure 2 Only four modules of device 210 are shown in the diagram, but device 210 may also include other modules, such as a control or processing module, a color space conversion module, etc. Furthermore, the encoding module 216 may have both encoding and decoding capabilities; that is, the encoding module 216 can be an encoding / decoding module.

[0029] like Figure 2As shown, device 220 can be the shared end (i.e., the receiving end) during screen sharing, and it may include a transmission module 222, a decoding module 224, a drawing module 226, and a display module 228. The transmission module 222 can receive encoded images from device 210 via a network, and the decoding module 224 can decode the encoded images. The drawing module 226 is used to draw or render the decoded images, and displays the drawn images through the display module 228, for example, displaying a GUI 229.

[0030] It can be seen that the GUI 229 displayed in device 220 is the shared screen view of the GUI 213 displayed in device 210. It should be understood that, although... Figure 2 Only four modules of device 220 are shown in the diagram, but device 220 may also include other modules, such as a control or processing module, an inverse color space conversion module, etc. Furthermore, the encoding module 2224 may have both encoding and decoding capabilities; that is, the encoding module 224 can be an encoding / decoding module.

[0031] Furthermore, whenever the decoding module 224 of device 220 successfully decodes an image, it sends feedback to device 210, informing device 210 that the image has been received and decoded. Therefore, device 210 always knows how many images it has encoded and transmitted, and how many image decoding feedbacks it has received from device 220. According to embodiments of the subject matter described herein, if the number of images that have been encoded but not yet decoded (referred to as the "queue number of images") is too large (e.g., device 210 has encoded and transmitted 10 images, but has only received decoding feedback for 5 images), device 210 will pause image processing for a period of time, such as pausing image capture or pausing encoding and transmission operations after image capture, until it receives more decoding feedback.

[0032] The inventors discovered two main reasons for the excessive number of images in the queue. The first reason is the poor available network bandwidth between devices 210 and 220, which requires a longer time to send images. In this situation, device 210 needs to pause image processing for a period of time; otherwise, too many images will remain in the transmission channel, leading to increased overall latency.

[0033] The second reason is the poor decoding capability of the receiving end (such as low decoding speed). For example, the processing capability of device 220 may be poor. That is, although device 220 can receive the encoded images from device 210 in a timely manner, it cannot decode these encoded images in real time. Therefore, device 210 also needs to pause image processing for a period of time; otherwise, there will be too many images waiting to be decoded at device 220, which will also lead to a large overall delay.

[0034] In embodiments of the subject matter described herein, the term "frame rate" generally refers to the number of images captured or displayed per second. A higher frame rate results in smoother visuals, while a lower frame rate leads to more jerky visuals. The term "resolution" refers to the length and width of the image, i.e., the image size, while the term "bitrate" refers to the amount of data after compressing the images captured per second. Bitrate control is a crucial aspect of screen sharing; a bitrate that is too high may result in high latency and low frame rates, while a bitrate that is too low may result in poor image quality. The following references... Figure 3 Figure 7 illustrates further details of bitrate control according to an embodiment of the subject matter described herein.

[0035] Figure 3 A flowchart of a method 300 for real-time screen sharing according to an embodiment of the subject matter described herein is shown. It should be understood that method 300 can be derived from references... Figure 1 The described computing device / server 100 or reference Figure 2 The described device 210 performs the operation.

[0036] In 302, in response to the first device (e.g.) Figure 2 The image encoded by the illustrated device 210 is compared with that of the second device (e.g., Figure 2 The image processing is paused on the first device when the decoded images of the illustrated device 220 meet certain predetermined conditions (referred to as the "first predetermined conditions"), and the first device shares the screen with the second device.

[0037] Typically, the transmitting end (e.g., device 210) records the number of images or the amount of data (such as the number of bits) that have been encoded and transmitted. Furthermore, the transmitting end receives decoding feedback from the receiving end (e.g., device 220), which sends feedback back to the transmitting end whenever it receives and successfully decodes an image. In some embodiments, the first predetermined condition may be that the difference between the number of images encoded by the transmitting end and the number of images decoded by the receiving end is greater than a predetermined number. That is, when too many images have been transmitted but not yet decoded, it indicates that the number of images in the queue is too large, and therefore image processing needs to be paused and a waiting period should be initiated. In another embodiment, the first predetermined condition may also be that the difference between the number of bits of the images encoded by the transmitting end and the number of bits of the images decoded by the receiving end is greater than a predetermined number of bits. The following references... Figure 4 An example embodiment of pausing image processing is further described.

[0038] At 304, in response to the pause duration satisfying another predetermined condition (referred to as the "second predetermined condition"), parameters associated with the image compression ratio (e.g., quantization parameters) are adjusted. The quantization parameter (QP) is the sequence number of the quantization step size (Qstep). In H.264 / AVC and H.265 / HEVC, for an 8-bit input image / video, its value is typically an integer between, for example, 0 and 51. The quantization step size is determined based on the value of the quantization parameter, and the quantization step size determines the image compression ratio and the transmission bitrate shared by the screen. Generally, increasing the quantization parameter value degrades the image size and quality while decreasing the transmission bitrate, and vice versa. See the following reference... Figure 5 An example embodiment of adjusting the quantization parameters is further described.

[0039] At 306, in response to resuming image processing on the first device, the newly captured image on the first device is encoded using adjusted parameters. For example, after adjusting the quantization parameters, when image capture resumes, the adjusted quantization parameters will be used for encoding. See the following reference... Figure 6 An example embodiment of encoding images is further described.

[0040] Therefore, according to an embodiment of the method 300 described herein, the transmission bitrate during screen sharing is controlled based on the pause time of the paused image processing, without the need to estimate or determine the specific network bandwidth or encoding / decoding speed, thereby reducing latency and effectively ensuring the user experience during screen sharing.

[0041] Figure 4 A flowchart illustrating a method 400 for controlling screen sharing according to an embodiment of the subject matter described herein is shown. It should be understood that method 400 can be derived from references... Figure 1 The described computing device / server 100 or reference Figure 2 The described device 210 performs this function. Furthermore... Figure 4 Actions 402-406 in the text can be... Figure 3 Actions that are executed before the action in box 302, action 408 can be an example of the action in box 302, and actions 410-418 can be examples of the action in box 306 and its subsequent actions.

[0042] In 402, the first device (e.g.) Figure 2 The illustrated device 210) begins to move towards the second device (e.g. Figure 2 The illustrated device 220) shares a screen. At 404, the number of queued images that the first device has encoded but the second device has not yet decoded is determined, and at 406 it is determined whether the number of queued images is greater than a predetermined number.

[0043] If so, the first device pauses image processing at 408 and waits for a period of time, then returns to 404 to continue determining the number of queued images until the number of queued images is less than a predetermined number, at which point the paused image processing ends. According to embodiments of the subject matter described herein, the total time elapsed for each pause capture is recorded as the "pause time" for each pause capture. If the pause time for each pause capture is too long, it may result in a low frame rate, making the screen-shared image less smooth and realistic.

[0044] If the number of queued images is less than a predetermined number, the first device continues to capture images at 410, for example, capturing the screen at a fixed number of frames per second, and encodes the captured images at 412. At 414, the first device sends the encoded images to the second device; optionally, the time elapsed for each image transmission, also known as the "delay time," can be recorded. At 416, it is determined whether to continue sharing the screen. If to continue, the process returns to box 404, and the screen sharing operation continues; otherwise, at 418, the first device stops sharing the screen with the second device.

[0045] Figure 5 A flowchart of a method 500 for adjusting quantization parameters associated with an image compression ratio, according to an embodiment of the subject matter described herein, is shown. It should be understood that method 500 can be derived from references... Figure 1 The described computing device / server 100 or reference Figure 2 The described device 210 performs this function. Furthermore... Figure 5 Actions 502-516 in the text can be used as a reference. Figure 2 The sub-actions of the action in the described box 304.

[0046] In step 502, the pause time for pausing image processing is determined, along with the delay time for sending the image. For example, it can be determined at a reference... Figure 4 The pause time experienced during each pause in the described box 408 (if there is no pause, the pause time can be set to zero by default), and determines the reference. Figure 4 The delay time for sending the current image in the described box 414.

[0047] At step 504, it is determined whether the pause time and delay time each exceed a threshold time. If one or both are satisfied, it means that the screen sharing delay is relatively long. At step 506, the quantization parameter value can be increased (e.g., incremented by 1 each time) to reduce latency by decreasing image quality and reducing the transmission bitrate. In some embodiments, the quantization parameter value can be increased only if the pause time is greater than the threshold time, without specifying the length of the delay time. Increasing the quantization parameter value results in a lower transmission bitrate and lower image quality. A lower transmission bitrate not only alleviates latency caused by insufficient network bandwidth but also speeds up the decoding at the receiving end, thereby simultaneously maintaining the frame rate and reducing latency.

[0048] If neither the pause time nor the delay time exceeds its threshold time, it indicates that the current latency of screen sharing is very small, and it is possible to consider whether to improve the image quality, that is, to reduce the quantization parameter value. At 508, it is determined whether the number of consecutively transmitted images (referred to as the "third number") that meets the condition that neither the pause time nor the delay time exceeds its threshold time is greater than a predetermined number (referred to as the "second predetermined number"). If the number of consecutive images is greater than the second predetermined number, then at 510 the quantization parameter value is reduced (e.g., by 1 each time).

[0049] If the number of sustained images is less than the second predetermined number, then at 512, it is determined whether the number of sustained images is greater than another predetermined number (referred to as the "third predetermined number," whose value is less than the second predetermined number), and simultaneously, it is determined whether the duration since the last increase in the quantization parameter value is greater than a predetermined time. If the conditions in block 512 are met, then at 514, the quantization parameter value is decreased (e.g., by 1 each time); otherwise, the quantization parameter value remains unchanged. It should be understood that although in Figure 5 In this context, action 508 is shown to be performed before action 512; however, action 508 may also be performed after action 512 or both may be performed substantially simultaneously.

[0050] In some cases, upper and lower limits can be set for the quantization parameter values, and adjustment of the quantization parameter values ​​can be stopped when the upper or lower limit is reached, thereby ensuring that the adjusted quantization parameter values ​​are always within a reasonable range. Therefore, according to the method 500 described in this paper, by using pause time, delay time, the number of continuous images, and duration to dynamically adjust the quantization parameter values ​​used to determine the compression ratio and transmission bitrate of the image, bitrate adjustment becomes more accurate and real-time, thereby effectively improving the user experience of screen sharing.

[0051] Figure 6 A flowchart of a screen sharing method 600 based on multiple codecs, according to an embodiment of the subject matter described herein, is shown. It should be understood that method 600 can be derived from references... Figure 1 The described computing device / server 100 or reference Figure 2 The described device 210 performs this function. Furthermore... Figure 6 Actions 602-606 in the text can be used as a reference. Figure 2 The sub-action of action 306 described.

[0052] In 602, in response to the detection in the first device (e.g.) Figure 2 The first and second images captured consecutively on the described device 210 are partially identical, and the changing regions in the second image that include content different from that in the first image are identified. For example, in some scenarios, only a portion of the screen changes over time, while other portions of the screen remain unchanged. In this case, only the changed regions need to be encoded, while the unchanging regions do not need to be encoded or transmitted.

[0053] For example, when playing a video in a browser, only the content of the video playback area changes, while other parts of the screen remain unchanged. In some embodiments, when network bandwidth is insufficient, the quantization parameter value of the changing area can be increased while the quantization parameter value of the unchanging area remains unchanged or decreases. This results in only the image quality of the changing area deteriorating, while the image quality of the unchanging area remains unchanged or improves. For example, if the unchanging area remains unchanged for a predetermined number of images, the quantization parameter value for the unchanging area can be decreased (e.g., decremented by 1). Those skilled in the art will understand that when network bandwidth and image encoding quality are sufficiently good, it is not necessary to detect changing areas on the screen, thereby reducing the computational complexity of the screen sharing process.

[0054] In some embodiments, the screen can be divided into multiple blocks (e.g., 16×16 blocks), and for each block, differences between blocks can be detected by pixel-by-pixel comparison or subsampled pixel comparison. In other embodiments, differences between images can also be compared using "dirty region" information provided by the operating system, where a "dirty region" refers to an area of ​​the screen where everything else remains unchanged except for the dirty region itself. In still other embodiments, the "dirty region" information provided by the operating system can be used to perform an initial comparison, followed by a pixel-by-pixel comparison to further compare the "dirty regions" and extract more precise areas of change.

[0055] At 604, a suitable codec for the changing region is selected from a set of codecs, wherein the set of codecs includes multiple codecs with different codec sizes. For example, the codec corresponding to the smallest codec region capable of covering the changing region is selected. In some embodiments, the set of codecs may predefine multiple codecs for different region sizes; for example, the multiple codecs in the set may correspond to encoding / decoding of the entire screen, encoding / decoding of a quarter screen, encoding / decoding of a sixteenth screen, and encoding / decoding of a fixed minimum region (such as 256×256 pixels, if it is smaller than a sixteenth screen size). As another example, the multiple codecs in the set may also correspond to encoding / decoding of the entire screen, encoding / decoding of the left half screen, encoding / decoding of the right half screen, encoding / decoding of a quarter screen, and so on. It should be understood that multiple codecs according to embodiments of the subject matter described in this disclosure may be pre-configured to encode / decode regions of various sizes.

[0056] In some embodiments, the screen size is assumed to be w × h pixels, where w represents the number of pixels in the width of the screen and h represents the number of pixels in the height of the screen. A first codec in the codec set can be configured for a size of w × h and is used to encode the entire screen. A second codec in the codec set can be configured for a size of w / 2 × h / 2, a third codec can be configured for a size of w / 4 × h / 4, and a fourth codec can be configured for a size of 256 × 256 pixels (if the size of the third codec is larger than the size of the fourth codec). In some embodiments, the transmitting end can select a codec with a resolution that is a multiple of 16.

[0057] At 606, the selected codec is used to encode the second image. Encoded region information needs to be recorded, including the start position and size of the encoded region for the selected codec. Then, during transmission, the encoded second image and the encoded region information are sent from the first device to the second device. In this way, the second device only receives and decodes image data for the encoded regions, and decodes and draws the received image data based on the encoded region information, thereby reducing the amount of data transmitted during screen sharing.

[0058] In some embodiments, when the coding regions of the encoders and decoders of the first image and the second image are the same size, if the coding region of the first image can cover the changing region, then the coding region information of the first image is used as the coding region information of the second image; otherwise, the coding region information of the second image is determined based on the changing region, for example, by placing the changing region at the middle position of the coding region. In some embodiments, the encoder and decoder can also determine the coding region information by selecting a suitable starting position (e.g., the upper left corner of the matrix coding region).

[0059] Optionally, when multiple small sub-variable regions exist on the screen, these small sub-variable regions can be merged to form a large variable region. Alternatively, in screen-sharing scenarios such as document input, where only the cursor position and word count position may change, instead of merging these small variable regions into a single large variable region, multiple small variable regions can be encoded separately. For example, multiple sub-variable regions can be extracted from the variable region, such that the area outside the multiple sub-variable regions remains unchanged, and the sum of multiple sub-encoded regions for each of the multiple sub-variable regions can be determined. If the sum of the multiple sub-encoded regions is less than a predetermined proportion (e.g., 50%) of the encoded region for the variable region, then multiple codecs for each of the multiple sub-variable regions are used to encode the image.

[0060] Therefore, according to an embodiment of the method 600 described herein, when only a portion of a continuous image is changed, by employing an appropriate encoder-decoder to encode a portion of the image, preprocessing time (e.g., color space conversion time), encoding time, transmission time, decoding time, post-processing time (e.g., inverse color space conversion time), and rendering time can be reduced.

[0061] Figures 7A-7D Example graphical user interfaces (GUIs) 700, 730, 750, and 780 for screen sharing, according to embodiments of the subject matter described herein, are shown. It should be understood that... Figures 7A-7D The examples are presented merely as examples of embodiments of the subject matter described herein and do not limit the scope of protection of the subject matter described herein.

[0062] Figure 7A An example GUI 700 for sharing a screen is shown, according to an embodiment of the subject matter described herein. For example, device 210 shares a slideshow in the screen with device 220, which is a text description and image introduction about a leaf shape, wherein the left side of the GUI 700 is a text area 710 and the right side is an image area 720. Figure 7BThe GUI 730 shown is an example image captured at the next moment. The upper part of image region 720 changes, while other regions remain unchanged. Therefore, the changed area of ​​GUI 730 relative to GUI 700 is determined to be the changed region 735. Assuming the smallest encoding region capable of covering the changed region 735 is encoding region 740, the codec corresponding to encoding region 740 is used to encode only the image content within encoding region 740 of GUI 730.

[0063] Figure 7C The GUI 750 shown is the image captured at the next moment. In this image, the lower part of image region 720 changes, while other regions remain unchanged. Therefore, the changed area of ​​GUI 760 relative to GUI 730 is the changed area 755. Assuming the smallest encoding region that can cover the changed area 755 is encoding region 760, the codec corresponding to encoding region 760 is used to encode only the image content within encoding region 760 of GUI 730. For example... Figure 7C As shown, since the coding region 740 of the previous moment cannot cover the current changed region 755, a new coding region 760 is determined.

[0064] Figure 7D The GUI 780 shown is an image captured at the next moment, where the lower part of the image in image region 720 changes, while other regions remain unchanged. Since the encoding region 760 from the previous moment can cover the currently changed region 755, encoding region 760 is still used. Furthermore, in GUI 780, part of the text in text region 710 also changes, thus including a changed region 785. The smallest encoding region capable of covering the changed region 785 is, for example, encoding region 790.

[0065] exist Figure 7D In this example, although only two small regions, namely regions 755 and 785, change, their content is significantly different due to their distance from each other. If a single encoding region is used, the entire screen would need to be encoded. However, if encoding regions are selected separately for the small changing regions, only regions 760 and 790 need to be encoded, their total size being less than half the entire screen. Therefore, according to embodiments of the subject matter described herein, two codecs of different sizes can be used to encode the two sub-encoding regions (e.g., encoding regions 760 and 790) separately, thereby further reducing the amount of data to be transmitted.

[0066] The methods and functions described herein can be performed, at least in part, by one or more hardware logic components. Examples, but not limited to, illustrative types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), etc.

[0067] Program code used to implement the methods described herein can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0068] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0069] Furthermore, while the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the topics described herein. Certain features described in the context of a single implementation can also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation can also be implemented individually or in any suitable sub-combination in multiple implementations.

[0070] The following are some example implementations of this disclosure.

[0071] In another aspect, a method is provided. The method includes: pausing image processing on the first device in response to a first predetermined condition being met between an image encoded by a first device and an image decoded by a second device; sharing a screen between the first device and the second device; processing including one or more of capturing, encoding, and transmitting the image; adjusting parameters associated with an image compression ratio in response to a second predetermined condition being met during the pause period; and encoding a new image captured on the first device using the adjusted parameters in response to resuming image processing on the first device.

[0072] In some embodiments, the first predetermined condition includes: the difference between a first number of images encoded by the first device and a second number of images decoded by the second device is greater than a predetermined number; or the difference between a first number of bits of images encoded by the first device and a second number of bits of images decoded by the second device is greater than a predetermined number of bits.

[0073] In some embodiments, the parameter is a quantization parameter, and adjusting the parameter associated with the image compression ratio includes: increasing the quantization parameter value in response to a pause time greater than a predetermined pause time.

[0074] In some embodiments, adjusting the parameter associated with the image compression ratio further includes: determining the delay time for a specific image to be transmitted from the first device to the second device; and reducing the quantization parameter value in response to a pause time being less than a predetermined pause time, a delay time being less than a predetermined delay time, and a third predetermined condition being met.

[0075] In some embodiments, the predetermined number is a first predetermined number, and reducing the quantization parameter value includes: determining a third number of images to be transmitted if the pause time is less than a predetermined pause time and the delay time is less than a predetermined delay time; and reducing the quantization parameter value in response to the third number being greater than a second predetermined number.

[0076] In some embodiments, reducing the quantization parameter value further includes: in response to a third number being less than a second predetermined number and greater than a third predetermined number: determining the length of time elapsed since the last increase in the quantization parameter value; and in response to the length of time reaching a predetermined threshold, reducing the quantization parameter value.

[0077] In some embodiments, encoding a new image captured on a first device using adjusted parameters includes: in response to detecting that a first image and a second image captured consecutively on the first device are partially identical, determining a variation region in the second image that includes content different from that in the first image; selecting a suitable codec for the variation region from a set of codecs, the set of codecs including a plurality of codecs with different codec sizes; and encoding the second image using the selected codec.

[0078] In some embodiments, the method further includes: sending an encoded second image and encoded region information from a first device to a second device, the encoded region information including the starting position and size of the encoded region of a selected codec.

[0079] In some embodiments, the method further includes: responding to the fact that the coding regions of the codecs of the first image and the second image are the same size; responding to the fact that the coding region of the first image can cover the changing region, using the coding region information of the first image as the coding region information of the second image; and responding to the fact that the coding region of the first image cannot cover the changing region, determining the coding region information of the second image based on the changing region.

[0080] In some embodiments, encoding a second image using a selected codec includes: extracting a plurality of sub-variable regions in a variable region such that regions in the variable region other than the plurality of sub-variable regions remain unchanged; determining the sum of a plurality of sub-coded regions for each of the plurality of sub-variable regions; and encoding the second image using a plurality of codecs for each of the plurality of sub-variable regions in response to the sum of the plurality of sub-coded regions being less than a predetermined proportion of the encoded regions for the variable region.

[0081] In one aspect, an electronic device is provided. The electronic device includes: a processing unit; and a memory coupled to the processing unit and storing instructions that, when executed by the processing unit, perform the following actions: pausing image processing on the electronic device in response to a first predetermined condition being met between an image encoded by the electronic device and an image decoded by another electronic device; sharing a screen with the other electronic device; and processing including one or more of capturing, encoding, and transmitting the image; adjusting parameters associated with an image compression ratio in response to a second predetermined condition being met; and encoding a new image captured on the electronic device using the adjusted parameters in response to resuming image processing on the electronic device.

[0082] In some embodiments, the first predetermined condition includes: the difference between a first number of images encoded by the electronic device and a second number of images decoded by another electronic device is greater than a predetermined number; or the difference between a first number of bits of images encoded by the electronic device and a second number of bits of images decoded by another electronic device is greater than a predetermined number of bits.

[0083] In some embodiments, the parameter is a quantization parameter, and adjusting the parameter associated with the image compression ratio includes: increasing the quantization parameter value in response to a pause time greater than a predetermined pause time.

[0084] In some embodiments, adjusting the parameters associated with the image compression ratio further includes: determining the delay time at which a particular image being transmitted is sent from one electronic device to another; and reducing the quantization parameter value in response to a pause time being less than a predetermined pause time, a delay time being less than a predetermined delay time, and a third predetermined condition being met.

[0085] In some embodiments, the predetermined number is a first predetermined number, and reducing the quantization parameter value includes: determining a third number of images to be transmitted if the pause time is less than a predetermined pause time and the delay time is less than a predetermined delay time; and reducing the quantization parameter value in response to the third number being greater than a second predetermined number.

[0086] In some embodiments, reducing the quantization parameter value further includes: in response to a third number being less than a second predetermined number and greater than a third predetermined number: determining the length of time elapsed since the last increase in the quantization parameter value; and in response to the length of time reaching a predetermined threshold, reducing the quantization parameter value.

[0087] In some embodiments, encoding a new image captured on an electronic device using adjusted parameters includes: in response to detecting that a first image and a second image captured consecutively on the electronic device are partially identical, determining a variation region in the second image that includes content different from that in the first image; selecting a suitable codec for the variation region from a set of codecs, the set of codecs including a plurality of codecs with different codec sizes; and encoding the second image using the selected codec.

[0088] In some embodiments, the action further includes: sending an encoded second image and encoded region information from an electronic device to another electronic device, the encoded region information including the starting position and size of the encoded region of the selected codec.

[0089] In some embodiments, the action further includes: responding to the fact that the coding regions of the codecs of the first image and the second image are the same size; responding to the fact that the coding region of the first image can cover the changing region, using the coding region information of the first image as the coding region information of the second image; and responding to the fact that the coding region of the first image cannot cover the changing region, determining the coding region information of the second image based on the changing region.

[0090] In some embodiments, encoding a second image using a selected codec includes: extracting a plurality of sub-variable regions in a variable region such that regions in the variable region other than the plurality of sub-variable regions remain unchanged; determining the sum of a plurality of sub-coded regions for each of the plurality of sub-variable regions; and encoding the second image using a plurality of codecs for each of the plurality of sub-variable regions in response to the sum of the plurality of sub-coded regions being less than a predetermined proportion of the encoded regions for the variable region.

[0091] In another aspect, a computer program product is provided. This computer program product is stored in a non-transient computer storage medium and includes machine-executable instructions that, when executed in a device, cause the device to: suspend image processing on the first device in response to a first predetermined condition being met between an image encoded by the first device and an image decoded by the second device; share a screen with the second device; and process one or more of the following: capturing, encoding, and transmitting the image; adjust parameters associated with an image compression ratio in response to a second predetermined condition being met during the pause period of the suspension; and encode a new image captured on the first device using the adjusted parameters in response to resuming image processing on the first device.

[0092] In some embodiments, the first predetermined condition includes: the difference between a first number of images encoded by the first device and a second number of images decoded by the second device is greater than a predetermined number; or the difference between a first number of bits of images encoded by the first device and a second number of bits of images decoded by the second device is greater than a predetermined number of bits.

[0093] In some embodiments, the parameter is a quantization parameter, and adjusting the parameter associated with the image compression ratio includes: increasing the quantization parameter value in response to a pause time greater than a predetermined pause time.

[0094] In some embodiments, adjusting the parameter associated with the image compression ratio further includes: determining the delay time for a specific image to be transmitted from the first device to the second device; and reducing the quantization parameter value in response to a pause time being less than a predetermined pause time, a delay time being less than a predetermined delay time, and a third predetermined condition being met.

[0095] In some embodiments, the predetermined number is a first predetermined number, and reducing the quantization parameter value includes: determining a third number of images to be transmitted if the pause time is less than a predetermined pause time and the delay time is less than a predetermined delay time; and reducing the quantization parameter value in response to the third number being greater than a second predetermined number.

[0096] In some embodiments, reducing the quantization parameter value further includes: in response to a third number being less than a second predetermined number and greater than a third predetermined number: determining the length of time elapsed since the last increase in the quantization parameter value; and in response to the length of time reaching a predetermined threshold, reducing the quantization parameter value.

[0097] In some embodiments, encoding a new image captured on a first device using adjusted parameters includes: in response to detecting that a first image and a second image captured consecutively on the first device are partially identical, determining a variation region in the second image that includes content different from that in the first image; selecting a suitable codec for the variation region from a set of codecs, the set of codecs including a plurality of codecs with different codec sizes; and encoding the second image using the selected codec.

[0098] In some embodiments, the machine-executable instructions, when executed in the device, further cause the device to: send an encoded second image and encoded region information from the first device to the second device, the encoded region information including the starting position and size of the encoded region of the selected codec.

[0099] In some embodiments, the machine-executable instructions, when executed in the device, further cause the device to: respond to the fact that the coding regions of the codecs of the first image and the second image are the same size; respond to the fact that the coding region of the first image can cover the changing region, use the coding region information of the first image as the coding region information of the second image; and respond to the fact that the coding region of the first image cannot cover the changing region, determine the coding region information of the second image based on the changing region.

[0100] In some embodiments, encoding a second image using a selected codec includes: extracting a plurality of sub-variable regions in a variable region such that regions in the variable region other than the plurality of sub-variable regions remain unchanged; determining the sum of a plurality of sub-coded regions for each of the plurality of sub-variable regions; and encoding the second image using a plurality of codecs for each of the plurality of sub-variable regions in response to the sum of the plurality of sub-coded regions being less than a predetermined proportion of the encoded regions for the variable region.

[0101] Although this disclosure has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method comprising: In response to a first predetermined condition being met between an image encoded by the first device and an image decoded by the second device, image processing is paused on the first device, and the first device shares its screen with the second device. The processing includes one or more of capturing, encoding, and transmitting the image. In response to the pause duration satisfying a second predetermined condition, the parameter associated with the image compression ratio is adjusted; and In response to recovering the processed image on the first device, the newly captured image on the first device is encoded using adjusted parameters. Encoding the new image captured on the first device using adjusted parameters includes: In response to detecting that a first image and a second image captured consecutively on the first device are partially identical, a region of variation in the second image that includes content different from that in the first image is determined. Select a suitable codec for the varying region from a set of codecs, the set including multiple predefined codecs for different region sizes; and The second image is encoded using the selected codec to obtain information about the encoded region. The process of using a selected codec to encode the second image to obtain information about the encoded region includes: Extract multiple sub-change regions from the change region, such that the region of the change region other than the multiple sub-change regions remains unchanged; Select multiple codecs from the set of codecs for the multiple sub-variable regions respectively; Determine the sum of multiple sub-coding regions corresponding to the selected multiple codecs; and In response to the fact that the sum of the plurality of sub-coded regions is less than a predetermined proportion of the coded region for the change region, the second image is encoded using a plurality of codecs for the plurality of sub-change regions respectively.

2. The method according to claim 1, wherein the first predetermined condition includes: The difference between the first number of images encoded by the first device and the second number of images decoded by the second device is greater than a predetermined number; or The difference between the first number of bits of the image encoded by the first device and the second number of bits of the image decoded by the second device is greater than a predetermined number of bits.

3. The method of claim 2, wherein the parameter is a quantization parameter, and adjusting the parameter associated with the image compression ratio includes: In response to the pause time being greater than the predetermined pause time, the quantization parameter value of the quantization parameter is increased.

4. The method of claim 3, wherein adjusting the parameter associated with the image compression ratio further comprises: Determine the delay time at which a specific image is transmitted from the first device to the second device; as well as In response to the pause time being less than the predetermined pause time, the delay time being less than the predetermined delay time, and the third predetermined condition being met, the value of the quantization parameter is reduced.

5. The method of claim 4, wherein the predetermined number is a first predetermined number, and reducing the quantization parameter value comprises: A third number of images to be transmitted under the condition that the pause time is less than the predetermined pause time and the delay time is less than the predetermined delay time is determined; as well as In response to the third number being greater than the second predetermined number, the quantization parameter value is reduced.

6. The method according to claim 5, wherein reducing the quantization parameter value further comprises: In response to the third number being less than the second predetermined number and greater than the third predetermined number: Determine the length of time elapsed since the last increase in the value of the quantization parameter; as well as In response to the time length reaching a predetermined threshold, the quantization parameter value is reduced.

7. The method according to claim 1, further comprising: The encoded second image and encoded region information are sent from the first device to the second device, the encoded region information including the starting position and size of the encoded region of the selected codec.

8. The method according to claim 7, further comprising: The encoding regions of the codecs for the first image and the second image are the same size: In response to the fact that the coded region of the first image can cover the changed region, the coded region information of the first image is used as the coded region information of the second image; as well as In response to the fact that the encoded region of the first image cannot cover the changed region, the encoded region information of the second image is determined based on the changed region.

9. An electronic device, comprising: Processing unit; A memory, coupled to the processing unit, stores instructions that, when executed by the processing unit, perform the following actions: In response to a first predetermined condition being met between an image encoded by the electronic device and an image decoded by another electronic device, image processing is paused on the electronic device, and the electronic device shares its screen with the other electronic device. The processing includes one or more of capturing, encoding, and transmitting the image. In response to the pause duration satisfying a second predetermined condition, the parameter associated with the image compression ratio is adjusted; and In response to recovering and processing images on the electronic device, newly captured images on the electronic device are encoded using adjusted parameters. Encoding new images captured on the electronic device using adjusted parameters includes: In response to detecting that a first image and a second image captured consecutively on the electronic device are partially identical, a region of variation in the second image that includes content different from that in the first image is determined. Select a suitable codec for the varying region from a set of codecs, the set including multiple predefined codecs for different region sizes; and The second image is encoded using the selected codec to obtain information about the encoded region. The process of using a selected codec to encode the second image to obtain information about the encoded region includes: Extract multiple sub-change regions from the change region, such that the region of the change region other than the multiple sub-change regions remains unchanged; Select multiple codecs from the set of codecs for the multiple sub-variable regions respectively; Determine the sum of multiple sub-coding regions corresponding to the selected multiple codecs; and In response to the fact that the sum of the plurality of sub-coded regions is less than a predetermined proportion of the coded region for the change region, the second image is encoded using a plurality of codecs for the plurality of sub-change regions respectively.

10. The electronic device of claim 9, wherein the first predetermined condition includes: The difference between the first number of images encoded by the electronic device and the second number of images decoded by the other electronic device is greater than a predetermined number; or The difference between the first number of bits of the image encoded by the electronic device and the second number of bits of the image decoded by the other electronic device is greater than a predetermined number of bits.

11. The electronic device of claim 10, wherein the parameter is a quantization parameter, and adjusting the parameter associated with the image compression ratio includes: In response to the pause time being greater than the predetermined pause time, the quantization parameter value of the quantization parameter is increased.

12. The electronic device of claim 11, wherein adjusting the parameter associated with the image compression ratio further comprises: Determine the delay time at which a specific image is transmitted from the electronic device to the other electronic device; as well as In response to the pause time being less than the predetermined pause time, the delay time being less than the predetermined delay time, and the third predetermined condition being met, the value of the quantization parameter is reduced.

13. The electronic device of claim 12, wherein the predetermined number is a first predetermined number, and reducing the quantization parameter value comprises: A third number of images to be transmitted under the condition that the pause time is less than the predetermined pause time and the delay time is less than the predetermined delay time is determined; as well as In response to the third number being greater than the second predetermined number, the quantization parameter value is reduced.

14. The electronic device of claim 13, wherein reducing the quantization parameter value further comprises: In response to the third number being less than the second predetermined number and greater than the third predetermined number: Determine the length of time elapsed since the last increase in the value of the quantization parameter; as well as In response to the time length reaching a predetermined threshold, the quantization parameter value is reduced.

15. The electronic device according to claim 9, wherein the action further includes: The electronic device transmits the encoded second image and encoded region information, including the starting position and size of the encoded region of the selected codec, to the other electronic device.

16. A computer program product stored in a non-transitory computer storage medium and comprising machine-executable instructions that, when executed in a first device, cause the first device to: In response to a first predetermined condition being met between an image encoded by the first device and an image decoded by the second device, image processing is paused on the first device, and the first device shares its screen with the second device. The processing includes one or more of capturing, encoding, and transmitting the image. In response to the pause duration satisfying a second predetermined condition, the parameter associated with the image compression ratio is adjusted; and In response to recovering the processed image on the first device, the newly captured image on the first device is encoded using adjusted parameters. Encoding the new image captured on the first device using adjusted parameters includes: In response to detecting that a first image and a second image captured consecutively on the first device are partially identical, a region of variation in the second image that includes content different from that in the first image is determined. Select a codec suitable for the changing region from a set of codecs, the set of codecs including multiple predefined codecs for different region sizes; as well as The second image is encoded using the selected codec to obtain information about the encoded region. The process of using a selected codec to encode the second image to obtain information about the encoded region includes: Extract multiple sub-change regions from the change region, such that the region of the change region other than the multiple sub-change regions remains unchanged; Select multiple codecs from the set of codecs for the multiple sub-variable regions respectively; Determine the sum of multiple sub-coding regions corresponding to the selected multiple codecs; and In response to the fact that the sum of the plurality of sub-coded regions is less than a predetermined proportion of the coded region for the change region, the second image is encoded using a plurality of codecs for the plurality of sub-change regions respectively.

17. The computer program product of claim 16, wherein the first predetermined condition includes: The difference between the first number of images encoded by the first device and the second number of images decoded by the second device is greater than a predetermined number; or The difference between the first number of bits of the image encoded by the first device and the second number of bits of the image decoded by the second device is greater than a predetermined number of bits.

18. The computer program product of claim 17, wherein the parameter is a quantization parameter, and adjusting the parameter associated with the image compression ratio includes: In response to the pause time being greater than the predetermined pause time, the quantization parameter value of the quantization parameter is increased.

19. The computer program product of claim 18, wherein adjusting the parameters associated with the image compression ratio further includes: Determine the delay time at which a specific image is transmitted from the first device to the second device; as well as In response to the pause time being less than the predetermined pause time, the delay time being less than the predetermined delay time, and the third predetermined condition being met, the value of the quantization parameter is reduced.

20. The computer program product of claim 19, wherein the predetermined number is a first predetermined number, and reducing the quantization parameter value comprises: A third number of images to be transmitted under the condition that the pause time is less than the predetermined pause time and the delay time is less than the predetermined delay time is determined; as well as In response to the third number being greater than the second predetermined number, the quantization parameter value is reduced.

21. The computer program product of claim 20, wherein reducing the quantization parameter value further comprises: In response to the third number being less than the second predetermined number and greater than the third predetermined number: Determine the length of time elapsed since the last increase in the value of the quantization parameter; as well as In response to the time length reaching a predetermined threshold, the quantization parameter value is reduced.

22. The computer program product of claim 16, wherein the machine-executable instructions, when executed in the first device, further cause the first device to: The encoded second image and encoded region information are sent from the first device to the second device, the encoded region information including the starting position and size of the encoded region of the selected codec.

23. The computer program product of claim 22, wherein the machine-executable instructions, when executed in the first device, further cause the first device to: The encoding regions of the codecs for the first image and the second image are the same size: In response to the fact that the encoded region of the first image can cover the changed region, the encoded region information of the first image is used as the encoded region information of the second image; and In response to the fact that the encoded region of the first image cannot cover the changed region, the encoded region information of the second image is determined based on the changed region.

Images