Image coding method and device, image decoding method and device, equipment and storage medium

Abstract

The invention discloses an image coding method and device, an image decoding method and device, equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a to-be-transmitted desktop image, and carrying out the foreground and background separation of the desktop image, so as to recognize a first rectangular region corresponding to an image background and a second rectangular region corresponding to an image foreground; coding the first rectangular region based on a first preset coding rule to obtain a first coding result; coding the second rectangular region based on a second preset coding rule to obtain a second coding result; and compressing the first coding result and the second coding result to obtain a target coding result, and transmitting the target coding result to a client. According to the invention, the data transmission quantity can be obviously reduced, and the consumption of network bandwidth is reduced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to image encoding methods, decoding methods, apparatus, devices, and storage media. Background Technology

[0002] In the field of Virtual Desktop Infrastructure (VDI), one of the core challenges is how to efficiently transmit the server-side desktop image to the client in real time to enable users to visualize and operate the remote desktop. Transmission efficiency directly affects the smoothness of the user experience, server-side resource consumption, and network bandwidth usage. Therefore, desktop images are typically compressed before transmission.

[0003] Current image coding methods mainly employ a holistic coding strategy, which processes the entire desktop image (including background and foreground) uniformly, such as through slider buffering or integral image matching to encode and transmit image blocks. However, in desktop images, the background area is usually composed of continuous identical pixels, while the foreground area is composed of discrete pixels, and the characteristics of the two are significantly different. The uniform coding strategy does not distinguish between these two characteristics, and redundant coding is still performed on large areas of repetitive background areas, resulting in excessively large amounts of encoded data and low coding efficiency.

[0004] Therefore, the aforementioned technical problems urgently need to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an image encoding method, decoding method, apparatus, device, and storage medium that can significantly reduce data transmission volume and lower network bandwidth consumption. The specific solution is as follows:

[0006] A first aspect of this application provides an image encoding method applied to a server, the method comprising:

[0007] The desktop image to be transmitted is acquired, and the foreground and background of the desktop image are separated to identify a first rectangular region corresponding to the background of the image and a second rectangular region corresponding to the foreground of the image.

[0008] The first rectangular region is encoded based on the first preset encoding rule to obtain the first encoding result;

[0009] The second rectangular region is encoded based on the second preset encoding rule to obtain the second encoding result;

[0010] The first encoding result and the second encoding result are compressed to obtain the target encoding result, and the target encoding result is transmitted to the client.

[0011] Optionally, encoding the first rectangular region based on a first preset encoding rule to obtain a first encoding result includes:

[0012] Record the first position information of the first rectangular region in the desktop image;

[0013] The target pixel value corresponding to the first rectangular region is determined, and the first position information and the target pixel value are used as the first encoding result after encoding the first rectangular region.

[0014] Optionally, encoding the second rectangular region based on a second preset encoding rule to obtain a second encoding result includes:

[0015] Record the second position information of the second rectangular region in the desktop image;

[0016] Calculate the hash value of the pixel data in the second rectangular region, and use the hash value as the region identifier;

[0017] Query whether the region identifier exists in the preset foreground region buffer and obtain the corresponding query results;

[0018] Based on the second location information and the query result, a second encoding result is obtained after encoding the second rectangular region.

[0019] Optionally, obtaining the second encoding result after encoding the second rectangular region based on the second location information and the query result includes:

[0020] If the region identifier exists in the foreground region buffer, then the second location information and the region identifier are used as the second encoding result after encoding the second rectangular region;

[0021] If the region identifier is not present in the foreground region buffer, the pixel data within the second rectangular region is compressed to obtain compressed data, and the second position information and the compressed data are used as the second encoding result after encoding the second rectangular region. The region identifier and the compressed data are stored in the foreground region buffer.

[0022] Optionally, compressing the pixel data within the second rectangular region to obtain compressed data includes:

[0023] If the pixel data within the second rectangular area is text data, then the pixel data within the second rectangular area is compressed using a preset lossless compression algorithm to obtain compressed data.

[0024] If the pixel data within the second rectangular area is image-type data, then a preset lossy compression algorithm is used to compress the pixel data within the second rectangular area to obtain compressed data.

[0025] A second aspect of this application provides an image encoding method applied to a client, the method comprising:

[0026] Receive the target encoding result transmitted from the server;

[0027] The target encoding result is decompressed to obtain a first encoding result and a second encoding result; the first encoding result is obtained by the server encoding a first rectangular region corresponding to the background of the image in the desktop image based on a first preset encoding rule; the second encoding result is obtained by the server encoding a second rectangular region corresponding to the foreground of the image in the desktop image based on a second preset encoding rule; the first rectangular region and the second rectangular region are rectangular regions obtained by the server after separating the foreground and background of the desktop image;

[0028] Based on the first encoding result, render the first rectangular area on the local canvas to obtain the image background;

[0029] Based on the second encoding result, the second rectangular area is rendered on a local canvas that has already been rendered with the image background to recover the desktop image.

[0030] Optionally, the first encoding result is the first position information and the target pixel value corresponding to the first rectangular region;

[0031] Accordingly, rendering the first rectangular region on the local canvas based on the first encoding result to obtain the image background includes:

[0032] The first rectangular area is rendered on the local canvas based on the first location information and the target pixel value to obtain the image background.

[0033] Optionally, the second encoding result is the second location information corresponding to the second rectangular region, as well as the region identifier or compressed data;

[0034] Accordingly, rendering the second rectangular region on a local canvas with the rendered image background based on the second encoding result to recover the desktop image includes:

[0035] If the second encoding result includes a region identifier, then pixel data corresponding to the second rectangular region is obtained from the preset foreground region buffer based on the region identifier;

[0036] If the second encoding result includes compressed data, then the compressed data is decompressed to obtain pixel data corresponding to the second rectangular region;

[0037] Based on the second location information and the pixel data corresponding to the second rectangular region, the second rectangular region is rendered on a local canvas that has already been rendered with the image background, so as to recover the desktop image.

[0038] A third aspect of this application provides an image encoding apparatus for use on a server, the apparatus comprising:

[0039] The foreground / background separation module is used to acquire the desktop image to be transmitted and to separate the foreground / background of the desktop image to identify a first rectangular region corresponding to the background of the image and a second rectangular region corresponding to the foreground of the image.

[0040] The first encoding module is used to encode the first rectangular region based on a first preset encoding rule to obtain a first encoding result;

[0041] The second encoding module is used to encode the second rectangular region based on the second preset encoding rule to obtain the second encoding result;

[0042] The transmission module is used to compress the first encoding result and the second encoding result to obtain the target encoding result, and transmit the target encoding result to the client.

[0043] A fourth aspect of this application provides an image decoding apparatus for use in a client, the apparatus comprising:

[0044] The receiving module is used to receive the target encoding result transmitted from the server.

[0045] The decompression module is used to decompress the target encoding result to obtain a first encoding result and a second encoding result; the first encoding result is the result obtained by the server encoding a first rectangular region corresponding to the background of the image in the desktop image based on a first preset encoding rule; the second encoding result is the result obtained by the server encoding a second rectangular region corresponding to the foreground of the image in the desktop image based on a second preset encoding rule; the first rectangular region and the second rectangular region are rectangular regions obtained by the server after separating the foreground and background of the desktop image;

[0046] The first rendering module is used to render the first rectangular area on the local canvas based on the first encoding result to obtain the image background;

[0047] The second rendering module is used to render the second rectangular area on a local canvas that has been rendered with the image background based on the second encoding result, so as to recover the desktop image.

[0048] A fifth aspect of this application provides an electronic device comprising a processor and a memory; wherein the memory is used to store a computer program loaded and executed by the processor to implement the aforementioned disclosed image encoding method.

[0049] A sixth aspect of this application provides an electronic device including a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the aforementioned disclosed image decoding method.

[0050] The seventh aspect of this application provides a computer-readable storage medium storing computer-executable instructions, which, when loaded and executed by a processor, implement the aforementioned image encoding method or image decoding method.

[0051] As can be seen, this application obtains the desktop image to be transmitted through the server, and performs foreground-background separation on the desktop image to identify a first rectangular region corresponding to the image background and a second rectangular region corresponding to the image foreground; encodes the first rectangular region based on a first preset encoding rule to obtain a first encoding result; encodes the second rectangular region based on a second preset encoding rule to obtain a second encoding result; compresses the first encoding result and the second encoding result to obtain a target encoding result, and transmits the target encoding result to the client.

[0052] Beneficial Effects: This application separates the foreground and background of a desktop image to identify a first rectangular region corresponding to the image background and a second rectangular region corresponding to the image foreground. Furthermore, this application encodes the first rectangular region using a first preset encoding rule and the second rectangular region using a second preset encoding rule. That is, considering the characteristic differences between the foreground and background regions in the desktop image, this application uses different encoding rules for each, thereby achieving more targeted and efficient encoding. Compared to overall encoding, this significantly reduces the total amount of encoded data and lowers the transmission burden. Finally, this application compresses the obtained overall encoding result before transmitting it to the client, further reducing data transmission volume and network bandwidth consumption. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0054] Figure 1 This is a flowchart of an image encoding method disclosed in this application;

[0055] Figure 2 This application discloses a flowchart of a server-side image encoding process.

[0056] Figure 3 This is a flowchart of an image decoding method disclosed in this application;

[0057] Figure 4 This application discloses an interaction flowchart between a server and a client.

[0058] Figure 5 This application discloses a client-side image decoding process flowchart;

[0059] Figure 6 This is a schematic diagram of the structure of an image encoding device disclosed in this application;

[0060] Figure 7 This is a schematic diagram of the structure of an image decoding device disclosed in this application;

[0061] Figure 8 This is a structural diagram of an electronic device disclosed in this application;

[0062] Figure 9 This is another electronic device structure diagram disclosed in this application. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] In existing technologies, image encoding methods mainly employ a holistic encoding strategy, which uniformly processes the entire desktop image (including background and foreground), such as encoding and transmitting image blocks through methods like slider buffering and integral image matching. However, in desktop images, the background area is typically composed of continuous identical pixels, while the foreground area is composed of discrete pixels, with significant differences in their characteristics. The unified encoding strategy fails to distinguish between these two characteristics, resulting in redundant encoding of large, repetitive background areas, leading to excessively large amounts of encoded data and low encoding efficiency. Therefore, this application discloses an image encoding method, decoding method, apparatus, device, and storage medium that can significantly reduce data transmission volume and lower network bandwidth consumption.

[0065] Figure 1This is a flowchart illustrating an image encoding method provided in an embodiment of this application, applied to a server. See also... Figure 1 As shown, the image encoding method includes:

[0066] Step S11: Obtain the desktop image to be transmitted, and perform foreground-background separation on the desktop image to identify a first rectangular region corresponding to the image background and a second rectangular region corresponding to the image foreground.

[0067] In this embodiment, the server first acquires the desktop image to be transmitted. In scenarios such as remote desktops and cloud desktops, the server captures the desktop image currently displayed in real time. It is understood that remote desktop images in office scenarios generally have the following characteristics: these images are generated by computers and are non-natural images, so the number of colors is relatively small. In addition, there are many text, blank areas, and continuous color pixels in the image.

[0068] Therefore, for this type of image, this application requires foreground-background separation of the desktop image to identify a first rectangular region (i.e., the background rectangular region) corresponding to the image background and a second rectangular region (i.e., the foreground rectangular region) corresponding to the image foreground, so that the first and second rectangular regions can be encoded one by one subsequently. It should be noted that both the background rectangular region and the foreground rectangular region are axis-aligned rectangular regions, that is, the four sides of the rectangle are parallel to the coordinate axes of the image coordinate system; among them, the pixels in the foreground rectangular region are mostly different, so the foreground rectangular region is an axis-aligned rectangular region composed of discontinuous identical pixels, while all pixels in the background rectangular region are the same, so the background rectangular region is an axis-aligned rectangular region composed of continuous identical pixels.

[0069] In a specific implementation, when performing foreground-background separation on a desktop image, the image can first be divided into a series of largest, axis-aligned background rectangles with continuous internal pixel colors, thus obtaining the first rectangular region. Then, the remaining regions that do not satisfy this continuity are divided into foreground rectangles, thus obtaining the second rectangular region. During this process, a minimum rectangle size (e.g., 4×4 pixels or 8×8 pixels) needs to be set as one of the termination conditions for recursive segmentation, thereby preventing infinite subdivision and avoiding the generation of a large number of meaningless 1×1 pixel small rectangles. The specific segmentation process can be as follows:

[0070] 1. Initialization: Use the entire desktop image as the first candidate region to be processed;

[0071] 2. Determine candidate regions:

[0072] If the area is determined to be a solid color, then mark it as a background rectangle;

[0073] If the area is not a solid color, check its size:

[0074] If the minimum segmentation granularity has been reached, then it is forcibly marked as a foreground rectangle;

[0075] If the minimum granularity has not yet been reached, the region is divided into four sub-regions, and these four sub-regions are added to the processing queue as new candidate regions.

[0076] 3. Iteration: Repeat step 2 until all candidate regions have been processed.

[0077] Furthermore, to reduce the number of background rectangles, all background rectangles can be traversed, and adjacent rectangles with the same color can be merged into a larger rectangle, which can significantly reduce the number of background rectangles that need to be transmitted.

[0078] Step S12: Encode the first rectangular region based on the first preset encoding rule to obtain the first encoding result.

[0079] In this embodiment, the first rectangular region is encoded using a first preset encoding rule to obtain a first encoding result.

[0080] Specifically, encoding the first rectangular region based on a first preset encoding rule to obtain a first encoding result includes: recording the first position information of the first rectangular region in the desktop image; determining the target pixel value corresponding to the first rectangular region; and using the first position information and the target pixel value as the first encoding result after encoding the first rectangular region.

[0081] That is, such as Figure 2 As shown, this application first needs to record the position of the first rectangular region in the desktop image to obtain first position information, which can be the coordinates of the upper left and lower right corners of the first rectangular region. Furthermore, since the first rectangular region is an axis-aligned rectangular region composed of consecutive identical pixels, only its corresponding target pixel value needs to be recorded to complete the encoding process of the first rectangular region. That is, the first encoding result corresponding to the first rectangular region is the first position information and the target pixel value.

[0082] Step S13: Encode the second rectangular region based on the second preset encoding rule to obtain the second encoding result.

[0083] In this embodiment, the second rectangular region is encoded using a second preset encoding rule to obtain a second encoding result.

[0084] Specifically, the step of encoding the second rectangular region based on the second preset encoding rule to obtain the second encoding result includes: recording the second position information of the second rectangular region in the desktop image; calculating a hash value for the pixel data of the second rectangular region and using the hash value as a region identifier; querying whether the region identifier exists in a preset foreground region cache and obtaining the corresponding query result; and obtaining the second encoding result after encoding the second rectangular region based on the second position information and the query result.

[0085] That is, such as Figure 2 As shown, this application also needs to record the position of the second rectangular region in the desktop image to obtain the second position information, which can specifically be the coordinates of the upper left and lower right corners of the second rectangular region. Furthermore, since the second rectangular region is an axis-aligned rectangular region composed of discontinuous identical pixels (i.e., the pixels within this region are not identical), using the same processing method as the first rectangular region would be meaningless. Therefore, this application first needs to calculate the hash value of the pixel data of the second rectangular region and use the hash value as the region identifier (i.e., ID) of the rectangular region. To avoid repeatedly compressing the same rectangular region, this application pre-sets a foreground region cache to store data related to the foreground rectangular region. Therefore, this application queries the foreground region cache to determine if the region identifier exists and obtains the corresponding query result. Then, based on the second position information and the query result, it obtains the second encoding result after encoding the second rectangular region. That is, considering the feature differences between the foreground and background regions in the desktop image, this application uses different encoding rules for encoding, thereby achieving more targeted and efficient encoding. Compared to overall encoding, this significantly reduces the total amount of encoded data and lowers the transmission burden.

[0086] It should be noted that the purpose of calculating the hash value of the pixel data in the second rectangular region in this application is to generate a unique and fast identifier for the foreground rectangular region, which is used for fast lookup and matching in the cache. The advantage of this method is that it is not sensitive to position and is only related to pixel content. In a specific implementation, the pixel data in the rectangular region can be expanded into a continuous byte array in row-major order, and this byte array can be used as input to pass to the selected hash algorithm to calculate a fixed-length hash value, which is the region identifier of the foreground rectangular region. This application does not limit the type of hash algorithm; for example, MD5 (Message-Digest Algorithm 5) or SHA-1 (Secure Hash Algorithm 1) can be used.

[0087] Understandably, traditional slider caching schemes divide the desktop image into multiple fixed-size pixel blocks (i.e., sliders) and cache each transmitted block. When transmitting a new frame, the server uses a sliding window to scan the current frame, searching for image regions that match existing blocks in the cache. If a match is found, only the cache index (or ID) of that block is transmitted, rather than its complete pixel data, thus saving bandwidth. However, the sliding window requires pixel-by-pixel sliding and matching across the entire frame, a computationally intensive operation. Furthermore, to maintain a high cache hit rate, a large number of image blocks and their feature information need to be stored, leading to significant memory consumption. Additionally, when the user scrolls the page, the image content shifts on the screen. The sliding window scheme is position-sensitive; unless a full-image search is performed, it's difficult to identify cached content, resulting in a decreased cache hit rate.

[0088] In this application, when a foreground block (such as an icon) moves on the screen, its hash value remains unchanged, and it can still hit the cache. Unlike slider caching schemes, which may fail to match or mismatch due to position changes, this solves the problem of image rendering errors caused by cache conflicts and ensures image accuracy. Furthermore, this solution is naturally friendly to common office scenarios such as scrolling and dragging, maintaining a high cache hit rate and is suitable for all types of desktop image scenarios. In addition, in typical office scenarios, the area of ​​the foreground region (such as text or icons) is much smaller than the entire image. Therefore, the memory usage of the foreground region historical cache is naturally much smaller than that of existing solutions that require caching a large number of full image blocks.

[0089] Specifically, obtaining the second encoding result after encoding the second rectangular region based on the second location information and the query result includes: if the region identifier exists in the foreground region buffer, then the second location information and the region identifier are used as the second encoding result after encoding the second rectangular region; if the region identifier does not exist in the foreground region buffer, then the pixel data in the second rectangular region is compressed to obtain compressed data, and the second location information and the compressed data are used as the second encoding result after encoding the second rectangular region, and the region identifier and the compressed data are stored in the foreground region buffer.

[0090] That is, in one specific implementation, if the region identifier exists in the foreground region buffer, it means that the same foreground region rectangle information has been sent before. Therefore, the second position information and the region identifier are used as the second encoding result after encoding the second rectangle region, and there is no need to retransmit the compressed data corresponding to the second rectangle region.

[0091] In another specific implementation, if the region identifier is not present in the foreground region buffer, it indicates that this is a completely new foreground region rectangle. Therefore, the pixel data within the second rectangle region needs to be compressed to obtain compressed data. The second position information and the compressed data are then used as the second encoding result after encoding the second rectangle region. Simultaneously, the region identifier and the compressed data obtained in this calculation also need to be stored in the foreground region buffer so that when the same region identifier is matched subsequently, only the region identifier needs to be transmitted.

[0092] It should also be noted that the compression of pixel data within the second rectangular region to obtain compressed data includes: if the pixel data within the second rectangular region is text data, then a preset lossless compression algorithm is used to compress the pixel data within the second rectangular region to obtain compressed data; if the pixel data within the second rectangular region is image data, then a preset lossy compression algorithm is used to compress the pixel data within the second rectangular region to obtain compressed data. That is, this application can classify and compress the pixel data within the second rectangular region to obtain corresponding compressed data by using appropriate compression algorithms for different types of pixel data. Specifically, if the pixel data within the second rectangular region is text data, since text needs to maintain clarity and pixel distortion is not allowed, a lossless compression algorithm can be used for compression, such as LZ77 or Huffman coding. Furthermore, if the pixel data within the second rectangular region is image data, since the human eye is not very sensitive to image detail loss, lossy compression (such as the JPEG algorithm) can discard some visual redundancy information to obtain a compression ratio much higher than lossless compression. If lossless compression is used on images, the file size will be very large.

[0093] Step S14: Compress the first encoding result and the second encoding result to obtain the target encoding result, and transmit the target encoding result to the client.

[0094] In this embodiment, as Figure 2 As shown, after all foreground matrix regions and background rectangular regions have been processed, the final overall encoding result is compressed again before being transmitted to the client, thereby further reducing data transmission volume and network bandwidth consumption. This embodiment does not limit the compression algorithm used here; for example, common lossless compression algorithms can also be used.

[0095] As can be seen, this application separates the foreground and background of the desktop image to identify a first rectangular region corresponding to the image background and a second rectangular region corresponding to the image foreground. Furthermore, this application encodes the first rectangular region using a first preset encoding rule and the second rectangular region using a second preset encoding rule. That is, considering the feature differences between the foreground and background regions in the desktop image, this application uses different encoding rules for each, thereby achieving more targeted and efficient encoding. Compared to overall encoding, this significantly reduces the total amount of encoded data and lowers the transmission burden. Finally, this application compresses the obtained overall encoding result before transmitting it to the client, further reducing data transmission volume and network bandwidth consumption.

[0096] Figure 3 This is a flowchart illustrating an image decoding method provided in an embodiment of this application, applied to a client. See also... Figure 3 As shown, the image decoding method includes:

[0097] Step S21: Receive the target encoding result transmitted by the server.

[0098] In this embodiment, the receiving server transmits the target encoding result after encoding the desktop image using the aforementioned disclosed image encoding method. The interaction process between the server and the client is as follows: Figure 4 As shown, the server captures the desktop image and processes it using the image encoding module and image caching module to obtain the final encoded result. Then, it is transmitted to the client's network receiving module through the network sending module. The client then processes the received target encoded result through its local image decoding module and image caching module to obtain the decoded original image data, and finally renders it locally.

[0099] Step S22: Decompress the target encoding result to obtain a first encoding result and a second encoding result; the first encoding result is the result obtained by the server encoding the first rectangular region corresponding to the background of the image in the desktop image based on the first preset encoding rule; the second encoding result is the result obtained by the server encoding the second rectangular region corresponding to the foreground of the image in the desktop image based on the second preset encoding rule; the first rectangular region and the second rectangular region are rectangular regions obtained by the server after separating the foreground and background of the desktop image.

[0100] In this embodiment, the server encodes the first rectangular region corresponding to the background of the desktop image based on a first preset encoding rule, obtaining a first encoding result; and encodes the second rectangular region corresponding to the foreground of the desktop image based on a second preset encoding rule, obtaining a second encoding result. Finally, the first and second encoding results are compressed together to obtain the final target encoding result. Therefore, the client needs to use a corresponding decompression algorithm to decompress the target encoding result to obtain the first and second encoding results.

[0101] Step S23: Render the first rectangular area on the local canvas based on the first encoding result to obtain the image background.

[0102] In this embodiment, the first rectangular area is first rendered on the local canvas based on the first encoding result to obtain the image background. It is understood that, to reduce the number of background areas, the server-side will merge background areas while ignoring foreground areas. Therefore, in this embodiment, the background area must be rendered first, followed by the foreground area, to avoid rendering errors where background color blocks incorrectly cover foreground content.

[0103] Wherein, the first encoding result is the first position information and target pixel value corresponding to the first rectangular region; correspondingly, rendering the first rectangular region on the local canvas based on the first encoding result to obtain the image background includes: rendering the first rectangular region on the local canvas based on the first position information and the target pixel value to obtain the image background. That is, after obtaining the position information and pixel value of the background rectangular region, the background rectangular region can be rendered on the local canvas. For example, the corresponding region can be found first based on the position information, and then the region can be filled with the color corresponding to the target pixel value.

[0104] Step S24: Based on the second encoding result, render the second rectangular area on the local canvas that has been rendered with the image background to recover the desktop image.

[0105] In this embodiment, on a local canvas that has already been rendered with an image background, a second rectangular area is rendered based on the second encoding result to recover the desktop image.

[0106] Wherein, the second encoding result is the second position information corresponding to the second rectangular region and the region identifier or compressed data; correspondingly, the step of rendering the second rectangular region on a local canvas with the rendered image background based on the second encoding result to recover the desktop image includes: if the second encoding result includes a region identifier, then obtaining pixel data corresponding to the second rectangular region from a preset foreground region cache according to the region identifier; if the second encoding result includes compressed data, then decompressing the compressed data to obtain pixel data corresponding to the second rectangular region; and rendering the second rectangular region on a local canvas with the rendered image background based on the second position information and the pixel data corresponding to the second rectangular region to recover the desktop image.

[0107] That is, such as Figure 5 As shown, besides the fixed second position information, the second encoding result also includes either a region identifier or compressed data corresponding to the foreground rectangular region. It's also important to note that the client maintains a foreground region cache, synchronized with the server's foreground region cache, which stores the associated region identifier and compressed data. If the second encoding result includes a region identifier, it means that data for the same foreground rectangular region has already been received. Therefore, the pixel data corresponding to the second rectangular region can be directly retrieved from the local foreground region cache based on the region identifier. If the second encoding result includes compressed data, the compressed data is decompressed to obtain the pixel data corresponding to the second rectangular region. Simultaneously, the compressed data and the region identifier are stored in the local foreground region cache. Alternatively, the pixel data can be stored directly in the foreground region cache. Finally, based on the second position information and the pixel data corresponding to the second rectangular region, the second rectangular region is rendered on the local canvas with the already rendered image background to reconstruct the desktop image.

[0108] As can be seen, after receiving the target encoding result transmitted from the server, this application decompresses the target encoding result to obtain the first encoding result and the second encoding result. It is understandable that the server side, considering the feature differences between the foreground and background regions in the desktop image, uses different encoding rules to encode the rectangular regions corresponding to the image foreground and background respectively, thereby achieving more targeted and efficient encoding. Compared to overall encoding, this significantly reduces the total amount of encoded data and lowers the transmission burden. Furthermore, the client first renders the image background on its local canvas and then overlays it with the rendered foreground, thus perfectly reproducing the desktop image from the server and avoiding rendering errors where background color blocks incorrectly cover foreground content.

[0109] See Figure 6 As shown in the figure, this application also discloses an image encoding device applied to a server, the device comprising:

[0110] The foreground / background separation module 01 is used to acquire the desktop image to be transmitted and to perform foreground / background separation on the desktop image to identify a first rectangular region corresponding to the background of the image and a second rectangular region corresponding to the foreground of the image.

[0111] The first encoding module 02 is used to encode the first rectangular region based on a first preset encoding rule to obtain a first encoding result;

[0112] The second encoding module 03 is used to encode the second rectangular region based on the second preset encoding rule to obtain the second encoding result;

[0113] The transmission module 04 is used to compress the first encoding result and the second encoding result to obtain the target encoding result, and transmit the target encoding result to the client.

[0114] As can be seen, this application separates the foreground and background of the desktop image to identify a first rectangular region corresponding to the image background and a second rectangular region corresponding to the image foreground. Furthermore, this application encodes the first rectangular region using a first preset encoding rule and the second rectangular region using a second preset encoding rule. That is, considering the feature differences between the foreground and background regions in the desktop image, this application uses different encoding rules for each, thereby achieving more targeted and efficient encoding. Compared to overall encoding, this significantly reduces the total amount of encoded data and lowers the transmission burden. Finally, this application compresses the obtained overall encoding result before transmitting it to the client, further reducing data transmission volume and network bandwidth consumption.

[0115] In some specific embodiments, the first encoding module 02 specifically includes:

[0116] The first position recording unit is used to record the first position information of the first rectangular area in the desktop image;

[0117] The first encoding unit is used to determine the target pixel value corresponding to the first rectangular region, and to use the first position information and the target pixel value as the first encoding result after encoding the first rectangular region.

[0118] In some specific embodiments, the second encoding module 03 may specifically include:

[0119] The second position recording unit is used to record the second position information of the second rectangular region in the desktop image;

[0120] A hash calculation unit is used to calculate a hash value for the pixel data of the second rectangular region and use the hash value as a region identifier.

[0121] The query unit is used to query whether the region identifier exists in the preset foreground region buffer and obtain the corresponding query result;

[0122] The second encoding unit is used to obtain a second encoding result after encoding the second rectangular region based on the second location information and the query result.

[0123] In some specific embodiments, the second encoding unit specifically includes:

[0124] The first result determination unit is used to, if the region identifier exists in the foreground region buffer, use the second position information and the region identifier as the second encoding result after encoding the second rectangular region;

[0125] The second result determination unit is configured to, if the region identifier is not present in the foreground region buffer, compress the pixel data within the second rectangular region to obtain compressed data, use the second position information and the compressed data as the second encoding result after encoding the second rectangular region, and store the region identifier and the compressed data in the foreground region buffer.

[0126] In some specific embodiments, the second result determining unit further includes:

[0127] The first compression unit is used to compress the pixel data in the second rectangular area using a preset lossless compression algorithm to obtain compressed data if the pixel data in the second rectangular area is text data.

[0128] The second compression unit is used to compress the pixel data in the second rectangular area using a preset lossy compression algorithm to obtain compressed data if the pixel data in the second rectangular area is image-type data.

[0129] See Figure 7 As shown in the illustration, this application also discloses an image decoding device for use on a client side, the device comprising:

[0130] Receiver module 11 is used to receive the target encoding result transmitted by the server;

[0131] The decompression module 12 is used to decompress the target encoding result to obtain a first encoding result and a second encoding result; the first encoding result is the result obtained by the server encoding a first rectangular region corresponding to the background of the image in the desktop image based on a first preset encoding rule; the second encoding result is the result obtained by the server encoding a second rectangular region corresponding to the foreground of the image in the desktop image based on a second preset encoding rule; the first rectangular region and the second rectangular region are rectangular regions obtained by the server after separating the foreground and background of the desktop image;

[0132] The first rendering module 13 is used to render the first rectangular area on the local canvas based on the first encoding result to obtain the image background;

[0133] The second rendering module 14 is used to render the second rectangular area on a local canvas that has been rendered with the image background based on the second encoding result, so as to recover the desktop image.

[0134] As can be seen, after receiving the target encoding result transmitted from the server, this application decompresses the target encoding result to obtain the first encoding result and the second encoding result. It is understandable that the server side, considering the feature differences between the foreground and background regions in the desktop image, uses different encoding rules to encode the rectangular regions corresponding to the image foreground and background respectively, thereby achieving more targeted and efficient encoding. Compared to overall encoding, this significantly reduces the total amount of encoded data and lowers the transmission burden. Furthermore, the client first renders the image background on its local canvas and then overlays it with the rendered foreground, thus perfectly reproducing the desktop image from the server and avoiding rendering errors where background color blocks incorrectly cover foreground content.

[0135] In some specific embodiments, the first encoding result is the first position information and the target pixel value corresponding to the first rectangular region;

[0136] Accordingly, the first rendering module 13 is specifically used to render the first rectangular area on the local canvas based on the first position information and the target pixel value to obtain the image background.

[0137] In some specific embodiments, the second encoding result is the second location information corresponding to the second rectangular region, as well as the region identifier or compressed data;

[0138] Accordingly, the second rendering module 14 may specifically include:

[0139] The first acquisition unit is configured to, if the second encoding result includes a region identifier, acquire pixel data corresponding to the second rectangular region from a preset foreground region buffer based on the region identifier;

[0140] The second acquisition unit is used to decompress the compressed data if the second encoding result includes compressed data, so as to obtain pixel data corresponding to the second rectangular region.

[0141] The rendering unit is used to render the second rectangular area on a local canvas that has been rendered with the image background based on the second position information and the pixel data corresponding to the second rectangular area, so as to recover the desktop image.

[0142] Furthermore, embodiments of this application also provide an electronic device. Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0143] Figure 8 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the image encoding method disclosed in any of the foregoing embodiments.

[0144] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0145] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0146] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the image encoding method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include image quality control strategies collected by the electronic device 20, etc.

[0147] This application also provides another electronic device. Figure 9 This is a structural diagram of an electronic device 30 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0148] Figure 9 This is a schematic diagram of the structure of an electronic device 30 provided in an embodiment of this application. The electronic device 30 may specifically include: at least one processor 31, at least one memory 32, a power supply 33, a communication interface 34, an input / output interface 35, and a communication bus 36. The memory 32 stores a computer program, which is loaded and executed by the processor 31 to implement the relevant steps in the image decoding method disclosed in any of the foregoing embodiments.

[0149] In this embodiment, the power supply 33 is used to provide operating voltage for each hardware device on the electronic device 30; the communication interface 34 can create a data transmission channel between the electronic device 30 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 35 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0150] In addition, the memory 32, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 321, computer program 322 and data 323, etc., and the storage method can be temporary storage or permanent storage.

[0151] The operating system 321 manages and controls the various hardware devices on the electronic device 30 and the computer program 322 to enable the processor 31 to perform calculations and processing on the massive amount of data 323 in the memory 32. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 322, in addition to including a computer program capable of performing the image decoding method executed by the electronic device 30 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 323 may include image quality control strategies collected by the electronic device 30, etc.

[0152] Furthermore, this application also discloses a storage medium storing a computer program, which, when loaded and executed by a processor, implements the steps of the image encoding method or image decoding method disclosed in any of the foregoing embodiments.

[0153] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0154] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0155] The image encoding method, decoding method, apparatus, device, and storage medium provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An image encoding method, characterized in that, Applied to the server side, the method includes: The desktop image to be transmitted is acquired, and the foreground and background of the desktop image are separated to identify a first rectangular region corresponding to the background of the image and a second rectangular region corresponding to the foreground of the image. The first rectangular region is encoded based on the first preset encoding rule to obtain the first encoding result; The second rectangular region is encoded based on the second preset encoding rule to obtain the second encoding result; The first encoding result and the second encoding result are compressed to obtain the target encoding result, and the target encoding result is transmitted to the client.

2. The image encoding method according to claim 1, characterized in that, The step of encoding the first rectangular region based on a first preset encoding rule to obtain a first encoding result includes: Record the first position information of the first rectangular region in the desktop image; The target pixel value corresponding to the first rectangular region is determined, and the first position information and the target pixel value are used as the first encoding result after encoding the first rectangular region.

3. The image encoding method according to claim 1, characterized in that, The step of encoding the second rectangular region based on the second preset encoding rule to obtain the second encoding result includes: Record the second position information of the second rectangular region in the desktop image; Calculate the hash value of the pixel data in the second rectangular region, and use the hash value as the region identifier; Query whether the region identifier exists in the preset foreground region buffer and obtain the corresponding query results; Based on the second location information and the query result, a second encoding result is obtained after encoding the second rectangular region.

4. The image encoding method according to claim 3, characterized in that, The step of obtaining a second encoding result based on the second location information and the query result, after encoding the second rectangular region, includes: If the region identifier exists in the foreground region buffer, then the second location information and the region identifier are used as the second encoding result after encoding the second rectangular region; If the region identifier is not present in the foreground region buffer, the pixel data within the second rectangular region is compressed to obtain compressed data, and the second position information and the compressed data are used as the second encoding result after encoding the second rectangular region. The region identifier and the compressed data are stored in the foreground region buffer.

5. The image encoding method according to claim 3, characterized in that, The step of compressing the pixel data within the second rectangular region to obtain compressed data includes: If the pixel data within the second rectangular area is text data, then the pixel data within the second rectangular area is compressed using a preset lossless compression algorithm to obtain compressed data. If the pixel data within the second rectangular area is image-type data, then a preset lossy compression algorithm is used to compress the pixel data within the second rectangular area to obtain compressed data.

6. An image decoding method, characterized in that, Applied to a client, the method includes: Receive the target encoding result transmitted from the server; The target encoding result is decompressed to obtain a first encoding result and a second encoding result; the first encoding result is obtained by the server encoding a first rectangular region corresponding to the background of the image in the desktop image based on a first preset encoding rule; the second encoding result is obtained by the server encoding a second rectangular region corresponding to the foreground of the image in the desktop image based on a second preset encoding rule; the first rectangular region and the second rectangular region are rectangular regions obtained by the server after separating the foreground and background of the desktop image; Based on the first encoding result, render the first rectangular area on the local canvas to obtain the image background; Based on the second encoding result, the second rectangular area is rendered on a local canvas that has already been rendered with the image background to recover the desktop image.

7. The image decoding method according to claim 6, characterized in that, The first encoding result is the first position information and the target pixel value corresponding to the first rectangular region; Accordingly, rendering the first rectangular region on the local canvas based on the first encoding result to obtain the image background includes: The first rectangular area is rendered on the local canvas based on the first location information and the target pixel value to obtain the image background.

8. The image decoding method according to claim 6, characterized in that, The second encoding result is the second location information corresponding to the second rectangular region, as well as the region identifier or compressed data; Accordingly, rendering the second rectangular region on a local canvas with the rendered image background based on the second encoding result to recover the desktop image includes: If the second encoding result includes a region identifier, then pixel data corresponding to the second rectangular region is obtained from the preset foreground region buffer based on the region identifier; If the second encoding result includes compressed data, then the compressed data is decompressed to obtain pixel data corresponding to the second rectangular region; Based on the second location information and the pixel data corresponding to the second rectangular region, the second rectangular region is rendered on a local canvas that has already been rendered with the image background, so as to recover the desktop image.

9. An image encoding device, characterized in that, Applied to the server side, the device includes: The foreground / background separation module is used to acquire the desktop image to be transmitted and to separate the foreground / background of the desktop image to identify a first rectangular region corresponding to the background of the image and a second rectangular region corresponding to the foreground of the image. The first encoding module is used to encode the first rectangular region based on a first preset encoding rule to obtain a first encoding result; The second encoding module is used to encode the second rectangular region based on the second preset encoding rule to obtain the second encoding result; The transmission module is used to compress the first encoding result and the second encoding result to obtain the target encoding result, and transmit the target encoding result to the client.

10. An image decoding device, characterized in that, Applied to a client, the device includes: The receiving module is used to receive the target encoding result transmitted from the server. The decompression module is used to decompress the target encoding result to obtain a first encoding result and a second encoding result; the first encoding result is the result obtained by the server encoding a first rectangular region corresponding to the background of the image in the desktop image based on a first preset encoding rule; the second encoding result is the result obtained by the server encoding a second rectangular region corresponding to the foreground of the image in the desktop image based on a second preset encoding rule; the first rectangular region and the second rectangular region are rectangular regions obtained by the server after separating the foreground and background of the desktop image; The first rendering module is used to render the first rectangular area on the local canvas based on the first encoding result to obtain the image background; The second rendering module is used to render the second rectangular area on a local canvas that has been rendered with the image background based on the second encoding result, so as to recover the desktop image.

11. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein: The memory is used to store computer programs; The computer program is loaded and executed by the processor to implement the image encoding method as described in any one of claims 1 to 5.

12. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein: The memory is used to store computer programs; The computer program is loaded and executed by the processor to implement the image decoding method as described in any one of claims 6 to 8.

13. A computer-readable storage medium, characterized in that, Used to store computer-executable instructions, which, when loaded and executed by a processor, implement the method as described in any one of claims 1 to 8.

Images