Image coding transmission method and device based on cache mechanism, equipment and medium

Abstract

The invention provides an image coding transmission method and device based on a cache mechanism, equipment and a medium, relates to the technical field of computers, and solves the problem that the transmission bandwidth is wasted due to the fact that the same image content is coded for multiple times in the prior art. According to the scheme, the macro blocks subjected to lossless coding can be stored at the coding side by utilizing the cache mechanism, so that the macro blocks with the same image content do not need to be coded again in subsequent processing, the coding times of the same image content are effectively reduced, the coding efficiency of the image is further improved, and the coding efficiency of the image is improved. The code rate of cloud desktop hybrid coding is reduced, and the effect of saving bandwidth is achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image encoding and transmission method, apparatus, device and medium based on a caching mechanism. Background Technology

[0002] Cloud desktop applications capture, encode, and transmit desktop content from cloud servers, then decode, render, and display it on a terminal, enabling control of the cloud desktop from any local device. In related technologies, methods for encoding and transmitting cloud desktop content typically employ a hybrid encoding method combining lossless and lossy encoding. However, this method only considers sending different areas of the image to different encoding modules based on the content, neglecting the reproducibility of the image content. Furthermore, when the same image content appears in different positions across different frames, related technologies encode the same content multiple times during the encoding process, resulting in wasted transmission bandwidth. Summary of the Invention

[0003] This application provides an image encoding and transmission method, apparatus, device, and medium based on a caching mechanism, which solves the problem in related technologies of wasting transmission bandwidth by encoding the same image content multiple times. This solution can use a caching mechanism to store macroblocks using lossless encoding to reduce the number of times the same image content is encoded, thereby reducing the bitrate of cloud desktop hybrid encoding and saving bandwidth.

[0004] In a first aspect, this application provides an image encoding and transmission method based on a caching mechanism, applied to an encoding end device, wherein both the encoding end device and the corresponding decoding end device are equipped with a preset buffer, and the method includes: Given the desktop image of the cloud desktop, image content segmentation is performed on the desktop image to determine the first macroblock to be lossy encoded and the second macroblock to be lossless encoded in the desktop image; Determine the cache state of the second macroblock in the preset cache, so as to determine whether the first target macroblock in the cache state is stored or the second target macroblock in the cache state is not stored; Based on the impact parameters of the second target macroblock on the bitrate, the encoding method of the second target macroblock is re-determined, so as to determine the first sub-target macroblock with lossy encoding and the second sub-target macroblock with lossless encoding in the second target macroblock. The impact parameters are related to the information entropy of the second target macroblock. The second sub-target macroblock is added to the preset cache, and the cache information corresponding to the first target macroblock and the second sub-target macroblock is determined. The cache information is used to represent the first position parameter of the macroblock in the desktop image and the second position parameter of the macroblock in the preset cache. Based on a preset encoding algorithm, lossy encoding is performed on the first macroblock and the first sub-target macroblock, and lossless encoding is performed on the second sub-target macroblock to obtain encoded data. The encoded data and cache information are then sent to the decoding device so that the decoding device can restore the desktop image.

[0005] Secondly, this application also provides an image encoding and transmission device based on a caching mechanism, applied to an encoding end device, wherein both the encoding end device and the corresponding decoding end device are equipped with a preset cache, and the device includes: The image segmentation module is configured to perform image content segmentation on the desktop image obtained from the cloud desktop to determine the first macroblock to be lossily encoded and the second macroblock to be losslessly encoded in the desktop image; The cache query module determines the cache status of the second macroblock in the preset cache, thereby identifying the first target macroblock whose cache status is stored and the second target macroblock whose cache status is not stored in the corresponding second macroblock. The encoding reconfiguration module is configured to redetermine the encoding method of the second target macroblock based on the impact parameter of the second target macroblock on the bitrate, so as to determine the first sub-target macroblock with lossy encoding and the second sub-target macroblock with lossless encoding in the second target macroblock. The impact parameter is related to the information entropy of the second target macroblock. The information extraction module is configured to add the second sub-target macroblock to a preset cache and determine the cache information of the first target macroblock and the second sub-target macroblock. The cache information is used to represent the first position parameter of the macroblock in the desktop image and the second position parameter of the macroblock in the preset cache. The data encoding module is configured to perform lossy encoding on the first macroblock and the first sub-target macroblock and lossless encoding on the second sub-target macroblock based on a preset encoding algorithm to obtain encoded data, and send the encoded data and cache information to the decoding end device so that the decoding end device can restore the desktop image.

[0006] Thirdly, this application also provides an electronic device comprising: One or more processors; A storage device is provided for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the image encoding and transmission method based on a caching mechanism as described in this application.

[0007] Fourthly, this application also provides a storage medium for storing computer-executable instructions, which, when executed by a processor, are used to execute the image encoding and transmission method based on a caching mechanism of this application.

[0008] This application's solution utilizes a caching mechanism to store lossless encoded macroblocks on the encoding side, so that macroblocks with the same image content do not need to be encoded again in subsequent processing. This effectively reduces the number of times the same image content is encoded, thereby improving the encoding efficiency of the image and helping to reduce the bitrate of cloud desktop hybrid encoding, thus saving bandwidth. Attached Figure Description

[0009] Figure 1 This is a schematic diagram illustrating the steps of an image encoding and transmission method based on a caching mechanism provided in an embodiment of this application.

[0010] Figure 2 This is a schematic diagram illustrating the steps for re-determining the encoding method according to an embodiment of this application.

[0011] Figure 3 This is a schematic diagram of the structure of an image encoding and transmission device based on a caching mechanism provided in an embodiment of this application.

[0012] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of this application and are not intended to limit the scope of this application. Furthermore, it should be noted that, for ease of description, the accompanying drawings only show the parts related to the embodiments of this application, not all structures. Those skilled in the art, after reading this specification, should be able to conceive that any combination of technical features can constitute an optional implementation method, provided that the technical features do not contradict each other.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, not limited in number; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this application, "multiple" means two or more, and "several" means one or more.

[0015] Cloud desktop applications achieve the effect of controlling the cloud desktop from any local device by capturing, encoding, and transmitting desktop content from a cloud server, and then decoding and rendering it on the terminal. Typically, cloud desktop content includes text, lines, icons, images, and videos, and is often directly encoded and transmitted using H.264 / H.265 video encoders. In related technologies, methods for encoding and transmitting cloud desktop content often employ a hybrid encoding method combining lossless and lossy encoding. However, this method only considers sending different areas of the screen to different encoding modules for encoding according to different encoding methods, without considering the reproducibility of the content. For example, in a video stream, if the same content appears in different positions in different frames, the solutions provided by related technologies will encode the same content multiple times during the entire encoding process, resulting in wasted transmission bandwidth.

[0016] To address this, this application provides an image encoding and transmission method based on a caching mechanism. Figure 1 This diagram illustrates the steps of an image encoding and transmission method based on a caching mechanism according to an embodiment of this application. The method is applied to an encoding device in a remote transmission system, where data is transmitted between the encoding device and a decoding device. For example, the encoding device encodes video data and transmits it as a bitstream to the decoding device. It is conceivable that the encoding device can be a cloud server, computer, or other similar device. Furthermore, both the encoding device and the corresponding decoding device are equipped with preset buffers to enable the encoding device to utilize the buffers for data encoding and transmission. Specific steps include S110-S150.

[0017] Step S110: After obtaining the desktop image of the cloud desktop, perform image content segmentation on the desktop image to determine the first macroblock to be lossy encoded and the second macroblock to be lossless encoded in the desktop image.

[0018] The desktop image of the cloud desktop serves as the image to be encoded. In remote control scenarios, the desktop image to be transmitted consists of multiple frames. The encoding device encodes these multiple frames before transmitting them to the decoding device. At the encoding end, the image to be encoded is segmented into different macroblocks, and these macroblocks are distinguished according to the determined encoding type. The encoding type includes lossy encoding and lossless encoding, thereby determining the first macroblock to be lossily encoded and the second macroblock to be losslessly encoded.

[0019] Optionally, the macroblock division of the desktop image can be based on a preset segmentation direction and a preset macroblock size. The preset macroblock size can be determined based on the coding basic unit adapted to the coding standard, such as 4×4, 8×8, and 16×16. The preset segmentation direction can be from top to bottom or from left to right according to the preset macroblock size, or other segmentation directions conforming to coding specifications, thereby dividing the image to be processed into multiple macroblocks. Within the divided macroblocks, all macroblocks are traversed, and their information entropy is compared with a preset filtering threshold to further divide the macroblocks into a first macroblock to be lossy encoded and a second macroblock to be lossless encoded. For example, when the information entropy of a macroblock is less than or equal to the filtering threshold, the macroblock is determined to be a second macroblock to be lossless encoded; when the information entropy of a macroblock is greater than the filtering threshold, the macroblock is determined to be a first macroblock to be lossy encoded.

[0020] Optionally, the filtering threshold includes a first threshold parameter corresponding to the information entropy and a second threshold parameter corresponding to the number of parameters. When distinguishing between the first macroblock and the second macroblock, the first threshold parameter corresponding to the information entropy can be used to determine whether the current macroblock is the second macroblock according to the above judgment method. When determining the number of second macroblocks, the corresponding number of second threshold parameters can be used as the upper limit to limit the number of second macroblocks and avoid reducing the coding efficiency due to too many second macroblocks to be losslessly encoded on a single frame.

[0021] Step S120: Determine the cache state of the second macroblock in the preset cache, so as to determine whether the cache state of the second macroblock is the first target macroblock that has been stored or the second target macroblock that has not been stored.

[0022] The preset buffer is used to compare macroblocks already stored in the buffer during the encoding of each frame of image. This comparison determines whether the second macroblock has a cached state of stored first target macroblocks or not. It's conceivable that during the encoding of the first frame of desktop image, the initial judgment determines that none of the divided second macroblocks are currently stored in the preset buffer. However, during subsequent processing, some macroblocks are stored in the preset buffer. Consequently, during the encoding of other frames of image, macroblocks with identical content may already be stored in the preset buffer. For example, the second frame image contains a macroblock (macroblock II) with the same content as macroblock I in the first frame image. Macroblock II is the second macroblock to be losslessly encoded, and during the encoding of the first frame image, a macroblock (macroblock I) with the same content as macroblock II is already stored in the preset buffer. Therefore, during the encoding of the second frame image, macroblock II can be determined to be a cached first target macroblock.

[0023] Optionally, after storing a macroblock in a preset cache, the hash value of the macroblock can be calculated. Then, when determining the cache status of each macroblock in the subsequent encoding process of other frame images, the hash value of the corresponding macroblock can be calculated, and the Hamming distance between the hash value of the macroblock and the hash value of the macroblock in the preset cache can be determined to determine whether the two macroblocks are macroblocks with the same content. The cache status of the macroblock can be determined in the encoding process of other frame images.

[0024] Step S130: Based on the impact parameters of the second target macroblock on the bit rate, redetermine the encoding method of the second target macroblock, so as to determine the first sub-target macroblock with lossy encoding and the second sub-target macroblock with lossless encoding in the second target macroblock.

[0025] The influence parameter is related to the information entropy of the second target macroblock to facilitate the re-distinguishing of the encoding method of the second target macroblock. During the encoding process of the current frame image, after determining all second target macroblocks, the second target macroblocks are traversed sequentially, and the corresponding influence parameters are determined to filter out the first sub-target macroblocks with lossy encoding and the second sub-target macroblocks with lossless encoding from all second target macroblocks. It is understandable that after the initial determination of the macroblock encoding method (i.e., dividing into first and second macroblocks), for second target macroblocks not stored in the preset buffer, they are further divided into macroblocks with different encoding methods. It is conceivable that after the above processing, some macroblocks may have been initially classified as lossless encoding but are now classified as lossy encoding in the subsequent determination.

[0026] Step S140: Add the second sub-target macroblock to the preset cache and determine the cache information corresponding to the first target macroblock and the second sub-target macroblock.

[0027] The second sub-target macroblock, whose encoding method is ultimately determined to be lossless, is added to a preset buffer to facilitate the determination of its buffer status during the encoding processing of subsequent frames. It can be conceivable that during the encoding processing of the current frame, a macroblock is identified as the first target macroblock in the preceding checks, and a macroblock as the second sub-target macroblock is identified in subsequent checks. The buffer information corresponding to the first target macroblock and the second sub-target macroblock is recorded and transmitted to the decoding device for decoding. The buffer information represents the macroblock's first position parameter in the desktop image and its second position parameter in the preset buffer; that is, the buffer information indicates the macroblock's position in the image and in the preset buffer, enabling the decoding device to reconstruct the original image.

[0028] Step S150: Based on a preset encoding algorithm, lossy encoding is performed on the first macroblock and the first sub-target macroblock, and lossless encoding is performed on the second sub-target macroblock to obtain encoded data. The encoded data and cache information are then sent to the decoding device so that the decoding device can restore the desktop image.

[0029] It is conceivable that the preset encoding algorithms include both lossy and lossless encoding algorithms; any encoding algorithm employing relevant techniques will suffice, and will not be elaborated upon further. In the desktop image of the current frame, some macroblocks that constitute the first macroblock and the first sub-target macroblock require lossy encoding; while some macroblocks that constitute the second sub-target macroblock require lossless encoding. This yields the encoded data corresponding to all macroblocks requiring encoding, and information about the macroblocks constituting the second sub-target macroblock needs to be added to the buffer information. For the macroblocks constituting the first target macroblock, they are already stored in the preset buffer and can be identified using the buffer information. The buffer information and encoded data are then packaged and sent to the decoding device for image reconstruction. It is noteworthy that this buffer information also stores information corresponding to the second sub-target macroblock, allowing the decoding device to store the received data corresponding to the second sub-target macroblock in its own preset buffer according to the buffer information.

[0030] It is understandable that both the encoding and decoding devices have preset buffers. These preset buffers can have the same structure and access method, so that after the decoding device receives data transmitted by the encoding device, it can store and retrieve macroblocks in the preset buffer on the decoding device's side based on the buffer information to recover the original image. For example, after receiving the data of the first frame image, the decoding device decodes the encoded data to recover the macroblocks. Since there is no macroblock in the first frame image that serves as the first target macroblock, the first frame image can be recovered based on the macroblocks obtained from decoding the encoded data. Furthermore, the currently received buffer information only contains information about macroblocks that serve as the second sub-target macroblocks. Therefore, the macroblocks that serve as the second sub-targets are stored in the preset buffer for subsequent retrieval. If some macroblocks in the second frame image are identical in image content to macroblocks in the first frame image stored in the preset cache on the encoding end device side, then after the decoding end device receives the data of the second frame image, it can obtain some macroblocks that are the first target macroblocks in the second frame image based on the cache information from the preset cache on the decoding end device side, while other macroblocks are obtained by decoding the encoded data, thereby recovering the second frame image.

[0031] Optionally, the preset caches set in the encoding device and the corresponding decoding device can also adopt different structures and access methods. Then, when the decoding device stores macroblocks through the preset cache, it stores them according to the pre-configured mapping relationship. It is conceivable that the mapping relationship is used to represent the storage position relationship between the preset caches of the two devices. For example, position A in the preset cache on the encoding device side corresponds to position B in the preset cache on the decoding device side, so that the macroblock stored at position A on the encoding device side is correspondingly stored at position B on the decoding device side.

[0032] Therefore, this solution utilizes a caching mechanism to store lossless encoded macroblocks on the encoding side, so that macroblocks with the same image content do not need to be encoded again in subsequent processing. This effectively reduces the number of times the same image content is encoded, thereby improving the encoding efficiency of the image and helping to reduce the bitrate of cloud desktop hybrid encoding, thus saving bandwidth.

[0033] Figure 2 This is a schematic diagram illustrating the steps of re-determining the encoding method according to an embodiment of this application. In one embodiment, for a second target macroblock whose cache state is not stored, the first sub-target macroblock whose encoding method is lossy and the second sub-target macroblock whose encoding method is lossless are determined again. The specific steps include steps S210-S240: Step S210: Determine the expected number of times for the second target macroblock.

[0034] Step S220: Based on the expected number of times and the information entropy of the second target macroblock, determine the short-term bitrate impact value and long-term bitrate impact value corresponding to the second target macroblock, so as to serve as the impact parameters of the second target macroblock on the bitrate.

[0035] Step S230: Based on the short-term bitrate impact value and the long-term bitrate impact value, redetermine the encoding method for the second target macroblock.

[0036] Step S240: If the encoding method of the second target macroblock is determined to be lossy encoding, the second target macroblock is added to the processing sequence of the corresponding first macroblock, so as to use the second target macroblock as a macroblock to be lossy encoded.

[0037] Understandably, the expected number of occurrences is the number of times the second target macroblock appears in subsequently acquired desktop images within a preset time period. Optionally, this expected number of occurrences can be estimated using a CNN (Convolutional Neural Network). This network takes a macroblock's pixel value matrix as input and outputs the corresponding expected number of occurrences. The network can be trained using data collected over long-term testing. The network includes convolutional layers, pooling layers, and fully connected layers. The network normalizes the input macroblock to obtain a pixel matrix and performs multi-scale convolutions to extract features strongly correlated with the recurrence count (such as low texture of background macroblocks and high texture of moving objects). Then, through pooling and dimensionality compression, a fixed-dimensional feature vector is obtained, thus mapping the multi-dimensional feature vector to a non-negative expected number of occurrences. Of course, a loss function is also constructed during training to optimize the network parameters.

[0038] Furthermore, the parameters affecting bitrate include short-term and long-term bitrate impact values. These parameters are determined based on the expected number of occurrences and the information entropy of the second target macroblock. Optionally, the information entropy of the second target macroblock can be used as the short-term bitrate impact value, and the ratio of the information entropy to the expected number of occurrences can be determined as the long-term bitrate impact value. The corresponding calculation formula is as follows:

[0039]

[0040] Among them, Eff short Eff is the short-term bitrate impact value. long The long-term bitrate impact value is Ent, where Ent is the information entropy and T is the long-term bitrate impact E Let be the expected value of the order. It should be noted that the information entropy of a macroblock can be determined using Shannon's theorem.

[0041] Then, based on the determined short-term and long-term bitrate impact values, the encoding method for the second target macroblock is reassessed. It's conceivable that the short-term bitrate impact value corresponds to the short-term influence of the macroblock on the bitrate, while the long-term bitrate impact value corresponds to the long-term influence of the macroblock on the bitrate. Optionally, the macroblock can be determined as either a lossless or lossy macroblock to be encoded by comparing the sum of its short-term and long-term bitrate impact values ​​with a threshold. Alternatively, the encoding method can be determined by comparing the weighted cumulative sum of the short-term and long-term bitrate impact values ​​with a threshold. The second target macroblock is then further subdivided. For example, if the encoding method for the second target macroblock is determined to be lossy, the second target macroblock is added to the processing sequence of the corresponding first macroblock, becoming the first sub-target macroblock. The first sub-target macroblock is then subjected to the same lossy encoding as the first macroblock. If the encoding method of the second target macroblock is determined to be lossless encoding, then the macroblock is added to the preset cache as the second sub-target macroblock.

[0042] Therefore, this scheme re-divides the second target macroblock that is not stored in the preset cache, taking into account its short-term and long-term impact on the bitrate, and identifies macroblocks that have a high probability of appearing in subsequent images to store in the preset cache. This reduces the number of times the same image content is repeatedly encoded in subsequent processing, which helps to improve the encoding efficiency of the entire video stream.

[0043] Optionally, in one embodiment, when redetermining the encoding method of the second target macroblock, the filtering strategy is optimized using short-term and long-term bitrate impact values. Specifically, the long-term and short-term bitrate impact values ​​of the second target macroblock can be gradually accumulated according to the numerical order of the long-term bitrate impact values ​​to obtain the total bitrate impact. For example, macroblocks can be sorted according to the numerical order of their long-term bitrate impact values, such as in ascending order. The sum of the long-term and short-term bitrate impact values ​​of the first macroblock in the sequence (i.e., the macroblock with the smallest long-term bitrate impact value) can be calculated according to this order to obtain the total bitrate impact of one macroblock. Then, the long-term and short-term bitrate impact values ​​of the second macroblock in the sequence can be accumulated to obtain the total bitrate impact of two macroblocks. This process is repeated to continuously accumulate and determine the total bitrate impact.

[0044] Furthermore, if the total bitrate impact exceeds the bitrate set for lossless encoding, the encoding method of all macroblocks accumulated from the previous total bitrate impact is determined to be lossless encoding, and the encoding method of the remaining macroblocks is determined to be lossy encoding. Referring to the example above, if the current total bitrate impact accumulates to the value of three macroblocks, and this value is greater than the bitrate set for lossless encoding, then the first and second macroblocks in the sequence are considered as all macroblocks accumulated from the previous total bitrate impact. Therefore, the encoding method of the first and second macroblocks in the sequence is determined to be lossless encoding, while the encoding method of the other macroblocks in the sequence is updated to lossy encoding. Thus, this scheme determines macroblocks for lossless encoding based on their short-term and long-term impact on the bitstream, thereby re-evaluating the encoding method of all macroblocks that are considered as the second target macroblocks. This reduces the number of times the same image content is repeatedly encoded in subsequent processing, helping to improve the encoding efficiency of the entire video stream.

[0045] In one embodiment, the encoding device estimates the current available bandwidth value before sending data to determine the corresponding bitrate for lossless and lossy coding. Specifically, it performs bandwidth estimation processing on the available transmission capacity of the current network channel to obtain the available bandwidth value. It is conceivable that the available transmission capacity of a network channel is the maximum currently available rate or bandwidth for transmitting data through that channel, representing the amount of data that can be transmitted through the network channel per unit time. It is typically measured in bits per second or bytes per second, depending on the current network environment, including the performance of network devices, network topology, signal interference, and limitations of the transmission protocol. For example, the available bandwidth value can be determined using algorithms such as XdB bandwidth estimation, mean threshold bandwidth estimation, and power spectral density distribution inflection point bandwidth estimation.

[0046] Then, based on the available bandwidth and a preset bitrate ratio, a first bitrate is set for lossless encoding and a second bitrate is set for lossy encoding. This bitrate ratio represents the allocation of the current available network bandwidth between lossy and lossless encoding. It can be used to balance the bitrate output of lossless and lossy encoding, reducing the risk that the output bitrate of the mixed encoding may not be suitable for the current network environment due to excessive differences in the bitrate outputs of lossless and lossy encoding. For example, if the current available bandwidth is BW and the bitrate ratio is α:1-α, then the first bitrate for lossless encoding is BW×α, and the second bitrate for lossy encoding is BW×(1-α). This bitrate ratio can be adjusted according to the actual network environment and the output bitrate after mixed encoding to balance the bitrate allocation of lossless and lossy encoding. It is conceivable that this first bitrate can also be used as a corresponding judgment threshold when re-determining the encoding method of the second target macroblock. Based on the first and second bitrates, encoded data and buffer information are transmitted to the decoding device. This solution determines the current channel bandwidth through bandwidth estimation, and then allocates the corresponding bitrate to facilitate better data transmission and avoid network congestion.

[0047] In one embodiment, the encoding device and the decoding device send data in the form of a bitstream. After the encoding device sends data to the decoding device, it redetermines the set bitrate ratio and the filtering threshold for image content segmentation based on the bitstream sent to the decoding device. The filtering threshold is used to filter second macroblocks and limit the number of second macroblocks. For example, the filtering threshold includes a first threshold parameter corresponding to the information entropy and a second threshold parameter corresponding to the number of macroblocks. When distinguishing between first and second macroblocks, the first threshold parameter corresponding to the information entropy can be used to determine whether the current macroblock is a second macroblock according to the above-described judgment method. When determining the number of second macroblocks, the corresponding number of second threshold parameters can be used as an upper limit to limit the number of second macroblocks, avoiding an excessive number of second macroblocks to be losslessly encoded in a single frame, which would reduce encoding efficiency. The bitrate ratio is then redetermined based on the bitrate corresponding to different encoding methods in the bitstream.

[0048] Optionally, in the output bitstream, a first output bitrate corresponding to lossy encoding and a second output bitrate corresponding to lossless encoding are determined. Then, based on these two bitrates, the set bitrate ratio is updated. It's conceivable that when the second output bitrate is higher than the allocated bitrate, the proportion of lossless encoding in the bitrate ratio can be adjusted. Furthermore, the filtering threshold is updated based on the first output bitrate of the lossy encoding, the second output bitrate of the lossless encoding, and a preset mapping relationship. The preset mapping relationship records different channel states of the network channel and the corresponding filtering thresholds. Different channel states correspond to different filtering thresholds, and the channel state of the network channel is related to the first and second output bitrates. Therefore, after determining the first and second output bitrates, the current channel state can be determined, and a new filtering threshold can be obtained by combining the mapping relationship. This allows for adjustment of the number of second macroblocks to be losslessly encoded during macroblock partitioning, ensuring they are suitable for the current network channel and helping to avoid bandwidth waste.

[0049] Figure 3 This is a schematic diagram of an image encoding and transmission device based on a caching mechanism according to an embodiment of this application. The device is applied to an encoding end device, and both the encoding end device and the corresponding decoding end device are equipped with preset caches. The device is used to implement the image encoding and transmission method based on a caching mechanism provided in the above embodiment, and possesses the functional modules and beneficial effects of implementing the method. As shown in the figure, the device includes an image segmentation module 301, a cache query module 302, an encoding reconfiguration module 303, an information extraction module 304, and a data encoding module 305.

[0050] The image segmentation module 301 is configured to perform image content segmentation on the desktop image when the desktop image of the cloud desktop is acquired, to determine the first macroblock to be lossy encoded and the second macroblock to be lossless encoded in the desktop image; The cache query module 302 determines the cache status of the second macroblock in the preset cache, so as to determine the first target macroblock whose cache status is stored and the second target macroblock whose cache status is not stored in the corresponding second macroblock. The encoding reconfiguration module 303 is configured to redetermine the encoding method of the second target macroblock based on the influence parameter of the second target macroblock on the bit rate, so as to determine the first sub-target macroblock with lossy encoding and the second sub-target macroblock with lossless encoding in the second target macroblock, and the influence parameter is related to the information entropy of the second target macroblock; The information extraction module 304 is configured to add the second sub-target macroblock to a preset cache and determine the cache information of the first target macroblock and the second sub-target macroblock. The cache information is used to represent the first position parameter of the macroblock in the desktop image and the second position parameter of the macroblock in the preset cache. The data encoding module 305 is configured to perform lossy encoding on the first macroblock and the first sub-target macroblock and lossless encoding on the second sub-target macroblock based on a preset encoding algorithm to obtain encoded data, and send the encoded data and cache information to the decoding end device so that the decoding end device can restore the desktop image.

[0051] Based on the above embodiments, the encoding reconfiguration module 303 is specifically configured as follows: Determine the expected number of occurrences of the second target macroblock. The expected number of occurrences is the number of times the second target macroblock appears in subsequently acquired desktop images within a preset time period. Based on the expected number of times and the information entropy of the second target macroblock, the short-term bitrate impact value and long-term bitrate impact value of the second target macroblock are determined as parameters of the impact of the second target macroblock on the bitrate. Based on the short-term and long-term bitrate impact values, the encoding method for the second target macroblock is redefined; If the encoding method of the second target macroblock is determined to be lossy encoding, the second target macroblock is added to the processing sequence of the corresponding first macroblock so that the second target macroblock is used as the macroblock to be lossy encoded.

[0052] Based on the above embodiments, the encoding reconfiguration module 303 is further configured as follows: The information entropy of the second target macroblock is used as the short-term bitrate impact value; Determine the ratio of information entropy to expected number of times as a long-term bitrate impact value.

[0053] Based on the above embodiments, the encoding reconfiguration module 303 is further configured as follows: According to the numerical order of the long-term bitrate impact value, the long-term bitrate impact value and the short-term bitrate impact value of the second target macroblock are gradually accumulated to obtain the total bitrate impact; If the total bitrate impact exceeds the bitrate set for lossless encoding, the encoding method of all macroblocks accumulated from the previous total bitrate impact is determined as lossless encoding, and the encoding method of the remaining macroblocks is determined as lossy encoding.

[0054] Based on the above embodiments, the device further includes a bandwidth estimation module, which is specifically configured as follows: The available transmission capacity of the current network channel is estimated to obtain the available bandwidth value; Based on the available bandwidth value and the preset bitrate ratio, determine the first bitrate set for lossless encoding and the second bitrate set for lossy encoding; Based on the first bit rate and the second bit rate, encoded data and buffered information are transmitted.

[0055] Based on the above embodiments, the device further includes a bitstream statistics module, which is configured as follows: Based on the bitstream sent to the decoding device, the set bitrate ratio and the filtering threshold used for image content segmentation are redefined. The filtering threshold is used to filter second macroblocks and limit the number of second macroblocks.

[0056] Based on the above embodiments, the bitstream statistics module is specifically configured as follows: Determine the first output bitrate of the corresponding lossy code and the second output bitrate of the corresponding lossless code; Update the set bitrate ratio based on the first output bitrate of the corresponding lossy code and the second output bitrate of the corresponding lossless code; Based on the first output bitrate corresponding to the lossy coding, the second output bitrate corresponding to the lossless coding, and the preset mapping relationship, the filtering threshold is updated. The preset mapping relationship is used to record different channel states of the network channel and the filtering threshold corresponding to the channel state. The channel state of the network channel is associated with the first output bitrate and the second output bitrate.

[0057] It is worth noting that in the embodiments of the above-mentioned device, the modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each module are only for easy differentiation and are not used to limit the protection scope of the embodiments of this application.

[0058] Figure 4This is a schematic diagram of an electronic device provided in an embodiment of this application. The device is used to execute the image encoding and transmission method based on a caching mechanism provided in the above embodiment, and has corresponding functional modules and beneficial effects for executing the method. As shown in the figure, the device includes a processor 401, a memory 402, an input device 403, and an output device 404. The number of processors 401 can be one or more; one processor 401 is shown as an example in the figure. The processor 401, memory 402, input device 403, and output device 404 can be connected via a bus or other means; a bus connection is shown as an example in the figure. The memory 402, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the image encoding and transmission method based on a caching mechanism in the embodiments of this application. The processor 401 executes various corresponding functional applications and data processing by running the software programs, instructions, and modules stored in the memory 402, thereby realizing the above-mentioned image encoding and transmission method based on a caching mechanism. The memory 402 may mainly include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a function; the data storage area can store data recorded or created during use. Furthermore, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 402 may further include memory remotely located relative to processor 401, and these remotely located memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. Input device 403 can be used to input corresponding numerical or character information to processor 401, and to generate key signal inputs related to user settings and function control of the device; output device 404 can be used to send or display key signal outputs related to user settings and function control of the device.

[0059] This application also provides a storage medium storing computer-executable instructions, which, when executed by a processor, are used to perform related operations in the image encoding and transmission method based on a caching mechanism provided in any embodiment of this application.

[0060] Computer-readable storage media include both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0061] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0062] Note that the above description is merely a preferred embodiment and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, and the scope of this application is determined by the scope of the appended claims.

Claims

1. An image encoding and transmission method based on a caching mechanism, characterized in that, Applied to encoding devices, where both the encoding device and the corresponding decoding device are equipped with preset buffers, the method includes: When a desktop image of the cloud desktop is obtained, the desktop image is segmented to determine the first macroblock to be lossily encoded and the second macroblock to be losslessly encoded in the desktop image; Determine the cache state of the second macroblock in the preset cache, so as to determine the first target macroblock whose cache state is stored and the second target macroblock whose cache state is not stored. Based on the impact parameter of the second target macroblock on the bitrate, the encoding method of the second target macroblock is re-determined, so as to determine a first sub-target macroblock with lossy encoding and a second sub-target macroblock with lossless encoding in the second target macroblock. The impact parameter is related to the information entropy of the second target macroblock. The second sub-target macroblock is added to the preset cache, and the cache information corresponding to the first target macroblock and the second sub-target macroblock is determined. The cache information is used to represent the first position parameter of the macroblock in the desktop image and the second position parameter of the macroblock in the preset cache. Based on a preset encoding algorithm, lossy encoding is performed on the first macroblock and the first sub-target macroblock, and lossless encoding is performed on the second sub-target macroblock to obtain encoded data. The encoded data and the cache information are then sent to the decoding device so that the decoding device can restore the desktop image.

2. The image encoding and transmission method based on a caching mechanism according to claim 1, characterized in that, The step of redetermining the encoding method for the second target macroblock based on the parameter affecting the bitrate of the second target macroblock includes: Determine the expected number of occurrences of the second target macroblock, wherein the expected number of occurrences is the number of times the second target macroblock appears in subsequently acquired desktop images within a preset time period; Based on the expected number of times and the information entropy of the second target macroblock, the short-term bitrate impact value and long-term bitrate impact value corresponding to the second target macroblock are determined as parameters of the impact of the second target macroblock on the bitrate. Based on the short-term bitrate impact value and the long-term bitrate impact value, the encoding method for the second target macroblock is re-determined; If the encoding method of the second target macroblock is determined to be lossy encoding, the second target macroblock is added to the processing sequence corresponding to the first macroblock, so that the second target macroblock is used as a macroblock to be lossy encoded.

3. The image encoding and transmission method based on a caching mechanism according to claim 2, characterized in that, The step of determining the short-term bitrate impact value and long-term bitrate impact value corresponding to the second target macroblock based on the expected number of occurrences and the information entropy of the second target macroblock includes: The information entropy of the second target macroblock is used as the short-term bitrate impact value; The ratio of the information entropy to the expected number of times is determined as the long-term bitrate impact value.

4. The image encoding and transmission method based on a caching mechanism according to claim 2 or 3, characterized in that, The step of redetermining the encoding method for the second target macroblock based on the short-term bitrate impact value and the long-term bitrate impact value includes: According to the numerical order of the long-term bitrate impact value, the long-term bitrate impact value and the short-term bitrate impact value of the second target macroblock are gradually accumulated to obtain the total bitrate impact; If the sum of the bitrate effects is greater than the bitrate set for lossless coding, the encoding method of all macroblocks accumulated by the previous sum of bitrate effects is determined to be lossless coding, and the encoding method of the remaining macroblocks is determined to be lossy coding.

5. The image encoding and transmission method based on a caching mechanism according to claim 1 or 2, characterized in that, Before sending the encoded data and the cache information to the decoding terminal device, the method further includes: The available transmission capacity of the current network channel is estimated to obtain the available bandwidth value; Based on the available bandwidth value and the preset bitrate ratio, a first bitrate is set for lossless encoding and a second bitrate is set for lossy encoding. Based on the first bitrate and the second bitrate, the encoded data and the cache information are transmitted.

6. The image encoding and transmission method based on a caching mechanism according to claim 1 or 2, characterized in that, Also includes: Based on the bitstream sent to the decoding device, the set bitrate ratio and the filtering threshold for image content segmentation are redefined. The filtering threshold is used to filter the second macroblock and limit the number of the second macroblock.

7. The image encoding and transmission method based on a caching mechanism according to claim 6, characterized in that, The step of redetermining the set bitrate ratio and the filtering threshold for image content segmentation based on the bitstream sent to the decoding terminal device includes: Determine the first output bitrate of the corresponding lossy code and the second output bitrate of the corresponding lossless code; Update the set bitrate ratio based on the first output bitrate of the corresponding lossy code and the second output bitrate of the corresponding lossless code; Based on the first output bitrate corresponding to the lossy encoding, the second output bitrate corresponding to the lossless encoding, and a preset mapping relationship, the filtering threshold is updated. The preset mapping relationship is used to record different channel states of the network channel and the filtering threshold corresponding to the channel state. The channel state of the network channel is associated with the first output bitrate and the second output bitrate.

8. An image encoding and transmission device based on a caching mechanism, characterized in that, Applied to encoding end devices, wherein both the encoding end device and the corresponding decoding end device are equipped with preset buffers, the device includes: The image segmentation module is configured to, upon acquiring a desktop image of a cloud desktop, perform image content segmentation on the desktop image to determine a first macroblock to be lossily encoded and a second macroblock to be losslessly encoded in the desktop image; The cache query module determines the cache status of the second macroblock in the preset cache, thereby determining the first target macroblock whose cache status is stored and the second target macroblock whose cache status is not stored in the corresponding second macroblock. The encoding reconfiguration module is configured to redetermine the encoding method of the second target macroblock based on the impact parameter of the second target macroblock on the bitrate, so as to determine a first sub-target macroblock with lossy encoding and a second sub-target macroblock with lossless encoding in the second target macroblock, wherein the impact parameter is associated with the information entropy of the second target macroblock; The information extraction module is configured to add the second sub-target macroblock to the preset cache and determine the cache information of the first target macroblock and the second sub-target macroblock. The cache information is used to represent the first position parameter of the macroblock in the desktop image and the second position parameter of the macroblock in the preset cache. The data encoding module is configured to perform lossy encoding on the first macroblock and the first sub-target macroblock and lossless encoding on the second sub-target macroblock based on a preset encoding algorithm to obtain encoded data, and send the encoded data and the cache information to the decoding end device so that the decoding end device can restore the desktop image.

9. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the image encoding and transmission method based on a caching mechanism as described in any one of claims 1-7.

10. A storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a processor, are used to perform the image encoding and transmission method based on a caching mechanism as described in any one of claims 1-7.

Images