Video coding method, and cloud desktop image coding method and device

Abstract

The embodiment of the invention provides a video coding method and a cloud desktop image coding method and device, and the video coding method comprises the steps: dividing a to-be-coded video frame into a plurality of macro blocks, and determining the macro block time domain complexity of each macro block according to the macro block inter-frame change degree of each macro block in the to-be-coded video frame; determining the frame time domain complexity of the video frame to be coded according to the macro block time domain complexity of each macro block, and calculating the quantization adjustment coefficient of the target macro block according to the frame time domain complexity and the target macro block time domain complexity of the target macro block; according to the quantization adjustment coefficient of the target macro block, updating the basic quantization parameter of the target macro block to obtain an updated quantization parameter; and coding the target macro block according to the updated quantization parameter. The basic quantization parameters are controlled and updated based on the macro block time domain complexity, so that the macro blocks with different change degrees can be coded by adopting different quantization parameters, the coding image quality of the macro blocks with different change degrees is controlled in a targeted manner, and the video coding effect is ensured.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a video encoding method, a cloud desktop image encoding method, and an apparatus. Background Technology

[0002] With the rapid development of computer, communication, and network technologies, a wide variety of videos are emerging, and watching videos has become an important form of leisure and entertainment. Because videos contain a large amount of data, video encoding technology is needed to compress them for easier transmission and storage. Video encoding is a lossy compression process; the decoded video is not identical to the original. Therefore, balancing video quality and compression loss is a key research focus in video encoding.

[0003] In existing technologies, the basic quantization parameters of the macroblock to be encoded in a video frame can be calculated first, and encoding can be performed based on these parameters. However, in the above encoding methods, if local scene changes are very frequent and drastic, the image quality of the scene content in the non-drastically changing areas will be severely affected after encoding by the video encoder, resulting in poor video encoding performance. Therefore, there is an urgent need for a method to improve video encoding performance. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a video encoding method. One or more embodiments of this specification also relate to a cloud desktop image encoding method, a video encoding apparatus, a cloud desktop image encoding apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a video encoding method is provided, comprising:

[0006] The video frame to be encoded is divided into multiple macroblocks, and the macroblock temporal complexity of each macroblock is determined based on the degree of inter-frame variation of each macroblock in the video frame to be encoded.

[0007] The frame time-domain complexity of the video frame to be encoded is determined based on the macroblock time-domain complexity of each macroblock. The quantization adjustment coefficient of the target macroblock is calculated based on the frame time-domain complexity and the target macroblock time-domain complexity of the target macroblock. The target macroblock is the current macroblock to be encoded.

[0008] Based on the quantization adjustment coefficient of the target macroblock, the basic quantization parameters of the target macroblock are updated to obtain the updated quantization parameters;

[0009] The target macroblock is encoded according to the updated quantization parameters.

[0010] According to a second aspect of the embodiments of this specification, a method for encoding cloud desktop images is provided, applied to a backend server of a cloud application, comprising:

[0011] The cloud desktop image to be encoded is acquired and divided into multiple macroblocks. Based on the degree of inter-frame variation of each macroblock in the cloud desktop image, the macroblock temporal complexity of each macroblock is determined.

[0012] The frame time-domain complexity of the cloud desktop image is determined based on the macroblock time-domain complexity of each macroblock. The quantization adjustment coefficient of the target macroblock is calculated based on the frame time-domain complexity and the target macroblock time-domain complexity of the target macroblock. The target macroblock is the current macroblock to be encoded.

[0013] Based on the quantization adjustment coefficient of the target macroblock, the basic quantization parameters of the target macroblock are updated to obtain the updated quantization parameters;

[0014] The target macroblock is encoded according to the updated quantization parameters, and the encoded macroblock data is transmitted to the client of the cloud application.

[0015] According to a third aspect of the embodiments of this specification, a video encoding apparatus is provided, comprising:

[0016] The first determining module is configured to divide the video frame to be encoded into multiple macroblocks and determine the macroblock temporal complexity of each macroblock based on the degree of macroblock variation in each macroblock of the video frame to be encoded.

[0017] The first calculation module is configured to determine the frame time-domain complexity of the video frame to be encoded based on the macroblock time-domain complexity of each macroblock, and to calculate the quantization adjustment coefficient of the target macroblock based on the frame time-domain complexity and the target macroblock time-domain complexity of the target macroblock, wherein the target macroblock is the current macroblock to be encoded.

[0018] The first update module is configured to update the basic quantization parameters of the target macroblock according to the quantization adjustment coefficient of the target macroblock, so as to obtain updated quantization parameters;

[0019] The encoding module is configured to encode the target macroblock according to the updated quantization parameters.

[0020] According to a fourth aspect of the embodiments of this specification, an encoding apparatus for cloud desktop images is provided, applied to a backend server of a cloud application, comprising:

[0021] The second determining module is configured to acquire the cloud desktop image to be encoded, divide the cloud desktop image into multiple macroblocks, and determine the macroblock temporal complexity of each macroblock based on the degree of inter-frame variation of each macroblock in the cloud desktop image.

[0022] The second calculation module is configured to determine the frame time-domain complexity of the cloud desktop image based on the macroblock time-domain complexity of each macroblock, and to calculate the quantization adjustment coefficient of the target macroblock based on the frame time-domain complexity and the target macroblock time-domain complexity of the target macroblock, wherein the target macroblock is the current macroblock to be encoded.

[0023] The second update module is configured to update the basic quantization parameters of the target macroblock according to the quantization adjustment coefficient of the target macroblock, so as to obtain updated quantization parameters;

[0024] The transmission module is configured to encode the target macroblock according to the updated quantization parameters and transmit the encoded macroblock data to the client of the cloud application.

[0025] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0026] Memory and processor;

[0027] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the above-described video encoding method or cloud desktop image encoding method.

[0028] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the video encoding method or the cloud desktop image encoding method described above.

[0029] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the video encoding method or the cloud desktop image encoding method described above.

[0030] This specification provides a video encoding method in one embodiment, which involves dividing a video frame to be encoded into multiple macroblocks; determining the macroblock temporal complexity of each macroblock based on the degree of inter-frame variation among macroblocks in the video frame to be encoded; determining the frame temporal complexity of the video frame to be encoded based on the macroblock temporal complexity of each macroblock; calculating the quantization adjustment coefficient of the target macroblock based on the frame temporal complexity and the target macroblock temporal complexity, wherein the target macroblock is the current macroblock to be encoded; updating the basic quantization parameters of the target macroblock based on the quantization adjustment coefficient of the target macroblock to obtain updated quantization parameters; and encoding the target macroblock based on the updated quantization parameters.

[0031] One embodiment of this specification implements a method to calculate the macroblock temporal complexity of a macroblock based on the degree of inter-frame variation among macroblocks in the video frame to be encoded. Then, combining this with the frame temporal complexity of the video frame to be encoded, the quantization adjustment coefficients of the target macroblock are calculated. Based on these quantization adjustment coefficients, the basic quantization parameters are updated to obtain the updated quantization parameters, and the target macroblock is then encoded. In this way, the degree of change in the image content of the region where the macroblock is located is predicted based on the macroblock temporal complexity. A higher macroblock temporal complexity indicates lower inter-frame correlation, meaning the macroblock changes more frequently, which can reduce the image quality of the macroblock. Conversely, a lower macroblock temporal complexity indicates higher inter-frame correlation, meaning the macroblock changes slowly, which can enhance the image quality of the macroblock. In other words, by controlling the update of the basic quantization parameters based on the macroblock temporal complexity, macroblocks with different degrees of change can be encoded using different quantization parameters. This targeted control of the encoded image quality of macroblocks with different degrees of change ensures the effectiveness of video encoding and optimizes its performance. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating a video encoding method provided in one embodiment of this specification;

[0033] Figure 2a This is a schematic diagram of macroblock encoding provided in one embodiment of this specification;

[0034] Figure 2b This is a schematic diagram of an encoding process provided in one embodiment of this specification;

[0035] Figure 3a This is a flowchart illustrating a cloud desktop image encoding method provided in one embodiment of this specification;

[0036] Figure 3b This is a flowchart illustrating the processing procedure of a video encoding method provided in one embodiment of this specification;

[0037] Figure 4 This is a schematic diagram of the structure of a video encoding device provided in one embodiment of this specification;

[0038] Figure 5 This is a schematic diagram of the structure of a cloud desktop image encoding device provided in one embodiment of this specification;

[0039] Figure 6 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0040] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0041] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0042] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0043] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0044] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0045] Cloud desktop: This is a new model that uses cloud computing technology to replace traditional computers. The computer host is virtualized in the backend server. The front-end devices mainly use lightweight clients to connect to the monitor and input devices. After installing the client, users can access the virtual machine host on the backend server through a unique communication protocol to achieve interactive operation and achieve the same experience as the computer host.

[0046] Video encoding: During video transmission, if the video is not compressed, the existing bandwidth cannot meet the transmission requirements. Therefore, video compression is necessary, and this process is called video encoding. More specifically, encoding is the technology of converting information from one form (format) to another using a specified method. Video encoding refers to the use of data compression technology to remove redundant information from digital video signals, converting the original video format file into another video format file, thereby reducing the bitrate required to represent the original video and facilitating the transmission and storage of video data.

[0047] Bitrate: The amount of data a video file uses per unit of time, often simply called bitrate, is the most important part of video encoding for controlling picture quality. Generally speaking, at the same resolution, the higher the bitrate of a video, the lower the compression ratio and the degree of distortion, resulting in higher picture quality.

[0048] Subjective quality: A method for evaluating video quality by having humans rate the video after watching it, reflecting the perceived quality of the video in the eyes of humans.

[0049] Bitrate control: Video encoding is a lossy encoding process. The goal of video encoding is to save bitrate as much as possible while ensuring the subjective quality of the video. Bitrate control is an important tool for balancing bitrate and subjective quality.

[0050] Quantization refers to the process of mapping continuous values ​​(or a large number of possible discrete values) of a signal to a finite number of discrete values, achieving a many-to-one mapping of signal values. In video / image coding, after the residual signal undergoes DCT, the transform coefficients often have a large dynamic range. Therefore, quantizing the transform coefficients can effectively reduce the signal value space and achieve better compression results.

[0051] Quantization parameter (QP): In the quantization process, the quantization parameter is a crucial parameter used to control the dispersion of variables and is a major factor affecting video bitrate and compression levels. The quantization parameter is the threshold selected during quantization to discretize continuous values. It reflects the compression of spatial details. A small QP preserves most details, resulting in better encoded image quality; a larger QP leads to some detail loss, lower bitrate, but also increased image distortion and reduced quality, resulting in poor encoded image quality. In other words, QP and bitrate are inversely proportional, and this inverse relationship becomes more pronounced as the complexity of the video source increases. The core of bitrate control is determining the QP of macroblocks.

[0052] Macroblock (MB): When selecting quantization parameters for a frame of image, a fixed-size sliding window is typically used as the unit. The pixel block enclosed by this sliding window is called a macroblock. A macroblock is the basic unit of coding processing, typically 16x16 pixels in size. A coded image must first be divided into multiple blocks (4x4 pixels) before processing; obviously, macroblocks should consist of an integer number of blocks. Macroblocks are divided into I, P, and B macroblocks: I macroblocks (intra-prediction macroblocks) can only use previously decoded pixels in the current frame as references for intra-prediction; P macroblocks (inter-prediction macroblocks) can use previously decoded images as reference images for intra-prediction; B macroblocks (inter-bidirectional prediction macroblocks) use both forward and backward reference images for intra-prediction.

[0053] Basic quantization parameters: These are calculated based on the characteristics of the macroblocks to be encoded in the video frame to be encoded, and serve as the initial values ​​for macroblock quantization.

[0054] Temporal adaptive quantization: a video compression technique that automatically adjusts quantization parameters by analyzing the dynamic changes in video footage to achieve higher compression efficiency and image quality.

[0055] Blocking effect: refers to the jagged, blocky, or discontinuous appearance of an image caused by limitations in quantization parameters during video compression.

[0056] Macroblock Tree (MB-Tree): The main purpose of this algorithm is to estimate the amount of information in each macroblock in order to help predict future video frames more likely. This estimated information allows the macroblock tree to weight the quality of each macroblock according to its contribution. To this end, the macroblock tree works in the opposite direction of prediction, propagating information from future frames back to the current frame to be encoded.

[0057] It should be noted that in cloud desktop scenarios, when the local screen changes very frequently and drastically, the image quality of the non-drastically changing areas will be severely affected after being encoded by the video encoder, resulting in obvious blockiness or blurring, which will affect the user experience.

[0058] One common approach is to use an MB-tree, which first analyzes the propagation relationships and allocates bits based on the contribution of MBs to inter prediction. However, this tool introduces encoding latency and is not suitable for scenarios like cloud desktops that require real-time encoding.

[0059] This specification provides a video coding method in one embodiment that achieves temporal adaptive quantization. This means that quantization parameters can be adaptively updated based on the temporal complexity of macroblocks to control the image quality of different macroblocks. By improving the bitrate control mechanism of video coding, the performance of video coding is optimized, thereby improving the image quality of cloud desktop video images. Specifically, the inter-frame temporal complexity of a macroblock can be defined and calculated. A higher temporal complexity indicates lower inter-frame correlation, meaning more frequent changes in that region, which can reduce the image quality in that region. Conversely, a lower temporal complexity suggests that the region containing the macroblock is not expected to change drastically, making it worthwhile to allocate more bandwidth for encoding to improve image quality. During encoding, the quantization adjustment coefficient and quantization parameter compensation coefficient of the macroblock are calculated based on the ratio between the temporal complexity of the macroblock and the average temporal complexity of the encoded frame, and then superimposed on the original bitrate control result. This method can analyze the temporal changes of different regions of the desktop image, identify slowly changing regions and enhance their image quality, while maintaining overall bitrate balance.

[0060] This specification provides a video encoding method, and also relates to a video encoding apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0061] See Figure 1 , Figure 1 A flowchart of a video encoding method according to an embodiment of this specification is shown, specifically including the following steps 102-108.

[0062] Step 102: Divide the video frame to be encoded into multiple macroblocks, and determine the macroblock temporal complexity of each macroblock based on the degree of inter-frame variation of each macroblock in the video frame to be encoded.

[0063] It should be noted that the video encoding method provided in the embodiments of this specification can be applied to both client-side and cloud-side applications. When applied to a client-side application, the video to be encoded can be obtained from a cloud server or other devices, encoded, and then transmitted to the other devices or the cloud server. When applied to a cloud server, the video to be encoded can be obtained from the client, other servers, or the cloud server itself, encoded, and then transmitted to the client for display. The cloud server can connect to one or more clients via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. These clients may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. The client can interact with the user through a graphical user interface to control operations such as uploading, encoding, and displaying the video to be encoded, thereby implementing the video encoding method provided in the embodiments of this specification.

[0064] In practice, during video encoding, the video frame to be encoded can be divided into multiple macroblocks, and encoding is performed on a macroblock basis. Specifically, during the encoding process, the video frame to be encoded can be divided into multiple macroblocks based on the division method and size allowed by the encoding rules used. For example, the video frame to be encoded can be divided into 16*16 macroblocks, 32*32 macroblocks, and 64*64 macroblocks, with 16*16 macroblocks being the most common.

[0065] It's important to note that after dividing macroblocks, the degree of inter-frame variation within each macroblock can be determined, and the macroblock temporal complexity of each macroblock can be calculated. A higher macroblock temporal complexity indicates lower inter-frame correlation, meaning more frequent regional changes within that macroblock; conversely, lower macroblock temporal complexity indicates higher inter-frame correlation, meaning slower regional changes within that macroblock. This facilitates subsequent control of macroblock quantization parameters during the encoding process based on macroblock temporal complexity, thereby enabling targeted control of the encoded image quality for macroblocks with varying degrees of variation.

[0066] In one optional implementation of this embodiment, the macroblock temporal complexity of each macroblock is determined based on the degree of inter-frame variation of each macroblock in the video frame to be encoded, including:

[0067] Based on the pixel differences of each macroblock in the target video frame and the video frame preceding the target video frame, the degree of inter-frame variation of each macroblock in the target video frame is determined, wherein the target video frame is the video frame preceding the video frame to be encoded in the video to be encoded.

[0068] Based on the degree of inter-frame variation of each macroblock in the target video frame, the degree of inter-frame variation of each macroblock in the video frame to be encoded is calculated.

[0069] The macroblock temporal complexity of each macroblock in the video frame to be encoded is determined based on the degree of inter-frame variation of each macroblock in the video frame to be encoded, the weighting factor, and the historical temporal complexity of each macroblock in the previous video frame of the video frame to be encoded.

[0070] In practice, when calculating the inter-frame variation of macroblocks in a target video frame, it is necessary to consider the preceding video frame. Based on the pixel differences between the two frames, the inter-frame variation of each macroblock in the target video frame can be calculated. The pixel difference can be the difference in the grayscale values ​​of individual pixels. All video frames preceding the target video frame are considered target video frames, and the inter-frame variation of each macroblock within the target video frame can be calculated.

[0071] Then, for each macroblock in the target video frame, the inter-frame variation of the macroblock in each target video frame preceding the video frame to be encoded can be summed to obtain the inter-frame variation of the corresponding macroblock in the video frame to be encoded. Thus, by combining the inter-frame variation of each macroblock in each target video frame preceding the video frame to be encoded, the inter-frame variation of each macroblock in the video frame to be encoded can be calculated. Then, based on the inter-frame variation of each macroblock in the video frame to be encoded, the weight factor, and the historical temporal complexity of each macroblock in the previous video frame, the temporal complexity of each macroblock in the video frame to be encoded can be determined.

[0072] In practice, the video frame to be encoded is the nth video frame, and the macroblock temporal complexity of the macroblock at position (i, j) can be calculated using the following formula (1):

[0073] MSSE(i,j,n)=Yn(i,j)*Factor+MSSE(i,j,n-1)*(1-Factor)(1)

[0074] Where MSSE(i,j,n) represents the macroblock temporal complexity at position (i,j) in the nth video frame; Yn(i,j) represents the inter-frame variation of the macroblock at position (i,j) in the nth video frame; Factor represents a pre-configured weight factor, which is used to adjust the proportion of the inter-frame variation of the macroblock in the current video frame to be encoded and the previous video frame to be encoded. This weight factor is generally configured to 0.2; MSSE(i,j,n-1) is the macroblock temporal complexity at position (i,j) in the (n-1)th video frame, which is also the historical temporal complexity.

[0075] It should be noted that the macroblock temporal complexity of each macroblock in the current video frame to be encoded can be calculated by combining the degree of macroblock inter-frame variation and historical temporal complexity of each macroblock in the video frame to be encoded. This fully considers the degree of macroblock variation in the video frames before the current video frame to be encoded, improves the accuracy of macroblock temporal complexity, and thus ensures the accuracy of quantization parameters controlled based on macroblock temporal complexity, thereby improving encoding performance.

[0076] In one optional implementation of this embodiment, determining the inter-frame variation degree of each macroblock in the target video frame based on the pixel differences of each macroblock in the target video frame and the preceding video frame includes:

[0077] For each macroblock in the target video frame, the pixel value of each pixel in the macroblock is subtracted from the pixel value of each pixel in the macroblock at the same position in the previous video frame to obtain the pixel difference of each pixel in the macroblock; the pixel differences of each pixel are summed to obtain the degree of inter-frame variation of the macroblock.

[0078] In practice, SSE (Sum of Squared Errors) can be used to measure the degree of change of each macroblock in each target video frame, thereby calculating the inter-frame variation of macroblocks in each target video frame. Specifically, for each macroblock in each target video frame, the pixel difference of each pixel in the macroblock at the same position in two consecutive video frames can be obtained; the inter-frame variation of the macroblock is obtained by summing the pixel differences; thus, the inter-frame variation of macroblocks in each macroblock in the target video frame can be obtained.

[0079] In addition, for each macroblock in the video frame to be encoded, the inter-frame variation of the corresponding macroblock in each target video frame can be summed to obtain the inter-frame variation of the macroblock in the video frame to be encoded.

[0080] In practice, the degree of inter-frame variation of the macroblock at position (i, j) in the video frame to be encoded can be calculated using the following formula (2):

[0081]

[0082] Where (d,k) represents the position coordinates of a pixel in the macroblock; Cur (i,j) (d,k) represents the pixel grayscale value of the pixel at position (d,k) of the macroblock at position (i,j) in the v-th target video frame; Pred (i,j) (d,k) represents the pixel grayscale value of the pixel at position (d,k) of the macroblock at position (i,j) in the (v-1)th target video frame (i.e., the previous frame of the target video); This indicates the degree of inter-frame variation of the macroblock in the v-th target video frame.

[0083] Combining formulas (1) and (2) above, we can obtain the formula for calculating the macroblock temporal complexity of the macroblock at position (i, j) in the nth video frame, as shown in formula (3) below:

[0084]

[0085] Example, Figure 2a This is a schematic diagram of macroblock encoding provided in one embodiment of this specification, such as... Figure 2a As shown, the macroblock (MB) at position ((x, x+16), (y, y+16)) in the (N-1)th video frame is also located at ((x, x+16), (y, y+16)) in the Nth video frame. Based on the pixel difference between the macroblocks at the same position in the two consecutive video frames, the degree of inter-frame variation of the macroblock can be determined.

[0086] It should be noted that the sum of squared errors can be used to measure the drastic changes of macroblocks in each video frame. Then, based on the current video frame to be encoded and the previous video frame, combined with weighting factors, the historical average value is calculated as the temporal complexity of the macroblock in the current video frame to be encoded. This combines the degree of macroblock change in the current video frame and historical video frames, improving the accuracy of the macroblock temporal complexity, thereby ensuring the accuracy of controlling quantization parameters based on macroblock temporal complexity and improving encoding performance. A higher macroblock temporal complexity indicates higher temporal complexity, meaning more drastic and frequent changes, which can reduce image quality for that macroblock; conversely, a lower temporal complexity indicates lower temporal complexity, meaning slower and less frequent changes, which can enhance image quality.

[0087] Step 104: Determine the frame time-domain complexity of the video frame to be encoded based on the macroblock time-domain complexity of each macroblock. Calculate the quantization adjustment coefficient of the target macroblock based on the frame time-domain complexity and the target macroblock time-domain complexity, where the target macroblock is the current macroblock to be encoded.

[0088] Among them, the frame time-domain complexity is the time-domain complexity of the entire image of the video frame to be encoded.

[0089] In actual implementation, the quantization adjustment coefficient of the target macroblock can be calculated based on the frame temporal complexity and the target macroblock temporal complexity. This quantization adjustment coefficient is used to control the quantization parameters of the target macroblock, thereby controlling the image quality when encoding different macroblocks.

[0090] In one optional implementation of this embodiment, determining the frame temporal complexity of the video frame to be encoded based on the macroblock temporal complexity of each macroblock includes:

[0091] Determine the sum of the macroblock time-domain complexity of each macroblock;

[0092] The frame time complexity of the video frame to be encoded is determined based on the sum of the macroblock time-domain complexities of each macroblock and the number of macroblocks in the video frame to be encoded.

[0093] It should be noted that the sum of the macroblock time-domain complexity of each macroblock in the video frame to be encoded can be determined. The sum of the macroblock time-domain complexity of each macroblock can be divided by the number of macroblocks in the video frame to be encoded to obtain the average time-domain complexity of each macroblock in the video frame to be encoded, which can be used as the frame time-domain complexity of the video frame to be encoded.

[0094] In practice, the temporal complexity of the video frame to be encoded can be calculated using the following formula (4):

[0095]

[0096] Where, Avg(n) MSSELet (i,j) represent the frame time complexity of the nth video frame; (i,j) represent the macroblock position; and Mb represents the number of macroblocks in the nth video frame.

[0097] In the embodiments of this specification, the average temporal complexity of each macroblock in the video frame to be encoded can be determined as the frame temporal complexity of the video frame to be encoded. This facilitates the subsequent calculation of the quantization adjustment coefficient of the target macroblock by combining the overall frame temporal complexity of the video frame to be encoded with the target macroblock temporal complexity of the target macroblock to be encoded, thereby controlling the quantization parameters of the target macroblock and thus controlling the encoded image quality of the target macroblock.

[0098] In an optional implementation of this embodiment, the quantization adjustment coefficient of the target macroblock is calculated based on the frame temporal complexity and the target macroblock temporal complexity, including:

[0099] Determine the ratio parameter between the temporal complexity of the target macroblock and the temporal complexity of the frame;

[0100] The quantization adjustment coefficient of the target macroblock is calculated based on the ratio parameter and the set adjustment coefficient in the encoding rule.

[0101] In practice, the ratio of the target macroblock temporal complexity to the frame temporal complexity can be calculated using the following formula (5):

[0102] Scale(i,j,n)=MSSE(i,j,n) / Avg(n) MSSE (5)

[0103] Here, Scale(i,j,n) represents the ratio of the macroblock at position (i,j) in the nth video frame to the temporal complexity of the frame.

[0104] In the embodiments of this specification, the proportion of the target macroblock's temporal complexity to the frame's temporal complexity can be determined. Then, based on this proportion and the adjustment coefficients set in the encoding rules, the quantization adjustment coefficient of the target macroblock is calculated. By measuring the proportion of temporal complexity, the degree of change of a macroblock relative to other macroblocks in a video frame can be measured, and the quantization adjustment coefficient of the macroblock can be calculated. Thus, for macroblocks in the video to be encoded, different degrees of change will determine different quantization adjustment coefficients, thereby enabling macroblocks with different degrees of change to be encoded with different encoding quality.

[0105] In one optional implementation of this embodiment, the adjustment coefficients include a threshold for frame temporal complexity, an adjustment coefficient for frame temporal complexity, and a quantization parameter adjustment coefficient; the step of calculating the quantization adjustment coefficient of the target macroblock based on the proportional parameter and the set adjustment coefficients in the encoding rules includes:

[0106] Subtract the threshold of the frame temporal complexity from the ratio parameter to obtain the first intermediate value;

[0107] Subtract the adjustment coefficient of the frame temporal complexity from the ratio parameter to obtain the second intermediate value;

[0108] The ratio of the first intermediate value to the second intermediate value is multiplied by the quantization parameter adjustment coefficient to obtain the quantization adjustment coefficient of the target macroblock.

[0109] Among them, the frame temporal complexity threshold is the adjustment threshold. When the ratio parameter (Scale) of the macroblock at position (i, j) in the nth video frame to the frame temporal complexity is less than the frame temporal complexity threshold, it means that the change of the macroblock is relatively small and the image quality needs to be enhanced.

[0110] In practice, the frame temporal complexity threshold, the frame temporal complexity adjustment coefficient, and the quantization parameter adjustment coefficient can all be pre-configured. The higher the desired increase in image quality, the larger the frame temporal complexity threshold, the smaller the frame temporal complexity adjustment coefficient, and the larger the quantization parameter adjustment coefficient. For example, the frame temporal complexity threshold can typically be set to 10000, the frame temporal complexity adjustment coefficient can typically be set to 9900, and the quantization parameter adjustment coefficient can typically be set to 12000.

[0111] Specifically, the quantization adjustment coefficient of the target macroblock is calculated based on the ratio parameter and the set adjustment coefficient in the encoding rule, which can be achieved by the following formula (6):

[0112]

[0113] Where DeltaQp(i,j,n) represents the quantization adjustment coefficient of the macroblock at position (i,j) in the nth video frame; Model threshold The threshold representing the temporal complexity of a frame, Model Alpha The adjustment factor representing the time-domain complexity of a frame, Model Beta express.

[0114] In the embodiments described in this specification, the frame temporal complexity threshold, frame temporal complexity adjustment coefficient, quantization parameter adjustment coefficient, and other adjustment coefficients are flexibly configured according to the required level of image quality enhancement. Then, based on the frame temporal complexity threshold, frame temporal complexity adjustment coefficient, and quantization parameter adjustment coefficient, the quantization adjustment coefficient of the target macroblock is calculated, thereby controlling the quantization parameters of the target macroblock and controlling the encoded image quality. In this way, the degree of image quality enhancement can be flexibly controlled, optimizing the encoded signal.

[0115] In an optional implementation of this embodiment, after calculating the quantization adjustment coefficient of the target macroblock based on the frame temporal complexity and the target macroblock temporal complexity, the method further includes:

[0116] Based on the quantization adjustment coefficients of each macroblock, the quantization parameter compensation coefficients of the video frame to be encoded are determined, wherein the quantization parameter compensation coefficients are used to control the bitrate balance of the video frame to be encoded.

[0117] In practice, the quantization parameter compensation coefficient of the video frame to be encoded can be determined based on the quantization adjustment coefficient of each macroblock. This quantization parameter compensation coefficient is used to reduce the image quality of other macroblocks while enhancing the image quality of a certain macroblock.

[0118] It should be noted that since the image quality of some macroblocks is enhanced, in order not to affect the overall bandwidth performance, it is necessary to reduce the image quality of other areas to ensure the overall bitrate balance. Therefore, in addition to introducing quantization adjustment coefficients to control the encoded image quality, quantization parameter compensation coefficients can also be introduced to control the overall bitrate balance of the video frames to be encoded.

[0119] In one optional implementation of this embodiment, the quantization parameter compensation coefficients of the video frame to be encoded are determined based on the quantization adjustment coefficients of each macroblock, including:

[0120] Macroblocks whose quantization adjustment coefficient is less than a set threshold are identified as macroblocks to be enhanced.

[0121] The quantization parameter compensation coefficients of the video frame to be encoded are calculated based on the quantization adjustment coefficients of the macroblock to be enhanced.

[0122] The threshold is a pre-configured value used to determine whether a macroblock needs enhanced image quality; for example, the threshold can be set to 0.

[0123] In the embodiments described in this specification, if the quantization adjustment coefficient is less than a set threshold, it indicates that the image quality of the macroblock needs to be enhanced; if the quantization adjustment coefficient is not less than the set threshold, it indicates that the image quality of the macroblock does not need to be enhanced. Therefore, macroblocks with quantization adjustment coefficients less than the set threshold can be identified as macroblocks to be enhanced. Then, quantization parameter compensation coefficients are calculated based on the macroblocks to be enhanced, thereby using the quantization parameter compensation coefficients to reduce the image quality of other macroblocks and ensure bitrate balance.

[0124] In one optional implementation of this embodiment, the quantization parameter compensation coefficients of the video frame to be encoded are calculated based on the quantization adjustment coefficients of the macroblock to be enhanced, including:

[0125] Determine the sum of the quantization adjustment coefficients for each macroblock to be enhanced;

[0126] The quantization parameter compensation coefficient of the video frame to be encoded is calculated based on the sum of the quantization adjustment coefficients of each macroblock to be enhanced and the number of macroblocks in the video frame to be encoded.

[0127] In practice, the quantization parameter compensation coefficient of the video frame to be encoded is calculated by adding the quantization adjustment coefficients of all image enhancement macroblocks in the video frame to be encoded and dividing by the total number of macroblocks.

[0128] Specifically, the quantization parameter compensation coefficients of the video frame to be encoded are calculated based on the quantization adjustment coefficients of the macroblock to be enhanced, which can be achieved by the following formula (7):

[0129]

[0130] Where, F(n) comp DeltaQp(k,n) represents the quantization parameter compensation coefficient of the nth video frame; DeltaQp(k,n) represents the quantization adjustment coefficient of the kth macroblock to be enhanced in the nth video frame; Num aq_mb Indicates the number of macroblocks to be enhanced; Num mb This indicates the number of macroblocks in the video frame to be encoded.

[0131] In the embodiments of this specification, the sum of the quantization adjustment coefficients of each macroblock to be enhanced is divided by the number of macroblocks in the video frame to be encoded to obtain the quantization parameter compensation coefficient, thereby using the quantization parameter compensation coefficient to reduce the image quality of other macroblocks and ensure bitrate balance.

[0132] Step 106: Update the basic quantization parameters of the target macroblock based on the quantization adjustment coefficients of the target macroblock to obtain the updated quantization parameters.

[0133] It should be noted that the higher the temporal complexity of the target macroblock, the larger the calculated quantization adjustment coefficient, resulting in poorer encoded image quality. Conversely, the lower the temporal complexity of the target macroblock, the smaller the calculated quantization adjustment coefficient, resulting in better encoded image quality. Therefore, based on the quantization adjustment coefficient of the target macroblock, the basic quantization parameters of the target macroblock are updated to obtain updated quantization parameters. Encoding of the target macroblock is then performed based on these updated quantization parameters. If the target macroblock changes more frequently, the calculated updated quantization parameters will be larger, thus reducing the image quality of the target macroblock; similarly, the more frequent the changes in the target macroblock, the larger the calculated updated quantization parameters will be, thus reducing the image quality of the target macroblock.

[0134] In one optional implementation of this embodiment, the basic quantization parameters of the target macroblock are updated according to the quantization adjustment coefficients of the target macroblock to obtain updated quantization parameters, including:

[0135] The updated quantization parameters are obtained by updating the basic quantization parameters of the target macroblock based on the quantization parameter compensation coefficient and the quantization adjustment coefficient of the target macroblock.

[0136] It should be noted that if a quantization parameter compensation coefficient is introduced, the basic quantization parameters of the target macroblock can be updated by combining the quantization parameter compensation coefficient and the quantization adjustment coefficient of the target macroblock. This allows macroblocks with different degrees of variation to be encoded using different quantization parameters, thereby specifically controlling the encoded image quality of macroblocks with different degrees of variation while ensuring overall bitrate balance.

[0137] In one optional implementation of this embodiment, the step of updating the basic quantization parameters of the target macroblock based on the quantization parameter compensation coefficient and the quantization adjustment coefficient of the target macroblock to obtain updated quantization parameters includes:

[0138] The initial adjustment parameters are obtained by subtracting the quantization adjustment coefficient of the target macroblock from its base quantization parameters.

[0139] The sum of the initial adjustment parameter and the quantization parameter compensation coefficient is determined as the updated quantization parameter.

[0140] In actual implementation, the basic quantization parameters of the target macroblock are updated based on the quantization adjustment coefficient and quantization parameter compensation coefficient, and then superimposed onto the original code control result. Specifically, the updated quantization parameters are obtained by updating the basic quantization parameters of the target macroblock based on the quantization parameter compensation coefficient and the quantization adjustment coefficient of the target macroblock, which can be achieved by the following formula (8):

[0141] QP'(i,j)=QP(i,j)-DeltaQp(i,j,n)+F(n) comp (8)

[0142] Where QP′(i,j) represents the updated quantization parameter of the macroblock at position (i,j); QP(i,j) represents the basic quantization parameter of the macroblock at position (i,j).

[0143] In the embodiments of this specification, the initial adjustment parameters can be obtained by subtracting the quantization adjustment coefficient of the target macroblock from the basic quantization parameters of the target macroblock; then, the sum of the initial adjustment parameters and the quantization parameter compensation coefficient is determined as the updated quantization parameters, thereby superimposing the quantization adjustment coefficient into the code control result, so as to achieve different encoding quality for macroblocks with different degrees of change, optimize encoding performance, and improve encoding effect.

[0144] Step 108: Encode the target macroblock according to the updated quantization parameters.

[0145] Example, Figure 2b This is a schematic diagram of an encoding process provided in one embodiment of this specification, such as... Figure 2b As shown, after inputting the video frames to be encoded, the macroblock temporal complexity (MSSE) of each macroblock is determined based on the degree of inter-frame variation. Then, the frame temporal complexity of the video frames to be encoded is calculated based on the macroblock temporal complexity. The quantization adjustment coefficient (DQP) of the target macroblock is calculated based on the macroblock temporal complexity and the frame temporal complexity. The quantization parameter compensation coefficient is calculated based on the quantization adjustment coefficient of each macroblock. The quantization adjustment coefficient and the quantization parameter compensation coefficient are applied to the initial base quantization parameter (QP) to obtain the updated quantization parameter (QP') for quantization.

[0146] The process involves inverse quantization and inverse transform, followed by deblocking filtering, frame reconstruction, and reference frame determination. Intra-frame prediction / inter-frame prediction is then performed on the input video frame to be encoded to obtain the prediction result. The video frame to be encoded and the prediction result are then transformed and quantized to output the encoded result.

[0147] The video encoding method provided in the embodiments of this specification calculates the macroblock temporal complexity of each macroblock based on the degree of inter-frame variation in each macroblock within the video frame to be encoded. Then, combined with the frame temporal complexity of the video frame to be encoded, it calculates the quantization adjustment coefficient of the target macroblock. Based on the quantization adjustment coefficient of the macroblock to be enhanced, it calculates the quantization parameter compensation coefficient of the video frame to be encoded. The basic quantization parameters are updated by combining the quantization adjustment coefficient and the quantization parameter compensation coefficient to obtain updated quantization parameters, which are then used to encode the target macroblock. In this way, by predicting the degree of change in the image content of the region where the macroblock is located based on the macroblock temporal complexity, macroblocks with different degrees of change can be encoded using different quantization parameters. This allows for targeted control of the encoded image quality of macroblocks with different degrees of change. Furthermore, by introducing the quantization parameter compensation coefficient, while enhancing the image quality of some macroblocks, the image quality of other macroblocks is reduced, ensuring overall bitrate balance and thus guaranteeing the video encoding effect and optimizing video encoding performance.

[0148] See Figure 3a , Figure 3a A flowchart of an encoding method for a cloud desktop image according to an embodiment of this specification is shown, which is applied to a backend server of a cloud application and specifically includes the following steps 302-308.

[0149] Step 302: Obtain the cloud desktop image to be encoded, divide the cloud desktop image into multiple macroblocks, and determine the macroblock temporal complexity of each macroblock based on the degree of inter-frame variation of each macroblock in the cloud desktop image.

[0150] In one optional implementation of this embodiment, the macroblock temporal complexity of each macroblock is determined based on the degree of inter-frame variation of each macroblock in the cloud desktop image, including:

[0151] Based on the pixel differences of each macroblock in the target image frame and the previous image frame, the degree of inter-frame variation of each macroblock in the target image frame is determined. The target image frame is the image frame before the cloud desktop image in the cloud desktop video to be encoded.

[0152] Based on the degree of inter-frame variation of each macroblock in the target image frame, the degree of inter-frame variation of each macroblock in the cloud desktop image is calculated.

[0153] The macroblock temporal complexity of each macroblock in the cloud desktop image is determined based on the degree of inter-frame variation of each macroblock in the cloud desktop image, the weighting factor, and the historical temporal complexity of each macroblock in the previous image frame of the cloud desktop image.

[0154] Step 304: Determine the frame time-domain complexity of the cloud desktop image based on the macroblock time-domain complexity of each macroblock. Calculate the quantization adjustment coefficient of the target macroblock based on the frame time-domain complexity and the target macroblock time-domain complexity, where the target macroblock is the current macroblock to be encoded.

[0155] In an optional implementation of this embodiment, after calculating the quantization adjustment coefficient of the target macroblock based on the frame temporal complexity and the target macroblock temporal complexity, the method further includes:

[0156] Based on the quantization adjustment coefficients of each macroblock, the quantization parameter compensation coefficients of the cloud desktop image are determined. These quantization parameter compensation coefficients are used to control the bitrate balance of the cloud desktop image.

[0157] Step 306: Update the basic quantization parameters of the target macroblock based on the quantization adjustment coefficients of the target macroblock to obtain the updated quantization parameters.

[0158] In an optional implementation of this embodiment, if after calculating the quantization adjustment coefficient of the target macroblock, the quantization parameter compensation coefficient of the cloud desktop image is also calculated, then the basic quantization parameters of the target macroblock are updated according to the quantization adjustment coefficient of the target macroblock to obtain updated quantization parameters, including:

[0159] The updated quantization parameters are obtained by updating the basic quantization parameters of the target macroblock based on the quantization parameter compensation coefficient and the quantization adjustment coefficient of the target macroblock.

[0160] Step 308: Encode the target macroblock according to the updated quantization parameters, and transmit the encoded macroblock data to the cloud application client.

[0161] The cloud desktop image encoding method provided in the embodiments of this specification can be applied to the backend server of cloud applications. It calculates the macroblock temporal complexity of each macroblock based on the degree of inter-frame variation in the macroblocks within the cloud desktop image. Then, combining this with the frame temporal complexity of the cloud desktop image, it calculates the quantization adjustment coefficient of the target macroblock. Based on the calculated quantization adjustment coefficient, it calculates the quantization parameter compensation coefficient of the cloud desktop image. The basic quantization parameters are updated by combining the quantization adjustment coefficient and the quantization parameter compensation coefficient to obtain the updated quantization parameters. The target macroblock is then encoded, and the encoded macroblock data is transmitted to the front-end client for display. Thus, for areas with slower changes during rapid scrolling of the cloud desktop image, the calculated quantization adjustment coefficient is smaller, resulting in a smaller update of the quantization parameters. Most details are preserved, enhancing the encoded image quality. Simultaneously, the quantization parameter compensation coefficient ensures overall bitrate balance, making areas with slower changes during rapid scrolling of the cloud desktop image clearer than before optimization.

[0162] The above is an illustrative scheme of a cloud desktop image encoding method according to this embodiment. It should be noted that the technical solution of this cloud desktop image encoding method belongs to the same concept as the technical solution of the video encoding method described above. For details not described in detail in the technical solution of the cloud desktop image encoding method, please refer to the description of the technical solution of the video encoding method described above.

[0163] The following is in conjunction with the appendix Figure 3b This document uses the video encoding method provided in this manual as an example of its application in a cloud desktop to further explain the video encoding method. Specifically, Figure 3b This specification illustrates a flowchart of the processing procedure of a video encoding method according to one embodiment. Figure 3b As shown, the backend server of the cloud desktop obtains the cloud desktop image to be encoded, which is a video frame to be encoded, and divides the cloud desktop image into multiple macroblocks.

[0164] The macroblock temporal complexity of each macroblock is determined based on the degree of inter-frame variation. The sum of the macroblock temporal complexities of each macroblock is determined, and the frame temporal complexity of the cloud desktop image is determined based on the sum of the macroblock temporal complexities of each macroblock and the number of macroblocks in the cloud desktop image.

[0165] For scenarios involving rapid scrolling of images on cloud desktops, such as the target macroblock corresponding to the task manager area, the quantization adjustment coefficient of the target macroblock is calculated by combining the frame time-domain complexity and the target macroblock time-domain complexity.

[0166] The quantization parameter compensation coefficients of the cloud desktop image are determined based on the quantization adjustment coefficients of each macroblock.

[0167] By combining the quantization adjustment coefficients and quantization parameter compensation coefficients of the target macroblock, the basic quantization parameters of the target macroblock are updated, and the updated quantization parameters are output. The updated quantization parameters are then used to encode the task manager area, and the encoded desktop image is pushed to the front-end device for display.

[0168] In the embodiments of this specification, when the cloud desktop image scrolls rapidly, the text in the task manager area changes slowly, the calculated quantization adjustment coefficient is small, and the quantization parameter is updated less, so most of the details are preserved, which enhances the encoded image quality. At the same time, the quantization parameter compensation coefficient is used to ensure the overall bitrate balance. When the desktop image scrolls rapidly, the text in the task manager area is clearer than before optimization.

[0169] Corresponding to the above method embodiments, this specification also provides embodiments of video encoding apparatus. Figure 4 A schematic diagram of a video encoding apparatus according to one embodiment of this specification is shown. Figure 4 As shown, the device includes:

[0170] The first determining module 402 is configured to divide the video frame to be encoded into multiple macroblocks and determine the macroblock temporal complexity of each macroblock based on the degree of macroblock variation of each macroblock in the video frame to be encoded.

[0171] The first calculation module 404 is configured to determine the frame time-domain complexity of the video frame to be encoded based on the macroblock time-domain complexity of each macroblock, and to calculate the quantization adjustment coefficient of the target macroblock based on the frame time-domain complexity and the target macroblock time-domain complexity of the target macroblock, wherein the target macroblock is the current macroblock to be encoded.

[0172] The first update module 406 is configured to update the basic quantization parameters of the target macroblock according to the quantization adjustment coefficient of the target macroblock, so as to obtain updated quantization parameters;

[0173] Encoding module 408 is configured to encode the target macroblock according to the updated quantization parameters.

[0174] Optionally, the first determining module 402 is further configured as follows:

[0175] Based on the pixel differences of each macroblock in the target video frame and the video frame preceding the target video frame, the degree of inter-frame variation of each macroblock in the target video frame is determined, wherein the target video frame is the video frame preceding the video frame to be encoded in the video to be encoded.

[0176] Based on the degree of inter-frame variation of each macroblock in the target video frame, the degree of inter-frame variation of each macroblock in the video frame to be encoded is calculated.

[0177] The macroblock temporal complexity of each macroblock in the video frame to be encoded is determined based on the degree of inter-frame variation of each macroblock in the video frame to be encoded, the weighting factor, and the historical temporal complexity of each macroblock in the previous video frame of the video frame to be encoded.

[0178] Optionally, the first determining module 402 is further configured as follows:

[0179] For each macroblock in the target video frame, the pixel value of each pixel in the macroblock is subtracted from the pixel value of each pixel in the macroblock at the same position in the previous video frame to obtain the pixel difference of each pixel in the macroblock; the pixel differences of each pixel are summed to obtain the degree of inter-frame variation of the macroblock.

[0180] Optionally, the first computing module 404 is further configured as follows:

[0181] Determine the sum of the macroblock time-domain complexity of each macroblock;

[0182] The frame time complexity of the video frame to be encoded is determined based on the sum of the macroblock time-domain complexities of each macroblock and the number of macroblocks in the video frame to be encoded.

[0183] Optionally, the first computing module 404 is further configured as follows:

[0184] Determine the ratio parameter between the temporal complexity of the target macroblock and the temporal complexity of the frame;

[0185] The quantization adjustment coefficient of the target macroblock is calculated based on the ratio parameter and the set adjustment coefficient in the encoding rule.

[0186] Optionally, the setting adjustment coefficient includes a threshold for frame time-domain complexity, an adjustment coefficient for frame time-domain complexity, and an adjustment coefficient for quantization parameters; the first calculation module 404 is further configured to:

[0187] Subtract the threshold of the frame temporal complexity from the ratio parameter to obtain the first intermediate value;

[0188] Subtract the adjustment coefficient of the frame temporal complexity from the ratio parameter to obtain the second intermediate value;

[0189] The ratio of the first intermediate value to the second intermediate value is multiplied by the quantization parameter adjustment coefficient to obtain the quantization adjustment coefficient of the target macroblock.

[0190] Optionally, the device further includes a third determining module configured to:

[0191] Based on the quantization adjustment coefficients of each macroblock, the quantization parameter compensation coefficients of the video frame to be encoded are determined, wherein the quantization parameter compensation coefficients are used to control the bitrate balance of the video frame to be encoded.

[0192] Accordingly, the first update module 406 is further configured as follows:

[0193] The updated quantization parameters are obtained by updating the basic quantization parameters of the target macroblock based on the quantization parameter compensation coefficient and the quantization adjustment coefficient of the target macroblock.

[0194] Optionally, the third determining module is further configured as follows:

[0195] Macroblocks whose quantization adjustment coefficient is less than a set threshold are identified as macroblocks to be enhanced.

[0196] The quantization parameter compensation coefficients of the video frame to be encoded are calculated based on the quantization adjustment coefficients of the macroblock to be enhanced.

[0197] Optionally, the third determining module is further configured as follows:

[0198] Determine the sum of the quantization adjustment coefficients for each macroblock to be enhanced;

[0199] The quantization parameter compensation coefficient of the video frame to be encoded is calculated based on the sum of the quantization adjustment coefficients of each macroblock to be enhanced and the number of macroblocks in the video frame to be encoded.

[0200] Optionally, the first update module 406 is further configured as follows:

[0201] The initial adjustment parameters are obtained by subtracting the quantization adjustment coefficient of the target macroblock from its base quantization parameters.

[0202] The sum of the initial adjustment parameter and the quantization parameter compensation coefficient is determined as the updated quantization parameter.

[0203] The video encoding apparatus provided in the embodiments of this specification includes a first determining module, a first calculating module, a first updating module, and an encoding module. These modules are used to predict the degree of change in the image content of the region where a macroblock is located based on the temporal complexity of the macroblock. This allows macroblocks with different degrees of change to be encoded using different quantization parameters, enabling targeted control of the encoded image quality of macroblocks with different degrees of change. Furthermore, a quantization parameter compensation coefficient is introduced to enhance the image quality of certain macroblocks while reducing the image quality of other macroblocks, ensuring overall bitrate balance and thus guaranteeing the effectiveness of video encoding and optimizing its performance.

[0204] The above is an illustrative scheme of a video encoding device according to this embodiment. It should be noted that the technical solution of this video encoding device and the technical solution of the video encoding method described above belong to the same concept. For details not described in detail in the technical solution of the video encoding device, please refer to the description of the technical solution of the video encoding method described above.

[0205] Corresponding to the above method embodiments, this specification also provides embodiments of a cloud desktop image encoding device. Figure 5 A schematic diagram of a cloud desktop image encoding device according to one embodiment of this specification is shown. Figure 5 As shown, the device includes:

[0206] The second determining module 502 is configured to acquire the cloud desktop image to be encoded, divide the cloud desktop image into multiple macroblocks, and determine the macroblock temporal complexity of each macroblock based on the degree of inter-frame variation of each macroblock in the cloud desktop image.

[0207] The second calculation module 504 is configured to determine the frame time-domain complexity of the cloud desktop image based on the macroblock time-domain complexity of each macroblock, and to calculate the quantization adjustment coefficient of the target macroblock based on the frame time-domain complexity and the target macroblock time-domain complexity of the target macroblock, wherein the target macroblock is the current macroblock to be encoded.

[0208] The second update module 506 is configured to update the basic quantization parameters of the target macroblock according to the quantization adjustment coefficient of the target macroblock, so as to obtain updated quantization parameters;

[0209] The transmission module 508 is configured to encode the target macroblock according to the updated quantization parameters and transmit the encoded macroblock data to the client of the cloud application.

[0210] Optionally, the second determining module 502 is further configured as follows:

[0211] Based on the pixel difference between each macroblock in the target image frame and the previous image frame, the degree of inter-frame variation of each macroblock in the target image frame is determined, wherein the target image frame is the image frame preceding the cloud desktop image in the cloud desktop video to be encoded.

[0212] Based on the degree of inter-frame variation of each macroblock in the target image frame, the degree of inter-frame variation of each macroblock in the cloud desktop image is calculated.

[0213] The macroblock temporal complexity of each macroblock in the cloud desktop image is determined based on the degree of inter-frame variation of each macroblock in the cloud desktop image, the weighting factor, and the historical temporal complexity of each macroblock in the previous image frame of the cloud desktop image.

[0214] Optionally, the device further includes a fourth determining module, configured to:

[0215] Based on the quantization adjustment coefficients of each macroblock, the quantization parameter compensation coefficients of the cloud desktop image are determined, wherein the quantization parameter compensation coefficients are used to control the bitrate balance of the cloud desktop image.

[0216] Accordingly, the second update module 506 is further configured as follows:

[0217] The updated quantization parameters are obtained by updating the basic quantization parameters of the target macroblock based on the quantization parameter compensation coefficient and the quantization adjustment coefficient of the target macroblock.

[0218] The cloud desktop image encoding device provided in the embodiments of this specification can be applied to the backend server of cloud applications. It includes a second determining module, a second calculating module, a second updating module, and a transmission module. By using the above modules, for areas where the cloud desktop image changes slowly when scrolling rapidly, the calculated quantization adjustment coefficient is smaller, and thus the updated quantization parameter is smaller. Most of the details are preserved, enhancing the encoded image quality. At the same time, the quantization parameter compensation coefficient ensures the overall bitrate balance. When the cloud desktop image scrolls rapidly, areas where the change is slow are clearer than before optimization.

[0219] The above is a schematic scheme of a cloud desktop image encoding device according to this embodiment. It should be noted that the technical solution of this cloud desktop image encoding device belongs to the same concept as the technical solutions of the video encoding method and the cloud desktop image encoding method described above. For details not described in detail in the technical solution of the cloud desktop image encoding device, please refer to the description of the technical solutions of the video encoding method and the cloud desktop image encoding method described above.

[0220] Figure 6 A structural block diagram of a computing device according to one embodiment of this specification is shown. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0221] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0222] In one embodiment of this specification, the above-described components of the computing device 600 and Figure 6 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 6 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0223] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0224] The processor 620 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the aforementioned video encoding method or cloud desktop image encoding method.

[0225] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solutions of the video encoding method and the cloud desktop image encoding method described above. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the video encoding method and the cloud desktop image encoding method described above.

[0226] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described video encoding method or cloud desktop image encoding method.

[0227] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the video encoding method and the cloud desktop image encoding method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solutions of the video encoding method and the cloud desktop image encoding method described above.

[0228] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described video encoding method or cloud desktop image encoding method.

[0229] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solutions of the video encoding method and the cloud desktop image encoding method described above. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solutions of the video encoding method and the cloud desktop image encoding method described above.

[0230] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0231] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0232] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0233] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0234] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for video coding, comprising: dividing a video frame to be coded into a plurality of macroblocks, determining macroblock temporal complexity of each macroblock in the video frame to be coded according to macroblock inter-frame variation of the macroblock; determining frame temporal complexity of the video frame to be coded according to the macroblock temporal complexity of each macroblock, calculating a quantization adjustment coefficient of a target macroblock according to the frame temporal complexity and target macroblock temporal complexity of the target macroblock, wherein the target macroblock is a current macroblock to be coded; updating a base quantization parameter of the target macroblock according to the quantization adjustment coefficient of the target macroblock to obtain an updated quantization parameter; and coding the target macroblock according to the updated quantization parameter. 2.The method of claim 1, wherein the macroblock temporal complexity of each macroblock in the video frame to be coded is determined according to macroblock inter-frame variation of each macroblock in a target video frame and a previous video frame of the target video frame, wherein the target video frame is a video frame before the video frame to be coded in the video to be coded; the macroblock inter-frame variation of each macroblock in the target video frame is calculated according to the macroblock inter-frame variation of each macroblock in the target video frame; and the macroblock temporal complexity of each macroblock in the video frame to be coded is determined according to the macroblock inter-frame variation of each macroblock in the video frame to be coded, a weight factor and historical temporal complexity of each macroblock in a previous video frame of the video frame to be coded. 3.The method of claim 2, wherein the macroblock inter-frame variation of each macroblock in the target video frame is determined according to pixel difference of each pixel point in the macroblock in the target video frame and the macroblock in the previous video frame of the target video frame, wherein the pixel difference of each pixel point in the macroblock is obtained by subtracting pixel value of each pixel point in the macroblock in the previous video frame of the target video frame from pixel value of each pixel point in the macroblock in the target video frame; and the macroblock inter-frame variation of the macroblock is obtained by summing the pixel difference of each pixel point. 4.The method of any one of claims 1-3, wherein the frame temporal complexity of the video frame to be coded is determined according to the sum of the macroblock temporal complexity of each macroblock; and the frame temporal complexity of the video frame to be coded is determined according to the sum of the macroblock temporal complexity of each macroblock and the number of macroblocks in the video frame to be coded. 5.The method of any one of claims 1-3, wherein the quantization adjustment coefficient of the target macroblock is calculated according to a proportional parameter of the target macroblock temporal complexity and the frame temporal complexity; and the quantization adjustment coefficient of the target macroblock is calculated according to the proportional parameter and a set adjustment coefficient in a coding rule. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 6. The video coding method of claim 5, wherein the setting the adjustment factor comprises a frame temporal complexity threshold, a frame temporal complexity adjustment factor, and a quantization parameter adjustment factor; and wherein the calculating the quantization adjustment factor of the target macroblock according to the scaling parameter and the set adjustment factor in the coding rule comprises: subtracting the frame temporal complexity threshold from the scaling parameter to obtain a first intermediate value; subtracting the frame temporal complexity adjustment factor from the scaling parameter to obtain a second intermediate value; multiplying the scaling parameter by a ratio of the first intermediate value and the second intermediate value to obtain the quantization adjustment factor of the target macroblock.

7. The video coding method of any one of claims 1-3, wherein after the calculating the quantization adjustment factor of the target macroblock according to the frame temporal complexity and the target macroblock temporal complexity of the target macroblock, the method further comprises: determining a quantization parameter compensation factor of the video frame to be coded according to the quantization adjustment factor of each macroblock, wherein the quantization parameter compensation factor is used to control a code rate balance of the video frame to be coded; and correspondingly, the updating the base quantization parameter of the target macroblock according to the quantization adjustment factor of the target macroblock to obtain an updated quantization parameter comprises: updating the base quantization parameter of the target macroblock according to the quantization parameter compensation factor and the quantization adjustment factor of the target macroblock to obtain the updated quantization parameter.

8. The video coding method of claim 7, wherein the determining the quantization parameter compensation factor of the video frame to be coded according to the quantization adjustment factor of each macroblock comprises: determining a macroblock whose quantization adjustment factor is less than a set threshold as an enhanced macroblock; and calculating the quantization parameter compensation factor of the video frame to be coded according to the quantization adjustment factor of the enhanced macroblock.

9. The video coding method of claim 8, wherein the calculating the quantization parameter compensation factor of the video frame to be coded according to the quantization adjustment factor of the enhanced macroblock comprises: determining a sum of the quantization adjustment factors of the enhanced macroblocks; and calculating the quantization parameter compensation factor of the video frame to be coded according to the sum of the quantization adjustment factors of the enhanced macroblocks and a number of macroblocks of the video frame to be coded.

10. The video coding method of claim 7, wherein the updating the base quantization parameter of the target macroblock according to the quantization parameter compensation factor and the quantization adjustment factor of the target macroblock to obtain the updated quantization parameter comprises: obtaining an initial adjustment parameter by subtracting the quantization adjustment factor of the target macroblock from the base quantization parameter of the target macroblock; and determining a sum of the initial adjustment parameter and the quantization parameter compensation factor as the updated quantization parameter.

11. A cloud desktop image coding method applied to a backend server of a cloud application, comprising: obtaining a cloud desktop image to be coded, and dividing the cloud desktop image into a plurality of macroblocks, and determining a macroblock temporal complexity of each macroblock in the cloud desktop image according to a macroblock inter-frame variation degree of the macroblock. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ determine a frame temporal complexity of the cloud desktop image according to the macroblock temporal complexity of each macroblock, calculate a quantization adjustment coefficient of a target macroblock according to the frame temporal complexity and a target macroblock temporal complexity of the target macroblock, wherein the target macroblock is a current to-be-encoded macroblock; update a basic quantization parameter of the target macroblock according to the quantization adjustment coefficient of the target macroblock to obtain an updated quantization parameter; encode the target macroblock according to the updated quantization parameter, and transmit encoded macroblock data to a client of the cloud application.

12. The cloud desktop image encoding method of claim 11, wherein the macroblock temporal complexity of each macroblock in the cloud desktop image is determined according to a macroblock interframe variation degree of each macroblock in the cloud desktop image, and the macroblock interframe variation degree of each macroblock in the target image frame is determined according to a pixel difference value of each macroblock in the target image frame and a previous image frame of the target image frame.

12. The cloud desktop image encoding method of claim 11, wherein the macroblock temporal complexity of each macroblock in the cloud desktop image is determined according to a macroblock interframe variation degree of each macroblock in the cloud desktop image, and the macroblock interframe variation degree of each macroblock in the target image frame is determined according to a pixel difference value of each macroblock in the target image frame and a previous image frame of the target image frame.

13. The cloud desktop image encoding method of claim 11 or 12, wherein after the quantization adjustment coefficient of the target macroblock is calculated according to the frame temporal complexity and the target macroblock temporal complexity of the target macroblock, the method further comprises: determining a quantization parameter compensation coefficient of the cloud desktop image according to the quantization adjustment coefficient of each macroblock, wherein the quantization parameter compensation coefficient is used to control a code rate balance of the cloud desktop image.

13. The cloud desktop image encoding method of claim 11 or 12, wherein after the quantization adjustment coefficient of the target macroblock is calculated according to the frame temporal complexity and the target macroblock temporal complexity of the target macroblock, the method further comprises: determining a quantization parameter compensation coefficient of the cloud desktop image according to the quantization adjustment coefficient of each macroblock, wherein the quantization parameter compensation coefficient is used to control a code rate balance of the cloud desktop image.

14. A video encoding apparatus, comprising: a first determining module configured to divide a to-be-encoded video frame into a plurality of macroblocks, and determine a macroblock temporal complexity of each macroblock in the to-be-encoded video frame according to a macroblock variation degree of each macroblock in the to-be-encoded video frame; a first calculating module configured to determine a frame temporal complexity of the to-be-encoded video frame according to the macroblock temporal complexity of each macroblock, and calculate a quantization adjustment coefficient of a target macroblock according to the frame temporal complexity and a target macroblock temporal complexity of the target macroblock, wherein the target macroblock is a current to-be-encoded macroblock; a first updating module configured to update a basic quantization parameter of the target macroblock according to the quantization adjustment coefficient of the target macroblock to obtain an updated quantization parameter; and an encoding module configured to encode the target macroblock according to the updated quantization parameter. ​ ​ ​ ​ ​ ​ ​ ​ 15. An apparatus for encoding a cloud desktop image, applied to a backend server of a cloud application, comprising: a second determining module configured to obtain a cloud desktop image to be encoded, divide the cloud desktop image into a plurality of macroblocks, and determine a macroblock time domain complexity of each macroblock in the cloud desktop image according to a macroblock inter-frame variation degree of the each macroblock; a second calculating module configured to determine a frame time domain complexity of the cloud desktop image according to the macroblock time domain complexity of the each macroblock, calculate a quantization adjustment coefficient of a target macroblock according to the frame time domain complexity and a target macroblock time domain complexity of the target macroblock, wherein the target macroblock is a current macroblock to be encoded; a second updating module configured to update a basic quantization parameter of the target macroblock according to the quantization adjustment coefficient of the target macroblock, and obtain an updated quantization parameter; a transmitting module configured to encode the target macroblock according to the updated quantization parameter, and transmit encoded macroblock data to a client of the cloud application.

16. A computing device, comprising: a memory and a processor; the memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, which, when executed by the processor, implement steps of the video encoding method of any one of claims 1 to 10 or the cloud desktop image encoding method of any one of claims 11 to 13.

17. A computer readable storage medium storing computer programs / instructions, which, when executed by a processor, implement steps of the video encoding method of any one of claims 1 to 10 or the cloud desktop image encoding method of any one of claims 11 to 13.

18. A computer program product comprising computer programs / instructions, which, when executed by a processor, implement steps of the video encoding method of any one of claims 1 to 10 or the cloud desktop image encoding method of any one of claims 11 to 13.

Images