Coding method, electronic device, storage medium and computer program product

Abstract

The invention provides a coding method, electronic equipment, a storage medium and a computer program product, and the method comprises the steps: carrying out the pixel reconstruction of a plurality of TUs in a video frame in parallel, and obtaining a reconstruction value corresponding to the pixel value of each pixel in each TU; performing mode decision on the plurality of TUs in parallel to obtain a mode decision result corresponding to each TU, and coding the TU based on the mode decision result corresponding to each TU; wherein the first TU refers to a reconstruction value corresponding to a pixel value of a pixel in the second TU in a mode decision process; the first TU represents one TU in the plurality of TUs, and the second TU represents a TU adjacent to the first TU.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an encoding method, electronic device, storage medium, and computer program product. Background Technology

[0002] In related technologies, during the encoding of a Transform Unit (TU) within a video frame, mode decisions are made with reference to the pixels of the adjacent encoded TUs, and then encoding is performed based on the mode decision results, resulting in low encoding efficiency. Summary of the Invention

[0003] To address the related technical problems, embodiments of this application provide an encoding method, an electronic device, a storage medium, and a computer program product.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides an encoding method, the method comprising: Pixel reconstruction is performed on multiple TUs within a video frame in parallel, and the reconstructed value corresponding to the pixel value of each pixel in each TU is obtained respectively; Mode decisions are performed on the multiple TUs in parallel to obtain the mode decision result for each TU, and the TU is encoded based on the mode decision result for each TU; wherein, the first TU refers to the reconstructed value corresponding to the pixel value of the pixel in the second TU during the mode decision process; the first TU represents one of the multiple TUs, and the second TU represents the TU adjacent to the first TU.

[0005] In the above scheme, when reconstructing a pixel in one of the plurality of TUs, the method includes: Based on the size of the TU, the prediction mode corresponding to the TU is determined from multiple set prediction modes; Based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU, the reconstructed value corresponding to the pixel value of each pixel in the TU is determined.

[0006] In the above scheme, the set multiple prediction modes are determined from the multiple candidate prediction modes based on the first statistical result; the first statistical result is used to describe the distribution of the selection probability corresponding to the multiple candidate prediction modes respectively; the selection probability is used to describe the probability of deciding to select the corresponding prediction mode in the historical mode decision-making process.

[0007] In the above scheme, determining the prediction mode corresponding to the TU from a set of multiple prediction modes based on the TU's size includes: When the size of the TU is characterized as a set first size, the set first prediction mode is determined as the prediction mode corresponding to the TU; and / or, When the size of the TU is represented as a size other than the first size, the prediction mode corresponding to the TU is determined from a plurality of set second prediction modes based on a plurality of first costs corresponding to the TU; each of the plurality of first costs represents the bit rate loss cost of performing pixel prediction on the TU based on one of the plurality of second prediction modes; the prediction mode corresponding to the TU is represented as the second prediction mode corresponding to the minimum first cost among the plurality of first costs.

[0008] In the above scheme, the first prediction mode is characterized as a DC mode; and / or, When the length and width of the TU are equal, the set plurality of second prediction modes include: DC mode, V mode, H mode, and D45 mode; and / or, When the length and width of the TU are not equal, the set multiple second prediction modes include: V mode, H mode and D45 mode.

[0009] In the above scheme, determining the reconstructed value corresponding to the pixel value of each pixel in the TU based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU includes: Based on the prediction mode corresponding to the TU, pixel prediction is performed on the pixels in the TU to obtain the predicted value corresponding to the pixel value of each pixel in the TU; wherein, the first TU refers to the pixel value of the pixel in the second TU during the pixel prediction process; The difference between the pixel value and the corresponding predicted value of each pixel in the TU is calculated to obtain the first residual value for each pixel in the TU; The multiple first residual values ​​corresponding to the TU are transformed, quantized, dequantized and inverse transformed to obtain the second residual value corresponding to each pixel in the TU. The predicted value corresponding to the pixel value of each pixel in the TU is summed with the second residual value to obtain the reconstructed value corresponding to the pixel value of each pixel in the TU.

[0010] In the above scheme, the multiple TUs represent all TUs within the same coding tree unit (CTU) in the video frame.

[0011] This application also provides an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor. The processor is used to run the computer program and execute the steps of any of the aforementioned methods.

[0012] This application also provides a storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of any of the aforementioned methods.

[0013] This application also provides a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of any of the aforementioned methods.

[0014] In this embodiment, pixel reconstruction is performed on multiple Units (TUs) within a video frame in parallel, obtaining the reconstructed value corresponding to the pixel value of each pixel in each TU. Then, mode decision is performed on multiple TUs in parallel, obtaining the mode decision result corresponding to each TU. The TU is then encoded based on the mode decision result corresponding to each TU. The first TU references the reconstructed value corresponding to the pixel value of a pixel in a second TU during the mode decision process. The first TU represents one of the multiple TUs, and the second TU represents the TU adjacent to the first TU. In the above scheme, the reconstructed pixels of each TU in the multiple TUs within the video frame are determined in advance through parallel pixel reconstruction processing. Then, for each TU, mode decision is performed with reference to the reconstructed pixels of its adjacent TUs. Thus, when performing mode decision on a TU based on reconstructed pixels, it is not necessary to rely on the final encoding result of the previous TU. That is, it is not necessary to perform mode decision after completing the mode decision on the previous TU. Based on this, mode decision and encoding of multiple TUs based on reconstructed pixels can be performed in parallel, improving encoding efficiency while ensuring encoding accuracy. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of a pattern decision-making process in related technologies; Figure 2 This is a schematic diagram of an encoding sequence in related technologies; Figure 3 This is a schematic diagram of another encoding order in related technologies; Figure 4 A schematic diagram illustrating the implementation flow of an encoding method provided in an embodiment of this application; Figure 5 A schematic diagram of statistical results provided for an embodiment of this application; Figure 6 This is a schematic diagram illustrating another statistical result provided for an embodiment of this application; Figure 7 A schematic diagram of the processing flow of an encoding method provided for an application embodiment of this application; Figure 8 This is a schematic diagram of the processing flow of an encoding method in related technologies; Figure 9A schematic diagram illustrating the processing of a predicted reconstruction model provided for an application embodiment of this application; Figure 10 A schematic diagram of the processing flow of a reconstruction estimation method provided for an application embodiment of this application; Figure 11 A schematic diagram of the processing flow of a pattern prediction method provided for an application embodiment of this application; Figure 12 This is a schematic diagram of the structure of an encoding device provided in an embodiment of this application; Figure 13 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0016] In related technologies, during the encoding of a TU within a video frame, mode decisions are made with reference to the pixels of the adjacent encoded TUs, and encoding is performed based on the obtained mode decision results, thereby compressing the video through encoding.

[0017] For example, see Figure 1 TU-0, TU-1, TU-2, and TU-3 are located in the same CTU. During the mode decision-making process for TU-3, the reconstructed pixels of TU-0, TU-1, and TU-2 are required. Reconstructed pixels can be understood as the reconstructed values ​​corresponding to the pixel values ​​of individual pixels. These reconstructed pixels are obtained based on the encoded TU-0, TU-1, and TU-2. In other words, only after encoding TU-0, TU-1, and TU-2 can the required reconstructed pixels for TU-3 be obtained, thus enabling mode decision-making and further encoding of TU-3.

[0018] Based on this, during the encoding of multiple TUs within a video frame, mode decisions for multiple TUs need to be made in a serial processing manner, thus requiring serial encoding of multiple TUs, resulting in low encoding efficiency. For example, Figure 2 This illustrates the processing order for encoding individual TUs within a CTU using a serial processing approach.

[0019] In related technologies, to improve coding efficiency, during the mode decision-making process for a TU, the original pixels in its neighboring TUs are used as a reference to process the TU. These original pixels are uncoded pixels. Thus, mode decisions can be performed on multiple TUs within a video frame in a parallel manner. For example, Figure 3The diagram illustrates the processing order for mode decisions within a CTU based on parallel processing. However, there is a significant error between the reconstructed pixel and the original pixel. Therefore, this method of directly using the original pixel as a reference for mode decisions results in a large error in mode decisions, which severely reduces the accuracy of coding and causes a loss in coding performance.

[0020] Based on this, in this embodiment, pixel reconstruction is performed on multiple TUs within a video frame in parallel, obtaining the reconstructed value corresponding to the pixel value of each pixel in each TU; then, mode decision is performed on multiple TUs in parallel, obtaining the mode decision result corresponding to each TU, and the TU is encoded based on the mode decision result corresponding to each TU; wherein, the first TU refers to the reconstructed value corresponding to the pixel value of the pixel in the second TU during the mode decision process; the first TU represents one of the multiple TUs, and the second TU represents the TU adjacent to the first TU. In the above scheme, through parallel pixel reconstruction processing, the reconstructed pixels of each TU in the multiple TUs within the video frame are determined in advance, and then for each TU in the multiple TUs, mode decision is performed with reference to the reconstructed pixels of the adjacent TUs. Thus, when performing mode decision on a TU based on the reconstructed pixels, it is not necessary to rely on the final encoding result of the previous TU, that is, it is not necessary to perform it after completing the mode decision on the previous TU. On this basis, mode decision and encoding of multiple TUs based on reconstructed pixels can be performed in parallel processing, improving encoding efficiency while ensuring encoding accuracy.

[0021] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0022] This application provides an encoding method, see [link to relevant documentation] Figure 4 The method includes: Step 401: Perform pixel reconstruction on multiple TUs within the video frame in parallel, and obtain the reconstructed value corresponding to the pixel value of each pixel in each TU.

[0023] In practical applications, a video frame can be represented as a single image frame in a video. A video frame can contain one or more CTUs, each CTU can contain one or more TUs, and each TU can contain one or more pixels.

[0024] In practical applications, when performing pixel reconstruction on multiple transition units (TUs) within a video frame in parallel, the pixel reconstruction process for multiple TUs can be performed simultaneously, rather than reconstructing the pixels of one TU after completing the pixel reconstruction of the next TU. Compared to related technologies, this effectively decouples the TU-level dependencies in the pixel reconstruction process, improving the efficiency of pixel reconstruction.

[0025] By reconstructing pixels from a unit (TU), the reconstructed value corresponding to the pixel value of each pixel within that TU can be obtained. The pixel value of a pixel can also be represented as the original pixel, and the reconstructed value corresponding to the pixel value of a pixel can also be represented as the reconstructed pixel.

[0026] In practical applications, when multiple TUs within a video frame are reconstructed in parallel, for each TU, pixel reconstruction can be performed based on the original pixels of its neighboring TUs.

[0027] Step 402: Perform mode decisions on multiple TUs in parallel, obtain the mode decision results for each TU, and encode the TU based on the mode decision results for each TU.

[0028] In this process, the first TU refers to the reconstructed value corresponding to the pixel value of the pixel in the second TU; the first TU represents one of the multiple TUs, and the second TU represents the TU adjacent to the first TU.

[0029] In practical applications, the second TU can represent the TU adjacent to the first TU, and can also represent one of multiple TUs.

[0030] Here, before performing mode decisions on multiple TUs in parallel, the reconstructed pixels of multiple TUs are determined in advance through parallel pixel reconstruction. Therefore, during the parallel mode decision-making process for multiple TUs, the reconstructed pixels that the first TU needs to reference during mode decision-making can be known, instead of needing to determine the reconstructed pixels that the first TU needs to reference only after completing the mode decision for the second TU. This ensures the implementation of parallel mode decision-making for multiple TUs. Compared to related technologies, this is equivalent to decoupling the TU-level dependencies in the mode decision-making process, thus improving the efficiency of mode decision-making.

[0031] Furthermore, the first TU references the reconstructed pixels of the second TU during the mode decision-making process, rather than the original pixels of the second TU. This improves the accuracy of mode decision-making compared to the method of making mode decisions based on the original pixels of the second TU.

[0032] In practical applications, mode decision can also be understood as intra-frame mode prediction. The mode decision result can be used to indicate the predicted mode. During the encoding of the TU based on the mode decision result, the TU can be encoded based on the predicted mode indicated by the mode decision result. For example, the predicted mode can include DC mode, V mode, H mode, and D45 mode, etc. V mode, H mode, and D45 mode can all be understood as partial angle modes.

[0033] In the process of encoding a TU based on a prediction pattern, pixel prediction can be performed on the TU based on the prediction pattern to obtain the predicted pixel values ​​of the pixels in the TU. The predicted pixel values ​​of the pixels can also be expressed as predicted pixels. Then, based on each predicted pixel of the TU and the corresponding original pixel, the residual value of the pixel value can be determined. The residual value of the pixel value can also be expressed as a residual pixel. For example, the residual pixel can be characterized as the difference between the predicted pixel and the original pixel. After that, the residual pixels can be encoded into the corresponding bitstream to describe the image content of the TU. For example, the residual pixels can be encoded into the corresponding bitstream based on entropy coding.

[0034] In practical applications, residual pixels can contain less data than original pixels. Describing the image content of a TU using residual pixels takes up less storage space than describing the image content of a TU using original pixels. This is equivalent to compressing the TU and thus encoding it.

[0035] In practical applications, multiple TUs can be encoded in parallel based on the decision results of multiple modes corresponding to multiple TUs.

[0036] In this embodiment, the reconstructed pixels of each TU in a video frame are determined in advance through parallel pixel reconstruction processing. Then, for each TU, mode decision is made with reference to the reconstructed pixels of its neighboring TUs. Thus, when making mode decision for a TU based on the reconstructed pixels, it does not need to rely on the final encoding result of the previous TU. In other words, it does not need to be done after the mode decision for the previous TU is completed. Based on this, mode decision and encoding of multiple TUs can be performed in parallel based on the reconstructed pixels, which improves encoding efficiency while ensuring encoding accuracy.

[0037] In one embodiment, multiple TUs are characterized as all TUs within the same CTU in a video frame.

[0038] In practical applications, during the encoding of video frames, Continuous Transmission Units (CTUs) can be used as processing units, and multiple CTUs within a video frame can be processed sequentially. While processing a single CTU within a video frame, pixel reconstruction can be performed in parallel on all TUs within that CTU, and mode decisions can then be made in parallel on all TUs within that CTU. During mode decision-making, a first TU within that CTU references the reconstructed pixels of its corresponding second TU. Compared to related technologies, this effectively decouples the dependency between pixel reconstruction and mode decision-making from the TU level to the CTU level, improving the efficiency of pixel reconstruction and mode decision-making, and thus improving encoding efficiency.

[0039] In practical applications, TUs within the same CTU can have high spatial correlation. Encoding video frames using CTUs as processing units can improve the accuracy of pixel reconstruction and pattern prediction, thereby improving the overall coding accuracy. Furthermore, performing parallel pixel reconstruction and parallel pattern decision-making on multiple TUs within a CTU can avoid an excessive number of TUs in the parallel processing, reducing computational complexity and thus saving computational resources.

[0040] The pixel reconstruction method in the embodiments of this application will be further described below.

[0041] In one embodiment, when reconstructing a pixel of one of a plurality of TUs, the encoding method provided in this application includes: Based on the size of the TU, the prediction mode corresponding to the TU is determined from multiple set prediction modes; Based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU, the reconstructed value corresponding to the pixel value of each pixel in the TU is determined.

[0042] In practical applications, different prediction modes can be applied to different image content. For example, using the DC mode in flat areas of an image can achieve better coding performance, while using partial angle modes such as the D45 mode in edge or texture-rich areas of an image can achieve better coding performance. Coding performance can include one or more of the following: coding accuracy, coding processing efficiency, and coding compression ratio.

[0043] Here, when reconstructing pixels from a TU, the prediction mode corresponding to the TU is determined from multiple predefined prediction modes based on the TU's size. This allows for adaptive selection of a suitable prediction mode for subsequent pixel reconstruction, taking into account the image content characteristics of the TU. This improves pixel reconstruction performance and, consequently, coding performance. Pixel reconstruction performance can include one or more of the following: accuracy and efficiency of pixel reconstruction.

[0044] In practical applications, when reconstructing a TU, pixel prediction of the TU can be performed based on the prediction mode to obtain the predicted pixels of the TU, and then the reconstructed pixels of the TU can be determined based on the predicted pixels of the TU.

[0045] In one embodiment, determining the reconstructed value corresponding to the pixel value of each pixel in the TU based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU includes: Based on the prediction mode corresponding to the TU, pixel prediction is performed on the pixels in the TU to obtain the predicted value corresponding to the pixel value of each pixel in the TU; wherein, the first TU refers to the pixel value of the pixel in the second TU during the pixel prediction process; The difference between the pixel value and the corresponding predicted value of each pixel in the TU is calculated to obtain the first residual value for each pixel in the TU; The multiple first residual values ​​corresponding to the TU are transformed, quantized, dequantized and inverse transformed to obtain the second residual value corresponding to each pixel in the TU. The predicted value corresponding to the pixel value of each pixel in the TU is summed with the second residual value to obtain the reconstructed value corresponding to the pixel value of each pixel in the TU.

[0046] In practical applications, during the process of pixel prediction for the first TU based on the prediction mode corresponding to the first TU, the original pixels of the adjacent TUs of the first TU, i.e. the second TU, can be used as reference data to perform pixel prediction for the first TU based on the prediction mode.

[0047] In practical applications, transform processing can be used to convert data from the spatial domain to the frequency domain. For example, transform processing can include discrete cosine transform processing. Inverse transform processing can be considered as the inverse operation of transform processing and can be used to convert data from the frequency domain to the spatial domain.

[0048] Quantization can be used to compress data within a numerical range to reduce its volume. In practice, quantization may result in the loss of some information within the data, essentially performing lossy compression. Inverse quantization, the reverse operation, can be used to restore quantized data to its original state as close as possible.

[0049] In practical applications, the second residual value can be understood as the encoded pixel value obtained after encoding the TU based on the prediction mode corresponding to the TU. The reconstructed pixel value of the pixel in the TU can be understood as the pixel value recovered after encoding and decoding the TU.

[0050] In this embodiment, the prediction mode corresponding to the TU is determined based on the size of the TU, and then pixel prediction is performed based on the prediction mode corresponding to the TU to obtain the predicted pixels of the TU. This is equivalent to adaptively selecting a suitable prediction mode to perform pixel prediction on the TU according to the image content characteristics of the TU, which improves the accuracy and processing efficiency of pixel prediction. On this basis, the reconstructed pixels of the TU are determined based on the predicted pixels of the TU, which improves the performance of pixel reconstruction and thus improves the coding performance.

[0051] The following section provides a further explanation of how the prediction model is determined.

[0052] In one embodiment, a plurality of prediction modes are determined from a plurality of candidate prediction modes based on a first statistical result; the first statistical result is used to describe the distribution of the selection probability corresponding to each of the plurality of candidate prediction modes; the selection probability is used to describe the probability of deciding to select the corresponding prediction mode in the historical mode decision-making process.

[0053] In practical applications, multiple candidate prediction modes can be understood as a set of prediction modes that may be used during the encoding of TU. The selection probability of a prediction mode can also be understood as the probability of using that prediction mode, or simply the probability of occurrence.

[0054] In practical applications, by statistically analyzing the usage of multiple candidate prediction modes in historical mode decisions, the selection probability of these prediction modes relative to different TUs can be determined, thus obtaining a first statistical result. This first statistical result can then be used to reflect the selection of one or more prediction modes with higher probabilities under specific conditions, such as when processing TUs of a set size. This provides a limited number of alternatives for prediction mode selection, thereby improving the efficiency of determining the prediction mode corresponding to the TU and making the selected prediction mode a better match with the TU to be processed, thus improving the performance of mode decision-making and pixel reconstruction.

[0055] For example, Figure 5 It shows the size of When TU makes a mode decision, the distribution of the selection probabilities of multiple candidate prediction modes is shown, that is, it shows the multiple candidate prediction modes relative to The probability of TU occurring. Figure 6 The diagram shows the distribution of selection probabilities for multiple candidate prediction modes when making mode decisions for TUs of different sizes. In other words, it shows the probability of occurrence of multiple candidate prediction modes relative to TUs of different sizes. Figure 5 and Figure 6 All of these can be understood as the first statistical result.

[0056] In one embodiment, based on the size of the TU, the prediction mode corresponding to the TU is determined from a set of prediction modes, including: When the size of the TU is characterized as a set first size, the set first prediction mode is determined as the prediction mode corresponding to the TU; and / or, When the size of the TU is represented as a size other than the first size, the prediction mode corresponding to the TU is determined from a plurality of set second prediction modes based on a plurality of first costs corresponding to the TU; each of the plurality of first costs represents the bit rate loss cost of performing pixel prediction on the TU based on one of the plurality of second prediction modes; the prediction mode corresponding to the TU is represented as the second prediction mode corresponding to the smallest first cost among the plurality of first costs.

[0057] In practical applications, based on the first statistical results, it can be determined that when the size of the TU is a specific size, the selection probability of the specific prediction mode is higher than that of other prediction modes. Therefore, the first size is set to the specific size, and the first prediction mode is set to the specific prediction mode.

[0058] In practical applications, based on the first statistical result, it can be determined that when TU has different shapes, the prediction mode with higher probability should be selected, and then the second prediction mode should be set to the prediction mode with higher probability.

[0059] In one embodiment, the first prediction mode is characterized as a DC mode; and / or, With the length and width of the TU being equal, the set second prediction modes include: DC mode, V mode, H mode, and D45 mode; and / or, When the length and width of the TU are not equal, several second prediction modes are set, including: V mode, H mode and D45 mode.

[0060] In practical applications, see Figure 5 In the case of size When using TU for mode decision-making, the DC mode can correspond to a higher selection probability compared to other prediction modes. Based on this, the first size can be... That is, the length corresponding to the first dimension is 64, and the width corresponding to the first dimension is 64. The first prediction mode can be DC mode, thereby improving the performance of mode decision and pixel reconstruction.

[0061] In practical applications, see Figure 6 The probability of selecting DC mode relative to square TU can be higher than that of selecting rectangular TU, and the sum of the selection probabilities of DC mode, V mode, H mode and D45 mode can be higher than 20%.

[0062] based on Figure 6When the length and width of the TU are equal, i.e., the TU is square, the DC mode, V mode, H mode, and D45 mode can be regarded as the prediction modes with higher selection probability. The multiple second prediction modes set can include the DC mode, V mode, H mode, and D45 mode. When the length and width of the TU are not equal, i.e. the TU is rectangular, the V mode, H mode, and D45 mode can be regarded as the prediction modes with higher selection probability. The multiple second prediction modes set can include the V mode, H mode, and D45 mode, thereby improving the performance of mode decision and pixel reconstruction.

[0063] Here, when the size of the TU is represented as a size other than the first size, the prediction mode corresponding to the TU is determined from a set of a plurality of second prediction modes based on a plurality of first costs corresponding to the TU.

[0064] In practical applications, based on the multiple first costs corresponding to the TU, the second prediction mode corresponding to the minimum first cost can be selected from the multiple set second prediction modes, and the second prediction mode corresponding to the minimum first cost can be determined as the prediction mode corresponding to the TU.

[0065] The multiple second prediction modes can be understood as a set of candidate modes used to select the second prediction mode corresponding to the minimum first cost when the size of TU is represented as a size other than the first size.

[0066] In practical applications, during image encoding based on pixel prediction, a trade-off must be struck between the compression level corresponding to the bitrate and the image distortion caused by the encoding, resulting in a cost. For example, with a higher bitrate, the cost may include higher image distortion, i.e., poorer image quality; with lower image distortion, the cost may include a lower bitrate.

[0067] The first cost can be used to measure the overall cost required in the corresponding processing. Determining the second prediction mode corresponding to the minimum first cost as the prediction mode for TU ensures that processing TU based on this prediction mode incurs a lower overall cost compared to other prediction modes, thereby improving processing performance. The second prediction mode corresponding to the minimum first cost can also be described as the optimal mode.

[0068] For example, the optimal mode can be determined as follows:

[0069] in, This can be characterized as the optimal mode. It can be characterized as multiple predefined second prediction modes. This can be represented as the first cost.

[0070] For example, multiple second prediction modes are set. It can be represented as: .

[0071] For example, the first cost It can be represented as: , in, This can be characterized as the distortion caused by the Hadamard transformation, or understood as a parameter related to the absolute error and the cost (SATD). In practical applications, the Hadamard transformation can be performed on the residual coefficient matrix corresponding to TU to obtain multiple transformed coefficients. Then, the absolute values ​​of these transformed coefficients are summed to obtain... The residual coefficient matrix may include one or more residual coefficients, which can be determined based on the difference between the original pixels and the predicted pixels of the TU. The predicted pixels used to determine the residual coefficients can be determined based on the pixel prediction of the TU according to the corresponding prediction mode.

[0072] This can be represented by the number of bits corresponding to the prediction mode, and different prediction modes correspond to... The values ​​can be the same.

[0073] This can be characterized as a Lagrange factor related to the quantification parameters, corresponding to different prediction models. The values ​​can be different.

[0074] Quantization parameters can be considered as key parameters used to control the balance between compression rate and image quality during video encoding. Quantization parameters can directly affect the quantization accuracy of residual pixels, thereby determining encoding efficiency and the quality of reconstructed images.

[0075] For example, This can be characterized as the balance coefficient in rate-distortion optimization (RDO). It can be represented as: , in, It can be characterized as a quantization parameter.

[0076] In this embodiment, based on the size of the TU, the determination of the prediction mode corresponding to the TU is divided into different cases for adaptive processing. In the case where the size of the TU is other than the first size, the optimal mode is selected from a limited number of second prediction modes in combination with the first cost corresponding to the TU. This improves the selection efficiency, and pixel prediction of the TU based on the selected prediction mode can generate a smaller overall cost, thereby improving the performance of pixel prediction and thus improving the performance of pixel reconstruction and subsequent encoding.

[0077] The present application will be further described in detail below with reference to application examples.

[0078] This application provides an encoder that can perform video encoding based on the encoding method provided in this application.

[0079] In practical applications, see Figure 7 The encoding process of the encoder in the application embodiments of this application can mainly include the following steps: Step 1: Read the pixels.

[0080] In practical applications, the pixel values ​​of each pixel within a video frame can be read, that is, the raw pixels can be read.

[0081] Step 2: Reconstruct the estimate.

[0082] In practical applications, CTU-level reconstruction estimation can be performed, which can also be understood as pixel reconstruction. During the pixel reconstruction of each CTU within a video frame, multiple TUs within that CTU can be reconstructed in parallel, obtaining the reconstructed pixels of each TU.

[0083] Step 3: Pattern Decision.

[0084] In practical applications, mode decisions can be performed at the CTU level. During the mode decision-making process for each CTU within a video frame, mode decisions can be performed in parallel for multiple TUs within that CTU, yielding the mode decision result for each TU. When performing mode decisions for a single TU within a CTU, the reconstructed pixels of its neighboring TUs can be used as reference data for mode decision-making on that TU.

[0085] In practical applications, before making a mode decision for a TU within a CTU, a coarse mode decision can be performed on that TU to obtain the corresponding coarse mode decision result. Then, based on the coarse mode decision result, the mode decision for that TU can be made. During the coarse mode decision process, the TU can refer to the reconstructed pixels of neighboring TUs.

[0086] For example, in the process of making a coarse mode decision for a TU, 3 to 8 prediction modes can be selected from 56 candidate prediction modes based on the bit rate loss cost, and the selected prediction modes are determined as the coarse mode prediction results. In the process of making a mode decision for a TU based on the coarse mode decision results, an optimal prediction mode can be selected from the coarse mode decision results through mode decision, and this prediction mode is determined as the mode decision result.

[0087] Step 4: Entropy coding.

[0088] In practical applications, CTU-level entropy coding can be performed. During the entropy coding process for each CTU within a video frame, multiple TUs within that CTU can be entropy coded in parallel based on the mode decision results corresponding to each TU within that CTU, thereby achieving the coding of the CTU.

[0089] To facilitate understanding of the differences between the encoding method in the application embodiments of this application and related technologies, Figure 8 The processing flow of an encoding method in the related art is shown. In the related art, the encoding processing flow can mainly include the following steps: reading pixels, mode decision, and entropy coding. Among them, mode decision and entropy coding are performed at the TU level. That is, mode decision for the next TU can only be performed after the mode decision for the TU has been completed, and entropy coding for the next TU can only be performed after the entropy coding for the TU has been completed.

[0090] Comparison Figure 7 and Figure 8 As can be seen, compared with related technologies, the application embodiment of this application adds a reconstruction estimation processing step before the pattern decision processing, which completes the pixel reconstruction process in advance and independently, thereby generating the reconstructed pixels required for the pattern decision in the next step. This is equivalent to decoupling the dependency between pixel reconstruction and pattern decision from the TU level to the CTU level. In this way, the pattern decision can be performed on the TU in parallel, improving the efficiency of pixel reconstruction and pattern decision, and thus improving the coding efficiency.

[0091] Furthermore, the TU references the reconstructed pixels of neighboring TUs instead of the original pixels during the pattern decision-making process. Compared with the method of making pattern decisions based on the original pixels of neighboring TUs, this improves the accuracy of pattern decisions and thus improves the accuracy of encoding.

[0092] In practical applications, the encoder provided in the application embodiments of this application may include a reconstruction estimation model, which can be used to estimate the reconstruction of each TU.

[0093] In practical applications, see Figure 9In the process of processing a TU based on the prediction reconstruction model, the input of the prediction reconstruction model may include: the original pixels of the TU, the size of the TU and multiple candidate prediction modes. The output of the prediction reconstruction model may include the reconstructed pixels of the TU. The reconstructed pixels output by the prediction reconstruction model can be used for mode decision of the neighboring TUs of the TU.

[0094] For example, see Figure 9 The reconstruction model can output the reconstruction pixels required by TU-3 in the mode decision process by reconstructing TU-0, TU-1 and TU-2.

[0095] In practical applications, see Figure 10 The process of reconstructing a TU using a reconstruction model can mainly include the following steps: Step 1: Pattern prediction.

[0096] In practical applications, the prediction mode corresponding to the TU can be determined from multiple candidate prediction modes based on the size of the TU.

[0097] Step 2: Pixel prediction.

[0098] In practical applications, the pixels in the TU can be predicted based on the prediction mode corresponding to the TU, thus obtaining the predicted pixels.

[0099] Step 3: Calculate residual pixels.

[0100] In practical applications, the difference between the predicted pixel of the TU and the corresponding original pixel can be calculated to obtain the residual pixel.

[0101] Step 4: Transformation, quantization, dequantization, and inverse transform.

[0102] In practical applications, the residual pixels in the TU can be transformed, quantized, dequantized, and inverse transformed to obtain the processed residual pixels.

[0103] Step 5: Pixel reconstruction.

[0104] In practical applications, the original pixels of the TU can be summed with the processed residual pixels to obtain the reconstructed pixels.

[0105] In practical applications, see Figure 11 The process of predicting the mode of a TU using a reconstruction model can mainly include the following steps: Step 1: Determine if the dimensions of the TU are correct. .

[0106] In practical applications, when the size of the TU is determined to be... In this case, the DC mode can be determined as the prediction mode corresponding to the TU.

[0107] In practical applications, when the size of the TU is determined to be... In cases other than these dimensions, you can proceed to step 2.

[0108] Step 2: Determine whether the TU is a square.

[0109] In practical applications, when the TU is determined to be rectangular, the prediction mode with the smallest bit rate loss cost can be selected from multiple predefined partial angle modes based on the bit rate loss cost, and this prediction mode can be determined as the prediction mode corresponding to the TU.

[0110] In practical applications, when the TU is determined to be square, the prediction mode with the smallest bit rate loss cost can be selected from the DC mode and the multiple partial angle modes based on the bit rate loss cost, and this prediction mode can be determined as the prediction mode corresponding to the TU.

[0111] The various partial angle modes that can be set include: V mode, H mode, and D45 mode.

[0112] In practical applications, the bitrate loss cost can be calculated based on the SATD cost correlation coefficient. Selecting a prediction mode based on the bitrate loss cost can also be understood as mode selection based on SATD.

[0113] In the application embodiments of this application, the prediction reconstruction model in the encoder is based on the size of the TU, and the determination of the prediction mode corresponding to the TU is divided into different cases for adaptive processing, and a limited number of alternatives are set, so as to efficiently determine the optimal prediction mode, improve the performance of pixel prediction, and thus improve the performance of pixel reconstruction and subsequent encoding.

[0114] Based on the embodiments described above, this application also provides an encoding device, see [link to previous document]. Figure 12 The encoding device includes: The first determining unit 1201 is used to perform pixel reconstruction on multiple TUs in a video frame in parallel, and obtain the reconstruction value corresponding to the pixel value of each pixel in each TU. The second determining unit 1202 is used to perform mode decisions on the plurality of TUs in parallel, obtain the mode decision result corresponding to each TU, and encode the TU based on the mode decision result corresponding to each TU; wherein, the first TU refers to the reconstructed value corresponding to the pixel value of the pixel in the second TU during the mode decision process; the first TU represents one of the plurality of TUs, and the second TU represents the TU adjacent to the first TU.

[0115] In one embodiment, when the first determining unit 1201 performs pixel reconstruction on one of the plurality of TUs, it is used to: Based on the size of the TU, the prediction mode corresponding to the TU is determined from multiple set prediction modes; Based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU, the reconstructed value corresponding to the pixel value of each pixel in the TU is determined.

[0116] In one embodiment, the set plurality of prediction modes are determined from a plurality of candidate prediction modes based on a first statistical result; the first statistical result is used to describe the distribution of the selection probability corresponding to each of the plurality of candidate prediction modes; the selection probability is used to describe the probability of deciding to select the corresponding prediction mode in the historical mode decision-making process.

[0117] In one embodiment, the first determining unit 1201 determines the prediction mode corresponding to the TU from a set of prediction modes based on the size of the TU, including: When the size of the TU is characterized as a set first size, the set first prediction mode is determined as the prediction mode corresponding to the TU; and / or, When the size of the TU is represented as a size other than the first size, the prediction mode corresponding to the TU is determined from a plurality of set second prediction modes based on a plurality of first costs corresponding to the TU; each of the plurality of first costs represents the bit rate loss cost of performing pixel prediction on the TU based on one of the plurality of second prediction modes; the prediction mode corresponding to the TU is represented as the second prediction mode corresponding to the minimum first cost among the plurality of first costs.

[0118] In one embodiment, the set first prediction mode is characterized as a DC mode; and / or, When the length and width of the TU are equal, the set plurality of second prediction modes include: DC mode, V mode, H mode, and D45 mode; and / or, When the length and width of the TU are not equal, the set multiple second prediction modes include: V mode, H mode and D45 mode.

[0119] In one embodiment, the first determining unit 1201 determines the reconstructed value corresponding to the pixel value of each pixel in the TU based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU, including: Based on the prediction mode corresponding to the TU, pixel prediction is performed on the pixels in the TU to obtain the predicted value corresponding to the pixel value of each pixel in the TU; wherein, the first TU refers to the pixel value of the pixel in the second TU during the pixel prediction process; The difference between the pixel value and the corresponding predicted value of each pixel in the TU is calculated to obtain the first residual value for each pixel in the TU; The multiple first residual values ​​corresponding to the TU are transformed, quantized, dequantized and inverse transformed to obtain the second residual value corresponding to each pixel in the TU. The predicted value corresponding to the pixel value of each pixel in the TU is summed with the second residual value to obtain the reconstructed value corresponding to the pixel value of each pixel in the TU.

[0120] In one embodiment, the plurality of TUs represents all TUs within the same CTU in the video frame.

[0121] In practical applications, the first determining unit 1201 and the second determining unit 1202 can be implemented by the processor in the encoding device.

[0122] It should be noted that the encoding device provided in the above embodiments is only illustrated by the division of the above program modules during encoding processing. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the encoding device and encoding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0123] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, this application also provides an electronic device, see [link to relevant documentation]. Figure 13 The electronic device includes: Communication interface 1 enables information exchange with other devices; Processor 2 is connected to communication interface 1 to enable information interaction with other devices and, when running a computer program, executes the methods provided by one or more technical solutions in the above embodiments. The computer program is stored in memory 3.

[0124] Specifically, the processor 2 is configured to perform pixel reconstruction on multiple TUs within a video frame in parallel, obtaining the reconstructed value corresponding to the pixel value of each pixel in each TU; and, Mode decisions are performed on the multiple TUs in parallel to obtain the mode decision result for each TU, and the TU is encoded based on the mode decision result for each TU; wherein, the first TU refers to the reconstructed value corresponding to the pixel value of the pixel in the second TU during the mode decision process; the first TU represents one of the multiple TUs, and the second TU represents the TU adjacent to the first TU.

[0125] In one embodiment, when the processor 2 performs pixel reconstruction on one of the plurality of TUs, it is configured to: Based on the size of the TU, the prediction mode corresponding to the TU is determined from multiple set prediction modes; Based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU, the reconstructed value corresponding to the pixel value of each pixel in the TU is determined.

[0126] In one embodiment, the set plurality of prediction modes are determined from a plurality of candidate prediction modes based on a first statistical result; the first statistical result is used to describe the distribution of the selection probability corresponding to each of the plurality of candidate prediction modes; the selection probability is used to describe the probability of deciding to select the corresponding prediction mode in the historical mode decision-making process.

[0127] In one embodiment, the processor 2 determines the prediction mode corresponding to the TU from a set of prediction modes based on the size of the TU, including: When the size of the TU is characterized as a set first size, the set first prediction mode is determined as the prediction mode corresponding to the TU; and / or, When the size of the TU is represented as a size other than the first size, the prediction mode corresponding to the TU is determined from a plurality of set second prediction modes based on a plurality of first costs corresponding to the TU; each of the plurality of first costs represents the bit rate loss cost of performing pixel prediction on the TU based on one of the plurality of second prediction modes; the prediction mode corresponding to the TU is represented as the second prediction mode corresponding to the minimum first cost among the plurality of first costs.

[0128] In one embodiment, the set first prediction mode is characterized as a DC mode; and / or, When the length and width of the TU are equal, the set plurality of second prediction modes include: DC mode, V mode, H mode, and D45 mode; and / or, When the length and width of the TU are not equal, the set multiple second prediction modes include: V mode, H mode and D45 mode.

[0129] In one embodiment, the processor 2 determines the reconstructed value corresponding to the pixel value of each pixel in the TU based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU, including: Based on the prediction mode corresponding to the TU, pixel prediction is performed on the pixels in the TU to obtain the predicted value corresponding to the pixel value of each pixel in the TU; wherein, the first TU refers to the pixel value of the pixel in the second TU during the pixel prediction process; The difference between the pixel value and the corresponding predicted value of each pixel in the TU is calculated to obtain the first residual value for each pixel in the TU; The multiple first residual values ​​corresponding to the TU are transformed, quantized, dequantized and inverse transformed to obtain the second residual value corresponding to each pixel in the TU. The predicted value corresponding to the pixel value of each pixel in the TU is summed with the second residual value to obtain the reconstructed value corresponding to the pixel value of each pixel in the TU.

[0130] In one embodiment, the plurality of TUs represents all TUs within the same CTU in the video frame.

[0131] It should be noted that the specific processing procedure of communication interface 1 can be understood by referring to the above method.

[0132] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 13 The general will label all buses as Bus System 4.

[0133] The memory 3 in this embodiment is used to store various types of data to support operation in the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0134] The methods disclosed in the embodiments of this application can be applied to the processor 2, or implemented by the processor 2. The processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 2 or by instructions in the form of software. The processor 2 mentioned above may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 3. The processor 2 reads the information in the memory 3 and combines its hardware to complete the steps of the aforementioned method.

[0135] In an exemplary embodiment, the electronic device may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0136] It is understood that the memory 3 in the embodiments of this application can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0137] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 3 storing a computer program, which can be executed by the processor 2 of an electronic device to complete the steps of the aforementioned image processing method.

[0138] Computer-readable storage media can be FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0139] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 2 of an electronic device to perform the steps described in the aforementioned image processing method.

[0140] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0141] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, the term "one or more" in this document refers to any combination of at least two of any one or more elements from a set of A, B, and C. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set of A, B, and C. Additionally, the term "one or more" in this document is an exemplary expression and can be replaced with any possible expressions, such as one or more, at least one, or at least one of, etc.

[0142] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0143] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. An encoding method, characterized in that, The method includes: Pixel reconstruction is performed on multiple transform units (TUs) within a video frame in parallel, and the reconstructed value corresponding to the pixel value of each pixel in each TU is obtained respectively; Mode decisions are performed on the multiple TUs in parallel to obtain the mode decision result for each TU, and the TU is encoded based on the mode decision result for each TU; wherein, the first TU refers to the reconstructed value corresponding to the pixel value of the pixel in the second TU during the mode decision process; the first TU represents one of the multiple TUs, and the second TU represents the TU adjacent to the first TU.

2. The method according to claim 1, characterized in that, When reconstructing a pixel in one of the plurality of TUs, the method includes: Based on the size of the TU, the prediction mode corresponding to the TU is determined from multiple set prediction modes; Based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU, the reconstructed value corresponding to the pixel value of each pixel in the TU is determined.

3. The method according to claim 2, characterized in that, The set multiple prediction modes are determined from the multiple candidate prediction modes based on the first statistical result; the first statistical result is used to describe the distribution of the selection probability corresponding to the multiple candidate prediction modes respectively; the selection probability is used to describe the probability of deciding to select the corresponding prediction mode in the historical mode decision-making process.

4. The method according to claim 2 or 3, characterized in that, The step of determining the prediction mode corresponding to the TU from a set of multiple prediction modes based on the TU size includes: When the size of the TU is characterized as a set first size, the set first prediction mode is determined as the prediction mode corresponding to the TU; and / or, When the size of the TU is represented as a size other than the first size, the prediction mode corresponding to the TU is determined from a plurality of set second prediction modes based on a plurality of first costs corresponding to the TU; each of the plurality of first costs represents the bit rate loss cost of performing pixel prediction on the TU based on one of the plurality of second prediction modes; the prediction mode corresponding to the TU is represented as the second prediction mode corresponding to the minimum first cost among the plurality of first costs.

5. The method according to claim 4, characterized in that, The first prediction mode is defined as a DC mode; and / or, When the length and width of the TU are equal, the set plurality of second prediction modes include: DC mode, V mode, H mode, and D45 mode; and / or, When the length and width of the TU are not equal, the set multiple second prediction modes include: V mode, H mode and D45 mode.

6. The method according to claim 2, characterized in that, The step of determining the reconstructed value corresponding to the pixel value of each pixel in the TU based on the prediction mode corresponding to the TU and the pixel value of each pixel in the TU includes: Based on the prediction mode corresponding to the TU, pixel prediction is performed on the pixels in the TU to obtain the predicted value corresponding to the pixel value of each pixel in the TU; wherein, the first TU refers to the pixel value of the pixel in the second TU during the pixel prediction process; The difference between the pixel value and the corresponding predicted value of each pixel in the TU is calculated to obtain the first residual value for each pixel in the TU; The multiple first residual values ​​corresponding to the TU are transformed, quantized, dequantized and inverse transformed to obtain the second residual value corresponding to each pixel in the TU. The predicted value corresponding to the pixel value of each pixel in the TU is summed with the second residual value to obtain the reconstructed value corresponding to the pixel value of each pixel in the TU.

7. The method according to claim 1, characterized in that, The multiple TUs refer to all TUs within the same coding tree unit (CTU) in the video frame.

8. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. The processor is used to run the computer program and perform the steps of the method according to any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images