Continuity cost determination method, decoding method, apparatus and storage medium

Abstract

The embodiments of the present application belong to the technical field of video encoding and decoding. Disclosed are a continuity cost determination method, a decoding method, an apparatus and a storage medium. The method comprises: on the basis of a predicted value of a current block, an assumed residual value and an adjacent sample, determining the continuity cost of the current block. By using the solution provided in the embodiments of the present application, the effect of saving codewords can be achieved while ensuring the decoding accuracy.

Description

Continuity cost determination method, decoding method, device and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular to a continuity cost determination method, decoding method, device, and storage medium. Background Art

[0002] During the video decoding process, the decoder needs to use intra-frame prediction or inter-frame prediction to predict the current block to obtain the predicted block of the current block; on the other hand, the decoder needs to decode the bitstream to obtain the quantization coefficient matrix, and perform inverse quantization and inverse transformation on the quantization coefficient matrix to obtain the residual block, and then add the predicted block and the residual block to obtain the reconstructed block of the current block.

[0003] Since the quantization coefficient may be positive or negative, the sign of the quantization coefficient needs to be encoded into the bitstream during the entropy coding process.

[0004] Summary of the Invention

[0005] The present invention provides a method for determining continuity cost, a decoding method, an apparatus, and a storage medium. The technical solution is as follows:

[0006] In one aspect, an embodiment of the present application provides a method for determining continuity cost, the method comprising:

[0007] A continuity cost of the current block is determined based on a prediction value of the current block, a hypothesized residual value, and neighboring samples.

[0008] On the other hand, an embodiment of the present application provides a decoding method, the method comprising:

[0009] Determine the predicted value of the current block;

[0010] performing an inverse transform on the quantized coefficients according to a hypothesized sign combination of the quantized coefficients to obtain a hypothesized residual value of the current block;

[0011] Determining, based on the predicted value of the current block, the residual value of the hypothesis, and neighboring samples, a continuity cost of the current block under the sign combination of the hypothesis;

[0012] The code stream is decoded according to the sign combination of the hypothesis corresponding to the minimum continuity cost.

[0013] On the other hand, an embodiment of the present application provides a decoding method, the method comprising:

[0014] Determine the predicted value of the current block;

[0015] performing an inverse transform on the quantized coefficient according to a hypothetical transform kernel of the quantized coefficient to obtain a hypothetical residual value of the current block;

[0016] Determining a continuity cost of the current block under the assumed transformation kernel based on the predicted value of the current block, the assumed residual value, and neighboring samples;

[0017] A transform kernel is determined from the assumed transform kernels according to the continuity cost, and a code stream is decoded based on the determined transform kernel.

[0018] On the other hand, an embodiment of the present application provides a decoder, comprising:

[0019] The decoding unit is configured to determine a continuity cost of the current block based on a predicted value of the current block, an assumed residual value, and neighboring samples.

[0020] On the other hand, an embodiment of the present application provides a decoder, comprising:

[0021] A decoding unit for determining a prediction value of a current block;

[0022] The decoding unit is configured to perform an inverse transform on the quantization coefficients according to a hypothesized sign combination of the quantization coefficients to obtain a hypothesized residual value of the current block;

[0023] The decoding unit is configured to determine a continuity cost of the current block under the hypothesized sign combination based on the predicted value of the current block, the hypothesized residual value, and neighboring samples;

[0024] The decoding unit is configured to decode the code stream according to the hypothetical sign combination corresponding to the minimum continuity cost.

[0025] On the other hand, an embodiment of the present application provides a decoder, comprising:

[0026] A decoding unit for determining a prediction value of a current block;

[0027] The decoding unit is configured to perform an inverse transform process on the quantized coefficient according to a hypothetical transform kernel of the quantized coefficient to obtain a hypothetical residual value of the current block;

[0028] The decoding unit is configured to determine a continuity cost of the current block under the assumed transformation kernel based on the predicted value of the current block, the assumed residual value, and neighboring samples;

[0029] The decoding unit is configured to determine a transform kernel from the assumed transform kernels according to the continuity cost, and decode the bitstream based on the determined transform kernel.

[0030] On the other hand, an embodiment of the present application provides a decoder, which includes a memory and a processor, wherein the memory is used to store a computer program running on the processor; and the processor is used to execute the continuity cost determination method or the decoding method as described in the above aspects when running the computer program.

[0031] On the other hand, an embodiment of the present application provides a non-volatile computer-readable storage medium storing a code stream, wherein the code stream is decoded by using a decoding method of a decoder, wherein the decoding method is the decoding method described in the above aspect.

[0032] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, the computer instructions are stored in a computer-readable storage medium, the processor obtains the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the continuity cost determination method or decoding method as described in the above aspects.

[0033] In an embodiment of the present application, the continuity cost of the current block is determined based on the predicted value of the current block, the assumed residual value, and the neighboring samples. Since the continuity cost can characterize the continuity of pixels between the current block and the neighboring blocks under the assumed residual value, and the continuity is related to the accuracy of the assumed residual value, the sign of the quantization coefficient can be predicted based on the continuity cost corresponding to each of the different assumed residual values. Using the solution provided in the embodiment of the present application, the encoder can encode the binary symbol that characterizes the accuracy of the prediction of the quantization coefficient sign into the bitstream. Since the accuracy of the symbol prediction based on the continuity cost is higher than the error rate, the effect of saving codewords can be achieved; accordingly, the decoder can decode the sign of the quantization coefficient based on the continuity cost and the binary symbol that characterizes the accuracy of the prediction of the quantization coefficient sign in the bitstream, thereby ensuring decoding accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a schematic diagram of an encoding process at an encoding end according to an exemplary embodiment of the present application;

[0035] FIG2 is a schematic diagram of a decoding process at a decoding end according to an exemplary embodiment of the present application;

[0036] FIG3 is a flow chart of a method for determining continuity cost according to an exemplary embodiment of the present application;

[0037] FIG4 is a schematic diagram illustrating an implementation of a continuity cost determination process according to an exemplary embodiment of the present application;

[0038] FIG5 is a flowchart of a decoding method according to an exemplary embodiment of the present application;

[0039] FIG6 is a flowchart of a decoding method according to another exemplary embodiment of the present application;

[0040] FIG7 is a schematic diagram illustrating an implementation of an estimated value determination process according to an exemplary embodiment of the present application;

[0041] FIG8 is a schematic diagram of a sample in a current block and feature-adjacent samples according to an exemplary embodiment of the present application;

[0042] FIG9 is a flow chart of an estimated value determination process shown in an exemplary embodiment of the present application;

[0043] FIG10 is a schematic diagram of an intra-frame prediction mode;

[0044] FIG11 is a schematic diagram illustrating an implementation of an estimated value determination process according to an exemplary embodiment of the present application;

[0045] FIG12 is a flowchart of a process for determining an estimated value according to another exemplary embodiment of the present application;

[0046] FIG13 is a schematic diagram of a texture feature classification method according to an exemplary embodiment of the present application;

[0047] FIG14 is a schematic diagram of reference neighboring samples according to an exemplary embodiment of the present application;

[0048] FIG15 is a schematic diagram of sample weight combinations under different types shown in an exemplary embodiment of the present application;

[0049] FIG16 is a schematic diagram showing a group division method according to an exemplary embodiment of the present application;

[0050] FIG17 is a schematic diagram showing a group division method according to another exemplary embodiment of the present application;

[0051] FIG18 shows a structural block diagram of an encoder provided by an exemplary embodiment of the present application;

[0052] FIG19 shows a structural block diagram of an encoder provided by another exemplary embodiment of the present application;

[0053] FIG20 shows a structural block diagram of an encoder provided by another exemplary embodiment of the present application;

[0054] FIG21 shows a structural block diagram of a decoder provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0056] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0057] Currently, common video codec standards all use a block-based hybrid coding framework. A video frame is divided into square Largest Coding Units (LCUs) or Coding Tree Units (CTUs) of the same size (e.g., 256×256, 128×128, 64×64, etc.). LCUs or CTUs can be divided into rectangular Coding Units (CUs) based on rules. Coding Units may also be divided into Prediction Units (PUs) and Transform Units (TUs).

[0058] The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filter. The prediction module includes intra prediction and inter prediction. Inter prediction includes motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels in the same frame image, the intra prediction method can be used in video coding and decoding technology to eliminate spatial redundancy between adjacent pixels. Since there is a strong similarity between adjacent frame images in a video, the inter prediction method can be used in video coding and decoding technology to eliminate temporal redundancy between adjacent frame images, thereby improving coding efficiency.

[0059] At the encoding end, as shown in Figure 1, the encoder first divides the image into multiple coding blocks, and then uses the intra-frame or inter-frame prediction (including motion estimation and motion compensation) algorithm for the current block to generate a prediction block of the current block, and uses the original block of the current block to subtract the prediction block to obtain a residual block, and then transforms and quantizes the residual block to obtain quantization coefficients, and finally encodes the quantization coefficients into the bitstream through entropy coding.

[0060] At the decoding end, as shown in Figure 2, the decoder uses an intra-frame or inter-frame prediction algorithm to predict the current block. It also performs entropy decoding on the bitstream to obtain quantization coefficients. These coefficients are then dequantized and inversely transformed to obtain a residual block. The predicted and residual blocks are then added together to form a reconstructed block. The decoder then combines the reconstructed blocks to produce a reconstructed image, which it then performs loop filtering on to produce the decoded image.

[0061] Since the sign of the quantization coefficient may be positive or negative, and different signs of the coefficients will result in different residual blocks, and thus different reconstructed blocks, when encoding the quantization coefficient, if the quantization coefficient is a non-zero coefficient, the sign of the quantization coefficient needs to be encoded.

[0062] In related art, the sign of the quantized coefficient is encoded using a bypass mode. However, since the bypass mode assumes that the sign of the quantized coefficient is equally likely to be positive or negative, the encoded sign of the quantized coefficient requires a large number of code words.

[0063] Taking into account the certain correlation between adjacent pixels, that is, the pixels between the current block and its neighboring blocks are usually continuous, the sign of the quantization parameter can be predicted based on the continuity between the pixels. Accordingly, the encoder and decoder use the same method to determine the continuity cost. The decoder can select the symbol prediction result with the smallest continuity cost based on the continuity costs corresponding to different symbol prediction results, and encode the difference between the true sign of the quantization coefficient and the symbol prediction result corresponding to the minimum continuity cost into the bitstream in the form of a binary symbol. Since the prediction accuracy of the symbol prediction result corresponding to the minimum continuity cost is significantly greater than the prediction error rate, encoding the difference between the true symbol and the symbol prediction result can achieve the effect of saving codewords.

[0064] The decoder similarly selects the symbol prediction with the lowest continuity cost based on the continuity costs associated with the different symbol predictions. Based on the difference between the decoded true sign of the quantized coefficient and the selected symbol prediction, the decoder corrects the selected symbol prediction to obtain the true sign of the quantized coefficient. Because the encoder and decoder determine the continuity cost in the same way, and the bitstream contains the difference between the true symbol and the symbol prediction corresponding to the lowest continuity cost, the decoder can ultimately accurately determine the sign of the quantized coefficient, thereby ensuring decoding accuracy.

[0065] The continuity cost determination method will be described in detail below using an exemplary embodiment. It should be noted that the continuity cost determination method can be used in an encoder or a decoder. For ease of description, the following embodiments are described using the decoder as an example.

[0066] Please refer to Figure 3, which shows a flow chart of a method for determining continuity cost provided by an exemplary embodiment of the present application. The method may include the following steps:

[0067] Step 301 : Determine the continuity cost of the current block based on the predicted value of the current block, the assumed residual value, and neighboring samples.

[0068] In some embodiments, the decoder and the encoder determine a current block based on consistent block partitioning information, wherein the current block may be a current coding unit, a current prediction unit, or a current transform unit.

[0069] In some embodiments, the decoder performs intra-frame or inter-frame prediction on the current block based on the prediction information contained in the bitstream and the decoded blocks to obtain a prediction value of the current block, wherein the prediction value is a pixel value of a pixel in the current block.

[0070] After the decoder performs entropy decoding and inverse quantization on the bitstream to obtain quantized coefficients, it needs to further perform an inverse transform on these coefficients to obtain the residual value of the current block. This residual value represents the difference between the predicted value and the original value of the current block. Since the sign of non-zero quantized coefficients can be positive or negative, the sign of the non-zero quantized coefficients must be determined before performing the inverse transform.

[0071] In this embodiment, since the sign of the quantization coefficient is not directly encoded into the bitstream, the decoder needs to make a residual value assumption based on the quantization coefficient to obtain an assumed residual value. Optionally, the assumed residual value can be determined in any of the following ways:

[0072] 1. The assumed residual value is determined based on the assumed sign combination of the quantization coefficients.

[0073] In a possible implementation, for a non-zero quantized coefficient, the decoder assumes a sign of the non-zero quantized coefficient, and performs an inverse transform process on the quantized coefficient after the sign assumption to obtain an assumed residual value.

[0074] When there are N non-zero quantization coefficients, since the signs of the non-zero quantization coefficients include positive and negative possibilities, we can get 2 N For each hypothetical sign combination, the decoder can obtain the corresponding hypothetical residual value, and a total of 2 N The residual value of the hypothesis.

[0075] For example, when there are two non-zero quantization coefficients, four hypothetical sign combinations (++, +-, -+, and --) and four hypothetical residual values ​​can be obtained.

[0076] Furthermore, when the assumed residual value is determined based on the assumed sign combinations of the quantized coefficients, the decoder determines the continuity costs corresponding to the various assumed sign combinations.

[0077] 2. The assumed residual value is determined based on the assumed transformation kernel of the quantized coefficient.

[0078] Different from the first method, after predicting the signs of different non-zero quantized coefficients, the quantized coefficients after sign prediction are inversely transformed. In the second method, the decoder does not predict the signs of the non-zero quantized coefficients one by one, but uses different transform kernels to inversely transform the quantized coefficients.

[0079] In one possible implementation, the encoder and decoder are configured with several consistent hypothetical transform kernels (or candidate transform kernels). After obtaining the quantized coefficients, the decoder uses different hypothetical transform kernels to perform inverse transform processing on the quantized coefficients to obtain hypothetical residual values ​​corresponding to the different hypothetical transform kernels.

[0080] When M hypothetical transform kernels are set, M hypothetical residual values ​​can be obtained by performing inverse transform processing on the quantized coefficients using different hypothetical transform kernels.

[0081] Furthermore, when the assumed residual values ​​are determined based on the assumed transform kernels of the quantized coefficients, the decoder determines the continuity costs corresponding to the various assumed transform kernels.

[0082] It should be noted that in the above embodiment, only two methods of determining residual values, namely, sign prediction and transform kernel selection, are used as examples for explanation. The decoder can also adopt other methods to make residual value assumptions based on unsigned quantization coefficients. The embodiments of the present application do not constitute a limitation on this.

[0083] In some embodiments, the neighboring samples used to determine the continuity cost are selected from neighboring blocks of the current block. The neighboring blocks may be reconstructed blocks adjacent to the current block in a specified direction, and the neighboring samples may be samples in the neighboring blocks that are adjacent to samples in the current block in the specified direction. For example, the neighboring blocks may be reconstructed blocks located above and to the left of the current block, and the neighboring samples may be samples in the above reconstructed block that are adjacent to the upper edge samples of the current block, and samples in the left reconstructed block that are adjacent to the left edge samples of the current block.

[0084] In some embodiments, the prediction value used to determine the continuity cost may be the prediction value of all samples (ie, pixels) in the current block, or the prediction value of some samples in the current block.

[0085] Correspondingly, the residual value used to determine the hypothesis of continuity cost may be the residual value corresponding to all samples in the current block, or the residual value corresponding to all samples in the current block.

[0086] Optionally, the part of samples may be samples located at a specified edge in the current block.

[0087] In one possible design, the portion of samples may be the first n rows of samples at the upper edge of the current block, and the first m rows of samples at the left edge, where n and m are both positive integers. Accordingly, the parameter values ​​used to determine the hypothesis of the continuity cost include the residual values ​​corresponding to the first n rows of samples at the upper edge of the current block, and the residual values ​​corresponding to the first m rows of samples at the left edge.

[0088] Schematically, as shown in Figure 4, the decoder determines the continuity cost of the current block 42 based on the predicted values ​​of the first row sample 411 of the upper edge and the first column sample 412 of the left edge in the current block 41, the residual values ​​corresponding to the first row sample 411 of the upper edge and the first column sample 412 of the left edge, and the neighboring samples 42 located above and to the left of the current block 41.

[0089] Regarding the calculation method of the continuity cost, in one possible implementation, the decoder determines the hypothesized reconstruction value of the current block based on the predicted value of the current block and the hypothesized residual value, and then determines the continuity cost based on the correlation between the hypothesized reconstruction value and the neighboring samples.

[0090] In some embodiments, the smaller the continuity cost, the better the continuity between the current block and the neighboring samples (the higher the correlation) under the assumed residual value. Conversely, the larger the continuity cost, the worse the continuity between the current block and the neighboring samples (the lower the correlation) under the assumed residual value.

[0091] In summary, in the embodiment of the present application, the continuity cost of the current block is determined based on the predicted value of the current block, the assumed residual value and the adjacent samples. Since the continuity cost can characterize the continuity of the pixels between the current block and the adjacent blocks under the assumed residual value, and the continuity is related to the accuracy of the assumed residual value, the sign of the quantization coefficient can be predicted based on the continuity cost corresponding to each of the different assumed residual values. Using the solution provided in the embodiment of the present application, the encoder can encode the binary symbol that characterizes the accuracy of the prediction of the quantization coefficient sign into the bitstream. Since the accuracy of the symbol prediction based on the continuity cost is higher than the error rate, the effect of saving codewords can be achieved; accordingly, the decoder can decode the sign of the quantization coefficient based on the continuity cost and the binary symbol that characterizes the accuracy of the prediction of the quantization coefficient sign in the bitstream, thereby ensuring decoding accuracy.

[0092] The above continuity cost determination method can be applied to a decoding process based on sign prediction or a decoding process based on transform kernel selection, and will be described below using exemplary embodiments.

[0093] Please refer to Figure 5, which shows a flowchart of a decoding method provided by an exemplary embodiment of the present application. The method may include the following steps:

[0094] Step 501: Determine the prediction value of the current block.

[0095] During the decoding process, the decoder performs intra-frame or inter-frame prediction on the current block based on the prediction information contained in the bitstream and the decoded blocks to obtain the predicted value of the current block.

[0096] Step 502: Perform inverse transformation on the quantized coefficients according to the assumed sign combination of the quantized coefficients to obtain the assumed residual value of the current block.

[0097] In one possible implementation, the decoder determines non-zero quantized coefficients in the quantized coefficients (obtained through entropy decoding and inverse quantization), and assumes the signs of the non-zero quantized coefficients through permutations and combinations, thereby obtaining several assumed sign combinations.

[0098] The decoder processes the non-zero quantized coefficients based on the assumed positive and negative sign combination (adds signs to the non-zero quantized coefficients), and performs inverse transformation on the processed quantized coefficients to obtain the assumed residual value of the current block.

[0099] For example, when three of the four quantization coefficients corresponding to the current block are non-zero, the eight hypothetical sign combinations are +++, ++-, +-+, +--, -++, -+-, --+, and ---. When the four quantization coefficients are 1, 0, 5, and 3, respectively, the quantization coefficients are processed based on the hypothetical sign combination "+++" to obtain the quantization coefficients of +1, 0, +5, and +3, respectively. Inverse transform processing is performed on the quantization coefficients of +1, 0, +5, and +3 to obtain the hypothetical residual values ​​corresponding to the hypothetical sign combination "+++." The quantization coefficients are processed based on the hypothetical sign combination "++-" to obtain the hypothetical residual values ​​corresponding to the hypothetical sign combination "++-." Similarly, eight hypothetical residual values ​​corresponding to eight hypothetical sign combinations are obtained.

[0100] Step 503 : Determine the continuity cost of the current block under the assumed sign combination based on the predicted value of the current block, the assumed residual value, and the neighboring samples.

[0101] Similar to step 301 , the decoder determines the continuity cost of the current block under different hypothesized sign combinations based on the prediction value of the current block, the hypothesized residual value, and neighboring samples.

[0102] Combining the examples in the above steps, the decoder determines the continuity costs of the current block under 8 combinations of positive and negative signs.

[0103] Step 504: Decode the code stream according to the hypothetical sign combination corresponding to the minimum continuity cost.

[0104] In one possible implementation, during the encoding process, the encoder determines the continuity costs corresponding to different hypothetical sign combinations in the same manner as the decoder. The encoder determines the hypothetical sign combination corresponding to the minimum continuity cost, determines the difference between the hypothetical sign combination corresponding to the minimum continuity cost and the actual sign combination of the quantized coefficients, and encodes a binary symbol representing the difference into the bitstream.

[0105] Optionally, the encoder compares the symbol at the same position in the assumed positive and negative sign combination corresponding to the minimum continuity cost and the real positive and negative sign combination. If the symbols are consistent (that is, the symbol prediction is correct), the binary symbol corresponding to the position is 1; if the symbols are inconsistent (that is, the symbol prediction is wrong), the binary symbol corresponding to the position is 0.

[0106] Indicatively, when the assumed sign combination corresponding to the minimum continuity cost is “+++” and the actual sign combination is “+++”, the comparison result can be expressed as 111; when the assumed sign combination corresponding to the minimum continuity cost is “+++” and the actual sign combination is “++-”, the comparison result can be expressed as 110.

[0107] Since the prediction accuracy of the positive and negative signs in the hypothetical positive and negative sign combinations corresponding to the minimum continuity cost is significantly higher than the prediction error rate, code words can be saved when encoding the comparison results using the context prediction model.

[0108] Because the decoder and encoder use the same continuity cost determination method, the hypothetical sign combinations corresponding to the minimum continuity costs determined by the decoder and encoder are consistent. Furthermore, the decoder can determine the actual sign combination corresponding to the quantization coefficients based on the comparison results in the bitstream and the hypothetical sign combination corresponding to the minimum continuity cost. After performing an inverse transform on the quantization coefficients under the actual sign combination to obtain the residual value, the decoder can then derive the reconstructed block based on the predicted value of the current block and the residual value, and then obtain the reconstructed image based on the reconstructed block.

[0109] In illustration, when the assumed sign combination corresponding to the minimum continuity cost is "+++" and the comparison result contained in the bitstream is 111, the decoder determines that the true sign combination of the quantization coefficient is "+++"; when the assumed sign combination corresponding to the minimum continuity cost is "+++" and the comparison result contained in the bitstream is 110, the decoder determines that the true sign combination of the quantization coefficient is "++-".

[0110] As can be seen, when applying the above continuity cost determination method to the decoding process based on sign prediction, encoding the comparison result between the sign combination corresponding to the minimum continuity cost and the true sign combination can save codewords compared to directly encoding the signs of the quantized coefficients. Furthermore, the encoder and decoder use a consistent continuity cost determination scheme, ensuring that the decoder restores the true sign combination and guarantees decoding accuracy.

[0111] Please refer to Figure 6, which shows a flowchart of a decoding method provided by another exemplary embodiment of the present application. The method may include the following steps:

[0112] Step 601: Determine the prediction value of the current block.

[0113] During the decoding process, the decoder performs intra-frame or inter-frame prediction on the current block based on the prediction information contained in the bitstream and the decoded blocks to obtain the predicted value of the current block.

[0114] Step 602: Perform inverse transformation on the quantized coefficients according to the assumed transformation kernel of the quantized coefficients to obtain the assumed residual value of the current block.

[0115] In a possible implementation, after entropy decoding and inverse quantization are performed to obtain quantization coefficients, the decoder uses different assumed transform kernels to perform inverse transform processing on the quantization coefficients to obtain assumed residual values ​​of the current block under the different assumed transform kernels.

[0116] Illustratively, when the decoder is provided with four candidate transform kernels, four hypothetical residual values ​​are obtained after inverse transform processing is performed on the quantized coefficients using different candidate transform kernels.

[0117] Step 603 : Determine the continuity cost of the current block under the assumed transformation kernel based on the prediction value of the current block, the assumed residual value, and the neighboring samples.

[0118] Similar to step 301 , the decoder determines the continuity cost of the current block under different assumed transform kernels based on the prediction value of the current block, the assumed residual value, and the neighboring samples.

[0119] In combination with the examples in the above steps, the decoder determines the continuity costs of the current block under four assumed transformation kernels.

[0120] Step 604 : Determine a transform kernel from the assumed transform kernels according to the continuity cost, and decode the bitstream based on the determined transform kernel.

[0121] In one possible implementation, the decoder determines the hypothetical transform kernel corresponding to the minimum continuity cost as the transform kernel for the current block. The decoder then uses the determined transform kernel to perform an inverse transform on the quantized coefficients to obtain a residual value. The decoder then reconstructs the block based on the predicted value of the current block and the residual value, and then reconstructs the image based on the reconstructed block. It should be noted that in this approach, the encoder does not need to encode information related to the transform kernel selection in the bitstream.

[0122] In another possible implementation, during the encoding process, the encoder determines the continuity costs corresponding to different hypothesized transform kernels in the same manner as the decoder, and sorts the hypothesized transform kernels (in descending or ascending order) based on the continuity costs. Furthermore, the encoder determines the position of the actual transform kernel in the sequence and encodes the position information into the bitstream.

[0123] Because lower continuity costs increase the probability that the corresponding hypothetical transform kernel is the actual transform kernel, and higher continuity costs decrease the probability that the corresponding hypothetical transform kernel is the actual transform kernel, when encoding the position of the actual transform kernel in the sequence, the probability varies at different positions. Accordingly, using a context prediction model to encode position information into the bitstream can save codewords.

[0124] Since the decoder and encoder use the same continuity cost determination method and the configured hypothetical transform kernels are consistent, the decoder can sort the hypothetical transform kernels based on the continuity cost and determine the actual transform kernel based on the position information of the transform kernel actually used in the sequence in the bitstream.

[0125] Illustratively, the decoder determines the order of the assumed transform cores as transform core 3, transform core 1, transform core 2, and transform core 4 based on the ascending order of continuity cost. When the position information of the transform core actually used in the bitstream in the sequence is 1, the decoder determines that the transform core actually used is transform core 3. If the order of the assumed transform cores is transform core 1, transform core 3, transform core 4, and transform core 2, and the position information of the transform core actually used in the bitstream in the sequence is 1, the decoder determines that the transform core actually used is transform core 1.

[0126] It can be seen that when the above-mentioned continuity cost determination method is applied to the decoding process based on transform kernel selection, compared with directly encoding the sign of the quantization coefficient, encoding the position information of the actually used transform kernel in the transform kernel sequence can save codewords.

[0127] Regarding the specific method for determining the continuity cost, because neighboring samples are correlated with samples in the current block, they can be used to estimate the samples in the current block. The higher the correlation, the more accurate the estimation result. Based on this characteristic, the decoder can determine the continuity cost of the current block based on the estimation result of the neighboring samples.

[0128] In one possible implementation, the decoder determines an estimated value of the current block based on neighboring samples, and determines a continuity cost of the current block based on a difference between the estimated value and a sum of a predicted value and a hypothesized residual value.

[0129] The sum of the predicted value and the assumed residual value can be understood as the assumed reconstructed value of the current block. The smaller the difference between the assumed reconstructed value and the estimated value, the higher the correlation between the current block and the neighboring samples under the assumed residual value, and accordingly, the more accurate the assumed residual value. Conversely, the larger the difference between the assumed reconstructed value and the estimated value, the lower the correlation between the current block and the neighboring samples under the assumed residual value, and accordingly, the less accurate the assumed residual value.

[0130] Schematically, for the sample (x, y) in the current block, the continuity cost of the sample can be expressed as: cost x,y =|(P x,y +r x,y )-P′ x,y |

[0131] Among them, P′ x,y Represents the estimated value of the sample, P x,y Represents the predicted value of the sample, r x,y Represents the residual value of the hypothesis for this sample.

[0132] Furthermore, the sum of the continuity costs of the samples in the current block is the continuity cost of the current block.

[0133] Optionally, the decoder determines a hypothetical residual value corresponding to the minimum difference value, and reconstructs the current block based on the hypothetical residual value to obtain a reconstructed block.

[0134] In some embodiments, determining the estimated value of the current block based on neighboring samples may adopt either of the following two methods.

[0135] 1. Determine the estimated value of the current block based on the neighboring samples of the current block in a preset direction.

[0136] In a possible implementation, the decoder may estimate samples of the current block using neighboring samples of the current block in a preset direction.

[0137] In one possible design, the decoder may use neighboring samples above the current block to determine an estimated value of an upper edge sample in the current block; and may use neighboring samples to the left of the current block to determine an estimated value of a left edge sample in the current block.

[0138] Optionally, for the sample in the current block, the decoder determines at least two neighboring samples of the sample in a preset direction, and determines the sample change trend based on the at least two neighboring samples, thereby estimating the sample based on the at least two neighboring samples and the sample change trend to obtain an estimated value of the sample.

[0139] The sample change trend can be represented by the sample difference between adjacent samples.

[0140] Schematically, as shown in FIG7 , for an upper edge sample 711 in a current block 71, the decoder can determine an estimated value of the upper edge sample 711 based on two adjacent samples 721 above the upper edge sample 711; for a left edge sample 712 in the current block 71, the decoder can determine an estimated value of the left edge sample 712 based on two adjacent samples 722 to the left of the upper edge sample 712. Furthermore, the continuity cost of the current block 71 can be expressed as:

[0141] Where w is the width of the current block, h is the height of the current block, and the position of the upper left corner sample of the current block is represented as (0, 0). R represents the neighboring sample, P represents the predicted value of the sample in the current block, and r represents the residual value of the hypothesis of the sample.

[0142] It should be noted that, in this embodiment, only the preset directions are upward and left, and the adjacent samples are two adjacent samples above the current block and two adjacent samples on the left are used as an example for explanation. The preset direction can also be set to other directions as needed, and the adjacent samples can be selected as needed. The embodiment of the present application does not limit this.

[0143] 2. Determine the estimated value of the current block based on the texture features of the neighboring samples of the current block.

[0144] Different from the first method, which only considers continuity in a preset direction (such as continuity in the vertical or horizontal direction), in order to further improve the accuracy of estimating the current block based on neighboring samples, and thus improve the accuracy of calculating the continuity cost, the decoder can determine the estimation method of the current block based on the texture features of the neighboring samples, and then obtain the estimated value of the current block.

[0145] When the image content changes along a certain direction, there is good continuity between the pixels in that direction. Accordingly, the accuracy of sample estimation for the current block based on neighboring samples in that direction is high. Therefore, in some embodiments, the texture feature may include a texture direction feature.

[0146] Regarding the representation of texture features, in one possible implementation, the encoder may determine the horizontal gradient of adjacent samples in the horizontal direction and the vertical gradient in the vertical direction as the texture features of the adjacent samples, that is, using gradients to characterize texture features.

[0147] The horizontal gradient is used to characterize the change direction and change intensity of the sample in the horizontal direction, and the vertical gradient is used to characterize the change direction and change intensity of the sample in the vertical direction.

[0148] Regarding determining the location of texture features, in one possible implementation, for the top edge sample in the current block, the decoder locates feature-adjacent samples in the vertical direction of the top edge sample and determines the texture features of the feature-adjacent samples. For the left edge sample in the current block, the decoder locates feature-adjacent samples in the horizontal direction of the left edge sample and determines the texture features of the feature-adjacent samples. The interval between the feature-adjacent samples and the top edge sample or the left edge sample is m, where m is a natural number.

[0149] 8 , for the upper edge sample 811 at (x, 0) in the current block 81, the decoder determines the neighboring sample at (x, -2) (with an interval of 1 from the upper edge sample) as a feature neighboring sample 821, and determines the estimated value of the upper edge sample 811 based on the texture features of the feature neighboring sample 821. For the left edge sample 812 at (0, y) in the current block 81, the decoder determines the neighboring sample at (-2, y) (with an interval of 1 from the left edge sample) as a feature neighboring sample 822, and determines the estimated value of the left edge sample 812 based on the texture features of the feature neighboring sample 822.

[0150] Optionally, the decoder may calculate the gradient of a position corresponding to the feature neighboring sample based on other samples surrounding the feature neighboring sample and a gradient operator.

[0151] The gradient operator may be a Sobel operator, a Prewitt operator, a Roberts operator, a Kirsch operator, etc. The embodiment of the present application does not limit the specific gradient operator used.

[0152] Taking the Sobel operator as an example, for the feature neighboring sample P located at (x, y) x,y , its corresponding horizontal gradient grad x =P x+1,y-1+2*P x+1,y +P x+1,y+1 -P x-1,y-1 -2*P x-1,y -P x-1,y+1 , vertical gradient grad y =P x-1,y+1 +2*P x,y+1 +P x+1,y+1 -P x-1,y-1 -2*P x,y-1 -P x+1,y-1 .

[0153] Of course, in addition to using the gradient operator to calculate the gradient, the decoder can also directly determine the difference between adjacent samples in the horizontal direction as the horizontal gradient, and determine the difference between adjacent samples in the vertical direction as the vertical gradient. The embodiments of the present application do not limit the specific gradient calculation method.

[0154] After determining the texture features of the neighboring samples, the estimated value of the current block may be determined in any of the following ways.

[0155] The first approach is to determine the intra-frame prediction mode based on texture features, and use neighboring samples to perform intra-frame prediction to determine the estimated value of the current block.

[0156] Since texture features can reflect the change of texture with direction, the intra-frame prediction mode corresponding to the samples in the current block is determined according to the texture features, and the intra-frame prediction logic is reused to determine the estimated value of the current block based on neighboring samples.

[0157] The second way is to determine the texture feature type to which the texture feature corresponding to the current block belongs, and determine the estimated value of the current block based on the neighboring samples and the texture feature type corresponding to the current block.

[0158] In a possible implementation, texture features may be divided into at least two texture feature types, and corresponding estimation algorithms may be set for different texture feature types, so as to determine an estimated value of the current block using the estimation algorithm and neighboring samples.

[0159] The above two methods of determining the estimated value of the current block are described in detail below using illustrative embodiments.

[0160] Please refer to Figure 9, which shows a flow chart of a method for determining an estimated value of a current block provided by an exemplary embodiment of the present application. The method may include the following steps:

[0161] Step 901: Determine the intra prediction mode of the current block based on texture features of neighboring samples of the current block.

[0162] In one possible implementation, the decoder is configured with a correspondence between texture features and intra prediction modes. When determining an estimated value of a sample in a current block, the decoder determines the intra prediction mode of the sample based on the texture features of the sample corresponding to neighboring samples.

[0163] It should be noted that this intra-frame prediction mode is different from the intra-frame prediction mode used in the code stream to determine the prediction value of the current block. In this embodiment, the intra-frame prediction mode determined based on the texture feature type is used to estimate the state (i.e., pixel value) that the sample should present, and the estimated state can be understood as the sum of the sample's prediction value and the residual value.

[0164] In some embodiments, when texture features include horizontal and vertical gradients, since the horizontal gradient can reflect the direction and intensity of the texture in the horizontal direction, and the vertical gradient can reflect the direction and intensity of the texture in the vertical direction, the overall texture direction and overall texture intensity of the neighboring samples can be determined based on the horizontal and vertical gradients. Furthermore, since the neighboring samples are relatively close to the current block, after mapping the overall texture direction and overall texture intensity to an intra-frame prediction mode, the samples in the current block can be estimated based on the neighboring samples adjacent to and to the left of the current block by reusing the intra-frame prediction logic.

[0165] In one possible implementation, the decoder determines the intra prediction mode of the current block based on the horizontal gradient and the vertical gradient of neighboring samples.

[0166] In some embodiments, when determining the intra-frame prediction mode based on the current block, for the samples in the current block, the decoder determines the candidate intra-frame prediction mode corresponding to each sample based on the horizontal gradient and vertical gradient of the adjacent samples corresponding to the sample, thereby determining the intra-frame prediction mode of the current block according to the candidate intra-frame prediction mode of the sample.

[0167] Optionally, this step includes the following sub-steps:

[0168] Step 901A: Determine a candidate intra prediction mode based on the numerical relationship between the horizontal gradient and the vertical gradient.

[0169] The numerical relationship between the horizontal gradient and the vertical gradient may include a gradient ratio and a sign relationship.

[0170] Optionally, the decoder is configured with a correspondence between a gradient ratio interval, a gradient sign and an intra-frame prediction mode, where the gradient ratio interval is the interval to which the ratio of the horizontal gradient to the vertical gradient belongs, and the gradient sign includes the gradient signs (positive, negative or 0) corresponding to the horizontal gradient and the vertical gradient.

[0171] For a sample in the current block, the decoder determines the gradient ratio based on the horizontal and vertical gradients of neighboring samples, and then determines the candidate intra prediction mode corresponding to the sample from the above correspondence according to the gradient ratio and the gradient sign.

[0172] Optionally, the corresponding relationship may be stored in a configuration table or other forms.

[0173] Optionally, the above correspondence may include all intra-frame prediction modes or some intra-frame prediction modes. For example, intra-frame prediction modes may be selected at intervals, and the above correspondence may be configured for the selected intra-frame prediction modes. Compared to configuring a correspondence for all intra-frame prediction modes, configuring a correspondence for some intra-frame prediction modes results in a larger gradient ratio range for the intra-frame prediction modes.

[0174] Schematically, as shown in Figure 10, in VVC, when the gradient amplitude of the horizontal gradient (i.e., the absolute value of the gradient) of a sample in the current block is 0 and the gradient amplitude of the vertical gradient is not 0, the decoder determines that the candidate intra-frame prediction mode of the sample is mode 18 based on the corresponding relationship.

[0175] When the vertical gradient magnitude of a sample in the current block is 0 and the horizontal gradient magnitude is not 0, the decoder determines that the candidate intra prediction mode of the sample is mode 50 based on the corresponding relationship.

[0176] When the gradient magnitudes of the horizontal gradient and the vertical gradient of a sample in the current block are both non-zero, and the gradient magnitudes of the horizontal gradient and the vertical gradient are the same, and the signs of the horizontal gradient and the vertical gradient are the same, the decoder determines that the candidate intra-frame prediction mode of the sample is mode 34 based on the corresponding relationship.

[0177] When the horizontal gradient magnitude of a sample in the current block is twice the vertical gradient magnitude, and the horizontal gradient and the vertical gradient have the same sign, the decoder determines that the candidate intra prediction mode for the sample is mode 40 based on the corresponding relationship.

[0178] Step 901B: Determine the cumulative magnitude corresponding to the candidate intra prediction mode based on the gradient magnitudes of the horizontal gradient and the vertical gradient.

[0179] Since the candidate intra-frame prediction modes corresponding to different samples in the current block may be different, when determining the intra-frame prediction mode of the current block, the importance of various candidate intra-frame prediction modes can be determined (which can reflect the significance of the texture features corresponding to the candidate intra-frame prediction modes), so that the candidate intra-frame prediction mode with the highest significance is determined as the intra-frame prediction mode of the current block.

[0180] Regarding the representation of the importance of the candidate intra prediction mode, in a possible implementation, the gradient magnitudes of the horizontal gradient and the vertical gradient may be used to represent the importance of the candidate intra prediction mode.

[0181] In one possible implementation, after determining a candidate intra prediction mode for a sample, the cumulative amplitude corresponding to the candidate intra prediction mode can be updated based on the gradient amplitudes of the horizontal gradient and the vertical gradient. After determining a candidate intra prediction mode for a sample in the current block, the cumulative amplitudes corresponding to various candidate intra prediction modes can be determined.

[0182] The larger the cumulative amplitude corresponding to the candidate intra prediction mode is, the more important the candidate intra prediction mode is. Accordingly, the more significant the texture feature corresponding to the candidate intra prediction mode in the current block is.

[0183] Among them, for the horizontal gradient, the corresponding gradient amplitude can be expressed as abs(grad x ), for the vertical gradient, the corresponding gradient amplitude can be expressed as abs(grad y ). Accordingly, the decoder is based on the sum of the horizontal gradient and the vertical gradient amplitude amp=abs(grad x )+abs(grad y ), update the cumulative amplitude corresponding to the candidate intra prediction mode.

[0184] Illustratively, the sum of the gradient amplitudes of the first sample corresponding to the neighboring samples in the current block is amp1, and the candidate intra-frame prediction mode corresponding to the first sample is intra-frame prediction mode 34. The sum of the gradient amplitudes of the second sample corresponding to the neighboring samples is amp2, and the candidate intra-frame prediction mode corresponding to the second sample is intra-frame prediction mode 34. The sum of the gradient amplitudes of the third sample corresponding to the neighboring samples is amp3, and the candidate intra-frame prediction mode corresponding to the third sample is intra-frame prediction mode 18. Ultimately, the cumulative amplitude corresponding to intra-frame prediction mode 18 is determined to be amp3, and the cumulative amplitude corresponding to intra-frame prediction mode 34 is amp1+amp2.

[0185] Step 901C: determine the candidate intra prediction mode corresponding to the maximum cumulative amplitude as the intra prediction mode of the current block.

[0186] After determining the cumulative amplitudes of various candidate intra-frame prediction modes, the encoder determines the candidate intra-frame prediction mode corresponding to the maximum cumulative amplitude as the intra-frame prediction mode of the current block, that is, the intra-frame prediction mode corresponding to the most significant texture feature of the neighboring samples is determined as the intra-frame prediction mode of the current block.

[0187] In combination with the example in the above steps, if amp1+amp2>amp3, the encoder determines the intra prediction mode 34 as the intra prediction mode of the current block.

[0188] In other possible implementations, in addition to determining the intra-frame prediction mode of the current block based on the cumulative amplitude of the candidate intra-frame prediction modes, the number of occurrences of the candidate intra-frame prediction modes can also be determined, so that the candidate intra-frame prediction mode corresponding to the highest number of occurrences is determined as the intra-frame prediction mode of the current block. The embodiments of the present application are not limited to this.

[0189] Step 902: Determine an estimated value of the current block based on the intra prediction mode of the current block and neighboring samples.

[0190] In one possible implementation, the decoder multiplexes intra prediction logic to determine estimated values ​​of samples in the current block based on neighboring samples above and to the left of the current block and an interpolation filter corresponding to the intra prediction mode of the current block.

[0191] Schematically, as shown in FIG11 , when the decoder determines the estimated values ​​of the top edge sample 1111 and the left edge sample 1112 in the current block 1110, it first determines a candidate intra-frame prediction mode for the top edge sample 1111 based on the horizontal and vertical gradients of the neighboring sample 1121 above the top edge sample 1111, and determines a candidate intra-frame prediction mode for the left edge sample 1112 based on the horizontal and vertical gradients of the neighboring sample 1122 to the left of the left edge sample 1112. The decoder then determines the intra-frame prediction mode corresponding to the current block 1110 based on the accumulated magnitudes of the candidate intra-frame prediction modes. Finally, the decoder determines the estimated values ​​of the top edge sample 1111 and the left edge sample 1112 through intra-frame prediction based on the upper and left adjacent neighboring samples 1130 of the current block 1110 and the intra-frame prediction mode of the current block.

[0192] In this embodiment, since the process of determining the estimated value refers to the texture of the adjacent samples (horizontal and vertical gradient representation), the estimated value determined by this scheme is more accurate; and the decoder reuses the intra-frame prediction logic when estimating the current block, without the need for additional estimated value calculation logic, thereby simplifying the implementation complexity of the estimation process.

[0193] Please refer to Figure 12, which shows a flow chart of a method for determining an estimated value of a current block provided by another exemplary embodiment of the present application. The method may include the following steps:

[0194] Step 1201: Determine the texture feature type corresponding to the current block based on the texture features of neighboring samples.

[0195] In one possible implementation, texture features are classified into several types, and different estimation calculation methods are configured for different texture feature types. When determining the estimated value of a sample in the current block, the decoder determines the texture feature type of the sample's corresponding neighboring samples and then uses the estimation calculation method corresponding to the texture feature type to determine the estimated value of the sample.

[0196] Regarding the classification of texture feature types, one possible design involves categorizing the texture feature types based on the texture direction angle represented by the texture feature, with different texture feature types corresponding to different texture direction angle ranges. When determining the texture feature type of a neighboring sample, the texture direction angle range to which the texture direction angle represented by the texture feature of the neighboring sample belongs is determined, and the texture feature type corresponding to the texture direction angle range is then determined as the texture feature type of the neighboring sample.

[0197] In some embodiments, when the texture feature includes a horizontal gradient and a vertical gradient, the decoder may determine the texture feature type according to the horizontal gradient and the vertical gradient.

[0198] In some embodiments, when determining the texture feature type based on the current block, for the samples in the current block, the decoder determines the candidate texture feature type corresponding to each sample based on the horizontal gradient and vertical gradient of the adjacent samples corresponding to the sample, thereby determining the texture feature type corresponding to the current block based on the candidate texture feature type corresponding to the sample.

[0199] Optionally, this step may include the following sub-steps:

[0200] Step 1201A: Determine a candidate texture feature type based on the numerical relationship between the horizontal gradient and the vertical gradient.

[0201] The numerical relationship between the horizontal gradient and the vertical gradient may include a gradient ratio and a sign relationship.

[0202] Optionally, the decoder is configured with a correspondence between a gradient ratio interval, a gradient sign and a type, where the gradient ratio interval is the interval to which the ratio of the horizontal gradient to the vertical gradient belongs, and the gradient sign includes the gradient signs (positive, negative or 0) corresponding to the horizontal gradient and the vertical gradient respectively.

[0203] For a sample in the current block, the decoder determines the gradient ratio based on the horizontal and vertical gradients of neighboring samples, and then determines the candidate texture feature type corresponding to the sample from the above correspondence according to the gradient ratio and gradient sign.

[0204] Optionally, the corresponding relationship may be stored in a configuration table or other forms.

[0205] In one possible division method, as shown in FIG13 , texture features can be divided into four types. Among them, the texture direction corresponding to the first texture feature type is a quasi-vertical direction (the texture direction angle range is an angle range with a first angle to the vertical direction), the texture direction corresponding to the second texture feature type is a quasi-horizontal direction (the texture direction angle range is an angle range with a second angle to the horizontal direction), the texture direction corresponding to the third texture feature type is a quasi-right oblique direction (the texture direction angle range is an angle range with a third angle to the right oblique 45° direction), and the texture direction corresponding to the fourth texture feature type is a quasi-left oblique direction (the texture direction angle range is an angle range with a fourth angle to the left oblique 45° direction). Accordingly, the correspondence between the gradient ratio interval, gradient sign, and type is shown in Table 1.

[0206] Table 1

[0207] In conjunction with the corresponding relationship shown in Table 1, in an illustrative example, when the horizontal gradient amplitude is greater than twice the vertical gradient amplitude, the decoder determines that the texture feature of the sample belongs to texture feature type 1;

[0208] In the case where the vertical gradient magnitude is greater than twice the horizontal gradient magnitude, the decoder determines that the texture feature of the sample belongs to texture feature type 2;

[0209] When the horizontal gradient amplitude is less than twice the vertical gradient amplitude, and the vertical gradient amplitude is less than twice the horizontal gradient amplitude, and the horizontal gradient and the vertical gradient have the same gradient sign, the decoder determines that the texture feature of the sample belongs to texture feature type 3;

[0210] When the horizontal gradient amplitude is less than twice the vertical gradient amplitude, and the vertical gradient amplitude is less than twice the horizontal gradient amplitude, and the horizontal gradient and the vertical gradient have different signs, the decoder determines that the texture feature of the sample belongs to texture feature type 4.

[0211] It should be noted that the above-mentioned classification method of texture features is for illustrative purposes only. In other possible implementations, texture features can be classified at a coarser granularity, such as into 2 or 3 types, or at a finer granularity, such as into 6, 8, or 12 types. The classification method and number of types can be dynamically set based on the computational load and accuracy requirements, and the embodiments of this application do not limit the specific classification method and number.

[0212] Step 1201B: Determine the cumulative amplitude corresponding to the candidate texture feature type based on the gradient amplitudes of the horizontal gradient and the vertical gradient.

[0213] Since the candidate texture feature types corresponding to different samples in the current block may be different, when determining the texture feature type corresponding to the current block, the importance of various candidate texture feature types can be determined (which can reflect the significance of the candidate texture feature types), so that the candidate texture feature type with the highest significance is determined as the texture feature type corresponding to the current block.

[0214] Regarding the representation of the importance of the candidate texture feature type, in a possible implementation, the gradient amplitudes of the horizontal gradient and the vertical gradient may be used to represent the importance of the candidate texture feature type.

[0215] Among them, for the horizontal gradient, the corresponding gradient amplitude can be expressed as abs(grad x ), for the vertical gradient, the corresponding gradient amplitude can be expressed as abs(grad y ). Accordingly, the decoder is based on the sum of the horizontal gradient and the vertical gradient amplitude amp=abs(grad x )+abs(grad y ), update the cumulative amplitude corresponding to the candidate texture feature type.

[0216] Illustratively, the sum of the gradient amplitudes of the first sample corresponding to the neighboring samples in the current block is amp1, and the first sample corresponds to texture feature type 1; the sum of the gradient amplitudes of the second sample corresponding to the neighboring samples is amp2, and the second sample corresponds to texture feature type 2; the sum of the gradient amplitudes of the third sample corresponding to the neighboring samples is amp3, and the third sample corresponds to texture feature type 1; the sum of the gradient amplitudes of the fourth sample corresponding to the neighboring samples is amp4, and the fourth sample corresponds to texture feature type 3. Ultimately, the cumulative amplitude corresponding to texture feature type 1 is determined to be amp1+amp3, the cumulative amplitude corresponding to texture feature type 2 is amp2, and the cumulative amplitude corresponding to texture feature type 3 is amp4.

[0217] Step 1201C: determine the candidate texture feature type corresponding to the maximum cumulative amplitude as the texture feature type corresponding to the current block.

[0218] After determining the cumulative magnitudes of various candidate types, the encoder determines the candidate type corresponding to the maximum cumulative magnitude as the texture feature type corresponding to the current block.

[0219] In combination with the example in the above steps, if amp1+amp3>amp2, and amp1+amp3>amp4, the encoder determines texture feature type 1 as the texture feature type corresponding to the current block.

[0220] In other possible implementations, in addition to determining the texture feature type of the current block based on the cumulative amplitude of the candidate texture feature types, the number of occurrences of the candidate texture feature types can also be determined, so that the candidate texture feature type corresponding to the highest number of occurrences is determined as the texture feature type corresponding to the current block. The embodiments of the present application are not limited to this.

[0221] Step 1202: Determine an estimated value of the current block based on the texture feature type corresponding to the current block and neighboring samples.

[0222] In a possible implementation, the decoder determines the estimated value of the sample in the current block based on the estimated value calculation method corresponding to the texture feature type corresponding to the current block and in combination with neighboring samples.

[0223] In one possible design, for different types of texture features, different neighboring samples are referenced when determining the estimated value of the sample in the current block.

[0224] In one possible implementation, the decoder determines reference neighboring samples based on the texture feature type, and determines an estimated value of the current block based on the reference neighboring samples, where different texture feature types correspond to reference neighboring samples in different orientations.

[0225] Optionally, for samples at different positions in the current block, different reference neighboring samples may be configured for the samples at different positions. For example, different reference neighboring samples may be configured for upper edge samples and lower edge samples.

[0226] Optionally, for a specific texture feature type, reference neighboring samples may be configured only for samples at a specific position in the current block, and the samples at the specific position may be estimated; for samples outside the specific position, reference neighboring samples may not be configured, and the samples outside the specific position may not be estimated.

[0227] Combined with the types described in Table 1, the correspondence between texture feature types and reference neighboring samples is shown in Table 2.

[0228] Table 2

[0229] Combined with Table 2, if the current block corresponds to texture feature type 1, for the upper edge sample (x, 0), the estimated value determined by the decoder based on the reference adjacent samples can be expressed as 2R x,-1 -R x,-2, the left edge sample (0, y) is not estimated. As shown in Figure 14, for the upper edge sample at (0, 0), the estimated value of the upper edge sample can be determined based on the neighboring samples at (0, -1) and (0, -2); for the upper edge sample at (1, 0), the estimated value of the upper edge sample can be determined based on the neighboring samples at (1, -1) and (1, -2).

[0230] If the current block corresponds to texture feature type 2, for the left edge sample (0, y), the estimated value determined by the decoder based on the reference adjacent samples can be expressed as 2R -1,y -R -2,y , the upper edge sample (x, 0) is not estimated. As shown in Figure 14, for the left edge sample at (0, 0), the estimated value of the left edge sample can be determined based on the neighboring samples at (-2, 0) and (-1, 0); for the left edge sample at (0, 1), the estimated value of the left edge sample can be determined based on the neighboring samples at (-2, 1) and (-1, 1).

[0231] If the current block corresponds to texture feature type 3, for the upper edge sample (x, 0), the estimated value determined by the decoder based on the reference adjacent samples can be expressed as 2R x-1,-1 -R x-2,-2 , for the left edge sample (0, y), the estimated value determined by the decoder based on the reference adjacent samples can be expressed as 2R -1,y-1 -R -2,y-2 As shown in FIG14 , for the upper edge sample at (1, 0), the estimated value of the upper edge sample can be determined based on the neighboring samples at (-1, -2) and (0, -1); for the left edge sample at (0, 1), the estimated value of the left edge sample can be determined based on the neighboring samples at (-2, -1) and (-1, 0).

[0232] If the current block corresponds to texture feature type 4, for the upper edge sample (x, 0), the estimated value determined by the decoder based on the reference adjacent samples can be expressed as 2R x+1,-1 -R x+2,-2 , for the left edge sample (0, y), the estimated value determined by the decoder based on the reference adjacent samples can be expressed as 2R -1,y+1 -R -2,y+2 As shown in FIG14 , for the upper edge sample at (1, 0), the estimated value of the upper edge sample can be determined based on the neighboring samples at (2, -1) and (3, -2); for the left edge sample at (0, 1), the estimated value of the left edge sample can be determined based on the neighboring samples at (-1, 2) and (-2, 3).

[0233] It should be noted that the above embodiments are only for illustrative purposes. When other types of division methods are adopted, the reference neighboring samples corresponding to the samples in the current block may be different from the above embodiments, and the method of determining the estimated value based on the reference neighboring samples may also be different from the above embodiments.

[0234] After determining the estimated value, the decoder performs continuity cost calculation based on the estimated value, the predicted value, and the assumed residual value.

[0235] Combined with the example in the above steps, for texture feature type 1, the continuity cost of the upper edge sample (x, 0) is x,0 =|(-R x,-2 +2R x,-1 -P x,0 )-r x,0 |, the continuity cost is not calculated for left edge samples.

[0236] For texture feature type 2, the continuity cost of the left edge sample (0, y) is 0,y =|(-R -2,y +2R -1,y -P 0,y )-r 0,y |, the upper edge samples do not calculate the continuity cost.

[0237] For texture feature type 3, the continuity cost of the upper edge sample (x, 0) is x,0 =|(-R x-2,-2 +2R x-1,-1 -P x,0 )-r x,0 |, the continuity cost of the left edge sample (0, y) 0,y =|(-R -2,y-2 +2R -1,y-1 -P 0,y )-r 0,y |.

[0238] For texture feature type 4, the continuity cost of the upper edge sample (x, 0) is x,0 =|(-R x+2,-2 +2R x+1,-1 -P x,0 )-r x,0 |, the continuity cost of the left edge sample (0, y) 0,y =|(-R -2,y+2 +2R -1,y+1 -P 0,y )-r 0,y |.

[0239] For samples in the current block for which continuity costs need to be calculated, after determining the continuity costs of the samples, the decoder determines the sum of the continuity costs of the samples as the continuity cost of the current block.

[0240] In another possible design, for different texture feature types, the neighboring samples referenced when determining the estimated value of the sample in the current block are the same, but the sample weights used when calculating the estimated value are different.

[0241] In one possible implementation, the decoder determines a sample weight combination based on the texture feature type, and determines an estimated value of the current block based on the sample weight combination and reference neighboring samples. The sample weight combination includes sample weights corresponding to different neighboring samples, and different texture feature types correspond to different sample weight combinations, and different texture feature types correspond to reference neighboring samples in the same orientation.

[0242] Optionally, the decoder performs weighted calculation on the reference neighboring samples based on the sample weights corresponding to the reference neighboring samples in the sample weight combination to obtain an estimated value of the sample in the current block.

[0243] In an illustrative example, as shown in FIG15 , when determining the estimated value of the upper edge sample 1511 in the current block 1510, reference is made to the neighboring samples 1520. If the current block 1510 corresponds to texture type 1, the sample weight combination corresponding to the neighboring samples 1520 is (0, 0, -1, 0, 0, 0, 0, 2, 0, 0); if the current block 1510 corresponds to texture type 3, the sample weight combination corresponding to the neighboring samples 1520 is (-1, 0, 0, 0, 0, 2, 0, 0, 0); if the current block 1510 corresponds to texture type 4, the sample weight combination corresponding to the neighboring samples 1520 is (0, 0, 0, 0, -1, 0, 0, 0, 2, 0).

[0244] In this embodiment, since the process of determining the estimated value refers to the texture of the adjacent samples (horizontal and vertical gradient representation), the accuracy of the estimated value determined by using this scheme is relatively high; and by configuring different sample estimation methods for different types of texture features, the specificity of sample estimation under different texture types is improved, which helps to further improve the accuracy of the determined estimated value.

[0245] In the above embodiment, before determining the continuity cost of the current block, the current block may be divided into several groups, and then the continuity cost corresponding to each group is determined. Finally, the continuity cost of the current block is determined based on the continuity cost of at least one group.

[0246] In one possible implementation, for a group in the current block, the decoder determines a continuity cost of the group based on the predicted value of the group, the assumed residual value, and neighboring samples, and determines a continuity cost of the current block based on the continuity cost of at least one group.

[0247] Optionally, the continuity cost of the current block is the sum of the continuity costs of at least one group.

[0248] In some embodiments, the group used to determine the continuity cost of the current block may be all the groups in the current block. Accordingly, the decoder determines the continuity cost of the current block based on the sum of the continuity costs of the various groups.

[0249] In other embodiments, the group used to determine the continuity cost of the current block may be one or at least two groups specified in the current block. Accordingly, the decoder determines the continuity cost of the current block based on the sum of the continuity costs of the specified groups (the continuity costs of groups other than the specified groups do not need to be calculated).

[0250] Optionally, the designated group may be a group selected according to a preset standard. For example, a group selected at intervals from all groups (e.g., groups with odd or even sequence numbers after sorting), or a group located at a specified position in the current block (e.g., a group located at the lower left edge, upper left edge, and upper right edge). The embodiments of the present application do not limit the method for selecting the designated group or the number of selected groups.

[0251] Regarding the group division method, any of the following methods can be used.

[0252] Division method 1: The upper edge samples and lower edge samples of the current block belong to the same group.

[0253] In this partitioning method, all samples in the current block used to calculate the continuity cost belong to the same group. Accordingly, the estimated values ​​of all samples in the current block are calculated in the same way.

[0254] In one possible implementation, the decoder determines the intra prediction mode of the current block based on candidate intra prediction modes corresponding to all samples in the current block, thereby determining estimated values ​​of samples in the current block according to the intra prediction mode of the current block.

[0255] In another possible implementation, the decoder determines the texture feature type corresponding to the current block based on candidate types of texture features corresponding to all samples in the current block, thereby determining an estimated value of the samples in the current block according to the texture feature type of the current block.

[0256] Division method 2: the upper edge samples of the current block belong to the first group, and the left edge samples of the current block belong to the second group.

[0257] To improve the accuracy of the estimated value calculation, the samples in the current block may be divided into at least two groups, and the estimated values ​​of the samples in each group may be calculated using corresponding methods for each group. The estimated values ​​of samples in the same group are calculated using the same method, while the methods used to calculate the estimated values ​​of different groups may be different.

[0258] Since samples at the same location in the current block have a high correlation, the samples can be grouped according to their location in the current block. For example, the top edge samples can be grouped into one group, and the left edge samples can be grouped into another group.

[0259] In one possible implementation, the decoder determines a first intra-frame prediction mode corresponding to the first group based on the candidate intra-frame prediction modes corresponding to the upper edge samples in the first group, thereby determining an estimated value of the upper edge samples in the first group based on the first intra-frame prediction mode. Based on the candidate intra-frame prediction modes corresponding to the left edge samples in the second group, the decoder determines a second intra-frame prediction mode corresponding to the second group, thereby determining an estimated value of the left edge samples in the second group based on the second intra-frame prediction mode.

[0260] In another possible implementation, the decoder determines that the first group corresponds to a first texture feature type based on the candidate types of texture features corresponding to the upper edge samples in the first group, and thereby determines estimated values ​​of the upper edge samples in the first group based on the first texture feature type. The decoder determines that the second group corresponds to a second texture feature type based on the texture feature types corresponding to the left edge samples in the second group, and thereby determines estimated values ​​of the left edge samples in the second group based on the second texture feature type.

[0261] Division method 3: n consecutive edge samples in the current block belong to the same group, n is a positive integer, and the edge samples belong to the upper edge samples and the lower edge samples of the current block.

[0262] Since consecutive samples usually have high similarity, the n consecutive edge samples in the current block can be divided into the same group. In addition, the more edge samples there are in the same group, the greater the computational complexity of calculating the estimated value, but the higher the accuracy of the estimated value. Conversely, the fewer edge samples there are in the same group, the smaller the computational complexity of calculating the estimated value, but the lower the accuracy of the estimated value.

[0263] Schematically, as shown in FIG16 , every two consecutive edge samples in the current block 1610 are divided into the same group 1611 .

[0264] In one possible implementation, the decoder determines the intra prediction mode of the group based on candidate intra prediction modes corresponding to edge samples in the same group, thereby determining the estimated value of the upper edge samples in the group according to the intra prediction mode of the group.

[0265] In another possible implementation, the decoder determines the texture feature type corresponding to the group based on candidate types of texture features corresponding to edge samples in the same group, and thereby determines the estimated value of the edge samples in the group according to the texture feature type corresponding to the group.

[0266] Division method 4: p consecutive upper edge samples in the current block belong to the same group, and q consecutive left edge samples in the current block belong to the same processing unit, where p and q are both positive integers.

[0267] Since the degree of texture variation in different directions may be different, for example, in some images the degree of texture variation in the horizontal direction is significantly greater than that in the vertical direction, in some embodiments, different grouping methods may be used for upper edge samples and lower edge samples.

[0268] Among the upper edge samples, p consecutive samples are divided into the same group, and among the left edge samples, q consecutive samples are divided into the same group. Here, p and q can be the same or different.

[0269] Optionally, when the texture variation degree in the horizontal direction is greater than that in the vertical direction, p<q may be set; and when the texture variation degree in the horizontal direction is less than that in the vertical direction, p>q may be set.

[0270] Schematically, as shown in FIG17 , in the current block 1710 , every two consecutive edge samples in the left edge samples are divided into the same group, and every four consecutive edge samples in the upper edge samples are divided into the same group.

[0271] In the above group division method, the process of determining the continuity cost of the group can refer to the continuity cost determination process based on the current block, which will not be described in detail in this embodiment.

[0272] Please refer to Figure 18, which shows a block diagram of an encoder provided by an exemplary embodiment of the present application. The encoder includes:

[0273] The decoding unit 1810 is configured to determine a continuity cost of the current block based on a prediction value of the current block, an assumed residual value, and neighboring samples.

[0274] Optionally, the decoding unit 1810 is configured to:

[0275] Determining an estimated value of the current block based on the neighboring samples;

[0276] The continuity cost of the current block is determined based on a difference between a sum of a residual value between the predicted value and the hypothesis and the estimated value.

[0277] Optionally, the decoding unit 1810 is configured to determine the estimated value of the current block based on the neighboring samples of the current block in a preset direction.

[0278] Optionally, the decoding unit 1810 is configured to determine the estimated value of the current block based on texture features of the neighboring samples of the current block.

[0279] Optionally, the decoding unit 1810 is configured to: determine an intra prediction mode of the current block based on the texture features of the neighboring samples of the current block;

[0280] The estimated value of the current block is determined based on the intra prediction mode of the current block and the neighboring samples.

[0281] Optionally, the texture feature includes a horizontal gradient and a vertical gradient;

[0282] The decoding unit 1810 is configured to determine an intra-frame prediction mode of the current block based on the horizontal gradient and the vertical gradient of the neighboring samples.

[0283] Optionally, the decoding unit 1810 is configured to: determine a candidate intra prediction mode based on a numerical relationship between the horizontal gradient and the vertical gradient;

[0284] Determining a cumulative amplitude corresponding to the candidate intra prediction mode based on the gradient amplitudes of the horizontal gradient and the vertical gradient;

[0285] The candidate intra prediction mode corresponding to the maximum cumulative amplitude is determined as the intra prediction mode of the current block.

[0286] Optionally, the decoding unit 1810 is configured to: determine a texture feature type corresponding to the current block based on the texture features of the neighboring samples;

[0287] The estimated value of the current block is determined based on the texture feature type corresponding to the current block and the neighboring samples.

[0288] Optionally, the texture feature includes a horizontal gradient and a vertical gradient;

[0289] The decoding unit 1810 is configured to: determine a candidate texture feature type based on a numerical relationship between the horizontal gradient and the vertical gradient;

[0290] Determining a cumulative amplitude corresponding to the candidate texture feature type based on the gradient amplitudes of the horizontal gradient and the vertical gradient;

[0291] The candidate texture feature type corresponding to the maximum cumulative amplitude is determined as the texture feature type corresponding to the current block.

[0292] Optionally, the decoding unit 1810 is configured to: determine reference neighboring samples based on a texture feature type corresponding to the current block, wherein different texture feature types correspond to reference neighboring samples in different orientations;

[0293] The estimated value of the current block is determined based on the reference neighboring samples.

[0294] Optionally, the decoding unit 1810 is configured to: determine a sample weight combination based on a texture feature type corresponding to the current block, wherein the sample weight combination includes sample weights corresponding to different neighboring samples, and different texture feature types correspond to different sample weight combinations;

[0295] The estimated value of the current block is determined based on the sample weight combination and reference neighboring samples, wherein different texture feature types correspond to reference neighboring samples of the same orientation.

[0296] Optionally, the decoding unit 1810 is configured to: determine, for a group in the current block, a continuity cost of the group based on the predicted value of the group, the assumed residual value, and the neighboring samples;

[0297] A continuity cost of the current block is determined based on the continuity cost of at least one group.

[0298] Optionally, the upper edge samples and the lower edge samples of the current block belong to the same group; or,

[0299] The upper edge sample of the current block belongs to the first group, and the left edge sample of the current block belongs to the second group; or,

[0300] n consecutive edge samples in the current block belong to the same group, n is a positive integer, and the edge samples belong to the upper edge samples and the lower edge samples of the current block; or,

[0301] P consecutive upper edge samples in the current block belong to the same group, and q consecutive left edge samples in the current block belong to the same processing unit, where p and q are both positive integers.

[0302] Optionally, the assumed residual value is determined based on a combination of assumed positive and negative signs of the quantization coefficients, or,

[0303] The assumed residual value is determined based on the assumed transformation kernel of the quantized coefficients.

[0304] Please refer to Figure 19, which shows a block diagram of the structure of an encoder provided by another exemplary embodiment of the present application. The encoder includes:

[0305] A decoding unit 1910, configured to determine a prediction value for a current block;

[0306] The decoding unit 1910 is configured to perform an inverse transform on the quantized coefficients according to the assumed sign combination of the quantized coefficients to obtain the assumed residual value of the current block;

[0307] The decoding unit 1910 is configured to determine a continuity cost of the current block under the hypothesized sign combination based on the predicted value of the current block, the hypothesized residual value, and neighboring samples;

[0308] The decoding unit 1910 is configured to decode the bitstream according to the hypothetical sign combination corresponding to the minimum continuity cost.

[0309] Please refer to Figure 20, which shows a block diagram of the structure of an encoder provided by another exemplary embodiment of the present application. The encoder includes:

[0310] A decoding unit 2010, configured to determine a prediction value of a current block;

[0311] The decoding unit 2010 is configured to perform an inverse transform process on the quantized coefficients according to an assumed transform kernel of the quantized coefficients to obtain an assumed residual value of the current block;

[0312] The decoding unit 2010 is configured to determine a continuity cost of the current block under the assumed transformation kernel based on the predicted value of the current block, the assumed residual value, and neighboring samples;

[0313] The decoding unit 2010 is configured to determine a transform kernel from the assumed transform kernels according to the continuity cost, and decode the bitstream based on the determined transform kernel.

[0314] It should be noted that the process of determining the continuity cost by the above decoding unit can refer to the above continuity cost determination method embodiment, and the process of implementing decoding by the above decoding unit can refer to the above decoding method embodiment, which will not be described in detail in this embodiment.

[0315] Please refer to Figure 21, which shows a block diagram of a decoder provided by an exemplary embodiment of the present application. The encoder may include one or more of the following components: a processor 2101 and a memory 2102. These components are coupled together via a bus system. It will be appreciated that the bus system is used to enable connectivity and communication between these components. In addition to a data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0316] The memory 2102 is configured to store a computer program that can be run on the processor 2101. The processor 2101 is configured to, when running the computer program, execute:

[0317] Determining a continuity cost of the current block based on a predicted value of the current block, a hypothesized residual value, and neighboring samples;

[0318] Alternatively, execute:

[0319] Determine the predicted value of the current block;

[0320] performing an inverse transform on the quantized coefficients according to a hypothesized sign combination of the quantized coefficients to obtain a hypothesized residual value of the current block;

[0321] Determining, based on the predicted value of the current block, the residual value of the hypothesis, and neighboring samples, a continuity cost of the current block under the sign combination of the hypothesis;

[0322] Decoding the bitstream according to the sign combination of the hypothesis corresponding to the minimum continuity cost;

[0323] Alternatively, execute:

[0324] Determine the predicted value of the current block;

[0325] performing an inverse transform on the quantized coefficient according to a hypothetical transform kernel of the quantized coefficient to obtain a hypothetical residual value of the current block;

[0326] Determining a continuity cost of the current block under the assumed transformation kernel based on the predicted value of the current block, the assumed residual value, and neighboring samples;

[0327] A transform kernel is determined from the assumed transform kernels according to the continuity cost, and a code stream is decoded based on the determined transform kernel.

[0328] It is understood that the memory 2102 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The description of memory 2102 herein is intended to include, but not be limited to, these and any other suitable types of memory.

[0329] Processor 2101 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-mentioned point cloud decoding method can be completed by hardware integrated logic circuits or software instructions in processor 2101. The above-mentioned processor 2101 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 2102, and the processor 2101 reads the information in the memory 2102 and completes the steps of the above method in combination with its hardware. It is understandable that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more ASICs, DSPs, digital signal processing devices (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), FPGAs, general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or a combination thereof. For software implementation, the technology described in this application can be implemented by modules (such as processes, functions, etc.) that perform the functions described in this application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor. Optionally, as another embodiment, the processor 2101 is also configured to execute the continuity cost determination method or the decoding method described in any of the above embodiments when running the computer program.

[0330] An embodiment of the present application further provides a non-volatile computer-readable storage medium for storing a code stream, wherein the code stream is decoded by a decoding method of a decoder, wherein the decoding method is the decoding method described in the above embodiment.

[0331] An embodiment of the present application also provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor obtains the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the continuity cost determination method or decoding method as described in the above embodiment.

[0332] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0333] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for determining continuity cost, characterized in that: The method comprises: A continuity cost of the current block is determined based on a prediction value of the current block, a hypothesized residual value, and neighboring samples.

2. The method according to claim 1, characterized in that The determining the continuity cost of the current block based on the predicted value of the current block, the assumed residual value, and the neighboring samples includes: Determining an estimated value of the current block based on the neighboring samples; The continuity cost of the current block is determined based on a difference between a sum of a residual value between the predicted value and the hypothesis and the estimated value.

3. The method according to claim 2, characterized in that The determining, based on the neighboring samples, an estimated value of the current block, includes: The estimated value of the current block is determined based on the neighboring samples of the current block in a preset direction.

4. The method according to claim 2, characterized in that The determining, based on the neighboring samples, an estimated value of the current block, includes: The estimated value of the current block is determined based on texture features of the neighboring samples of the current block.

5. The method according to claim 4, characterized in that The determining the estimated value of the current block based on the texture features of the neighboring samples of the current block includes: Determining an intra prediction mode of the current block based on the texture features of the neighboring samples of the current block; The estimated value of the current block is determined based on the intra prediction mode of the current block and the neighboring samples.

6. The method according to claim 5, characterized in that The texture features include horizontal gradient and vertical gradient; The determining, based on the texture features of the neighboring samples of the current block, the intra prediction mode of the current block comprises: An intra prediction mode of the current block is determined based on the horizontal gradient and the vertical gradient of the neighboring samples.

7. The method according to claim 6, characterized in that The determining, based on the horizontal gradient and the vertical gradient of the neighboring samples, the intra prediction mode of the current block includes: determining a candidate intra prediction mode based on a numerical relationship between the horizontal gradient and the vertical gradient; Determining a cumulative amplitude corresponding to the candidate intra prediction mode based on the gradient amplitudes of the horizontal gradient and the vertical gradient; The candidate intra prediction mode corresponding to the maximum cumulative amplitude is determined as the intra prediction mode of the current block.

8. The method according to claim 4, characterized in that The determining the estimated value of the current block based on the texture features of the neighboring samples of the current block includes: Determining a texture feature type corresponding to the current block based on the texture features of the neighboring samples; The estimated value of the current block is determined based on the texture feature type corresponding to the current block and the neighboring samples.

9. The method according to claim 8, characterized in that The texture features include horizontal gradient and vertical gradient; The determining, based on the texture features of the neighboring samples of the current block, a texture feature type corresponding to the current block includes: Determining a candidate texture feature type based on a numerical relationship between the horizontal gradient and the vertical gradient; Determining a cumulative amplitude corresponding to the candidate texture feature type based on the gradient amplitudes of the horizontal gradient and the vertical gradient; The candidate texture feature type corresponding to the maximum cumulative amplitude is determined as the texture feature type corresponding to the current block.

10. The method according to claim 8, characterized in that The determining the estimated value of the current block based on the texture feature type corresponding to the current block and the neighboring samples includes: Determining reference neighboring samples based on the texture feature type corresponding to the current block, wherein different texture feature types correspond to reference neighboring samples in different orientations; The estimated value of the current block is determined based on the reference neighboring samples.

11. The method according to claim 8, characterized in that The determining the estimated value of the current block based on the texture feature type corresponding to the current block and the neighboring samples includes: Determining a sample weight combination based on the texture feature type corresponding to the current block, wherein the sample weight combination includes sample weights corresponding to different neighboring samples, and different texture feature types correspond to different sample weight combinations; The estimated value of the current block is determined based on the sample weight combination and reference neighboring samples, wherein different texture feature types correspond to reference neighboring samples of the same orientation.

12. The method according to claim 1, characterized in that The determining the continuity cost of the current block based on the predicted value of the current block, the assumed residual value, and the neighboring samples includes: For a group in the current block, determining a continuity cost of the group based on the predicted value of the group, the residual value of the hypothesis, and the neighboring samples; A continuity cost of the current block is determined based on the continuity cost of at least one group.

13. The method according to claim 12, characterized in that The upper edge samples and the lower edge samples of the current block belong to the same group; or, The upper edge sample of the current block belongs to the first group, and the left edge sample of the current block belongs to the second group; or, n consecutive edge samples in the current block belong to the same group, where n is a positive integer, and the edge samples belong to upper edge samples and lower edge samples of the current block; or, P consecutive upper edge samples in the current block belong to the same group, and q consecutive left edge samples in the current block belong to the same processing unit, where p and q are both positive integers.

14. The method according to claim 1, wherein The assumed residual value is determined based on the assumed sign combination of the quantization coefficients, or, The assumed residual value is determined based on the assumed transformation kernel of the quantized coefficients.

15. A decoding method, characterized in that: The method comprises: Determine the predicted value of the current block; performing an inverse transform on the quantized coefficients according to a hypothesized sign combination of the quantized coefficients to obtain a hypothesized residual value of the current block; Determining, based on the predicted value of the current block, the residual value of the hypothesis, and neighboring samples, a continuity cost of the current block under the sign combination of the hypothesis; The code stream is decoded according to the sign combination of the hypothesis corresponding to the minimum continuity cost.

16. A decoding method, characterized in that: The method comprises: Determine the predicted value of the current block; performing an inverse transform on the quantized coefficient according to a hypothetical transform kernel of the quantized coefficient to obtain a hypothetical residual value of the current block; Determining a continuity cost of the current block under the assumed transformation kernel based on the predicted value of the current block, the assumed residual value, and neighboring samples; A transform kernel is determined from the assumed transform kernels according to the continuity cost, and a code stream is decoded based on the determined transform kernel.

17. A decoder, characterized in that: The decoder comprises: The decoding unit is configured to determine a continuity cost of the current block based on a predicted value of the current block, an assumed residual value, and neighboring samples.

18. A decoder, characterized in that The decoder comprises: A decoding unit for determining a prediction value of a current block; The decoding unit is configured to perform an inverse transform on the quantization coefficients according to a hypothesized sign combination of the quantization coefficients to obtain a hypothesized residual value of the current block; The decoding unit is configured to determine a continuity cost of the current block under the hypothesized sign combination based on the predicted value of the current block, the hypothesized residual value, and neighboring samples; The decoding unit is configured to decode the code stream according to the hypothetical sign combination corresponding to the minimum continuity cost.

19. A decoder, characterized in that: The decoder comprises: A decoding unit for determining a prediction value of a current block; The decoding unit is configured to perform an inverse transform process on the quantized coefficient according to a hypothetical transform kernel of the quantized coefficient to obtain a hypothetical residual value of the current block; The decoding unit is configured to determine a continuity cost of the current block under the assumed transformation kernel based on the predicted value of the current block, the assumed residual value, and neighboring samples; The decoding unit is configured to determine a transform kernel from the assumed transform kernels according to the continuity cost, and decode the bitstream based on the determined transform kernel.

20. A decoder, characterized in that The decoder includes a memory and a processor, the memory is used to store a computer program running on the processor; the processor is used to execute the continuity cost determination method according to any one of claims 1 to 14, or the decoding method according to claim 15, or the decoding method according to claim 16 when running the computer program.

21. A non-volatile computer-readable storage medium for storing a code stream, characterized in that: The code stream is decoded by using a decoding method of a decoder, wherein the decoding method is the decoding method according to claim 15 or 16.

22. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. The processor obtains the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the continuity cost determination method according to any one of claims 1 to 14, or the decoding method according to claim 15, or the decoding method according to claim 16.

Images