Decoding method, encoding method, decoder, and encoder
Patent Information
- Application Number
- MYPI2025006208
- Authority / Receiving Office
- MY · MY
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-11
- Publication Date
- 2024-10-11
AI Technical Summary
Existing digital video decompression technology still has room for improvement in improving video transmission bandwidth and traffic efficiency, resulting in insufficient decoding performance.
By determining the maximum number of reference lines allowed to be used in the current block, it is determined whether to use the template-based multi-reference line TMRL mode to predict the current block, thereby improving decoding performance.
It effectively improves the decoding performance of the decoder and optimizes resource utilization and coding efficiency in the video encoding and decoding process.
Smart Images

Figure PI2025006208_DRAWING_001 
Figure PI2025006208_DRAWING_002 
Figure PI2025006208_DRAWING_003
Abstract
Description
Decoding method, encoding method, decoder and encoder Technical Field
[0001] The embodiments of the present application relate to the field of coding and decoding technology, and more specifically, to a decoding method, an encoding method, a decoder, and an encoder. Background Art
[0002] Digital video compression technology primarily compresses large amounts of digital video data for easier transmission and storage. With the surge in Internet video usage and increasing demand for higher-quality video, while existing digital video compression standards are capable of implementing video decompression, there is still a need for better digital video decompression technologies to reduce bandwidth and traffic pressures associated with digital video transmission, thereby improving decoder performance.
[0003] Summary of the Invention
[0004] The embodiments of the present application provide a decoding method, an encoding method, a decoder, and an encoder, which can improve the decoding performance of the decoder.
[0005] In a first aspect, an embodiment of the present application provides a decoding method, comprising:
[0006] Determine the maximum number of reference lines allowed for the current block;
[0007] Based on the maximum number of rows, determining whether the current block uses a template-based multiple reference row (TMRL) mode;
[0008] If the current block uses the TMRL mode, determining the reference row used by the current block and the prediction mode used by the current block;
[0009] Based on the reference row used by the current block, the current block is predicted using the prediction mode used by the current block to obtain the predicted block.
[0010] In a second aspect, an embodiment of the present application provides an encoding method, including:
[0011] Determine the maximum number of reference lines allowed for the current block;
[0012] Encode this maximum number of rows.
[0013] In a third aspect, an embodiment of the present application provides a decoder, including:
[0014] A first determining unit, configured to determine a maximum number of reference rows allowed to be used by the current block;
[0015] a second determining unit, configured to determine, based on the maximum number of rows, whether the current block uses a template-based multiple reference row (TMRL) mode;
[0016] a third determining unit, configured to determine a reference row used by the current block and a prediction mode used by the current block if the current block uses the TMRL mode;
[0017] The prediction unit is configured to predict the current block based on a reference row used by the current block and using a prediction mode used by the current block to obtain the predicted block.
[0018] In a fourth aspect, an embodiment of the present application provides an encoder, including:
[0019] A determination unit, used to determine the maximum number of reference rows allowed to be used in the current block;
[0020] The encoding unit is used to encode the maximum number of rows.
[0021] In a fifth aspect, an embodiment of the present application provides a decoder, including:
[0022] a processor adapted to implement computer instructions; and,
[0023] A computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the decoding method in the first aspect or its various implementations mentioned above.
[0024] In one implementation, there are one or more processors and one or more memories.
[0025] In one implementation, the computer-readable storage medium may be integrated with the processor, or the computer-readable storage medium may be provided separately from the processor.
[0026] In a sixth aspect, an embodiment of the present application provides an encoder, including:
[0027] a processor adapted to implement computer instructions; and,
[0028] A computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the encoding method in the second aspect or its various implementations mentioned above.
[0029] In one implementation, there are one or more processors and one or more memories.
[0030] In one implementation, the computer-readable storage medium may be integrated with the processor, or the computer-readable storage medium may be provided separately from the processor.
[0031] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are read and executed by a processor of a computer device, the computer device executes the decoding method involved in the first aspect mentioned above or the encoding method involved in the second aspect mentioned above.
[0032] In an eighth aspect, embodiments of the present application provide a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the decoding method of the first aspect or the encoding method of the second aspect.
[0033] In a ninth aspect, an embodiment of the present application provides a code stream, which is the code stream involved in the method described in the first aspect above or the code stream generated by the method described in the second aspect above.
[0034] Based on the above technical solution, when the decoder predicts the current block, it first determines the maximum number of reference rows allowed for the current block; then, based on the maximum number of rows, it determines whether the current block uses the TMRL mode; if the current block uses the TMRL mode, it determines the reference rows used by the current block and the prediction mode used by the current block; then, based on the reference rows used by the current block, it uses the prediction mode used by the current block to predict the current block to obtain the predicted block. In other words, when the decoder determines whether to use the TMRL mode to predict the current block, it needs to use the maximum number of rows as a condition to determine whether the current block uses the TMRL mode, so that when the current block uses the TMRL mode, the maximum number of rows can meet the usage condition, thereby improving decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] FIG1 is a schematic block diagram of a coding framework provided in an embodiment of the present application.
[0036] FIG2 is a schematic block diagram of a decoding framework provided in an embodiment of the present application.
[0037] FIG3 is an example of four reconstructed rows located around a current block provided by an embodiment of the present application.
[0038] FIG4 is an example of a template and reference lines located around a current block provided by an embodiment of the present application.
[0039] FIG5 is a schematic flowchart of a decoding method provided in an embodiment of the present application.
[0040] FIG6 is a schematic flowchart of the encoding method provided in an embodiment of the present application.
[0041] FIG7 is a schematic block diagram of a decoder provided in an embodiment of the present application.
[0042] FIG8 is a schematic block diagram of an encoder provided in an embodiment of the present application.
[0043] FIG9 is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0044] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.
[0045] The solution provided in the embodiment of the present application can be applied to the field of digital video coding technology, for example, including but not limited to: image coding and decoding field, video coding and decoding field, hardware video coding and decoding field, dedicated circuit video coding and decoding field and real-time video coding and decoding field. In addition, the solution provided in the embodiment of the present application can be combined with the audio and video coding standard (Audio Video Coding Standard, AVS), the second generation AVS standard (AVS2) or the third generation AVS standard (AVS3). For example, including but not limited to: H.264 / Audio Video Coding (AVC) standard, H.265 / High Efficiency Video Coding (HEVC) standard and H.266 / Versatile Video Coding (VVC) standard. In addition, the solution provided in the embodiment of the present application can be used for lossy compression of images, and can also be used for lossless compression of images. The lossless compression can be visually lossless compression or mathematically lossless compression.
[0046] Video coding standards all adopt a block-based hybrid coding framework.
[0047] Each frame in a video is divided into square largest coding units (LCUs) or coding tree units (CTUs) of the same size (e.g., 128x128, 64x64, etc.). Each LCU or CTU can be divided into rectangular coding units (CUs) according to a rule. Coding units may also be divided into prediction units (PUs), transform units (TUs), etc. The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra prediction and inter prediction. Inter prediction includes motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels in a frame of a video, intra prediction is used in video coding and decoding technology to eliminate spatial redundancy between adjacent pixels. Intra prediction only refers to information from the same frame image to predict pixel information within the current partitioned block. Because adjacent frames in a video have strong similarities, inter-frame prediction is used in video coding and decoding to eliminate temporal redundancy between them, thereby improving coding efficiency. Inter-frame prediction refers to image information from different frames and uses motion estimation to search for the motion vector information that best matches the current block. Transformation converts the predicted image block to the frequency domain, redistributing the energy. Combined with quantization, it removes information that is insensitive to the human eye, eliminating visual redundancy. Entropy coding can eliminate character redundancy based on the current context model and the probabilistic information of the binary bitstream.
[0048] The basic process of video codec is as follows:
[0049] On the encoder side, a frame is divided into blocks. Intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The prediction block is subtracted from the original image block to obtain a residual block. The residual block is transformed and quantized to obtain a quantization coefficient matrix, which is then entropy-encoded and output to the bitstream. On the decoder side, intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The bitstream is then parsed to obtain a quantization coefficient matrix, which is then inversely quantized and inversely transformed to obtain a residual block. The prediction block and residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image, which is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoder side also performs similar operations as the decoder to obtain a decoded image. The decoded image can serve as a reference frame for inter-frame prediction in subsequent frames. The block division information determined by the encoder, as well as information about the prediction, transform, quantization, entropy coding, loop filtering, and other modes or parameters, are output to the bitstream if necessary. The decoding end determines the same block division information as the encoding end by parsing and analyzing the existing information, as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is also usually called a reconstructed image. The current block can be divided into prediction units during prediction, and the current block can be divided into transformation units during transformation. The division of prediction units and transformation units can be different. The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized, and this application does not make specific restrictions on this.
[0050] The current block may be a current coding unit (CU) or a current prediction unit (PU), etc.
[0051] It should be noted that the encoder also requires similar operations as the decoder to obtain a decoded image. The decoded image can serve as a reference frame for subsequent frames for inter-frame prediction. The block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode information or parameter information determined by the encoder are output to the bitstream if necessary. The decoder determines the same block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode information or parameter information as the encoder by parsing and analyzing existing information, thereby ensuring that the decoded image obtained by the encoder and the decoder are identical. The decoded image obtained by the encoder is also commonly referred to as a reconstructed image. During prediction, the current block can be divided into prediction units, and during transform, the current block can be divided into transform units. The division of prediction units and transform units can be the same or different. Of course, the above is only the basic process of a video codec within a block-based hybrid coding framework. With technological advancement, some modules of the framework or some steps of the process may be optimized. This application applies to the basic process of a video codec within this block-based hybrid coding framework.
[0052] FIG1 is a schematic block diagram of a coding framework 100 provided in an embodiment of the present application.
[0053] As shown in FIG1 , the coding framework 100 may include an intra-frame prediction unit 180, an inter-frame prediction unit 170, a residual unit 110, a transform and quantization unit 120, an entropy coding unit 130, an inverse transform and inverse quantization unit 140, and a loop filter unit 150. Optionally, the coding framework 100 may further include a decoded image buffer unit 160. The coding framework 100 may also be referred to as a hybrid framework coding mode.
[0054] The intra-frame prediction unit 180 or the inter-frame prediction unit 170 may predict the image block to be encoded and output a prediction block. The residual unit 110 may calculate a residual block based on the prediction block and the image block to be encoded, i.e., the difference between the prediction block and the image block to be encoded. The transform and quantization unit 120 is configured to perform operations such as transform and quantization on the residual block to remove information that is insensitive to the human eye, thereby eliminating visual redundancy. Optionally, the residual block before transformation and quantization by the transform and quantization unit 120 may be referred to as a time-domain residual block, and the time-domain residual block after transformation and quantization by the transform and quantization unit 120 may be referred to as a frequency residual block or a frequency-domain residual block. After receiving the transform and quantization coefficients output by the transform and quantization unit 120, the entropy coding unit 130 may output a bitstream based on the transform and quantization coefficients. For example, the entropy coding unit 130 may eliminate character redundancy based on a target context model and probability information of the binary bitstream. For example, the entropy coding unit 130 may be configured for context-based adaptive binary arithmetic entropy coding (CABAC). The entropy coding unit 130 may also be referred to as a header information coding unit. Optionally, in this application, the image block to be encoded may also be referred to as an original image block or a target image block, the prediction block may also be referred to as a prediction image block or an image prediction block, or may also be referred to as a prediction signal or prediction information, and the reconstruction block may also be referred to as a reconstructed image block or an image reconstruction block, or may also be referred to as a reconstruction signal or reconstruction information. In addition, for the encoding end, the image block to be encoded may also be referred to as a coding block or a coding image block, and for the decoding end, the image block to be encoded may also be referred to as a decoding block or a decoding image block. The image block to be encoded may be a CTU or a CU.
[0055] The encoding framework 100 calculates the residual between the prediction block and the image block to be encoded, generates a residual block, and transmits the residual block to the decoder through processes such as transformation and quantization. Correspondingly, the decoder receives and parses the bitstream, then performs inverse transformation and inverse quantization to obtain a residual block. The predicted block, obtained by the decoder, is then superimposed on the residual block to create a reconstructed block.
[0056] It should be noted that the inverse transform and inverse quantization unit 140, the loop filter unit 150, and the decoded image buffer unit 160 in the coding framework 100 can be used to form a decoder. This means that the intra-frame prediction unit 180 or the inter-frame prediction unit 170 can predict the image block to be coded based on the existing reconstructed block, thereby ensuring that the encoder and decoder have a consistent understanding of the reference frame. In other words, the encoder can replicate the decoder's processing loop and produce the same prediction as the decoder. Specifically, the quantized transform coefficients are inversely transformed and inversely quantized by the inverse transform and inverse quantization unit 140 to replicate the approximate residual block on the decoder side. This approximate residual block, after adding the prediction block, can pass through the loop filter unit 150 to smooth out block artifacts caused by block-based processing and quantization. The image block output by the loop filter unit 150 can be stored in the decoded image buffer unit 160 for use in subsequent image prediction.
[0057] It should be understood that FIG1 is merely an example of the present application and should not be construed as limiting the present application.
[0058] For example, the loop filtering unit 150 in the coding framework 100 may include a deblocking filter (DBF) and a sample adaptive offset filter (SAO). The function of DBF is to remove the blocking effect, and the function of SAO is to remove the ringing effect. In other embodiments of the present application, the coding framework 100 may adopt a neural network-based loop filtering algorithm to improve the compression efficiency of the video. In other words, the coding framework 100 may be a video coding hybrid framework based on a deep learning neural network. In one implementation, a convolutional neural network-based model may be used to calculate the result of pixel filtering on the basis of a deblocking filter and a sample adaptive offset filter. The network structure of the loop filtering unit 150 on the luminance component and the chrominance component may be the same or different. Considering that the luminance component contains more visual information, the luminance component may also be used to guide the filtering of the chrominance component to improve the reconstruction quality of the chrominance component.
[0059] FIG2 is a schematic block diagram of a decoding framework 200 provided in an embodiment of the present application.
[0060] As shown in FIG. 2 , the decoding framework 200 may include an entropy decoding unit 210 , an inverse transform and dequantization unit 220 , a residual unit 230 , an intra-frame prediction unit 240 , an inter-frame prediction unit 250 , a loop filtering unit 260 , and a decoded image buffer unit 270 .
[0061] After receiving and parsing the bitstream, the entropy decoding unit 210 obtains a prediction block and a frequency domain residual block. The frequency domain residual block is subjected to inverse transformation and inverse quantization by the inverse transform and inverse quantization unit 220 to obtain a time domain residual block. The residual unit 230 superimposes the prediction block predicted by the intra-frame prediction unit 240 or the inter-frame prediction unit 250 onto the time domain residual block that has been inverse transformed and inverse quantized by the inverse transform and inverse quantization unit 220 to obtain a reconstructed block. For example, the intra-frame prediction unit 240 or the inter-frame prediction unit 250 can obtain the prediction block by decoding header information of the bitstream.
[0062] JVET, the international video coding standards organization, has established a group to research coding models that surpass H.266 / VVC. This model, or platform testing software, has been named ECM. Building on VTM10.0, ECM incorporates newer and more efficient compression algorithms, currently surpassing VVC's coding performance by approximately 13%. ECM not only increases the coding unit size for specific resolutions but also integrates numerous intra- and inter-frame prediction techniques.
[0063] The following describes the related technologies involved in this application.
[0064] (1) Most probable modes (MPM).
[0065] MPM is an intra-frame prediction mode. MPM usually includes some modes that are most likely to be selected for the current block. The MPM list usually includes one or more of the following modes:
[0066] a) The prediction mode selected by the surrounding adjacent prediction blocks.
[0067] b) The prediction mode selected by the surrounding non-adjacent prediction blocks.
[0068] c) The surrounding reconstructed pixels use the prediction mode derived from the decoder side intra mode derivation (DIMD) mode.
[0069] d) Some similar angles of the angle patterns obtained in a and b.
[0070] e) The default and more commonly used intra-frame prediction mode.
[0071] The prediction mode selected by the adjacent prediction blocks may be the prediction mode used by the adjacent prediction blocks; similarly, the prediction mode selected by the non-adjacent prediction blocks may be the prediction mode used by the non-adjacent prediction blocks.
[0072] The MPM in the ECM includes 22 prediction modes, the first six modes are primary MPMs, and the last sixteen modes are secondary MPMs. Among the 22 MPM modes, the planar mode is located at the first position.
[0073] The parsing process of MRL related grammatical elements is roughly as shown in Table 1 below:
[0074] Table 1
[0075] As shown in Table 1, if mrl_flag is set, the decoder determines that it is not a multiple reference line (MRL) intra prediction mode (no extended reference line) and decodes mpm_flag; otherwise, mpm_flag is set to 1 by default; further, if mpm_flag is true, mpm_idx is parsed.
[0076] (2) Multiple reference line (MRL) mode.
[0077] The MRL mode allows intra-frame prediction to use not only the reconstructed line closest to the current block (Reference line 0) as a reference line, but also reconstructed lines that are not adjacent to the current block as reference lines. The MRL mode can use more reference pixels, thereby improving coding efficiency.
[0078] FIG3 is an example of four reference rows located around a current block provided by an embodiment of the present application.
[0079] As shown in Figure 3, the reference lines in the VCC that are not adjacent to the current block may be reference lines 1 and 2, and the reference lines in the ECM that are not adjacent to the current block may be reference lines 1, 3, 5, 7, and 12. Figure 3 shows four reference lines, 0 through 3, which can be divided into segments A through F.
[0080] The parsing process of CU-level MRL related syntax elements is roughly as shown in Table 2 below:
[0081] Table 2
[0082] As shown in Table 2, if !sps_mrl_flag||cu.timd||cu.dimd||isFirstLineOfCtu||cu.bdpcmMode||!cu.Y().valid, then multiRefIdx = 0, indicating that the decoder determines not to use MRL. Otherwise, multiRefIdx is obtained by parsing the bitstream, which indicates the reference line to use. MULTI_REF_LINE_IDX is a 6-length array containing {0, 1, 3, 5, 7, 12}. The decoder can obtain the current reference line index based on the parsed multiRefIdx and the array.
[0083] In addition, the MRL mode can be used simultaneously with the TIMD mode. When the current block is in TIMD mode, the CU-level MRL-related syntax element parsing process is roughly as shown in Table 3 below:
[0084] Table 3
[0085] As shown in Table 3, if !sps_mrl_flag||cu.timd||cu.dimd||isFirstLineOfCtu||cu.bdpcmMode||!cu.Y().valid, the decoder determines not to use MRL in TIMD mode. Otherwise, if TIMD mode is selected for the current coding unit, multiRefIdx is obtained by parsing the bitstream, which is used to indicate the reference line to be used. MULTI_REF_LINE_IDX is a 6-length array containing {0, 1, 3, 5, 7, 12}. The decoder can obtain the index of the current reference line based on the parsed multiRefIdx and the array.
[0086] It is worth noting that the MPM technology is a technology used to determine the prediction mode, and the MRL technology is a technology used to determine the reference row. The MPM technology and the MRL technology can be used in combination or separately, and this application does not make any specific restrictions on this.
[0087] (3) Template-based Multiple Reference Line (TMRL) mode.
[0088] The TMRL mode is an extension of the MRL technology. The TMRL mode uses template matching technology to sort the reconstruction lines and prediction modes that may be used for the current block, and construct the more likely combinations of reconstruction lines and prediction modes into a TMRL candidate list. The decoder parses the candidate index to select the reconstruction line and prediction mode selected for the current block from the TMRL candidate list and perform prediction.
[0089] For the TMRL mode, the decoder needs to complete the following steps:
[0090] 1. Get the reconstructed lines and possible intra prediction modes that can be used.
[0091] 2. Perform intra prediction on the template area using each possible reconstruction line and intra prediction mode, and calculate the difference between the prediction result and the reconstruction result for each combination. For example, the difference value can be the sum of absolute differences (SAD).
[0092] 3. Sort the SAD values of all combinations from small to large, and select the smaller combinations as the TMRL candidate list.
[0093] If the available reconstruction lines are {0, 1, 3, 5, 7, 12} and the possible intra prediction modes include 10 prediction modes, the decoder uses each possible reconstruction line and intra prediction mode to perform intra prediction on the template area and calculates the SAD of the prediction result and the reconstruction result of each combination to obtain 50 SADs; then the decoder sorts these 50 SADs from small to large and selects the 20 smallest combinations as the TMRL candidate list.
[0094] It is worth noting that since the TMRL mode is designed to obtain 10 possible intra prediction modes for each intra prediction block, and the length of the TMRL candidate list is 20, TMRL can only work properly when there are more than or equal to 2 reconstructed rows available for the current block.
[0095] FIG4 is an example of a template and reference lines located around a current block provided by an embodiment of the present application.
[0096] As shown in FIG4 , the area where reference line 0 is located is the template area of the current block, and the reconstructed lines of the current block include reference lines 1 to 6 .
[0097] The parsing process of the relevant syntax elements of the CU-level TMRL model is roughly as shown in Table 4 below:
[0098] Table 4
[0099] As shown in Table 4, if sps_mrl_flag&&! dimd&&! timd&&! isFirstLineOfCtu&&! bdpcmMode&&cu.Y().valid, the decoder decodes tmrlFlag; otherwise, if tmrlFlag is true, tmrlListIdx is obtained by parsing the codestream, which is used to indicate the candidate in the TMRL candidate list.
[0100] FIG5 is a schematic flow chart of a decoding method 300 provided in an embodiment of the present application. It should be understood that the decoding method 300 can be executed by a decoder. For example, it can be applied to the decoding framework 200 shown in FIG2 . For ease of description, the following description uses a decoder as an example.
[0101] As shown in FIG5 , the decoding method 300 may include:
[0102] S310: The decoder determines the maximum number of reference lines allowed for the current block. For example, if the current block is not the first line of the current CTU, the current block does not use the DIMD mode, the current block does not use the BDPCM mode, and the current block is a luminance block, the decoder determines the maximum number of lines by decoding the bitstream.
[0103] Exemplarily, the decoder determines the maximum number of rows by decoding the code stream.
[0104] Exemplarily, the reference row used by the current block may be a reference row that is not (or not) adjacent to the current block.
[0105] Exemplarily, the reference line used by the current block may be a reference line formed by reconstructed samples located above and to the left of the current block.
[0106] It should be understood that "the maximum number of reference rows allowed to be used by the current block" is intended to explain: the number of rows that can be used as reference rows for the current block. The decoder can select the reference rows actually used by the current block from the "reference rows that can be used as the current block", that is, the decoder can select the reference rows used to predict the current block from the "reference rows that can be used as the current block". In other words, "the maximum number of reference rows allowed to be used by the current block" can be understood as or equivalently replaced by: "the maximum number of reference rows allowed to be used by the current block", "the number of reference rows that can be used as the current block", "the maximum number of rows allowed to be used by the current block", "the maximum number of reference rows that can be used by the current block", "the maximum number of reference rows that can be used by the current block", and other terms with similar meanings or descriptions with similar meanings, and this application does not make specific limitations on this.
[0107] S320: The decoder determines whether the current block uses a template-based multiple reference line (TMRL) mode based on the maximum number of lines.
[0108] Exemplarily, if the maximum number of rows satisfies the usage conditions of the TMRL mode, the decoder determines that the current block uses the TMRL mode; otherwise, the decoder determines that the current block does not use the TMRL mode. For example, if the maximum number of rows satisfies the usage conditions of the TMRL mode, the decoder determines that the current block uses the TMRL mode by decoding the bitstream; otherwise, the decoder determines that the current block does not use the TMRL mode, that is, the decoder determines that the current block uses a prediction mode other than the TMRL mode by decoding the bitstream. The maximum number of rows satisfying the usage conditions of the TMRL mode can be understood as the maximum number of rows satisfying the decoder's construction of a TMRL mode list of a specific length.
[0109] S330: If the current block uses the TMRL mode, the decoder determines a reference row used by the current block and a prediction mode used by the current block.
[0110] Exemplarily, if the current block uses the TMRL mode, the decoder determines the reference row used by the current block and the prediction mode used by the current block by decoding the bitstream.
[0111] S340: The decoder predicts the current block based on the reference row used by the current block and using the prediction mode used by the current block to obtain the predicted block.
[0112] Exemplarily, if the prediction mode used by the current block is an angle prediction mode, the decoder can determine the reconstructed samples in the reference row used by the current block as the prediction samples in the current block according to the angle indicated by the angle prediction mode, and thus obtain the prediction block of the current block.
[0113] In this embodiment, when the decoder predicts the current block, it first determines the maximum number of reference lines allowed for the current block; then, based on the maximum number of lines, it determines whether the current block uses the TMRL mode; if the current block uses the TMRL mode, it determines the reference lines used by the current block and the prediction mode used by the current block; then, based on the reference lines used by the current block, it predicts the current block using the prediction mode used by the current block to obtain the predicted block. In other words, when the decoder determines whether to use the TMRL mode to predict the current block, it needs to use the maximum number of reference lines allowed for the current block as a condition to determine whether the current block uses the TMRL mode. When the maximum number of reference lines allowed for the current block satisfies the usage condition when the current block uses the TMRL mode, decoding performance can be improved.
[0114] In some embodiments, S310 may include:
[0115] The decoder obtains the first identifier;
[0116] Among them, when the value of the first identifier is a first value, it indicates that the current sequence to which the current block belongs does not use the multi-reference row MRL mode; when the value of the first identifier is a second value other than the first value, it indicates that the current sequence uses the MRL mode and indicates the maximum number of rows.
[0117] Exemplarily, the first identifier can be recorded as sps_mrl_enabled_flag, that is, sps_mrl_enabled_flag is a sequence-level MRL control identifier. The value range of sps_mrl_enabled_flag is 0 to x (x is a positive integer). When the value of sps_mrl_enabled_flag is 0, it indicates that the sequence-level MRL is closed. When the value of sps_mrl_enabled_flag is 1 to x, it indicates that the current sequence uses the MRL mode and indicates the maximum number of rows. For example, when the value of sps_mrl_enabled_flag is i (1≤i≤x), it indicates that the current sequence uses the MRL mode and indicates that the maximum number of rows is a value determined according to i. For example, in ECM, the value range of the second identifier is 0 to 5, and the second identifier can be represented by a 3-bit fixed-length code "u(3)".
[0118] Exemplarily, when the first identifier does not exist in the code stream obtained by the decoder, the value of the first identifier may default to 0 or other values.
[0119] Exemplarily, when the value of the first identifier is the second value, the decoder directly determines the second value as the maximum number of rows. For example, when the value of the first identifier is 5, the decoder determines the value of the first identifier (ie, 5) as the maximum number of rows.
[0120] For example, when the value of the first identifier is a second numerical value, the decoder may determine the number of rows corresponding to the second numerical value as the maximum number of rows based on a correspondence between at least one numerical value and at least one row number; the at least one numerical value includes the second numerical value. For example, when the value of the first identifier is 4, the decoder may determine the number of rows corresponding to 4 (e.g., 5) as the maximum number of rows based on a correspondence between at least one numerical value and at least one row number.
[0121] Of course, in other alternative embodiments, the first identifier may also be an image-level identifier or an image block-level identifier.
[0122] For example, if the first identifier is an image-level identifier, then: when the value of the first identifier is a first numerical value, it indicates that the current image to which the current block belongs does not use the multi-reference line MRL mode; when the value of the first identifier is a second numerical value other than the first numerical value, it indicates that the current image uses the MRL mode and indicates the maximum number of rows.
[0123] For another example, if the first identifier is an image block level identifier, then: when the value of the first identifier is a first numerical value, it indicates that the current block does not use the multi-reference row MRL mode; when the value of the first identifier is a second numerical value other than the first numerical value, it indicates that the current block uses the MRL mode and indicates the maximum number of rows.
[0124] It should be noted that when the value of the first identifier is the second numerical value, the value of the first identifier can be used only to indicate that the current sequence uses the MRL mode and indicates the maximum number of rows. It can also indicate the reference rows allowed to be used by the current block on the basis of indicating that the current sequence uses the MRL mode and indicates the maximum number of rows. In other words, the value of the first identifier is not only used to indicate the following content: the current sequence uses the MRL mode, the maximum number of rows, and the reference rows allowed to be used by the current block.
[0125] For example, if the maximum value of the first flag is 5, then:
[0126] When the value of the first flag is 0, it indicates that the current sequence does not use the MRL mode. When the value of the first flag is 1, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 1, and the reference row allowed to be used by the current block is reference row 1. When the value of the first flag is 2, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 2, and the reference rows allowed to be used by the current block include reference rows 1 and 3. When the value of the first flag is 3, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 3, and the reference rows allowed to be used by the current block include reference rows 1, 3, and 5. When the value of the first flag is 4, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 4, and the reference rows allowed to be used by the current block include reference rows 1, 3, 5, and 7. When the value of the first flag is 5, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 5, and the reference rows allowed to be used by the current block include reference rows 1, 3, 5, 7, and 12.
[0127] Of course, in other alternative embodiments, there may be other corresponding relationships between the second value and the reference row allowed to be used by the current block, and this application does not make specific limitations on this.
[0128] In some embodiments, S310 may include:
[0129] The decoder obtains the second identifier;
[0130] The decoder determines the number of rows indicated by the second identifier as the maximum number of rows.
[0131] Exemplarily, the first identifier may be recorded as num_non_adjacent_ref_line. The value range of num_non_adjacent_ref_line is 0 to x (x is a positive integer), indicating the maximum number of lines. For example, when the value of sps_mrl_enabled_flag is i (1≤i≤x), it indicates that the maximum number of lines is a value determined according to i. For example, in the ECM, the value range of the second identifier is 0 to 5, and the second identifier can be represented by a 3-bit fixed-length code "u(3)".
[0132] Exemplarily, when the second identifier does not exist in the code stream obtained by the decoder, the value of the second identifier may default to 0 or other values.
[0133] Exemplarily, the decoder directly determines the value of the second identifier as the maximum number of rows. For example, when the value of the second identifier is 5, the decoder determines the value of the second identifier (ie, 5) as the maximum number of rows.
[0134] Exemplarily, the decoder may determine the number of rows corresponding to the value of the second identifier as the maximum number of rows based on a correspondence between at least one numerical value and at least one row number; the at least one numerical value includes the value of the second identifier. For example, when the value of the second identifier is 4, the decoder may determine the number of rows corresponding to 4 (e.g., 5) as the maximum number of rows based on the correspondence between the at least one numerical value and the at least one row number.
[0135] Exemplarily, the second identifier is a sequence-level identifier.
[0136] Of course, in other alternative embodiments, the second identifier may also be an image-level identifier or an image block-level identifier.
[0137] For example, if the second identifier is a picture-level identifier, then: the second identifier indicates the number of reference lines allowed to be used by the current picture to which the current block belongs.
[0138] For another example, if the second flag is an image block level flag, then: the second flag indicates the number of reference rows that the current block is allowed to use.
[0139] It should be noted that the second identifier can be used only to indicate the maximum number of rows, and can also indicate the reference rows allowed to be used by the current block on the basis of indicating the maximum number of rows; in other words, the value of the second identifier is not only used to indicate the following: the maximum number of rows, the reference rows allowed to be used by the current block.
[0140] For example, if the maximum value of the second flag is 5, then:
[0141] When the value of the second identifier is 0, it indicates the following: the maximum number of rows is 0, and the reference row allowed to be used by the current block is empty. When the value of the second identifier is 1, it indicates the following: the maximum number of rows is 1, and the reference row allowed to be used by the current block is reference row 1. When the value of the second identifier is 2, it indicates the following: the maximum number of rows is 2, and the reference rows allowed to be used by the current block include reference rows 1 and 3. When the value of the second identifier is 3, it indicates the following: the maximum number of rows is 3, and the reference rows allowed to be used by the current block include reference rows 1, 3, and 5. When the value of the second identifier is 4, it indicates the following: the maximum number of rows is 4, and the reference rows allowed to be used by the current block include reference rows 1, 3, 5, and 7. When the value of the second identifier is 5, it indicates the following: the maximum number of rows is 5, and the reference rows allowed to be used by the current block include reference rows 1, 3, 5, 7, and 12.
[0142] Of course, in other alternative embodiments, there may be other corresponding relationships between the value of the second identifier and the reference row allowed to be used by the current block, and this application does not make specific limitations on this.
[0143] In some embodiments, S320 may include:
[0144] If the maximum number of rows is greater than a first preset threshold, the decoder obtains a third identifier;
[0145] If the third identifier indicates that the current sequence to which the current block belongs allows the use of the TMRL mode, the decoder obtains a fourth identifier;
[0146] The fourth flag indicates whether the current block uses the TMRL mode.
[0147] Exemplarily, if the maximum number of rows is greater than a first preset threshold, the decoder decodes the code stream to obtain a third identifier; if the third identifier indicates that the current sequence to which the current block belongs allows the use of the TMRL mode, the decoder decodes the code stream to obtain a fourth identifier; wherein the fourth identifier indicates whether the current block uses the TMRL mode.
[0148] Exemplarily, the first preset threshold may be a threshold predefined by a protocol.
[0149] Exemplarily, the first preset threshold may be 1, 2 or other values, which is not specifically limited in this application.
[0150] Exemplarily, the third flag may be recorded as sps_tmrl_enabled_flag, and the fourth flag may be recorded as cu_tmrl_flag.
[0151] If the maximum number of rows is greater than a first preset threshold (e.g., the first preset threshold is 1), the decoder decodes the bitstream to obtain sps_tmrl_enabled_flag; if sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode (e.g., the value of sps_tmrl_enabled_flag is 1), the decoder decodes the bitstream to obtain cu_tmrl_flag. When the value of cu_tmrl_flag is 1, it indicates that the current block uses the TMRL mode; when the value of cu_tmrl_flag is 0, it indicates that the current block does not use the TMRL mode.
[0152] In some embodiments, if the third flag indicates that the current sequence allows the use of the TMRL mode and the current block does not use the TIMD mode, the decoder obtains the fourth flag.
[0153] Exemplarily, the decoder may determine whether the current block uses the TIMD mode by decoding the code stream.
[0154] Exemplarily, the third flag may be recorded as sps_tmrl_enabled_flag, and the fourth flag may be recorded as cu_tmrl_flag.
[0155] If the maximum number of rows is greater than a first preset threshold (for example, the first preset threshold is 1), the decoder decodes the code stream to obtain sps_tmrl_enabled_flag; if sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1) and the current block does not use the TIMD mode, the decoder decodes the code stream to obtain cu_tmrl_flag. When the value of cu_tmrl_flag is 1, it indicates that the current block uses the TMRL mode; when the value of cu_tmrl_flag is 0, it indicates that the current block does not use the TMRL mode.
[0156] In some embodiments, S320 may include:
[0157] The decoder obtains the third identifier;
[0158] If the third flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode and the maximum number of rows is greater than the first preset threshold, the decoder obtains a fourth flag;
[0159] The fourth flag indicates whether the current block uses the TMRL mode.
[0160] Exemplarily, the decoder decodes the code stream to obtain a third identifier; if the third identifier indicates that the current sequence to which the current block belongs allows the use of the TMRL mode and the maximum number of rows is greater than a first preset threshold, the decoder decodes the code stream to obtain a fourth identifier; wherein the fourth identifier indicates whether the current block uses the TMRL mode.
[0161] Exemplarily, the first preset threshold may be a threshold predefined by a protocol.
[0162] Exemplarily, the first preset threshold may be 1, 2 or other values, which is not specifically limited in this application.
[0163] Exemplarily, the third flag may be recorded as sps_tmrl_enabled_flag, and the fourth flag may be recorded as cu_tmrl_flag.
[0164] The decoder decodes the bitstream to obtain sps_tmrl_enabled_flag; if sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1), and the maximum number of rows is greater than a first preset threshold (for example, the first preset threshold is 1), the decoder decodes the bitstream to obtain cu_tmrl_flag. When the value of cu_tmrl_flag is 1, it indicates that the current block uses the TMRL mode; when the value of cu_tmrl_flag is 0, it indicates that the current block does not use the TMRL mode.
[0165] In some embodiments, if the third flag indicates that the current sequence allows the use of the TMRL mode, the maximum number of rows is greater than the first preset threshold, and the current block does not use the template-based intra-frame mode to derive the TIMD mode, then it is determined that the current block uses the TMRL mode.
[0166] Exemplarily, the third flag may be recorded as sps_tmrl_enabled_flag, and the fourth flag may be recorded as cu_tmrl_flag.
[0167] The decoder decodes the bitstream to obtain sps_tmrl_enabled_flag; if sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1), and the maximum number of rows is greater than a first preset threshold (for example, the first preset threshold is 1), and the current block does not use the TIMD mode, the decoder decodes the bitstream to obtain cu_tmrl_flag. When the value of cu_tmrl_flag is 1, it indicates that the current block uses the TMRL mode; when the value of cu_tmrl_flag is 0, it indicates that the current block does not use the TMRL mode.
[0168] In some embodiments, if the current sequence allows the use of the MRL mode, the decoder obtains the third identifier.
[0169] Exemplarily, if the decoder decodes the bitstream and determines that the current sequence allows the use of the MRL mode, the decoder obtains the third identifier and, based on the third identifier, determines whether the current sequence allows the use of the TMRL mode. Otherwise, the decoder determines that the current sequence does not use the TMRL mode. Alternatively, if the decoder decodes the bitstream and determines that the current sequence allows the use of the MRL mode, the decoder obtains the third identifier and, based on the third identifier, determines whether the current sequence allows the use of the TMRL mode. Otherwise, the decoder determines that the current block does not use the TMRL mode. That is, the decoder determines, by decoding the bitstream, that the current block uses a prediction mode other than the TMRL mode.
[0170] Exemplarily, the third flag may be recorded as sps_tmrl_enabled_flag, and the flag indicating whether the current sequence is allowed to use the MRL mode may be recorded as sps_mrl_enabled_flag.
[0171] If the current sequence allows the use of the MRL mode (for example, sps_mrl_enabled_flag is 1), the decoder decodes the bitstream to obtain sps_tmrl_enabled_flag, and based on the value of sps_tmrl_enabled_flag, determines whether the current sequence allows the use of the TMRL mode. For example, a value of 1 for sps_tmrl_enabled_flag indicates that the current sequence allows the use of the TMRL mode, while a value of 0 for sps_tmrl_enabled_flag indicates that the current sequence does not allow the use of the TMRL mode.
[0172] In some embodiments, the decoder obtains a fifth identifier; if the fifth identifier indicates that there is no restriction on the value of the third identifier, or the fifth identifier does not restrict the value of the identifier used to indicate whether the current sequence allows the use of the multi-reference line MRL mode, the decoder obtains the third identifier.
[0173] Exemplarily, the decoder decodes the code stream to obtain the fifth identifier. If the fifth identifier indicates that there is no restriction on the value of the third identifier, or the fifth identifier does not restrict the value of the identifier used to indicate whether the current sequence allows the use of the MRL mode, the decoder decodes the code stream to obtain the third identifier.
[0174] Exemplarily, the fifth identifier is general constraints information (GCI) of the third identifier, which is used to restrict the value of the third identifier.
[0175] The third flag may be recorded as sps_tmrl_enabled_flag, and the GCI of the third flag may be recorded as gci_no_tmrl_constraint_flag, which indicates whether sps_tmrl_enabled_flag is restricted. For example, when the value of gci_no_tmrl_constraint_flag is 1, the value of sps_tmrl_enabled_flag must be equal to 0. Otherwise, no restriction is imposed on the value of sps_tmrl_enabled_flag.
[0176] The decoder decodes the codestream to obtain gci_no_tmrl_constraint_flag. If gci_no_tmrl_constraint_flag indicates that the value of sps_tmrl_enabled_flag is not restricted (for example, the value of gci_no_tmrl_constraint_flag is 0), the decoder decodes the codestream to obtain sps_tmrl_enabled_flag, and determines whether the current sequence is allowed to use the TMRL mode based on the value of sps_tmrl_enabled_flag. For example, when the value of sps_tmrl_enabled_flag is 1, it indicates that the current sequence is allowed to use the TMRL mode, and when the value of sps_tmrl_enabled_flag is 0, it indicates that the current sequence is not allowed to use the TMRL mode.
[0177] Exemplarily, the fifth flag is a GCI used to indicate whether the current sequence allows the use of the multiple reference line MRL mode, and is used to restrict the value of the flag used to indicate whether the current sequence allows the use of the multiple reference line MRL mode.
[0178] The third flag may be recorded as sps_tmrl_enabled_flag, and the GCI of the third flag may be recorded as gci_no_mrl_constraints_flag, which indicates whether sps_tmrl_enabled_flag and sps_mrl_enabled_flag are subject to restrictions. For example, when the value of gci_no_tmrl_constraint_flag is 1, the values of sps_tmrl_enabled_flag and sps_mrl_enabled_flag must both be equal to 0. Otherwise, no restrictions are imposed on the values of sps_tmrl_enabled_flag and sps_mrl_enabled_flag.
[0179] The decoder decodes the codestream to obtain gci_no_mrl_constraints_flag. If gci_no_mrl_constraints_flag indicates that sps_tmrl_enabled_flag and sps_mrl_enabled_flag are not restricted (for example, the value of gci_no_mrl_constraints_flag is 0), the decoder decodes the codestream to obtain sps_tmrl_enabled_flag, and determines whether the current sequence is allowed to use the TMRL mode based on the value of sps_tmrl_enabled_flag. For example, if the value of sps_tmrl_enabled_flag is 1, it indicates that the current sequence is allowed to use the TMRL mode, and if the value of sps_tmrl_enabled_flag is 0, it indicates that the current sequence is not allowed to use the TMRL mode.
[0180] In some embodiments, the decoder obtains a sixth identifier; if the sixth identifier indicates that the current sequence allows the use of a template-based mode, the decoder obtains the third identifier.
[0181] Exemplarily, the decoder decodes the code stream to obtain the sixth identifier; if the sixth identifier indicates that the current sequence allows the use of the template-based mode, the decoder decodes the code stream to obtain the third identifier.
[0182] Exemplarily, the third flag may be recorded as sps_tmrl_enabled_flag, and the sixth flag may be recorded as sps_tm_enabled_flag.
[0183] The decoder decodes the bitstream to obtain sps_tm_enabled_flag; if sps_tm_enabled_flag indicates that the current sequence allows the use of the template-based mode (for example, the value of sps_tm_enabled_flag is 1), the decoder decodes the bitstream to obtain sps_tmrl_enabled_flag, and determines whether the current sequence allows the use of the TMRL mode based on the value of sps_tmrl_enabled_flag. For example, when the value of sps_tmrl_enabled_flag is 1, it indicates that the current sequence allows the use of the TMRL mode, and when the value of sps_tmrl_enabled_flag is 0, it indicates that the current sequence does not allow the use of the TMRL mode.
[0184] In some embodiments, the decoder obtains the seventh identifier; if the seventh identifier indicates that there is no restriction on the value of the sixth identifier, the decoder obtains the sixth identifier.
[0185] Exemplarily, the seventh identifier is general constraints information (GCI) of the sixth identifier, and is used to restrict a value of the sixth identifier.
[0186] The third flag may be recorded as sps_tmrl_enabled_flag, the sixth flag may be recorded as sps_tm_enabled_flag, and the seventh flag may be recorded as gci_no_tm_constraints_flag, where gci_no_tm_constraints_flag indicates whether sps_tm_enabled_flag is restricted. For example, when the value of gci_no_tm_constraints_flag is 1, the value of sps_tm_enabled_flag must be 0; otherwise, no restriction is imposed on the value of sps_tm_enabled_flag.
[0187] The decoder decodes the codestream to obtain gci_no_tm_constraints_flag. If gci_no_tm_constraints_flag indicates that the value of sps_tm_enabled_flag is not restricted (for example, the value of gci_no_tm_constraints_flag is 0), the decoder decodes the codestream to obtain sps_tm_enabled_flag. If sps_tm_enabled_flag indicates that the current sequence allows the use of the template-based mode (for example, the value of sps_tm_enabled_flag is 1), the decoder decodes the codestream to obtain sps_tmrl_enabled_flag, and determines whether the current sequence allows the use of the TMRL mode based on the value of sps_tmrl_enabled_flag. For example, if the value of sps_tmrl_enabled_flag is 1, it indicates that the current sequence allows the use of the TMRL mode, and if the value of sps_tmrl_enabled_flag is 0, it indicates that the current sequence does not allow the use of the TMRL mode.
[0188] In some embodiments, S330 may include:
[0189] The decoder determines the reference row corresponding to the maximum number of rows as the reference row allowed to be used by the current block;
[0190] The decoder constructs a TMRL mode list based on the reference rows allowed to be used by the current block and the prediction modes allowed to be used by the current block;
[0191] The decoder obtains the first index;
[0192] The decoder determines the reference row and prediction mode included in the combination indicated by the first index in the TMRL mode list as the reference row used by the current block and the prediction mode used by the current block.
[0193] Exemplarily, the decoder may determine the reference row in the reference row set corresponding to the maximum reference row as the reference row allowed to be used by the current block based on the correspondence between at least one row number and at least one reference row set; wherein the at least one row number includes the maximum row number.
[0194] For example, if the maximum number of rows ranges from 0 to 5, then:
[0195] When the maximum number of rows is 0, the reference row allowed for the current block is empty. When the maximum number of rows is 1, the reference row allowed for the current block is reference row 1. When the maximum number of rows is 2, the reference rows allowed for the current block may include reference rows 1 and 3. When the maximum number of rows is 3, the reference rows allowed for the current block may include reference rows 1, 3, and 5. When the maximum number of rows is 4, the reference rows allowed for the current block may include reference rows 1, 3, 5, and 7. When the maximum number of rows is 5, the reference rows allowed for the current block may include reference rows 1, 3, 5, 7, and 12.
[0196] Of course, in other alternative embodiments, there may be other corresponding relationships between the maximum number of rows and the reference rows allowed to be used by the current block, and this application does not make any specific limitations on this.
[0197] Exemplarily, the decoder decodes the code stream to obtain the first index.
[0198] In some embodiments, for each of at least one combination obtained by the reference line allowed to be used by the current block and the prediction mode allowed to be used by the current block, the decoder predicts the template of the current block based on the reference line of each combination and uses the prediction mode of each combination to obtain the prediction cost of each combination; the decoder sorts the at least one combination in order of small to large prediction cost based on the prediction cost of each combination to obtain a combination sorting; the decoder determines the combination that is at the front of the middle position in the combination sorting as the combination in the TMRL mode list.
[0199] For example, assuming that the reference lines allowed for the current block include {0, 1, 3, 5, 7, 12} and the prediction modes allowed for the current block include 10 prediction modes, there are 50 possible combinations of reference lines and prediction modes allowed for the current block. The decoder performs prediction on the template area of the current block (i.e., predicts the template of the current block) based on the reference lines included in each of the 50 combinations and the prediction modes included in each combination. The decoder calculates the difference between the predicted result of the template of the current block and the reconstructed result of the template of the current block under each combination. For example, the difference value can be the sum of absolute differences (SAD). After obtaining the 50 SADs, the decoder can sort the 50 SADs from small to large and select the smaller combinations as the TMRL mode list. The number of combinations selected by the decoder can be determined based on the length of the TMRL mode list.
[0200] In some embodiments, the method 300 may further include:
[0201] If the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode.
[0202] Exemplarily, if the maximum number of rows is greater than a first preset threshold (for example, the first preset threshold is 1), the decoder decodes the code stream to obtain sps_tmrl_enabled_flag; if sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1), and the current block does not use the TIMD mode, the decoder decodes the code stream to obtain cu_tmrl_flag; if cu_tmrl_flag indicates that the current block does not use the TMRL mode (for example, the value of cu_tmrl_flag is 0), the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode.
[0203] Exemplarily, the decoder decodes the code stream to obtain sps_tmrl_enabled_flag. If sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1), and the maximum number of rows is greater than a first preset threshold (for example, the first preset threshold is 1), and the current block does not use the TIMD mode, the decoder decodes the code stream to obtain cu_tmrl_flag. If cu_tmrl_flag indicates that the current block does not use the TMRL mode (for example, the value of cu_tmrl_flag is 0), the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode.
[0204] Of course, in other alternative embodiments, when the decoder determines that the current sequence does not use the TMRL mode or the current block does not meet the conditions for using the TMRL mode, it determines that the current block does not use the TMRL mode. In this case, the decoder can directly determine the reference row used by the current block based on whether the current block uses the TIMD mode. In other words, when the decoder determines that the current sequence does not use the TMRL mode or the current block does not meet the conditions for using the TMRL mode, it can skip decoding cu_tmrl_flag and directly determine the reference row used by the current block based on whether the current block uses the TIMD mode. For example, if sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs does not use the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 0), the decoder can skip decoding cu_tmrl_flag and directly determine the reference row used by the current block based on whether the current block uses the TIMD mode. For another example, if the current block uses the TIMD mode, the decoder may skip decoding cu_tmrl_flag, and directly decode the code stream to determine the reference row used when the current block uses the TIMD mode.
[0205] Exemplarily, if the current block uses the TIMD mode, the decoder decodes the code stream to determine the reference row used when the current block uses the TIMD mode; if the current block does not use the TIMD mode, the decoder decodes the code stream to determine the reference row used when the current block does not use the TIMD mode.
[0206] In some embodiments, if the current block uses the TIMD mode, the decoder obtains a second index; if the current block does not use the TIMD mode, the decoder obtains a third index; wherein the second index indicates the reference row used when the current block uses the TIMD mode; and the third index indicates the reference row used when the current block does not use the TIMD mode.
[0207] Exemplarily, the second index may be the multiRefIdx shown in Table 3, which will not be described here to avoid repetition.
[0208] Exemplarily, the third index may be the multiRefIdx shown in Table 2, which will not be described again here to avoid repetition.
[0209] In some embodiments, the decoder obtains an eighth identifier; if the eighth identifier indicates that the current sequence to which the current block belongs allows the use of the MRL mode and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode.
[0210] Exemplarily, the decoder decodes the code stream to obtain the eighth identifier; if the eighth identifier indicates that the current sequence to which the current block belongs allows the use of the MRL mode and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode.
[0211] Exemplarily, the eighth flag may be recorded as sps_mrl_enabled_flag.
[0212] The decoder decodes the code stream to obtain sps_mrl_enabled_flag; if sps_mrl_enabled_flag indicates that the current sequence to which the current block belongs allows the use of the MRL mode (for example, the value of sps_mrl_enabled_flag is 1), and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode.
[0213] In some embodiments, if the maximum number of rows is greater than or equal to a second preset threshold, the decoder acquires an eighth identifier.
[0214] Exemplarily, the second preset threshold may be a threshold predefined by a protocol.
[0215] Exemplarily, the second preset threshold may be 0 or another value, which is not specifically limited in this application.
[0216] If the maximum number of rows is greater than or equal to the second preset threshold, the decoder decodes the code stream to obtain the sps_mrl_enabled_flag; if the sps_mrl_enabled_flag indicates that the current sequence to which the current block belongs allows the use of the MRL mode (for example, the value of sps_mrl_enabled_flag is 1), and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode.
[0217] In some embodiments, if the eighth identifier indicates that the current sequence allows the use of a multi-reference row MRL mode, and the maximum number of rows is greater than or equal to a second preset threshold, and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode.
[0218] The decoder decodes the code stream to obtain sps_mrl_enabled_flag; if sps_mrl_enabled_flag indicates that the current sequence to which the current block belongs allows the use of the MRL mode (for example, the value of sps_mrl_enabled_flag is 1), and the maximum number of rows is greater than or equal to the second preset threshold, and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode.
[0219] The following is an exemplary description of the sequence level identifiers involved in this application. In the following tables, the syntax elements in bold are syntax elements that can be obtained based on bitstream decoding.
[0220] 1.sps_mrl_enabled_flag.
[0221] Table 5
[0222] As shown in Table 5, the sequence parameter set (seq_parameter_set_rbsp) may include sps_mrl_enabled_flag. sps_mrl_enabled_flag controls whether to enable MRL for the current sequence. For example, when sps_mrl_enabled_flag is 1, MRL is enabled for the current sequence; when sps_mrl_enabled_flag is 0, MRL is not enabled for the current sequence.
[0223] 2. sps_tmrl_enabled_flag.
[0224] Table 6
[0225] As shown in Table 6, the sequence parameter set (seq_parameter_set_rbsp) includes sps_mrl_enabled_flag and sps_tmrl_enabled_flag.
[0226] sps_mrl_enabled_flag controls whether to enable MRL for the current sequence. For example, when sps_mrl_enabled_flag is 1, MRL is enabled for the current sequence; when sps_mrl_enabled_flag is 0, MRL is not enabled for the current sequence.
[0227] sps_tmrl_enabled_flag indicates whether the current sequence is allowed to use the TMRL mode. For example, when sps_tmrl_enabled_flag is 1, it indicates that the current sequence is allowed to use the TMRL mode; when sps_tmrl_enabled_flag is 0, it indicates that the current sequence is not allowed to use the TMRL mode.
[0228] Furthermore, there is a dependency relationship between sps_mrl_enabled_flag and sps_tmrl_enabled_flag, that is, if the current sequence allows the use of the MRL mode (for example, sps_mrl_enabled_flag is 1), the decoder decodes the code stream to obtain sps_tmrl_enabled_flag.
[0229] Table 7
[0230] As shown in Table 7, there may be no dependency relationship between sps_mrl_enabled_flag and sps_tmrl_enabled_flag, that is, the decoder decodes the bitstream to obtain sps_mrl_enabled_flag and sps_tmrl_enabled_flag.
[0231] When sps_mrl_enabled_flag and sps_tmrl_enabled_flag are introduced, the syntax elements at the CU level change as shown in Table 8 below:
[0232] Table 8
[0233] As shown in Table 8, when sps_mrl_enabled_flag and sps_tmrl_enabled_flag are introduced, if the current block is not the first row of the current CTU, the current block does not use the DIMD mode, the current block does not use the BDPCM mode, and the current block is a luminance block, the decoder continues to decode the code stream based on sps_tmrl_enabled_flag.
[0234] In this embodiment, sps_tmrl_enabled_flag is used to switch between TMRL and MRL.
[0235] If sps_tmrl_enabled_flag indicates that the current sequence allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1) and the current block does not use the TIMD mode, the decoder decodes the bitstream to obtain cu_tmrl_flag. If cu_tmrl_flag indicates that the current block uses the TMRL mode (for example, cu_tmrl_flag is 1), the decoder decodes the bitstream to obtain tmrl_list_idx, and determines the reference row and prediction mode included in the combination indicated by tmrl_list_idx in the TMRL mode list as the reference row and prediction mode used by the current block. Then, based on the reference row used by the current block, the current block is predicted using the prediction mode used by the current block to obtain the predicted block.
[0236] If sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs does not allow the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 0), or the current block uses the TIMD mode, or cu_tmrl_flag indicates that the current block does not use the TMRL mode (for example, cu_tmrl_flag is 0); further, if sps_mrl_enabled_flag indicates that the current sequence allows the use of the MRL mode (for example, the value of sps_mrl_enabled_flag is 1), and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode. For example, if the current block uses the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block uses the TIMD mode; if the current block does not use the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block does not use the TIMD mode.
[0237] 3. gci_no_mrl_constraint_flag and gci_no_tmrl_constraint_flag.
[0238] Considering that the GCI includes a series of flags that are used to restrict the presence of certain sequence-level flags in the codestream, when the GCI of a specific flag is greater than 0, it means that the portion of the codestream with that specific flag is specifically restricted. For example, when the GCI is greater than 0, a certain coding tool is restricted in the codestream, while when it is equal to 0, it means that no special restrictions are imposed on the portion of the codestream (such as a certain coding tool). For example, the standard text contains the gci_no_mrl_constraint_flag to indicate whether the sps_mrl_enabled_flag is restricted. When gci_no_mrl_constraint_flag is equal to 1, the sps_mrl_enabled_flag must be equal to 0. Otherwise, no restrictions are imposed on the value of the sps_mrl_enabled_flag.
[0239] Due to the introduction of the TMRL tool, it has been enhanced for MRL. In view of this, in the use of TMRL and MRL, this application introduces a sequence-level identifier sps_tmrl_enabled_flag for switching between them. In this case, the GCI can also be adjusted accordingly. For example, when the decoding of sps_tmrl_enabled_flag does not depend on the value of sps_mrl_enabled_flag, a tmrl GCI identifier can be added. For example, adjustments can be made as shown in Table 9 below.
[0240] Table 9
[0241] As shown in Table 9, gci_no_mrl_constraint_flag indicates whether sps_mrl_enabled_flag is restricted. When gci_no_mrl_constraint_flag is equal to 1, sps_mrl_enabled_flag must be equal to 0. Otherwise, no restriction is imposed on the value of sps_mrl_enabled_flag.
[0242] In addition, a gci_no_tmrl_constraint_flag may be added to restrict the value of sps_tmrl_enabled_flag. gci_no_tmrl_constraint_flag indicates whether sps_tmrl_enabled_flag is restricted. When gci_no_tmrl_constraint_flag is equal to 1, sps_tmrl_enabled_flag must be equal to 0. Otherwise, no restriction is imposed on the value of sps_tmrl_enabled_flag.
[0243] Of course, in other alternative embodiments, when the decoding of sps_tmrl_enabled_flag depends on the value of sps_mrl_enabled_flag, gci_no_mrl_constraints_flag may also be used to control both sps_mrl_enabled_flag and sps_tmrl_enabled_flag. This application does not make any specific limitation on this.
[0244] 4. sps_tm_enabled_flag and gci_no_tm_constraints_flag.
[0245] Taking into account that there are a large number of template-based technologies in ECM, for example, DIMD in the prediction mode, and TIMD also use templates for prediction tools. This application can also introduce a unified identifier to control all template-based technologies. That is, a sequence-level identifier sps_tm_enabled_flag can be introduced to control whether the template-based encoding and decoding tool is enabled at the sequence level, and there can also be a gci_no_tm_constraints_flag identifier to control whether restrictions are imposed on the sps_tm_enabled_flag in the code stream. If gci_no_tm_constraints_flag is equal to 1, sps_tm_enabled_flag must be equal to 0, otherwise, no restrictions will be imposed on the value of sps_tm_enabled_flag.
[0246] In this case, decoding of sps_tmrl_enabled_flag depends on the value of sps_tm_enabled_flag.
[0247] Table 10
[0248] As shown in Table 10, the decoder decodes the code stream to obtain gci_no_tm_constraints_flag. If gci_no_tm_constraints_flag is equal to 1, sps_tm_enabled_flag must be equal to 0. Otherwise, no restriction is imposed on the value of sps_tm_enabled_flag.
[0249] Table 11
[0250] As shown in Table 11, the decoder decodes the code stream to obtain sps_tm_enabled_flag and sps_mrl_enabled_flag. If the current sequence allows the use of the template-based mode (for example, sps_tm_enabled_flag is 1), the decoder decodes the code stream to obtain sps_tmrl_enabled_flag.
[0251] Table 12
[0252] As shown in Table 12, the decoder decodes the code stream to obtain sps_tm_enabled_flag and sps_mrl_enabled_flag. If the current sequence allows the use of the template-based mode (for example, sps_tm_enabled_flag is 1) and the current sequence allows the use of the MRL mode (for example, sps_mrl_enabled_flag is 1), the decoder decodes the code stream to obtain sps_tmrl_enabled_flag.
[0253] Since the MRL tool is enabled, it means that non-adjacent reference lines are allowed to be used. Therefore, the sps_mrl_enabled_flag can be expanded to indicate the number of non-adjacent reference lines that can be used (i.e., the maximum number of lines mentioned above). After the maximum number of lines is introduced, the implementation of CU-level syntax elements under different sequence parameter sets needs to be adjusted accordingly. The following is an exemplary description of the adjustment method.
[0254] Method 1:
[0255] The sequence parameter set (seq_parameter_set_rbsp) may be as shown in Table 13 below:
[0256] Table 13
[0257] As shown in Table 13, the sequence parameter set includes sps_mrl_enabled_flag and sps_tmrl_enabled_flag.
[0258] The value range of sps_mrl_enabled_flag is 0 to x (x is a positive integer). When the value of sps_mrl_enabled_flag is 0, it indicates that sequence-level MRL is disabled. When the value of sps_mrl_enabled_flag is 1 to x, it indicates that the current sequence uses the MRL mode and indicates the maximum number of rows. For example, when the value of sps_mrl_enabled_flag is i (1≤i≤x), it indicates that the current sequence uses the MRL mode and indicates that the maximum number of rows is determined by i. If sps_mrl_enabled_flag does not exist in the codestream, sps_mrl_enabled_flag defaults to 0.
[0259] For example, if the maximum value of sps_mrl_enabled_flag is 5, then:
[0260] When the sps_mrl_enabled_flag is 0, it indicates that the current sequence does not use the MRL mode. When the sps_mrl_enabled_flag is 1, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 1, and the reference row allowed to be used by the current block is reference row 1. When the sps_mrl_enabled_flag is 2, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 2, and the reference rows allowed to be used by the current block include reference rows 1 and 3. When the sps_mrl_enabled_flag is 3, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 3, and the reference rows allowed to be used by the current block include reference rows 1, 3, and 5. When the sps_mrl_enabled_flag is 4, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 4, and the reference rows allowed to be used by the current block include reference rows 1, 3, 5, and 7. When the sps_mrl_enabled_flag is 5, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 5, and the reference rows allowed to be used by the current block include reference rows 1, 3, 5, 7, and 12.
[0261] sps_tmrl_enabled_flag indicates whether the current sequence is allowed to use the TMRL mode. For example, when sps_tmrl_enabled_flag is 1, it indicates that the current sequence is allowed to use the TMRL mode; when sps_tmrl_enabled_flag is 0, it indicates that the current sequence is not allowed to use the TMRL mode. If sps_tmrl_enabled_flag does not exist in the codestream, sps_tmrl_enabled_flag defaults to 0.
[0262] There is a dependency relationship between sps_mrl_enabled_flag and sps_tmrl_enabled_flag, that is, if sps_mrl_enabled_flag>n (for example, n is 1), the decoder decodes the code stream to obtain sps_tmrl_enabled_flag.
[0263] Based on the sequence parameter set shown in Table 13, the implementation of the CU-level syntax elements can be as shown in Table 14 below:
[0264] Table 14
[0265] As shown in Table 14, when the syntax element with the maximum number of lines is introduced, if the current block is not the first line of the current CTU, the current block does not use the DIMD mode, the current block does not use the BDPCM mode, and the current block is a luminance block, the decoder continues to decode the code stream based on sps_tmrl_enabled_flag.
[0266] In this embodiment, sps_tmrl_enabled_flag is used to switch between TMRL and MRL.
[0267] If sps_tmrl_enabled_flag indicates that the current sequence allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1) and the current block does not use the TIMD mode, the decoder decodes the bitstream to obtain cu_tmrl_flag. If cu_tmrl_flag indicates that the current block uses the TMRL mode (for example, cu_tmrl_flag is 1), the decoder decodes the bitstream to obtain tmrl_list_idx, and determines the reference row and prediction mode included in the combination indicated by tmrl_list_idx in the TMRL mode list as the reference row and prediction mode used by the current block. Then, based on the reference row used by the current block, the current block is predicted using the prediction mode used by the current block to obtain the predicted block.
[0268] If sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs does not allow the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 0), or the current block uses the TIMD mode, or cu_tmrl_flag indicates that the current block does not use the TMRL mode (for example, cu_tmrl_flag is 0); further, if sps_mrl_enabled_flag indicates that the current sequence allows the use of the MRL mode (for example, the value of sps_mrl_enabled_flag is 1), and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode. For example, if the current block uses the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block uses the TIMD mode; if the current block does not use the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block does not use the TIMD mode.
[0269] Method 2:
[0270] The sequence parameter set (seq_parameter_set_rbsp) may be as shown in Table 15 below:
[0271] Table 15
[0272] As shown in Table 13, the sequence parameter set includes sps_mrl_enabled_flag and sps_tmrl_enabled_flag.
[0273] The value range of sps_mrl_enabled_flag is 0 to x (x is a positive integer). When sps_mrl_enabled_flag is 0, it indicates that sequence-level MRL is disabled. When sps_mrl_enabled_flag is 1 to x, it indicates that the current sequence uses the MRL mode and indicates the maximum number of rows. For example, when sps_mrl_enabled_flag is i (1 ≤ i ≤ x), it indicates that the current sequence uses the MRL mode and indicates that the maximum number of rows is determined by i. If sps_mrl_enabled_flag does not exist in the codestream, sps_mrl_enabled_flag defaults to 0.
[0274] For example, if the maximum value of sps_mrl_enabled_flag is 5, then:
[0275] When the sps_mrl_enabled_flag is 0, it indicates that the current sequence does not use the MRL mode. When the sps_mrl_enabled_flag is 1, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 1, and the reference row allowed to be used by the current block is reference row 1. When the sps_mrl_enabled_flag is 2, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 2, and the reference rows allowed to be used by the current block include reference rows 1 and 3. When the sps_mrl_enabled_flag is 3, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 3, and the reference rows allowed to be used by the current block include reference rows 1, 3, and 5. When the sps_mrl_enabled_flag is 4, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 4, and the reference rows allowed to be used by the current block include reference rows 1, 3, 5, and 7. When the sps_mrl_enabled_flag is 5, it indicates the following: the current sequence uses the MRL mode, the maximum number of rows is 5, and the reference rows allowed to be used by the current block include reference rows 1, 3, 5, 7, and 12.
[0276] sps_tmrl_enabled_flag indicates whether the current sequence is allowed to use the TMRL mode. For example, when sps_tmrl_enabled_flag is 1, it indicates that the current sequence is allowed to use the TMRL mode; when sps_tmrl_enabled_flag is 0, it indicates that the current sequence is not allowed to use the TMRL mode. If sps_tmrl_enabled_flag does not exist in the codestream, sps_tmrl_enabled_flag defaults to 0.
[0277] There is no dependency between sps_mrl_enabled_flag and sps_tmrl_enabled_flag, that is, the decoder decodes the code stream to obtain sps_mrl_enabled_flag and sps_tmrl_enabled_flag.
[0278] Based on the sequence parameter set shown in Table 15, the implementation of the CU-level syntax elements can be as shown in Table 16 below:
[0279] Table 16
[0280] As shown in Table 16, when the syntax element with the maximum number of lines is introduced, if the current block is not the first line of the current CTU, the current block does not use the DIMD mode, the current block does not use the BDPCM mode, and the current block is a luminance block, the decoder continues to decode the code stream based on sps_tmrl_enabled_flag.
[0281] In this embodiment, sps_tmrl_enabled_flag is used to switch between TMRL and MRL.
[0282] If sps_tmrl_enabled_flag indicates that the current sequence allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1), the current block does not use the TIMD mode, and num_non_adjacent_ref_line>n (for example, n is 1), the decoder decodes the bitstream to obtain cu_tmrl_flag. If cu_tmrl_flag indicates that the current block uses the TMRL mode (for example, cu_tmrl_flag is 1), the decoder decodes the bitstream to obtain tmrl_list_idx, and determines the reference row and prediction mode included in the combination indicated by tmrl_list_idx in the TMRL mode list as the reference row and prediction mode used by the current block. Then, based on the reference row used by the current block, the current block is predicted using the prediction mode used by the current block to obtain the predicted block.
[0283] The num_non_adjacent_ref_line variable is equal to the value of sps_mrl_enabled_flag, which is used to indicate the maximum number of lines.
[0284] If sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs does not allow the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 0), or the current block uses the TIMD mode, or cu_tmrl_flag indicates that the current block does not use the TMRL mode (for example, cu_tmrl_flag is 0); further, if sps_mrl_enabled_flag indicates that the current sequence allows the use of the MRL mode (for example, the value of sps_mrl_enabled_flag is 1), and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode. For example, if the current block uses the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block uses the TIMD mode; if the current block does not use the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block does not use the TIMD mode.
[0285] Method 3:
[0286] The sequence parameter set (seq_parameter_set_rbsp) may be as shown in Table 17 below:
[0287] Table 17
[0288] As shown in Table 17, sps_mrl_enabled_flag is still only a sequence-level flag used to control whether to enable MRL for the current sequence. For example, when sps_mrl_enabled_flag is 1, MRL is enabled for the current sequence; when sps_mrl_enabled_flag is 0, MRL is not enabled for the current sequence.
[0289] num_non_adjacent_ref_line is used to indicate the maximum number of lines. For example, in ECM, the value of num_non_adjacent_ref_line can range from 0 to 5, which can be represented by the 3-bit fixed-length code "u(3)". Of course, when num_non_adjacent_ref_line does not exist in the codestream, the value of num_non_adjacent_ref_line can default to 0.
[0290] Based on the sequence parameter set shown in Table 17, the implementation of the CU-level syntax elements can be as shown in Table 18 below:
[0291] Table 18
[0292] As shown in Table 18, when the syntax element of the maximum number of lines is introduced, if the current block is not the first line of the current CTU, the current block does not use the DIMD mode, the current block does not use the BDPCM mode, and the current block is a luminance block, the decoder continues to decode the code stream based on sps_tmrl_enabled_flag.
[0293] In this embodiment, sps_tmrl_enabled_flag is used to switch between TMRL and MRL.
[0294] If sps_tmrl_enabled_flag indicates that the current sequence allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1), the current block does not use the TIMD mode, and num_non_adjacent_ref_line>n (for example, n is 1), the decoder decodes the bitstream to obtain cu_tmrl_flag. If cu_tmrl_flag indicates that the current block uses the TMRL mode (for example, cu_tmrl_flag is 1), the decoder decodes the bitstream to obtain tmrl_list_idx, and determines the reference row and prediction mode included in the combination indicated by tmrl_list_idx in the TMRL mode list as the reference row and prediction mode used by the current block. Then, based on the reference row used by the current block, the current block is predicted using the prediction mode used by the current block to obtain the predicted block.
[0295] The num_non_adjacent_ref_line variable is equal to the value of sps_mrl_enabled_flag, which is used to indicate the maximum number of lines.
[0296] If sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs does not allow the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 0), or the current block uses the TIMD mode, or cu_tmrl_flag indicates that the current block does not use the TMRL mode (for example, cu_tmrl_flag is 0); further, if sps_mrl_enabled_flag indicates that the current sequence allows the use of the MRL mode (for example, the value of sps_mrl_enabled_flag is 1), and num_non_adjacent_ref_line>m (for example, m is 0), and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode. For example, if the current block uses the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block uses the TIMD mode; if the current block does not use the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block does not use the TIMD mode.
[0297] Since there is no dependency between num_non_adjacent_ref_line and sps_mrl_enabled_flag, and between num_non_adjacent_ref_line and sps_tmrl_enabled_flag, it is necessary to determine whether to decode the image block level flag based on num_non_adjacent_ref_line.
[0298] Method 4:
[0299] The sequence parameter set (seq_parameter_set_rbsp) may be as shown in Table 19 below:
[0300] Table 19
[0301] As shown in Table 19, sps_mrl_enabled_flag is still just a sequence-level flag that controls whether MRL is enabled for the current sequence. For example, when sps_mrl_enabled_flag is 1, MRL is enabled for the current sequence; when sps_mrl_enabled_flag is 0, MRL is not enabled for the current sequence. A new syntax element, num_non_adjacent_ref_line, is introduced to indicate the maximum number of lines. For example, in ECM, the value of num_non_adjacent_ref_line can range from 0 to 5, and can be represented by the 3-bit fixed-length code "u(3)". Of course, when num_non_adjacent_ref_line does not exist in the codestream, the value of num_non_adjacent_ref_line can default to 0.
[0302] The sps_mrl_enabled_flag and sps_tmrl_enabled_flag can each be dependent on num_non_adjacent_ref_line. If num_non_adjacent_ref_line > m (for example, n is 0), the decoder decodes the bitstream to obtain sps_mrl_enabled_flag. If num_non_adjacent_ref_line > n (for example, n is 1), the decoder decodes the bitstream to obtain sps_tmrl_enabled_flag.
[0303] Based on the sequence parameter set shown in Table 19, the implementation of the CU-level syntax elements can be as shown in Table 20 below:
[0304] Table 20
[0305] As shown in Table 20, when the syntax element with the maximum number of lines is introduced, if the current block is not the first line of the current CTU, the current block does not use the DIMD mode, the current block does not use the BDPCM mode, and the current block is a luminance block, the decoder continues to decode the code stream based on sps_tmrl_enabled_flag.
[0306] In this embodiment, sps_tmrl_enabled_flag is used to switch between TMRL and MRL.
[0307] If sps_tmrl_enabled_flag indicates that the current sequence allows the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 1) and the current block does not use the TIMD mode, the decoder decodes the bitstream to obtain cu_tmrl_flag. If cu_tmrl_flag indicates that the current block uses the TMRL mode (for example, cu_tmrl_flag is 1), the decoder decodes the bitstream to obtain tmrl_list_idx, and determines the reference row and prediction mode included in the combination indicated by tmrl_list_idx in the TMRL mode list as the reference row and prediction mode used by the current block. Then, based on the reference row used by the current block, the current block is predicted using the prediction mode used by the current block to obtain the predicted block.
[0308] If sps_tmrl_enabled_flag indicates that the current sequence to which the current block belongs does not allow the use of the TMRL mode (for example, the value of sps_tmrl_enabled_flag is 0), or the current block uses the TIMD mode, or cu_tmrl_flag indicates that the current block does not use the TMRL mode (for example, cu_tmrl_flag is 0); further, if sps_mrl_enabled_flag indicates that the current sequence allows the use of the MRL mode (for example, the value of sps_mrl_enabled_flag is 1), and the current block does not use the TMRL mode, the decoder determines the reference row used by the current block based on whether the current block uses the TIMD mode. For example, if the current block uses the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block uses the TIMD mode; if the current block does not use the TIMD mode, the decoder decodes the code stream to obtain intra_luma_ref_idx, where intra_luma_ref_idx indicates the reference row used when the current block does not use the TIMD mode.
[0309] It should be understood that the implementation of the CU-level syntax elements under the sequence parameter set described above is only an example of the present application and should not be understood as a limitation to the present application. For example, in other alternative embodiments, the parsing conditions of cu_tmrl_flag, intra_luma_ref_idx or intra_luma_ref_idx can also be constructed according to actual needs based on the identifier in the sequence parameter set, and the present application does not make specific limitations on this. In addition, when the value of the identifier involved above is 1, it controls or indicates that the current sequence uses a certain technology or mode, and when the value of the identifier is 0, it controls or indicates that the current sequence does not use a certain technology or mode. However, in other alternative embodiments, it can also be that when the value of the identifier is 0, it controls or indicates that the current sequence uses a certain technology or mode, and when the value of the identifier is 1, it controls or indicates that the current sequence does not use a certain technology or mode. The present application does not make specific limitations on this.
[0310] The decoding method according to the embodiment of the present application is described in detail above from the perspective of a decoder. The encoding method according to the embodiment of the present application will be described below from the perspective of an encoder in conjunction with FIG. 7 .
[0311] FIG6 is a schematic flow chart of an encoding method 400 provided in an embodiment of the present application. It should be understood that the encoding method 400 can be executed by an encoder, for example, in the encoding framework 100 shown in FIG1 .
[0312] As shown in FIG6 , the encoding method 400 may include:
[0313] S410, determining the maximum number of reference rows allowed to be used by the current block;
[0314] S420: Encode the maximum number of rows.
[0315] In some embodiments, S410 may include:
[0316] encoding the first identifier;
[0317] Among them, when the value of the first identifier is a first value, it indicates that the current sequence to which the current block belongs does not use the multi-reference row MRL mode; when the value of the first identifier is a second value other than the first value, it indicates that the current sequence uses the MRL mode and indicates the maximum number of rows.
[0318] In some embodiments, S410 may include:
[0319] encoding the second identifier;
[0320] The second identifier indicates the maximum number of rows.
[0321] In some embodiments, the method 400 may further include:
[0322] If the maximum number of rows is greater than a first preset threshold, encoding the third identifier;
[0323] If the third flag indicates that the current sequence allows the use of the template-based multiple reference line (TMRL) mode, encoding the fourth flag;
[0324] The fourth flag indicates whether the current block uses the TMRL mode.
[0325] In some embodiments, if the third flag indicates that the current sequence allows the use of the TMRL mode and the current block does not use the template-based intra-frame mode to derive the TIMD mode, the fourth flag is encoded.
[0326] In some embodiments, the method 400 may further include:
[0327] encoding the third identifier;
[0328] If the third flag indicates that the current sequence allows the use of the template-based multiple reference line TMRL mode and the maximum number of lines is greater than the first preset threshold, encoding the fourth flag;
[0329] The fourth flag indicates whether the current block uses the TMRL mode.
[0330] In some embodiments, if the third identifier indicates that the current sequence allows the use of the TMRL mode, the maximum number of rows is greater than the first preset threshold, and the current block does not use the template-based intra-frame mode to derive the TIMD mode, the fourth identifier is encoded.
[0331] In some embodiments, if the current sequence allows the use of the multiple reference line (MRL) mode, the third flag is encoded.
[0332] In some embodiments, the fifth identifier is encoded; if the fifth identifier indicates that there is no restriction on the value of the third identifier, or the fifth identifier does not restrict the value of the identifier used to indicate whether the current sequence allows the use of the multi-reference line MRL mode, the third identifier is encoded.
[0333] In some embodiments, the sixth identifier is encoded; if the sixth identifier indicates that the current sequence allows the use of the template-based mode, the third identifier is encoded.
[0334] In some embodiments, the seventh identifier is encoded; if the seventh identifier indicates that there is no restriction on the value of the sixth identifier, the sixth identifier is encoded.
[0335] In some embodiments, the method 400 may further include:
[0336] Determining the reference row corresponding to the maximum number of rows as the reference row allowed to be used by the current block;
[0337] Constructing a TMRL mode list based on the reference rows allowed to be used by the current block and the prediction modes allowed to be used by the current block;
[0338] Encoding the first index;
[0339] The reference row and prediction mode included in the combination indicated by the first index in the TMRL mode list are the reference row used by the current block and the prediction mode used by the current block.
[0340] In some embodiments, for each of at least one combination obtained by the reference line allowed to be used by the current block and the prediction mode allowed to be used by the current block, based on the reference line of each combination, the template of the current block is predicted using the prediction mode of each combination to obtain the prediction cost of each combination; based on the prediction cost of each combination, the at least one combination is sorted in order of small to large prediction cost to obtain a combination ranking; the combination that is at the front of the combination ranking is determined as the combination in the TMRL mode list.
[0341] In some embodiments, the method 400 may further include:
[0342] If the current block does not use the template-based multiple reference line TMRL mode, the reference line used by the current block is determined based on whether the current block uses the template-based intra-frame mode to derive the TIMD mode.
[0343] In some embodiments, the method 400 may further include:
[0344] If the current block uses the TIMD mode, encoding the second index;
[0345] If the current block does not use the TIMD mode, encoding the third index;
[0346] The second index indicates a reference row used when the current block uses the TIMD mode; and the third index indicates a reference row used when the current block does not use the TIMD mode.
[0347] In some embodiments, the eighth identifier is encoded; if the eighth identifier indicates that the current sequence to which the current block belongs allows the use of the multi-reference row MRL mode and the current block does not use the TMRL mode, then the reference row used by the current block is determined based on whether the current block uses the TIMD mode.
[0348] In some embodiments, if the maximum number of rows is greater than or equal to a second preset threshold, the eighth identifier is encoded.
[0349] In some embodiments, if the eighth identifier indicates that the current sequence allows the use of a multi-reference row MRL mode, and the maximum number of rows is greater than or equal to a second preset threshold, and the current block does not use the TMRL mode, then the reference row used by the current block is determined based on whether the current block uses the TIMD mode.
[0350] It should be understood that the encoding method can be understood as the inverse process of the decoding method. Therefore, the specific scheme of the encoding method 400 can refer to the relevant content of the decoding method 300. For the sake of ease of description, this application will not go into details.
[0351] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the embodiments mentioned above. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all fall within the scope of protection of the present application. For example, the various specific technical features described in the specific embodiments mentioned above can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present application will no longer describe the various possible combinations separately. For another example, the various different embodiments of the present application can also be arbitrarily combined, as long as they do not violate the ideas of the present application, they should also be regarded as the contents disclosed in the present application. It should also be understood that in the various method embodiments of the present application, the size of the sequence numbers of the processes mentioned above does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0352] The method embodiment of the present application is described in detail above. The device embodiment of the present application is described in detail below with reference to FIG. 7 to FIG. 9 .
[0353] FIG7 is a schematic block diagram of a decoder 500 according to an embodiment of the present application.
[0354] As shown in FIG7 , the decoder 500 may include:
[0355] A first determining unit 510 is configured to determine a maximum number of reference rows allowed to be used by the current block;
[0356] A second determining unit 520 is configured to determine whether the current block uses a template-based multiple reference line TMRL mode based on the maximum number of lines;
[0357] A third determining unit 530 is configured to determine a reference row used by the current block and a prediction mode used by the current block if the current block uses the TMRL mode;
[0358] The prediction unit 540 is configured to predict the current block based on the reference row used by the current block and using the prediction mode used by the current block to obtain the predicted block.
[0359] In some embodiments, the first determining unit 510 is specifically configured to:
[0360] Get the first identifier;
[0361] Among them, when the value of the first identifier is a first value, it indicates that the current sequence to which the current block belongs does not use the multi-reference row MRL mode; when the value of the first identifier is a second value other than the first value, it indicates that the current sequence uses the MRL mode and indicates the maximum number of rows.
[0362] In some embodiments, the first determining unit 510 is specifically configured to:
[0363] Obtain a second identifier;
[0364] The number of rows indicated by the second identifier is determined as the maximum number of rows.
[0365] In some embodiments, the second determining unit 520 is specifically configured to:
[0366] If the maximum number of rows is greater than a first preset threshold, obtaining a third identifier;
[0367] If the third flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode, obtaining a fourth flag;
[0368] The fourth flag indicates whether the current block uses the TMRL mode.
[0369] In some embodiments, the second determining unit 520 is specifically configured to:
[0370] If the third flag indicates that the current sequence allows the use of the TMRL mode and the current block does not use the template-based intra-frame mode to derive the TIMD mode, the fourth flag is obtained.
[0371] In some embodiments, the second determining unit 520 is specifically configured to:
[0372] Get the third identifier;
[0373] If the third flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode and the maximum number of rows is greater than a first preset threshold, obtaining a fourth flag;
[0374] The fourth flag indicates whether the current block uses the TMRL mode.
[0375] In some embodiments, the second determining unit 520 is specifically configured to:
[0376] If the third flag indicates that the current sequence allows the use of the TMRL mode, the maximum number of rows is greater than the first preset threshold, and the current block does not use the template-based intra-frame mode to derive the TIMD mode, it is determined that the current block uses the TMRL mode.
[0377] In some embodiments, the second determining unit 520 is specifically configured to:
[0378] If the current sequence allows the use of the multiple reference line (MRL) mode, the third identifier is obtained.
[0379] In some embodiments, the second determining unit 520 is specifically configured to:
[0380] Get the fifth identifier;
[0381] If the fifth flag indicates that the value of the third flag is not restricted, or the fifth flag does not restrict the value of the flag used to indicate whether the current sequence allows the use of the multiple reference line MRL mode, the third flag is obtained.
[0382] In some embodiments, the second determining unit 520 is specifically configured to:
[0383] Get the sixth identifier;
[0384] If the sixth identifier indicates that the current sequence allows the use of the template-based mode, the third identifier is obtained.
[0385] In some embodiments, the second determining unit 520 is specifically configured to:
[0386] Get the seventh mark;
[0387] If the seventh flag indicates that there is no restriction on the value of the sixth flag, the sixth flag is obtained.
[0388] In some embodiments, the third determining unit 530 is specifically configured to:
[0389] Determining the reference row corresponding to the maximum number of rows as the reference row allowed to be used by the current block;
[0390] Constructing a TMRL mode list based on the reference rows allowed to be used by the current block and the prediction modes allowed to be used by the current block;
[0391] Get the first index;
[0392] The reference row and prediction mode included in the combination indicated by the first index in the TMRL mode list are determined as the reference row used by the current block and the prediction mode used by the current block.
[0393] In some embodiments, the third determining unit 530 is specifically configured to:
[0394] For each of at least one combination obtained from a reference row allowed to be used by the current block and a prediction mode allowed to be used by the current block, predicting a template of the current block based on the reference row of each combination and using the prediction mode of each combination to obtain a prediction cost for each combination;
[0395] Based on the predicted cost of each combination, the at least one combination is sorted in ascending order of predicted cost to obtain a combination sorting;
[0396] The combination that is ranked higher in the middle is determined as the combination in the TMRL pattern list.
[0397] In some embodiments, the prediction unit 540 is further configured to:
[0398] If the current block does not use the TMRL mode, the reference row used by the current block is determined based on whether the current block uses the template-based intra-frame mode to derive the TIMD mode.
[0399] In some embodiments, the prediction unit 540 is specifically configured to:
[0400] If the current block uses the TIMD mode, obtaining a second index;
[0401] If the current block does not use the TIMD mode, obtain the third index;
[0402] The second index indicates a reference row used when the current block uses the TIMD mode; and the third index indicates a reference row used when the current block does not use the TIMD mode.
[0403] In some embodiments, the prediction unit 540 is specifically configured to:
[0404] Get the eighth identifier;
[0405] If the eighth flag indicates that the current sequence to which the current block belongs allows the use of the multi-reference line MRL mode and the current block does not use the TMRL mode, the reference line used by the current block is determined based on whether the current block uses the TIMD mode.
[0406] In some embodiments, the prediction unit 540 is specifically configured to:
[0407] If the maximum number of rows is greater than or equal to the second preset threshold, an eighth flag is obtained.
[0408] In some embodiments, the prediction unit 540 is specifically configured to:
[0409] If the eighth flag indicates that the current sequence allows the use of multiple reference row MRL mode, and the maximum number of rows is greater than or equal to the second preset threshold, and the current block does not use the TMRL mode, then the reference row used by the current block is determined based on whether the current block uses the TIMD mode.
[0410] FIG8 is a schematic block diagram of an encoder 600 according to an embodiment of the present application.
[0411] As shown in FIG8 , the encoder 600 may include:
[0412] A determining unit 610 is configured to determine a maximum number of reference rows allowed to be used by the current block;
[0413] The encoding unit 620 is configured to encode the maximum number of rows.
[0414] In some embodiments, the determining unit 610 is specifically configured to:
[0415] encoding the first identifier;
[0416] Among them, when the value of the first identifier is a first value, it indicates that the current sequence to which the current block belongs does not use the multi-reference row MRL mode; when the value of the first identifier is a second value other than the first value, it indicates that the current sequence uses the MRL mode and indicates the maximum number of rows.
[0417] In some embodiments, the determining unit 610 is specifically configured to:
[0418] encoding the second identifier;
[0419] The second identifier indicates the maximum number of rows.
[0420] In some embodiments, the encoding unit 620 is further configured to:
[0421] If the maximum number of rows is greater than a first preset threshold, encoding the third identifier;
[0422] If the third flag indicates that the current sequence allows the use of the template-based multiple reference line (TMRL) mode, encoding the fourth flag;
[0423] The fourth flag indicates whether the current block uses the TMRL mode.
[0424] In some embodiments, the encoding unit 620 is specifically configured to:
[0425] If the third flag indicates that the current sequence allows the use of the TMRL mode and the current block does not use the template-based intra-frame mode to derive the TIMD mode, the fourth flag is encoded.
[0426] In some embodiments, the encoding unit 620 is further configured to:
[0427] encoding the third identifier;
[0428] If the third flag indicates that the current sequence allows the use of the template-based multiple reference line TMRL mode and the maximum number of lines is greater than the first preset threshold, encoding the fourth flag;
[0429] The fourth flag indicates whether the current block uses the TMRL mode.
[0430] In some embodiments, the encoding unit 620 is specifically configured to:
[0431] If the third flag indicates that the current sequence allows the use of the TMRL mode, the maximum number of rows is greater than the first preset threshold, and the current block does not use the template-based intra-frame mode to derive the TIMD mode, the fourth flag is encoded.
[0432] In some embodiments, the encoding unit 620 is specifically configured to:
[0433] If the current sequence allows the use of the multiple reference line (MRL) mode, the third flag is encoded.
[0434] In some embodiments, the encoding unit 620 is specifically configured to:
[0435] encoding the fifth identifier;
[0436] If the fifth flag indicates that there is no restriction on the value of the third flag, or the fifth flag does not restrict the value of the flag used to indicate whether the current sequence allows the use of the multiple reference line MRL mode, the third flag is encoded.
[0437] In some embodiments, the encoding unit 620 is specifically configured to:
[0438] encoding the sixth identifier;
[0439] If the sixth flag indicates that the current sequence allows the use of the template-based mode, the third flag is encoded.
[0440] In some embodiments, the encoding unit 620 is specifically configured to:
[0441] Encoding the seventh identifier;
[0442] If the seventh identifier indicates that there is no restriction on the value of the sixth identifier, the sixth identifier is encoded.
[0443] In some embodiments, the encoding unit 620 is further configured to:
[0444] Determining the reference row corresponding to the maximum number of rows as the reference row allowed to be used by the current block;
[0445] Constructing a TMRL mode list based on the reference rows allowed to be used by the current block and the prediction modes allowed to be used by the current block;
[0446] Encoding the first index;
[0447] The reference row and prediction mode included in the combination indicated by the first index in the TMRL mode list are the reference row used by the current block and the prediction mode used by the current block.
[0448] In some embodiments, the encoding unit 620 is specifically configured to:
[0449] For each of at least one combination obtained from a reference row allowed to be used by the current block and a prediction mode allowed to be used by the current block, predicting a template of the current block based on the reference row of each combination and using the prediction mode of each combination to obtain a prediction cost for each combination;
[0450] Based on the predicted cost of each combination, the at least one combination is sorted in ascending order of predicted cost to obtain a combination sorting;
[0451] The combination that is ranked higher in the middle is determined as the combination in the TMRL pattern list.
[0452] In some embodiments, the determining unit 610 is further configured to:
[0453] If the current block does not use the template-based multiple reference line TMRL mode, the reference line used by the current block is determined based on whether the current block uses the template-based intra-frame mode to derive the TIMD mode.
[0454] In some embodiments, the encoding unit 620 is further configured to:
[0455] If the current block uses the TIMD mode, encoding the second index;
[0456] If the current block does not use the TIMD mode, encoding the third index;
[0457] The second index indicates a reference row used when the current block uses the TIMD mode; and the third index indicates a reference row used when the current block does not use the TIMD mode.
[0458] In some embodiments, the encoding unit 620 is specifically configured to:
[0459] Encode the eighth identifier; the determining unit 610 is specifically configured to:
[0460] If the eighth flag indicates that the current sequence to which the current block belongs allows the use of the multi-reference line MRL mode and the current block does not use the TMRL mode, the reference line used by the current block is determined based on whether the current block uses the TIMD mode.
[0461] In some embodiments, the determining unit 610 is further configured to:
[0462] If the maximum number of rows is greater than or equal to the second preset threshold, the eighth identifier is encoded.
[0463] In some embodiments, the determining unit 610 is further configured to:
[0464] If the eighth flag indicates that the current sequence allows the use of multiple reference row MRL mode, and the maximum number of rows is greater than or equal to the second preset threshold, and the current block does not use the TMRL mode, then the reference row used by the current block is determined based on whether the current block uses the TIMD mode.
[0465] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, the decoder 500 shown in Figure 7 may correspond to the corresponding subject in the method 300 for executing the embodiment of the present application, and the aforementioned and other operations and / or functions of the various units in the decoder 500 are respectively for implementing the corresponding processes in various methods such as the method 300. The encoder 600 shown in Figure 8 may correspond to the corresponding subject in the method 400 for executing the embodiment of the present application, that is, the aforementioned and other operations and / or functions of the various units in the encoder 600 are respectively for implementing the corresponding processes in various methods such as the method 400.
[0466] It should also be understood that the various units in the decoder 500 or encoder 600 involved in the embodiment of the present application can be separately or all merged into one or several other units to constitute, or some (some) units therein can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effect of the embodiment of the present application. The units involved in the above are divided based on logical functions. In practical applications, the function of a unit can also be realized by multiple units, or the function of multiple units is realized by one unit. In other embodiments of the present application, the decoder 500 or encoder 600 can also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, the decoder 500 or encoder 600 involved in the embodiment of the present application can be constructed by running a computer program (including program code) that can execute each step involved in the corresponding method on a general-purpose computing device including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), etc., to construct the decoder 500 or encoder 600 involved in the embodiment of the present application, and to implement the encoding method or decoding method of the embodiment of the present application. The computer program can be recorded on, for example, a computer-readable storage medium, loaded into an electronic device via the computer-readable storage medium, and run therein to implement the corresponding method of the embodiment of the present application.
[0467] In other words, the units mentioned above can be implemented in the form of hardware, can be implemented by instructions in the form of software, or can be implemented in the form of a combination of hardware and software. Specifically, the steps of the method embodiments in the embodiments of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software in a decoding processor. Optionally, the software can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps in the method embodiments mentioned above in combination with its hardware.
[0468] FIG9 is a schematic structural diagram of an electronic device 700 provided in an embodiment of the present application.
[0469] As shown in Figure 9, the electronic device 700 includes at least a processor 710 and a computer-readable storage medium 720. The processor 710 and the computer-readable storage medium 720 may be connected via a bus or other means. The computer-readable storage medium 720 is used to store a computer program 721, which includes computer instructions. The processor 710 is used to execute the computer instructions stored in the computer-readable storage medium 720. The processor 710 is the computing core and control core of the electronic device 700. It is suitable for implementing one or more computer instructions, and is specifically suitable for loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.
[0470] For example, the processor 710 may also be referred to as a central processing unit (CPU). The processor 710 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, discrete hardware components, and the like.
[0471] Exemplarily, the computer-readable storage medium 720 may be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory; optionally, it may also be at least one computer-readable storage medium located away from the aforementioned processor 710. Specifically, the computer-readable storage medium 720 includes, but is not limited to: volatile memory and / or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).
[0472] Exemplarily, the electronic device 700 may be an encoder or encoding framework involved in an embodiment of the present application; a first computer instruction is stored in the computer-readable storage medium 720; the processor 710 loads and executes the first computer instruction stored in the computer-readable storage medium 720 to implement the corresponding steps in the encoding method provided in an embodiment of the present application; in other words, the first computer instruction in the computer-readable storage medium 720 is loaded by the processor 710 and the corresponding steps are executed. To avoid repetition, they are not repeated here.
[0473] Exemplarily, the electronic device 700 may be a decoder or decoding framework involved in an embodiment of the present application; a second computer instruction is stored in the computer-readable storage medium 720; the processor 710 loads and executes the second computer instruction stored in the computer-readable storage medium 720 to implement the corresponding steps in the decoding method provided in an embodiment of the present application; in other words, the second computer instruction in the computer-readable storage medium 720 is loaded by the processor 710 and the corresponding steps are executed, which will not be repeated here to avoid repetition.
[0474] According to another aspect of the present application, the present application also provides a coding and decoding system, including the encoder and decoder mentioned above.
[0475] According to another aspect of the present application, the present application also provides a computer-readable storage medium (Memory), which is a memory device in the electronic device 700 for storing programs and data. For example, a computer-readable storage medium 720. It can be understood that the computer-readable storage medium 720 here can include both the built-in storage medium in the electronic device 700 and, of course, the extended storage medium supported by the electronic device 700. The computer-readable storage medium provides a storage space, which stores the operating system of the electronic device 700. In addition, one or more computer instructions suitable for being loaded and executed by the processor 710 are also stored in the storage space. These computer instructions can be one or more computer programs 721 (including program codes).
[0476] According to another aspect of the present application, a computer program product or computer program is provided. The computer program product or computer program includes computer instructions stored in a computer-readable storage medium. For example, computer program 721. In this case, data processing device 700 may be a computer, and processor 710 reads the computer instructions from computer-readable storage medium 720. Processor 710 executes the computer instructions, causing the computer to perform the encoding method or decoding method provided in the various optional embodiments mentioned above.
[0477] In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process of the embodiment of the present application is run in whole or in part or the functions of the embodiment of the present application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode.
[0478] Those skilled in the art will appreciate that the units and process steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0479] Finally, it should be noted that the above content is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.
Claims
1. A decoding method, characterized in that: include: Determine the maximum number of reference lines allowed for the current block; Based on the maximum number of rows, determining whether the current block uses a template-based multiple reference row TMRL mode; If the current block uses the TMRL mode, determining a reference row used by the current block and a prediction mode used by the current block; Based on the reference row used by the current block, the current block is predicted using the prediction mode used by the current block to obtain the predicted block.
2. The method according to claim 1, characterized in that The determining the maximum number of reference rows allowed to be used by the current block includes: Get the first identifier; When the value of the first identifier is a first value, it indicates that the current sequence to which the current block belongs does not use the multi-reference row MRL mode; when the value of the first identifier is a second value other than the first value, it indicates that the current sequence uses the MRL mode and indicates the maximum number of rows.
3. The method according to claim 1, characterized in that The determining the maximum number of reference rows allowed to be used by the current block includes: Obtaining a second identifier; The number of rows indicated by the second identifier is determined as the maximum number of rows.
4. The method according to any one of claims 1 to 3, characterized in that The determining, based on the maximum number of rows, whether the current block uses a template-based multi-reference row TMRL mode comprises: If the maximum number of rows is greater than a first preset threshold, obtaining a third identifier; If the third flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode, acquiring a fourth flag; The fourth flag indicates whether the current block uses the TMRL mode.
5. The method according to claim 4, characterized in that The obtaining of the fourth identifier includes: If the third flag indicates that the current sequence allows the use of the TMRL mode and the current block does not use the template-based intra-frame mode to derive the TIMD mode, the fourth flag is obtained.
6. The method according to any one of claims 1 to 3, characterized in that The determining, based on the maximum number of rows, whether the current block uses a template-based multi-reference row TMRL mode comprises: Get the third identifier; If the third flag indicates that the current sequence to which the current block belongs allows the use of the TMRL mode and the maximum number of rows is greater than a first preset threshold, acquiring a fourth flag; The fourth flag indicates whether the current block uses the TMRL mode.
7. The method according to claim 6, characterized in that The obtaining of the fourth identifier includes: If the third identifier indicates that the current sequence allows the use of the TMRL mode, and the maximum number of rows is greater than the first preset threshold, and the current block does not use the template-based intra-frame mode to derive the TIMD mode, it is determined that the current block uses the TMRL mode.
8. The method according to any one of claims 4 to 7, characterized in that The obtaining of the third identifier includes: If the current sequence allows the use of the multiple reference line (MRL) mode, the third identifier is obtained.
9. The method according to any one of claims 4 to 7, characterized in that The obtaining of the third identifier includes: Get the fifth identifier; If the fifth flag indicates that there is no restriction on the value of the third flag, or the fifth flag does not restrict the value of the flag used to indicate whether the current sequence allows the use of the multiple reference line MRL mode, the third flag is acquired.
10. The method according to any one of claims 4 to 9, characterized in that The obtaining of the third identifier includes: Get the sixth identifier; If the sixth identifier indicates that the current sequence allows the use of a template-based mode, the third identifier is acquired.
11. The method according to claim 10, characterized in that The obtaining of the sixth identifier includes: Get the seventh mark; If the seventh identifier indicates that there is no restriction on the value of the sixth identifier, the sixth identifier is obtained.
12. The method according to any one of claims 1 to 11, characterized in that The determining the reference row used by the current block and the prediction mode used by the current block includes: Determine the reference row corresponding to the maximum number of rows as the reference row allowed to be used by the current block; Constructing a TMRL mode list based on the reference rows allowed to be used by the current block and the prediction modes allowed to be used by the current block; Get the first index; The reference row and prediction mode included in the combination indicated by the first index in the TMRL mode list are determined as the reference row used by the current block and the prediction mode used by the current block.
13. The method according to claim 12, characterized in that The constructing a TMRL mode list based on the reference row allowed to be used by the current block and the prediction mode allowed to be used by the current block comprises: For each of at least one combination obtained by the reference row allowed to be used by the current block and the prediction mode allowed to be used by the current block, based on the reference row of each combination, using the prediction mode of each combination to predict the template of the current block, and obtain a prediction cost of each combination; Based on the predicted cost of each combination, the at least one combination is sorted in order of predicted cost from small to large to obtain a combination sorting; The combination that is ranked at the front in the middle is determined as the combination in the TMRL pattern list.
14. The method according to any one of claims 1 to 13, characterized in that The method further comprises: If the current block does not use the TMRL mode, a reference row used by the current block is determined based on whether the current block uses a template-based intra-mode to derive a TIMD mode.
15. The method according to claim 14, characterized in that The step of determining a reference row used by the current block based on whether the current block uses a template-based intra mode to derive a TIMD mode comprises: If the current block uses the TIMD mode, obtaining a second index; If the current block does not use the TIMD mode, obtaining a third index; The second index indicates a reference row used when the current block uses the TIMD mode; and the third index indicates a reference row used when the current block does not use the TIMD mode.
16. The method according to claim 14 or 15, characterized in that If the current block does not use the TMRL mode, determining the reference row used by the current block based on whether the current block uses a template-based intra mode to derive a TIMD mode includes: Get the eighth identifier; If the eighth flag indicates that the current sequence to which the current block belongs allows the use of the multi-reference line MRL mode and the current block does not use the TMRL mode, the reference row used by the current block is determined based on whether the current block uses the TIMD mode.
17. The method according to claim 16, characterized in that The obtaining of the eighth identification includes: If the maximum number of rows is greater than or equal to the second preset threshold, an eighth flag is obtained.
18. The method according to claim 16, characterized in that The determining, based on whether the current block uses the TIMD mode, a reference row used by the current block comprises: If the eighth flag indicates that the current sequence allows the use of multiple reference row MRL mode, and the maximum number of rows is greater than or equal to the second preset threshold, and the current block does not use the TMRL mode, then the reference row used by the current block is determined based on whether the current block uses the TIMD mode.
19. A coding method, characterized in that: include: Determine the maximum number of reference lines allowed for the current block; The maximum number of rows is encoded.
20. The method according to claim 19, characterized in that The encoding of the maximum number of rows comprises: encoding the first identifier; When the value of the first identifier is a first value, it indicates that the current sequence to which the current block belongs does not use the multi-reference row MRL mode; when the value of the first identifier is a second value other than the first value, it indicates that the current sequence uses the MRL mode and indicates the maximum number of rows.
21. The method according to claim 19, characterized in that The encoding of the maximum number of rows comprises: encoding the second identifier; The second identifier indicates the maximum number of rows.
22. The method according to any one of claims 19 to 21, characterized in that The method further comprises: If the maximum number of rows is greater than a first preset threshold, encoding the third identifier; If the third flag indicates that the current sequence allows the use of a template-based multiple reference line TMRL mode, encoding a fourth flag; The fourth flag indicates whether the current block uses the TMRL mode.
23. The method according to claim 22, characterized in that The encoding of the fourth identifier includes: If the third flag indicates that the current sequence allows the use of the TMRL mode and the current block does not use the template-based intra-mode to derive the TIMD mode, the fourth flag is encoded.
24. The method according to any one of claims 19 to 21, characterized in that The method further comprises: encoding a third identifier; If the third flag indicates that the current sequence allows the use of a template-based multiple reference line TMRL mode, and the maximum number of lines is greater than a first preset threshold, encoding a fourth flag; The fourth flag indicates whether the current block uses the TMRL mode.
25. The method according to claim 24, characterized in that The encoding of the fourth identifier includes: If the third flag indicates that the current sequence allows the use of the TMRL mode, and the maximum number of rows is greater than the first preset threshold, and the current block does not use the template-based intra-frame mode to derive the TIMD mode, the fourth flag is encoded.
26. The method according to any one of claims 22 to 25, characterized in that The encoding of the third identifier includes: If the current sequence allows the use of the multiple reference line (MRL) mode, the third identifier is encoded.
27. The method according to any one of claims 22 to 25, characterized in that The encoding of the third identifier includes: encoding a fifth identifier; If the fifth flag indicates that there is no restriction on the value of the third flag, or the fifth flag does not restrict the value of the flag used to indicate whether the current sequence allows the use of the multiple reference line MRL mode, the third flag is encoded.
28. The method according to any one of claims 22 to 27, characterized in that The encoding of the third identifier includes: encoding the sixth identifier; If the sixth identifier indicates that the current sequence allows the use of a template-based mode, the third identifier is encoded.
29. The method according to claim 28, characterized in that The encoding of the sixth identifier includes: encoding the seventh identifier; If the seventh identifier indicates that there is no restriction on the value of the sixth identifier, the sixth identifier is encoded.
30. The method according to any one of claims 19 to 29, characterized in that The method further comprises: Determine the reference row corresponding to the maximum number of rows as the reference row allowed to be used by the current block; Constructing a TMRL mode list based on the reference rows allowed to be used by the current block and the prediction modes allowed to be used by the current block; encoding a first index; The reference row and prediction mode included in the combination indicated by the first index in the TMRL mode list are the reference row used by the current block and the prediction mode used by the current block.
31. The method according to claim 30, characterized in that The constructing a TMRL mode list based on the reference row allowed to be used by the current block and the prediction mode allowed to be used by the current block comprises: For each of at least one combination obtained by the reference row allowed to be used by the current block and the prediction mode allowed to be used by the current block, based on the reference row of each combination, using the prediction mode of each combination to predict the template of the current block, and obtain a prediction cost of each combination; Based on the predicted cost of each combination, the at least one combination is sorted in order of predicted cost from small to large to obtain a combination sorting; The combination that is ranked at the front in the middle is determined as the combination in the TMRL pattern list.
32. The method according to any one of claims 19 to 31, characterized in that The method further comprises: If the current block does not use the template-based multi-reference line TMRL mode, the reference line used by the current block is determined based on whether the current block uses the template-based intra-frame mode to derive the TIMD mode.
33. The method according to claim 32, characterized in that The method further comprises: If the current block uses the TIMD mode, encoding the second index; If the current block does not use the TIMD mode, encoding the third index; The second index indicates a reference row used when the current block uses the TIMD mode; and the third index indicates a reference row used when the current block does not use the TIMD mode.
34. The method according to claim 32 or 33, characterized in that If the current block does not use the TMRL mode, determining the reference row used by the current block based on whether the current block uses a template-based intra mode to derive a TIMD mode includes: Encoding the eighth identifier; If the eighth flag indicates that the current sequence to which the current block belongs allows the use of the multi-reference line MRL mode and the current block does not use the TMRL mode, the reference row used by the current block is determined based on whether the current block uses the TIMD mode.
35. The method according to claim 34, characterized in that The encoding of the eighth identifier includes: If the maximum number of rows is greater than or equal to a second preset threshold, the eighth identifier is encoded.
36. The method according to claim 34, characterized in that The determining, based on whether the current block uses the TIMD mode, a reference row used by the current block comprises: If the eighth flag indicates that the current sequence allows the use of multiple reference row MRL mode, and the maximum number of rows is greater than or equal to the second preset threshold, and the current block does not use the TMRL mode, then the reference row used by the current block is determined based on whether the current block uses the TIMD mode.
37. A decoder, characterized in that include: A first determining unit, used to determine the maximum number of reference lines allowed to be used by the current block; A second determining unit, configured to determine whether the current block uses a template-based multiple reference line TMRL mode based on the maximum number of lines; A third determining unit, configured to determine a reference row used by the current block and a prediction mode used by the current block if the current block uses the TMRL mode; The prediction unit is used to predict the current block based on the reference row used by the current block and using the prediction mode used by the current block to obtain the predicted block.
38. An encoder, characterized in that include: A determination unit, used to determine the maximum number of reference lines allowed to be used by the current block; An encoding unit is used to encode the maximum number of rows.
39. An electronic device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the method according to any one of claims 1 to 18 or the method according to any one of claims 19 to 36.
40. A computer-readable storage medium, characterized in that Used to store a computer program, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 18 or the method according to any one of claims 19 to 36.
41. A computer program product comprising a computer program / instructions, characterized in that The computer program / instructions, when executed by a processor, implement the method according to any one of claims 1 to 18 or the method according to any one of claims 19 to 36.
42. A code stream, characterized in that The code stream is a code stream generated by the method according to any one of claims 1 to 18 or the method according to any one of claims 19 to 36.
