Coding method, decoding method, code stream, coders, decoders and storage media

Abstract

Disclosed in the embodiments of the present application are a coding method and a decoding method. The decoding method comprises: at a decoding end, when it is determined that a current image is filtered by means of a TALF, determining a filter coefficient; on the basis of vector information corresponding to a current position, determining at least one reference position corresponding to the current position; and performing filtering on the basis of a reconstruction sample of the at least one reference position and the filter coefficient, so as to determine a filtered reconstruction value of the current position.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to a coding and decoding method, a code stream, an encoder, a decoder and a storage medium. BACKGROUND

[0002] In the multi-functional video coding (VVC), the in loop filter includes a DeBlocking Filter (DBF), a Sample adaptive Offset (SAO), an Adaptive loop filter (ALF) and a Cross Component Adaptive Loop Filter (CCALF).

[0003] However, the common filtering technology does not fully utilize the related information of the time domain reconstructed image, and the filtering effect is not ideal, which reduces the coding and decoding performance.

[0004] SUMMARY

[0005] Embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder and a storage medium, which can improve the filtering effect and improve the coding and decoding performance.

[0006] The technical scheme of the embodiments of the present application can be implemented as follows:

[0007] In a first aspect, the embodiments of the present application provide a decoding method applied to a decoder, and the method comprises:

[0008] In a case where it is determined that the current image is filtered using the TALF, filter coefficients are determined;

[0009] According to the vector information corresponding to the current position, at least one reference position corresponding to the current position is determined;

[0010] Filtering is performed based on the reconstructed samples of the at least one reference position and the filter coefficients to determine the filtered reconstructed value of the current position.

[0011] In a second aspect, the embodiments of the present application provide an encoding method applied to an encoder, and the method comprises:

[0012] In a case where the current image is filtered using the TALF, filter coefficients are determined;

[0013] determine at least one reference position corresponding to the current position according to vector information corresponding to the current position;

[0014] perform filtering based on the reconstructed samples of the at least one reference position and the filter coefficients to determine the filtered reconstructed value of the current position.

[0015] In a third aspect, an embodiment of the present application provides a code stream, which is generated by bit encoding of to-be-encoded information; wherein the to-be-encoded information at least includes auxiliary filtering identification information.

[0016] In a fourth aspect, an embodiment of the present application provides an encoder, which includes a first determining unit; wherein

[0017] The first determining unit is configured to, when filtering is performed on a current image using TALF, determine filter coefficients; determine at least one reference position corresponding to the current position according to vector information corresponding to the current position; and perform filtering based on the reconstructed samples of the at least one reference position and the filter coefficients to determine the filtered reconstructed value of the current position.

[0018] In a fifth aspect, an embodiment of the present application provides an encoder, which includes a first memory and a first processor; wherein

[0019] The first memory is configured to store a computer program capable of running on the first processor;

[0020] The first processor is configured to, when the computer program is running, execute the encoding method as described above.

[0021] In a sixth aspect, an embodiment of the present application provides a decoder, which includes a second determining unit; wherein

[0022] The second determining unit is configured to, when it is determined that filtering is performed on a current image using TALF, determine filter coefficients; determine at least one reference position corresponding to the current position according to vector information corresponding to the current position; and perform filtering based on the reconstructed samples of the at least one reference position and the filter coefficients to determine the filtered reconstructed value of the current position.

[0023] In a seventh aspect, an embodiment of the present application provides a decoder, which includes a second memory and a second processor; wherein

[0024] The second memory is configured to store a computer program capable of running on the second processor;

[0025] The second processor is configured to, when the computer program is running, execute the decoding method as described above.

[0026] In an eighth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed to implement the decoding method in the first aspect, or implement the encoding method in the second aspect.

[0027] In a ninth aspect, an embodiment of the present application provides a computer readable storage medium, which is used to store a code stream generated by the encoding method in the first aspect.

[0028] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder and a storage medium. At a decoding end, when it is determined that a current image is filtered by TALF, filter coefficients are determined; at least one reference position corresponding to a current position is determined according to vector information corresponding to the current position; filtering is performed based on reconstructed samples of the at least one reference position and the filter coefficients to determine a filtered reconstructed value of the current position. At an encoding end, when the current image is filtered by TALF, filter coefficients are determined; at least one reference position corresponding to a current position is determined according to vector information corresponding to the current position; filtering is performed based on reconstructed samples of the at least one reference position and the filter coefficients to determine a filtered reconstructed value of the current position. It can be seen that the embodiments of the present application propose a time domain adaptive loop filter method. In the process of performing TALF filtering, vector information including a motion vector or a block vector corresponding to a sample position can be used to obtain a corresponding reference position, and TALF filtering of the sample position is completed based on the reference position, that is, the vector information is introduced to assist TALF filtering, so as to further improve the filtering effect and improve the coding and decoding performance. BRIEF DESCRIPTION OF DRAWINGS

[0029] FIG. 1 is an application schematic diagram of an encoding framework provided by the related art;

[0030] FIG. 2 is an application schematic diagram of a loop filter module provided by the related art;

[0031] FIG. 3 is a shape schematic diagram one of a common luma adaptive loop filter;

[0032] FIG. 4 is a shape schematic diagram two of a common chroma adaptive loop filter;

[0033] FIG. 5 is an implementation process schematic diagram of loop filtering;

[0034] FIG. 6 is a cross-component adaptive filter shape schematic diagram;

[0035] FIG. 7 is a relationship of encoding / decoding and playing order under a RA configuration;

[0036] FIG. 8 is a system block diagram of an encoder according to an embodiment of the present application;

[0037] FIG. 9 is a system block diagram of a decoder according to an embodiment of the present application;

[0038] FIG. 10 is a flowchart of a decoding method according to an embodiment of the present application;

[0039] FIG. 11 is a diagram illustrating an implementation of a TALF filter according to an embodiment of the present application;

[0040] FIG. 12 is a diagram illustrating another implementation of a TALF filter according to an embodiment of the present application;

[0041] FIG. 13 is a diagram illustrating a third implementation of a TALF filter according to an embodiment of the present application;

[0042] FIG. 14 is a diagram illustrating a fourth implementation of a TALF filter according to an embodiment of the present application;

[0043] FIG. 15 is a diagram illustrating a fifth implementation of a TALF filter according to an embodiment of the present application;

[0044] FIG. 16 is a diagram illustrating a sixth implementation of a TALF filter according to an embodiment of the present application;

[0045] FIG. 17 is a diagram illustrating a seventh implementation of a TALF filter according to an embodiment of the present application;

[0046] FIG. 18 is a diagram illustrating a first implementation of a filter according to an embodiment of the present application;

[0047] FIG. 19 is a diagram illustrating a second implementation of a filter according to an embodiment of the present application;

[0048] FIG. 20 is a diagram illustrating a third implementation of a filter according to an embodiment of the present application;

[0049] FIG. 21 is a diagram illustrating a fourth implementation of a filter according to an embodiment of the present application;

[0050] FIG. 22 is a diagram illustrating a fifth implementation of a filter according to an embodiment of the present application;

[0051] FIG. 23 is a diagram illustrating a sixth implementation of a filter according to an embodiment of the present application;

[0052] FIG. 24 is a diagram illustrating a seventh implementation of a filter according to an embodiment of the present application;

[0053] FIG. 25 is a flowchart of an encoding method according to an embodiment of the present application;

[0054] FIG. 26 is a diagram illustrating a first implementation of a filter position according to an embodiment of the present application;

[0055] FIG. 27 is a diagram illustrating a second implementation of a filter position according to an embodiment of the present application;

[0056] Fig. 28 is a schematic diagram of a filtering position according to an embodiment of the present application;

[0057] Fig. 29 is a schematic diagram of a filtering position according to an embodiment of the present application;

[0058] Fig. 30 is a schematic diagram of a filtering position according to an embodiment of the present application;

[0059] Fig. 31 is a schematic diagram of a filtering position according to an embodiment of the present application;

[0060] Fig. 32 is a schematic diagram of a filter according to an embodiment of the present application;

[0061] Fig. 33 is a schematic diagram of a filter according to an embodiment of the present application;

[0062] Fig. 34 is a schematic diagram of a structure of an encoder according to an embodiment of the present application;

[0063] Fig. 35 is a schematic diagram of a specific hardware structure of an encoder according to an embodiment of the present application;

[0064] Fig. 36 is a schematic diagram of a structure of a decoder according to an embodiment of the present application;

[0065] Fig. 37 is a schematic diagram of a specific hardware structure of a decoder according to an embodiment of the present application;

[0066] Fig. 38 is a schematic diagram of a structure of a codec system according to an embodiment of the present application. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It can be understood that the specific embodiments described herein are only used to explain the related application, but not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for convenience of description.

[0068] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict. It should be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0069] Digital video compression technology is mainly to compress the huge digital video data, so as to facilitate transmission and storage, etc. With the proliferation of Internet video and people's demand for video clarity is getting higher and higher, although the existing digital video compression standard can save a lot of video data, but at present still need to pursue better digital video compression technology, in order to reduce the bandwidth and traffic pressure of digital video transmission.

[0070] In the process of digital video encoding, the encoder reads unequal samples of the original video sequence of different color formats, including luminance component and chrominance component, that is, the encoder reads a black and white or color image. Then it is divided into blocks, and the block data is handed over to the encoder for encoding.

[0071] The general video coding standard adopts a hybrid coding framework based on blocks. Each frame of video image is divided into square maximum coding units (LCU) or coding tree units (CTU) of the same size (such as 128x128, 64x64, etc.), each maximum coding unit or coding tree unit can also be divided into rectangular coding units (CU) according to the rules; and the coding unit may also be divided into smaller prediction units (PU), transform units (TU), etc.

[0072] FIG. 1 is a schematic diagram of an application of a coding framework provided by the related art. As shown in FIG. 1, the hybrid coding framework can include a prediction module 11, a transform and quantization module 12, an entropy coding module 13, an inverse quantization and inverse transform module 14, a loop filtering module 15, and a decoded picture buffer module 16. The prediction module 11 can include an intra prediction module 11a and an inter prediction module 11b, and the inter prediction module 11b can include a motion estimation module and a motion compensation module. Because there is a strong correlation between adjacent samples in a frame of a video image, using an intra prediction mode in a video coding technology can eliminate spatial redundancy between adjacent samples. However, because there is also a strong similarity between adjacent frames in a video image, using an inter prediction mode in a video coding technology can eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency. The basic process of a video codec is as follows: at an encoding end, a frame of an image is divided into blocks, an intra prediction or an inter prediction is used for a coding block to generate a prediction block of the coding block, an original block of the coding block is subtracted from the prediction block to obtain a residual block, a transform and quantization are performed on the residual block to obtain a quantized coefficient matrix, and the quantized coefficient matrix is entropy coded and output to a bitstream. At a decoding end, an intra prediction or an inter prediction is used for a coding block to generate a prediction block of the coding block, and on the other hand, a quantized coefficient matrix is obtained by decoding the bitstream, the quantized coefficient matrix is inverse quantized and inverse transformed to obtain a residual block, and the prediction block and the residual block are added to obtain a reconstructed value. The reconstructed value constitutes a reconstructed image, and a decoded image is obtained by performing loop filtering on the reconstructed image on a block basis or on an image basis. The encoding end also needs to perform similar operations as the decoding end to obtain a decoded image. The decoded image can be used as a reference frame for inter prediction of subsequent frames. If necessary, block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode information or parameter information determined by the encoding end need to be output to the bitstream. The decoding end analyzes and determines the same block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode information or parameter information as the encoding end, so as to ensure that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is usually also called a reconstructed image. When prediction is performed, a coding block can be divided into prediction units, and when transform is performed, the coding block can be divided into transform units. The division of the prediction units and the transform units can be different. The above is the basic process of a video codec under a hybrid coding framework based on blocks. With the development of technology, some modules or steps of the framework or process can be optimized. Embodiments of the present application are applicable to the basic process of a video codec under the hybrid coding framework based on blocks, but are not limited to the framework and process.

[0073] It can be understood that, intra prediction only refers to the information of the same frame image, predicts the sample information in the current partition block, and is used to eliminate spatial redundancy; inter prediction can refer to the image information of different frames, uses motion estimation to search for a motion vector information most matching the current partition block, and is used to eliminate temporal redundancy; transformation converts the predicted image block to a frequency domain, energy is redistributed, and the information not sensitive to human eyes can be removed in combination with quantization, and is used to eliminate visual redundancy; entropy coding can eliminate character redundancy according to a current context model and probability information of a binary code stream; and loop filtering mainly processes samples after inverse transformation and inverse quantization, makes up for distortion information, and provides better reference for subsequent coding samples.

[0074] Loop filtering is generally present in existing video coding standards, which greatly improves the subjective and objective quality of reconstructed video. In VVC, loop filtering includes a DeBlocking Filter (DBF), a Sample adaptive Offset (SAO), an Adaptive loop filter (ALF), and a Cross Component Adaptive Loop Filter (CCALF). In the latest JVET traditional video coding exploration platform reference software test model (Enhanced Compression Model, ECM), loop filtering additionally includes a Cross component sample adaptive offset (CCSAO) and a Bilateral filter (BIF). For example, FIG. 2 is an application schematic diagram of a loop filtering module provided by the related art, and a block diagram of ECM loop filtering is shown in FIG. 2.

[0075] The principle of adaptive loop filtering and cross component adaptive loop filtering is to calculate one or more sets of filter coefficients through the relationship between reconstructed values and original sample values. The filter coefficients are transmitted through a code stream, and the decoder can construct a filter after obtaining the filter coefficients, and use the filter to filter the reconstructed image, so that the reconstructed image is closer to the original image after filtering.

[0076] The adaptive loop filtering in VVC can be applied to luminance and chrominance components. FIG. 3 is a shape schematic diagram of a common luminance adaptive loop filter one, as shown in FIG. 3, the filter on the luminance component is a 7x7 diamond-shaped coefficient symmetric filter. FIG. 4 is a shape schematic diagram of a common chrominance adaptive loop filter two, as shown in FIG. 4, the filter on the chrominance component is a 5x5 diamond-shaped coefficient symmetric filter.

[0077] The adaptive loop filter for luma and chroma in VVC uses the reconstructed samples under the filter window as input values, which are multiplied by adaptive filter coefficients and then summed to produce a correction value for the center position of the filter. The correction value is added to the reconstructed value at the center position to obtain the filtered value.

[0078] In ECM, the adaptive loop filter is more refined. The adaptive loop filter in ECM has more coefficients, and the input values of the filter are not limited to the current reconstructed value, but can also include residual values, reconstructed values before deblocking filtering, reconstructed values after deblocking filtering, reconstructed values after Gaussian filtering, and the like. Most of the inputs of the adaptive loop filter in ECM have undergone some pre-filtering processing, so that the adaptive loop filter does not need to transmit a large number of adaptive coefficients through the code stream as before.

[0079] Exemplarily, the cross-component adaptive filter in VVC is an 8-coefficient filter, which is only applicable to the chroma component. The cross-component adaptive filter uses the reconstructed value of luma as the filter input, and produces a correction value with the filter coefficients to be added to the chroma component to improve the subjective and objective quality of the chroma. In ECM, the number of filter coefficients is further increased to achieve higher filtering efficiency. At the same time, the residual value of luma and the reconstructed value of chroma are further made as inputs of the cross-component adaptive filter.

[0080] FIG. 5 is a schematic diagram of the implementation process of loop filtering. As shown in FIG. 5, the CCALF can take the reconstructed information and the residual information (l0) of luma as inputs, so the CCALF takes the luma reconstructed information and the luma residual information output by the SAO as inputs, calculates the filter coefficients to perform filtering processing, and obtains the filtering result (l1). or In addition, the chroma ALF itself filters the chroma reconstructed information output by the SAO to obtain the corresponding filtering result. Finally, the filtering result (l1) of the CCALF and the filtering result of the chroma ALF are added to obtain the final chroma filtering result. or

[0081] FIG. 6 is a schematic diagram of the shape of the cross-component adaptive filter. As shown in FIG. 6, the luma information (luma samples) can be used to minimize the error between the chroma reconstructed image and the original image, so as to complete the correction of the chroma samples.

[0082] ​Video encoding and decoding usually include three mainstream configurations: all intra (AI), random access (RA) and low delay (LD). In AI configuration, all video frames are encoded using intra coding technology; in RA configuration, according to the setting, a plurality of inter-coded frames are inserted between every two intra-coded frames; in LD configuration, except for the first encoded frame being an intra-coded frame, the rest are inter-coded frames. In AI and LD configurations, the encoding and decoding order of video frames is the same as the playing order, while in RA, the encoding and decoding order and the playing order can be different.

[0083] Exemplarily, FIG. 7 is a relationship between the encoding / decoding and playing order in an RA configuration. As shown in FIG. 7, the actual RA encoding / decoding order is not performed according to the variable (picture order count, POC) order used to determine the video playing order. For example, in the current ECM CTC test condition, the encoding / decoding order, POC value and temporal level (TemporalId, TID) relationship of RA are shown in Table 1 as follows:

[0084] Table 1

[0085] The inter-coded frame with a larger TID has more available reference images and is more likely to be compressed. Therefore, the higher the TID, the larger the quantization parameter used, and the lower the code rate can be.

[0086] Adaptive parameter set (APS) is a picture-level parameter set, which is used to transmit some adaptive parameters. The APS syntax element table in VVC is as follows:

[0087] The APS parameter set includes a plurality of syntax elements, wherein aps_adaptation_parameter_set_id is used to assign an index to the currently decoded APS set, aps_params_type is used to indicate the type of the APS parameter set, which in VVC can be a parameter set for encoding ALF adaptive parameters, or a parameter set for luma mapping with chroma scaling (LMCS) filter or SCALING. When this APS is an ALF type parameter set, the number of ALF filters, filter parameters and other data are further parsed.

[0088] Since the common filtering techniques do not fully utilize the correlation information of the time-domain reconstructed image, the filtering effect is not ideal, and the coding and decoding performance is reduced. In this regard, the TALF scheme is proposed in the embodiments of the present application, which can fully utilize the correlation information of the inter-frame reference image, thereby improving the filtering effect.

[0089] Further, the embodiments of the present application provide a coding and decoding method, a bitstream, an encoder, a decoder and a storage medium. In the process of performing TALF filtering, the vector information including the motion vector or the block vector corresponding to the sample position can be used to obtain the corresponding reference position, and the TALF filtering of the sample position is completed based on the reference position, that is, the vector information is introduced to assist the TALF filtering, thereby further improving the filtering effect and improving the coding and decoding performance.

[0090] Referring to FIG. 8, an example of a system composition block diagram of an encoder provided by the embodiments of the present application is shown. As shown in FIG. 8, the encoder 10 can include a partition unit 101, a prediction unit 102, a first adder 107, a transform unit 108, a quantization unit 109, a dequantization unit 110, an inverse transform unit 111, a second adder 112, a filtering unit 113, a decoded picture buffer (DPB) unit 114 and an entropy encoding unit 115. Here, the input of the encoder 10 can be a video composed of a series of pictures or a still picture, and the output of the encoder 10 can be a bitstream (also referred to as a “bitstream”) representing the compressed version of the input video.

[0091] The partition unit 101 partitions the pictures in the input video into one or more coding tree units (CTUs). The partition unit 101 divides the picture into a plurality of tiles (or tiles), and can further divide one tile into one or more bricks. Here, one tile or one brick can include one or more complete and / or partial CTUs. In addition, the partition unit 101 can form one or more slices, wherein one slice can include one or more tiles arranged in raster order in the picture, or one or more tiles covering a rectangular region in the picture. The partition unit 101 can also form one or more sub-pictures, wherein one sub-picture can include one or more slices, tiles or bricks.

[0092] In the encoding process of the encoder 10, the partition unit 101 delivers a CTU to the prediction unit 102. Generally, the prediction unit 102 can be composed of a block partition unit 103, a motion estimation (ME) unit 104, a motion compensation (MC) unit 105, and an intra prediction unit 106. Specifically, the block partition unit 103 iteratively partitions an input CTU into smaller coding units (CUs) using quad-tree partitioning, binary-tree partitioning, and ternary-tree partitioning. The prediction unit 102 can obtain an inter prediction block for a CU using the ME unit 104 and the MC unit 105. The intra prediction unit 106 can obtain an intra prediction block for a CU using various intra prediction modes including the MIP mode. In an example, a rate-distortion optimized motion estimation approach can be invoked by the ME unit 104 and the MC unit 105 to obtain the inter prediction block, and a rate-distortion optimized mode determination approach can be invoked by the intra prediction unit 106 to obtain the intra prediction block.

[0093] The prediction unit 102 outputs the prediction block of a CU, and the first adder 107 calculates the difference between the CU in the output of the partition unit 101 and the prediction block of the CU, i.e., a residual CU. The transform unit 108 reads the residual CU and performs one or more transform operations on the residual CU to obtain coefficients. The quantization unit 109 quantizes the coefficients and outputs quantized coefficients (i.e., levels). The inverse quantization unit 110 performs a scaling operation on the quantized coefficients to output reconstructed coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transforms in the transform unit 108 and outputs a reconstructed residual. The second adder 112 calculates a reconstructed CU by adding the reconstructed residual and the prediction block of the CU from the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 to be used as an intra prediction reference. After all CUs in a picture or sub-picture are reconstructed, the filter unit 113 performs loop filtering on the reconstructed picture or sub-picture. Here, the filter unit 113 contains one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping with chroma scaling (LMCS) filter, and a neural network based filter, etc. Alternatively, when the filter unit 113 determines that a CU is not to be used as a reference for encoding of other CUs, the filter unit 113 performs loop filtering on one or more target samples in the CU.

[0094] The output of the filtering unit 113 is decoded pictures or sub-pictures, which are buffered to the DPB unit 114. The DPB unit 114 outputs the decoded pictures or sub-pictures according to the timing and control information. Here, the pictures stored in the DPB unit 114 can also be used as reference for the inter prediction or intra prediction performed by the prediction unit 102. Finally, the entropy encoding unit 115 converts the parameters (such as control parameters and supplemental information, etc.) necessary for decoding the pictures from the encoder 10 into binary form, and writes such binary form into the bitstream according to the syntax structure of each data unit, i.e., the bitstream finally output by the encoder 10.

[0095] Further, the encoder 10 can be a computing device having a first processor and a first memory storing a computer program. When the first processor reads and runs the computer program, the encoder 10 reads the input video and generates the corresponding bitstream. In addition, the encoder 10 can also be a computing device having one or more chips. These units implemented as integrated circuits on the chip have similar connection and data exchange functions as the corresponding units in FIG. 9.

[0096] Referring to FIG. 9, an example of a system composition block diagram of a decoder provided by an embodiment of the present application is shown. As shown in FIG. 9, the decoder 20 can include a parsing unit 201, a prediction unit 202, an inverse quantization unit 205, an inverse transformation unit 206, an adder 207, a filtering unit 208, and a decoded picture buffer unit 209. Here, the input of the decoder 20 is a bitstream used to represent a compressed version of a video or a still picture, and the output of the decoder 20 can be a decoded video composed of a series of pictures or a decoded still picture.

[0097] The input bitstream of the decoder 20 can be the bitstream generated by the encoder 10. The parsing unit 201 parses the input bitstream and obtains the values of the syntax elements from the input bitstream. The parsing unit 201 converts the binary representation of the syntax elements into digital values and sends the digital values to the units in the decoder 20 to obtain one or more decoded pictures. The parsing unit 201 can also parse one or more syntax elements from the input bitstream to display the decoded pictures.

[0098] During the decoding process of the decoder 20, the parsing unit 201 sends the values of the syntax elements and one or more variables used to obtain one or more decoded pictures, which are set or determined according to the values of the syntax elements, to the units in the decoder 20.

[0099] The prediction unit 202 determines a prediction block for a current decoding block (e.g., a CU). Here, the prediction unit 202 can include a motion compensation unit 203 and an intra prediction unit 204. Specifically, when an inter decoding mode is indicated for decoding the current decoding block, the prediction unit 202 passes relevant parameters from the parsing unit 201 to the motion compensation unit 203 to obtain an inter prediction block; when an intra prediction mode (including a MIP mode indicated based on a MIP mode index value) is indicated for decoding the current decoding block, the prediction unit 202 passes relevant parameters from the parsing unit 201 to the intra prediction unit 204 to obtain an intra prediction block.

[0100] The inverse quantization unit 205 has the same function as the inverse quantization unit 110 in the encoder 10. The inverse quantization unit 205 performs a scaling operation on the quantized coefficients (i.e., levels) from the parsing unit 201 to obtain reconstructed coefficients.

[0101] The inverse transform unit 206 has the same function as the inverse transform unit 111 in the encoder 10. The inverse transform unit 206 performs one or more transform operations (i.e., inverse operations of one or more transform operations performed by the inverse transform unit 111 in the encoder 10) to obtain reconstructed residuals.

[0102] The adder 207 performs an addition operation on its inputs (a prediction block from the prediction unit 202 and reconstructed residuals from the inverse transform unit 206) to obtain a reconstructed block for the current decoding block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks encoded in the intra prediction mode.

[0103] After all CUs in a picture or sub-picture are reconstructed, the filter unit 208 performs loop filtering on the reconstructed picture or sub-picture. The filter unit 208 contains one or more filters, such as a deblocking filter, a sample adaptive offset filter, an adaptive loop filter, a luma mapping and chroma scaling filter, and a neural network based filter, etc. Alternatively, when the filter unit 208 determines that a reconstructed block is not used as a reference for decoding other blocks, the filter unit 208 performs loop filtering on one or more target samples in the reconstructed block. Here, the output of the filter unit 208 is a decoded picture or sub-picture, which is buffered to the DPB unit 209. The DPB unit 209 outputs the decoded picture or sub-picture according to the timing and control information. Pictures stored in the DPB unit 209 can also be used as a reference for performing inter prediction or intra prediction by the prediction unit 202.

[0104] Further, the decoder 20 can be a second processor and a second memory recording a computer program. When the first processor reads and runs the computer program, the decoder 20 reads the input code stream and generates the corresponding decoded video. In addition, the decoder 20 can also be a computing device with one or more chips. These units implemented as integrated circuits on the chip have similar connection and data exchange functions as the corresponding units in FIG. 8.

[0105] It should be further noted that when the embodiments of the present application are applied to the encoder 10, the "coding block" specifically refers to a current block to be encoded in the video image (which can also be referred to as "coding block" for short); when the embodiments of the present application are applied to the decoder 20, the "coding block" specifically refers to a current block to be decoded in the video image (which can also be referred to as "decoding block" for short).

[0106] Based on FIG. 7, the encoding method in the embodiments of the present application is mainly applied to the "filtering unit 113" part in the encoder 10.

[0107] Based on FIG. 8, the decoding method in the embodiments of the present application is mainly applied to the "filtering unit 208" part in the decoder 20.

[0108] That is, the encoding and decoding method in the embodiments of the present application can be applied to a video encoding system (referred to as "encoder" for short), a video decoding system (referred to as "decoder" for short), or even both a video encoding system and a video decoding system, but here is not limited.

[0109] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0110] An embodiment of the present application provides a decoding method, which is applied to a decoder and used in a scene of performing filtering processing through a TALF filter. FIG. 10 is a schematic diagram of the decoding method provided by the embodiment of the present application. As shown in FIG. 10, the method of the decoder performing decoding processing can include the following steps.

[0111] Step 1001: In a case where it is determined that the current image is filtered using TALF, filter coefficients are determined.

[0112] In the embodiments of the present application, if it is determined that the current image is filtered using TALF, the filter coefficients can be determined first.

[0113] It should be noted that in the embodiments of the present application, the first syntax element identification information can be determined by decoding the code stream. The first syntax element identification information can be used to determine whether the current block in the current picture is filtered using the TALF, i.e., based on the first syntax element identification information, it can be determined whether the TALF filter is used to filter the reconstructed value of the image component of the current block.

[0114] Further, in the embodiments of the present application, for the TALF filter, the corresponding input information can include at least one inter-frame reference picture corresponding to the current picture, and the at least one inter-frame reference picture can be a decoded reconstructed picture in the time domain. The TALF filtering of the current block in the current picture can be completed using the reconstructed sample value of the at least one inter-frame reference picture.

[0115] It can be understood that in the embodiments of the present application, the current block can be a CTU determined after the current picture is divided, the current block can also be a coding unit (CU) obtained after the CTU is divided, and the current block can also be a sub-region of any size obtained after the current picture is divided. The size and the obtaining manner of the current block in the current picture are not limited in the present application.

[0116] Further, in the embodiments of the present application, the first syntax element identification information can be used to determine whether the TALF is used to filter the current block. The first syntax element identification information can be a flag. If the current block is a CTU corresponding to the current picture, the first syntax element identification information can be a CTU-level flag. If the current block is a CU corresponding to the current picture, the first syntax element identification information can be a CU-level flag. Of course, for a sub-region of any size in the current picture, the first syntax element identification information can also be a flag of another block level. The present application does not make specific limitation.

[0117] Further, in the embodiments of the present application, whether the TALF is used to filter the current block can be determined by the value of the first syntax element identification information.

[0118] Exemplarily, in some embodiments, when the value of the first syntax element identification information is a first value, it is determined that the TALF is not used to filter the current block. When the value of the first syntax element identification information is not the first value, it is determined that the TALF is used to filter the current block.

[0119] It should be noted that in the embodiments of the present application, the first syntax element identification information can be used to indicate whether the TALF is used to filter the current block. In addition, the first value can be in the form of a parameter or in the form of a number, which is not limited herein.

[0120] Exemplarily, in some embodiments, if the first syntax element identifies information as a block level flag, in one specific example, the first value can be set as 0; in another specific example, the first value can also be set as false. The first value in the embodiments of the present application is not limited in any way.

[0121] Taking the first value as 0 as an example, in the embodiments of the present application, if the value of the first syntax element identifying information is 0, it can be determined that the current block does not use TALF for filtering. Otherwise, if the value of the first syntax element identifying information is not 0, it can be determined that the current block uses TALF for filtering.

[0122] Exemplarily, in some embodiments, if the current block of the current image is a CTU, the first syntax element identifying information can be a CTU level syntax, for example, the first syntax element identifying information can be represented by a syntax element control flag talf_ctb_idc[CtbAddrX][CtbAddrY], that is, talf_ctb_idc[CtbAddrX][CtbAddrY] indicates whether TALF is used for filtering the current block of the current image. If the value of talf_ctb_idc[CtbAddrX][CtbAddrY] is 0, it is determined that TALF is not used for filtering the current block, and if the value of talf_ctb_idc[CtbAddrX][CtbAddrY] is not 0, it is determined that TALF is used for filtering the current block. Of course, the value of talf_ctb_idc[CtbAddrX][CtbAddrY] is not limited to 0, and the present application does not make specific limitations.

[0123] Further, in the embodiments of the present application, the code stream can be decoded first to determine the second syntax element identifying information; in the case where it is determined based on the second syntax element identifying information that the current image is allowed to use TALF for filtering, the determination process of the first syntax element identifying information is performed, that is, the parsing process of the first syntax element identifying information is performed.

[0124] Further, in the embodiments of the present application, the second syntax element identifying information can be used to determine whether the current image is allowed to use TALF for filtering. The second syntax element identifying information can be a flag.

[0125] It should be noted that, in the embodiments of the present application, the second syntax element identifying information can be a picture level flag, a slice level (slice level) flag, a sub-picture level flag, or a tile level flag. The present application does not make specific limitations.

[0126] Exemplarily, in some embodiments, if the second syntax element identifies information is a flag at slice level, the second syntax element identifies information can be used to determine whether the current slice is allowed to use TALF for filtering.

[0127] Further, in embodiments of the present application, whether the current picture (and / or the current slice) is allowed to use TALF for filtering can be determined by the value of the second syntax element identifies information.

[0128] Exemplarily, in some embodiments, if the value of the second syntax element identifies information is the second value, it is determined that the current picture (and / or the current slice) is allowed to use TALF for filtering. If the value of the second syntax element identifies information is the third value, it is determined that the current picture (and / or the current slice) is not allowed to use TALF for filtering.

[0129] It should be noted that, in embodiments of the present application, the second syntax element identifies information can be used to indicate whether the current picture (and / or the current slice) is allowed to use TALF for filtering. In addition, the second value and the third value are different, and the second value and the third value can be in the form of a parameter or in the form of a number. In general, the second syntax element identifies information can be a parameter written in a slice header or a picture header, which is not limited herein.

[0130] It should be further noted that, if the second syntax element identifies information is a flag at picture level or a flag at slice level, in one specific example, the second value can be set to 1, and the third value can be set to 0; in another specific example, the second value can also be set to true, and the third value can also be set to false; or in yet another specific example, the second value can also be set to 0, and the third value can also be set to 1; or the second value can also be set to false, and the third value can also be set to true. The second value and the third value in embodiments of the present application are not limited.

[0131] Suppose the second syntax element identifies information is a flag at picture level, and the second value is 1 and the third value is 0, in embodiments of the present application, if the value of the second syntax element identifies information is 1, it is determined that the current picture uses TALF for filtering. Otherwise, if the value of the second syntax element identifies information is 0, it is determined that the current picture does not use TALF for filtering.

[0132] Exemplarily, in some embodiments, assuming the second syntax element identification information is a picture level flag, the second syntax element identification information can be represented by a syntax element ph_talf_enabled_flag, i.e., ph_talf_enabled_flag indicates whether the current picture is allowed to use TALF for filtering. If the value of ph_talf_enabled_flag is 0, it is determined that the current picture is not allowed to use TALF for filtering, and if the value of ph_talf_enabled_flag is 1, it is determined that the current picture is allowed to use TALF for filtering.

[0133] Exemplarily, in some embodiments, assuming the second syntax element identification information is a slice level flag, the second syntax element identification information can be represented by a syntax element sh_talf_enabled_flag, i.e., sh_talf_enabled_flag indicates whether the current slice is allowed to use TALF for filtering. If the value of sh_talf_enabled_flag is 0, it is determined that the current slice is not allowed to use TALF for filtering, and if the value of sh_talf_enabled_flag is 1, it is determined that the current slice is allowed to use TALF for filtering.

[0134] Of course, the value of sh_talf_enabled_flag is not limited to 0 and 1, and the present application does not make specific limitations.

[0135] Therefore, in the embodiments of the present application, the parsing of the first syntax element identification information can be dependent on the second syntax element identification information. That is, the block level identification can depend on the picture level identification, or the block level identification can depend on the slice level identification. For example, in the case of determining that the current picture (and / or the current slice) uses TALF for filtering based on the second syntax element identification information, the determination of the first syntax element identification information can be further performed, otherwise, the first syntax element identification information does not need to be decoded again.

[0136] Exemplarily, in some embodiments, an example of encoding a picture level TALF identifier (second syntax element identification information) in a picture header and a corresponding APS index can be as follows:

[0137] Wherein, ph_talf_enabled_flag is used to indicate whether the current picture uses TALF for filtering, and ph_talf_aps_id can be used to indicate in which APS parameter set the adaptive parameters (such as the number of filters, filter coefficients) of TALF and other information are obtained when TALF is used.

[0138] Further, in the embodiments of the present application, the code stream can be decoded first to determine the third syntax element identification information; and in a case that it is determined based on the third syntax element identification information that the current sequence is filtered using the TALF, the determination process of the second syntax element identification information is performed.

[0139] Further, in the embodiments of the present application, the third syntax element identification information can be used to determine whether the current sequence is filtered using the TALF. The third syntax element identification information can be a flag, for example, a sequence-level flag.

[0140] Further, in the embodiments of the present application, whether the current sequence is filtered using the TALF can be determined by the value of the third syntax element identification information.

[0141] For example, in some embodiments, in a case that the value of the third syntax element identification information is a fourth value, it is determined that the current sequence is filtered using the TALF. In a case that the value of the third syntax element identification information is a fifth value, it is determined that the current sequence is not filtered using the TALF.

[0142] It should be noted that, in the embodiments of the present application, the third syntax element identification information can be used to indicate whether the current sequence is filtered using the TALF. In addition, the fourth value and the fifth value are different, and the fourth value and the fifth value can be in the form of a parameter or in the form of a number, which are not limited herein.

[0143] It should be further noted that, if the third syntax element identification information is a sequence-level flag, in one specific example, the fourth value can be set to 1, and the fifth value can be set to 0; in another specific example, the fourth value can also be set to true, and the fifth value can also be set to false; or in yet another specific example, the fourth value can also be set to 0, and the fifth value can also be set to 1; or the fourth value can also be set to false, and the fifth value can also be set to true. The fourth value and the fifth value in the embodiments of the present application are not limited.

[0144] For example, assuming that the third syntax element identification information is a sequence-level flag, and the fourth value is 1 and the fifth value is 0, in the embodiments of the present application, if the value of the third syntax element identification information is 1, it is determined that the current sequence is filtered using the TALF. Otherwise, if the value of the third syntax element identification information is 0, it is determined that the current sequence is not filtered using the TALF.

[0145] Exemplarily, in some embodiments, assuming that the third syntax element identification information is a sequence level flag, the third syntax element identification information can be represented by a syntax element sps_talf_enabled_flag, i.e., sps_talf_enabled_flag indicates whether TALF is used for filtering the current sequence. If the value of sps_talf_enabled_flag is 0, it is determined that TALF is not used for filtering the current sequence, and if the value of sps_talf_enabled_flag is 1, it is determined that TALF is used for filtering the current sequence.

[0146] Of course, the value of sps_talf_enabled_flag is not limited to 0 and 1, and the present application does not make specific limitations.

[0147] Therefore, in the embodiments of the present application, the parsing of the second syntax element identification information can depend on the third syntax element identification information, i.e., the identification at the picture level (and / or slice level) can depend on the identification at the sequence level. For example, in the case of determining that TALF is used for filtering the current sequence based on the third syntax element identification information, the determination of the second syntax element identification information can be further performed, otherwise, the second syntax element identification information does not need to be decoded again.

[0148] That is, in the embodiments of the present application, TALF can include a sequence level enabling identification, i.e., the third syntax element identification information. For example, the manner of adding TALF sequence level identification in the standard text can be as follows:

[0149] Wherein, sps_talf_enabled_flag is used to indicate whether TALF is used for filtering the current sequence, and sps_alf_enabled_flag is used to indicate whether ALF is used for filtering the current sequence.

[0150] Further, in the embodiments of the present application, the sequence level TALF identification can depend on the sequence level ALF identification, i.e., the parsing of the third syntax element identification information can have a dependent relationship with the parsing of the sequence level syntax element of ALF.

[0151] Exemplarily, in some embodiments, it is assumed that the sequence level ALF indication can be represented by a syntax element sps_alf_enabled_flag, i.e., sps_alf_enabled_flag indicates whether ALF is used for filtering the current sequence. If sps_alf_enabled_flag indicates that ALF is not used for filtering the current sequence, then the determination of the third syntax element indication information can be selected not to be decoded any more; if sps_alf_enabled_flag indicates that ALF is used for filtering the current sequence, then the determination of the third syntax element indication information can be further performed.

[0152] Exemplarily, in some embodiments, the TALF sequence level indication (the third syntax element indication information) can depend on the ALF picture level indication (the picture level ALF indication), and the dependency relationship can be as follows:

[0153] Wherein, sps_talf_enabled_flag is used for indicating whether TALF is used for filtering the current sequence, sps_alf_enabled_flag is used for indicating whether ALF is used for filtering the current sequence, and only when the current sequence allows ALF, i.e., sps_alf_enabled_flag indicates that ALF is used for filtering the current sequence, sps_talf_enabled_flag can be further parsed.

[0154] Further, in the embodiments of the present application, the picture level (and / or slice level) indication depends on the sequence level indication, which can include that the picture level (and / or slice level) TALF indication depends on the sequence level TALF indication, and can also include that the picture level (and / or slice level) TALF indication depends on the sequence level ALF indication.

[0155] That is to say, in the embodiments of the present application, the parsing of the second syntax element indication information can depend on the third syntax element indication information, or can depend on the sequence level ALF indication.

[0156] Exemplarily, in some embodiments, it is assumed that the sequence level ALF indication can be represented by a syntax element sps_alf_enabled_flag, i.e., sps_alf_enabled_flag indicates whether ALF is used for filtering the current sequence. If sps_alf_enabled_flag indicates that ALF is not used for filtering the current sequence, then the determination of the second syntax element indication information can be selected not to be decoded any more; if sps_alf_enabled_flag indicates that ALF is used for filtering the current sequence, then the determination of the second syntax element indication information can be further performed.

[0157] Exemplarily, in some embodiments, the TALF picture level identification (TALF picture level identification) can depend on the ALF sequence level identification (ALF sequence level identification), and the dependency relationship can be as follows:

[0158] Wherein, sps_talf_enabled_flag is used to indicate whether the current sequence is filtered by TALF, sps_alf_enabled_flag is used to indicate whether the current sequence is filtered by ALF, ph_talf_enabled_flag is used to indicate whether the current picture is filtered by TALF, and ph_talf_aps_id can be used to indicate where the adaptive parameters (such as the number of filters, filter coefficients) of TALF and other information are obtained in the APS parameter set when TALF is used.

[0159] Further, in the embodiments of the present application, the TALF identification at the picture level (and / or slice level) can also depend on the ALF identification at the picture level (and / or slice level).

[0160] That is, in the embodiments of the present application, the parsing of the second syntax element identification information can also depend on the ALF identification at the picture level (and / or slice level).

[0161] Exemplarily, in some embodiments, it is assumed that the ALF identification at the picture level can be represented by the syntax element ph_alf_enabled_flag, that is, ph_alf_enabled_flag indicates whether the current picture is allowed to be filtered by ALF. If ph_alf_enabled_flag indicates that the current picture is not filtered by ALF, then the decoding of the second syntax element identification information can be selected to be stopped; if ph_alf_enabled_flag indicates that the current picture is allowed to be filtered by ALF, then the determination of the second syntax element identification information can be further performed.

[0162] Exemplarily, in some embodiments, the TALF picture level identification (TALF picture level identification) can depend on the ALF picture level identification (ALF picture level identification), and the dependency relationship can be as follows:

[0163] sps_talf_enabled_flag is used to indicate whether the current sequence is filtered by TALF, ph_alf_enabled_flag is used to indicate whether the current picture is filtered by ALF, ph_talf_enabled_flag is used to indicate whether the current picture is filtered by TALF, and ph_talf_aps_id can be used to indicate where the adaptive parameters (such as the number of filters and filter coefficients) of TALF are obtained from the APS parameter set when TALF is used.

[0164] Exemplarily, in some embodiments, the image-level (and / or slice-level) TALF identification can also depend on some high-level syntax, for example, the high-level syntax element pps_alf_info_in_ph_flag is used to identify the control identification of the image-level or slice-level of the syntax element used to parse the ALF of the image-level or slice-level, and the ALF identifier also exists in the slice header.

[0165] Exemplarily, in some embodiments, the slice-level TALF identification (slice-level TALF identification) can depend on the TALF control identification, and the dependency relationship can be as follows:

[0166] sps_talf_enabled_flag is used to indicate whether the current sequence is filtered by TALF, pps_alf_info_in_ph_flag is used to indicate whether the control identification of the image-level or slice-level of the syntax element used to parse the ALF of the image-level or slice-level is used, sh_talf_enabled_flag is used to indicate whether the current slice is filtered by TALF, and sh_talf_aps_id can be used to indicate where the adaptive parameters (such as the number of filters and filter coefficients) of TALF are obtained from the APS parameter set when TALF is used.

[0167] Further, in the embodiments of the present application, the vector information corresponding to the current position in the current picture can be selected to assist the TALF filtering. In the process of using the vector information corresponding to the current position to assist the TALF filtering, the position corresponding to the current position can be offset based on the vector information corresponding to the current position, so as to obtain at least one reference position corresponding to the current position.

[0168] It can be understood that in the embodiments of the present application, corresponding to different types of vector information, the images where the at least one reference position is determined to be located are different, for example, for a motion vector, the at least one reference position is determined to be located in an inter-frame reference image pointed to by the motion vector; for a block vector, the at least one reference position is determined to be located in the current image.

[0169] Further, in embodiments of the present application, if it is determined to use TALF for filtering, then the filter coefficients corresponding to the current block can be further determined.

[0170] Further, in embodiments of the present application, when determining the filter coefficients corresponding to the current block, the number parameter, the coefficient value parameter and the coefficient sign parameter corresponding to the current block can be determined first; then the number of groups of filters can be determined according to the number parameter; then the filter coefficients corresponding to the current block can be determined according to the number of groups of filters, the coefficient value parameter, the coefficient sign parameter and the first syntax element identification information.

[0171] It can be understood that, in embodiments of the present application, the number parameter can be used to determine the number of groups of filters corresponding to the current image. The coefficient value parameter can be used to determine the absolute value of the filter coefficients. The coefficient sign parameter can be used to determine the sign of the filter coefficients, including positive and negative.

[0172] Further, in embodiments of the present application, the number of groups of filters can be determined according to the number parameter. Wherein, the number of groups of filters can be determined according to the number parameter and a preset value.

[0173] It can be understood that, in embodiments of the present application, when determining the number of groups of filters according to the number parameter and the preset value, the value of the number parameter and the preset value can be subjected to mathematical operation, so as to obtain the number of groups of filters.

[0174] Exemplarily, in some embodiments, the number parameter can be represented by a syntax element talf_num_filters_signalled_minus1, which is a syntax element indicating the number of TALF filters contained in the current slice minus 1 (i.e. the preset value is 1). Wherein, assuming that a slice can have at most four TALF filters, the value of talf_num_filters_signalled_minus1 can be 0, 1, 2 or 3. The syntax element being 0 indicates that the current slice has one TALF filter, the syntax element being 1 indicates that the current slice has two TALF filters, the syntax element being 2 indicates that the current slice has three TALF filters, and the syntax element being 3 indicates that the current slice has four TALF filters. When the syntax element does not exist in the code stream, its value can be defaulted to 0.

[0175] Further, in the embodiments of the present application, when the filter coefficients corresponding to the current block are determined according to the group number of the filter, the coefficient numerical value parameter, the coefficient sign parameter and the first syntax element identification information, the candidate filter corresponding to the current block can be determined according to the first syntax element identification information and the group number of the filter first.

[0176] It can be understood that in the embodiments of the present application, the first syntax element identification information can also be used to determine the TALF information used by the current block, for example, the candidate filter corresponding to the current block can be determined through the value of the first syntax element identification information.

[0177] Exemplarily, in some embodiments, when the value of the first syntax element identification information is a first value, it is determined that the TALF is not used for filtering the current block. When the value of the first syntax element identification information is not the first value, it is determined that the TALF is used for filtering the current block, and the candidate filter corresponding to the current block can be further determined according to the first syntax element identification information.

[0178] Further, in the embodiments of the present application, when the candidate filter corresponding to the current block is determined according to the first syntax element identification information and the group number of the filter, the group number index of the filter corresponding to the current block can be determined according to the first syntax element identification information first, and then the candidate filter corresponding to the current block can be determined according to the group number of the filter and the group number index of the filter corresponding to the current block.

[0179] It can be understood that in the embodiments of the present application, when it is determined that the TALF is used for filtering the current block, the group number index of the filter corresponding to the current block can be determined according to the value of the first syntax element identification information. The group number index of the filter can represent the index of the TALF filter used by the current block in the filter set coded in the current image, that is, the group number index of the filter can be used to select the candidate filter corresponding to the current block from the plurality of filters corresponding to the current image.

[0180] Exemplarily, in some embodiments, if the current block of the current picture is a CTU, it is assumed that the first syntax element identification information is represented by syntax element talf_ctb_idc[CtbAddrX][CtbAddrY], if the value of talf_ctb_idc[CtbAddrX][CtbAddrY] is 0, it is determined that the current block is not filtered by using TALF, if the value of talf_ctb_idc[CtbAddrX][CtbAddrY] is 1, it is determined that the candidate filter corresponding to the current block can be the first group of filters in the at least one group of filters, if the value of talf_ctb_idc[CtbAddrX][CtbAddrY] is 2, it is determined that the candidate filter corresponding to the current block can be the second group of filters in the at least one group of filters.

[0181] It should be noted that in the embodiments of the present application, the value of the first syntax element identification information depends on the number of groups of filters. Wherein, it is assumed that the number of groups of filters corresponding to the filter can be determined by the number parameter talf_num_filters_signalled_minus1, then the value of the first syntax element identification information talf_ctb_idc[CtbAddrX][CtbAddrY] can depend on talf_num_filters_signalled_minus1, for example, the value range of talf_ctb_idc[CtbAddrX][CtbAddrY]-1 can be 0-talf_num_filters_signalled_minus1.

[0182] Further, in the embodiments of the present application, when determining the filter coefficient corresponding to the current block according to the candidate filter corresponding to the current block, the coefficient value parameter and the coefficient sign parameter, the value of the filter coefficient corresponding to the candidate filter can be determined according to the coefficient value parameter; at the same time, the sign of the filter coefficient corresponding to the candidate filter is determined according to the coefficient sign parameter; finally, the filter coefficient corresponding to the current block can be determined according to the value of the filter coefficient corresponding to the candidate filter and the sign of the filter coefficient corresponding to the candidate filter.

[0183] It can be understood that in the embodiments of the present application, after determining the candidate filter corresponding to the current block based on the first syntax element identification information, the reconstruction of the filter coefficient can be further combined with the coefficient value parameter and the coefficient sign parameter to obtain the filter coefficient corresponding to the current block.

[0184] Exemplarily, in some embodiments, the coefficient value parameter can be represented by a syntax element talf_coeff_abs[sfIdx][j], where talf_coeff_abs[sfIdx][j] can be used to determine the absolute value of the jth coefficient of the sfIdxth group of filters. For example, talf_coeff_abs[sfIdx][j] can be represented using a K-order exponential Golomb code, where K is 0.

[0185] It should be noted that in the embodiments of the present application, if the syntax element talf_coeff_abs[sfIdx][j] does not exist in the code stream, it can be defaulted to 0.

[0186] Exemplarily, in some embodiments, the coefficient sign parameter can be represented by a syntax element talf_coeff_sign[sfIdx][j], where talf_coeff_sign[sfIdx][j] can be used to determine the sign of the jth coefficient of the sfIdxth group of filters. For example, when the value of talf_coeff_sign[sfIdx][j] is 1, it indicates that the sign of the jth coefficient of the sfIdxth group of filters is negative, and when the value of talf_coeff_sign[sfIdx][j] is 0, it indicates that the sign of the jth coefficient of the sfIdxth group of filters is positive.

[0187] It should be noted that in the embodiments of the present application, if the syntax element talf_coeff_sign[sfIdx][j] does not exist in the code stream, it can be defaulted to 0.

[0188] That is, in the embodiments of the present application, if the candidate filter determined based on the first syntax element identification information is the sfIdxth group of filters, the reconstruction of the filter coefficients can be completed in combination with the coefficient value parameter talf_coeff_abs[sfIdx][j] and the coefficient sign parameter talf_coeff_sign[sfIdx][j].

[0189] Exemplarily, in some embodiments, based on the candidate filter corresponding to the current block, the filter coefficient absolute value and the filter

[0190] The coefficient sign reconstruction filter process is as follows:

[0191] Wherein, the numCoeff variable represents the number of filter coefficients in the filter.

[0192] Step 1002, determining at least one reference position corresponding to the current position according to the vector information corresponding to the current position.

[0193] In embodiments of the present application, at least one reference position corresponding to the current position can be determined according to the vector information corresponding to the current position.

[0194] It should be noted that in embodiments of the present application, the vector information can include a motion vector (MV) and a block vector (BV). Different methods are used to determine at least one reference position corresponding to different types of vector information.

[0195] It can be understood that in embodiments of the present application, the loop filtering link is performed on the reconstructed image after the current image has been reconstructed, so when the temporal loop filtering is performed, it can be determined that the coding block where the current position is located has selected intra prediction, inter prediction or IBC prediction. For example, in the ECM reference software, the block to which the current position belongs is an intra prediction block, if the selected is an intra TMP prediction mode, there will be one or more saved BVs in the motion cache; if the selected is an IBC prediction mode, there will also be one or more saved BVs in the motion cache; if the selected is an inter prediction mode, there will be one or more saved MVs in the motion cache.

[0196] Further, in embodiments of the present application, if the vector information corresponding to the current position includes at least one motion vector corresponding to the current position, when determining at least one reference position corresponding to the current position according to the vector information corresponding to the current position, at least one inter frame reference image corresponding to the current image can be determined according to the at least one motion vector; and then in the case that the at least one inter frame reference image exists in the reference image list corresponding to the current image, the at least one reference position corresponding to the current position is determined in the at least one inter frame reference image according to the at least one motion vector.

[0197] Further, in embodiments of the present application, when determining at least one inter frame reference image corresponding to the current image according to the at least one motion vector, the filtering mode corresponding to the current image can be determined first, and then at least one inter frame reference image corresponding to the current image is determined according to the filtering mode corresponding to the current image and the at least one motion vector.

[0198] Further, in embodiments of the present application, after determining at least one inter frame reference image corresponding to the current image according to the at least one motion vector, in the case that the at least one inter frame reference image does not exist in the reference image list corresponding to the current image, TALF filtering of the current position can be skipped.

[0199] Exemplarily, in some embodiments, if the filter mode corresponding to the current picture is denoted as talf_filter_mode, when determining the at least one reference position corresponding to the current position, if talf_filter_mode is 0, it is determined whether the prediction block to which the current position belongs selects MV0 pointing to a reference picture in the reference picture list 0, if yes, the reference picture rec0 pointed to by MV0 is used as input, otherwise, the filtering of the current position is skipped; if talf_filter_mode is 1, it is determined whether the prediction block to which the current position belongs selects MV1 pointing to a reference picture in the reference picture list 1, if yes, the reference picture rec1 pointed to by MV1 is used as input, otherwise, the filtering of the current position is skipped; if talf_filter_mode is 2, it is determined whether the prediction block to which the current position belongs selects both MV0 pointing to a reference picture in the reference picture list 0 and MV1 pointing to a reference picture in the reference picture list 1, if yes, the reference picture rec0 pointed to by MV0 and the reference picture rec1 pointed to by MV1 are used as input, otherwise, the filtering of the current position is skipped.

[0200] Further, in the embodiments of the present application, if the vector information corresponding to the current position includes at least one block vector corresponding to the current position, when determining the at least one reference position corresponding to the current position according to the vector information corresponding to the current position, the at least one reference position corresponding to the current position in the current picture can be directly determined according to the at least one block vector.

[0201] It can be understood that, in the embodiments of the present application, whether to perform filtering is determined according to the control identifier of the CTU, the selected filter mode, and whether the MV0 and MV1 or BV0 and BV1 pointing to the reference picture list 0 and the reference picture list 1 are available for motion buffer acquisition.

[0202] Exemplarily, in some embodiments, if the MV0 exists, the CTU control identifier indicates that filtering is turned on, and the selected filter mode is forward filtering, the current position can be filtered.

[0203] Exemplarily, in some embodiments, if the BV0 exists, the CTU control identifier indicates that filtering is turned on, and the selected filter mode is forward filtering, the current position can be filtered.

[0204] Exemplarily, in some embodiments, if the MV1 exists, the CTU control identifier indicates that filtering is turned on, and the selected filter mode is backward filtering, the current position can be filtered.

[0205] Exemplarily, in some embodiments, if the BV1 exists, the CTU control identifier indicates that filtering is turned on, and the selected filter mode is backward filtering, the current position can be filtered.

[0206] Exemplarily, in some embodiments, if MV0 and MV1 exist, the CTU control flag indicates that the filter is turned on, and the selected one is the bi-directional filter, the current position can be filtered.

[0207] Exemplarily, in some embodiments, if BV0 and BV1 exist, the CTU control flag indicates that the filter is turned on, and the selected one is the bi-directional filter, the current position can be filtered.

[0208] Further, in the embodiments of the present application, a flag can be introduced to further control whether to use the vector information assisted TALF filter. For example, the auxiliary filter identification information can be transmitted in the code stream, and the auxiliary filter identification information is used to determine whether to use the vector information assisted TALF filter.

[0209] Further, in the embodiments of the present application, the code stream can be decoded to determine the auxiliary filter identification information; in the case of determining to use the vector information assisted TALF filter based on the auxiliary filter identification information, the determination process of the at least one reference position is performed, that is, the scheme of step 1002 is performed.

[0210] Further, in the embodiments of the present application, the auxiliary filter identification information can be used to determine whether to allow the current position to use the vector information assisted TALF filter. The auxiliary filter identification information can be a flag.

[0211] It should be noted that, in the embodiments of the present application, the auxiliary filter identification information can be a picture level flag, a slice level (slice level) flag, a sub-picture level flag, or a tile level flag. The present application does not make specific limitations.

[0212] Further, in the embodiments of the present application, whether to use the vector information assisted TALF filter can be determined by the value of the auxiliary filter identification information.

[0213] Exemplarily, in some embodiments, in the case of the value of the auxiliary filter identification information being 0, it is determined not to use the vector information assisted TALF filter. In the case of the value of the auxiliary filter identification information being 1, it is determined to use the vector information assisted TALF filter.

[0214] Exemplarily, in some embodiments, in the case of the value of the auxiliary filter identification information being false, it is determined not to use the vector information assisted TALF filter. In the case of the value of the auxiliary filter identification information being true, it is determined to use the vector information assisted TALF filter.

[0215] It should be noted that in the embodiments of the present application, the value of the auxiliary filtering identification information can be in the form of a parameter or in the form of a number. In general, the auxiliary filtering identification information can be a parameter written in a slice header or a picture header, which is not limited herein.

[0216] Suppose the auxiliary filtering identification information is a picture-level flag, if the value of the auxiliary filtering identification information is 1, it can be determined that the current picture allows the use of vector information assisted TALF filtering. Otherwise, if the value of the auxiliary filtering identification information is 0, it can be determined that the current picture does not allow the use of vector information assisted TALF filtering.

[0217] Exemplarily, in some embodiments, the auxiliary filtering identification information can be represented by a syntax element talf_mv_guided_flag, i.e., talf_mv_guided_flag indicates whether the current position allows the use of vector information assisted TALF filtering. If the value of talf_mv_guided_flag is 0, it is determined that the current position does not allow the use of vector information assisted TALF filtering, and if the value of talf_mv_guided_flag is 1, it is determined that the current position allows the use of vector information assisted TALF filtering.

[0218] Of course, the value of talf_mv_guided_flag is not limited to 0 and 1, and the present application does not make specific limitations.

[0219] Exemplarily, one implementation is that the parsed slice-level syntax element needs to additionally include an identification indicating whether it is MV assisted TALF, and the parsed slice-level syntax element is as follows:

[0220] Wherein, talf_mv_guided_flag is used to identify whether the TALF under the current slice is based on motion vector or block vector assistance for filtering, when the value of the identification is 1, the temporal adaptive loop filter of the current slice is based on motion vector block vector (vector information) assistance; when the value of the identification is 0, the temporal adaptive loop filter of the current slice is not based on motion vector block vector assistance (vector information) assistance. When the syntax element does not exist in the code stream, its value can be defaulted to 0.

[0221] Further, in embodiments of the present application, when the at least one reference position corresponding to the current position in the at least one inter-frame reference image is determined according to the at least one motion vector, at least one first position offset value in a first direction and at least one second position offset value in a second direction can be determined according to the at least one motion vector first; and then the at least one reference position in the at least one inter-frame reference image is determined according to the at least one first position offset value and the at least one second position offset value.

[0222] It should be noted that in embodiments of the present application, the first offset value is used for position offset in the first direction, and the second offset value is used for position offset in the second direction. Wherein the first direction and the second direction can be horizontal direction and vertical direction respectively.

[0223] Exemplarily, in some embodiments, if MV0 and MV1 are found, since the MV used in inter-frame prediction is sub-pixel level accuracy, here the position offset on the reconstructed image pointed by the MV is obtained by rounding to the nearest pixel accuracy. Wherein the horizontal direction integer pixel position offset of MV0 is: Offset0X = Mv0.Hor < 0? - ((abs(Mv0.Hor) + 8) >> 4) : ((abs(Mv0.Hor) + 8) >> 4); the vertical direction integer pixel position offset of MV0 is: Offset0Y = Mv0.Ver < 0? - ((abs(Mv0.Ver) + 8) >> 4) : ((abs(Mv0.Ver) + 8) >> 4); the horizontal direction integer pixel position offset of MV1 is: Offset1X = Mv1.Hor < 0? - ((abs(Mv1.Hor) + 8) >> 4) : ((abs(Mv1.Hor) + 8) >> 4); and the vertical direction integer pixel position offset of MV1 is: Offset1Y = Mv1.Ver < 0? - ((abs(Mv1.Ver) + 8) >> 4) : ((abs(Mv1.Ver) + 8) >> 4).

[0224] Further, in embodiments of the present application, after determining the auxiliary filter identification information, in the case of determining not to use vector information assisted TALF filtering based on the auxiliary filter identification information, at least one inter-frame reference image corresponding to the current image and the filter coefficient are determined; the filter is performed based on the reconstructed samples of the at least one inter-frame reference image and the filter coefficient, to determine the filtered reconstructed value of the current position.

[0225] It can be understood that in the embodiments of the present application, if it is determined that the TALF filtering is not assisted by the vector information, at least one inter-frame reference picture corresponding to the current picture and the filter coefficients can be determined according to the conventional TALF filtering method, and then the TALF filtering is performed based on the at least one inter-frame reference picture and the filter coefficients.

[0226] That is to say, in the embodiments of the present application, if the TALF filtering is not assisted by the vector information corresponding to the current position in the current picture, at least one inter-frame reference picture corresponding to the current picture can be further determined.

[0227] Therefore, in the embodiments of the present application, the manner of determining the reference picture can be different, and the finally determined reference picture can also be different, for different scenarios of using the vector information corresponding to the current position to assist the TALF filtering and not using the vector information corresponding to the current position to assist the TALF filtering.

[0228] Exemplarily, in some embodiments, if the TALF filtering is not assisted by the vector information, the input picture of the filter determined according to the conventional TALF filtering procedure can be the inter-frame reference picture, and then the filtering processing can be performed according to the reconstructed samples in the at least one inter-frame reference picture; if the TALF filtering is assisted by the vector information, the input picture of the filter determined can be the inter-frame reference picture, or can be the current picture itself, and then the filtering processing can be performed according to the reconstructed samples in the at least one inter-frame reference picture, or according to the reconstructed samples in the current picture.

[0229] Exemplarily, in some embodiments, the filter mode corresponding to the current image is denoted as talf_filter_mode, when determining the input image of the filter according to talf_mv_guided_flag and talf_filter_mode, if talf_filter_mode is 0 and talf_mv_guided_flag is 0, the forward nearest reconstructed image rec0 is used as the input; if talf_filter_mode is 0 and talf_mv_guided_flag is 1, it is confirmed whether the prediction block to which the current position belongs selects the MV0 pointing to the reference image in the reference image list 0, if yes, the reference image rec0 pointed by the MV0 is used as the input, otherwise, the filtering of the current position is skipped; if talf_filter_mode is 1 and talf_mv_guided_flag is 0, the backward nearest reconstructed image rec1 is used as the input under RA, and the forward second nearest reconstructed image rec1 is used as the input under LD; if talf_filter_mode is 1 and talf_mv_guided_flag is 1, it is confirmed whether the prediction block to which the current position belongs selects the MV1 pointing to the reference image in the reference image list 1, if yes, the reference image rec1 pointed by the MV1 is used as the input, otherwise, the filtering of the current position is skipped; if talf_filter_mode is 2 and talf_mv_guided_flag is 0, the forward nearest reconstructed image rec0 and the backward nearest reconstructed image rec1 are used as the input under RA, and the forward nearest reconstructed image rec0 and the second nearest reconstructed image rec1 are used as the input under LD; if talf_filter_mode is 2 and talf_mv_guided_flag is 1, it is confirmed whether the prediction block to which the current position belongs selects the Mv0 pointing to the reference image in the reference image list 0 and the MV1 pointing to the reference image in the reference image list 1, if yes, the reference image rec0 pointed by the MV0 and the reference image rec1 pointed by the MV1 are used as the input, otherwise, the filtering of the current position is skipped.

[0230] Further, in the embodiments of the present application, if talf_mv_guided_flag is 0, Offset0X, Offset0Y, Offset1X and Offset1Y are set to 0.

[0231] Further, in the embodiments of the present application, if the TALF filtering is not assisted by the vector information, according to the conventional TALF filtering procedure, when determining the at least one inter-frame reference picture corresponding to the current picture and the filter coefficients corresponding to the current block, the TALF parameters corresponding to the current block can be determined first; wherein the TALF parameters include one or more of a mode parameter, a number parameter, a coefficient value parameter and a coefficient sign parameter; then the filtering mode and the number of filter groups are determined according to the TALF parameters; the at least one inter-frame reference picture can be determined according to the filtering mode; and the filter coefficients corresponding to the current block can be determined according to the number of filter groups, the coefficient value parameter, the coefficient sign parameter and the first syntax element identification information.

[0232] It can be understood that in the embodiments of the present application, the mode parameter can be used to determine the filtering mode of the TALF filtering, wherein the determined inter-frame reference picture corresponding to the current picture can be different for different filtering modes. The number parameter can be used to determine the number of filter groups corresponding to the current picture. The coefficient value parameter can be used to determine the absolute value size of the filter coefficients. The coefficient sign parameter can be used to determine the sign of the filter coefficients, including positive and negative.

[0233] Further, in the embodiments of the present application, when determining the TALF parameters corresponding to the current block, in the case of determining that the current picture uses TALF filtering based on the second syntax element identification information, the code stream can be decoded to determine the APS index; then the APS set can be determined according to the APS index, and the TALF parameters are determined based on the APS set.

[0234] It can be understood that in the embodiments of the present application, the APS index can be used to determine the APS parameter set corresponding to the current picture, that is, the APS set corresponding to the current picture can be determined through the APS index.

[0235] Exemplarily, in some embodiments, in the case of determining that the current picture uses TALF filtering based on the second syntax element identification information, the code stream can be further decoded to determine the corresponding APS index ph_talf_aps_id, which can be used to indicate where the adaptive parameters (TALF parameters) of the TALF corresponding to the current picture are obtained in the APS parameter set when using the TALF.

[0236] Exemplarily, in some embodiments, in the case that the current slice is determined to use TALF for filtering based on the second syntax element, the bitstream can be further decoded to determine a corresponding APS index sh_talf_aps_id, which can be used to indicate in which APS parameter set the adaptive parameters (TALF parameters) of the TALF corresponding to the current slice are obtained when TALF is used.

[0237] That is, in the embodiments of the present application, the TALF parameters corresponding to the current image can be saved in the APS, so that the TALF parameters can be obtained by parsing in the APS.

[0238] Exemplarily, in some embodiments, the implementation of determining the TALF parameters by parsing the APS is as follows:

[0239] Wherein, alf_data() is a function of parsing ALF, CCALF related syntax elements and parameters. The syntax elements and parameters of TALF can also be further parsed in alf_data().

[0240] Exemplarily, in some embodiments, the implementation of determining the TALF parameters by parsing alf_data() is as follows:

[0241] Wherein, talf_filter_signal_flag is used to identify whether there is a TALF parameter in the parameters of the current alf_data.

[0242] talf_filter_mode is a mode parameter, which is used to indicate which mode the TALF belongs to among the forward / backward / bidirectional modes if there is a TALF parameter.

[0243] talf_num_filters_signalled_minus1 is a number parameter, which is used to indicate the number of filter groups. For example, if there is a TALF parameter, the value of talf_num_filters_signalled_minus1 is the number of TALF filters coded in the current APS minus one.

[0244] talf_coeff_abs is a coefficient value parameter, which is used to indicate the absolute value of the coefficient in each coded filter.

[0245] talf_coeff_sign is a coefficient sign parameter, which is used to indicate the positive or negative of the non-zero coefficient in each coded filter.

[0246] The numCoeff variable represents the number of filter coefficients in the filter.

[0247] Further, in embodiments of the present application, when determining the TALF parameters corresponding to the current block, the TALF parameters can be decoded from the bitstream in the case that it is determined based on the second syntax element identification information that the current picture uses TALF for filtering.

[0248] That is, in embodiments of the present application, the TALF parameters corresponding to the current picture can be directly written into the bitstream, so that the TALF parameters can be obtained by decoding the bitstream. For example, the TALF parameters can be coded in the picture header or slice header.

[0249] Exemplarily, in some embodiments, the method of coding the TALF identification and TALF parameters in the picture header is as follows:

[0250] Further, in embodiments of the present application, when determining the filter mode and the number of filter groups according to the TALF parameters, the current configuration can be determined first; in the case that the current configuration is a random access (RA) configuration, the filter mode is determined in a first candidate mode according to the mode parameters; wherein the first candidate mode includes one or more of a forward filter mode, a backward filter mode and a bidirectional filter mode; in the case that the current configuration is a low delay (LD) configuration, the filter mode is determined in a second candidate mode according to the mode parameters; wherein the second candidate mode includes one or more of a first forward filter mode, a second forward filter mode and a third forward filter mode.

[0251] It should be noted that, in embodiments of the present application, the filter mode corresponding to the current picture can be determined in combination with the current configuration and the mode parameters in the TALF parameters. Wherein, the determination of the filter mode can be based on the first candidate mode or the second candidate mode according to the current configuration, and then the corresponding filter mode can be selected from the first candidate mode or the second candidate mode according to the mode parameters.

[0252] It can be understood that, in embodiments of the present application, for the RA configuration, the corresponding first candidate mode can include one or more of the forward filter mode, the backward filter mode and the bidirectional filter mode. Wherein, the forward filter mode can be understood as using the reconstructed information of the reconstructed picture with smaller POC than the current picture as the input of the filter, the backward filter mode can be understood as using the reconstructed information of the reconstructed picture with larger POC than the current picture as the input of the filter, and the bidirectional filter mode can be understood as using the reconstructed information of the reconstructed picture with smaller POC and the reconstructed information of the reconstructed picture with larger POC as the input of the filter.

[0253] It can be understood that, in the embodiments of the present application, for the LD configuration, the corresponding second candidate mode can include one or more of the first forward filtering mode, the second forward filtering mode and the third forward filtering mode. Wherein, for the coding under the LD configuration, the coding order is the same as the playing order, so only the reference image smaller than the current image POC can be used as the input of the filter, that is, only the forward filtering mode is supported. However, corresponding to different inter-frame reference images, the corresponding forward filtering mode can be distinguished into the first forward filtering mode, the second forward filtering mode and the third forward filtering mode.

[0254] Further, in the embodiments of the present application, after determining the filtering mode corresponding to the current image, at least one inter-frame reference image can be further determined according to the filtering mode.

[0255] It can be understood that, in the embodiments of the present application, the determination of the inter-frame reference image is dependent on the filtering mode, and for different filtering modes, the finally determined inter-frame reference image can be different.

[0256] Exemplarily, in some embodiments, if the POC value of the current image is N, for the RA configuration, if the determined filtering mode is the forward filtering mode, the image with the POC value of N-1 can be selected as the inter-frame reference image of the current image; if the determined filtering mode is the reverse filtering mode, the image with the POC value of N+1 can be selected as the inter-frame reference image of the current image; if the determined filtering mode is the bidirectional filtering mode, the images with the POC values of N-1 and N+1 can be selected as the inter-frame reference images of the current image.

[0257] Exemplarily, in some embodiments, if the POC value of the current image is N, for the LD configuration, the images with the POC values of N-1, N-2… can be used as the inter-frame reference images of the current image. For example, if the determined filtering mode is the first forward filtering mode, the image with the POC value of N-1 can be selected as the inter-frame reference image of the current image; if the determined filtering mode is the second forward filtering mode, the image with the POC value of N-2 can be selected as the inter-frame reference image of the current image; if the determined filtering mode is the third forward filtering mode, the images with the POC values of N-1 and N-2 can be selected as the inter-frame reference images of the current image.

[0258] That is, in the embodiments of the present application, the POC values of the inter-frame reference images determined based on different filter modes can be different, and the number of the determined inter-frame reference images can also be different. For example, for the forward filter mode, the backward filter mode in the RA configuration, and the first forward filter mode (the nearest first frame) and the second forward filter mode (the nearest second frame) in the LD configuration, the current image corresponds to one inter-frame reference image; for the bidirectional filter mode in the RA configuration, and the third forward filter mode (the nearest first frame and the nearest second frame) in the LD configuration, the current image corresponds to two inter-frame reference images.

[0259] It can be seen that, for the time-domain adaptive loop filter type, the intra-frame encoded image has no reference image, and the inter-frame encoded image has a reference image, so the time-domain filter of the present scheme is applied to the inter-frame encoded image, for example, the inter-frame encoded image under the encoding type of the RA and LD configurations. For the coding under the LD configuration, the coding sequence is the same as the playing sequence, so when the time-domain adaptive filter is used for the inter-frame encoded image, the reference image with a smaller POC than the current image is always used as the input of the filter. For the coding under the RA configuration, the coding sequence can be different from the playing sequence, and when the time-domain filter is used for the current inter-frame encoded image, the reference frame with a larger or smaller POC than the current frame can be used as the input of the filter.

[0260] The forward filter uses the reconstructed information of the reconstructed image with a smaller POC than the current image as the input of the filter to generate the filter value acting on the current reconstructed image. For example, under the LD configuration, the POC of the current image is N, N is a non-negative integer, and the reconstructed value of the reconstructed image with the POC value of N-1, N-2, … can be used as the input of the filter. For example, under the RA configuration, under the coding sequence, when the POC of the current image is N, the inter-frame encoded image with the TID of 5 can use the reconstructed value of the image with the POC value of N-1 as the input of the filter; the inter-frame encoded image with the TID of 4 can use the reconstructed value of the image with the POC value of N-2 as the input of the filter; the inter-frame encoded image with the TID of 3 can use the reconstructed value of the image with the POC value of N-4 as the input of the filter; the inter-frame encoded image with the TID of 2 can use the reconstructed value of the image with the POC value of N-8 as the input of the filter; the inter-frame encoded image with the TID of 1 can use the reconstructed value of the image with the POC value of N-16 as the input of the filter; and the inter-frame encoded image with the TID of 0 can use the reconstructed value of the image with the POC value of N-32 as the input of the filter.

[0261] It should be noted that, in the embodiments of the present application, the POC value of the reference frame used in the forward filter changes due to different encoding configurations.

[0262] Reverse filtering is to use the reconstructed information of the reconstructed image with a larger POC than the current image as the input of the filter to generate the filtered value acting on the current reconstructed image. In the current various coding configurations, reverse filtering is only used in the RA configuration. For example, in the RA configuration, when the POC of the current image is N in the coding order, the inter-coded image with TID of 5 can use the reconstructed value of the image with POC value of N+1 as the input of the filter; the TID of 4 can use the reconstructed value of the image with POC value of N+2 as the input of the filter; the TID of 3 can use the reconstructed value of the image with POC value of N+4 as the input of the filter; the TID of 2 can use the image with POC value of N+8 as the input of the filter; the TID of 1 can use the reconstructed value of the image with POC value of N+16 as the input of the filter; and the TID of 0 can use the reconstructed value of the image with POC value of N+32 as the input of the filter.

[0263] It should be noted that in the embodiments of the present application, the POC value of the reference frame used in reverse filtering will change due to different coding configurations.

[0264] Bidirectional filtering is to use the reconstructed information of the image with a smaller POC than the current image and the image with a larger POC than the current image as the input of the filter to generate the filtered value acting on the current reconstructed image. In the current various coding configurations, bidirectional filtering is only used in the RA configuration. For example, in the RA configuration, when the POC of the current image is N in the coding order, the inter-coded image with TID of 5 can use the reconstructed value of the image with POC value of N+1 and N-1 as the input of the filter; the TID of 4 can use the reconstructed value of the image with POC value of N+2 and N-2 as the input of the filter; the TID of 3 can use the reconstructed value of the image with POC value of N+4 and N-4 as the input of the filter; the TID of 2 can use the image with POC value of N+8 and N-8 as the input of the filter; the TID of 1 can use the reconstructed value of the image with POC value of N+16 and N-16 as the input of the filter; and the TID of 0 can use the reconstructed value of the image with POC value of N+32 and N-32 as the input of the filter.

[0265] It should be noted that in the embodiments of the present application, the POC value of the reference frame used in bidirectional filtering will change due to different coding configurations. In principle, in the RA configuration, bidirectional filtering always uses the reconstructed values of the two reconstructed images closest in forward and backward distance as the input of the filter.

[0266] Exemplarily, in some embodiments, assuming that the filtering mode is a forward filtering mode, the following forward filtering modes can be included but are not limited to the following:

[0267] 1. When in the RA configuration, the reference picture with the closest time distance and smaller POC than the current picture is used as the input of the temporal filter;

[0268] 2. When in the LD configuration, the reference picture with POC value of N-1 is used as the input of the temporal filter corresponding to the first forward filter mode when the current picture has POC value of N;

[0269] 3. The reference picture with POC value of N-2 is used as the input of the temporal filter corresponding to the second forward filter mode when the current picture has POC value of N in the LD configuration;

[0270] 4. The two reference pictures with POC values of N-2 and N-1 are used as the input of the temporal filter corresponding to the third forward filter mode when the current picture has POC value of N in the LD configuration.

[0271] Exemplarily, in some embodiments, when the current picture corresponds to one inter-reference picture, for example, the forward filter mode, the backward filter mode in the RA configuration, and the first forward filter mode (the nearest first frame), the second forward filter mode (the nearest second frame) in the LD configuration, a 7x7 symmetrical filter with 13 filter coefficients can be selected, and the value of the numCoeff variable can be 13; when the current picture corresponds to two inter-reference pictures, for example, the bidirectional filter mode in the RA configuration, and the third forward filter mode (the nearest first frame and the nearest second frame) in the LD configuration, two 5x5 symmetrical filters with 7 filter coefficients can be selected, and the value of the numCoeff variable can be 14.

[0272] It should be noted that in the embodiments of the present application, the image-level control identifier can be applied to the slice level, the sub-picture level, and the tile level, that is, the image-level syntax element identifier information can be replaced by the slice-level syntax element identifier information, the sub-picture-level syntax element identifier information, and the tile-level syntax element identifier information, and the type and form of the control identifier are not limited.

[0273] Step 1003, filtering based on the reconstructed samples of the at least one reference position and the filter coefficients to determine the filtered reconstructed value of the current position.

[0274] In the embodiments of the present application, after determining the at least one reference position corresponding to the current position according to the vector information corresponding to the current position, filtering can be further performed based on the reconstructed samples of the at least one reference position and the filter coefficients, so that the filtered reconstructed value of the current position can be determined.

[0275] It can be understood that in the embodiments of the present application, the filtering process using vector information to assist TALF filtering can also be understood as filtering the (x, y) position (current position) according to the obtained position offset (first position offset value and second position offset value), the reconstructed image (inter-frame reference image or current image), and the filter coefficient.

[0276] Further, in the embodiments of the present application, when filtering based on the reconstructed samples of the at least one reference position and the filter coefficient to determine the filtered reconstructed value of the current position, the reconstructed sample value of the current position and the reconstructed sample value of the reference position can be determined first; then according to the filter coefficient, the reconstructed sample value of the current position, and the reconstructed sample value of the reference position, the filtered reconstructed value of the current position is determined.

[0277] Further, in the embodiments of the present application, when filtering based on the reconstructed samples of the at least one reference position and the filter coefficient to determine the filtered reconstructed value of the current position, the reconstructed sample value of the current position, the reconstructed sample value of the first reference position, and the reconstructed sample value of the second reference position can be determined first; then according to the filter coefficient, the reconstructed sample value of the current position, the reconstructed sample value of the first reference position, and the reconstructed sample value of the second reference position, the filtered reconstructed value of the current position is determined.

[0278] Exemplarily, in some embodiments, assuming numCoeff is 14: x0 = Clip3(0, picWidth, x + Offset0X) y0 = Clip3(0, picHeight, y + Offset0Y) x1 = Clip3(0, picWidth, x + Offset1X) y1 = Clip3(0, picHeight, y + Offset1Y) filterldx = talf_ctb_idx - 1 sum = tAlfCoeff[filterldx][0]*(rec0[x0][y0] - rec[x][y]) + tAlfCoeff[filterldx][1]*(rec1[x1][y1] - rec[x][y]) + tAlfCoeff[filterldx][2]*(rec0[x0-1][y0] + rec0[x0+1][y0] - 2*rec[x][y]) + tAlfCoeff[filterldx][3]*(rec1[x1-1][y1] + rec1[x1+1][y1] - 2*rec[x][y]) + tAlfCoeff[filterldx][4]*(rec0[x0][y0-1] + rec0[x0][y0+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][5]*(rec1[x1][y1-1] + rec1[x1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][6]*(rec0[x0+1][y0-1] + rec0[x0-1][y0+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][7]*(rec1[x1+1][y1-1] + rec1[x1-1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][8]*(rec0[x0-1][y0-1] + rec0[x0+1][y0+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][9]*(rec1[x1-1][y1-1] + rec1[x1+1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0010] *(rec0[x0-2][y0] + rec0[x0+2][y0] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0011] *(rec1[x1-2][y1] + rec1[x1+2][y1] - 2*rec[x][y]) +tAlfCoeff[filterldx]

[0012] *(rec0[x0][y0-2]+rec0[x0][y0+2]-2*rec[x][y])+ tAlfCoeff[filterldx]

[0013] *(rec1[x1][y1-2]+rec1[x1][y1+2]-2*rec[x][y]) signSum = sum < 0? -1 : 1 scaledAbsSum = ( abs(sum) + (1 << (shift - 1)) ) » shift scaledSum = scaledAbsSum * signSum rec'[x][y] = max( min( rec[x][y] + scaledSum, (1 « BitDepth) - 1 ), 0 )

[0279] In the above process of calculating the filter, shift is 6, BitDepth is the bit depth of the luma component, x and y represent the horizontal and vertical coordinates of the reconstructed image array. x0, y0, x1, y1 are the coordinates of the center positions of the offset rec0 and rec1, respectively. When the reconstructed values in the coordinate positions rec0, rec1 array are obtained, the horizontal coordinate should be limited to 0 to the image width picWidth-1, and the vertical coordinate should be limited to 0 to the image height picHeight-1.

[0280] Exemplarily, in some embodiments, assuming numCoeff is 13 and talf_filter_mode is 0: x0 = Clip3(0, picWidth, x + Offset0X) y0 = Clip3(0, picHeight, y + Offset0Y) filterldx = talf_ctb_idc - 1 sum = tAlfCoeff[filterldx][0]*(rec0[x0][y0] - rec[x][y]) + tAlfCoeff[filterldx][1]*(rec0[x0-1][y0] + rec0[x0+1][y0] - 2*rec[x][y]) + tAlfCoeff[filterldx][2]*(rec0[x0][y0-1] + rec0[x0][y0+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][3]*(rec0[x0+1][y0-1] + rec0[x0-1][y0+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][4]*(rec0[x0-1][y0-1] + rec0[x0+1][y0+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][5]*(rec0[x0-2][y0] + rec0[x0+2][y0] - 2*rec[x][y]) + tAlfCoeff[filterldx][6]*(rec0[x0][y0-2] + rec0[x0][y0+2] - 2*rec[x][y]) + tAlfCoeff[filterldx][7]*(rec0[x0-2][y0-1] + rec0[x0+2][y0+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][8]*(rec0[x0-2][y0+1] + rec0[x0+2][y0-1] - 2*rec[x][y]) + tAlfCoeff[filterldx][9]*(rec0[x0-1][y0-2] + rec0[x0+1][y0+2] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0010] *(rec0[x0-1][y0+2] + rec0[x0+1][y0-2] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0011] *(rec0[x0-3][y0] + rec0[x0+3][y0] - 2*rec[x][y]) +tAlfCoeff[ filterldx ] [ 0 ] * ( rec0[ x0 ][ y0 - 3 ] + rec0[ x0 ][ y0 + 3 ] - 2 * rec[ x ][ y ] ) signSum = sum < 0? -1 : 1 scaledAbsSum = ( abs( sum ) + ( 1 << ( shift - 1 ) ) ) » shift scaledSum = scaledAbsSum * signSum rec' [ x ][ y ] = max( min( rec[ x ][ y ] + scaledSum, ( 1 << BitDepth ) - 1 ), 0 )

[0281] numCoeff is 13 and talf_filter_mode is 1: x1 = Clip3(0, picWidth, x + Offset1X) y1 = Clip3(0, picHeight, y + Offset1Y) filterldx = talf_ctb_idc - 1 sum = tAlfCoeff[filterldx][0] * (rec1[x1][y1] - rec[x][y]) + tAlfCoeff[filterldx][1] * (rec1[x1-1][y1] + rec1[x1+1][y1] - 2*rec[x][y]) + tAlfCoeff[filterldx][2] * (rec1[x1][y1-1] + rec1[x1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][3] * (rec1[x1+1][y1-1] + rec1[x1-1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][4] * (rec1[x1-1][y1-1] + rec1[x1+1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][5] * (rec1[x1-2][y1] + rec1[x1+2][y1] - 2*rec[x][y]) + tAlfCoeff[filterldx][6] * (rec1[x1][y1-2] + rec1[x1][y1+2] - 2*rec[x][y]) + tAlfCoeff[filterldx][7] * (rec1[x1-2][y1-1] + rec1[x1+2][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][8] * (rec1[x1-2][y1+1] + rec1[x1+2][y1-1] - 2*rec[x][y]) + tAlfCoeff[filterldx][9] * (rec1[x1-1][y1-2] + rec1[x1+1][y1+2] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0010] * (rec1[x1-1][y1+2] + rec1[x1+1][y1-2] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0011] * (rec1[x1-3][y1] + rec1[x1+3][y1] - 2*rec[x][y]) +tAlfCoeff[ filterldx ][ 0 ][ 0 ] * ( rec1[ x1 ][ y1 - 3 ] + rec1[ x1 ][ y1 + 3 ] - 2 * rec[ x ][ y ] ) signSum = sum < 0? -1 : 1 scaledAbsSum = ( abs( sum ) + ( 1 << ( shift - 1 ) ) ) » shift scaledSum = scaledAbsSum * signSum rec' [ x ][ y ] = max( min( rec[ x ][ y ] + scaledSum, ( 1 << BitDepth ) - 1 ), 0 )

[0282] Further, in embodiments of the present disclosure, if it is determined that the TALF filter is not assisted by the vector information, the filtered reconstructed value of the current position can be determined based on the reconstructed samples of the at least one inter-coded reference picture and the filter coefficients.

[0283] It should be noted that in embodiments of the present disclosure, after the first syntax element is identified to indicate that the current block is filtered by the TALF, and the at least one inter-coded reference picture corresponding to the current picture and the filter coefficients corresponding to the current block are determined, the current block can be further filtered based on the reconstructed samples of the at least one inter-coded reference picture and the filter coefficients corresponding to the current block to determine the filtered reconstructed value of the current block.

[0284] It can be understood that in embodiments of the present disclosure, since the POC values of the inter-coded reference pictures determined based on different filter modes can be different, and the number of the determined inter-coded reference pictures can also be different, when the current block of the current picture is filtered by the TALF, the filtering processes performed corresponding to different numbers of inter-coded reference pictures are also different.

[0285] Further, in embodiments of the present disclosure, when the current block is filtered based on the reconstructed samples of the at least one inter-coded reference picture and the filter coefficients corresponding to the current block to determine the filtered reconstructed value of the current block, for a current sample position in the current block, the reconstructed sample value of the current sample position is determined according to the reconstructed value of the current block, and the reconstructed sample value of a reference sample position corresponding to the current sample position is determined according to the reconstructed samples of the inter-coded reference pictures; the filtered reconstructed sample value of the current sample position is determined according to the filter coefficients corresponding to the current block, the reconstructed sample value of the current sample position, and the reconstructed sample value of the reference sample position; and the filtered reconstructed value of the current block is determined according to the filtered reconstructed sample value of the current sample position.

[0286] It can be understood that, in the embodiments of the present application, for the forward filtering mode, the backward filtering mode in the RA configuration, and the first forward filtering mode (the nearest first frame), the second forward filtering mode (the nearest second frame) in the LD configuration, the current image corresponds to one inter-frame reference image, and when the TALF filter is used to filter the reconstructed value of the current block based on the reconstructed sample of the one inter-frame reference image, the reference sample position corresponding to the current sample position can be determined in the one inter-frame reference image, and then the reconstructed sample value of the current sample position and the reconstructed sample value of the reference sample position are input into the TALF filter, and finally the filtered reconstructed sample value of the current sample position is determined by combining the determined filter coefficient corresponding to the current block, and the filtering of the current block is completed by traversing any sample position of the current block according to the above scheme, and the filtered reconstructed value of the current block is obtained.

[0287] Exemplarily, in some embodiments, it is assumed that the current image corresponds to one inter-frame reference image, and at this time, the process of filtering the sample value at the coordinate (x, y) position (the current sample position) in the current image using 13 filter coefficients and a 7x7 symmetric filter is as follows:

[0288] Wherein, rec' (x，y) is the reconstructed sample value at the (x, y) position after the correction of the temporal adaptive filter, rec (x，y) is the reconstructed sample value at the (x, y) position to be corrected, and are the reconstructed sample values at (x-u i , y-u i ) and (x+u i , y+u i ) in the inter-frame reference image, and c is the filter coefficient of the temporal adaptive filter.

[0289] Further, in the embodiments of the present application, when the filtered reconstructed sample value of the current sample position is determined according to the filter coefficient corresponding to the current block, the reconstructed sample value of the current sample position, and the reconstructed sample value of the reference sample position, the first sample difference value can be determined according to the reconstructed sample value of the current sample position and the reconstructed sample value of the reference sample position, and then the filtered reconstructed sample value of the current sample position is determined according to the filter coefficient corresponding to the current block, the reconstructed sample value of the current sample position, and the first sample difference value.

[0290] That is, in the embodiments of the present application, the first sample difference value can also be determined according to the reconstructed sample value of the current sample position and the reconstructed sample value of the reference sample position, and then the first sample difference value is used as the input of the filter.

[0291] Exemplarily, in some embodiments, assuming that there is one inter- reference picture corresponding to the current picture, the process of filtering the sample value at the position (x, y) (current sample position) in the current picture using 13 filter coefficients and 7x7 symmetric filter is as follows:

[0292] Further, in the embodiments of the present application, when determining the filtered reconstructed sample value at the current sample position according to the filter coefficients corresponding to the current block, the reconstructed sample value at the current sample position, and the reconstructed sample value at the reference sample position, the filtering position corresponding to the reference sample position in the current block can be determined, and the reconstructed sample value at the filtering position can be determined according to the reconstructed value of the current block; the second sample difference value can be determined according to the reconstructed sample value at the filtering position and the reconstructed sample value at the reference sample position; and the filtered reconstructed sample value at the current sample position can be determined according to the filter coefficients corresponding to the current block, the reconstructed sample value at the current sample position, and the second sample difference value.

[0293] That is, in the embodiments of the present application, for each filtering position, the second sample difference value can also be determined according to the reconstructed sample value at the filtering position and the reconstructed sample value at the reference sample position, and then the second sample difference value can be taken as the input of the filter.

[0294] Exemplarily, in some embodiments, assuming that there is one inter- reference picture corresponding to the current picture, the process of filtering the sample value at the position (x, y) (current sample position) in the current picture using 13 filter coefficients and 7x7 symmetric filter is as follows:

[0295] Further, in the embodiments of the present application, when filtering the current block based on the reconstructed samples of at least one inter- reference picture and the filter coefficients corresponding to the current block to determine the filtered reconstructed value of the current block, for the current sample position in the current block, the reconstructed sample value at the current sample position can be determined according to the reconstructed value of the current block, the reconstructed sample value at the first reference sample position corresponding to the current sample position can be determined according to the reconstructed samples of the first inter- reference picture, and the reconstructed sample value at the second reference sample position corresponding to the current sample position can be determined according to the reconstructed samples of the second inter- reference picture; the filtered reconstructed sample value at the current sample position can be determined according to the filter coefficients corresponding to the current block, the reconstructed sample value at the current sample position, the reconstructed sample value at the first reference sample position, and the reconstructed sample value at the second reference sample position; and the filtered reconstructed value of the current block can be determined according to the filtered reconstructed sample value at the current sample position.

[0296] It can be understood that, in the embodiments of the present application, for the bidirectional filtering mode in the RA configuration, and the third forward filtering mode (the nearest first frame and the nearest second frame) in the LD configuration, the current picture corresponds to two inter-frame reference pictures, and when the TALF filtering is performed on the reconstructed values of the current block based on the reconstructed samples of the two inter-frame reference pictures, the reference sample positions corresponding to the current sample position can be determined in the two inter-frame reference pictures respectively, and then the reconstructed sample value corresponding to the current sample position, the reconstructed sample values corresponding to the two reference sample positions are input into the TALF filter respectively, combined with the determined filter coefficients corresponding to the current block, to finally determine the filtered reconstructed sample value corresponding to the current sample position, and the above scheme is traversed for any sample position of the current block to complete the filtering of the current block and obtain the filtered reconstructed value of the current block.

[0297] Exemplarily, in some embodiments, assuming that the current picture corresponds to two inter-frame reference pictures, at this time, the process of filtering the sample value at the coordinate (x, y) position (the current sample position) in the current picture using 7 filter coefficients and 5x5 symmetrical 2 filters is as follows:

[0298] Wherein, rec' (x，y) is the reconstructed sample value at the (x, y) position after the correction by the temporal adaptive filter, rec (x，y) is the reconstructed sample value at the (x, y) position to be corrected, ref0 and ref1 are the reconstructed values of the first inter-frame reference picture and the second inter-frame reference picture respectively. and are the reconstructed sample values at (x-u i , y-u i ) and (x+u i , y+u i ) of the first inter-frame reference picture, and are the reconstructed sample values at (x-u i , y-u i ) and (x+u i , y+u i ) of the second inter-frame reference picture, and c is the temporal adaptive filter coefficient.

[0299] Further, in the embodiments of the present application, when the filtered reconstructed sample value of the current sample position is determined according to the filter coefficient corresponding to the current block, the reconstructed sample value of the current sample position, the reconstructed sample value of the first reference sample position, and the reconstructed sample value of the second reference sample position, the third sample difference value can be determined according to the reconstructed sample value of the current sample position and the reconstructed sample value of the first reference sample position; the fourth sample difference value can be determined according to the reconstructed sample value of the current sample position and the reconstructed sample value of the second reference sample position; and the filtered reconstructed sample value of the current sample position can be determined according to the filter coefficient corresponding to the current block, the reconstructed sample value of the current sample position, the third sample difference value, and the fourth sample difference value.

[0300] That is, in the embodiments of the present application, the third sample difference value and the fourth sample difference value can also be determined according to the reconstructed sample value of the current sample position and the reconstructed sample value of the two reference sample positions respectively, and then the third sample difference value and the fourth sample difference value can be taken as the input of the filter.

[0301] Exemplarily, in some embodiments, assuming that the current image corresponds to two inter-frame reference images, the process of filtering the sample value at the coordinate (x, y) position (the current sample position) in the current image using 7 filter coefficients and 2 filters of 5x5 symmetry is as follows:

[0302] Further, in the embodiments of the present application, when the filtered reconstructed sample value of the current sample position is determined according to the filter coefficient corresponding to the current block, the reconstructed sample value of the current sample position, the reconstructed sample value of the first reference sample position, and the reconstructed sample value of the second reference sample position, the filter position corresponding to the reference sample position in the current block can be determined, and the reconstructed sample value of the filter position can be determined according to the reconstructed value of the current block; the fifth sample difference value can be determined according to the reconstructed sample value of the filter position and the reconstructed sample value of the first reference sample position; the sixth sample difference value can be determined according to the reconstructed sample value of the filter position and the reconstructed sample value of the second reference sample position; and the filtered reconstructed sample value of the current sample position can be determined according to the filter coefficient corresponding to the current block, the reconstructed sample value of the current sample position, the fifth sample difference value, and the sixth sample difference value.

[0303] That is, in the embodiments of the present application, for each filter position, the fifth sample difference value and the sixth sample difference value can also be determined according to the reconstructed sample value of the filter position and the reconstructed sample value of the two reference sample positions respectively, and then the fifth sample difference value and the sixth sample difference value can be taken as the input of the filter.

[0304] Exemplarily, in some embodiments, assuming that there are two inter- frame reference pictures corresponding to the current picture, the filtering process of the sample value at the coordinate (x, y) position (current sample position) in the current picture using 7 filter coefficients and 5x5 symmetrical 2 filters is as follows:

[0305] It should be noted that in the embodiments of the present application, the filtering process shown in the above formula is an exemplary description of the TALF filtering proposed in the present application, and of course, in the actual application process, if the adaptive filter does not have the symmetry of the above example, the adaptive filter coefficients and the input values corresponding to each coefficient should also be adjusted accordingly. That is, the filtering process should be a process in which the adaptive filter coefficients and the reconstructed values of the reference picture at each related position act together.

[0306] It should be noted that in the embodiments of the present application, the filter coefficients can be integers, and the parameters used in the filtering process can also include bias and shift.

[0307] Further, in the embodiments of the present application, for the TALF filter, in addition to the at least one inter-frame reference picture that has been decoded in the time domain, the corresponding input information can also include the reconstructed value of the current block, for example, the reconstructed value of any one image component of the current block. Wherein, the reconstructed value of the current block can be obtained after processing the current block in the current picture based on any loop filtering method.

[0308] That is, in the embodiments of the present application, the determination method of the reconstructed value of the current block is not specifically limited, and correspondingly, the specific position of the TALF filtering in the loop filtering process is not specifically limited, that is, the position of the temporal adaptive loop filtering can be at any place in the loop filtering.

[0309] Exemplarily, in some embodiments, FIG. 11 is an implementation schematic diagram one of the TALF filtering proposed in the embodiments of the present application, as shown in FIG. 11, the input of the TALF can include the reconstructed value on the reconstructed image of the current image before ALF filtering and the reconstructed image of the current image that has been decoded in the time domain.

[0310] Exemplarily, in some embodiments, FIG. 12 is an implementation schematic diagram two of the TALF filtering proposed in the embodiments of the present application, as shown in FIG. 12, the input of the TALF can include the reconstructed value on the reconstructed image of the current image after Deblocking and the reconstructed image of the current image that has been decoded in the time domain, that is, the position of the TALF in the loop filtering can be parallel to the SAO.

[0311] Exemplarily, in some embodiments, FIG. 13 is a schematic diagram of implementation of TALF filtering according to an embodiment of the present application, as shown in FIG. 13, the position of TALF in loop filtering can be parallel to Deblocking.

[0312] Exemplarily, in some embodiments, FIG. 14 is a schematic diagram of implementation of TALF filtering according to an embodiment of the present application, as shown in FIG. 14, the position of TALF in loop filtering can be before Deblocking.

[0313] Exemplarily, in some embodiments, FIG. 15 is a schematic diagram of implementation of TALF filtering according to an embodiment of the present application, as shown in FIG. 15, the input of TALF can include the reconstructed value on the reconstructed image of the current image after Deblocking and the decoded reconstructed image in time domain, that is, the position of TALF in loop filtering can be after Deblocking.

[0314] Exemplarily, in some embodiments, FIG. 16 is a schematic diagram of implementation of TALF filtering according to an embodiment of the present application, as shown in FIG. 16, the input of TALF can include the reconstructed value on the reconstructed image of the current image after SAO and the decoded reconstructed image in time domain, that is, the position of TALF in loop filtering can be after SAO.

[0315] Exemplarily, in some embodiments, FIG. 17 is a schematic diagram of implementation of TALF filtering according to an embodiment of the present application, as shown in FIG. 17, the input of TALF can also include the reconstructed value on the reconstructed image of the current image after ALF filtering and the decoded reconstructed image in time domain, that is, the position of TALF in loop filtering can be after ALF.

[0316] Therefore, in some embodiments, TALF can be placed in any position in a series of loop filters, which means that the input of TALF includes the decoded image and the reconstructed value on the reconstructed image of any step in the current loop filtering.

[0317] Further, in the embodiments of the present application, the reconstructed value of the current block can be the reconstructed value of the luminance component of the current block, or the reconstructed value of the chroma component of the current block, which is not limited in the present application.

[0318] That is, in the embodiments of the present application, TALF filtering can be applied to any image component of the current image.

[0319] It should be noted that in the embodiments of the present application, in the video image, the image components of the current image can include a first image component, a second image component and a third image component. Among them, the three image components are a luminance component, a blue color component and a red color component respectively, specifically, the luminance component is usually represented by the symbol Y, the blue color component is usually represented by the symbol Cb or U, and the red color component is usually represented by the symbol Cr or V; in this way, the video image can be represented in YCbCr format or YUV format.

[0320] Further, in the embodiments of the present application, the shape, size and tap number of the filter used in the TALF filtering process are not specifically limited, that is, filters of any shape and size can be applied to the TALF filtering process. For example, the shape of the filter used in the TALF filtering process includes but is not limited to diamond shape, cross shape, rectangle, square.

[0321] Exemplarily, in some embodiments, FIG. 18 is a schematic diagram of a filter according to an embodiment of the present application, as shown in FIG. 18, a 5x5 symmetric filter contains 7 filter coefficients.

[0322] Exemplarily, in some embodiments, FIG. 19 is a schematic diagram of a filter according to an embodiment of the present application, as shown in FIG. 19, a 7x7 symmetric filter contains 13 filter coefficients.

[0323] Exemplarily, in some embodiments, FIG. 20 is a schematic diagram of a filter according to an embodiment of the present application, as shown in FIG. 20, a 9x9 symmetric filter contains 21 filter coefficients.

[0324] Among them, the above several are symmetric filters, each index value corresponds to a filter coefficient value, and each filter coefficient corresponds to two filter input values in the symmetric position except the filter center point.

[0325] Exemplarily, in some embodiments, FIG. 21 is a schematic diagram of a filter according to an embodiment of the present application, as shown in FIG. 21, a 5x5 symmetric filter contains 4 filter coefficients.

[0326] Exemplarily, in some embodiments, FIG. 22 is a schematic diagram of a filter according to an embodiment of the present application, as shown in FIG. 22, a 7x7 symmetric filter contains 6 filter coefficients.

[0327] Among them, the symmetric filter can also be that each index value corresponds to a filter coefficient value, and each filter coefficient corresponds to filter input values in multiple directions except the filter center point.

[0328] Of course, the filter can also be asymmetric, meaning that each filter input position has a different coefficient value.

[0329] Exemplarily, in some embodiments, Fig. 23 is a schematic diagram of a filter six proposed in embodiments of the present application, as shown in Fig. 23, a 5x5 asymmetric filter, containing 13 filter coefficients.

[0330] Exemplarily, in some embodiments, for the forward filtering and backward filtering under the RA configuration, and the forward filtering using the reconstructed value in the POC N-1 reference image as input and the forward filtering using the reconstructed value in the POC N-2 reference image as input under the LD configuration, a 7x7 symmetric filter containing 13 filter coefficients can be selected.

[0331] Exemplarily, in some embodiments, Fig. 24 is a schematic diagram of a filter seven proposed in embodiments of the present application, as shown in Fig. 24, two 5x5 symmetric filters, containing 14 filter coefficients in total. Among them, for the bidirectional filtering under the RA configuration and the forward filtering using the reconstructed values in the POC N-1 and N-2 reference images as input under the LD configuration, two filters as shown in Fig. 24 can be selected, which are used for the reconstructed values in the two reference images as input respectively.

[0332] It can be understood that the decoding method proposed in embodiments of the present application is a method of using the reconstructed sample values in the time domain as an information source to improve the current image reconstruction quality, and extracting information from the reconstructed samples of the reconstructed image by the TALF filter to filter the current image.

[0333] It can be understood that the decoding method proposed in embodiments of the present application can be pre-defined or transmitted through a code stream.

[0334] It can be understood that the decoding method proposed in embodiments of the present application can propose different filtering modes under different encoding configurations. Among them, under the RA configuration, the filtering mode can be divided into forward filtering mode, backward filtering mode and bidirectional filtering mode; under the LD configuration, the filtering mode can be divided into first forward filtering mode, second forward filtering mode and third forward filtering mode.

[0335] It can be understood that the decoding method proposed in embodiments of the present application can use TALF for luminance component or chrominance component.

[0336] It can be understood that the decoding method proposed in the embodiments of the present application can be any level of TALF switch identification, such as TALF switching in image units, TALF switching in CTU units, TALF switching in CU units, or other ways of dividing an image into sub-regions.

[0337] It can be understood that the decoding method proposed in the embodiments of the present application can also select to skip the coding and decoding processing of the CTU identifier, and directly decide whether to perform TALF filtering on the entire image through the image-level identifier. At this time, the current image corresponds to only one candidate filter.

[0338] The embodiments of the present application provide a decoding method. In the process of performing TALF filtering, the vector information including the motion vector or the block vector corresponding to the sample position can be used to obtain the corresponding reference position, and the TALF filtering of the sample position is completed based on the reference position, that is, the vector information is introduced to assist the TALF filtering, so as to further improve the filtering effect and improve the coding and decoding performance.

[0339] An embodiment of the present application proposes an encoding method. The method is applied to an encoder and used in a scene of performing filtering processing through a TALF filter. FIG. 25 is a schematic diagram of the encoding method proposed in the embodiments of the present application. As shown in FIG. 25, the method of the encoder for performing encoding processing can include the following steps.

[0340] Step 2001: In a case where it is determined that the current image uses TALF for filtering, determining filter coefficients.

[0341] In the embodiments of the present application, if it is determined that the current image uses TALF for filtering, the filter coefficients can be determined first.

[0342] Further, in the embodiments of the present application, each candidate mode can be traversed. In each candidate mode, the candidate filter corresponding to the current block is determined, and the filter coefficients corresponding to the current block are determined based on the candidate filter corresponding to the current block.

[0343] It can be understood that in the embodiments of the present application, each filtering mode (candidate mode) can be traversed. The current image reconstruction value in the corresponding mode, the reconstruction value of the reference reconstruction image (inter-frame reference image), and the original value of the current image are used to construct a Wiener-Hopf equation, and 1-4 groups of filter coefficients are solved. For each group of candidate filters that can be used for the current block, the filter coefficients corresponding to the candidate filter can be solved, that is, the filter coefficients corresponding to the current block can include the filter coefficients of each group of candidate filters.

[0344] Step 2002, determining at least one reference position corresponding to the current position according to the vector information corresponding to the current position.

[0345] In the embodiments of the present application, at least one reference position corresponding to the current position can be determined according to the vector information corresponding to the current position.

[0346] It should be noted that in the embodiments of the present application, the vector information can include a motion vector (MV) and a block vector (BV). Wherein, corresponding to different types of vector information, the method of determining at least one reference position is different.

[0347] It can be understood that in the embodiments of the present application, the loop filtering link is performed on the reconstructed image after the current image has been reconstructed, so when the temporal loop filtering is performed, it has been determined that the coding block where the current position is located has selected intra prediction, inter prediction or IBC prediction. For example, in the ECM reference software, the block to which the current position belongs is an intra prediction block, if the selected is an intra TMP prediction mode, there will be one or more saved BVs in the motion cache; if the selected is an IBC prediction mode, there will also be one or more saved BVs in the motion cache; if the selected is an inter prediction mode, there will be one or more saved MVs in the motion cache.

[0348] Further, in the embodiments of the present application, if the vector information corresponding to the current position includes at least one motion vector corresponding to the current position, when determining at least one reference position corresponding to the current position according to the vector information corresponding to the current position, at least one inter frame reference image corresponding to the current image can be determined according to the at least one motion vector; then in the case that the at least one inter frame reference image exists in the reference image list corresponding to the current image, the at least one reference position corresponding to the current position is determined in the at least one inter frame reference image according to the at least one motion vector.

[0349] Further, in the embodiments of the present application, when at least one inter frame reference image corresponding to the current image is determined according to the at least one motion vector, the filtering mode corresponding to the current image can be determined first, and then at least one inter frame reference image corresponding to the current image is determined according to the filtering mode corresponding to the current image and the at least one motion vector.

[0350] Further, in embodiments of the present application, after determining the at least one inter-frame reference picture corresponding to the current picture according to the at least one motion vector, if the at least one inter-frame reference picture does not exist in the reference picture list corresponding to the current picture, the TALF filtering of the current position can be skipped.

[0351] For example, in some embodiments, the filtering mode corresponding to the current picture is denoted as talf_filter_mode, and when determining the at least one reference position, if talf_filter_mode is 0, it is determined whether the prediction block to which the current position belongs selects MV0 pointing to a reference picture in reference picture list 0, and if yes, the reference picture rec0 pointed to by MV0 is used as input, otherwise the filtering of the current position is skipped; if talf_filter_mode is 1, it is determined whether the prediction block to which the current position belongs selects MV1 pointing to a reference picture in reference picture list 1, and if yes, the reference picture rec1 pointed to by MV1 is used as input, otherwise the filtering of the current position is skipped; if talf_filter_mode is 2, it is determined whether the prediction block to which the current position belongs selects both MV0 pointing to a reference picture in reference picture list 0 and MV1 pointing to a reference picture in reference picture list 1, and if yes, the reference picture rec0 pointed to by MV0 and the reference picture rec1 pointed to by MV1 are used as input, otherwise the filtering of the current position is skipped.

[0352] Further, in embodiments of the present application, if the vector information corresponding to the current position includes at least one block vector corresponding to the current position, when determining the at least one reference position corresponding to the current position according to the vector information corresponding to the current position, the at least one reference position corresponding to the current position in the current picture can be determined directly according to the at least one block vector.

[0353] It can be understood that in embodiments of the present application, whether to perform filtering is determined according to the control identifier at the CTU level, the selected filtering mode, and whether there are available MV0 and MV1 pointing to reference picture list 0 and reference picture list 1, or BV0 and BV1 in the motion buffer.

[0354] For example, in some embodiments, if MV0 exists, the CTU control identifier indicates that filtering is on, and the selected filtering mode is forward filtering, the current position can be filtered.

[0355] For example, in some embodiments, if BV0 exists, the CTU control identifier indicates that filtering is on, and the selected filtering mode is forward filtering, the current position can be filtered.

[0356] Exemplarily, in some embodiments, if MV1 exists, the CTU control flag indicates that the filter is on, and the selected one is the backward filter, the filter can be performed on the current position.

[0357] Exemplarily, in some embodiments, if BV1 exists, the CTU control flag indicates that the filter is on, and the selected one is the backward filter, the filter can be performed on the current position.

[0358] Exemplarily, in some embodiments, if MV0 and MV1 exist, the CTU control flag indicates that the filter is on, and the selected one is the bi-directional filter, the filter can be performed on the current position.

[0359] Exemplarily, in some embodiments, if BV0 and BV1 exist, the CTU control flag indicates that the filter is on, and the selected one is the bi-directional filter, the filter can be performed on the current position.

[0360] Further, in the embodiments of the present application, a flag can be introduced to further control whether to use the vector information assisted TALF filter. For example, the auxiliary filter identification information used to determine whether to use the vector information assisted TALF filter can be transmitted in the code stream.

[0361] Further, in the embodiments of the present application, the cost value when the vector information assisted TALF filter is used and the cost value when the vector information assisted TALF filter is not used are determined; the auxiliary filter identification information is determined according to the cost value when the vector information assisted TALF filter is used and the cost value when the vector information assisted TALF filter is not used, and the auxiliary filter identification information is written into the code stream; wherein the auxiliary filter identification information is used to determine whether to use the vector information assisted TALF filter.

[0362] Further, in the embodiments of the present application, when the auxiliary filter identification information is determined according to the cost value when the vector information assisted TALF filter is used and the cost value when the vector information assisted TALF filter is not used, in the case that the cost value when the vector information assisted TALF filter is used is greater than or equal to the cost value when the vector information assisted TALF filter is not used, the auxiliary filter identification information is set to indicate that the vector information assisted TALF filter is not used; in the case that the cost value when the vector information assisted TALF filter is used is less than the cost value when the vector information assisted TALF filter is not used, the auxiliary filter identification information is set to indicate that the vector information assisted TALF filter is used.

[0363] Further, in the embodiments of the present application, the auxiliary filter identification information can be used to determine whether to allow the current position to use the vector information assisted TALF filter. Wherein the auxiliary filter identification information can be a flag.

[0364] It should be noted that in the embodiments of the present application, the auxiliary filtering identification information can be a picture-level flag, a slice-level (slice-level) flag, a sub-picture-level flag, or a tile-level flag. The present application does not make specific limitations.

[0365] Further, in the embodiments of the present application, it can be determined whether to allow the use of vector information assisted TALF filtering through the value of the auxiliary filtering identification information.

[0366] For example, in some embodiments, when the value of the auxiliary filtering identification information is 0, it is determined not to use vector information assisted TALF filtering. When the value of the auxiliary filtering identification information is 1, it is determined to use vector information assisted TALF filtering.

[0367] For example, in some embodiments, when the value of the auxiliary filtering identification information is false, it is determined not to use vector information assisted TALF filtering. When the value of the auxiliary filtering identification information is true, it is determined to use vector information assisted TALF filtering.

[0368] It should be noted that in the embodiments of the present application, the value of the auxiliary filtering identification information can be in the form of a parameter or in the form of a number. In general, the auxiliary filtering identification information can be a parameter written in the Slice header or the picture header, which is not limited here.

[0369] Assuming that the auxiliary filtering identification information is a picture-level flag, if the value of the auxiliary filtering identification information is 1, it can be determined that the current picture allows the use of vector information assisted TALF filtering. Otherwise, if the value of the auxiliary filtering identification information is 0, it can be determined that the current picture does not allow the use of vector information assisted TALF filtering.

[0370] For example, in some embodiments, the auxiliary filtering identification information can be represented by a syntax element talf_mv_guided_flag, i.e. talf_mv_guided_flag indicates whether the current position allows the use of vector information assisted TALF filtering. If the value of talf_mv_guided_flag is 0, it is determined that the current position does not allow the use of vector information assisted TALF filtering, and if the value of talf_mv_guided_flag is 1, it is determined that the current position allows the use of vector information assisted TALF filtering.

[0371] Of course, the value of talf_mv_guided_flag is not limited to 0 and 1, and the present application does not make specific limitations.

[0372] Exemplarily, one implementation is that a slice-level syntax element needs to additionally include an identifier indicating whether TALF is assisted by MV, and the slice-level syntax element is as follows:

[0373] Wherein, talf_mv_guided_flag is used to identify whether TALF under the current slice is filtered based on motion vector or block vector assistance, when the value of the identifier is 1, the time domain adaptive loop filter of the current slice is based on motion vector block vector (vector information) assistance; when the value of the identifier is 0, the time domain adaptive loop filter of the current slice is not based on motion vector block vector (vector information) assistance. When the syntax element does not exist in the code stream, its value can be defaulted as 0.

[0374] Further, in the embodiments of the present application, when the at least one reference position corresponding to the current position in the at least one inter-frame reference image is determined according to the at least one motion vector, the at least one first position offset value in the first direction and the at least one second position offset value in the second direction can be determined according to the at least one motion vector first; and then the at least one reference position in the at least one inter-frame reference image is determined according to the at least one first position offset value and the at least one second position offset value.

[0375] It should be noted that in the embodiments of the present application, the first offset value is used for position offset in the first direction, and the second offset value is used for position offset in the second direction. Wherein, the first direction and the second direction can be horizontal direction and vertical direction respectively.

[0376] Exemplarily, in some embodiments, if MV0 and MV1 are found, since the MVs used in inter prediction are sub-pixel level precision, here the position offset on the reconstructed image pointed by the MVs is obtained by rounding to the nearest integer pixel precision. Wherein, the horizontal direction integer pixel position offset of MV0 is Offset0X = Mv0.Hor < 0? - ((abs(Mv0.Hor) + 8) >> 4) : ((abs(Mv0.Hor) + 8) >> 4); the vertical direction integer pixel position offset of MV0 is Offset0Y = Mv0.Ver < 0? - ((abs(Mv0.Ver) + 8) >> 4) : ((abs(Mv0.Ver) + 8) >> 4); the horizontal direction integer pixel position offset of MV1 is Offset1X = Mv1.Hor < 0? - ((abs(Mv1.Hor) + 8) >> 4) : ((abs(Mv1.Hor) + 8) >> 4); and the vertical direction integer pixel position offset of MV1 is Offset1Y = Mv1.Ver < 0? - ((abs(Mv1.Ver) + 8) >> 4) : ((abs(Mv1.Ver) + 8) >> 4).

[0377] Further, in the embodiments of the present application, after determining the auxiliary filter identification information, in the case that it is determined that the vector information assisted TALF filter is not used based on the auxiliary filter identification information, at least one inter-frame reference image corresponding to the current image and the filter coefficient are determined; the filter is performed based on the reconstructed samples of the at least one inter-frame reference image and the filter coefficient, to determine the filtered reconstructed value of the current position.

[0378] It can be understood that, in the embodiments of the present application, if it is determined that the vector information assisted TALF filter is not used, then the at least one inter-frame reference image corresponding to the current image and the filter coefficient can be determined according to the conventional TALF filter method, and then the TALF filter is performed based on the at least one inter-frame reference image and the filter coefficient.

[0379] That is to say, in the embodiments of the present application, if the vector information assisted TALF filter corresponding to the current position in the current image is not used, then the at least one inter-frame reference image corresponding to the current image can be further determined.

[0380] Therefore, it can be seen that, in the embodiments of the present application, for different scenarios of using the vector information assisted TALF filter corresponding to the current position and not using the vector information assisted TALF filter corresponding to the current position, the determined reference image can be different in different manners.

[0381] Exemplarily, in some embodiments, if the TALF filtering is not assisted by the vector information, the input image of the determined filter can be an inter reference picture according to the conventional TALF filtering procedure, and then the filtering process can be performed according to the reconstructed samples in the at least one inter reference picture; if the TALF filtering is assisted by the vector information, the input image of the determined filter can be an inter reference picture or the current picture itself, and then the filtering process can be performed according to the reconstructed samples in the at least one inter reference picture or the current picture.

[0382] Exemplarily, in some embodiments, the filter mode corresponding to the current picture is denoted as talf_filter_mode, when the input image of the filter is determined according to talf_filter_mode and talf_filter_mode, if talf_filter_mode is 0 and talf_filter_mode is 0, the forward nearest reconstructed picture rec0 is used as the input; if talf_filter_mode is 0 and talf_filter_mode is 1, it is confirmed whether the MV0 pointing to the reference picture in the reference picture list 0 is selected for the prediction block to which the current position belongs, if yes, the reference picture rec0 pointed by the MV0 is used as the input, otherwise the filtering of the current position is skipped; if talf_filter_mode is 1 and talf_filter_mode is 0, the backward nearest reconstructed picture rec1 is used as the input in RA, and the forward second nearest reconstructed picture rec1 is used as the input in LD; if talf_filter_mode is 1 and talf_filter_mode is 1, it is confirmed whether the MV1 pointing to the reference picture in the reference picture list 1 is selected for the prediction block to which the current position belongs, if yes, the reference picture rec1 pointed by the MV1 is used as the input, otherwise the filtering of the current position is skipped; if talf_filter_mode is 2 and talf_filter_mode is 0, the forward nearest reconstructed picture rec0 and the backward nearest reconstructed picture rec1 are used as the input in RA, and the forward nearest reconstructed picture rec0 and the second nearest reconstructed picture rec1 are used as the input in LD; if talf_filter_mode is 2 and talf_filter_mode is 1, it is confirmed whether the Mv0 pointing to the reference picture in the reference picture list 0 and the MV1 pointing to the reference picture in the reference picture list 1 are both selected for the prediction block to which the current position belongs, if yes, the reference picture rec0 pointed by the MV0 and the reference picture rec1 pointed by the MV1 are used as the input, otherwise the filtering of the current position is skipped.

[0383] Further, in embodiments of the present application, if talf mv guided flag is 0, Offset0X, Offset0Y, Offset1X and Offset1Y are set to 0.

[0384] Further, in embodiments of the present application, if the vector information assisted TALF filter is not used, in determining the at least one inter- frame reference picture corresponding to the current picture and the filter coefficients corresponding to the current block, the at least one inter- frame reference picture corresponding to the current picture and the filter coefficients corresponding to the current block can be determined respectively in the current configuration and each candidate mode corresponding to the current configuration according to a regular TALF filter procedure.

[0385] Further, in embodiments of the present application, for the TALF filter, the corresponding input information can include at least one inter- frame reference picture corresponding to the current picture, and the at least one inter- frame reference picture can be a temporally coded reconstructed picture. Wherein, the TALF filter for the current block in the current picture can be completed using the reconstructed sample values of the at least one inter- frame reference picture.

[0386] It can be understood that, in embodiments of the present application, the current block can be a CTU determined after the current picture is divided, the current block can also be a coding unit CU obtained after the CTU is divided, and the current block can also be a sub-region of any size obtained after the current picture is divided, and the present application does not limit the size and the obtaining manner of the current block in the current picture.

[0387] Further, in embodiments of the present application, in determining the at least one inter- frame reference picture corresponding to the current picture and the filter coefficients corresponding to the current block, the candidate mode can be determined according to the current configuration first; then in each candidate mode, the at least one inter- frame reference picture is determined, the candidate filter corresponding to the current block is determined, and the filter coefficients corresponding to the current block are determined based on the candidate filter corresponding to the current block.

[0388] Further, in embodiments of the present application, in determining the candidate mode according to the current configuration, in the case that the current configuration is a RA configuration, the candidate mode is determined as a first candidate mode; wherein the first candidate mode includes one or more of a forward filtering mode, a reverse filtering mode and a bidirectional filtering mode; in the case that the current configuration is a LD configuration, the candidate mode is determined as a second candidate mode; wherein the second candidate mode includes one or more of a first forward filtering mode, a second forward filtering mode and a third forward filtering mode.

[0389] It can be understood that in the embodiments of the present application, for the RA configuration, the corresponding first candidate mode can include one or more of a forward filtering mode, a backward filtering mode and a bidirectional filtering mode. Among them, the forward filtering mode can be understood as using the reconstruction information of the reconstructed image smaller than the POC of the current image as the input of the filter, the backward filtering mode can be understood as using the reconstruction information of the reconstructed image larger than the POC of the current image as the input of the filter, and the bidirectional filtering mode can be understood as using the reconstruction information of the image smaller than the POC of the current image and the image larger than the POC of the current image as the input of the filter.

[0390] It can be understood that in the embodiments of the present application, for the LD configuration, the corresponding second candidate mode can include one or more of a first forward filtering mode, a second forward filtering mode and a third forward filtering mode. Among them, for the coding under the LD configuration, the coding order is the same as the playing order, so only the reference image smaller than the POC of the current image can be used as the input of the filter, that is, only the forward filtering mode is supported. However, corresponding to different inter-frame reference images, the corresponding forward filtering mode can be distinguished into the first forward filtering mode, the second forward filtering mode and the third forward filtering mode.

[0391] Further, in the embodiments of the present application, after determining the filtering mode corresponding to the current image, at least one inter-frame reference image can be further determined according to the filtering mode.

[0392] It can be understood that in the embodiments of the present application, the determination of the inter-frame reference image depends on the filtering mode, and for different filtering modes, the finally determined inter-frame reference image can be different.

[0393] Exemplarily, in some embodiments, if the POC value of the current image is N, for the RA configuration, if the determined filtering mode is the forward filtering mode, the image with the POC value of N-1 can be selected as the inter-frame reference image of the current image; if the determined filtering mode is the backward filtering mode, the image with the POC value of N+1 can be selected as the inter-frame reference image of the current image; if the determined filtering mode is the bidirectional filtering mode, the image with the POC value of N-1 and the image with the POC value of N+1 can be selected as the inter-frame reference image of the current image.

[0394] Exemplarily, in some embodiments, if the POC value of the current image is N, for the LD configuration, the images with POC values of N-1, N-2, … can be used as the inter-frame reference images of the current image. For example, if the determined filter mode is the first forward filter mode, the image with POC value of N-1 can be selected as the inter-frame reference image of the current image; if the determined filter mode is the second forward filter mode, the image with POC value of N-2 can be selected as the inter-frame reference image of the current image; if the determined filter mode is the third forward filter mode, the images with POC values of N-1 and N-2 can be selected as the inter-frame reference images of the current image.

[0395] That is, in the embodiments of the present application, the POC values of the determined inter-frame reference images can be different based on different filter modes, and the number of the determined inter-frame reference images can also be different. For example, for the forward filter mode, the backward filter mode in the RA configuration, and the first forward filter mode (the nearest first frame), the second forward filter mode (the nearest second frame) in the LD configuration, the current image corresponds to one inter-frame reference image; for the bidirectional filter mode in the RA configuration, and the third forward filter mode (the nearest first frame and the nearest second frame) in the LD configuration, the current image corresponds to two inter-frame reference images.

[0396] It can be seen that, for the temporal adaptive loop filter type, the intra-frame encoded image has no reference image, and the inter-frame encoded image has a reference image, so the temporal filter of the present scheme is applied to the inter-frame encoded image, for example, the inter-frame encoded image under the encoding types of the RA and LD configurations. For the encoding and decoding under the LD configuration, the encoding and decoding order is the same as the playing order, so when the temporal adaptive filter is used for the inter-frame encoded image, the reference image with a smaller POC value than the current image is always used as the input of the filter. For the encoding and decoding under the RA configuration, the encoding and decoding order can be different from the playing order, and when the temporal filter is used for the current inter-frame encoded image, the reference frame with a larger or smaller POC value than the current frame can be used as the input of the filter.

[0397] The forward filter uses the reconstructed information of the reconstructed picture with a POC smaller than the POC of the current picture as the input of the filter to generate the filtered value acting on the current reconstructed picture. For example, in the LD configuration, the POC of the current picture is N, where N is a non-negative integer, the reconstructed value of the reconstructed picture with the POC value of N-1, N-2, … can be used as the input of the filter; for example, in the RA configuration, in the coding order, when the POC of the current picture is N, the inter-coded picture with the TID of 5 can use the reconstructed value of the picture with the POC value of N-1 as the input of the filter; the TID of 4 can use the reconstructed value of the picture with the POC value of N-2 as the input of the filter; the TID of 3 can use the reconstructed value of the picture with the POC value of N-4 as the input of the filter; the TID of 2 can use the picture with the POC value of N-8 as the input of the filter; the TID of 1 can use the reconstructed value of the picture with the POC value of N-16 as the input of the filter; and the TID of 0 can use the reconstructed value of the picture with the POC value of N-32 as the input of the filter.

[0398] It should be noted that in the embodiments of the present application, the POC value of the reference frame used in the forward filtering will change due to different coding configurations.

[0399] The backward filter uses the reconstructed information of the reconstructed picture with a POC larger than the POC of the current picture as the input of the filter to generate the filtered value acting on the current reconstructed picture. In the current various coding configurations, the backward filter is only used in the RA configuration. For example, in the RA configuration, in the coding order, when the POC of the current picture is N, the inter-coded picture with the TID of 5 can use the reconstructed value of the picture with the POC value of N+1 as the input of the filter; the TID of 4 can use the reconstructed value of the picture with the POC value of N+2 as the input of the filter; the TID of 3 can use the reconstructed value of the picture with the POC value of N+4 as the input of the filter; the TID of 2 can use the picture with the POC value of N+8 as the input of the filter; the TID of 1 can use the reconstructed value of the picture with the POC value of N+16 as the input of the filter; and the TID of 0 can use the reconstructed value of the picture with the POC value of N+32 as the input of the filter.

[0400] It should be noted that in the embodiments of the present application, the POC value of the reference frame used in the backward filtering will change due to different coding configurations.

[0401] Bidirectional filtering is to use the reconstructed information of the image with smaller POC than the current image and the image with larger POC than the current image as the input of the filter to generate the filtering value acting on the current reconstructed image. In the current various coding configurations, the reverse filtering is only used in the RA configuration. For example, in the RA configuration, when the POC of the current image is N in the coding order, the inter-coded image with TID of 5 can use the reconstructed values of the images with POC values of N+1 and N-1 as the input of the filter; the TID of 4 can use the reconstructed values of the images with POC values of N+2 and N-2 as the input of the filter; the TID of 3 can use the reconstructed values of the images with POC values of N+4 and N-4 as the input of the filter; the TID of 2 can use the images with POC values of N+8 and N-8 as the input of the filter; the TID of 1 can use the reconstructed values of the images with POC values of N+16 and N-16 as the input of the filter; and the TID of 0 can use the reconstructed values of the images with POC values of N+32 and N-32 as the input of the filter.

[0402] It should be noted that in the embodiments of the present application, the POC values of the reference frames used in bidirectional filtering will change due to different coding configurations. In principle, in the RA configuration, bidirectional filtering always uses the reconstructed values of the two frames of reconstructed images with the closest forward and backward distances as the input of the filter.

[0403] Exemplarily, in some embodiments, assuming that the filtering mode is a forward filtering mode, the following forward filtering modes can be included but are not limited to:

[0404] 1. In the RA configuration, the reference image with the closest temporal distance and smaller POC than the current image is used as the input of the temporal filtering;

[0405] 2. In the LD configuration, when the POC value of the current image is N, the reference image with POC value of N-1 is used as the input of the temporal filtering corresponding to the first forward filtering mode;

[0406] 3. In the LD configuration, when the POC value of the current image is N, the reference image with POC value of N-2 is used as the input of the temporal filtering corresponding to the second forward filtering mode;

[0407] 4. In the LD configuration, when the POC value of the current image is N, the two frames of reference images with POC values of N-2 and N-1 are used as the input of the temporal filtering corresponding to the third forward filtering mode.

[0408] It can be understood that in the embodiments of the present application, in the case of being configured as an RA configuration or an LD configuration according to the coding configuration, the mode allowed to be used by the TALF can be determined by traversing each candidate mode. Among them, the candidate modes of the TALF are three in the RA configuration and the LD configuration. For the RA configuration, in the corresponding first candidate mode, the forward filtering (forward filtering mode) is mode 1, the backward filtering (backward filtering mode) is mode 2, and the bidirectional filtering (bidirectional filtering mode) is mode 3; for the LD configuration, in the corresponding second candidate mode, using the reference image POC N-1 as the TALF filter input (first forward filtering mode) is mode 1, using the reference image POC N-2 as the TALF filter input (second forward filtering mode) is mode 2, and using the reference images POC N-1 and N-2 as the TALF filter input (third forward filtering mode) is mode 3.

[0409] Further, in the embodiments of the present application, each candidate mode can be traversed, in each candidate mode, the candidate filter corresponding to the current block is determined, and the filter coefficient corresponding to the current block is determined based on the candidate filter corresponding to the current block.

[0410] It can be understood that in the embodiments of the present application, each filtering mode (candidate mode) can be traversed, and the Wiener-Hopf equation can be constructed using the current image reconstruction value in the corresponding mode, the reconstruction value of the reference reconstructed image (inter-frame reference image) and the original value of the current image, and 1-4 sets of filter coefficients can be solved out. Among them, for each set of candidate filters that can be used for the current block, the filter coefficients corresponding to the candidate filter can be solved out, that is, the filter coefficients corresponding to the current block can include the filter coefficients of each set of candidate filters.

[0411] Step 2002, filtering based on the reconstructed samples of at least one reference position and the filter coefficients to determine the filtered reconstruction value of the current position.

[0412] In the embodiments of the present application, after determining the at least one reference position corresponding to the current position according to the vector information corresponding to the current position, the filtered reconstruction value of the current position can be determined based on the reconstructed samples of the at least one reference position and the filter coefficients.

[0413] It can be understood that in the embodiments of the present application, the filtering process of using vector information to assist TALF filtering can also be understood as filtering the (x, y) position (current position) according to the obtained position offset (first position offset value and second position offset value), the reconstructed image (inter-frame reference image or current image), and the filter coefficient.

[0414] Further, in the embodiments of the present application, when filtering based on the reconstructed samples of the at least one reference position and the filter coefficients to determine the filtered reconstructed value of the current position, the reconstructed sample value of the current position and the reconstructed sample value of the reference position can be determined first; then according to the filter coefficients, the reconstructed sample value of the current position and the reconstructed sample value of the reference position, the filtered reconstructed value of the current position is determined.

[0415] Further, in the embodiments of the present application, when filtering based on the reconstructed samples of the at least one reference position and the filter coefficients to determine the filtered reconstructed value of the current position, the reconstructed sample value of the current position, the reconstructed sample value of the first reference position and the reconstructed sample value of the second reference position can be determined first; then according to the filter coefficients, the reconstructed sample value of the current position, the reconstructed sample value of the first reference position and the reconstructed sample value of the second reference position, the filtered reconstructed value of the current position is determined.

[0416] Further, in the embodiments of the present application, if it is determined that the TALF filter assisted by the vector information is not used, the filtered reconstructed value of the current position can be determined based on the reconstructed samples of the at least one inter-frame reference image and the filter coefficients.

[0417] In the embodiments of the present application, after the at least one inter-frame reference image corresponding to the current image and the filter coefficients corresponding to the current block are determined, the current block can be further filtered based on the reconstructed samples of the at least one inter-frame reference image and the filter coefficients corresponding to the current block to determine the filtered reconstructed value of the current block.

[0418] It can be understood that in the embodiments of the present application, in each candidate mode, each group of candidate filters can be used respectively to filter the reconstructed value of the current block by using the filter coefficients corresponding to the current block in the candidate filter and the reconstructed samples of the at least one inter-frame reference image corresponding to the candidate mode, so as to determine the filtered reconstructed value of the current block.

[0419] It can be understood that in the embodiments of the present application, since the POC values of the inter-frame reference images determined based on different filter modes can be different, and the number of the determined inter-frame reference images can also be different, when the TALF filter is performed on the current block of the current image, the filtering processes performed corresponding to different numbers of inter-frame reference images are also different.

[0420] Further, in the embodiments of the present application, when the current block is filtered based on the filter coefficients corresponding to the current block, the reconstructed sample of the current sample position and the reconstructed sample of the reference sample position corresponding to the current sample position, the filtered reconstructed value of the current block is determined as follows: the reconstructed sample value of the current sample position is determined according to the reconstructed value of the current block; the reconstructed sample value of the reference sample position is determined according to the reconstructed sample of the inter-frame reference picture; the filtered reconstructed sample value of the current sample position is determined according to the filter coefficients corresponding to the current block, the reconstructed sample value of the current sample position and the reconstructed sample value of the reference sample position; and the filtered reconstructed value of the current block is determined according to the filtered reconstructed sample value of the current sample position.

[0421] It can be understood that, in the embodiments of the present application, for the forward filtering mode, the backward filtering mode in the RA configuration, and the first forward filtering mode (the nearest first frame) and the second forward filtering mode (the nearest second frame) in the LD configuration, the current picture corresponds to one inter-frame reference picture, and when the TALF filtering is performed on the reconstructed value of the current block based on the reconstructed sample of the one inter-frame reference picture, the reference sample position corresponding to the current sample position can be determined in the one inter-frame reference picture, and then the reconstructed sample value of the current sample position and the reconstructed sample value of the reference sample position are input into the TALF filter, and finally the filtered reconstructed sample value of the current sample position is determined according to the determined filter coefficients corresponding to the current block, and the above scheme is used to traverse any sample position of the current block to complete the filtering of the current block and obtain the filtered reconstructed value of the current block.

[0422] Exemplarily, in some embodiments, assuming that the current picture corresponds to one inter-frame reference picture, the process of filtering the sample value of the position (x, y) (the current sample position) in the current picture using 13 filter coefficients and a 7x7 symmetric filter is as shown in formula (1).

[0423] Further, in the embodiments of the present application, when the filtered reconstructed sample value of the current sample position is determined according to the filter coefficients corresponding to the current block, the reconstructed sample value of the current sample position and the reconstructed sample value of the reference sample position, the first sample difference value can be determined according to the reconstructed sample value of the current sample position and the reconstructed sample value of the reference sample position; and then the filtered reconstructed sample value of the current sample position is determined according to the filter coefficients corresponding to the current block, the reconstructed sample value of the current sample position and the first sample difference value.

[0424] That is, in the embodiments of the present application, the first sample difference value can also be determined according to the reconstructed sample value of the current sample position and the reconstructed sample value of the reference sample position, and then the first sample difference value is used as the input of the filter.

[0425] Exemplarily, in some embodiments, assuming that there is one inter- reference picture corresponding to the current picture, the process of filtering the sample value at the position (x, y) (current sample position) in the current picture using 13 filter coefficients and a 7x7 symmetric filter is as shown in formula (2).

[0426] Further, in the embodiments of the present application, when determining the filtered reconstructed sample value at the current sample position according to the filter coefficients corresponding to the current block, the reconstructed sample value at the current sample position, and the reconstructed sample value at the reference sample position, the filtering position corresponding to the reference sample position in the current block can be determined, and the reconstructed sample value at the filtering position can be determined according to the reconstructed value of the current block; the second sample difference value can be determined according to the reconstructed sample value at the filtering position and the reconstructed sample value at the reference sample position; and the filtered reconstructed sample value at the current sample position can be determined according to the filter coefficients corresponding to the current block, the reconstructed sample value at the current sample position, and the second sample difference value.

[0427] That is, in the embodiments of the present application, for each filtering position, the second sample difference value can also be determined according to the reconstructed sample value at the filtering position and the reconstructed sample value at the reference sample position, and then the second sample difference value can be taken as the input of the filter.

[0428] Exemplarily, in some embodiments, assuming that there is one inter- reference picture corresponding to the current picture, the process of filtering the sample value at the position (x, y) (current sample position) in the current picture using 13 filter coefficients and a 7x7 symmetric filter is as shown in formula (3).

[0429] Further, in the embodiments of the present application, when filtering the current block based on the reconstructed samples of at least one inter- reference picture and the filter coefficients corresponding to the current block to determine the filtered reconstructed value of the current block, for the current sample position in the current block, the reconstructed sample value at the current sample position can be determined according to the reconstructed value of the current block, the reconstructed sample value at the first reference sample position corresponding to the current sample position can be determined according to the reconstructed samples of the first inter- reference picture, and the reconstructed sample value at the second reference sample position corresponding to the current sample position can be determined according to the reconstructed samples of the second inter- reference picture; the filtered reconstructed sample value at the current sample position can be determined according to the filter coefficients corresponding to the current block, the reconstructed sample value at the current sample position, the reconstructed sample value at the first reference sample position, and the reconstructed sample value at the second reference sample position; and the filtered reconstructed value of the current block can be determined according to the filtered reconstructed sample value at the current sample position.

[0430] It can be understood that, in the embodiments of the present application, for the bidirectional filter mode in the RA configuration, and the third forward filter mode (the nearest first frame and the nearest second frame) in the LD configuration, the current picture corresponds to two inter-frame reference pictures, and when the TALF filter is used to filter the reconstructed value of the current block based on the reconstructed samples of the two inter-frame reference pictures, the reference sample positions corresponding to the current sample position can be determined in the two inter-frame reference pictures respectively, and then the reconstructed sample value corresponding to the current sample position, the reconstructed sample values corresponding to the two reference sample positions are input into the TALF filter, combined with the determined filter coefficients corresponding to the current block, to finally determine the filtered reconstructed sample value corresponding to the current sample position. According to the above scheme, any sample position of the current block is traversed to complete the filtering of the current block and obtain the filtered reconstructed value of the current block.

[0431] Exemplarily, in some embodiments, assuming that the current picture corresponds to two inter-frame reference pictures, at this time, the process of filtering the sample value at the coordinate (x, y) position (the current sample position) in the current picture using 7 filter coefficients and 5x5 symmetric 2 filters is as formula (4).

[0432] Further, in the embodiments of the present application, when the filtered reconstructed sample value of the current sample position is determined according to the filter coefficients corresponding to the current block, the reconstructed sample value of the current sample position, the reconstructed sample value of the first reference sample position, the reconstructed sample value of the second reference sample position, the filtered reconstructed sample value of the current sample position can be determined according to the reconstructed sample value of the current sample position and the reconstructed sample value of the first reference sample position; the fourth sample difference value is determined according to the reconstructed sample value of the current sample position and the reconstructed sample value of the second reference sample position; and the filtered reconstructed sample value of the current sample position is determined according to the filter coefficients corresponding to the current block, the reconstructed sample value of the current sample position, the third sample difference value and the fourth sample difference value.

[0433] That is, in the embodiments of the present application, the third sample difference value and the fourth sample difference value can also be determined according to the reconstructed sample value of the current sample position and the reconstructed sample values of the two reference sample positions respectively, and then the third sample difference value and the fourth sample difference value are used as the input of the filter.

[0434] Exemplarily, in some embodiments, assuming that the current picture corresponds to two inter-frame reference pictures, at this time, the process of filtering the sample value at the coordinate (x, y) position (the current sample position) in the current picture using 7 filter coefficients and 5x5 symmetric 2 filters is as formula (5).

[0435] Further, in the embodiments of the present application, when the filtered reconstructed sample value of the current sample position is determined according to the filter coefficient corresponding to the current block, the reconstructed sample value of the current sample position, the reconstructed sample value of the first reference sample position, the reconstructed sample value of the second reference sample position, the filtered reconstructed sample value of the current sample position can be determined as follows: a filtering position corresponding to the reference sample position in the current block is determined, and the reconstructed sample value of the filtering position is determined according to the reconstructed value of the current block; a fifth sample difference value is determined according to the reconstructed sample value of the filtering position and the reconstructed sample value of the first reference sample position; a sixth sample difference value is determined according to the reconstructed sample value of the filtering position and the reconstructed sample value of the second reference sample position; and the filtered reconstructed sample value of the current sample position is determined according to the filter coefficient corresponding to the current block, the reconstructed sample value of the current sample position, the fifth sample difference value and the sixth sample difference value.

[0436] That is, in the embodiments of the present application, for each filtering position, the fifth sample difference value and the sixth sample difference value can also be determined according to the reconstructed sample value of the filtering position and the reconstructed sample values of the two reference sample positions respectively, and then the fifth sample difference value and the sixth sample difference value are taken as the input of the filter.

[0437] Exemplarily, in some embodiments, assuming that the current image corresponds to two inter-frame reference images, the filtering process of the sample value at the position (x, y) (current sample position) in the current image using 7 filter coefficients and 2 filters with 5x5 symmetry is as shown in formula (6).

[0438] It should be noted that in the embodiments of the present application, the filtering process shown in the above formula is an exemplary description of the TALF filtering proposed in the present application, and of course, in actual application, if the adaptive filter does not have the symmetry of the above example, the adaptive filter coefficients and the input values corresponding to each coefficient should also be adjusted accordingly. That is, the filtering process should be a process in which the adaptive filter coefficients and the reconstructed values of the reference image at each related position act together.

[0439] It should be noted that in the embodiments of the present application, the filter coefficients can be integers, and the parameters used in the filtering process can also include bias and shift.

[0440] Further, in the embodiments of the present application, for the TALF filter, in addition to the at least one inter-frame reference image that has been decoded in the time domain, the corresponding input information can also include the reconstructed value of the current block, for example, the reconstructed value of any image component of the current block. Wherein, the reconstructed value of the current block can be obtained after the current block in the current image is processed based on any loop filtering method.

[0441] That is, in the embodiments of the present application, the manner of determining the reconstructed value of the current block is not specifically limited, and correspondingly, the specific position of the TALF filtering in the loop filtering process is not specifically limited, i.e., the position of the temporal adaptive loop filtering can be at any position in the loop filtering.

[0442] Exemplarily, in some embodiments, FIG. 11 is an implementation schematic diagram one of the TALF filtering according to the embodiments of the present application, as shown in FIG. 11, the input of the TALF can include the reconstructed value on the reconstructed image of the current image before the ALF filtering and the decoded reconstructed image in the time domain, i.e., the position of the TALF in the loop filtering can be parallel to the ALF.

[0443] Exemplarily, in some embodiments, FIG. 12 is an implementation schematic diagram two of the TALF filtering according to the embodiments of the present application, as shown in FIG. 12, the input of the TALF can include the reconstructed value on the reconstructed image of the current image after the Deblocking and the decoded reconstructed image in the time domain, i.e., the position of the TALF in the loop filtering can be parallel to the SAO.

[0444] Exemplarily, in some embodiments, FIG. 13 is an implementation schematic diagram three of the TALF filtering according to the embodiments of the present application, as shown in FIG. 13, the position of the TALF in the loop filtering can be parallel to the Deblocking.

[0445] Exemplarily, in some embodiments, FIG. 14 is an implementation schematic diagram four of the TALF filtering according to the embodiments of the present application, as shown in FIG. 14, the position of the TALF in the loop filtering can be before the Deblocking.

[0446] Exemplarily, in some embodiments, FIG. 15 is an implementation schematic diagram five of the TALF filtering according to the embodiments of the present application, as shown in FIG. 15, the input of the TALF can include the reconstructed value on the reconstructed image of the current image after the Deblocking and the decoded reconstructed image in the time domain, i.e., the position of the TALF in the loop filtering can be after the Deblocking.

[0447] Exemplarily, in some embodiments, FIG. 16 is an implementation schematic diagram six of the TALF filtering according to the embodiments of the present application, as shown in FIG. 16, the input of the TALF can include the reconstructed value on the reconstructed image of the current image after the SAO and the decoded reconstructed image in the time domain, i.e., the position of the TALF in the loop filtering can be after the SAO.

[0448] Exemplarily, in some embodiments, FIG. 17 is a schematic diagram of an implementation of the TALF filtering according to the embodiments of the present application. As shown in FIG. 17, the input of the TALF can also include the reconstructed value on the reconstructed image of the decoded reconstructed image in the time domain and the current image after the ALF filtering, i.e., the position of the TALF in the loop filtering can be after the ALF.

[0449] Therefore, in some embodiments, the TALF can also be placed at any position in the series of loop filters, which means that the input of the TALF includes the reconstructed value on the reconstructed image of any loop in the current loop filtering and the decoded image.

[0450] Further, in the embodiments of the present application, the reconstructed value of the current block can be the reconstructed value of the luma component of the current block or the reconstructed value of the chroma component of the current block, which is not specifically limited in the present application.

[0451] That is to say, in the embodiments of the present application, the TALF filtering can be applied to any image component of the current image.

[0452] It should be noted that, in the embodiments of the present application, the image component of the current image in the video image can include a first image component, a second image component and a third image component. The three image components are a luma component, a blue color component and a red color component, respectively. Specifically, the luma component is usually represented by the symbol Y, the blue color component is usually represented by the symbol Cb or U, and the red color component is usually represented by the symbol Cr or V. In this way, the video image can be represented in the YCbCr format or the YUV format.

[0453] Further, in the embodiments of the present application, the shape, size and tap number of the filter used in the TALF filtering process are not specifically limited, i.e., any shape and size of the filter can be applied to the TALF filtering process. For example, the shape of the filter used in the TALF filtering process includes but is not limited to a diamond shape, a cross shape, a rectangle and a square.

[0454] Exemplarily, in some embodiments, FIG. 18 is a schematic diagram of a filter according to the embodiments of the present application, as shown in FIG. 18, a 5x5 symmetric filter includes 7 filter coefficients.

[0455] Exemplarily, in some embodiments, FIG. 19 is a schematic diagram of a filter according to the embodiments of the present application, as shown in FIG. 19, a 7x7 symmetric filter includes 13 filter coefficients.

[0456] Exemplarily, in some embodiments, FIG. 20 is a schematic diagram three of a filter according to an embodiment of the present application, as shown in FIG. 20, a 9x9 symmetric filter, containing 21 filter coefficients.

[0457] wherein the above are symmetric filters, each index value corresponds to a filter coefficient value, and each filter coefficient corresponds to two filter input values in symmetric positions, except for the filter center point.

[0458] Exemplarily, in some embodiments, FIG. 21 is a schematic diagram four of a filter according to an embodiment of the present application, as shown in FIG. 21, a 5x5 symmetric filter, containing 4 filter coefficients.

[0459] Exemplarily, in some embodiments, FIG. 22 is a schematic diagram five of a filter according to an embodiment of the present application, as shown in FIG. 22, a 7x7 symmetric filter, containing 6 filter coefficients.

[0460] wherein the symmetric filter can also be that each index value corresponds to a filter coefficient value, and each filter coefficient corresponds to filter input values in multiple directions, except for the filter center point.

[0461] Of course, the filter can also be asymmetric, which means that each filter input position has different coefficient values.

[0462] Exemplarily, in some embodiments, FIG. 23 is a schematic diagram six of a filter according to an embodiment of the present application, as shown in FIG. 23, a 5x5 asymmetric filter, containing 13 filter coefficients.

[0463] Exemplarily, in some embodiments, for the forward filtering and backward filtering under RA configuration, and the forward filtering using the reconstructed value in the POC N-1 reference image as input and the forward filtering using the reconstructed value in the POC N-2 reference image as input under LD configuration, a 7x7 symmetric filter containing 13 filter coefficients can be selected.

[0464] Exemplarily, in some embodiments, FIG. 24 is a schematic diagram seven of a filter according to an embodiment of the present application, as shown in FIG. 24, two 5x5 symmetric filters, containing 14 filter coefficients in total. Wherein, for the bidirectional filtering under RA configuration and the forward filtering using the reconstructed value in the POC N-1 and N-2 reference images as input under LD configuration, two filters as shown in FIG. 24 can be selected, which are used for the reconstructed value in the two reference images as input respectively.

[0465] In step 2003, a first generation value is determined according to the filtered reconstructed value of the current block, and a first syntax element identification information is determined based on the first generation value, and the first syntax element identification information is written into the bitstream; wherein the first syntax element identification information is used to determine whether the current block is filtered using TALF.

[0466] In the embodiments of the present application, after the current block is filtered based on the reconstructed samples of the at least one inter-frame reference image and the filter coefficients corresponding to the current block, and the filtered reconstructed value of the current block is determined, a first generation value can be further determined according to the filtered reconstructed value of the current block, and a first syntax element identification information is determined based on the first generation value, and the first syntax element identification information is written into the bitstream; wherein the first syntax element identification information is used to determine whether the current block is filtered using TALF.

[0467] Further, in the embodiments of the present application, when the first generation value is determined according to the filtered reconstructed value of the current block, in each candidate mode, a second generation value of the filter coefficients corresponding to the current block is determined according to the filtered reconstructed value of the current block; and the smallest second generation value is determined as the first generation value corresponding to the case that the current block is filtered using TALF.

[0468] It can be understood that in the embodiments of the present application, each candidate filter can be traversed in each candidate mode, and a second generation value corresponding to the case that the current block is filtered using the filter coefficients corresponding to the current block is determined respectively, wherein the method for calculating the generation value includes but is not limited to rate-distortion optimization algorithm.

[0469] Correspondingly, in the embodiments of the present application, in each candidate mode, a plurality of second generation values can be calculated respectively under the traversal of the candidate filter and the plurality of possible combinations of the filter coefficients, and then the smallest generation value in the plurality of second generation values is determined as the first generation value corresponding to the case that the current block is filtered using TALF in the candidate mode.

[0470] Further, in the embodiments of the present application, when the first syntax element identification information is determined based on the first generation value, a third generation value corresponding to the case that the current block is not filtered using TALF can be determined; in the case that the first generation value is greater than or equal to the third generation value, the first syntax element identification information is set to indicate that the current block is not filtered using TALF; in the case that the first generation value is less than the third generation value, the first syntax element identification information is set to indicate that the current block is filtered using TALF.

[0471] It can be understood that in the embodiments of the present application, the optimal filter set number, whether each CTU uses TALF and which set of TALF filters is calculated by rate distortion optimization in each mode according to 1-4 respectively to determine whether each CTU opens TALF filtering.

[0472] It should be noted that in the embodiments of the present application, the calculation of the value of the cost can be performed in any manner, including but not limited to the rate distortion optimization (Rate-distortion optimization, RDO) algorithm.

[0473] Further, in the embodiments of the present application, the first syntax element identification information can be used to determine whether the current block in the current image is filtered using TALF, that is, based on the first syntax element identification information, it can be determined whether to filter the reconstructed value of the image component of the current block using the TALF filter.

[0474] Further, in the embodiments of the present application, the first syntax element identification information can be used to determine whether to filter the current block using TALF. The first syntax element identification information can be a flag, wherein if the current block is a CTU corresponding to the current image, the first syntax element identification information can be a CTU-level flag, and if the current block is a CU corresponding to the current image, the first syntax element identification information can be a CU-level flag. Of course, corresponding to any size sub-region in the current image, the first syntax element identification information can also be a flag of other block levels. The present application does not make specific limitations.

[0475] Further, in the embodiments of the present application, it can be determined whether to filter the current block using TALF by the value of the first syntax element identification information.

[0476] Exemplarily, in some embodiments, in the case that the value of the first syntax element identification information is a first value, it is determined that the current block is not filtered using TALF. In the case that the value of the first syntax element identification information is not the first value, it is determined that the current block is filtered using TALF.

[0477] It should be noted that in the embodiments of the present application, the first syntax element identification information can be used to indicate whether the current block is filtered using TALF. In addition, the first value can be in the form of a parameter or in the form of a number, which is not limited here.

[0478] Exemplarily, in some embodiments, if the first syntax element identification information is a block-level flag, in one specific example, the first value can be set as 0; in another specific example, the first value can also be set as false. The first value in the embodiments of the present application is not limited in any way.

[0479] Taking the first value as 0 as an example, in the embodiments of the present application, if the value of the first syntax element identification information is 0, it can be determined that the current block does not use TALF for filtering. Otherwise, if the value of the first syntax element identification information is not 0, it can be determined that the current block uses TALF for filtering.

[0480] Exemplarily, in some embodiments, if the current block of the current picture is a CTU, the first syntax element identification information can be a CTU-level syntax, for example, the first syntax element identification information can be represented by a syntax element control identification talf_ctb_idc[CtbAddrX][CtbAddrY], that is, talf_ctb_idc[CtbAddrX][CtbAddrY] indicates whether TALF is used for filtering the current block of the current picture. If the value of talf_ctb_idc[CtbAddrX][CtbAddrY] is 0, it is determined that TALF is not used for filtering the current block, and if the value of talf_ctb_idc[CtbAddrX][CtbAddrY] is not 0, it is determined that TALF is used for filtering the current block. Of course, the value of talf_ctb_idc[CtbAddrX][CtbAddrY] is not limited to 0, and the present application does not make specific limitations.

[0481] Further, in the embodiments of the present application, the fourth generation value corresponding to each candidate mode when the current picture uses TALF for filtering can also be determined according to the filtered reconstructed value of the current block under each candidate mode; and the smallest fourth generation value is determined as the fifth generation value corresponding to when the current picture uses TALF for filtering.

[0482] Further, in the embodiments of the present application, the sixth generation value corresponding to when the current picture does not use TALF for filtering is determined; in the case where the fifth generation value is greater than or equal to the sixth generation value, the second syntax element identification information is set to indicate that the current picture does not use TALF for filtering; and in the case where the fifth generation value is less than the sixth generation value, the second syntax element identification information is set to indicate that the current picture uses TALF for filtering.

[0483] Further, in embodiments of the present application, the first syntax element is written into the bitstream in the case that the second syntax element identification information indicates that the current picture is filtered using TALF.

[0484] Further, in embodiments of the present application, the second syntax element identification information can be used to determine whether the current picture is allowed to be filtered using TALF. In some embodiments, the second syntax element identification information can be a flag.

[0485] It should be noted that, in embodiments of the present application, the second syntax element identification information can be a picture-level flag, a slice-level flag, a sub-picture-level flag, or a tile-level flag. The present application does not make any specific limitation.

[0486] For example, in some embodiments, if the second syntax element identification information is a slice-level flag, the second syntax element identification information can be used to determine whether the current slice is allowed to be filtered using TALF.

[0487] Further, in embodiments of the present application, the value of the second syntax element identification information can be used to determine whether the current picture (and / or the current slice) is allowed to be filtered using TALF.

[0488] For example, in some embodiments, in the case that the value of the second syntax element identification information is a second value, it is determined that the current picture (and / or the current slice) is allowed to be filtered using TALF. In the case that the value of the second syntax element identification information is a third value, it is determined that the current picture (and / or the current slice) is not filtered using TALF.

[0489] It should be noted that, in embodiments of the present application, the second syntax element identification information can be used to indicate whether the current picture (and / or the current slice) is filtered using TALF. In addition, the second value and the third value are different, and the second value and the third value can be in the form of a parameter or in the form of a number, which are not limited in the present application.

[0490] It should be further noted that, if the second syntax element identification information is a picture-level flag or a slice-level flag, in one specific example, the second value can be set to 1, and the third value can be set to 0; in another specific example, the second value can be set to true, and the third value can be set to false; or in yet another specific example, the second value can be set to 0, and the third value can be set to 1; or the second value can be set to false, and the third value can be set to true. The second value and the third value in embodiments of the present application are not limited in any way.

[0491] For example, assuming the second syntax element identification information is a picture-level flag, and the second value is 1 and the third value is 0, in the embodiments of the present application, if the second syntax element identification information has the value of 1, it can be determined that the current picture uses TALF for filtering. Otherwise, if the second syntax element identification information has the value of 0, it can be determined that the current picture does not use TALF for filtering.

[0492] For example, in some embodiments, assuming the second syntax element identification information is a picture-level flag, the second syntax element identification information can be represented by the syntax element ph_talf_enabled_flag, i.e., ph_talf_enabled_flag indicates whether the current picture is allowed to use TALF for filtering. If the value of ph_talf_enabled_flag is 0, it is determined that the current picture does not use TALF for filtering, and if the value of ph_talf_enabled_flag is 1, it is determined that the current picture is allowed to use TALF for filtering.

[0493] For example, in some embodiments, assuming the second syntax element identification information is a slice-level flag, the second syntax element identification information can be represented by the syntax element sh_talf_enabled_flag, i.e., sh_talf_enabled_flag indicates whether the current slice is allowed to use TALF for filtering. If the value of sh_talf_enabled_flag is 0, it is determined that the current slice does not use TALF for filtering, and if the value of sh_talf_enabled_flag is 1, it is determined that the current slice is allowed to use TALF for filtering.

[0494] Of course, the value of sh_talf_enabled_flag is not limited to 0 and 1, and the present application does not make specific limitations.

[0495] As can be seen, in the embodiments of the present application, the coding of the first syntax element identification information can be dependent on the second syntax element identification information. That is, the block-level identification can depend on the picture-level identification, or the block-level identification can depend on the slice-level identification. For example, in the case of determining that the current picture (and / or the current slice) uses TALF for filtering based on the second syntax element identification information, the coding of the first syntax element identification information can be further performed, otherwise, the first syntax element identification information does not need to be coded.

[0496] Further, in the embodiments of the present application, third syntax element identification information can be determined and written into the bitstream, wherein the third syntax element identification information is used to determine whether the current sequence is filtered by TALF; and the second syntax element is written into the bitstream in the case that the third syntax element identification information indicates that the current sequence is filtered by TALF.

[0497] Further, in the embodiments of the present application, the third syntax element identification information can be used to determine whether the current sequence is filtered by TALF. The third syntax element identification information can be a flag, for example, a sequence-level flag.

[0498] Further, in the embodiments of the present application, whether the current sequence is filtered by TALF can be determined according to the value of the third syntax element identification information.

[0499] For example, in some embodiments, in the case that the value of the third syntax element identification information is a fourth value, it is determined that the current sequence is filtered by TALF; and in the case that the value of the third syntax element identification information is a fifth value, it is determined that the current sequence is not filtered by TALF.

[0500] It should be noted that, in the embodiments of the present application, the third syntax element identification information can be used to indicate whether the current sequence is filtered by TALF. In addition, the fourth value and the fifth value are different, and the fourth value and the fifth value can be in the form of a parameter or in the form of a number, which are not limited herein.

[0501] It should also be noted that, if the third syntax element identification information is a sequence-level flag, in one specific example, the fourth value can be set to 1 and the fifth value can be set to 0; in another specific example, the fourth value can also be set to true and the fifth value can also be set to false; or in yet another specific example, the fourth value can also be set to 0 and the fifth value can also be set to 1; or the fourth value can also be set to false and the fifth value can also be set to true. The fourth value and the fifth value in the embodiments of the present application are not limited.

[0502] For example, assuming that the third syntax element identification information is a sequence-level flag, the fourth value is 1 and the fifth value is 0, in the embodiments of the present application, if the value of the third syntax element identification information is 1, it is determined that the current sequence is filtered by TALF; otherwise, if the value of the third syntax element identification information is 0, it is determined that the current sequence is not filtered by TALF.

[0503] Exemplarily, in some embodiments, assuming that the third syntax element identification information is a sequence level flag, the third syntax element identification information can be represented by a syntax element sps_talf_enabled_flag, i.e., sps_talf_enabled_flag indicates whether TALF is used for filtering the current sequence. If the value of sps_talf_enabled_flag is 0, it is determined that TALF is not used for filtering the current sequence, and if the value of sps_talf_enabled_flag is 1, it is determined that TALF is used for filtering the current sequence.

[0504] Of course, the value of sps_talf_enabled_flag is not limited to 0 and 1, and the present application does not make specific limitations.

[0505] Therefore, in the embodiments of the present application, the coding of the second syntax element identification information can be dependent on the third syntax element identification information, i.e., the identification at the picture level (and / or slice level) can be dependent on the identification at the sequence level. For example, in the case of determining that TALF is used for filtering the current sequence based on the third syntax element identification information, the coding of the second syntax element identification information can be further performed, otherwise, the second syntax element identification information does not need to be coded.

[0506] Further, in the embodiments of the present application, the TALF identification at the sequence level can be dependent on the ALF identification at the sequence level, i.e., the coding of the third syntax element identification information can have a dependent relationship with the coding of the sequence level syntax element of ALF.

[0507] Further, in the embodiments of the present application, the identification at the picture level (and / or slice level) dependent on the identification at the sequence level can include that the TALF identification at the picture level (and / or slice level) dependent on the TALF identification at the sequence level, and can also include that the TALF identification at the picture level (and / or slice level) dependent on the ALF identification at the sequence level.

[0508] That is, in the embodiments of the present application, the coding of the second syntax element identification information can be dependent on the third syntax element identification information, or can be dependent on the ALF identification at the sequence level.

[0509] Further, in the embodiments of the present application, the TALF identification at the picture level (and / or slice level) can also be dependent on the ALF identification at the picture level (and / or slice level).

[0510] That is, in the embodiments of the present application, the coding of the second syntax element identification information can also be dependent on the ALF identification at the picture level (and / or slice level).

[0511] Exemplarily, in some embodiments, the TALF identification at the picture level (and / or slice level) can also rely on some high level syntax, for example, a high level syntax element pps alf info in ph flag to identify the control identification of the picture level or slice level of the ALF using the syntax element of the picture level or slice level to parse the ALF, and the ALF identifier also exists in the slice header.

[0512] Further, in the embodiments of the present application, after the optimal candidate mode, the candidate filter and the corresponding filter coefficient are selected based on the rate-distortion optimization algorithm, the TALF parameters can be further determined, and the TALF parameters and the filter coefficient corresponding to the current block are written into the bitstream. Among them, the TALF parameters include one or more of the mode parameter, the number parameter, the coefficient value parameter and the coefficient sign parameter.

[0513] It can be understood that in the embodiments of the present application, the mode parameter can be used to determine the filter mode of the TALF filter, wherein for different filter modes, the inter prediction reference picture corresponding to the current picture determined can be different. The number parameter can be used to determine the number of groups of the filter corresponding to the current picture. The coefficient value parameter can be used to determine the absolute value size of the filter coefficient. The coefficient sign parameter can be used to determine the sign of the filter coefficient, including positive and negative.

[0514] Exemplarily, in some embodiments, the coefficient value parameter can be determined according to the value of the filter coefficient corresponding to the current block; the coefficient sign parameter can be determined according to the sign of the filter coefficient corresponding to the current block; and the TALF parameter is determined according to the coefficient value parameter and the coefficient sign parameter, that is, the coefficient value parameter and the coefficient sign parameter in the TALF parameter are determined.

[0515] Exemplarily, in some embodiments, the filter mode corresponding to the fifth generation value and the number of groups of candidate filters can be determined; the mode parameter can be determined according to the filter mode and the current configuration; the number parameter can be determined according to the number of groups of candidate filters; and the TALF parameter is determined according to the mode parameter and the number parameter, that is, the mode parameter and the number parameter in the TALF parameter are determined.

[0516] Further, in the embodiments of the present application, the APS set corresponding to the TALF parameter can be determined; then the APS index corresponding to the APS set is determined, and the APS index is written into the bitstream.

[0517] It can be understood that in the embodiments of the present application, the APS index can be used to determine the APS parameter set corresponding to the current picture, that is, the APS set corresponding to the current picture can be determined through the APS index.

[0518] That is, in the embodiments of the present application, the TALF parameters corresponding to the current image can be saved in the APS, so that the TALF parameters can be obtained by parsing in the APS.

[0519] Further, in the embodiments of the present application, the TALF parameters can also be directly written into the code stream.

[0520] That is, in the embodiments of the present application, the TALF parameters corresponding to the current image can also be directly written into the code stream. For example, the TALF parameters can be coded in the image header or the slice header.

[0521] It can be understood that, in the embodiments of the present application, the first syntax element identification information can also be used to determine the TALF information used by the current block, for example, the value of the first syntax element identification information can be set according to the candidate filter corresponding to the current block finally determined.

[0522] For example, in some embodiments, when the value of the first syntax element identification information is a first value, it is indicated that the TALF is not used for filtering the current block. When the value of the first syntax element identification information is not the first value, it is indicated that the TALF is used for filtering the current block, and the value of the first syntax element identification information can be set to indicate the group number index of the filter corresponding to the current block.

[0523] It can be understood that, in the embodiments of the present application, when it is determined that the TALF is used for filtering the current block, the value of the first syntax element identification information can be set to indicate the group number index of the filter corresponding to the current block. The group number index of the filter can represent the index of the TALF filter used by the current block in the filter set coded in the current image, that is, the group number index of the filter can be used to select the candidate filter corresponding to the current block from the multiple filters corresponding to the current image.

[0524] For example, in the embodiments of the present application, the optimal filter mode, the group number of the filter, and the CTU switch optimized by the rate-distortion optimization are used to filter the reconstructed image, and the filter mode, the group number of the filter, and the filter coefficient are coded in the image header, the CTU switch, and the like.

[0525] It can be understood that the coding method proposed in the embodiments of the present application is a method of using the reconstructed sample value in the time domain as an information source to improve the reconstruction quality of the current image, and the information is extracted from the reconstructed sample of the reconstructed image by the TALF filter to filter the current image.

[0526] It can be understood that the filter coefficient of the filter in the time domain in the coding method proposed in the embodiments of the present application can be predefined or transmitted through the code stream.

[0527] It can be understood that the encoding method proposed in the embodiments of the present application can refer to different encoding configurations, and different filtering modes under different filtering modes are proposed. Among them, under the RA configuration, the filtering mode can be divided into forward filtering mode, reverse filtering mode and bidirectional filtering mode; under the LD configuration, the filtering mode can be divided into first forward filtering mode, second forward filtering mode and third forward filtering mode.

[0528] It can be understood that the encoding method proposed in the embodiments of the present application can be used for the luminance component or the chrominance component.

[0529] It can be understood that the encoding method proposed in the embodiments of the present application can be any level of switch identification of TALF, such as TALF which can be switched as a unit of image, or switched as a unit of CTU, or switched as a unit of CU, or switched in other ways of dividing the image into sub-regions.

[0530] It can be understood that the encoding method proposed in the embodiments of the present application can also select to skip the coding and decoding processing of the CTU identifier, and directly decide whether to perform TALF filtering on the entire image through the image-level identifier, at this time, the current image only corresponds to one candidate filter.

[0531] The embodiments of the present application provide an encoding method, in the process of performing TALF filtering, the vector information including motion vector or block vector corresponding to the sample position can be used to obtain the corresponding reference position, and the TALF filtering of the sample position is completed based on the reference position, that is, the vector information is introduced to assist the TALF filtering, so as to further improve the filtering effect and improve the coding and decoding performance.

[0532] Based on the above embodiments, the embodiments of the present application propose a time domain adaptive loop filtering technology, which is a technology of using the reconstructed image on the time domain as an information source to perform adaptive loop filtering on the current frame reconstructed image. Further, in the process of performing TALF filtering, the vector information including motion vector or block vector corresponding to the sample position can be used to obtain the corresponding reference position, and the TALF filtering of the sample position is completed based on the reference position, that is, the vector information is introduced to assist the TALF filtering, so as to further improve the filtering effect and improve the coding and decoding performance.

[0533] Embodiments of the present application propose a technique of using reconstructed images in the time domain to improve the quality of a current image. The encoder can obtain one or more sets of filter coefficients for improving the quality of the current reconstructed image and transmit the filter coefficients through a bitstream. The decoder uses information including but not limited to reconstructed values in the time domain as input to filter the current image by using the filter coefficients parsed.

[0534] Embodiments of the present application propose a technique of using reconstructed images in the time domain to improve the quality of a current image. The encoder can select appropriate filters and coefficients from a predefined set of filters and coefficients, and transmit syntax elements related to whether the filters and coefficients are used through a bitstream. The decoder determines the filters and coefficients by parsing the syntax elements, and uses information including but not limited to reconstructed values in the time domain as input to filter the current image.

[0535] Embodiments of the present application propose a technique of using reconstructed images in the time domain to improve the quality of a current image. The encoder can select appropriate filters and coefficients from a predefined set of filters and coefficients, and transmit syntax elements related to whether the filters and coefficients are used through a bitstream. The decoder determines the filters and coefficients by parsing the syntax elements, and uses information including but not limited to reconstructed values in the time domain as input to filter the current image.

[0536] Exemplarily, in some embodiments, a block-level identifier (first syntax element identifier information) can be used to indicate whether the current block is filtered using TALF.

[0537] Exemplarily, in some embodiments, a CTU-level identifier is used to control whether TALF is enabled. In addition, since a frame of image can have multiple TALF (candidate filter) for selection, when the CTU-level TALF identifier is true, the relevant identifier indicating the selected TALF filter for filtering the current CTU should also be parsed.

[0538] Exemplarily, in some embodiments, the parsing process of the first syntax element identifier information talf_ctb_idc[CtbAddrX][CtbAddrY] is as follows:

[0539] When the current slice / current picture allows the use of TALF, the CTU-level control flag talf_ctb_idc[CtbAddrX][CtbAddrY] is also coded when coding each CTU-level syntax, and the flag is non-zero when the current CTU uses TALF, and the value of talf_ctb_idc[CtbAddrX][CtbAddrY] - 1 corresponds to the index of the TALF filter used by the current CTU in the filter set coded in the current picture / slice. For example, if there are 3 sets of TALF filter coefficients coded in the current picture / slice, talf_ctb_idc[CtbAddrX][CtbAddrY] = 0 means that the current CTU does not use TALF, talf_ctb_idc[CtbAddrX][CtbAddrY] = 1 means that the first set of TALF filter coefficients is used to filter the current CTU, talf_ctb_idc[CtbAddrX][CtbAddrY] = 2 means that the second set of TALF filter coefficients is used to filter the current CTU, and talf_ctb_idc[CtbAddrX][CtbAddrY] = 3 means that the third set of TALF filter coefficients is used to filter the current CTU.

[0540] Of course, the CTU flag can also be discarded, and a decision can be made directly on whether to perform TALF filtering on the entire picture, or a CU-level flag can be used instead of the CTU, or a block-level flag of another size can be used.

[0541] Exemplarily, in some embodiments, up to 4 sets of TALF coefficients (4 sets of filter coefficients corresponding to 4 sets of candidate filters) can be coded for each picture. Assuming that TALF is placed after ALF, a reference picture (inter-frame reference picture) can be used to further improve the quality of the reconstructed picture after ALF filtering.

[0542] Exemplarily, in some embodiments, a 7x7 symmetric filter and a 5x5 symmetric filter can be used. For TALF using one reconstructed picture (inter-frame reference picture) as input, 1 7x7, 13-coefficient symmetric filter is used for filtering; for TALF using two reconstructed pictures as input, 2 5x5, 7-coefficient symmetric filters are used for filtering.

[0543] Exemplarily, in some embodiments, the difference between the reconstructed value of the reconstructed picture and the reconstructed value of the current picture can be selected as the input of the TALF filter, assuming that the current picture corresponds to an inter-frame reference picture, at this time, the process of filtering the sample value of the coordinate (x, y) position (current sample position) in the current picture using a 13-filter coefficient, 7x7 symmetric filter is as follows:

[0544] Exemplarily, in some embodiments, the difference between the reconstructed value of the reconstructed image and the current image reconstructed value can be selected as the TALF filter input, assuming that the current image corresponds to two inter-frame reference images, at this time, the process of filtering the sample value of the coordinate (x, y) position (current sample position) in the current image using 7 filter coefficients, 5x5 symmetric 2 filters is as follows:

[0545] It should be noted that in the embodiments of the present application, the filter related parameters (TALF parameters and filter coefficients) can be coded and decoded at the slice header (or image header), or can be written into the APS parameter set, which is not specifically limited in the present application.

[0546] It should be noted that in the embodiments of the present application, TALF filtering can be selected to be used for the luminance component, or TALF filtering can be selected to be used for the chrominance component (color component).

[0547] It should be noted that in the embodiments of the present application, when TALF filtering is only used for the luminance component, the chrominance component can also use the information of the reconstructed image in the time domain as auxiliary to improve the reconstructed quality of the current chrominance image.

[0548] Exemplarily, in some embodiments, at the encoding end, it is assumed that TALF is placed after ALF, that is, after ALF filtering, the decision process of TALF is entered.

[0549] First, according to the encoding configuration RA or LD, the mode allowed to be used by TALF is judged, and the TALF mode is three under RA and LD. The forward filtering is mode 1, the backward filtering is mode 2, and the bidirectional filtering is mode 3 under RA. The reference image POC N-1 is used as the TALF filter input for mode 1, the reference image POC N-2 is used as the TALF filter input for mode 2, and the reference images POC N-1 and N-2 are used as the TALF filter input for mode 3 under LD.

[0550] Each filtering mode is traversed, the current image reconstructed value under the corresponding mode is used, the reconstructed value of the reference reconstructed image and the original value of the current image are used to construct the Wiener-Hopf equation and analyze 1-4 groups of filter coefficients, and whether each CTU opens TALF filtering is judged according to 1-4, respectively, the optimal filter group number under each mode is calculated, whether each CTU uses TALF and which group of TALF filter is used.

[0551] Further, the optimal filter mode, filter set number, and CTU switch optimized by the usage rate distortion are used to filter the reconstructed image, and the filter mode, filter set number, and filter coefficient are encoded in the image header, CTU switch, and the like.

[0552] For example, in some embodiments, at the decoding end, the decoder parses the code stream to obtain the filter mode, filter number, filter coefficient, whether each CTU uses the TALF filter, and which filter set is used for filtering of the current image.

[0553] After ALF filtering, the CTU in the current image for which the TALF is selected is filtered using the corresponding filter.

[0554] For example, in some embodiments, when performing the temporal adaptive loop filtering, a sequence-level syntax element, such as a third syntax element, can be parsed, and the parsing process is as follows:

[0555] The sequence identifier sps_talf_enabled_flag is used to indicate whether the current sequence can use the TALF. When the syntax element does not exist in the code stream, the value of the syntax element is inferred to be 0. When the value of the syntax element is 1, the current sequence can use the TALF, and when the value of the syntax element is 0, the TALF cannot be used.

[0556] For example, in some embodiments, when performing the temporal adaptive loop filtering, a slice-level syntax element, such as a second syntax element, can be parsed, and the parsing process is as follows:

[0557] The sh_talf_enabled_flag syntax element is a slice header TALF enable identifier. When the syntax element does not exist in the code stream, the value of the syntax element is inferred to be 0. When the value of the syntax element is 1, the current slice can use the TALF, and when the value of the syntax element is 0, the TALF cannot be used.

[0558] The talf_filter_mode syntax element is a syntax element indicating which TALF mode is used for the current slice. When the value of the syntax element is 0, forward filtering is used under the RA configuration or forward filtering is used on the first frame before the LD. When the value of the syntax element is 1, backward filtering is used under the RA configuration or forward filtering is used on the second frame before the LD. When the value of the syntax element is 2, bidirectional filtering is used under the RA configuration or filtering is performed on the two frames before the LD. When the syntax element does not exist in the code stream, the value of the syntax element is 0.

[0559] The talf num filters signalled minusl syntax element is a syntax element indicating the number of TALF filters contained in the current slice minus one. In the present scheme, since a slice can have at most four TALF filters, the value of talf num filters signalled minusl can be 0, 1, 2 or 3. The value of 0 of this syntax element indicates that the current slice has one TALF filter, the value of 1 of this syntax element indicates that the current slice has two TALF filters, the value of 2 of this syntax element indicates that the current slice has three TALF filters, and the value of 3 of this syntax element indicates that the current slice has four TALF filters. When this syntax element is not present in the bitstream, its value is 0.

[0560] The numCoeff variable indicates the number of filter coefficients in the filter. Since the filter used in the forward and backward directions in the RA configuration and the first and second nearest frames in the LD configuration in the present scheme is a 13-coefficient 7x7 symmetric filter, numCoeff is equal to 13 in the corresponding case (i.e., talf_filter_mode is 0 or 1). The filter used in the bidirectional filtering in the RA configuration and the use of the two nearest frames in the LD configuration is two 5x5 7-coefficient filters, a total of 14 coefficients, so numCoeff is equal to 14 in the corresponding case (i.e., talf_filter_mode is 2).

[0561] The talf_coeff_abs[sfIdx][j] syntax element indicates the absolute value of the jth coefficient of the sfIdxth group of filters, which is represented using K-order exponential Golomb code, where K is 0. When this syntax element is not present in the bitstream, its value is 0.

[0562] The talf_coeff_sign[sfIdx][j] syntax element indicates the positive or negative sign of the jth coefficient of the sfIdxth group of filters. The value of 1 of this syntax element indicates that the coefficient is negative, and the value of 0 of this syntax element indicates that the coefficient is positive. When this syntax element is not present in the bitstream, its value is 0.

[0563] Exemplarily, in some embodiments, when performing time-domain adaptive loop filtering, a CTU-level syntax element, such as a first syntax element identification information, can be parsed, and the parsing process is as follows:

[0564] talf_ctb_idc[CtbAddrX][CtbAddrY] syntax element indicates whether the coding tree block of the horizontal direction number CtbAddrX and the vertical direction number CtbAddrY uses the TALF filter, and the value range thereof is 0-talf_num_filters_signalled_minus1. For example, when talf_num_filters_signalled_minus1 is 1, the value of talf_ctb_idc can be 0 or 1, and for example, when talf_num_filters_signalled_minus1 is 2, the value of talf_ctb_idc can be 0, 1 or 2. talf_ctb_idc is 0, which indicates that the current coding tree block does not use the TALF filter, 1 indicates that the first group of filters is used for filtering, and 2 indicates that the second group of filters is used for filtering. When the syntax element does not exist in the code stream, the value thereof is 0.

[0565] Further, in the embodiments of the present application, the filter process can be reconstructed according to the parsed filter number, filter type, filter coefficient absolute value and filter coefficient sign.

[0566] Further, in the embodiments of the present application, assuming that the reconstructed value of the luminance component is filtered by TALF, i.e., the current block is a luminance block (luminance coding tree block), when the talf_ctb_idc identifier of the current luminance coding tree block is non-zero, the TALF filter needs to be performed.

[0567] Exemplarily, in some embodiments, the input of the filter process includes:

[0568] 1. a reconstructed luminance image array rec filtered by the luminance adaptive loop filter,

[0569] 2. a corresponding time domain reconstructed luminance image array, for example, when the talf_filter_mode syntax element is 0, the forward nearest one frame time domain reconstructed image is used under the RA configuration or the first frame reconstructed image rec0 is used under the LD; when the talf_filter_mode syntax element is 1, the backward nearest one frame time domain reconstructed image is used under the RA configuration or the first frame reconstructed image rec1 is used under the LD; and when the talf_filter_mode syntax element is 2, two reconstructed image arrays rec0 and rec1 are used,

[0570] 3. the luminance coordinates (xCtb, yCtb) of the current luminance coding tree block in the current image,

[0571] 4. the width tAlfWidth and the height tAlfHeight of the luminance coding tree block,

[0572] 5. Coefficients of the time adaptive loop filter tAlfCoeff [ ][ ]

[0573] Correspondingly, the filtering calculation for each position in the luminance coding block is as follows:

[0574] numCoeff is 14:

[0575] Wherein, in the above filtering calculation process, shift is a positive integer, shift is 6 in the present scheme, BitDepth is the bit depth of the luminance component, and x and y represent the horizontal and vertical coordinates of the reconstructed image array.

[0576] numCoeff is 13 and talf_filter_mode is 0:

[0577] numCoeff is 13 and talf_filter_mode is 1:

[0578] It should be noted that in the embodiments of the present application, in the filtering process, the same image edge padding method as ALF is used to obtain the TALF input value outside the image range required in the filtering process.

[0579] Exemplarily, in some embodiments, the ALF filtering is padded at the edge of the coding tree block, and the filtering in one coding tree block does not use the reconstructed value of other coding tree blocks, in which case the TALF can also be padded at the edge of the coding tree block.

[0580] In a video, objects usually change positions in different images, in order to better capture the displacement of objects in the video, motion vectors are usually used to guide inter-frame coding, and the motion vector represents the position difference between the current coding block and the similar block in the reference image. Through the motion vector, a large amount of inter-frame coding redundancy can be saved. In the intra-frame coding frame, the object can also appear repeatedly in the current frame, which is usually guided by the block vector for coding, and the block vector represents the position difference between the current coding block and the reference block in the current image. Through the block vector, a large amount of intra-frame coding redundancy can be saved.

[0581] In the conventional TALF scheme, when the time domain adaptive filter is used, the corresponding current position on the reconstructed image can be selected as the input of the filter.

[0582] Exemplarily, in some embodiments, FIG. 26 is a schematic diagram of a filtering position according to an embodiment of the present application. As shown in FIG. 26, in the forward / backward filtering, the corresponding position of the current position (current filtering position) of the reference image can be selected as the filtering input.

[0583] Considering the motion vector of the object movement in the inter-frame coding, an embodiment of the present application proposes that the corresponding position of the reconstructed image / current image can be further offset based on the vector information, such as the motion vector MV or the block vector BV, used by the coding block of the current position when filtering the current position, and the offset corresponding position and the surrounding reconstructed pixels thereof are used as the input of the filter.

[0584] Exemplarily, in some embodiments, FIG. 27 is a schematic diagram of a filtering position according to an embodiment of the present application. As shown in FIG. 27, in the forward / backward filtering, the corresponding position of the current position can be offset by the MV, i.e., the filtering center position of the reference image can be offset by the MV.

[0585] In which, since the current image can have one or more reference images, it means that the MV can be directed to any one of the reference images, depending on the MV used by the coding block of the current filtering position and the reference image directed by the MV.

[0586] Exemplarily, in some embodiments, FIG. 28 is a schematic diagram of a filtering position according to an embodiment of the present application. As shown in FIG. 28, in the forward / backward filtering, the corresponding position of the current position can be offset by the BV, i.e., the filtering center position of the current image can be offset by the BV.

[0587] For the case of two reconstructed images as input, the corresponding positions can be offset by two MVs or BVs.

[0588] Exemplarily, in some embodiments, FIG. 29 is a schematic diagram of a filtering position according to an embodiment of the present application. As shown in FIG. 29, for the case of bidirectional filtering or two reconstructed images as input, the offset can be based on two MVs.

[0589] Exemplarily, in some embodiments, FIG. 30 is a schematic diagram of a filtering position according to an embodiment of the present application. As shown in FIG. 30, for the case of bidirectional filtering as input, the offset can be based on two BVs.

[0590] Exemplarily, in some embodiments, FIG. 31 is a schematic diagram of a filtering position according to an embodiment of the present application. As shown in FIG. 31, the bidirectional filtering of one MV and one BV combination can also occur.

[0591] It can be understood that the loop filtering link is performed on the reconstructed image after the current image has been reconstructed, so when the temporal loop filtering is performed, it can be determined that the current position is selected in the intra prediction, inter prediction or IBC prediction. For example, in the ECM reference software, the block to which the current position belongs is an intra prediction block, if the selected is an intra TMP prediction mode, there will be one or more saved BVs in the motion buffer; if the selected is an IBC prediction mode, there will also be one or more saved BVs in the motion buffer; if the selected is an inter prediction mode, there will be one or more saved MVs in the motion buffer.

[0592] Further, according to the CTU level control identifier, the selected filtering mode, and whether the motion buffer obtains the available MV0 and MV1 or BV0 and BV1 pointing to the reference picture list 0 and the reference picture list 1, it is determined whether to perform filtering.

[0593] For example, if the MV0 exists, the CTU control identifier filtering is turned on, and the selected is the forward filtering, the filtering can be performed on the current position.

[0594] For example, if the BV0 exists, the CTU control identifier filtering is turned on, and the selected is the forward filtering, the filtering can be performed on the current position.

[0595] For example, if the MV1 exists, the CTU control identifier filtering is turned on, and the selected is the backward filtering, the filtering can be performed on the current position.

[0596] For example, if the BV1 exists, the CTU control identifier filtering is turned on, and the selected is the backward filtering, the filtering can be performed on the current position.

[0597] For example, if the MV0 and MV1 exist, the CTU control identifier filtering is turned on, and the selected is the bidirectional filtering, the filtering can be performed on the current position.

[0598] For example, if the BV0 and BV1 exist, the CTU control identifier filtering is turned on, and the selected is the bidirectional filtering, the filtering can be performed on the current position.

[0599] Next, taking the MV as an example, the scheme of using vector information to assist the TALF filtering proposed in the embodiments of the present application is exemplarily described, which includes the TALF method of identifying whether to offset the reference position.

[0600] For example, one implementation is that the parsed slice level syntax element needs to additionally include an identification indicating whether the TALF is assisted by the MV, and the parsed slice level syntax element is as follows:

[0601] Wherein, talf mv guided flag is used to identify whether the TALF under the current slice is filtered based on the motion vector or the block vector assistance, when the identified value is 1, the time domain adaptive loop filter of the current slice is based on the motion vector block vector (vector information) assistance; when the identified value is 0, the time domain adaptive loop filter of the current slice is not based on the motion vector block vector (vector information) assistance. When the syntax element does not exist in the code stream, the value thereof can be 0 by default.

[0602] Further, when the TALF filtering is performed on the luma coding tree block, the talf_ctb_idc of the current luma coding tree block (current block) is identified as non-zero, then the TALF filtering is needed, that is, talf_ctb_idc indicates that the current block is filtered by TALF. Wherein, the input of TALF filtering is:

[0603] 1, a reconstructed luma image array rec filtered by luma adaptive loop filter

[0604] 2, the mode value of talf_filter_mode

[0605] 3, the identification value of talf_mv_guided_flag

[0606] 4, indicating the luma coordinates (xCtb, yCtb) of the current luma coding tree block in the current image

[0607] 5, the width tAlfWidth and the height tAlfHeight of the luma coding tree block

[0608] 6, the coefficient tAlfCoeff of the time domain adaptive loop filter

[0609] It should be noted that for each position (x, y) in the current luma coding tree block, x takes the value range of xCtb to xCtb+tAlfWidth-1, and y takes the value range of yCtb to yCtb+tAlfHeight-1.

[0610] Further, when the TALF filtering is performed on the luma coding tree block, the talf_ctb_idc of the current luma coding tree block (current block) is identified as non-zero, then the TALF filtering is needed, that is, talf_ctb_idc indicates that the current block is filtered by TALF. Wherein, the input of TALF filtering is:

[0611] First, get the reconstructed image and MV:

[0612] Determine the input image of the filter according to talf_mv_guided_flag and talf_filter_mode

[0613] If talf_filter_mode is 0 and talf_mv_guided_flag is 0, the forward nearest reconstructed image rec0 is used as the input;

[0614] If talf_filter_mode is 0 and talf_mv_guided_flag is 1, check whether the prediction block to which the current position belongs selects MV0 pointing to a reference picture in reference picture list 0, and if so, use the reference picture rec0 pointed to by MV0 as input, otherwise skip filtering at the current position;

[0615] If talf_filter_mode is 1 and talf_mv_guided_flag is 0, use the backward nearest reconstructed picture rec1 as input for RA and the forward second nearest reconstructed picture rec1 as input for LD;

[0616] If talf_filter_mode is 1 and talf_mv_guided_flag is 1, check whether the prediction block to which the current position belongs selects MV1 pointing to a reference picture in reference picture list 1, and if so, use the reference picture rec1 pointed to by MV1 as input, otherwise skip filtering at the current position;

[0617] If talf_filter_mode is 2 and talf_mv_guided_flag is 0, use the forward nearest reconstructed picture rec0 and the backward nearest reconstructed picture rec1 as input for RA and the forward nearest reconstructed picture rec0 and the second nearest reconstructed picture rec1 as input for LD;

[0618] If talf_filter_mode is 2 and talf_mv_guided_flag is 1, check whether the prediction block to which the current position belongs selects both MV0 pointing to a reference picture in reference picture list 0 and MV1 pointing to a reference picture in reference picture list 1, and if so, use the reference picture rec0 pointed to by MV0 and the reference picture rec1 pointed to by MV1 as input, otherwise skip filtering at the current position.

[0619] Second step, get the shift value:

[0620] If MV0 and MV1 are found in the first step, since the MVs used in inter prediction are in sub-pixel accuracy, here the position offset on the reconstructed picture pointed to by the MV is obtained by rounding to the nearest integer pixel accuracy.

[0621] The horizontal integer pixel position offset of MV0 is: Offset0X = Mv0.Hor < 0? -((abs(Mv0.Hor) + 8) >> 4) : ((abs(Mv0.Hor) + 8) >> 4)

[0622] OffsetOY = MvO.Ver < 0? - (( abs( MvO.Ver ) + 8 ) » 4 ) : ( ( abs( MvO.Ver ) + 8 ) » 4 )

[0623] OffsetlX = Mv1.Hor < 0? - (( abs( Mv1.Hor ) + 8 ) » 4 ) : ( ( abs( Mv1.Hor ) + 8 ) » 4 )

[0624] OffsetlY = Mv1.Ver < 0? - (( abs( Mv1.Ver ) + 8 ) » 4 ) : ( ( abs( Mv1.Ver ) + 8 ) » 4 )

[0625] If talf mv guided flag is equal to 0 in the first step, OffsetOx, OffsetOY, OffsetlX and OffsetlY are set equal to 0.

[0626] Third step, according to the position offset obtained, reconstruct the image, filter the coefficient of (x, y) position, when numCoeff is 14: x0 = Clip3 (0, picWidth, x + Offset0X) y0 = Clip3 (0, picHeight, y + Offset0Y) x1 = Clip3 (0, picWidth, x + Offset1X) y1 = Clip3 (0, picHeight, y + Offset1Y) filterIdx = talf_ctb_idx - 1 sum = tAlfCoeff [filterIdx] [0] * (rec0 [x0] [y0] - rec [x] [y]) + tAlfCoeff [filterIdx] [1] * (rec1 [x1] [y1] - rec [x] [y]) + tAlfCoeff [filterIdx] [2] * (rec0 [x0-1] [y0] + rec0 [x0+1] [y0] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [3] * (rec1 [x1-1] [y1] + rec1 [x1+1] [y1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [4] * (rec0 [x0] [y0-1] + rec0 [x0] [y0+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [5] * (rec1 [x1] [y1-1] + rec1 [x1] [y1+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [6] * (rec0 [x0+1] [y0-1] + rec0 [x0-1] [y0+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [7] * (rec1 [x1+1] [y1-1] + rec1 [x1-1] [y1+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [8] * (rec0 [x0-1] [y0-1] + rec0 [x0+1] [y0+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [9] * (rec1 [x1-1] [y1-1] + rec1 [x1+1] [y1+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx]

[0010] * (rec0 [x0-2] [y0] + rec0 [x0+2] [y0] -2 * rec [x] [y]) +tAlfCoeff[ filterldx ] [ 0011 ] * ( rec1[ x1-2 ][ y1 ] + rec1[ x1+2 ][ y1 ] - 2 * rec[ x ][ y ] ) + tAlfCoeff[ filterldx ] [ 0012 ] * ( rec0[ x0 ][ y0-2 ] + rec0[ x0 ][ y0+2 ] - 2 * rec[ x ][ y ] ) + tAlfCoeff[ filterldx ] [ 0013 ] * ( rec1[ x1 ][ y1-2 ] + rec1[ x1 ][ y1+2 ] - 2 * rec[ x ][ y ] ) signSum = sum < 0? -1 : 1 scaledAbsSum = ( abs( sum ) + ( 1 << ( shift - 1 ) ) ) » shift scaledSum = scaledAbsSum * signSum rec'[ x ][ y ] = max( min( rec[ x ][ y ] + scaledSum, ( 1 << BitDepth ) - 1 ), 0 )

[0627] In the above process of calculating the filter, shift is 6, BitDepth variable is the bit depth of the luma component, x and y represent the horizontal and vertical coordinates of the reconstructed image array. x0, y0, x1, y1 are the coordinates of the center positions of the offset rec0 and rec1, respectively. When the reconstructed values in the coordinate positions rec0, rec1 array are obtained, the horizontal coordinate should be limited to 0 to the image width picWidth-1, and the vertical coordinate should be limited to 0 to the image height picHeight-1.

[0628] numCoeff is 13 and talf_filter_mode is 0: x0 = Clip3(0, picWidth, x + Offset0X) y0 = Clip3(0, picHeight, y + Offset0Y) filterldx = talf_ctb_idc - 1 sum = tAlfCoeff[filterldx][0] * (rec0[x0][y0] - rec[x][y]) + tAlfCoeff[filterldx][1] * (rec0[x0-1][y0] + rec0[x0+1][y0] - 2 * rec[x][y]) + tAlfCoeff[filterldx][2] * (rec0[x0][y0-1] + rec0[x0][y0+1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][3] * (rec0[x0+1][y0-1] + rec0[x0-1][y0+1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][4] * (rec0[x0-1][y0-1] + rec0[x0+1][y0+1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][5] * (rec0[x0-2][y0] + rec0[x0+2][y0] - 2 * rec[x][y]) + tAlfCoeff[filterldx][6] * (rec0[x0][y0-2] + rec0[x0][y0+2] - 2 * rec[x][y]) + tAlfCoeff[filterldx][7] * (rec0[x0-2][y0-1] + rec0[x0+2][y0+1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][8] * (rec0[x0-2][y0+1] + rec0[x0+2][y0-1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][9] * (rec0[x0-1][y0-2] + rec0[x0+1][y0+2] - 2 * rec[x][y]) + tAlfCoeff[filterldx]

[0010] * (rec0[x0-1][y0+2] + rec0[x0+1][y0-2] - 2 * rec[x][y]) + tAlfCoeff[filterldx]

[0011] * (rec0[x0-3][y0] + rec0[x0+3][y0] - 2 * rec[x][y]) +tAlfCoeff[ filterldx ] [ 0 ] * ( rec0[ x0 ][ y0 - 3 ] + rec0[ x0 ][ y0 + 3 ] - 2 * rec[ x ][ y ] ) signSum = sum < 0? -1 : 1 scaledAbsSum = ( abs( sum ) + ( 1 << ( shift - 1 ) ) ) » shift scaledSum = scaledAbsSum * signSum rec' [ x ][ y ] = max( min( rec[ x ][ y ] + scaledSum, ( 1 << BitDepth ) - 1 ), 0 )

[0629] numCoeff is 13 and talf_filter_mode is 1: x1 = Clip3(0, picWidth, x + Offset1X) y1 = Clip3(0, picHeight, y + Offset1Y) filterldx = talf_ctb_idc - 1 sum = tAlfCoeff[filterldx][0] * (rec1[x1][y1] - rec[x][y]) + tAlfCoeff[filterldx][1] * (rec1[x1-1][y1] + rec1[x1+1][y1] - 2*rec[x][y]) + tAlfCoeff[filterldx][2] * (rec1[x1][y1-1] + rec1[x1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][3] * (rec1[x1+1][y1-1] + rec1[x1-1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][4] * (rec1[x1-1][y1-1] + rec1[x1+1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][5] * (rec1[x1-2][y1] + rec1[x1+2][y1] - 2*rec[x][y]) + tAlfCoeff[filterldx][6] * (rec1[x1][y1-2] + rec1[x1][y1+2] - 2*rec[x][y]) + tAlfCoeff[filterldx][7] * (rec1[x1-2][y1-1] + rec1[x1+2][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][8] * (rec1[x1-2][y1+1] + rec1[x1+2][y1-1] - 2*rec[x][y]) + tAlfCoeff[filterldx][9] * (rec1[x1-1][y1-2] + rec1[x1+1][y1+2] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0010] * (rec1[x1-1][y1+2] + rec1[x1+1][y1-2] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0011] * (rec1[x1-3][y1] + rec1[x1+3][y1] - 2*rec[x][y]) +tAlfCoeff[ filterldx ][ 0012 ] * ( rec1[ x1 ][ y1 - 3 ] + rec1[ x1 ][ y1 + 3 ] - 2 * rec[ x ][ y ] ) signSum = sum < 0? -1 : 1 scaledAbsSum = ( abs ( sum ) + ( 1 << ( shift - 1 ) ) ) » shift scaledSum = scaledAbsSum * signSum rec' [ x ][ y ] = max ( min ( rec[ x ][ y ] + scaledSum, ( 1 << BitDepth ) - 1 ), 0 )

[0630] In the filtering process, the present application uses the same image edge padding method as ALF to obtain the TALF input value outside the image range.

[0631] In some embodiments, the ALF filter is padded with the coding tree block as the unit, and the filter in one coding tree block does not use the reconstructed value of other coding tree blocks, in which case the TALF can also be padded with the coding tree block as the unit.

[0632] Next, taking the MV as an example, the scheme of using vector information to assist TALF filtering proposed in the embodiments of the present application is exemplarily described.

[0633] Exemplarily, one implementation is to use the TALF method of default using offset to obtain the reference position, that is, the parsed slice-level syntax element no longer additionally includes a flag indicating whether it is the MV-assisted TALF, and the slice-level syntax element is parsed as follows:

[0634] Further, when performing TALF filtering on the luma coding tree block, the TALF filtering needs to be performed when the talf_ctb_idc of the current luma coding tree block (current block) is non-zero, that is, talf_ctb_idc indicates that the current block performs TALF filtering. The input of the TALF filtering is:

[0635] 1. A reconstructed luma image array rec subjected to luma adaptive loop filtering

[0636] 2. The mode value of talf_filter_mode

[0637] 3. The luma coordinates (xCtb, yCtb) of the current luma coding tree block in the current image

[0638] 4. The width tAlfWidth and height tAlfHeight of the luma coding tree block

[0639] 5. Coefficients of the time adaptive loop filter tAlfCoeff[]

[0640] It should be noted that for each position (x, y) in the current luma coding tree block, x takes values from xCtb to xCtb+tAlfWidth-1, and y takes values from yCtb to yCtb+tAlfHeight-1.

[0641] Further, when performing TALF filtering on the luma coding tree block, the following steps can be included:

[0642] First, obtain the reconstructed image and MVs:

[0643] Determine the input image of the filter according to talf_filter_mode

[0644] If talf_filter_mode is 0, determine whether the prediction block to which the current position belongs selects MV0 pointing to the reference image in the reference image list 0, and if so, use the reference image rec0 pointed to by MV0 as the input, otherwise skip the filtering of the current position;

[0645] If talf_filter_mode is 1, determine whether the prediction block to which the current position belongs selects MV1 pointing to the reference image in the reference image list 1, and if so, use the reference image rec1 pointed to by MV1 as the input, otherwise skip the filtering of the current position;

[0646] If talf_filter_mode is 2, determine whether the prediction block to which the current position belongs selects both MV0 pointing to the reference image in the reference image list 0 and MV1 pointing to the reference image in the reference image list 1, and if so, use the reference image rec0 pointed to by MV0 and the reference image rec1 pointed to by MV1 as the input, otherwise skip the filtering of the current position.

[0647] Second, obtain the shift value:

[0648] If Mv0 and Mv1 are found in the first step, since sub-pixel level precision Mv is used in inter prediction, here the position offset on the reconstructed image pointed to by Mv is obtained by rounding the positive pixel precision.

[0649] The horizontal integer pixel position offset of MV0 is: Offset0X = Mv0.Hor < 0? - ((abs(Mv0.Hor) + 8) >> 4) : ((abs(Mv0.Hor) + 8) >> 4)

[0650] OffsetOY = MvO.Ver < 0? -((abs(MvO.Ver) + 8) » 4) : ((abs(MvO.Ver) + 8) » 4)

[0651] OffsetIX = MvI.Hor < 0? -((abs(MvI.Hor) + 8) » 4) : ((abs(MvI.Hor) + 8) » 4)

[0652] OffsetIY = MvI.Ver < 0? -((abs(MvI.Ver) + 8) » 4) : ((abs(MvI.Ver) + 8) » 4)

[0653] Third step, according to the position offset obtained, reconstruct the image, filter the coefficient of (x, y) position, when numCoeff is 14: x0 = Clip3 (0, picWidth, x + Offset0X) y0 = Clip3 (0, picHeight, y + Offset0Y) x1 = Clip3 (0, picWidth, x + Offset1X) y1 = Clip3 (0, picHeight, y + Offset1Y) filterIdx = talf_ctb_idx - 1 sum = tAlfCoeff [filterIdx] [0] * (rec0 [x0] [y0] - rec [x] [y]) + tAlfCoeff [filterIdx] [1] * (rec1 [x1] [y1] - rec [x] [y]) + tAlfCoeff [filterIdx] [2] * (rec0 [x0-1] [y0] + rec0 [x0+1] [y0] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [3] * (rec1 [x1-1] [y1] + rec1 [x1+1] [y1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [4] * (rec0 [x0] [y0-1] + rec0 [x0] [y0+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [5] * (rec1 [x1] [y1-1] + rec1 [x1] [y1+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [6] * (rec0 [x0+1] [y0-1] + rec0 [x0-1] [y0+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [7] * (rec1 [x1+1] [y1-1] + rec1 [x1-1] [y1+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [8] * (rec0 [x0-1] [y0-1] + rec0 [x0+1] [y0+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx] [9] * (rec1 [x1-1] [y1-1] + rec1 [x1+1] [y1+1] -2 * rec [x] [y]) + tAlfCoeff [filterIdx]

[0010] * (rec0 [x0-2] [y0] + rec0 [x0+2] [y0] -2 * rec [x] [y]) +tAlfCoeff[ filterldx ] [ 0011 ] * ( rec1[ x1-2 ][ y1 ] + rec1[ x1+2 ][ y1 ] - 2 * rec[ x ][ y ] ) + tAlfCoeff[ filterldx ] [ 0012 ] * ( rec0[ x0 ][ y0-2 ] + rec0[ x0 ][ y0+2 ] - 2 * rec[ x ][ y ] ) + tAlfCoeff[ filterldx ] [ 0013 ] * ( rec1[ x1 ][ y1-2 ] + rec1[ x1 ][ y1+2 ] - 2 * rec[ x ][ y ] ) signSum = sum < 0? -1 : 1 scaledAbsSum = ( abs( sum ) + ( 1 << ( shift - 1 ) ) ) » shift scaledSum = scaledAbsSum * signSum rec'[ x ][ y ] = max( min( rec[ x ][ y ] + scaledSum, ( 1 << BitDepth ) - 1 ), 0 )

[0654] In the above process of calculating the filter, shift is 6, BitDepth variable is the bit depth of the luma component, x and y represent the horizontal and vertical coordinates of the reconstructed image array. x0, y0, x1, y1 are the coordinates of the center positions of the offset rec0 and rec1, respectively. When the reconstructed values in the coordinate positions rec0, rec1 array are obtained, the horizontal coordinate should be limited to 0 to the image width picWidth-1, and the vertical coordinate should be limited to 0 to the image height picHeight-1.

[0655] numCoeff is 13 and talf_filter_mode is 0: x0 = Clip3(0, picWidth, x + Offset0X) y0 = Clip3(0, picHeight, y + Offset0Y) filterldx = talf_ctb_idc - 1 sum = tAlfCoeff[filterldx][0] * (rec0[x0][y0] - rec[x][y]) + tAlfCoeff[filterldx][1] * (rec0[x0-1][y0] + rec0[x0+1][y0] - 2 * rec[x][y]) + tAlfCoeff[filterldx][2] * (rec0[x0][y0-1] + rec0[x0][y0+1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][3] * (rec0[x0+1][y0-1] + rec0[x0-1][y0+1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][4] * (rec0[x0-1][y0-1] + rec0[x0+1][y0+1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][5] * (rec0[x0-2][y0] + rec0[x0+2][y0] - 2 * rec[x][y]) + tAlfCoeff[filterldx][6] * (rec0[x0][y0-2] + rec0[x0][y0+2] - 2 * rec[x][y]) + tAlfCoeff[filterldx][7] * (rec0[x0-2][y0-1] + rec0[x0+2][y0+1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][8] * (rec0[x0-2][y0+1] + rec0[x0+2][y0-1] - 2 * rec[x][y]) + tAlfCoeff[filterldx][9] * (rec0[x0-1][y0-2] + rec0[x0+1][y0+2] - 2 * rec[x][y]) + tAlfCoeff[filterldx]

[0010] * (rec0[x0-1][y0+2] + rec0[x0+1][y0-2] - 2 * rec[x][y]) + tAlfCoeff[filterldx]

[0011] * (rec0[x0-3][y0] + rec0[x0+3][y0] - 2 * rec[x][y]) +tAlfCoeff[ filterldx ] [ 0 ] * ( rec0[ x0 ][ y0 - 3 ] + rec0[ x0 ][ y0 + 3 ] - 2 * rec[ x ][ y ] ) signSum = sum < 0? -1 : 1 scaledAbsSum = ( abs( sum ) + ( 1 << ( shift - 1 ) ) ) » shift scaledSum = scaledAbsSum * signSum rec' [ x ][ y ] = max( min( rec[ x ][ y ] + scaledSum, ( 1 << BitDepth ) - 1 ), 0 )

[0656] numCoeff is 13 and talf_filter_mode is 1: x1 = Clip3(0, picWidth, x + Offset1X) y1 = Clip3(0, picHeight, y + Offset1Y) filterldx = talf_ctb_idc - 1 sum = tAlfCoeff[filterldx][0] * (rec1[x1][y1] - rec[x][y]) + tAlfCoeff[filterldx][1] * (rec1[x1-1][y1] + rec1[x1+1][y1] - 2*rec[x][y]) + tAlfCoeff[filterldx][2] * (rec1[x1][y1-1] + rec1[x1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][3] * (rec1[x1+1][y1-1] + rec1[x1-1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][4] * (rec1[x1-1][y1-1] + rec1[x1+1][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][5] * (rec1[x1-2][y1] + rec1[x1+2][y1] - 2*rec[x][y]) + tAlfCoeff[filterldx][6] * (rec1[x1][y1-2] + rec1[x1][y1+2] - 2*rec[x][y]) + tAlfCoeff[filterldx][7] * (rec1[x1-2][y1-1] + rec1[x1+2][y1+1] - 2*rec[x][y]) + tAlfCoeff[filterldx][8] * (rec1[x1-2][y1+1] + rec1[x1+2][y1-1] - 2*rec[x][y]) + tAlfCoeff[filterldx][9] * (rec1[x1-1][y1-2] + rec1[x1+1][y1+2] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0010] * (rec1[x1-1][y1+2] + rec1[x1+1][y1-2] - 2*rec[x][y]) + tAlfCoeff[filterldx]

[0011] * (rec1[x1-3][y1] + rec1[x1+3][y1] - 2*rec[x][y]) +tAlfCoeff[ filterldx ][ 0 ][ 0 ] * ( rec1[ x1 ][ y1 - 3 ] + rec1[ x1 ][ y1 + 3 ] - 2 * rec[ x ][ y ] ) signSum = sum < 0? -1 : 1 scaledAbsSum = ( abs( sum ) + ( 1 << ( shift - 1 ) ) ) » shift scaledSum = scaledAbsSum * signSum rec' [ x ][ y ] = max( min( rec[ x ][ y ] + scaledSum, ( 1 << BitDepth ) - 1 ), 0 )

[0657] In the filtering process, the present application adopts the same image edge padding method as ALF to obtain the TALF input values outside the image range.

[0658] In some embodiments, the ALF filtering is edge padded in units of coding tree blocks, and the filtering in one coding tree block does not use the reconstructed values of other coding tree blocks. In this case, the TALF can also be edge padded in units of coding tree blocks.

[0659] The present application provides a coding and decoding method, which is a time domain adaptive loop filtering method. The method can perform TALF filtering processing on the reconstructed values of a current block based on the reconstructed samples of at least one inter-frame reference image and corresponding filter coefficients. In the TALF filtering process, the related information of the at least one inter-frame reference image is fully utilized, thereby improving the filtering effect and enhancing the coding and decoding performance. Further, in the process of performing TALF filtering, the vector information including the motion vector or block vector corresponding to the sample position can be used to obtain the corresponding reference position, and the TALF filtering of the sample position is completed based on the reference position, that is, the vector information is introduced to assist the TALF filtering, thereby further improving the filtering effect and enhancing the coding and decoding performance.

[0660] Based on the above embodiments, the present application provides a time domain adaptive loop filtering technology, which is a technology of using the reconstructed image in the time domain as an information source to perform improved adaptive loop filtering on the current frame reconstructed image. Further, in the TALF filtering process, on the one hand, the multiplexed historical decoded time domain adaptive loop filter can be selected, on the other hand, the nonlinear time domain adaptive loop filter can be selected, on the other hand, the adaptive precision of the time domain loop filter can be introduced, and on the other hand, the time domain loop filter assisted by the vector information can be selected. Thus, the coding and decoding performance can be maximally improved.

[0661] Below, the TALF filtering scheme proposed by the embodiments of the present application and the related schemes based on TALF are exemplarily described.

[0662] Exemplarily, a 7x7 symmetrical filter and two 5x5 symmetrical filters can be selected, but the present application is not limited thereto. Among them, a 7x7, 13-coefficient symmetrical filter is used for filtering for TALF using one reconstructed image input; two 5x5, 7-coefficient symmetrical filters are used for filtering for TALF using two reconstructed image inputs.

[0663] Exemplarily, the filter-related parameters can be coded at the slice header, but the present application is not limited thereto.

[0664] Exemplarily, the TALF technology can be selected for use in the luminance component, and in some embodiments, the chrominance can also be improved by using the information of the reconstructed image in the time domain as an auxiliary to improve the reconstruction quality of the current chrominance image, but the present application is not limited thereto.

[0665] In an embodiment, in the TALF filtering process, the historical decoded time domain adaptive loop filter can be selected to be multiplexed, a nonlinear time domain adaptive loop filter is used at the same time, and the adaptive precision of the time domain loop filter is introduced. Among them, a maximum of four time domain adaptive loop filters are allowed to be used in one slice (strip), and the TALF can be located after the ALF for loop filtering.

[0666] The slice header syntax elements are parsed as follows:

[0667] Among them, when at least one of the following three conditions is true, the FIFO of the historical time domain filter needs to be emptied:

[0668] pcSlice->isIDRorBLA(): This condition indicates whether the current slice is an IDR or BLA; both IDR and BLA are NAL types, indicating the beginning of a sequence; all slices in a picture should have the same NAL type.

[0669] pcSlice->getPendingRasInit(): This condition indicates whether the POC is accumulated to the maximum allowed value and reset to 0.

[0670] pcSlice->isInterGDR(): This condition means whether it is GDR.

[0671] The tAlfParamPool.clear() operation indicates that the FIFO of the historical time domain filter is emptied.

[0672] The talf_reuse_flag syntax element is an identifier indicating whether the current slice is coded with the history temporal loop filter reused or not. The value of 1 indicates that the history temporal loop filter is reused, and the value of 0 indicates that the history temporal loop filter is not reused. When the identifier is not present in the bitstream, the value is 0.

[0673] The talf_reuse_index syntax element is an index indicating the history temporal loop filter FIFO reused by the current slice. Since the maximum length of the FIFO used in the present scheme is 8, the index can take the values of 0, 1, 2, 3, 4, 5, 6 or 7, and is represented by a fixed-length code with a length of 3 bins. When the syntax element is not present in the bitstream, the value is 0.

[0674] The talf_shift_minus5 syntax element indicates the variable precision used by one or more temporal loop filters coded in the current slice. The value of the syntax element is the shift value used in the filtering minus 5. The syntax element is represented by a fixed-length code with a length of 2 bins, and can take the values of 0, 1, 2 or 3. When the syntax element is not present in the bitstream, the value is 0.

[0675] The talf_k_order[sfIdx] syntax element indicates the order of the exponential Golomb used by the absolute value of the sfIdx-th filter coefficient coded in the current slice. The value of the syntax element is 0 or 1. The value of 0 indicates that the 0th order exponential Golomb code is used, and the value of 1 indicates that the 1st order exponential Golomb code is used. When the syntax element is not present in the bitstream, the value is 0.

[0676] The operation tAlfParamPool[talf_filter_mode][0].shift = talf_shift_minus5 + 5 indicates that the parsed talf_shift_minus5 is restored to the shift value and is updated at the head of the FIFO of the history temporal loop filter.

[0677] The talf_clip_flag[sfIdx] syntax element indicates whether the sfIdx-th temporal loop filter used in the current slice is clipped or not. The value of the syntax element is 0 or 1. The value of 1 indicates that the sfIdx-th temporal loop filter is clipped, and the value of 0 indicates that the sfIdx-th temporal loop filter is not clipped. When the syntax element is not present in the bitstream, the value is 0.

[0678] talf_clip_idx[ sfldx ][ j ] indicates the nonlinear index value used by the i-th coefficient of the sfldx-th temporal loop filter of the current slice, the value of this syntax element is 0, 1, 2 or 3. The value of 0 means that the input of the filter coefficient at this position does not use nonlinear, 1, 2, 3 means the index of the corresponding nonlinear clipping value, which is used to get the clipping value from the table of nonlinear clipping values. This syntax element is coded using a fixed length code with length of 2 bins. When this syntax element is not present in the bitstream, its value is 0.

[0679] tAlfParamPool[ talf_filter_mode ][ 0 ].clipldx[ sfldx ][ j ] = talf_clip_idx[ sfldx ][ j ] operation means that the parsed talf_clip_idx value is assigned to the FIFO of the corresponding history temporal loop filter.

[0680] Further, when reconstructing the temporal filter coefficient, considering that the multiplexing mechanism is introduced, if sh_talf_enabled_flag of the current slice is 1 and talf_reuse_flag is 0, the coefficient value, the shift value and the nonlinear clipping value of the temporal adaptive loop filter need to be reconstructed.

[0681] The process of obtaining the filter coefficient tAlfCoeff of the current slice is as follows:

[0682] The process of obtaining the nonlinear clipping value tAlfClip of the current slice is as follows:

[0683] - Construct a nonlinear clipping value table according to the pixel depth inputBitdepth of the luma component

[0684] - Obtain the nonlinear clipping value of each coefficient of each filter of the current slice:

[0685] The process of obtaining the shift value of the adaptive loop filter of the current slice is as follows:

[0686] shift = talf_shift_minus5 + 5

[0687] If sh_talf_enabled_flag of the current slice is 1 and talf_reuse_flag is 1, the coefficient value, the shift value and the nonlinear clipping value need to be obtained from the FIFO of the history temporal adaptive loop filter.

[0688] The process of obtaining the filter coefficient tAlfCoeff of the current slice is as follows:

[0689] The process of obtaining the non-linear clipping value tAlfClip for the current slice is as follows:

[0690] - Construct a non-linear clipping value table according to the pixel depth inputBitdepth of the luma component

[0691] - Obtain the non-linear clipping value for each coefficient of each filter of the current slice:

[0692] The process of obtaining the shift value of the adaptive loop filter for the current slice is as follows: shift = tAlfParamPool[talf_filter_mode][talf_reuse_idx].shift

[0693] Further, when performing TALF filtering on a luma coding tree block, the talf_ctb_idc identifier of the current luma coding tree block is non-zero, which means that TALF filtering is required.

[0694] The input of the filtering process includes:

[0695] 1. A reconstructed luma image array rec that has been filtered by the luma adaptive loop filter,

[0696] 2. A corresponding time-domain reconstructed luma image array, for example, when the talf_filter_mode syntax element is 0, the forward nearest one frame time-domain reconstructed image rec0 is used under the RA configuration or the first frame reconstructed image rec1 is used under the LD; when it is 1, the backward nearest one frame time-domain reconstructed image rec1 is used under the RA configuration or the first frame reconstructed image rec0 is used under the LD; when it is 2, two reconstructed image arrays rec0 and rec1 are used,

[0697] 3. The luma coordinates (xCtb, yCtb) of the current luma coding tree block in the current image,

[0698] 4. The width tAlfWidth and height tAlfHeight of the luma coding tree block,

[0699] 5. The coefficient tAlfCoeff of the time-domain adaptive loop filter,

[0700] 6. The non-linear clipping value tAlfClip of the time-domain adaptive loop filter,

[0701] 7. The shift value shift of the time-domain adaptive loop filter.

[0702] The filtering calculation for each position in the luma coding tree block is as follows:

[0703] numCoeff is 14:

[0704] BitDepth variable is the bit depth of the luma component, x and y represent the horizontal and vertical coordinates of the reconstructed image array. K(a, b) = min(b, max(-b, a))

[0705] numCoeff is 13 and talf_filter_mode is 0:

[0706] numCoeff is 13 and talf_filter_mode is 1:

[0707] In the process of filtering, the scheme uses the same image edge padding method as ALF to obtain the TALF input value outside the image range required.

[0708] It should be noted that through experiments, the coding and decoding method proposed in the present application improves the compression performance of the ECM reference software under RA and LD configurations. For example, implementing the above method on the ECM-13.0 reference software can obtain some BD-rates improvement under the RA configuration, and the performance improvement of the test sequences class-B, class-C, class-D and class-F is as shown in Table 2.

[0709] Table 2

[0710] At the same time, some BD-rates improvement can be obtained under the LD configuration, and the performance improvement of the test sequences class-C, class-D and class-E is as shown in Table 3.

[0711] Table 3

[0712] In another embodiment, in the TALF filtering process, a history decoded time domain adaptive loop filter can be selected, a nonlinear time domain adaptive loop filter is used at the same time, the adaptive precision of the time domain loop filter is introduced, and the vector information assisted time domain loop filtering is based. Among them, a maximum of eight time domain adaptive loop filters are allowed to be used in one slice (strip), and TALF can be located between ALF and SAO for loop filtering.

[0713] The slice header syntax elements are parsed as follows:

[0714] The FIFO of the history temporal filter needs to be emptied when at least one of the following three conditions is true:

[0715] pcSlice->isIDRorBLA(): This condition indicates whether the current slice is an IDR or BLA; both IDR and BLA are NAL types, indicating the start of a sequence; all slices in an image should have the same NAL type.

[0716] pcSlice->getPendingRasInit(): This condition indicates whether the POC is accumulated to the maximum allowed value and reset to 0.

[0717] pcSlice->isInterGDR(): This condition indicates whether it is GDR, as the name implies.

[0718] The tAlfParamPool.clear() operation indicates that the FIFO of the history temporal filter is emptied.

[0719] The talf_reuse_flag syntax element is an identifier indicating whether the history temporal loop filter is reused for the current slice coding, and the identifier is 1 to indicate that the history temporal loop filter is reused, and 0 to indicate that the history temporal loop filter is not reused. When the identifier does not exist in the code stream, the value is 0.

[0720] The talf_reuse_index syntax element is an index indicating the history temporal loop filter FIFO reused by the current slice. Since the maximum length of the FIFO used in the present scheme is 8, the index can be 0, 1, 2, 3, 4, 5, 6, or 7, and a fixed-length code with a length of 3 bins is used to represent. When the syntax element does not exist in the code stream, the value is 0.

[0721] talf_mv_guided_flag is an identifier indicating whether the motion vector assisted TALF is used, and the identifier is 1 to indicate that the motion vector assisted TALF is used for the current slice; and the identifier is 0 to indicate that the motion vector assisted TALF is not used for the current slice. When the identifier does not exist in the code stream, the value is 0.

[0722] talf_shift_minus6 indicates the variable precision used by one or more temporal loop filters coded in the current slice, and its meaning is that the shift value used in filtering is reduced by 6. This syntax element uses a fixed-length code with a length of 2 bins in the present scheme, and the value is 0, 1, 2, or 3. When the syntax element does not exist in the code stream, the value is 0.

[0723] talf_k_order[ sfldx ] specifies the order of the exponential Golomb used for the absolute value of the filter coefficients of the current slice. It can take the value 0 or 1, where 0 means that the 0th order exponential Golomb is used and 1 means that the 1st order exponential Golomb is used. When the syntax element is not present in the bitstream, its value is 0.

[0724] tAlfParamPool[ talf mv guided flag ][ 0 ]. shift = talf shift minus6 + 6 specifies that the value of talf shift minus6 is restored to a shift value and is updated at the head of the history-based temporal loop filter (HTLF) FIFO.

[0725] talf clip flag[ sfldx ] specifies whether the non-linearity is used for the sfldx-th HTLF of the current slice. It can take the value 0 or 1, where 1 means that the non-linearity is used for the sfldx-th HTLF and 0 means that the non-linearity is not used for the sfldx-th HTLF. When the syntax element is not present in the bitstream, its value is 0.

[0726] talf clip idx[ sfldx ][ j ] specifies the index of the non-linearity used for the j-th coefficient of the sfldx-th HTLF of the current slice. It can take the value 0, 1, 2 or 3, where 0 means that the non-linearity is not used for the j-th coefficient of the sfldx-th HTLF, and 1, 2 or 3 means that the index of the non-linearity is used for the j-th coefficient of the sfldx-th HTLF. The syntax element is coded using a fixed length code with a bin length of 2. When the syntax element is not present in the bitstream, its value is 0.

[0727] tAlfParamPool[ talf mv guided flag ][ 0 ]. clip idx[ sfldx ][ j ] = talf clip idx[ sfldx ][ j ] specifies that the value of talf clip idx[ sfldx ][ j ] is assigned to the corresponding HTLF FIFO.

[0728] Further, when reconstructing the HTLF coefficients, considering that the reuse mechanism is introduced, if sh talf enabled flag of the current slice is equal to 1 and talf reuse flag is equal to 0, the values of the HTLF coefficients, the shift value and the non-linearity limit value of the HTLF of the current slice need to be reconstructed.

[0729] The process of obtaining the filter coefficients tAlfCoeff of the current slice is as follows:

[0730] The process of obtaining the non-linear clipping value tAlfClip of the current slice is as follows:

[0731] -constructing a non-linear clipping value table according to the pixel depth inputBitdepth of the luminance component

[0732] -Obtaining the non-linear clipping value of each filter of each coefficient of the current slice:

[0733] The process of obtaining the shift value of the current slice of the adaptive loop filter is as follows: shift=talf_shift_minus6+6

[0734] If sh_talf_enabled_flag of the current slice is 1 and talf_reuse_flag is 1, the coefficient value, the shift value and the non-linear limit value need to be obtained from the FIFO of the historical temporal adaptive loop filter.

[0735] The process of obtaining the filter coefficient tAlfCoeff of the current slice is as follows:

[0736] The process of obtaining the non-linear clipping value tAlfClip of the current slice is as follows:

[0737] -constructing a non-linear clipping value table according to the pixel depth inputBitdepth of the luminance component

[0738] -Obtaining the non-linear clipping value of each filter of each coefficient of the current slice:

[0739] The process of obtaining the shift value of the current slice of the adaptive loop filter is as follows: shift=tAlfParamPool[talf_mv_guided_flag][talf_reuse_idx].shift

[0740] Further, when the TALF filter is applied to the luminance coding tree block, the talf_ctb_idc of the current luminance coding tree block is identified as non-zero, and the TALF filter is applied.

[0741] The input of the filtering process includes:

[0742] 1. A reconstructed luminance image array rec filtered by the luminance adaptive loop filter,

[0743] 2. The mode value of talf_filter_mode,

[0744] 3. The identification value of talf_mv_guided_flag,

[0745] 4. the luma coordinates (xCtb, yCtb) of the current luma coding tree block in the current picture,

[0746] 5. the width tAlfWidth and height tAlfHeight of the luma coding tree block,

[0747] 6. the coefficients tAlfCoeff[][] of the temporal adaptive loop filter.

[0748] For each position (x, y) in the current luma coding tree block, x takes values from xCtb to xCtb + tAlfWidth - 1, y takes values from yCtb to yCtb + tAlfHeight - 1

[0749] First, get the reconstructed picture and Mv:

[0750] Determine the input picture of the filter according to talf_mv_guided_flag and talf_filter_mode

[0751] If talf_filter_mode is 0 and talf_mv_guided_flag is 0, use the forward nearest reconstructed picture rec0 as the input;

[0752] If talf_filter_mode is 0 and talf_mv_guided_flag is 1, check whether the prediction block to which the current position belongs selects Mv0 pointing to the reference picture in the reference picture list 0, if yes, use the reference picture rec0 pointed by Mv0 as the input, otherwise skip the filtering of the current position;

[0753] If talf_filter_mode is 1 and talf_mv_guided_flag is 0, use the backward nearest reconstructed picture rec1 as the input in RA and the forward second nearest reconstructed picture rec1 as the input in LD;

[0754] If talf_filter_mode is 1 and talf_mv_guided_flag is 1, check whether the prediction block to which the current position belongs selects Mv1 pointing to the reference picture in the reference picture list 1, if yes, use the reference picture rec1 pointed by Mv1 as the input, otherwise skip the filtering of the current position;

[0755] If talf_filter_mode is equal to 2 and talf_mv_guided_flag is equal to 0, the forward nearest reconstructed picture rec0 and the backward nearest re...

Claims

1. A decoding method applied to a decoder, the method comprising: in a case where it is determined that a current picture is filtered using temporal adaptive loop filtering (TALF), determining filter coefficients; determining at least one reference position corresponding to a current position according to vector information corresponding to the current position; determining a filtered reconstructed value of the current position based on a reconstructed sample of the at least one reference position and the filter coefficients.

2. The method of claim 1, wherein, The vector information corresponding to the current position comprises at least one motion vector corresponding to the current position, and the determining the at least one reference position corresponding to the current position according to the vector information corresponding to the current position comprises: determining at least one inter-frame reference picture corresponding to the current picture according to the at least one motion vector; in a case where the at least one inter-frame reference picture exists in a reference picture list corresponding to the current picture, determining the at least one reference position corresponding to the current position in the at least one inter-frame reference picture according to the at least one motion vector.

3. The method of claim 2, wherein, The determining the at least one inter-frame reference picture corresponding to the current picture according to the at least one motion vector comprises: determining the at least one inter-frame reference picture corresponding to the current picture according to a filter mode corresponding to the current picture and the at least one motion vector.

4. The method of claim 2, wherein, The determining the at least one reference position corresponding to the current position in the at least one inter-frame reference picture according to the at least one motion vector comprises: determining at least one first position offset value in a first direction and at least one second position offset value in a second direction according to the at least one motion vector; determining the at least one reference position in the at least one inter-frame reference picture according to the at least one first position offset value and the at least one second position offset value.

5. The method of any one of claims 2-4, wherein, The method further comprises: in a case where the at least one inter-frame reference picture does not exist in the reference picture list corresponding to the current picture, skipping TALF filtering of the current position.

6. The method of claim 1, wherein, The vector information corresponding to the current position comprises at least one block vector corresponding to the current position, and the determining the at least one reference position corresponding to the current position according to the vector information corresponding to the current position comprises: determining the at least one reference position corresponding to the current position in the current picture according to the at least one block vector.

7. The method of any one of claims 1-4, wherein, The method further comprises: decoding a bitstream to determine auxiliary filtering identification information; in a case where it is determined, based on the auxiliary filtering identification information, that vector information is used to assist TALF filtering, performing the determining procedure of the at least one reference position.

8. The method of claim 7, wherein, The method further comprises: in a case where it is determined, based on the auxiliary filtering identification information, that vector information is not used to assist TALF filtering, determining at least one inter-frame reference picture corresponding to the current picture and the filter coefficients; determining a filtered reconstructed value of the current position based on a reconstructed sample of the at least one inter-frame reference picture and the filter coefficients.

9. The method of claim 1, wherein, The filtering based on the reconstructed sample of the at least one reference position and the filter coefficient determines a filtered reconstructed value of the current position, comprising: determining a reconstructed sample value of the current position and a reconstructed sample value of the reference position; determining a filtered reconstructed value of the current position according to the filter coefficient, the reconstructed sample value of the current position, and the reconstructed sample value of the reference position.

10. The method of claim 1, wherein, The filtering based on the reconstructed sample of the at least one reference position and the filter coefficient determines a filtered reconstructed value of the current position, comprising: determining a reconstructed sample value of the current position, a reconstructed sample value of a first reference position, and a reconstructed sample value of a second reference position; determining a filtered reconstructed value of the current position according to the filter coefficient, the reconstructed sample value of the current position, the reconstructed sample value of the first reference position, and the reconstructed sample value of the second reference position.

11. An encoding method applied to an encoder, the method comprising: determining a filter coefficient when filtering a current image using TALF; determining at least one reference position corresponding to a current position according to vector information corresponding to the current position; filtering based on a reconstructed sample of the at least one reference position and the filter coefficient to determine a filtered reconstructed value of the current position.

12. The method of claim 11, wherein, The vector information corresponding to the current position comprises at least one motion vector corresponding to the current position, and the determining the at least one reference position corresponding to the current position according to the vector information corresponding to the current position comprises: determining at least one inter-frame reference image corresponding to the current image according to the at least one motion vector; in a case where the at least one inter-frame reference image exists in a reference image list corresponding to the current image, determining the at least one reference position corresponding to the current position in the at least one inter-frame reference image according to the at least one motion vector.

13. The method of claim 12, wherein, The determining the at least one inter-frame reference image corresponding to the current image according to the at least one motion vector comprises: determining at least one inter-frame reference image corresponding to the current image according to a filter mode corresponding to the current image and the at least one motion vector.

14. The method of claim 12, wherein, The determining the at least one reference position corresponding to the current position in the at least one inter-frame reference image according to the at least one motion vector comprises: determining at least one first position offset value in a first direction and at least one second position offset value in a second direction according to the at least one motion vector; determining the at least one reference position in the at least one inter-frame reference image according to the at least one first position offset value and the at least one second position offset value.

15. The method of any one of claims 12-14, wherein, The method further comprises: in a case where the at least one inter-frame reference image does not exist in the reference image list corresponding to the current image, skipping the TALF filtering of the current position.

16. The method of claim 11, wherein, The vector information corresponding to the current position comprises at least one block vector corresponding to the current position, and the determining of the at least one reference position corresponding to the current position according to the vector information corresponding to the current position comprises: determining the at least one reference position corresponding to the current position in the current image according to the at least one block vector.

17. The method of any of claims 11-14, wherein, The method further comprises: determining a cost value when the TALF filtering is assisted by the vector information and a cost value when the TALF filtering is not assisted by the vector information; determining the auxiliary filtering identification information according to the cost value when the TALF filtering is assisted by the vector information and the cost value when the TALF filtering is not assisted by the vector information, and writing the auxiliary filtering identification information into a bitstream; wherein the auxiliary filtering identification information is used to determine whether the TALF filtering is assisted by the vector information.

18. The method of claim 13, wherein, The determining of the auxiliary filtering identification information according to the cost value when the TALF filtering is assisted by the vector information and the cost value when the TALF filtering is not assisted by the vector information comprises: in a case where the cost value when the TALF filtering is assisted by the vector information is greater than or equal to the cost value when the TALF filtering is not assisted by the vector information, setting the auxiliary filtering identification information to indicate that the TALF filtering is not assisted by the vector information; in a case where the cost value when the TALF filtering is assisted by the vector information is less than the cost value when the TALF filtering is not assisted by the vector information, setting the auxiliary filtering identification information to indicate that the TALF filtering is assisted by the vector information.

19. A code stream, wherein the code stream is generated by bit-encoding information to be encoded; and wherein, The to-be-encoded information at least comprises the auxiliary filtering identification information.

20. An encoder, comprising a first determining unit; wherein the first determining unit is configured to, when a current image is filtered using TALF, determine filter coefficients, determine at least one reference position corresponding to a current position according to vector information corresponding to the current position, and determine a filtered reconstructed value of the current position based on filtering of reconstructed samples of the at least one reference position and the filter coefficients.

21. An encoder, comprising a first memory and a first processor; wherein the first memory is configured to store a computer program capable of running on the first processor; the first processor is configured to, when the computer program is run, execute the method according to any one of claims 11 to 18.

22. A decoder, comprising a second determining unit; wherein the second determining unit is configured to, when it is determined that a current image is filtered using TALF, determine filter coefficients, determine at least one reference position corresponding to a current position according to vector information corresponding to the current position, and determine a filtered reconstructed value of the current position based on filtering of reconstructed samples of the at least one reference position and the filter coefficients.

23. A decoder, comprising a second memory and a second processor; wherein the second memory is configured to store a computer program capable of running on the second processor; The second processor is configured to implement the method according to any one of claims 1-10 when the computer program is run.

24. A computer readable storage medium storing a computer program, the computer program being executed to implement the decoding method according to any one of claims 1-10, or to implement the encoding method according to any one of claims 11-18.

25. A computer readable storage medium configured to store a bitstream generated by the encoding method according to any one of claims 11-18.

Images