Cascade vector-based cross-component model inheritance method and apparatus

Abstract

This invention discloses a method and apparatus for encoding and decoding chrominance components using Cross-Component Model (CCM) information based on cascaded motion vectors or block vectors. According to the method, when referencing CCM information from one or more neighboring blocks, if at least one neighboring block has a first motion vector (MV) or a first block vector (BV): one or more cascaded vectors are derived, wherein each cascaded vector is recursively derived as the sum of tracking vectors starting from the first MV or the first BV; target CCM information is determined based on the one or more cascaded vectors, and a merge list is derived accordingly; and the current second color block is encoded or decoded using encoding / decoding information containing the target CCM information, wherein the corresponding prediction data generates the current second color block by applying the cross-component model with the target CCM information to the current first color block.

Description

[0001] Related citations This invention is a non-provisional application of U.S. Provisional Patent Application No. 63 / 620,208 (filed January 12, 2024), and claims priority thereto. The aforementioned U.S. Provisional Patent Application is incorporated herein by reference in its entirety. Technical Field

[0002] This invention relates to a video encoding and decoding system using encoding and decoding tools that include one or more cross-component model correlation modes. Specifically, this invention relates to encoding and decoding chroma components using cross-component model information based on cascaded motion vectors or block vectors. Background Technology

[0003] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Video Coding Experts Group (VCEG) of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the Joint Video Experts Team (JVET) of the Moving Picture Experts Group (MPEG) of the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC). This standard was published as an ISO standard in February 2021: ISO / IEC 23090-3:2021, Information technology—Coded representation of immersive media—Part 3: Versatile Video Coding. VVC was developed based on its predecessor, High Efficiency Video Coding (HEVC). It improves encoding and decoding efficiency by adding more encoding and decoding tools, and can handle various types of video sources, including three-dimensional (3D) video signals.

[0004] Figure 1AAn exemplary adaptive inter-frame / intra-frame video coding system incorporating loop processing is illustrated. For intra-frame prediction, prediction data is derived from previously encoded / decoded image video data in the current frame. For inter-frame prediction 112, motion estimation (ME) is performed on the encoder side, and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other images and motion data. Switch 114 selects either intra-frame prediction 110 or inter-frame prediction 112 and provides the selected prediction data to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118, followed by quantization (Q) 120. The residual from the transform and quantization is then encoded by entropy encoder 122 to be included in the video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information (such as motion and encoding / decoding modes associated with intra-frame and inter-frame prediction) and other information such as parameters associated with loop filters applied to the underlying image regions. Figure 1A As shown, auxiliary information related to intra-frame prediction 110, inter-frame prediction 112, and loop filter 130 is provided to entropy encoder 122. When inter-frame prediction mode is used, one or more reference images must also be reconstructed on the encoder side. Therefore, the residuals from transform and quantization are processed by inverse quantization (IQ) 124 and inverse transformation (IT) 126 to recover the residuals. The residuals are then added back to the prediction data 136, reconstructing the video data at reconstruction (REC) 128. The reconstructed video data can be stored in reference image buffer 134 and used for prediction in other frames.

[0005] like Figure 1A As shown, the incoming video data undergoes a series of processes in the encoding system. The reconstructed video data from REC 128 may be subject to various degradations due to these processes. Therefore, before storing the reconstructed video data in the reference image buffer 134, a loop filter 130 is typically applied to the reconstructed video data to improve video quality. For example, a deblocking filter (DF), sample adaptive offset (SAO), and adaptive loop filter (ALF) may be used. It may be necessary to incorporate loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to the entropy encoder 122 for inclusion in the bitstream. Figure 1A In the process, the loop filter 130 is applied to the reconstructed video, and then the reconstructed samples are stored in the reference image buffer 134. Figure 1A The system described herein is intended to illustrate an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0006] like Figure 1B As shown, apart from transform 118 and quantization 120, the decoder can use the same or partially the same function blocks as the encoder, since the decoder only needs inverse quantization 124 and inverse transform 126. The decoder uses entropy decoder 140 instead of entropy encoder 122 to decode the video bitstream into quantized transform coefficients and the required encoding / decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 at the decoder end does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from entropy decoder 140, without motion estimation.

[0007] To improve the encoding and decoding performance of systems using cross-component models, methods and apparatuses using cross-component models associated with concatenated vectors are disclosed. Summary of the Invention

[0008] This invention discloses a method and apparatus for encoding and decoding color images or videos using an encoding / decoding tool that includes one or more cross-component model correlation modes. According to the method, input data associated with a current block is received, the current block including a current first color block and a current second color block, wherein the input data includes pixel data to be encoded at the encoder end or data associated with the current block to be decoded at the decoder end. When referencing Cross-Component Model (CCM) information from one or more neighboring blocks, if at least one neighboring block has a first Motion Vector (CCM) or a first Block Vector (BV), the following steps are performed: one or more concatenated vectors are derived, wherein each concatenated vector is recursively derived as a sum of traced vectors starting from the first MV or the first BV; target CCM information is determined based on the one or more concatenated vectors; a merge list containing the target CCM information is determined; and the current second color block is encoded or decoded using the merge list, wherein, when the target CCM information is selected, the corresponding prediction data for the current second color block is generated by applying the cross-component model with the target CCM information to the current first color block.

[0009] In one embodiment, if the second reference block indicated by the current cascaded vector has a second motion vector or a second block vector, then the second motion vector or the second block vector is used as the next tracking vector, and the next cascaded vector is formed by adding the next tracking vector to the current cascaded vector, wherein the current cascaded vector is initially set to the first motion vector or the first block vector. In one embodiment, each tracking vector corresponds to an L0 motion vector, an L1 motion vector, or a block vector.

[0010] In one embodiment, for each recursion, if a concatenated vector indicates a reference block that has multiple motion vectors or block vectors, then multiple concatenated vectors are derived. In one embodiment, a set of concatenated vectors is derived for different numbers of tracking depths. In another embodiment, a set of concatenated vectors is derived from the sum of all possible tracking vectors, where these tracking vectors correspond to the target tracking depth.

[0011] In one embodiment, the tracking depth associated with the one or more cascaded vectors corresponds to a finite number less than a maximum limit. In another embodiment, the tracking depth associated with the one or more cascaded vectors corresponds to an infinite number. In yet another embodiment, the tracking depth associated with the one or more cascaded vectors corresponds to a predetermined number.

[0012] In one embodiment, each of the one or more adjacent blocks corresponds to CU / CB, PU, ​​TU / TB, or a corresponding block with the same size as the current block.

[0013] In one embodiment, when exporting the corresponding CCM information to be stored in the target block, multiple sets of CCM information are stored in the target block after the encoding / decoding of the target block is completed. In one embodiment, if the target block has a usable target MV or BV, and the target block is encoded and decoded using Cross-Component Prediction (CCP), then in addition to storing the CCM information used by the target block, the CCM information of one or more reference blocks located by one or more concatenated vectors is also stored in the target block. In another embodiment, if the target block has a usable target MV or BV, and the target block is not encoded and decoded using Cross-Component Prediction (CCP), then the CCM information of one or more reference blocks located by one or more concatenated vectors is stored. In yet another embodiment, the maximum set of CCM information allowed to be stored in a block is predefined.

[0014] In one embodiment, if the number of available CCM information sets exceeds the maximum allowed number, the priority of the CCM information to be stored is predefined. In one embodiment, if the target block is encoded and decoded using CCP, the CCM information used by that target block has the highest priority. In one embodiment, the priority is determined based on the tracking depth of the target concatenated vector. In one embodiment, CCM information associated with shorter tracking depths has a higher priority. Attached Figure Description

[0015] Figure 1A An exemplary adaptive inter-frame / intra-frame video encoding / decoding system incorporating loop processing is shown.

[0016] Figure 1B Show Figure 1A The corresponding decoder for the encoder.

[0017] Figure 2 An example is shown for deriving the sample locations of α and β in a cross-component model.

[0018] Figure 3 An example is shown where adjacent samples are divided into two groups for MMLM.

[0019] Figure 4 The spatial portion of the convolution filter is shown.

[0020] Figure 5 The four gradient modes of GLM are shown.

[0021] Figure 6 Five adjacent blocks are shown as candidates for deriving the VVC space merge.

[0022] Figure 7 The following shows possible candidate pairs for considering redundancy checks in VVC.

[0023] Figure 8 An example of temporal candidate derivation is shown, in which a scaled motion vector is derived based on the Picture Order Count (POC) distance.

[0024] Figure 9 This shows the position of the time candidate selected between candidate C0 and C1.

[0025] Figure 10 This shows the spatially adjacent blocks used to derive spatial merging candidates.

[0026] Figure 11 An example of a search region used for intra-frame template matching is shown.

[0027] Figure 12 This shows an example of using an IntraTMP block vector for an IBC block.

[0028] Figure 13 This shows a sample system block diagram of the Cross-Component Residual Model (CCRM).

[0029] Figure 14 The relationship between luminance samples L0, ..., L5 and chrominance sample C is shown.

[0030] Figure 15 An example is shown on how to derive Auto-Relocated Block Vector Prediction (AR-BVP).

[0031] Figure 16 This shows how to derive the block vector BV. n,n+1 And for block B n An example of performing five spatial location checks.

[0032] Figure 17 An example of a Chained CMVP candidate set is shown, which is based on a pre-exported MV and is derived by recursively tracing the sum of MVs and BVs, which are derived from the pre-exported MVs of the inter-block merge candidate set list.

[0033] Figure 18This demonstrates how to derive the MV by checking if there exists an MV or BV in memory that corresponds to all five positions of the current block. k(m) Examples.

[0034] Figure 19A and Figure 19B Two search modes for predetermined locations and a predetermined order for deriving candidate sets of non-adjacent spatial neighbors are shown.

[0035] Figure 20 An example of CCM information propagation based on block vectors is shown, where the dashed blocks (i.e., A, E, G) are encoded and decoded using a cross-component model.

[0036] Figure 21 An example of CCM information propagation based on motion vectors is shown, where the dashed blocks (i.e., A, E, G) are encoded and decoded using a cross-component model.

[0037] Figure 22 An example of CCM information propagation with co-location is shown, where the dashed block (i.e., A) is encoded and decoded using a cross-component model.

[0038] Figure 23 An example of cascaded vector derivation is shown, where the cascaded vector is derived from the sum of the motion vectors and block vectors of adjacent blocks through recursive tracing.

[0039] Figure 24 A flowchart illustrating an exemplary video encoding / decoding system using CCM information associated with concatenated vectors according to an embodiment of the present invention is shown. Detailed Implementation

[0040] The components of the present invention, as generally described and illustrated in the figures, can be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention, as illustrated in the figures, is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the invention. References to “one embodiment,” “an embodiment,” or similar language throughout this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of the invention. Therefore, the phrases “in one embodiment” or “in one embodiment” appearing throughout this specification do not necessarily refer to the same embodiment.

[0041] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the invention can be practiced without using one or more specific details, or using other methods, components, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the invention. Embodiments of the invention will be best understood by referring to the accompanying drawings, wherein identical parts are identified by the same numbers throughout. The following description is illustrative only and only illustrates embodiments of specific selected apparatus and methods consistent with the invention declared herein.

[0042] I. Related Technologies I.1 Cross-Component Linear Model Prediction To reduce cross-component redundancy, VVC uses a cross-component linear model (CCLM) prediction mode, where chrominance samples are predicted using a linear model based on reconstructed luminance samples from the same CU: (1) Among them, pred C (i,j) represents the predicted chromaticity sample in CU, rec L (i,j) represents the downsampled reconstructed luminance samples from the same CU. α is called the scaling parameter, and β is called the offset parameter.

[0043] In this disclosure, the terms {CCLM_LT, CCLM_L, CCLM_T} and {CCLM_LA, CCLM_L, CCLM_A} and {LM_LA, LM_L, LM_A} are used interchangeably.

[0044] The CCLM parameters (α and β) are derived from at most four adjacent chroma samples and their corresponding downsampled luminance samples. Assuming the current chroma block size is W×H, then W'' and H' are set as follows: When using LM_LA mode, W' = W, H' = H; When using LM_A mode, W' = W + H; When using LM_L mode, H' = H + W.

[0045] Let the adjacent positions above be S[0,-1],…,S[W'-1,-1], and the adjacent positions to the left be S[-1,0],…,S[-1,H'-1], then the four samples selected are as follows: When the LM_LA mode is applied and the samples above and to the left are available, S[W' / 4, -1], S[3 * W' / 4, -1], S[-1, H' / 4], S[-1, 3 * H' / 4]; When applying LM_A mode or when only the upper adjacent sample is available, S[W' / 8, -1], S[3 * W' / 8, -1], S[5 * W' / 8, -1], S[7 * W' / 8, -1]; and When applying LM_L mode or when only the left adjacent sample is available, S[-1, H' / 8 ], S[ -1, 3 * H' / 8 ], S[ -1, 5 * H' / 8 ], S[ -1, 7 * H' / 8 ].

[0046] The four adjacent brightness samples at the selected location are downsampled and compared four times to find the two larger values: x 0 A and x 1 A, and two smaller values: x 0 B and x 1 B. Their corresponding chromaticity sample values ​​are represented as y. 0 A, y 1 A, y 0 B and y 1 B. Then, x A x B y A and y B The following formula can be derived: X a =(x 0 A + x 1 A +1)>>1; X b =(x 0 B + x 1 B +1)>>1; Y a =(y 0 A + y 1 A +1)>>1; Y b =(y 0 B + y 1 B +1)>>1 (2) Finally, the linear model parameters α and β are obtained according to the following equations.

[0047] (3) (4) Figure 2 This shows an example of the positions of the top-left sample and the sample in the current block involved in the LM_LA mode.

[0048] In addition to the upper and left templates being used together to calculate linear model coefficients, they can also be used alternately in two other LM modes (LM_A and LM_L modes).

[0049] In LM_A mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0050] In LM_LA mode, the left and upper templates are used to calculate the coefficients of the linear model.

[0051] I.2 Multiple Model CCLM (MMLM) In JEM (J. Chen, E. Alshina, GJ Sullivan, J.-R. Ohm, and J. Boyce, “Algorithmic Description of Joint Exploration Test Model 7”, JVET-G1001, ITU-T / ISO / IEC Joint Video Exploration Team (JVET), July 2017), the Multi-Model CCLM (MMLM) mode was proposed to predict chroma samples from luminance samples across the entire CU using two models. In MMLM, neighboring luminance and chroma samples of the current block are divided into two groups, each serving as a training set to derive a linear model (i.e., deriving specific α and β for a specific group). Furthermore, samples from the current luminance block are also classified based on the same classification rules as neighboring luminance samples.

[0052] In this disclosure, the terms MMLM_LT and MMLM_LA are used interchangeably.

[0053] Figure 3 This illustrates an example of dividing adjacent samples into two groups. The threshold is calculated as the average of adjacent reconstructed brightness samples. Rec′ L Neighboring samples with [x,y]<=threshold are classified into group 1; while Rec′ L The neighboring samples of [x,y]>threshold are classified into group 2.

[0054] (5) I.3 Convolutional Cross-Component Model (CCCM) In CCCM, convolutional models are used to improve chromaticity prediction performance. This convolutional model has a 7-tap filter, which consists of a 5-tap positive-shaped spatial component, a nonlinear term, and a bias term. The input to the 5-tap spatial component of the filter consists of a center (C) luminance sample, which is co-located with the chromaticity sample to be predicted and its adjacent samples above / north (N), below / south (S), left / west (W), and right / east (E), as shown below. Figure 4 As shown.

[0055] The non-linear term (denoted as P) is represented as a power of 2 of the center brightness sample C, and the range of sample values ​​scaled to the content: P = (C*C + midVal)>>bitDepth.

[0056] In other words, for 10 bits of content, the calculation is as follows: P = (C*C + 512)>>10.

[0057] The bias term (denoted as B) represents the scalar offset between the input and output (similar to the offset term in CCLM), and is set to an intermediate chroma value (512 for 10-bit content).

[0058] The filter output is calculated as the convolution of the filter coefficients ci with the input values, truncated to the effective chromaticity sample range: predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B.

[0059] I.4 Gradient Linear Model (GLM) For the YUV4:2:0 color format, the gradient linear model (GLM) method can be used to predict chromaticity samples based on the gradient of luminance samples. This method supports two modes: two-parameter GLM mode and three-parameter GLM mode.

[0060] Compared to CCLM, two-parameter GLM does not utilize downsampled luminance values; instead, it uses luminance sample gradients to derive a linear model. Specifically, when two-parameter GLM is applied, the input processed by CCLM, i.e., the downsampled luminance sample L, is replaced by the luminance sample gradient G. Other parts of CCLM (e.g., parameter derivation, linear transformation of predicted samples) remain unchanged. .

[0061] In the three-parameter generalized linear model (GLM), chromaticity samples can be predicted based on the gradient of luminance samples and downsampled luminance values ​​with different parameters. The model parameters of the three-parameter generalized linear model (GLM) are derived from 6 rows and 6 columns of adjacent samples using the MSE minimization method based on LDL decomposition used in CCCM.

[0062] .

[0063] For signaling, when the current CU has CCLM mode enabled, a flag bit is sent to indicate whether GLM is enabled for both Cb and Cr components; if GLM is enabled, another flag bit is sent to indicate which of the two GLM modes is selected, and a syntax element is sent to select one of the four gradient filters for gradient calculation.

[0064] like Figure 5 As shown, GLM enables four gradient filters (510-540).

[0065] I.5 Space Candidate Derivation The derivation of spatial merge candidates in VVC is the same as that in HEVC, except that the positions of the first two merge candidates are swapped. Figure 6 Among the candidates at the indicated positions, a maximum of four merge candidates (B0, A0, B1, and A1) can be selected for the current CU 610. The derivation order is B0, A0, B1, A1, and B2. Position B2 is considered only if one or more adjacent CUs at positions B0, A0, B1, and A1 are unavailable (e.g., belonging to another segment or tile) or if intra-frame encoding / decoding is being performed. After adding a candidate for position A1, the addition of the remaining candidates requires redundancy checking to ensure that candidates with the same motion information are not included in the list, thereby improving encoding / decoding efficiency. To reduce computational complexity, the above redundancy checking does not consider all possible candidate pairs. Instead, only... Figure 7 The pairs connected by arrows are added to the list only if the corresponding candidates used for redundancy check do not have the same motion information.

[0066] I.6 Time Candidate Derivation In this step, only one candidate is added to the list. Specifically, when deriving the temporal merging candidate for the current CU 810, the scaling motion vector is derived based on the co-located CU 820 belonging to the co-located reference image, such as... Figure 8 As shown. The list of reference images and reference indices used to derive the co-located CUs are explicitly indicated in the fragment header. The scaled motion vectors of the temporal merging candidates are shown in Figure 830. Figure 8 As shown by the dashed line, motion vector 830 is scaled from motion vector 840 of the co-located CU using Picture Order Count (POC) distances tb and td, where tb is defined as the POC difference between the reference image and the current image, and td is defined as the POC difference between the reference image and the co-located image. The reference image index of the temporal merging candidate is set to zero.

[0067] The position of the time candidate is selected between candidate C0 and C1, such as... Figure 9 As shown. If the CU at position C0 is unavailable, is intra-frame encoded or decoded, or is located outside the current CTU line, then position C1 is used. Otherwise, position C0 is used when deriving the time merge candidate.

[0068] I.7 Non-adjacent spatial candidates In JVET-L0399, non-adjacent space merge candidate blocks are inserted after the TMVP in the regular merge candidate block list. The pattern of space merge candidate blocks is as follows: Figure 10 As shown. The distance between a non-adjacent spatial candidate block and the current coding block is based on the width and height of the current coding block. Line buffer limits are not applied.

[0069] I.8 Intra-frame Template Matching Prediction Intra-Template Matching Prediction (IntraTMP) is a special intra-prediction mode that copies the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. Within a predetermined search range, the encoder searches for the template most similar to the current template in the reconstructed portion of the current frame and uses the corresponding block as the prediction block. The encoder then sends this mode for use, and the same prediction operation is performed at the decoder.

[0070] The prediction signal is obtained by comparing the L-shaped, top-only, or left-only causal neighborhood of the current block with... Figure 11 Another block in the predefined search area is matched to generate the result, and the search area includes: R1: All or any subset of the current CTU R2: Top left corner CTU R3: Above CTU R4: Left CTU exist Figure 11In the image, the current block 1110 in R1 matches the corresponding block 1112 in R2. The templates for the current block and the matching block are displayed as darker L-shaped regions. Region 1122 corresponds to the reconstructed region in the current image 1120. The sum of absolute differences (SAD) is used as the cost function. Within each region, the decoder searches for the template with the minimum SAD relative to the current block and uses its corresponding block as the prediction block.

[0071] The dimensions of all regions (SearchRange_w, SearchRange_h) are set to be proportional to the block size (BlkW, BlkH) to ensure a fixed number of SAD comparisons per pixel. That is: SearchRange_w = a * BlkW, SearchRange_h = a * BlkH, Here, "a" is a constant used to control the trade-off between gain and complexity. In fact, "a" equals 5.

[0072] To expedite template matching, the search range for all search regions is subsampled by a factor of 3. After finding the best match, a refinement process is performed. This refinement is accomplished by conducting a second template matching search around the best match, thus narrowing the search range.

[0073] For CUs with a width and height less than or equal to 64, enable the intra-frame template matching tool. The maximum CU size for intra-frame template matching is configurable.

[0074] When the current CU does not use Decoder-side Intra Mode Derivation (DIMD), the intra-template matching prediction mode is sent at the CU level via a dedicated flag.

[0075] I.9 IntraTMP Exported Block Vector Candidates for IBC In this method, the block vector (BV) derived from IntraTMP (Intra-Temporal Matching Prediction) is used for intra-block copy (IBC). The IntraTMP BV and IBC BV stored in adjacent blocks are used as spatial BV candidates in the construction of the IBC candidate list.

[0076] IntraTMP block vectors are stored in the IBC block vector buffer. The current IBC block can use the IBC BV and the IntraTMP BV of adjacent blocks as BV candidates for the IBC BV candidate list, such as... Figure 12 As shown.

[0077] exist Figure 12 In the diagram, block 1210 corresponds to the current block, and block 1212 corresponds to the adjacent IntraTMP block. IntraTMP BV1216 is used to locate the best matching block 1222 based on the matching cost between template 1214 and template 1224. Region 1232 corresponds to the reconstructed region in the current image 1230. The IntraTMP block vector is added as a spatial candidate to the IBC block vector candidate list.

[0078] I.10 Cross-component residual model (CCRM) Similar to JVET-AD0108, when blocks use inter-frame prediction or intra-block copy (IBC), a cross-component residual model (CCRM) is applied to predict chroma samples from the reconstructed luminance samples. Figure 13 The decoder side of the method is shown. The cross-component filter is derived using the predicted signals for both luminance and chrominance. The derived filter is applied to the reconstructed luminance signal to produce the final chrominance prediction. In step 1320, the predicted signals (i.e., predY 1310 and predCb 1312 or predCr 1314) are used to derive the filter coefficients for each chrominance component, respectively, and the filter is applied to the reconstructed luminance signal in step 1330, as shown... Figure 13 As shown. The reconstructed luminance signal is formed by combining the luminance prediction (PredY) 1310 and the residual luminance signal (resY) using adder 1322. After applying a filter, step 1330 generates the filtered predicted Cb 1340 and the filtered predicted Cr 1350. The reconstructed Cb signal is formed by combining the filtered predicted Cb 1340 and the residual Cb signal (i.e., resCb) using adder 1342. Similarly, the reconstructed Cr signal is formed by combining the filtered predicted Cr 1350 and the residual Cr signal (i.e., resCr) using adder 1352.

[0079] The proposed 8-tap filter consists of six spatial luminance samples, one nonlinear term, and one bias term. The spatial luminance samples (L0, ..., L5) are obtained by selecting the six luminance samples closest to the chromaticity position C from the luminance grid, without the need for downsampling. Figure 14 As shown. The predicted chromaticity values ​​are as follows: predChromaVal = c0L0+ c1L1 + c2L2 + c3L3 + c4L4 + c5L5 + c6nonlinear((L0+L3+1)>>1) + c7B, Where nonlinearity refers to the nonlinear operand of CCCM, and B is the bias.

[0080] The filter coefficients are derived using the division-free Gaussian elimination method of ECM, and the necessary offsets are applied to the samples before the filter derivation.

[0081] When the block has fewer than 64 chroma samples, intra-frame reference samples are used as additional input samples in filter derivation. A maximum of 6 rows and 6 columns of intra-frame reference samples are used in the CCCM design.

[0082] Blocks with 256 or more chroma samples will be divided into sub-blocks containing a maximum of 256 chroma samples each. Sub-blocks with zero luminance residuals will be skipped.

[0083] I.11 JVET-AG0091: EE2-1.8: Automatic Reset Block Vector Prediction In EE2-1.8, Auto-Relocated Block Vector Prediction (AR-BVP) was introduced into the construction of the IBC merge / AMVP candidate list.

[0084] like Figure 15 As shown, the bootstrap block vector BV associated with the current block B0 0,1 Point to reference block B1. If B1 has a BV that points to reference block B2, denoted as BV. 1,2 Then BV 0,2 (By BV) 0,2 = BV 0,1 + BV 1,2 (Given) is defined as being by BV 0,1 Guided AR-BVP. Similarly, BV 0,n+1 It can be derived in the following way: BV 0,n+1 =BV 0,n +BV n,n+1 = BV 0,1 +BV 1,2 +…+BV n-1,n +BV n,n+1 .

[0085] This EE underwent three tests. In EE2-1.8a, the length of the AR-BVP tracing path was 1 (i.e., n=1). In EE2-1.8b, the length of the AR-BVP tracing path was 2 (i.e., n=2). In EE2-1.8c, there was no limit to the length of the AR-BVP tracing path.

[0086] In BV 0,n BV is derived under the guidance of n,n+1 At that time, it is necessary to check B. n All five positions, including the top left corner (e.g. Figure 16LT in the middle), top right corner (e.g.) Figure 16 RT in the middle), center (e.g. Figure 16 (Ctr in the middle), bottom left corner (e.g.) Figure 16 (LB in the middle) and the bottom right corner (e.g.) Figure 16 (RB in the middle) to find BV n,n+1 .

[0087] In our implementation, the initial guiding block vector (BV) 0,1 It was set as an existing BVP in the IBC merge / AMVP candidate list.

[0088] AR-BVP candidates are inserted after HBVP candidates. The size of the IBC merge / AMVP candidate list remains unchanged.

[0089] I.12 JVET-AG0073: Non-EE2: Chain Motion Vector Prediction This contribution introduces chained motion vector prediction (CMVP) into the construction of the inter-frame merging candidate list.

[0090] like Figure 17 As shown, CMVP candidates can be derived from a pre-exported motion vector (MV) used for the inter-frame merging candidate list, by summing the MV and BV of the recursively traced MV. For example, a CMVP candidate consists of a set of motion vectors MV. k / m and reference image RefPic k / m It can be exported in the following ways: MV k / m =MV k(0) + BV k(0) +MV k(1) +MVk (2) + … + MV k(m) , RefPic k / m =RefPic k(m) , Where k and m represent the number of merged indexes and tracking depths of the CMVP.

[0091] Exporting MV k / m At that time, MV k(m) By checking if there are any locations in the MV / BV storage that match all five positions of the current block (e.g.) Figure 18 As shown, the MV or BV corresponding to the center, top left, top right, bottom left, and bottom right of the current block can be used to find it.

[0092] When the pre-exported merge candidate of the CMVP candidate has two MVs, one MV will be exported for each list (i.e., L0 and L1) and for each tracking depth. k / m Each list and each tracking depth can derive a maximum of two MVs, and the MV sets are inserted sequentially into the inter-frame merge candidate list.

[0093] Traceable reference images are only present in the reference image list. For regular merges and TM merges, CMVP candidates are inserted after HMVP candidates. When exporting CMVP candidates, hpelIfIdx, bcwIdx, licFlag, and mhpFlag are not inherited. CMVP candidates are not exported when TMVP is disabled.

[0094] II. The proposed method The following methods have been proposed to improve cross-component prediction accuracy or encoding / decoding performance.

[0095] II.1 Inheriting adjacent model parameters to refine cross-component model parameters When a cross-component prediction codec tool is applied to the current block to generate a prediction signal, cross-component model (CCM) information, including model parameters (see the "Inheriting CCM Information" section for more details), can be inherited from neighboring blocks.

[0096] In one embodiment, if the inherited neighboring blocks are encoded and decoded in CCLM mode, the final scaling parameter of the current block will be inherited from the neighboring blocks. Once the final scaling parameter is determined, the offset parameter (e.g., β in CCLM) is derived based on the inherited scaling parameter and the average of the luminance and chrominance samples of the current block's neighbors.

[0097] In one embodiment, if the inherited neighboring blocks are encoded and decoded using CCLM mode, the offset parameters can be further refined by dB after inheriting the model parameters. For example, if the final offset parameters are inherited from selected neighboring blocks, and the inherited offset parameters are... The final offset parameter is ( + dB). In another embodiment, the final offset parameter is inherited from the history list or adjacent blocks, but is not further refined using dB.

[0098] In another embodiment, if the inherited adjacent blocks are encoded and decoded using CCCM, then the filter coefficients (c i The offset parameter (e.g., in CCCM) will be inherited. or This can be re-derived based on inherited parameters and the average values ​​of luminance and chrominance samples at adjacent positions of the current block. In another embodiment, the offset parameters (e.g., in CCCM) or No need to re-derive.

[0099] In another embodiment, if an inherited candidate block applies a GLM gradient mode to its luminance reconstruction sample, then the current block should also inherit the GLM gradient mode of the candidate block and apply it to the current luminance reconstruction sample.

[0100] In another embodiment, if the inherited neighboring blocks are encoded and decoded using multiple cross-component models (e.g., MMLM, or CCCM with multiple models), the classification threshold is also inherited to divide the neighboring samples of the current block into multiple groups, and the inherited multiple cross-component model parameters are further assigned to each group.

[0101] II.2 Inheriting CCM Information In one embodiment, inherited cross-component model (CCM) information can be stored along with inherited model parameters. The CCM information can be inherited along with the inherited model parameters. Predictions for the current block can be generated based on the inherited CCM information and the inherited model parameters.

[0102] As mentioned above in this disclosure, CCM information includes, but is not limited to, prediction patterns (e.g., CCLM, MMLM, CCCM, CCCM with multiple models, 2-parameter GLM, 3-parameter GLM (GLM with a luminance term), CCRM), information indicating whether nonlinear terms are used in the model, model index indicating which model shape is used in the convolutional model, classification threshold for multiple models, information indicating whether non-downsampled samples are used in the convolutional model, downsampled filter flags, downsampled filter indexes when using multiple downsampled filters, information indicating whether multiple downsampled filters are used, the number of adjacent lines used to derive the model, template type used to derive the model (e.g., top left, top, left), multiple model flags, post-filter flags, or model parameters.

[0103] In another embodiment, a hybrid CCCM model consisting of various terms (e.g., spatial, gradient, positional, nonlinear, and bias terms) can be inherited. The gradient term can be computed in either a downsampled or non-downsampled domain. The positional term can be computed relative to the top-left corner coordinates of the current block or image. In addition to storing model parameters, prediction patterns can be stored in the CCM information to indicate that the inherited model is a hybrid CCCM model consisting of various terms. If multiple types of hybrid CCCM models exist, model indexes can also be stored in the CCM information to indicate which type of hybrid CCCM model is being inherited. For example, the gradient and location-based CCCM (GL-CCCM) proposed in JVET-AB0119 is a hybrid CCCM model consisting of a spatial term located at the center, two gradient terms for the horizontal and vertical directions respectively, two positional terms X and Y for the relative horizontal and vertical positions respectively, a nonlinear term, and a bias term. In addition to storing model parameters, prediction patterns can also be stored in the CCM information to indicate that the inherited model is a GL-CCCM model.

[0104] II.3 Inheritance Space Adjacency Model Parameters In another embodiment, the inherited model parameters can come from the block that is immediately adjacent to the model. Models from blocks at predetermined positions are added to the candidate list in a predetermined order.

[0105] In one embodiment, the predetermined position and predetermined order can be the same as the spatial candidate positions and predetermined order of the inter-frame merging mode.

[0106] In one embodiment, the predetermined location may be Figure 6 The positions shown in the "Spatial Candidate Derivation" section can be in the following predetermined order: B0, A0, B1, A1 and B2, or A0, B0, B1, A1 and B2.

[0107] In one embodiment, assuming the current block's position, width, and height are (x, y), W, and H, respectively, the predetermined position may include the position directly above the current block. For example, if W is greater than or equal to a threshold TH, the predetermined position is (x + W>>1, y-1) or (x + (W+1)>>1, y-1). The predetermined position may also include the position directly to the left of the current block. For example, if H is greater than or equal to a threshold TH, the predetermined position is (x-1, y+H>>1) or (x-1, y+(H+1)>>1). TH can be 2, 4, 8, 16, 32, or 64. If W is greater than or equal to TH, the predetermined position includes the position immediately above (W>>1) or ((W>>1) – 1); if H is greater than or equal to TH, the predetermined position includes the position immediately to the left (H>>1) or ((H>>1) – 1).

[0108] II.4 Inheritance Time Adjacency Model Parameters In one embodiment, if the current segment / image is a non-intra-frame segment / image, the inherited model parameters can be derived from blocks in previously encoded / decoded segments / images. In one embodiment, the current block position is (x, y), and the block size is w×h. Two value sets αx and αy are defined as: , .

[0109] α x and α y All values ​​of x are positive. Let (x) mid ,y mid ) = (x + w / 2, y + h / 2). Inherited model parameters can come from previously encoded / decoded segments / images located at... The block of location.

[0110] In one sub-implementation, .For example, .

[0111] In one sub-implementation, .For example, as well as In another embodiment, the current block position is (x, y), and the block size is w×h. The inherited model parameters can be derived from previously encoded / decoded segments / images located at... The block of location.

[0112] In one sub-implementation, .For example, .

[0113] In another sub-implementation, .For example, , .

[0114] In one embodiment, the location is close to The model with the closest position (x, y) will be added to the final merge candidate list first. In another embodiment, the model with the closest position (x, y) will be added to the final merge candidate list first.

[0115] In one embodiment, let and These are two fixed positive numbers. The inherited model parameters can come from the positions of previously encoded / decoded segments / images. .

[0116] In another embodiment, the current block position is (x, y), and the block size is... .make and These are two fixed positive numbers. The inherited model parameters can be derived from positions in previous encoded / decoded segments / images. .

[0117] In another embodiment, the current block position is (x, y), and the block size is w×h. The inherited model parameters can come from blocks at specific predetermined positions (x', y') in previous encoded / decoded segments / images. For example, these positions are located within the corresponding regions of the current encoded block, i.e. and Inherited model parameters can come from... For example, these locations are outside the corresponding region of the current encoded block, i.e. ,or Inherited model parameters can come from... The block at that location.

[0118] In one embodiment, the inherited model parameters can come from specific predetermined positions within the block. These predetermined positions and inclusion order can be the same as in the inter-frame merge mode.

[0119] The previously encoded / decoded image from which the inherited parameter model originates is referred to as the co-image below.

[0120] In one embodiment, the previously encoded or decoded image from which the inherited parameter model originates, i.e., the isotopic image, is one of the images in the reference list.

[0121] In one embodiment, the co-position image can be the same as the co-position image in the inter-frame merge mode.

[0122] In one embodiment, the co-position image is sent in the image / fragment header. A reference list and reference index are sent in the image / fragment header. For example, the co-position image is selected as L0[0]. As another example, the co-position image is selected as L1[0].

[0123] In one embodiment, the corresponding image is selected as the image with the smallest POC difference between its reference image and the current image in the reference list. For example, if the current image has a POC of 8, the images in reference list 0 have a POC of {7, 6, 5, 0}, and the images in reference list 1 have a POC of {7, 6, 5, 4}, then L0[0] (equivalent to L1[0]) is selected because it has the smallest POC difference.

[0124] In another sub-implementation, if there are two images whose reference POC differences with the current image are both minimal, the image with the smaller POC is selected.

[0125] In another sub-implementation, if there are two images whose POC difference between the reference image and the current image is the smallest, the image with the larger POC is selected.

[0126] In one embodiment, the co-position image is selected as the image in the reference list whose QP difference is the smallest between the reference image and the current image. For example, if the QP of the current image is 28, the QP of the images in reference list 0 is {19, 26, 23}, and the QP of the images in reference list 1 is {23, 22, 21}, then L0[1] is selected.

[0127] In one embodiment, the corresponding image is selected as the image with the smallest QP in the reference list. In another embodiment, the corresponding image is selected as the image with the largest QP in the reference list.

[0128] In one embodiment, the rules for selecting / not selecting co-located images described in the paragraph above can be combined.

[0129] II.5 Inheriting the Non-Closely Adjacent Spatial Adjacency Model In one embodiment, the inherited model parameters can come from non-immediately adjacent spatial blocks. Models from blocks at predetermined locations are added to the candidate list in a predetermined order.

[0130] In one sub-implementation, the predetermined positions and predetermined order are the same as the predetermined positions and predetermined order of non-immediately adjacent spatial blocks in the inter-frame merging mode.

[0131] In one sub-implementation, the predetermined position and predetermined order are as follows: Figure 19A and Figure 19BAs shown. The positions of the numbered squares are predetermined positions. The number within each square indicates the predetermined order. Positions in Pattern 1 (1910) are added to the list before positions in Pattern 2 (1920). The distance between each predetermined position is proportional to the width and height of the current block.

[0132] In one embodiment, the number of inheritance models from non-immediately adjacent spatial blocks that can be added to the candidate list has a maximum number, and this maximum number is less than the number of predetermined positions.

[0133] II.6 Inheriting Model Parameters from History Tables In one embodiment, the inherited model parameters can come from a cross-component model history table. This history table stores CCM information for valid previously encoded blocks. A valid previously encoded block refers to any block containing valid CCM information. Cross-component models in the history table can be added to a candidate list in a predetermined order. In one embodiment, the order in which historical candidates are added can be from the beginning to the end of the table. In another embodiment, the order in which historical candidates are added can be from a specific predetermined position to the end of the table. In yet another embodiment, the order in which historical candidates are added can be from the end to the beginning of the table. In yet another embodiment, the order in which historical candidates are added can be staggered (e.g., the first candidate added starts from the beginning of the table, the second candidate added starts from the end of the table, and so on).

[0134] In one embodiment, a single cross-component model history table can be maintained to store previous cross-component models, and the cross-component model history table can be reset at the current image, current slice, current tile, every M CTU rows, or at the beginning of every N CTU, where N and M can be any value greater than 0. In another embodiment, the cross-component model history table can be reset at the current image, current slice, current tile, current CTU row, or at the end of the current CTU.

[0135] II.7 Inheritance from Fusion Mode A fusion mode refers to any mode that merges two predictions to generate a final prediction. In chroma intra-frame fusion mode, chroma intra-frame predictions not generated using cross-component prediction (CCP) codecs (such as CCLM, MMLM, CCCM) are merged with another chroma intra-frame prediction generated using a cross-component prediction codec. For example, intra-frame predictions from non-CCLM codecs are merged with intra-frame predictions from CCLM codecs to obtain the final intra-frame prediction.

[0136] In one embodiment, when cross-component model parameters are inherited from the block / position of the chroma intra-fusion mode codec, the model parameters used to obtain intra-prediction of the CCP codec are also inherited and further refined.

[0137] In one embodiment, in addition to inheriting and refining the CCP model parameters, the fusion weights and encoding / decoding modes of non-CCP codec intra-frame prediction are also inherited. That is, the chroma intra-frame fusion mode is inherited.

[0138] II.8 Constructing the Candidate List In one embodiment, the candidate list is constructed by adding candidates in a predetermined order until the maximum number of candidates is reached. The added candidates may include all or some of the candidates described above, but are not limited to these. For example, the candidate list may include spatially adjacent candidates, temporally adjacent candidates, historical candidates, non-immediately adjacent candidates, and single-model candidates generated based on other inherited models. As another example, the candidate list may contain the same candidates as in the previous example, but in a different order.

[0139] In another embodiment, if all predetermined adjacent candidates and historical candidates have been added, but the maximum number of candidates has not been reached, some default candidates are added to the candidate list until the maximum number of candidates is reached.

[0140] In one sub-implementation, the default candidate can be a CCLM model. The final scaling parameter α comes from the set {0, 1 / 8, -1 / 8, +2 / 8, -2 / 8, +3 / 8, -3 / 8, +4 / 8, -4 / 8}, and the offset parameter... Alternatively, it can be derived based on adjacent luminance and chrominance samples. For example, if the average values ​​of adjacent luminance and chrominance samples are lumaAvg and chromaAvg, respectively, then β is derived from β = chromaAvg - α·lumaAvg. The average value of adjacent luminance samples (lumaAvg) can be calculated using all selected luminance samples. Similarly, the average value of adjacent chrominance samples (chromaAvg) can be calculated using all selected chrominance samples.

[0141] In another embodiment, the default candidate can be an earlier candidate refined by an incremental scaling parameter. The earlier candidate is a CCLM model. For example, if the scaling parameter of the earlier candidate is α, then the scaling parameter of the default candidate is (α+Δα), where Δα can be 1 / 8, -1 / 8, +2 / 8, -2 / 8, +3 / 8, -3 / 8, +4 / 8, or -4 / 8. The offset parameter of the default candidate is derived by using (α+Δα) and the average of the luminance and chrominance samples adjacent to the current block.

[0142] II.9 Send inheritance candidate indexes in the list An on / off flag is sent to indicate whether the current block inherits cross-component model parameters from neighboring blocks. This flag can be sent by CU / CB, PU, ​​TU / TB, color component, or chroma color component. High-level syntax can be sent in SPS, PPS, PH, or SH to indicate whether the proposed method is permitted for the current sequence, image, or fragment.

[0143] The maximum allowed candidate number is sent to indicate the maximum size of the candidate list to be merged. This number can be sent by CU / CB, PU, ​​TU / TB, color component, or chroma color component. A high-level syntax can be sent in SPS, PPS, PH, or SH to indicate whether the proposed method is allowed for the current sequence, image, or fragment. The maximum allowed candidate number for the proposed method can be shared with the maximum allowed candidate number for the merging mode.

[0144] If the current block inherits cross-component model parameters from neighboring blocks, the inherited candidate index is sent. This index can be sent (e.g., using truncated unary codes, exponential Golomb codes, or fixed-length codes) and shared between the current Cb and Cr blocks. For example, the index can be sent for the color components. For instance, one inherited index is sent for the Cb components, notifying another inherited index that is sent for the Cr components. For another example, the inherited index can be stored using chroma intra-frame prediction syntax (e.g., IntraPredModeC[xCb][yCb]).

[0145] II.10 Cross-component model of vector propagation In one embodiment, after encoding / decoding a block, the Cross-Component Model (CCM) information of the current block is exported and stored in the current block. The stored CCM information is available for reference by subsequent encoding / decoding blocks. Subsequent encoding / decoding blocks can inherit the CCM information from the current block. The definition of CCM information is in the "Inheriting CCM Information" section. The stored CCM information can be inherited as (but is not limited to) the following types of candidates: spatial candidates (as described in the "Inheriting Spatial Adjacency Model Parameters" section), non-adjacent candidates (as described in the "Inheriting Non-Adjacent Spatial Adjacency Model" section), temporal candidates (as described in the "Inheriting Temporal Adjacency Model Parameters" section), and historical candidates (as described in the "Inheriting Model Parameters from History Table" section).

[0146] In one embodiment, if the current block is encoded / decoded using Cross-Component Prediction (CCP), the cross-component model used by the current block can be stored and referenced by subsequent encoded / decoded blocks. When a block is encoded / decoded using CCP, this means that the block uses a cross-component model to generate predictions for the block based on the cross-component model. The block can use a cross-component model inherited from neighboring blocks, a cross-component model derived based on neighboring luma and chroma prediction / reconstruction sample values ​​(e.g., CCLM, MMLM, CCCM, CCRM), a cross-component model used in chroma fusion (meaning that chroma prediction is based on adding one or more cross-component prediction hypotheses to one or more existing non-cross-component prediction hypotheses), or any combination of the above models.

[0147] In one embodiment, if the current block has not been CCP encoded / decoded, and a usable block vector exists in the current block (e.g., the current luma block is encoded / decoded using IBC or IntraTMP mode, and the co-located luma block is encoded / decoded using IBC or IntraTMP mode), then the CCM information of the current block can be derived by copying the CCM information of a reference block located by the block vector. For example, as... Figure 20 As shown, block B has not undergone CCP encoding / decoding, and a usable block vector exists at block B. Reference block A is located by the block vector. The CCM information of reference block A, using the cross-component model, is copied and stored in block B. In one embodiment, if the reference block located by the block vector has also not undergone CCP encoding / decoding, but stores CCM information in the reference block, the CCM information of the current block can be derived by copying the CCM information stored in the reference block. That is, even if the reference block has not undergone CCP encoding / decoding, as long as the reference block stores valid CCM information, the current block can refer to that stored CCM information. For example, as... Figure 20 As shown, current block C has an available block vector, and its reference block B (not CCP encoded / decoded) stores CCM information. The CCM information of block B is copied and stored in block C. The CCM information stored in block B is copied from block A, therefore the CCM information of block A is propagated to block C. Block C can obtain the CCM information originally from block A simply by accessing block B. In one embodiment, if the reference block located by the block vector is not CCP encoded / decoded and does not store CCM information, then no CCM information is stored for the current block.

[0148] In one embodiment, when the current block has multiple available block vectors (e.g., block vectors can be bidirectional, the block can have multiple IntraTMP block vectors, or the current chroma block is co-located with multiple luma blocks and the multiple luma blocks have block vectors), in order to derive the CCM information of the current block, if only one of the reference blocks located by the block vector has CCM information, then the CCM information of the reference block with CCM information is copied to the current block and stored therein. For example, as... Figure 20 As shown, assume block F has two block vectors, corresponding to two reference blocks G and H respectively. Block G has CCM information, while block H does not. The CCM information of block G is copied to block F and stored therein.

[0149] In another embodiment, when the current block has multiple block vectors and multiple reference blocks located by these block vectors have CCM information, one of the reference blocks is selected based on a set of predetermined rules. The CCM information of the selected reference block is then copied and stored in the current block.

[0150] For a sub-implementation, a reference block encoded and decoded by CCP is selected.

[0151] For a sub-implementation, the reference block with the smallest distance to the current block is selected. The CCM information of the selected reference block is copied and stored in the current block. The reference block and the current block are located at (x... r , y r ) and (x c , y c Its distance can be reached through Calculate. (x) r , y r ) and (x c , y c The reference block and the current block can be located at the top left, top right, bottom left, bottom right, or center position.

[0152] For a sub-implementation, the minimum horizontal distance |x r - x c The reference block is selected. The CCM information of the selected reference block is copied and stored in the current block.

[0153] For another sub-implementation, the minimum vertical distance |y r - y c The reference block is selected. The CCM information of the selected reference block is copied and stored in the current block.

[0154] For a sub-implementation, the previously described rules can be combined, and not all of the previously described rules need to be applied. For example, a reference block encoded by CCP is selected. If multiple reference blocks encoded by CCP exist, the block with the shortest distance to the current block among the CCP-encoded reference blocks is selected. If multiple CCP-encoded reference blocks with the smallest distance to the current block exist, the horizontal distance |x r -x cThe smallest reference block is selected. For example, a reference block encoded by CCP is selected. If multiple CCP-encoded reference blocks exist, the block with the shortest distance to the current block is selected. If multiple CCP-encoded reference blocks have the smallest distance to the current block, the block with the vertical distance |y| is selected. r -y c The smallest reference block is selected. The CCM information of the selected reference block will be copied and stored in the current block.

[0155] In one embodiment, if the current block is not encoded or decoded by CCP and a usable motion vector exists in the current block (e.g., the current luma block is inter-frame encoded or decoded), the CCM information of the current block can be derived by copying the CCM information of a reference block in a reference image, which is located by the motion vector of the current block. For example, as Figure 21 As shown, block B is not encoded or decoded by CCP, and a usable motion vector exists at block B. Reference block A is located by the motion vector. The CCM information of reference block A, using the cross-component model, is copied and stored in block B. In one embodiment, if the reference block located by the motion vector is also not encoded or decoded by CCP, but CCM information is stored in the reference block, the CCM information of the current block can be derived by copying the CCM information stored in the reference block. That is, even if the reference block is not encoded or decoded by CCP, as long as the reference block stores valid CCM information, the current block can refer to that stored CCM information. For example, as... Figure 21 As shown, current block C has available motion vectors, and its reference block B (not encoded or decoded by CCP) stores CCM information. The CCM information of block B is copied and stored in block C. The CCM information stored in block B is copied from block A, therefore the CCM information of block A is propagated to block C. Block C only needs to access block B to obtain the CCM information originally from block A. In one embodiment, if the reference block located by motion vectors is not encoded or decoded by CCP and does not store CCM information, then the current block does not store any CCM information.

[0156] In one embodiment, when the current block uses bidirectional prediction for inter-frame encoding and decoding, in order to derive the CCM information of the current block, if only one of the reference blocks located by motion vectors has CCM information, then the CCM information of the reference block with CCM information is copied and stored in the current block. For example, as... Figure 21 As shown, assume that block F uses bidirectional prediction for inter-frame encoding and decoding. The two reference blocks located by motion vectors are block G and block H. Block G stores CCM information, while block H does not. The CCM information of block G is copied and stored in block F.

[0157] In another embodiment, when the current block uses bidirectional prediction for inter-frame encoding and decoding, and both reference blocks located by motion vectors store CCM information, the CCM information of the current block is derived by combining all or part of the CCM models of its reference blocks.

[0158] In another embodiment, when the current block uses bidirectional prediction for inter-frame encoding and decoding, and both reference blocks located by motion vectors have stored CCM information, one of the reference blocks is selected based on a set of predetermined rules. The CCM information of the selected reference block is then copied and stored in the current block.

[0159] For one sub-implementation, a reference block using CCP encoding / decoding is selected.

[0160] For a sub-implementation, a reference block whose reference image (i.e., the image containing the reference block) has a smaller POC distance than the current image is selected. The CCM information of the selected reference block is then copied and stored in the current block. For example... Figure 21 As shown, block F uses bidirectional prediction for inter-frame encoding and decoding. The two reference blocks located by motion vectors are block G and block H. Assuming that both blocks G and H have stored CCM information, and |N1-N2| is less than |N3-N2|, then block G is selected, and the CCM information of block G is copied and stored in block F.

[0161] In one sub-implementation, a reference block with a smaller QP difference between the reference image and the current image is selected. The CCM information of the selected reference block is copied and stored in the current block.

[0162] In one sub-implementation, a reference block with a smaller QP value is selected. In another sub-implementation, a reference block with a larger QP value is selected.

[0163] For a sub-implementation, the aforementioned rules can be combined, and it is not necessary to apply all of the aforementioned rules.

[0164] In one embodiment, if the current block is encoded / decoded inter-frame, or if available block vectors exist in the current block, the CCM information of the current block can be derived by copying the CCM information of a reference block located by motion vectors or block vectors. For example, as Figure 22 As shown, current block C has available block vectors, and its reference block B has available motion vectors. The CCM information of block B is copied from block A. Then, the CCM information of block B is copied to block C. Therefore, the CCM information of block A is propagated to current block C.

[0165] II.11 Cascaded Vector Cross-Component Model In one embodiment, when referencing cross-component model (CCM) information from neighboring blocks, the concatenated vector can be derived based on the motion vectors or block vectors of the neighboring blocks if the neighboring blocks have available motion vectors or block vectors. The CCM information of the block indicated by the concatenated vector can also be inherited. The definition of CCM information is in the section entitled "Inheriting CCM Information". Neighboring blocks can be, but are not limited to, the following types of candidates: spatial candidates (as described in the section entitled "Inheriting Spatial Adjacency Model Parameters"), non-closely adjacent candidates (as described in the section entitled "Inheriting Non-Closely Adjacent Spatial Adjacency Model"), temporal candidates (as described in the section entitled "Inheriting Temporal Adjacency Model Parameters"), and historical candidates (as described in the section entitled "Inheriting Model Parameters from History Tables").

[0166] like Figure 23 As shown, the concatenated vector can be derived from the motion vectors or block vectors of adjacent blocks by recursively tracing the sum of the motion vectors and block vectors. For example, the concatenated vector can be derived as follows: Let the L0 motion vector of the adjacent block be MVL0(0), the block vector of the block indicated by MVL0(0) be BV(0), the L0 motion vector of the block indicated by BV(0) be MVL0(1), and so on. The concatenated vector MV_m is: MV_m = MVL0(0) + BV(0) + MVL0(1) + … + MVL0(m). The reference image for MVL0_m is: RefPicL0_m = RefPicL0(m).

[0167] Here, m is the tracking depth, which is the number of reference images being referenced.

[0168] A cascaded vector can be the sum of the L0 motion vector, the L1 motion vector, and the block vector.

[0169] For each recursion, if the base block has multiple motion vectors or multiple block vectors, multiple concatenated vectors can be derived. For example, suppose the block indicated by MVL0(1) is bidirectional prediction and has two motion vectors MVL0(2) and MVL1(2). The two concatenated vectors can be derived as MV_1 + MVL0(2) and MV_1 + MVL1(2), where MV_1 = MVL0(0) + BV(0) + MVL0(1).

[0170] The tracking depth m can be a finite value, for example, m = 1. That is, the maximum value of m is finite. The number of backreferences to the reference image is limited. The tracking depth m can also be infinite. That is, all tracking depth values ​​are allowed, and there is no limit to the number of backreferences to the reference image. The tracking depth m can be predefined. That is, there can be a predetermined threshold for the maximum number of tracking depths.

[0171] A set of concatenated vectors can be derived for different m. That is, for all allowed tracking depths, one or more concatenated vectors can be derived.

[0172] Adjacent blocks can be CU / CB, PU, ​​TU / TB, or corresponding blocks of the same size as the current block.

[0173] Multiple vector propagation cross-component model As described in the "Vector Propagation Cross-Component Model" section, after a block is encoded / decoded, the cross-component model (CCM) information for the current block is exported and stored in the current block. In one embodiment, multiple CCM information entries can be stored in the current block.

[0174] In one embodiment, if the current block has available motion vectors or block vectors, and the current block uses cross-component prediction (CCP) encoding / decoding, then in addition to storing the cross-component model used by the current block, the CCM information of the reference block located by the concatenated vectors (as described in the "Concatenated Vector Cross-component Model" section) can also be stored in the current block for future reference. It should be noted that when the tracking depth is 0, the concatenated vectors degenerate into the motion vectors or block vectors of the current block. Therefore, the CCM information of the reference block located by the motion vectors or block vectors of the current block can also be stored in the current block for future reference. If the current block is not CCP encoded / decoded, the CCM information of the reference block located by the concatenated vectors (as described in the "Concatenated Vector Cross-component Model" section) can also be stored in the current block for future reference.

[0175] The maximum amount of CCM information allowed to be stored in a block can be predefined. If the amount of available CCM information exceeds the maximum allowed, the priority of the CCM information to be stored can be predefined. For example, if the current block is encoded and decoded by CCP, then the current CCM information has the highest priority. Alternatively, priority can be determined based on the tracking depth of the concatenated vectors. The shorter the tracking depth, the higher the priority of the CCM information. These rules can be combined.

[0176] The cascaded vector derivation method and its associated CCM information proposed above can be implemented in the encoder and / or decoder. For example, the proposed methods can be implemented in the inter-frame / intra-frame / prediction module of the encoder and / or the inter-frame / intra-frame / prediction module of the decoder. Alternatively, any of the proposed methods can be implemented as circuitry coupled to the inter-frame / intra-frame / prediction module of the encoder and / or the inter-frame / intra-frame / prediction module of the decoder to provide the information required by the inter-frame / intra-frame / prediction module.

[0177] The concatenated vector derivation method and its associated CCM information described above can be implemented on the encoder side or the decoder side. For example, any of the proposed methods can be implemented in the intra-frame encoding / decoding module of the decoder (e.g., Figure 1B Intra-prediction 150 in the encoder or the intra-encoder module of the encoder (e.g., Figure 1A Intra-frame prediction (110) is implemented. Any of the proposed propagating cross-component predictions can also be implemented as circuitry coupled to an intra / inter-frame codec module of the decoder or encoder. However, the decoder or encoder can also implement propagating cross-component prediction processing using additional processing units. While the intra-frame prediction unit (e.g., Figure 1A Unit 110 and Figure 1B Unit 150 in the diagram is shown as a separate processing unit, which may correspond to executable software or firmware code stored on a medium (such as a hard disk or flash memory) for a central processing unit (CPU) or a programmable device (e.g., a digital signal processor (DSP) or a field programmable gate array (FPGA)).

[0178] Figure 24A flowchart illustrating an exemplary video encoding / decoding system using CCM information associated with concatenated vectors according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as program code executable on one or more processors (e.g., one or more CPUs) at the encoder or decoder end. The steps shown in the flowchart can also be implemented based on hardware, for example, one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 2410, input data associated with the current block is received, the current block including a current first color block and a current second color block, wherein the input data includes pixel data to be encoded at the encoder end or data associated with the current block to be decoded at the decoder end. In step 2420, when referencing cross-component model (CCM) information from one or more neighboring blocks, it is checked whether at least one neighboring block has a first motion vector (MV) or a first block vector (BV). If at least one neighboring block has a first MV or a first BV (i.e., the "yes" path), steps 2430-2460 are executed. Otherwise (i.e., the "no" path), steps 2430-2460 are skipped. In step 2430, one or more concatenated vectors are derived, each of which is the sum of tracking vectors starting from the first MV or the first BV, derived recursively. In step 2440, target CCM information is determined based on the one or more concatenated vectors. In step 2450, a merge list including the target CCM information is determined. In step 2460, the current second color patch is encoded or decoded using the merge list, wherein, when target CCM information is selected, the corresponding prediction data for the current second color patch is generated by applying a cross-component model with target CCM information to the current first color patch.

[0179] The flowchart shown is intended to illustrate examples of video encoding and decoding according to the present invention. Those skilled in the art can modify each step, rearrange the steps, split the steps, or combine the steps to practice the invention without departing from its spirit. In this disclosure, specific syntax and semantics are used to illustrate examples of embodiments of the invention. Those skilled in the art can practice the invention by substituting equivalent syntax and semantics without departing from its spirit.

[0180] The above description is intended to enable those skilled in the art to practice the invention in the context of specific applications and their requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the invention is not intended to be limited to the specific embodiments shown and described, but rather to be given the broadest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details have been set forth to provide a thorough understanding of the invention. However, the invention can be practiced by those skilled in the art.

[0181] As described above, embodiments of the present invention can be implemented through various hardware, software code, or a combination of both. For example, embodiments of the present invention may be program code integrated into one or more circuits in a video compression chip, or integrated into video compression software to perform the processes described herein. Embodiments of the present invention may also be program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The present invention may also relate to multiple functions performed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the invention, defining the specific methods embodied in the invention by executing machine-readable software code or firmware code. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, different code formats, styles, and languages ​​of the software code, as well as other means of configuring the code to perform tasks according to the invention, do not depart from the spirit and scope of the invention.

[0182] This invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The examples described are to be regarded as illustrative in all respects, not limiting. Therefore, the scope of the invention should be indicated by the appended claims rather than the foregoing description. All variations within the meaning and equivalence of the claims should be included within its scope.

Claims

1. A method for encoding and decoding a color image using multiple encoding and decoding tools, said multiple encoding and decoding tools including multiple modes associated with one or more cross-component models, the method comprising: Receive input data associated with the current block, which includes a current first color block and a current second color block, wherein the input data includes pixel data to be encoded at the encoder end or data associated with the current block to be decoded at the decoder end; When referencing cross-component model information from one or more neighboring blocks, if at least one neighboring block has a first motion vector or a first block vector, then: Derive one or more cascaded vectors, wherein each cascaded vector is recursively derived from the sum of multiple tracking vectors starting from the first motion vector or the first block vector; The target cross-component model information is determined based on one or more concatenated vectors; The decision includes a merged list of cross-component model information for the target; and The merge list is used to encode or decode the current second color block, wherein, when the target cross-component model information is selected, the corresponding prediction data for the current second color block is generated by applying the cross-component model with the target cross-component model information to the current first color block.

2. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 1, characterized in that, If the second reference block indicated by the current cascade vector has a second motion vector or a second block vector, then the second motion vector or the second block vector is used as the next tracking vector, and the next tracking vector is added to the current cascade vector to form the next cascade vector, wherein the current cascade vector is initially set to the first motion vector or the first block vector.

3. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 1, characterized in that, Each tracking vector corresponds to an L0 motion vector, an L1 motion vector, or a block vector.

4. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 1, characterized in that, For each recursion, if a cascade vector indicates a reference block that has multiple motion vectors or multiple block vectors, then multiple cascade vectors are derived.

5. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 4, characterized in that, A set of concatenated vectors is derived for different numbers of tracking depths.

6. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 4, characterized in that, A set of concatenated vectors is derived from the sum of all possible tracking vectors, wherein the multiple tracking vectors correspond to the target tracking depth.

7. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 1, characterized in that, The tracking depth associated with one or more cascaded vectors corresponds to a finite number less than the maximum.

8. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 1, characterized in that, The tracking depth associated with one or more cascaded vectors corresponds to an infinite number.

9. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 1, characterized in that, The tracking depth associated with the one or more cascaded vectors corresponds to a predetermined number.

10. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 1, characterized in that, Each of the one or more adjacent blocks corresponds to CU / CB, PU, ​​TU / TB, or a corresponding block with the same size as the current block.

11. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 1, characterized in that, When acquiring the corresponding cross-component model information that needs to be stored in the target block, after the encoding / decoding of the target block is completed, multiple sets of cross-component model information are stored in the target block.

12. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 11, characterized in that, If the target block has available target motion vectors or block vectors, and the target block is encoded and decoded via cross-component prediction, then in addition to storing the cross-component model information used by the target block, the cross-component model information of one or more reference blocks located by one or more concatenated vectors is also stored.

13. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 11, characterized in that, If the target block has available target motion vectors or block vectors, and the target block is not encoded or decoded by cross-component prediction, then the cross-component model information of one or more reference blocks located by one or more concatenated vectors is stored.

14. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 11, characterized in that, The maximum number of groups of cross-component model information that can be stored in a block is predefined.

15. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 11, characterized in that, If the number of available groups for the cross-component model information exceeds the maximum allowed number, the priority of the cross-component model information to be stored is predefined.

16. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 15, characterized in that, If the target block is encoded and decoded via cross-component prediction, then the cross-component model information used by the target block has the highest priority.

17. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 15, characterized in that, This priority is determined by the tracking depth of the target concatenated vector.

18. The method for encoding and decoding a color image using multiple encoding and decoding tools as described in claim 17, characterized in that, Information from this cross-component model, which is associated with shorter tracking depths, has higher priority.

19. An apparatus for encoding and decoding a color image or video using a plurality of encoding and decoding tools, said plurality of encoding and decoding tools including a plurality of modes associated with one or more cross-component models, the apparatus including one or more electronic circuits or processors for: Receive input data associated with the current block, which includes the current first color block and the current second color block, wherein, The input data includes pixel data to be encoded at the encoder end or data associated with the current block to be decoded at the decoder end; When referencing cross-component model information from one or more neighboring blocks, if at least one neighboring block has a first motion vector or a first block vector, then: Derive one or more cascaded vectors, wherein each cascaded vector is recursively derived from the sum of multiple tracking vectors starting from the first motion vector or the first block vector; The target cross-component model information is determined based on one or more concatenated vectors; The decision includes a merged list of cross-component model information for the target; and The merge list is used to encode or decode the current second color block, wherein, when the target cross-component model information is selected, the corresponding prediction data for the current second color block is generated by applying the cross-component model with the target cross-component model information to the current first color block.

Images