Dynamic reference conversion for different coding properties in video coding

Abstract

A method and apparatus for video encoding / decoding using reference regions with different attributes from a current block. According to one method, a first attribute associated with the current block is determined. One or more neighborhoods are determined for the current block, wherein each of the one or more neighborhoods is associated with a second attribute, and one or more target second attributes associated with one or more target neighborhoods differ from the first attribute. Multiple reference samples in the one or more target neighborhoods are converted into multiple transformed reference samples to match the first attribute. Predicted data is derived for the current block based on the multiple transformed reference samples. The predicted data is used to encode or decode the current block. In another method, reference samples in the target neighborhoods are converted into transformed reference samples to match the first attribute.

Description

[0001] Cross-referencing

[0002] This invention is a non-provisional application of U.S. Provisional Patent Application No. 63 / 617,803, filed on January 5, 2024, and claims priority thereto. The entire contents of the U.S. Provisional Patent Application are incorporated herein by reference. Technical Field

[0003] This invention relates to video encoding and decoding systems. Specifically, it discloses the use of a reference image with encoding and decoding attributes different from the current block in a video encoding and decoding system. Background Technology

[0004] Versatile Video Coding (VVC) is the latest international video coding standard developed by the ITU-T Video Coding Experts Group (VCEG) and the Joint Video Experts Team (JVET) of the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC Moving Picture Experts Group (MPEG)). This standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology—Coding representation of immersive media—Part 3: Versatile Video Coding, published in February 2021. VVC was developed based on its predecessor, High Efficiency Video Coding (HEVC), by adding more coding tools to improve coding efficiency and enabling it to handle various types of video sources, including three-dimensional (3D) video signals. Bross et al. have published a review of the Multifunctional Video Coding (VVC) standard and its applications in IEEE Transactions on Circuits and Systems for Video Technology, Volume 31, Issue 10, October 2021, pp. 3736-3764.

[0005] VVC employs adaptive intra / inter-frame prediction and quantizes and transform-codes the prediction residuals, similar to previous video coding standards. However, VVC also incorporates several new coding tools. Figure 1An exemplary block diagram of a VVC encoder is shown. VVC also employs block-based coding, where each input image is divided into Coding Tree Units (CTUs), typically 128x128 or 64x64 in size. Input video is usually in 4:2:0 YCbCr format, where Cb and Cr have a resolution of 1 / 4 that of the luminance component (half in both the horizontal and vertical directions). A 4:2:0 YCbCr video sequence can be converted from 4:4:4 RGB video through a predefined conversion process. Since the human eye has low sensitivity to chroma information, the chroma channels are spatially downsampled, retaining only 1 / 4 of the luminance sample for chroma. After decoding, if the display device requires 4:4:4 RGB format to display frames, the decoded 4:2:0 YCbCr video can be converted to the format required by the display device for viewing.

[0006] like Figure 1 As shown, the input video signal 102 is subtracted from the prediction signal 111 by the subtractor 112. The output of the subtractor 112 corresponds to the residual signal. For VVC, the luma mapping 110 is applied to the input video signal 102 before prediction. In addition to intra-frame prediction 142 and inter-frame prediction 146, VVC introduces a new type of prediction called Combined Inter / Intra Prediction (CIIP, 144). Inter-frame prediction 146 is based on the reconstructed image stored in the decoded image buffer 170 and motion information from motion estimation 148. Since the input video signal is luma mapped by 110, a corresponding process (i.e., luma mapping 114) must be applied after the output inter-frame prediction 146 to select and use it as the prediction signal 111. The selection mechanism 116 is used to adaptively select between intra-frame prediction 142, inter-frame prediction 146, and combined inter-frame / intra-frame prediction (CIIP, 144).

[0007] Similar to HEVC, the residual signal output from subtractor 112 undergoes a transformation, scaling, and quantization process 120 to generate quantization transform coefficients (labeled "B"). The encoder also needs to generate a reconstructed image for subsequent image encoding. To generate the reconstructed image, the quantization transform coefficients (labeled "B") undergo scaling and inverse transform processes 130 to generate the reconstruction residual. VVC also introduces chroma scaling 132, applied to the reconstruction residual. After chroma scaling, the reconstruction residual is added to the prediction signal 111 to form the reconstructed signal. For intra-frame prediction, the reconstructed signal can be used directly without loop filtering. For inter-frame prediction, the reconstruction residual undergoes inverse luma mapping, deblocking, Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF) 160. Since luma mapping has already been applied to the input video signal, inverse luma mapping must be applied in the reconstruction path. VVC adaptively selects the filtering parameters. To determine the filtering parameters, the filter control analysis 150 determines the filtering parameters based on the input video signal 102 and the reconstruction residual.

[0008] Within the encoder, multiple processing modules need to determine modes, parameters, or other control information. For example, prediction mode selection 116 determines whether to use intra-frame, inter-frame, or CIIP modes based on mode selection information from general encoder control 190. Furthermore, general encoder control 190 also provides control information to intra-frame image estimation 140, inter-frame image estimation 146, and scaling and inverse transform 130. The quantization transform coefficients (labeled "B") generated by the transform, scaling, and quantization process 120 are encoded and incorporated into the encoded bitstream via header formatting and context-adaptive binary arithmetic coding (CABAC) 180. Additionally, various related encoder information, such as general encoder control information (labeled "A"), intra-frame prediction mode information (labeled "C", from intra-frame image estimation 140), inter-frame prediction mode information (labeled "E", from inter-frame image estimation 146), and filter control information (labeled "D", from filter control analysis 150), are also incorporated into the encoded bitstream via header formatting and CABAC 180.

[0009] Since the encoder also needs to generate a reconstructed signal for prediction, it contains some key processing units that are also used by the decoder. Figure 1 The processing modules shown in gray are the same modules used by the decoder. The encoded bitstream is received at the decoder. Modules that perform header formatting and CABAC 180 inverse functions are used to recover the quantization transform coefficients (marked "B") and various additional or control information (i.e., Figure 1(As shown in "A", "C", "D", and "E"). The decoder operates on the quantized transform coefficients to generate the reconstructed video. At the decoder end, intra-frame predicted mode information and inter-frame predicted motion information can be generated using the required additional information (i.e., "C" and "F"). However, for decoder-side intra-mode derivation (DIMD), the decoder can derive the mode information without encoder signaling.

[0010] Recently, P. Wennersten et al. disclosed Motion Compensated Temporal Filtering (MCTF) (“AHG10: Encoder-only GOP-based temporal filter”, ITU SG 16 WP 3 and IEC JTC 1 / SC 29 / WG 11 Joint Video Experts Group (JVET), 15th meeting: Gothenburg, Sweden, July 3-12, 2019, document: JVET-O0549). This filter is performed as a preprocessing step at the encoder end, where motion estimation and motion compensation methods are applied to neighboring images on an 8x8 luma block. References Figure 1 The encoder system block diagram shows that the MCTF process is located before the input video signal 102.

[0011] The image is divided into a series of codec tree units (CTUs). The concept of a CTU is the same as that of HEVC. For an image with three sample arrays, a CTU consists of an N×N luma sample block and two corresponding chroma sample blocks. The maximum allowable size of the luma block in a CTU is specified as 128×128 (although the maximum size of the luma transform block is 64×64).

[0012] In HEVC, the CTU is divided into codec units (CUs) using a quadtree (QT) structure (called a codec tree) to accommodate various local characteristics. The decision of whether to use inter-frame (temporal) or intra-frame (spatial) prediction to encode image regions is made at the leaf CU level. Each leaf CU can be further divided into one, two, or four prediction units (PUs) based on the PU partitioning type. Within a PU, the same prediction process is used, and relevant information is transmitted to the decoder in PU units. After obtaining residual blocks by applying the prediction process according to the PU partitioning type, the leaf CU can be divided into transform units (TUs) according to another quadtree structure similar to the CU codec tree. A key feature of the HEVC structure is its multiple partitioning concepts, including CUs, PUs, and TUs.

[0013] In VVC, a nested quadtree structure with multiple tree types is used, employing binary and ternary partitioning structures to replace the concept of multiple partitioning unit types. This removes the separation of CU, PU, ​​and TU unless the CU size exceeds the maximum transform length requirement, and supports more flexible CU partitioning shapes. In the codec tree structure, CUs can have square or rectangular shapes. Codec tree units (CTUs) are first partitioned using a quadtree (also known as a four-way tree) structure. Then, the leaf nodes of the quadtree can be further partitioned using multiple tree types. For example... Figure 2 As shown, the multi-type tree structure has four partitioning types: vertical binary partitioning (SPLIT_BT_VER 210), horizontal binary partitioning (SPLIT_BT_HOR 220), vertical ternary partitioning (SPLIT_TT_VER 230), and horizontal ternary partitioning (SPLIT_TT_HOR 240). The leaf nodes of the multi-type tree are called codec units (CUs). Unless the CU size is too large to exceed the maximum transform length, this partitioning is used for prediction and transform processing without further partitioning. This means that in most cases, CUs, PUs, and TUs have the same block size in a nested multi-type tree quadtree codec block structure. Exceptions occur when the maximum supported transform length is less than the color component width or height of the CU.

[0014] Intra-frame prediction

[0015] The prediction of a CU (Cumulative Unit) is based on the pixels contained in a PU (Prediction Unit). Predictions can be formed from neighboring pixels or pixels in a reference frame. These pixels are available at both the encoder and decoder ends, so the encoding flow is efficient for the encoder and decoder, generating the same predictions between them. Prediction involving only neighboring pixels is intra-frame prediction. Various intra-frame prediction methods are used in VVC, including DC, planar, and angular predictions. A common step is to obtain pixels from neighboring pixels. These neighboring pixels may come from multiple CUs. For example, in... Figure 3 The image shows the L-shaped neighborhood used for angle prediction. Pixels from multiple neighboring CUs are used in the prediction process.

[0016] Inter-frame prediction

[0017] The basic process of inter-frame prediction is to form a prediction by referring to the pixels in the reference frame. Figure 4 This is an example of bidirectional inter-frame prediction. Two motion vectors, MVL0 and MVL1, are used to acquire pixels.

[0018] The region acquired in the reference frame may also involve multiple CUs. For example... Figure 5 As shown, the acquisition area is represented by the dark dashed line area, which involves nine CUs.

[0019] Combined Inter and Intra Prediction (CIIP)

[0020] This is a prediction method that combines inter-frame prediction and intra-frame prediction. The final prediction is obtained by blending pixels from both predictions. As described in previous sections, the prediction before blending can be formed by multiple CUs.

[0021] Intra-template matching prediction (TMP) and intra-block copy (IBC)

[0022] Figure 6A -B illustrates the concept of Intra-Frame TMP, where an L-shaped template is used to search regions R1 to R6 to obtain a displacement vector (i.e., a block vector (BV)), similar to a motion vector. Intra-Frame TMP is an intra-frame coding mode that can only search the available region that has been reconstructed so far. This region is available to both the encoder and decoder when processing the current CU. As for IBC, its encoding process is similar to Intra-Frame TMP, except that IBC uses the raw pixels of the current CU to search its search region to obtain the BV. The BV of IBC should be explicitly signaled in the bitstream so that the decoder can use this information to generate a prediction for the current CU.

[0023] In this invention, a method and apparatus for using a reference region with different encoding attributes than the current block are disclosed. Summary of the Invention

[0024] A method and apparatus for a video encoding / decoding system are disclosed, the system using reference regions with different encoding / decoding attributes to a current block. According to one method, input data associated with a current block in a current image is received, wherein the input data includes pixel data to be encoded at an encoder end or encoded data associated with the current block to be decoded at a decoder end. A first attribute associated with the current block is determined. One or more neighborhoods are determined for the current block, wherein each of the one or more neighborhoods is associated with a second attribute, and one or more target second attributes associated with one or more target neighborhoods are different from the first attribute. A plurality of reference samples in the one or more target neighborhoods are converted into a plurality of transformed reference samples to match the first attribute. Predicted data is derived for the current block based on the plurality of transformed reference samples. The predicted data is used to encode or decode the current block.

[0025] In one embodiment, the first attribute and the second attribute correspond to scaling, cross-component transformation, hue rotation, pixel value mapping, or a combination thereof.

[0026] In one embodiment, for intra-frame coding / decoding, the plurality of reference samples in the one or more target neighborhoods correspond to a plurality of target samples in neighboring codec units (CUs) or a plurality of target samples pointed to by a block vector (BV). In one embodiment, for inter-frame coding / decoding, the plurality of reference samples in the one or more target neighborhoods correspond to a temporal reference region pointed to by a motion vector (MV).

[0027] In one embodiment, the first attribute is determined based on frame-level, region-level, codec tree unit (CTU)-level, or codec unit (CU)-level syntax elements.

[0028] In one embodiment, the plurality of reference samples in the one or more target neighborhoods are first converted into a plurality of intermediate reference samples having a third attribute, and the plurality of intermediate reference samples are subsequently converted into a plurality of transformed reference samples having the first attribute. In one embodiment, the first attribute, the one or more target second attributes, and the third attribute correspond to scaling ratios, and the most commonly used scaling ratio, one or more uniform scaling ratios, or the median scaling ratio among the one or more target second attributes is derived as the third attribute. In one embodiment, the third attribute is selected from a plurality of uniform scaling ratios. In one embodiment, a plurality of buffers are used to store a plurality of reconstructed reference samples associated with the plurality of uniform scaling ratios, and the plurality of transformed reference samples are provided without real-time computation. In one embodiment, the plurality of uniform scaling ratios are signaled or parsed in a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Picture Header (PH), a Slice Header (SH), or a combination thereof.

[0029] Alternatively, a first attribute associated with the current block is determined. Multiple target reference samples are selected from one or more buffers based on this first attribute. Predicted data is derived for the current block based on these multiple target reference samples. The predicted data is then used to encode or decode the current block.

[0030] According to another method, a first attribute associated with one or more neighborhoods of the current block is determined. First predicted data is derived based on multiple reference samples in the one or more neighborhoods. The first predicted data is converted into second predicted data, wherein the second predicted data has the same attribute as the current block. The current block is then encoded or decoded using the second predicted data. Attached Figure Description

[0031] Figure 1The example illustrates a system block diagram for a VVC encoder.

[0032] Figure 2 The examples illustrate various tree structures corresponding to vertical binary splits (SPLIT_BT_VER), horizontal binary splits (SPLIT_BT_HOR), vertical ternary splits (SPLIT_TT_VER), and horizontal ternary splits (SPLIT_TT_HOR).

[0033] Figure 3 The example illustrates a pixel in an L-shaped neighborhood involving multiple neighboring codec units (CUs).

[0034] Figure 4 The example illustrates bidirectional inter-frame prediction.

[0035] Figure 5 The example illustrates how pixel regions and codec units (CUs) involved in a reference frame are obtained from motion vectors (MV).

[0036] Figure 6A The example illustrates the concept of intra-frame TMP, where an L-shaped template is used to search regions R1 to R6 to obtain the displacement vector (i.e., block vector).

[0037] Figure 6B The example illustrates how to derive the intra-frame TMP for the current block based on the displacement vector (i.e., the block vector).

[0038] Figure 7 The example illustrates a candidate location for predicting the generated inferential scaling (SR).

[0039] Figure 8 The example illustrates a flowchart of a video encoding / decoding system that uses one or more reference regions having one or more different attributes from the current block, according to an embodiment of the present invention.

[0040] Figure 9 The example illustrates a flowchart of another video codec system that uses one or more reference regions with one or more different attributes from the current block, according to an embodiment of the present invention.

[0041] Figure 10 The example illustrates a flowchart of another video codec system that uses one or more reference regions having one or more different attributes from the current block, according to an embodiment of the present invention. Detailed Implementation

[0042] It is readily understood that the components of the present invention, as generally described and illustrated in the figures, can be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the system and method of the present invention, as shown in the figures, is not intended to limit the scope of the invention as described in the claims, but is merely representative of selected embodiments of the invention. References to "an embodiment," "one embodiment," or similar language throughout the specification mean that a particular feature, structure, or characteristic described in relation to that embodiment may be included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in one embodiment" appearing in different places in the specification do not necessarily refer to the same embodiment.

[0043] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize that the invention can be practiced without one or more specific details, or using other methods, components, etc. In other instances, well-known structures or operations have not been shown or described in detail to avoid obscuring certain aspects of the invention. Illustrative embodiments of the invention will be best understood by referring to the accompanying drawings, in which like parts are identified by the same numerals throughout the drawings. The following description is merely illustrative, briefly illustrating certain selected apparatus and method embodiments consistent with the claims of this invention.

[0044] In this invention, attribute adjustments are performed when acquiring reference pixels for prediction generation. These attributes include scaling, cross-component transformation, hue rotation, and pixel value mapping (e.g., luminance mapping, gamma curve mapping, or high dynamic range (HDR) tone mapping).

[0045] Intra-frame encoding and decoding with scaling neighborhood

[0046] Reference Picture Resampling (RPR) is an encoding tool that allows the resolution to change throughout the encoding of a video sequence. To display the video, frames with changed resolution are restored to the display resolution. For certain sequences with carefully chosen scaling, RPR-encoded bitstreams can exhibit better rate-distortion (RD) performance than non-RPR-encoded versions. Another application of RPR is adapting to network bandwidth. When network bandwidth is insufficient, enabling RPR and setting a scaling factor to encode the current image at a lower resolution is an effective and practical way to overcome variations in video transmission bandwidth. The same scaling concept can be applied to region (involving multiple codec tree units (CTUs)), CTU-level, or codec unit (CU)-level encoding. For example, a CTU or CU can be encoded using a scaling factor other than 1.0. Blocks with less texture can be encoded more efficiently by reducing pixel samples, providing better rate-distortion (RD) performance. Figure 3 As shown, there are eight CUs, denoted as CU_T0, CU_T1, CU_T2, CU_L0, CU_L1, CU_TL, CU_BL, and CU_TR. These CUs can be encoded using various scaling ratio (SR) options, such as SR = 1.0, 1.25, 1.5, and 2.0. An SR greater than 1.0 indicates that the CU is encoded using a scaling-down process. The possible processing steps are as follows: 1.A At the encoder, the original pixels are downsized, and a downsized prediction is generated. A residual is obtained using the downsized original pixels and the predicted pixels. This residual is then transformed and encoded. At the decoder, the residual is received, a downsized prediction is generated, and the received residuals are summed to obtain the downsized reconstruction. The reconstruction is then upsized to SR = 1.0 to obtain the final reconstruction.

[0047] 1.B At the encoder, the residual is obtained by subtracting the predicted (SR = 1.0) pixels from the original pixels. A reduced residual is generated. This residual is then transformed and encoded. The decoder receives the reduced residual and generates a corresponding amplified residual. A prediction of SR 1.0 is generated and added to the amplified residual to obtain the reconstruction with SR = 1.0.

[0048] Process 1.A (Type A) scales both the original and predicted pixels, while Process 1.B (Type B) scales the residual. Both processes require generating the prediction for the current CU with a specific SR. Process A requires generating the prediction with the target SR of the current CU, while Process B requires generating the prediction with SR = 1.0. However, neighboring CUs may not be encoded with the desired target SR. We propose several methods to handle the prediction generation process, where the SR used for prediction generation is based on the SR of the encoded CU itself or the SRs of neighboring CUs.

[0049] In one embodiment (Mi.1), the reconstructed pixels of neighboring CUs are scaled to the target SR encoding the current CU. Predictions are generated using these scaled pixels. In another embodiment (Mi.2.1), the most frequently used SR among the neighboring CUs is selected as the target SR. If two most frequently used SRs are used the same number of times when encoding neighboring CUs, the SR closer to 1.0 is selected as the target SR. The reconstructed pixels of the neighboring CUs are scaled to the target SR, and a prediction for the current CU is generated. If the target SR generated by the neighboring prediction (denoted as SR) is... P ) and the SR of the CU encoding (denoted as SR) C If the values ​​are different, the generated predictions should be scaled to the resolution required for encoding.

[0050] Table 1 – Exemplary SRs for CU

[0051] For example, as shown in Table 1, the most frequently used SRs in the neighborhood participating in the prediction generation are 1.5 and 1.25, because they are used three times in the encoding. SR 1.25 is closer to 1.0, and therefore is selected as the target SR for prediction generation. SR 1.25 is different from the SR 1.5 encoded by the current CU. Therefore, the generated prediction will be scaled to 1.5 for a Type A residual encoding process. If a Type B residual encoding process is used, the prediction will be scaled to SR 1.0 accordingly. In another embodiment, if two most frequently used SRs are used the same number of times when encoding neighboring CUs, the SR that is closer to the current CU (indicated by an explicit syntax element) is selected as the target SR. However, in some cases, the differences between the two most frequently used SRs and the SR of the current CU are the same. For example, the current CU will be encoded with SR = 1.5, while the two most frequently used SRs are 2.0 and 1.0. The difference between them is 0.5. A predefined strategy can be used to determine the SRs for prediction generation.

[0052] This strategy can select a scaling factor (SR) closer to 1.0 as the predicted SR if the differences between multiple neighboring SRs are the same. Another strategy can consider the area of ​​the neighboring codec unit (CU) or the distance between the neighboring CU and the current CU. The popular SR with a larger cumulative CU area wins, or the popular SR with a shorter cumulative CU distance wins. Therefore, the SR used for prediction is determined by multiple strategies. In another implementation, the SR parsing strategy can be signaled through Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header (PH), Slice Header (SH), etc. In another implementation, to count the number of times the surrounding SRs are used, the SR used to encode the current CU is also considered. C ) SR C The encoder is known, and explicit signaling is given to the decoder in the bitstream. Therefore, when considering the use of SR in the statistical neighborhood, SR... C You can also participate in the process.

[0053] In another embodiment (Mi.2.2), the SR from a specified location in the neighborhood is used for prediction generation. For example... Figure 7 As shown, the SR from position (1), (2) or (3) can be used as the SR generated by prediction.

[0054] In another embodiment (Mi.2.3), the median of the SRs of neighboring CUs is used as the predicted SR. Figure 7For example, the median SR is selected from (1), (2), and (3) as the SR generated for prediction. If the median SR differs from the SR used to encode the current CU, the generated prediction needs to be scaled to calculate the residual. Similar to the most popular SR in the statistical neighborhood, the SR... C It can also be used for median SR generation.

[0055] The predicted predictive signature (SR) determined by the neighborhood can be viewed as an on-the-fly prediction generation method. However, real-time methods require additional computation. For example, if the SR used to encode the current CU changes, the predicted SR will also change, and the neighborhood needs to be rescaled using the changed SR. We propose using a unified SR to scale the neighborhood reconstruction and generate predictions accordingly.

[0056] In one embodiment (Mi.3.1), the unified SR used for neighborhood reconstruction sample scaling to generate future CU predictions can be specified by SPS / PPS / PH / SH, etc. In another embodiment, the unified SR can be a default value of 1.0 (or 1.5 or 2.0). In another implementation (Mi.3.2), the default SR can be updated via syntax elements such as SPS / PPS / PH / SH. Multiple unified SRs can be used in encoding, rather than a single unified SR. For example, SRs 1.0 / 1.5 / 2.0 can be used simultaneously. Three reconstruction buffers are maintained for prediction generation, and the best prediction of a particular SR is selected for prediction generation based on the rate-distortion optimization results in the video encoder. The SR used for encoding CUs is signaled via region / code-decoder tree unit (CTU) / code-decoder unit (CU) level syntax elements.

[0057] In one embodiment (Mi.4.1), multiple unified SRs come from a predefined set of SRs. In another embodiment (Mi.4.2), multiple unified SRs perform signaling in SPS / PPS / PH / SH, etc. The number of allowed SRs selected from these multiple SRs is limited, and additional required frame-level buffers can be pre-allocated for each SR to store reconstructed samples. After encoding the CU, the CU reconstruction is generated against all unified SRs. Then, the reconstructions of all multiple unified SRs are stored in their corresponding buffers for prediction generation of future CU encodings. Possible multiple unified SRs are listed in Table 2.

[0058] Table 2 – Multiple Possible Unified SRs

[0059] Table 2 is not an exhaustive list of multiple unified SRs, but only an example of possible choices for an implementation. It should be noted that the SR used to encode the CU may differ from the SR specified by a single or multiple unified SRs. In the case of multiple unified SRs, it is practically feasible to encode the CU using one of the multiple unified SRs and avoids scaling the prediction of the current CU encoding. If the SR used to encode the current CU differs from any unified SR, a prediction is first generated based on the unified SR. The first prediction is then scaled to match the SR. C Alignment serves as a second prediction for encoding. If the SR used to encode the current CU does not belong to any of the multiple unified SRs, a strategy can be adopted, such as selecting the SR that best matches the current CU. C The SR used for prediction is determined by selecting one of several unified SRs that are closer to 1.0, or by choosing one of several unified SRs that are closer to 1.0. First, predictions are generated based on the determined SR, and then the first prediction is scaled down to the SR. C This serves as the second prediction for encoding the current CU in the encoding process.

[0060] While real-time methods require additional computation, they require less storage. Real-time methods do not need an image-level buffer for the prediction output; they only need storage for neighborhood scaling, and the required storage is typically less than that of an image buffer.

[0061] In another embodiment (Mi.5), a region / CTU / CU level flag is signaled to backtrack the SR to 1.0 for prediction generation. A region refers to an image sub-region that is typically larger than the CTU, and its boundaries may not align with the CTU boundaries. This syntax element introduces another option for rate distortion optimization (RDO), which may be beneficial for generating additional coding gain for certain sequences. This additional flag changes the default behavior of selecting the predicted SR based on neighborhood (e.g., most popular SR, predefined location SR, CU median SR, etc.) and provides a shortcut to selecting SR 1.0. This syntax element exists when the derived SR is not 1.0. In another embodiment, backtracking the SR is signaled in SPS / PPS / PH / SH. When the neighborhood-derived SR differs from the SR signaled by SPS / PPS / PH / SH, the region / CTU / CU level flag is signaled to determine whether to backtrack the predicted SR to the SR signaled by SPS / PPS / PH / SH.

[0062] In another embodiment (Mi.6), region / CTU / CU level syntax elements are used as indexes for signaling to specify the SRs used for prediction generation. This index can be an indicator of the predefined order of the SRs shown in Table 3. Alternatively, the index can serve as an indicator for selecting SRs based on the popularity of neighboring SRs, as shown in Table 4. To understand popularity, the usage of neighboring SRs is collected and sorted according to the number of times they are used.

[0063] Table 3 – SR Indexes with Predefined Order

[0064] Table 4 – Index and Popularity Order of SRs

[0065] Inter-frame encoding / decoding and scaling reference

[0066] For inter-frame encoding / decoding, motion search is invoked to obtain motion vectors (MV). If region, codec tree unit (CTU) level, or codec unit (CU) level encoding / decoding is applied in the reference image, the search region needs to be converted to a uniform scaling factor (SR) for motion search. In one implementation (M.1), the search region (for motion estimation (ME)) or the predicted reference pixel (for motion compensation (MC)) is transformed in real time. Pixels belonging to CUs with different SRs are transformed to the SR of the current CU. Real-time methods can generally be implemented with less memory. However, real-time transformation of the search region can be computationally intensive. Therefore, a more practical approach is to generate a representative frame with a certain SR for each frame for motion search. Furthermore, the current CU is encoded with another SR, which may be the same as or different from the SR of the representative frame. We propose several methods to accomplish the encoding task.

[0067] In one implementation (M.2), the sequence parameter set (SPS), picture parameter set (PPS), picture header (PH), and title sequence (SH) are signaled for the SR of the representative frame. After encoding or decoding a frame, a representative frame is generated for that image based on the SR so that it can be referenced by other frames during inter-frame encoding and decoding. In another implementation, if no syntax element is used to specify the SR, the default SR of the representative frame is 1.0. If the SR of the current CU is different from the SR of the selected representative frame, one of the following procedures can be applied during motion search: 2. Before motion search, the raw pixels of the CU are scaled to align with the SR (denoted as SR) representing the frame. U Perform motion search to obtain the MV of the CU and use it for encoding.

[0068] 2.B Before motion search, both the original pixels of the CU and the acquired search range are scaled to the current CU's SR (SR). C Get the MV and use it for encoding.

[0069] 2.C Before motion search, the acquired search area is scaled to SR 1.0, and the scaled search area is then used for ME using the unscaled original pixels. MV is then acquired and used for encoding.

[0070] If encoding time permits, all procedures 2.A, 2.B, and 2.C can be tried to obtain the minimum RDO cost. The choice of which procedure to use for encoding can be signaled via CU-level syntax elements.

[0071] For prediction generation during motion compensation in inter-frame encoding and decoding, if the SR representing the frame... U With the current CU's SR (SR) C The methods differ. For method 2.A, pixels are obtained from representative frames and the prediction is scaled to align with the SR. C This completes the MC process. For method 2.B, the search range is first scaled to SR. C Therefore, predictions obtained through motion compensation can be directly used for encoding. However, for methods 2.A and 2.C, after generating predictions through the motion compensation process, the predictions may need to be scaled to SR before calculating the residuals. C Furthermore, when calculating the residuals of methods 2.A and 2.B at the encoder end, the original pixels also need to be scaled to SR respectively. U and SR C .

[0072] In another implementation (M.3), multiple SRs are specified in SPS / PPS / PH / SH, etc., or from a predefined set of SRs. After encoding or decoding a frame, representative frames are generated based on these SRs, involving multiple frame-level buffers storing scaled reconstructed samples. For frames with SRs... C When the CU performs inter-frame encoding and decoding (which may be one of multiple SRs, or it may not be), the SR of the reference frame to be used is determined according to the SR. C Select based on the difference between the SR and the representative frame. Select based on the SR. C The representative frame with the smallest difference (which can be zero) is used as a reference to generate predictions. If two representative frames with different SRs are compared with each other, the prediction will be generated based on the SR. C If the difference metric is the same, other decision-making strategies can be applied to determine the final representative frame. For example, only representative frames with an SR closer to 1.0 can be used.

[0073] In another implementation (M.4), SR C Choose from multiple SRs used to represent a frame. This selection can be accomplished using one or more additional region / CU / CT level syntax elements. Therefore, for a P-film, the CU can always find a match with an SR. C A representative frame with the same SR is used as a reference. For film B, there are two reference frames, SR. CThe selected SR can be chosen from the intersection of the SRs specified for each reference frame. In another implementation of B-film, the CU's encoding SR is selected from the SRs of a reference frame. If another reference frame does not have the selected SR, the reference samples of that other reference frame can be scaled to the selected SR for bidirectional encoding of ME and MC.

[0074] In another implementation, for an I-frame (containing only I-slices) that serves as an inter-frame coding and decoding reference frame, the SR representing the frame is aligned with the unified SR used for intra-frame prediction selection as described in the previous section.

[0075] Encoding and decoding of Cross Component Transform (CCT)

[0076] To improve coding gain, a cross-component transform (CCT) is applied before encoding. For example, for the input Cb and Cr, a cross-component chromaticity linear transform can be applied to obtain Cp and Cq, as shown in the following formula: Forward transformation: Cp = a Cb + b Cr + c (Eq.1) Cq = d Cb + e Cr + f (Eq.2) Inverse transformation: Cb =[e (Cp-c) - b (Cq-f)] / (a eb d) (Eq.3) Cr = [d (Cp-c) - a (Cq-f)] / (b da e) (Eq.4) Example of coefficients a through f used to encode 10-bit video: a = 1 / 2, b = 1 / 2, c = 0, d = 1 / 2, e = -1 / 2, f = 512 Cp = 1 / 2 Cb + 1 / 2 Cr + 0 = (Cb + Cr) / 2 Cq = 1 / 2 Cb - 1 / 2 Cr + 512 = (Cb - Cr + 1024) / 2 Cb = [-1 / 2 Cp - 1 / 2 (Cq-512)] / (-1 / 2) = Cp + Cq - 512 Cr = [1 / 2 Cp - 1 / 2 (Cq-512)] / (1 / 2) = Cp - Cq + 512 In the above formula, after Cross-Component Transform (CCT), Cp and Cq are then encoded using a chroma coding process. This transformation can be applied at the frame level, region level, codec tree unit (CTU) level, or codec unit (CU) level. For intra-frame coding, a neighborhood consists of multiple codec units (CUs). Some CUs may be encoded using Cross-Component Transform (CCT), while others may not. Therefore, to generate a coding prediction for the current CU, the neighborhood should be converted to the same domain (i.e., CCT or non-CCT). In one embodiment (Mc.1), the neighborhood conversion is consistent with the coding mode selected by the current CU. The CU coding mode selection can depend on frame-level, region-level, CTU-level, or CU-level settings. For example, if Cross-Component Transform (CCT) is applied to the entire CTU, then the CUs contained in that CTU should be encoded using CCT. Frame-level or region-level CCT settings can be signaled in the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header (PH), or Sheet Header (SH). In frame-level intra-frame coding and decoding, CCT coding does not require converting neighboring domains to the same domain because the entire frame uses the same coding mode (i.e., CCT or non-CCT).

[0077] In another embodiment (Mc.2.1), for CCT encoding at the region / CTU / CU level, predefined locations (e.g.) Figure 7 The location shown is selected as the predicted uniform coding attribute (i.e., CCT or non-CCT). In another embodiment (Mc.2.2), for CCT coding at the region / CTU / CU level, if most CUs in the neighborhood use CCT coding, the current CU is inferred to use CCT coding by default. Otherwise, the current CU is inferred to use a non-CCT coding mode. Similar to the scaling factor commonly used in statistical neighborhoods, the coding mode of the current CU can be considered or ignored when statistically analyzing the popularity of CCT attributes. In yet another embodiment (Mc.3), an additional flag is signaled to determine whether to use the inferred coding mode for CU coding. If the flag indicates that the default inferred mode is not used, another mode is used for coding.

[0078] For inter-frame coding, a reference transformation is required based on the CCT domain needed for coding. In another embodiment (Mc.4), if the coding mode of the reference frame is different from that of the current CU, the reference frame should be transformed into the same domain in order to perform prediction generation such as motion estimation and motion compensation (MC).

[0079] In another inter-frame coding embodiment (Mc.5), prediction is first generated using the CCT attributes of the reference frame. Then, if the coding attributes of the current CU are different from those of the reference frame, a forward CCT or a backward CCT is applied to the prediction to obtain another prediction consistent with the attributes of the current CU, which is then used to predict the current CU.

[0080] In another embodiment (Mc.6), to implement image-level CCT, two reconstruction buffers are maintained. If forward CCT is applied to the current image, the reconstructed samples are stored in one of the reconstruction buffers after adaptive loop filtering (ALF). Subsequently, backward CCT transform is applied to the reconstructed samples, and the result is stored in the other reconstruction buffer. If forward CCT is not applied to the current image, the reconstructed samples are stored in one of the reconstruction buffers after ALF processing. Subsequently, forward CCT transform is applied to the reconstructed samples, and the result is stored in the other reconstruction buffer. These dual reconstruction buffers can be used for image-level coding, predicting and generating the next frame based on the CCT attributes of the current CU.

[0081] Hue Rotation (CPHR) Encoding

[0082] For region / CTU / CU level intra-frame coding, if the CU coding attributes in the reference region differ from the current CU, a transformation should be performed in the inferred region. Similar to Cross-Component Transform (CCT), Hue Rotation (CPHR) can be applied to the chroma components during coding. The forward hue rotation process is described below: 1. Convert Cb and Cr to U and V (for 10-bit video, subtract 512 from Cb and Cr to obtain U and V). 2. Use the rotation matrix of phase θ to pair [U,V] T Rotate to obtain [U', V'] T

[0083] 3. Convert U' and V' to Cb' and Cr' for encoding (for 10-bit video, add 512 to U' and V' to obtain Cb' and Cr').

[0084] Backward hue rotation is similar to the forward process, but vector rotation uses the opposite phase (-θ).

[0085] The Cb' and Cr' obtained by forward hue rotation are used for encoding. After reconstruction, backward hue rotation is applied to the chrominance components to obtain a final reconstruction consistent with the phase of the original video. Hue rotation (CPHR) can be applied at the frame level, region level, CTU level, or CU level. For intra-frame coding, a neighborhood consists of multiple CUs. Some CUs are coded using CPHR, while others are not. Therefore, to generate the coding prediction for the current CU, the neighborhood should be converted to the same domain (CPHR or non-CPHR). In one embodiment (Mr. 1), the neighborhood conversion is consistent with the coding mode selected for the current CU. The coding mode selection of the CU can depend on frame level, region level, CTU level, or CU level settings. For example, if CPHR is applied to the entire CTU, then the CUs contained in that CTU should be coded using CPHR. For frame-level intra-frame coding, CPHR encoding does not require converting the neighborhood to the same domain because the entire frame uses the same domain.

[0086] In another embodiment (Mr. 2.1), for region / CTU / CU level CPHR coding, a predefined location (e.g.) Figure 7 The location shown is selected as the unified coding attribute (i.e., CPHR or non-CPHR) for prediction generation. In another embodiment (Mr. 2.2), for region / CTU / CU level CPHR coding, if most CUs in the neighborhood use CPHR for coding, then the current CU is inferred to use CPHR for coding by default. Similar to the scaling factor commonly used in statistical neighborhoods, the coding mode of the current CU can be considered or ignored when calculating the popularity of the CPHR attribute. In yet another embodiment (Mr. 3), an additional flag is signaled to indicate whether the inferred coding mode is used to code the CU. If the flag indicates that the default inferred mode is not used, then another mode is used for coding.

[0087] For inter-frame coding, a reference conversion is required based on the CPHR domain needed for coding. In another embodiment (Mr. 4), if the coding mode of the reference frame is different from that of the current CU, the reference frame should be converted to the same domain in order to perform motion estimation and prediction generation, etc.

[0088] In another inter-frame coding embodiment (Mr. 5), prediction generation is first performed using the CPHR attribute of the reference frame. Then, if the coding attributes of the current CU are different from those of the reference frame, forward CPHR or backward CPHR is applied to the prediction to obtain another prediction consistent with the attributes of the current CU, which is used to predict the current CU.

[0089] In another embodiment (Mr.6), similar to CCT, a dual reconstruction buffer with non-CPHR and CPHR formats can be used for image-level coding, which is generated based on the CPHR properties of the current CU.

[0090] Encoding of pixel value mapping

[0091] Similar to context-adaptive binary arithmetic coding (CABAC), pixel value mapping (PVM) can be applied to each component during the encoding and decoding process, as described in the following formula: Forward PVM: p' = LUT F [p] (LUT F Forward lookup table) Inverse PVM: p'' = LUT B [p'] (LUT B (Reverse lookup table) PVM can be applied at the frame level, region level, codec tree unit (CTU) level, or codec unit (CU) level. For intra-frame coding, a neighborhood consists of multiple codec units (CUs). Some CUs may use PVM coding, while others may not. Therefore, to generate a prediction for encoding the current CU, the neighborhood should be converted to the same domain (i.e., PVM or non-PVM). In one embodiment (Mp.1), the neighborhood conversion is consistent with the coding mode selected by the current CU. The coding mode selection of a CU can depend on frame-level, region-level, codec tree unit (CTU) level, or codec unit (CU) level settings. For example, if PVM is applied to the entire codec tree unit (CTU), then the CUs contained in that CTU should use PVM coding. For frame-level intra-frame coding, PVM coding does not require the neighborhood to be converted to the same domain because the entire frame is encoded using the same domain.

[0092] In another embodiment (Mp.2.1), for region / code-decode tree unit (CTU) / code-decode unit (CU) level PVM encoding and decoding, predefined locations (e.g.) Figure 7 The location shown is selected as the predicted uniform coding attribute (PVM or non-PVM). In another embodiment (Mp.2.2), for PVM coding at the region / code-decode tree unit (CTU) / code-decode unit (CU) level, if a majority of CUs in the neighborhood use PVM coding, then the current CU is inferred to use PVM coding by default. In yet another embodiment (Mp.3), an additional signaling flag is used to indicate whether the inferred coding mode is used to encode the CU. If the flag indicates that the default inferred mode is not used, then another mode is used for encoding.

[0093] For frame-level inter-frame encoding and decoding, reference conversion needs to be performed according to the PVM domain required for encoding. In another embodiment (Mp.4), if the reference frame does not use the same encoding mode as the current CU, the reference frame should be converted to the same domain in order to perform operations such as motion estimation and prediction generation.

[0094] In another inter-frame coding / decoding embodiment (Mp.5), prediction generation is first performed using the PVM attributes of the reference frame. Subsequently, if the coding attributes of the current CU differ from those of the reference frame, either forward PVM or reverse PVM is applied to the prediction to obtain another prediction consistent with the attributes of the current CU, which is then used to predict the current CU.

[0095] In another embodiment (Mp.6), similar to binary arithmetic codec (CABAC), a dual reconstruction buffer in both non-PVM and PVM formats can be used for predictive generation of image-level coding based on the PVM properties of the current CU.

[0096] Intra-frame encoding / decoding and block vectors (BV)

[0097] Intra-frame coding and decoding with block vector (BV) (IntraTMP and IBC) face similar challenges to inter-frame coding and decoding with motion vector (MV). The proposed method for inter-frame coding and decoding with motion vector (MV) can be extended to intra-frame coding and decoding with block vector (BV) to unify the coding attributes of prediction generation and subsequent coding and decoding processes. It is assumed that multiple reconstruction buffers exist for reference (e.g., one buffer per scaling factor, one buffer per attribute value), storing reconstructed samples with specific attributes. In one embodiment, for intra-frame coding and decoding with block vector (BV), the attributes of the reference region are aligned with the attributes of the current CU. The reconstructed samples are transformed to attributes consistent with those used in the current CU coding and decoding, including scaling, cross-component transform (CCT), hue rotation (CPHR), and pixel value mapping (PVM).

[0098] In another embodiment, for block vector (BV) intra-frame encoding and decoding, a prediction is first generated using reference attributes, and then the prediction is adjusted according to the attributes of the current CU to generate a second prediction.

[0099] Any of the methods described above that employ one or more reference regions with attributes different from the current block can be implemented in the encoder and / or decoder. For example, any method that employs one or more reference regions with attributes different from the current block can be implemented in the inter-frame / intra-frame / prediction module of the encoder and / or the inter-frame / intra-frame / prediction module of the decoder. Alternatively, any of the above methods can also be implemented as circuitry connected to the inter-frame / intra-frame / prediction module of the encoder and / or the inter-frame / intra-frame / prediction module of the decoder to provide the necessary information to the inter-frame / intra-frame / prediction module.

[0100] Figure 8 A flowchart illustrating a video encoding / decoding system employing one or more reference regions with attributes different from the current block, according to an embodiment of the present invention, is provided. The steps shown in the flowchart can be executed as program code at the decoder end of one or more processors (e.g., one or more central processing units (CPUs)). The steps shown in the flowchart can also be implemented in hardware, such as one or more electronic devices or processors configured to perform the steps in the flowchart. According to a method, in step 810, input data associated with a current block in the current image is received, wherein the input data includes pixel data to be encoded at the encoder end or encoded data associated with the current block to be decoded at the decoder end. In step 820, a first attribute associated with the current block is determined. In step 830, one or more neighborhoods are determined for the current block, wherein each of the one or more neighborhoods is associated with a second attribute, and one or more target second attributes associated with one or more target neighborhoods are different from the first attribute. In step 840, a plurality of reference samples in the one or more target neighborhoods are converted into a plurality of transformed reference samples to match the first attribute. In step 850, predicted data is derived for the current block based on the plurality of transformed reference samples. In step 860, the predicted data is used to encode or decode the current block.

[0101] Figure 9 A flowchart illustrating another video encoding / decoding system employing one or more reference regions with attributes different from the current block, according to an embodiment of the present invention, is shown. According to the method, in step 910, input data related to the current block in the current image is received, wherein the input data includes pixel data to be encoded at the encoder end or encoded data related to the current block to be decoded at the decoder end. In step 920, a first attribute related to the current block is determined. In step 930, a plurality of target reference samples are selected from one or more buffers based on the first attribute. In step 940, predicted data is derived for the current block based on the plurality of target reference samples. In step 950, the predicted data is used to encode or decode the current block.

[0102] Figure 10A flowchart illustrating another video encoding / decoding system employing one or more reference regions with attributes different from the current block, according to an embodiment of the present invention, is shown. According to yet another method, in step 1010, input data related to the current block in the current image is received, wherein the input data includes pixel data to be encoded at the encoder end or encoded data related to the current block to be decoded at the decoder end. In step 1020, a first attribute related to one or more neighborhoods of the current block is determined. In step 1030, first prediction data is derived based on multiple reference samples in the one or more neighborhoods. In step 1040, the first prediction data is converted into second prediction data, wherein the second prediction data has the same first attribute as the current block. In step 1050, the current block is encoded or decoded using the second prediction data.

[0103] The flowchart shown is intended to illustrate a video encoding / decoding example according to the present invention. Those skilled in the art can modify each step, rearrange the steps, split a step, or combine steps to implement the present invention without departing from its spirit. Specific syntax and semantics are used in this specification to illustrate examples of implementing embodiments of the present invention. Those skilled in the art can implement the present invention by replacing the syntax and semantics with equivalent syntax and semantics without departing from its spirit.

[0104] The foregoing description is intended to enable those skilled in the art to practice the invention according to specific applications and their needs. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the invention is not intended to be limited to the specific embodiments shown and described, but should be given the broadest scope consistent with the principles and novel features disclosed herein. In the foregoing detailed description, various specific details have been set forth in order to provide a full understanding of the invention. However, those skilled in the art will understand that the invention can be practiced.

[0105] As described above, embodiments of the present invention can be implemented in various hardware, software code, or combinations thereof. For example, embodiments of the present invention may be one or more circuits integrated into a video compression chip, or program code integrated into video compression software, to perform the processes described herein. Embodiments of the present invention may also be program code that executes on a digital signal processor (DSP) to perform the processes described herein. The present invention may also relate to multiple functions performed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors can be configured to implement the specific methods embodied in the present invention by executing machine-readable software code or firmware code. The software code or firmware code may be developed in different programming languages, different formats, or styles. The software code may also be compiled for different target platforms. However, different formats, styles, and languages ​​of the software code, as well as other configuration codes, to achieve the tasks of the present invention, do not depart from the spirit and scope of the present invention.

[0106] This invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The examples described are for illustrative purposes only and not for limitation. Therefore, the scope of this invention is defined by the appended claims rather than the foregoing description. All modifications within the meaning and equivalence of the claims should be included within their scope.

Claims

1. A video encoding / decoding method, comprising: Receive input data related to the current block in the current image, wherein the input data includes pixel data to be encoded at the encoder end or encoded data related to the current block to be decoded at the decoder end; Determine the first attribute associated with the current block; One or more neighborhoods are determined for the current block, each of the one or more neighborhoods being associated with a second attribute, and one or more target second attributes associated with the one or more target neighborhoods being different from the first attribute; Multiple reference samples in one or more target neighborhoods are converted into multiple transformed reference samples to match the first attribute; Based on these multiple transformed reference samples, the prediction data for the current block is derived. as well as Use the predicted data to encode or decode the current block.

2. The method of claim 1, wherein the first attribute and the second attribute correspond to scaling ratio, cross-component transformation, hue rotation, pixel value mapping, or a combination thereof.

3. The method of claim 1, wherein for intra-frame coding and decoding, the plurality of reference samples in the one or more target neighborhoods correspond to a plurality of target samples in neighboring coding and decoding units or a plurality of target samples pointed to by a block vector.

4. The method of claim 1, wherein, for inter-frame coding, the plurality of reference samples in the one or more target neighborhoods correspond to a plurality of temporal reference regions pointed to by motion vectors.

5. The method of claim 1, wherein the first attribute is determined based on frame-level, region-level, codec tree unit-level, or codec unit-level syntax elements.

6. The method of claim 1, wherein the plurality of reference samples in one or more target neighborhoods are first converted into a plurality of intermediate reference samples having a third attribute, and the plurality of intermediate reference samples are subsequently converted into a plurality of transformed reference samples having the first attribute.

7. The method of claim 6, wherein the first attribute, the one or more target second attributes, and the third attribute correspond to scaling ratios, and the most commonly used scaling ratio, one or more uniform scaling ratios, or median scaling ratios among the one or more target second attributes are derived as the third attribute.

8. The method of claim 7, wherein the third attribute is selected from a plurality of uniform scaling ratios.

9. The method of claim 8, wherein multiple buffers are used to store multiple reconstructed reference samples related to the multiple uniform scaling ratios, and the multiple transformed reference samples are provided without real-time calculation.

10. The method of claim 8, wherein the plurality of uniform scaling ratios are signaled or parsed in a sequence parameter set, an image parameter set, an image header, a title sequence, or a combination thereof.

11. A video encoding / decoding apparatus, comprising one or more electronic devices or processors, configured to: Receive input data related to the current block in the current image, wherein the input data includes pixel data to be encoded at the encoder end or encoded data related to the current block to be decoded at the decoder end; Determine the first attribute associated with the current block; One or more neighborhoods are determined for the current block, wherein each of the one or more neighborhoods is associated with a second attribute, and one or more target second attributes associated with one or more target neighborhoods are different from the first attribute; Multiple reference samples in one or more target neighborhoods are converted into multiple transformed reference samples to match the first attribute; Based on these multiple transformed reference samples, the prediction data for the current block is derived. as well as Use the predicted data to encode or decode the current block.

12. A video encoding / decoding method, comprising: Receive input data related to the current block in the current image, wherein the input data includes pixel data to be encoded at the encoder end or encoded data related to the current block to be decoded at the decoder end; Determine the first attribute associated with the current block; Based on this first attribute, multiple target reference samples are selected from one or more buffers; Based on these multiple target reference samples, the prediction data for the current block is derived; as well as Use the predicted data to encode or decode the current block.

13. The method of claim 12, wherein the first attribute corresponds to scaling, cross-component transformation, hue rotation, pixel value mapping, or a combination thereof.

14. The method of claim 12, wherein a plurality of buffers are allocated to store a plurality of reference samples corresponding to different attributes.

15. The method of claim 12, wherein the first attribute is determined based on frame-level, region-level, codec tree unit-level, or codec unit-level syntax elements.

16. A video encoding / decoding apparatus, comprising one or more electronic devices or processors, configured to: Receive input data related to the current block in the current image, wherein the input data includes pixel data to be encoded at the encoder end or encoded data related to the current block to be decoded at the decoder end; Determine the first attribute associated with the current block; Based on this first attribute, multiple target reference samples are selected from one or more buffers; Based on these multiple target reference samples, the prediction data for the current block is derived; as well as Use the predicted data to encode or decode the current block.

17. A video encoding / decoding method, comprising: Receive input data related to the current block in the current image, wherein the input data includes pixel data to be encoded at the encoder end or encoded data related to the current block to be decoded at the decoder end; Determine a first attribute associated with one or more neighborhoods of the current block; The first prediction data is derived based on multiple reference samples in one or more neighborhoods; The first prediction data is converted into second prediction data, wherein the second prediction data has the same attributes as the current block; and Use the second prediction data to encode or decode the current block.

18. The method of claim 17, wherein the first attribute corresponds to scaling, cross-component transformation, hue rotation, pixel value mapping, or a combination thereof.

19. The method of claim 17, wherein, for intra-frame coding / decoding, the plurality of reference samples in the one or more target neighborhoods correspond to a plurality of target samples in neighboring coding / decoding units or a plurality of target samples pointed to by a block vector.

20. The method of claim 17, wherein, for inter-frame encoding / decoding, the plurality of reference samples in the one or more target neighborhoods correspond to a plurality of time reference regions pointed to by motion vectors.

21. The method of claim 17, wherein the first attribute is determined based on frame-level, region-level, codec tree unit-level, or codec unit-level syntax elements.

22. A video encoding / decoding apparatus, the apparatus comprising one or more electronic devices or processors configured to: Receive input data related to the current block in the current image, wherein the input data includes pixel data to be encoded at the encoder end or encoded data related to the current block to be decoded at the decoder end; Determine a first attribute associated with one or more neighborhoods of the current block; The first prediction data is derived based on multiple reference samples in one or more neighborhoods; The first prediction data is converted into second prediction data, wherein the second prediction data has the same attributes as the current block; and Use the second prediction data to encode or decode the current block.

Images