Motion compensated bi-prediction based on local illumination compensation

Abstract

Systems, methods, and devices for calculating local illumination compensation (LIC) parameters for bi-predicted coding units (CUs) are provided.SOLUTION: LIC parameters may be used to generate adjusted samples for the current CU to account for local illumination changes that may exist between temporal neighboring pictures. The LIC parameters are calculated based on the bi-predicted reference template samples and the template samples for the current CU. The bi-predicted reference template samples may be generated based on reference template samples adjacent to the temporal reference CU. Bi-predicted reference template samples are generated based on averaging of the reference template samples. The reference template sample may correspond to the template sample for the current CU. The CU may be a coding block and / or a sub-block that can be derived by partitioning a coding block, or may include a coding block and / or a sub-block.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 62 / 617,964, filed January 16, 2018, U.S. Provisional Patent Application No. 62 / 647,273, filed March 23, 2018, and U.S. Provisional Patent Application No. 62 / 786,671, filed December 31, 2018, the contents of which are incorporated herein by reference. [Background technology]

[0002] Video coding systems may be used to compress video signals to, for example, reduce the storage space and / or transmission bandwidth of such signals. Various types of video coding systems may exist, such as block-based, wavelet-based, object-based, and / or block-based hybrid video coding systems. Block-based video coding systems may support international video coding standards such as MPEG 1 / 2 / 4 part 2, H.264 / MPEG-4 part 10 AVC, VC-1, and / or High Efficiency Video Coding (HEVC). Summary of the Invention [Problem to be solved by the invention]

[0003] A system, method, and apparatus are provided for calculating local illumination compensation (LIC) parameters based on bi-predicted reference template samples and template samples.

[0004] Described herein are systems, methods, and devices for calculating local illumination compensation (LIC) parameters based on bi-predicted reference template samples and template samples. The calculated LIC parameters may be used to account for local illumination changes that may exist between temporally adjacent pictures. For example, the calculated LIC parameters for a current coding unit (CU) may be used to generate adjusted samples for the current CU to account for the local illumination changes.

[0005] A video CU may be processed. For example, the video CU may be or include a coding block or a sub-block. The sub-block may be derived by dividing a standard coding block. The current CU may be received in a video bitstream.

[0006] The current CU may be predicted. It may be determined whether the current CU is bi-predicted. If the current CU is bi-predicted, a reference template sample associated with the current CU may be identified. For example, the reference template sample may be a neighboring temporal reference CU of the current CU. The reference template sample may correspond to a template sample for the current CU that is neighboring the current CU. For example, the template sample for the current CU may be a neighboring sample of the current CU. The reference template sample that is neighboring the temporal reference CU may be identified based on, for example, a motion vector of the current CU.

[0007] The bi-predicted reference template sample may be generated based on the reference template sample. For example, the bi-predicted reference template sample may be generated based on averaging reference template samples adjacent to the temporal reference CU. The reference template sample for the current CU may be identified. The reference template sample may be adjacent to the temporal reference CU of the current CU corresponding to the template sample for the current CU.

[0008] The bi-predicted reference template sample may be used to calculate an LIC parameter. For example, the IC parameter may be calculated based on the generated bi-predicted reference template sample and the template sample for the current CU. In an embodiment, the LIC parameter may be calculated by minimizing the difference between the bi-predicted reference template sample and the template sample for the current CU.

[0009] The difference between the set of bi-predicted reference template samples and the set of template samples for the current CU may be minimized based on a least mean square error (LMSE) approach. In an embodiment, the LIC parameters may be calculated based on a linear model approach using minimum and maximum values ​​associated with the set of bi-predicted reference template samples and the set of template samples for the current CU. The calculated LIC parameters may be applied to the bi-predicted CU to generate an adjusted bi-predicted current CU.

[0010] The LIC parameters may be applied in an overlapped block motion compensation (OBMC) stage. For example, it may be determined whether a subblock of a current CU is on a boundary of the current CU. If the subblock of the current CU is on a boundary of the current CU, one or more neighboring subblocks of the subblock may be identified. One or more motion vectors associated with the neighboring subblocks may be applied to samples in the subblock to derive one or more template reference samples. OBMC predicted samples may be generated based on the template reference samples. For example, the OBMC predicted samples may be generated based on averaging the template reference samples. The LIC parameters may be applied to the generated OBMC predicted samples. In an embodiment, the LIC parameters calculated in the motion compensation stage may be cached in a memory or buffer and used in the OBMC stage. In an embodiment, the LIC parameters may be fetched from a memory or buffer and used in the OBMC stage.

[0011] In an embodiment, a template sample for a current CU may be identified. The template sample may be subsampled. LIC parameters may be calculated based on the subsampled template sample. A template sample for the current CU may be identified among the subsampled template samples.

[0012] In an embodiment, a reference template sample used for LIC parameter calculation may be identified. The reference template sample may be adjacent to a temporal reference CU corresponding to the template sample for the current CU. The reference template sample may be subsampled. The LIC parameters may be calculated based on the subsampled reference template sample. The reference template sample used for LIC calculation (e.g., adjacent to a temporal reference CU corresponding to the template sample for the current CU) may be identified from the subsampled reference template sample. [Effects of the Invention]

[0013] A system, method, and apparatus are provided for calculating local illumination compensation parameters based on novel bi-predicted reference template samples and template samples. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 illustrates an example of a block-based hybrid video encoding system. [Figure 2] FIG. 1 illustrates an example of a block-based video decoder. [Figure 3] FIG. 1 illustrates an example of local illumination compensation (LIC) processing. [Figure 4] FIG. 1 illustrates an example of an improved temporal motion vector prediction (ATMVP) operation. [Figure 5] FIG. 1 is a diagram illustrating an example of spatial-temporal motion vector prediction (STMV) calculation. [Figure 6A] FIG. 1 is a diagram illustrating an example of frame-rate up conversion (FRUC) calculation using template matching. [Figure 6B]FIG. 10 is a diagram illustrating an example of FRUC calculation using bilateral-matching. [Figure 7] FIG. 1 illustrates an example of an overlapped block motion compensation (OBMC) operation. [Figure 8A] FIG. 10 illustrates an example of a motion compensation operation associated with generating a prediction signal for a coding block when LIC is applied. [Figure 8B] FIG. 10 illustrates an example of a motion compensation operation associated with generating a prediction signal for a coding block when LIC is applied. [Figure 9] FIG. 10 is a diagram illustrating an example of bidirectional LIC calculation. [Figure 10A] FIG. 10 is a diagram illustrating an example of motion compensation calculation when bidirectional LIC is applied. [Figure 10B] FIG. 10 is a diagram illustrating an example of motion compensation calculation when bidirectional LIC is applied. [Figure 11A] FIG. 10 illustrates an example of a motion compensation operation when LIC parameters are derived once at the OBMC stage. [Figure 11B] FIG. 10 illustrates an example of a motion compensation operation when LIC parameters are derived once at the OBMC stage. [Figure 12A] FIG. 10 illustrates an example motion compensation operation during which LIC parameters derived from standard motion compensation can be reused for OBMC. [Figure 12B] FIG. 10 illustrates an example motion compensation operation during which LIC parameters derived from standard motion compensation can be reused for OBMC. [Figure 13A] FIG. 10 illustrates an example compensation operation in which LIC is performed after a prediction signal is generated for OBMC. [Figure 13B] FIG. 10 illustrates an example compensation operation in which LIC is performed after a prediction signal is generated for OBMC. [Figure 14A] FIG. 10 illustrates an example compensation operation in which LIC parameters are derived in a standard motion compensation stage and LIC-based sample adjustment is performed after OBMC. [Figure 14B]FIG. 10 illustrates an example compensation operation in which LIC parameters are derived in a standard motion compensation stage and LIC-based sample adjustment is performed after OBMC. [Figure 15] 1 shows an example block diagram of a video encoder with generalized bi-prediction (GBi) support. [Figure 16] 1 shows an example block diagram of a GBi module. [Figure 17] 1 shows an exemplary block diagram of a video decoder that supports GBi modules. [Figure 18] 1 shows an exemplary block diagram of a GBi module. [Figure 19A] FIG. 1 is a system diagram illustrating an example communication system in which one or more disclosed embodiments may be implemented. [Figure 19B] 19B illustrates an exemplary wireless transmit / receive unit (WTRU) that may be used within the communications system of FIG. 19A, according to an embodiment. [Figure 19C] 19B illustrates an exemplary radio access network (RAN) and core network (CN) that may be used within the communications system of FIG. 19A, according to an embodiment. [Figure 19D] 19B is a system diagram illustrating a further exemplary RAN and CN that can be used within the communication system of FIG. 19A, according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] A detailed description of exemplary embodiments will now be described with reference to various figures. While this description provides detailed examples of possible implementations, it should be noted that the details are intended to be illustrative and not to limit the scope of the present application.

[0016] A video coding system may include a digital video signal to, for example, reduce storage space and / or transmission bandwidth associated with storing and / or distributing the digital video signal. The video coding system may include a block-based system, a wavelet-based system, an object-based system, etc. The block-based video coding system may support international video coding standards such as MPEG-1 / 2 / 4 part 2, H.264 / MPEG-4 part 10 AVC, VC-1, and / or High Efficiency Video Coding (HEVC).

[0017] HEVC can provide bitrate savings (e.g., approximately 50%) or comparable perceptual quality compared to previous generation video coding technologies (e.g., H.264 / MPEG-AVC). Superior coding efficiency can be achieved through HEVC (e.g., through additional coding tools). A software code base, such as the Joint Exploration Model (JEM), may be based on the HEVC Model (HM). Coding may be integrated into the JEM software and tested, for example, using the JVET common test conditions (CTC). The HM and / or JEM software may be based on a block-based hybrid video coding framework.

[0018] FIG. 1 shows an example of a block-based hybrid video coding system 600. An input video signal 602 may be processed in coding units (CUs). A CU may include one or more video blocks or sub-blocks. A CU (e.g., a video block or sub-block) may be associated with a particular size (e.g., a number of pixels) and may be used to compress high-resolution (e.g., 1080 pixels or higher) video signals (e.g., in HEVC). A CU may include, for example, 64×64 pixels. A CU may be partitioned (e.g., into prediction units (PUs)). Separate (e.g., the same or different) prediction procedures may be applied to PUs. For example, spatial prediction 660 and / or temporal prediction 662 may be performed on (e.g., each) input video block (e.g., macroblock (MB) or CU).

[0019] Spatial prediction (e.g., intra prediction) may predict a current video block, for example, by using pixels from samples (e.g., reference samples) of one or more already-encoded neighboring blocks. The current video block and one or more already-encoded neighboring blocks may be in the same video picture or video slice. Spatial prediction may reduce temporal redundancy that may be inherent in a video signal. Temporal prediction (e.g., inter prediction, motion-compensated prediction, etc.) may predict a current video block, for example, by using reconstructed pixels from one or more already-encoded video pictures. Temporal prediction may reduce spatial redundancy that may be inherent in a video signal. A temporal prediction signal for a given video block may be signaled, for example, by one or more motion vectors (MVs), which may indicate the amount and / or direction of motion between the current block and a reference block. For example, when multiple reference pictures can be supported (e.g., for H.264 / AVC or HEVC), a reference picture index may be transmitted (e.g., for each video block). The reference index may be used to identify a reference picture (eg, in the reference picture store 664) from which the temporal prediction signal can be derived.

[0020] A mode decision and control logic unit 680 in the encoder may select (e.g., best) prediction mode (e.g., after spatial prediction and / or temporal prediction), for example, based on a rate-distortion optimization procedure. The prediction block may be subtracted from the current video block 616. The prediction residual may be decorrelated (e.g., in transform unit 604) and quantized (e.g., in quantization unit 606). The quantized residual coefficients may be inverse quantized (e.g., in 610) and inverse transformed (e.g., in 612), for example, to form a reconstructed residual. The reconstructed residual may be added back to the prediction block 626, for example, to form a reconstructed video block. For example, in-loop filtering (e.g., a deblocking filter and / or an adaptive loop filter) may be applied to the reconstructed video block (e.g., by loop filter 666) before being placed in the reference picture store 664. The reconstructed video block may be used to encode a subsequent video block. The coding mode information (e.g., inter or intra), prediction mode information, motion information, and / or quantized residual coefficients may be transmitted to the entropy encoding unit 608. One or more of the information may be compressed by the entropy encoding unit 608 and packed into the output bitstream 620.

[0021] 2 shows an example of a block-based video decoding system (e.g., a video decoder). A video bitstream 202 may be unpacked and entropy decoded in an entropy decoding unit 208. Coding mode and / or prediction mode information may be sent to a spatial prediction unit 260 (e.g., when intra-coded) or a temporal prediction unit such as a motion compensation prediction unit 262 (e.g., when inter-coded), for example, to form a prediction block. Residual transform coefficients may be provided to an inverse quantization unit 210 and an inverse transform unit 212, for example, to reconstruct a residual block. The prediction block and the residual block may be added together, for example, via an addition operation, at 226. In-loop filtering 266 may be applied to the reconstructed block before being stored in a reference picture store 264, for example. The reconstructed video 220 before being stored in a reconstructed picture store 264 may be sent (e.g., from the reference picture store 264) to drive a display device and / or predict a later video block.

[0022] The encoding system may implement the exemplary encoding / decoding workflow shown in Figures 1 and 2. The encoding system may include one or more of the functional units shown in Figures 1 and 2, such as a spatial prediction unit (e.g., for intra prediction), a temporal prediction unit (e.g., for inter prediction), a transform unit, a quantization unit, an entropy coding unit, and / or a loop filter.

[0023] In an embodiment (e.g., when motion-compensated prediction is used), motion information (e.g., motion vectors (MVs) and / or reference picture indices) may be used (e.g., for each intra-coded block) to track corresponding matching blocks in corresponding reference pictures (e.g., which can be synchronized between an encoder and a decoder). Multiple modes (e.g., two modes) may be used to encode the motion information of inter-blocks, e.g., to reduce overhead associated with transmitting motion information in a bitstream. The modes may include, for example, a merge mode and a non-merge mode. When a block is encoded using a non-merge mode, the MV may be coded using an MV predictor (e.g., differentially coded), and the difference between the MV and the MV predictor may be transmitted to the decoder. When a block is encoded using a merge mode, the motion information of the block may be derived from spatial and / or temporal neighboring blocks, and a competition-based scheme may be applied to select a motion (e.g., best motion) based on a group of candidate blocks. An index of the selected motion (e.g., best motion) may be used to re-establish the same motion information at the decoder.

[0024] To address local illumination changes, local illumination compensation (LIC) may be used. The local illumination changes may exist, for example, between temporally adjacent pictures. LIC may be based on a linear model. LIC may apply a scaling factor and an offset to a reference sample. For example, LIC may apply a scaling factor and an offset to the reference sample to obtain a predicted sample for the current block. LIC may be expressed by a mathematical formula such as the following Equation (1):

[0025]

number

[0026] P(x,y) may be a prediction signal of the current block at coordinates (x,y). r (x+v x ,y+v y ) is the motion vector (v x ,v y ) The LIC parameters α and β may represent a scaling factor and an offset, respectively, that can be applied to the reference block.

[0027] FIG. 3 illustrates an example of an LIC operation. As shown, when LIC is applied to a video block such as a CU, a least mean square error (LMSE) approach may be employed to derive LIC parameters (e.g., α and β). The process may include minimizing differences between neighboring samples of the current block (e.g., template samples in template T, as shown in FIG. 3) and their corresponding reference samples in one or more temporal reference pictures (e.g., reference template samples such as T0 and / or T1, as shown in FIG. 3). For example, the process may minimize differences between template samples for the current block and reference template samples neighboring a temporal reference CU that corresponds to the template samples for the current CU. This may be illustrated by equations (2) and (3).

[0028]

number

[0029]

number

[0030] The parameter N may represent the number of template samples that can be used to derive the LIC parameters. i ,y i ) is the coordinate (x i ,y i) may represent the template sample of the current block in

[0031]

number

[0032] may represent a reference sample corresponding to a template sample (e.g., a reference template sample) based on a motion vector associated with the current block (e.g., MV0 associated with L0 or MV1 associated with L1). The template sample (e.g., for the current CU) and the reference template sample (e.g., adjacent to the temporal reference CU) may be subsampled (e.g., via 2:1 subsampling) to derive the LIC parameters. For example, the shaded samples shown in FIG. 3 may be used to derive the LIC parameters.

[0033] LIC parameters may be derived and applied to prediction directions, for example, L0 and L1. In an embodiment, when LIC is applied to a bidirectional block or when the current block is predicted by two temporal predictive blocks, LIC parameters may be derived and applied to prediction directions, for example, L0 and L1. LIC parameters may be derived and applied to different directions (e.g., separately). FIG. 3 shows that two reference template samples T0 and T1 can be obtained based on two motion vectors MV0 and MV1. Corresponding pairs of LIC parameters in the two directions may be derived according to Equations (2) and (3), for example, by separately minimizing the distortion between T0 and T and between T1 and T. A bidirectionally predicted signal (e.g., a bidirectionally predicted signal) of the current block may be generated by combining two LIC uni-predictive blocks, as shown in Equation (4).

[0034]

number

[0035] α0 and β0 are L0 motion vectors

[0036]

number

[0037] , and β1 are the LIC parameters associated with the L1 motion vector.

[0038]

number

[0039] and the associated LIC parameter.

[0040]

number

[0041] and

[0042]

number

[0043] may be the corresponding temporal reference blocks of the current block from lists L0 and L1, respectively.

[0044] Motion compensation may be sub-block-based. Sub-block-level motion compensation approaches may include advanced temporal motion vector prediction (ATMVP), spatial-temporal motion vector prediction (STMVP), and / or frame-rate up conversion (FRUC) modes, etc. As described herein, a coding block may be associated with a motion vector for a prediction direction. A coding block may be further divided into multiple small sub-blocks (e.g., in one or more of the above-mentioned coding modes), and a motion block for (e.g., each) sub-block may be derived separately. For example, sub-block motion information may be used to generate a prediction signal for a sub-block (e.g., ultimately for a coding block) in the motion compensation stage. A coding unit referred to herein may be or include a coding block or a sub-block. A CU may include multiple video processing and distribution units (VPDUs). For example, a CU may include multiple VPDUs depending on its size. A VPDU may process a region (e.g., a 64x64 square region) before starting processing another region (e.g., another 64x64 square region). The VPDU may be included in a hardware implementation. The VPDU may impose one or more CU partitioning constraints. For example, if the parent block size is larger than 64 samples in one direction, triple tree (TT) partitioning may not be allowed. For example, if the resulting child CU size is smaller than 64 samples in one direction and larger than 64 samples in the other direction, binary tree (BT) partitioning may not be allowed.Those skilled in the art will recognize that CU or CU level as used throughout this specification may include VPDU or VPDU level, and CU or CU level may be used interchangeably with VPDU or VPDU level.

[0045] ATMVP can improve temporal motion vector prediction, for example, by allowing a block to derive multiple pieces of motion information (e.g., information related to motion vectors and reference indices) for sub-blocks within the current block. The motion information for a sub-block may be derived, for example, from a corresponding small block in a temporally neighboring picture of the current picture. One or more of the following may be performed: A block, such as a collocated block, corresponding to the current block may be identified in a temporal reference picture (e.g., an aligned picture). The current block may be divided into one or more sub-blocks. The motion information of a sub-block may be derived based on the corresponding small block in the aligned picture.

[0046] FIG. 4 illustrates an example of an ATMVP operation. The aligned blocks and aligned pictures may be identified, for example, based on motion information associated with one or more spatially neighboring blocks of the current block. In an embodiment, as shown in FIG. 4, the first available candidate in a merge candidate list may be considered. For example, block A may be estimated to be the first available merge candidate for the current block based on the scan order of the merge candidate list associated with the current block. To identify the aligned pictures and aligned blocks for the current block, the corresponding motion vector (e.g., MV) of block A may be used. A ) and its reference index may be used. For example, the motion vector (e.g., MV A ) to the coordinates of the current block, the position of the aligned block within the aligned picture may be determined.

[0047] For example, motion information associated with a sub-block of the current block may be derived based on the corresponding small blocks of the sub-blocks in the ordered block (e.g., as indicated by short arrows in FIG. 4). This may be done for one or more sub-blocks in the current block. The identified motion information of (e.g., each) small block in the ordered block may be converted into a motion vector reference index for the corresponding sub-block in the current block. The conversion may be performed in a manner similar to temporal motion vector prediction (TMVP), for example, where temporal motion vector scaling may be applied.

[0048] Motion information associated with sub-blocks of a coding block may be derived in a recursive manner (e.g., in STVMP). FIG. 5 illustrates an example of an STMVP operation. For example, FIG. 5 shows that a current block may include four sub-blocks (e.g., A, B, C, and D). Neighboring small blocks (e.g., sub-blocks) of the current sub-block A (e.g., spatial neighbors of sub-block A) may be labeled a, b, c, and d (e.g., shaded blocks shown in FIG. 5). The neighboring small blocks may have (e.g., each has) the same size as the current sub-block A. Motion derivation for sub-block A may identify its spatial neighbors (e.g., two spatial neighbors). The first neighbor may be small block c, which may be located above the current sub-block A. If a small block is not available or is intra-coded, one or more other neighboring small blocks (e.g., blocks above the current block) may be checked in a particular order (e.g., from left to right).

[0049] A second neighbor of sub-block A may be small block b, which may be located to the left of the current sub-block A. If small block b is not available or is intra-coded, one or more other neighboring small blocks (e.g., blocks to the left of the current block) may be checked in a particular order (e.g., from top to bottom). Motion information associated with one or more spatial neighbors of sub-block A may be fetched. Motion information associated with one or more temporal neighbors of sub-block A may be obtained by following a procedure similar to the TMVP process. Motion information associated with available spatial and temporal neighbors (e.g., up to three) may be averaged and used as motion information for sub-block A. The STMVP process described herein may be repeated to derive motion information for sub-blocks within the current video block. For example, the STMVP process may be repeated based on a raster scan order to derive motion information for sub-blocks within the current video block.

[0050] For example, FRUC may be performed on inter-coded blocks. FRUC may skip signaling motion information associated with coded blocks (e.g., information associated with motion vectors and / or reference indices). The motion information may be derived at the decoder side, for example, using template matching and / or bilateral matching techniques. A merge candidate list and / or a set of preliminary motion vectors associated with the current block may be checked (e.g., during a motion derivation procedure in the decoder) to identify a candidate that can provide the smallest sum of absolute differences (SAD). The set of preliminary motion vectors may be generated based on one or more motion vectors associated with one or more temporally aligned blocks of the current block. A candidate may be selected as a starting point. A local search around the starting point may be performed. For example, the local search may be based on template matching and / or bilateral matching, etc. The MV that may result in the smallest SAD may be considered as the MV for the entire current block. The motion information may be refined at the sub-block level.

[0051] 6A and 6B are diagrams illustrating an example of FRUC. FIG. 6A illustrates an example of template matching. FIG. 6B illustrates an example of bilateral matching. Template matching (e.g., as shown in FIG. 6A) may be used to derive motion information for a current block by, for example, finding a match (e.g., a best match) between a template in a current picture (e.g., an upper and / or left neighboring block of the current block) and a block in a reference picture (e.g., having the same size as the template). Bilateral matching (e.g., as shown in FIG. 6B) may be used to derive motion information for a current block by, for example, finding a best match between two blocks along the motion trajectory of the current block in multiple (e.g., two) different reference pictures. The motion estimation process for bilateral matching may be based on the motion trajectory. In an embodiment, motion vectors (e.g., MV0 and MV1) indicating reference blocks may be proportional to the temporal distance between the current picture and (e.g., each) reference picture (e.g., T0 and / or T1).

[0052] FRUC motion estimation (e.g., in the context of template matching and bilateral matching) may be performed. CU-level motion estimation may be performed. An initial MV may be derived for the entire CU (e.g., the current CU). Motion information associated with the CU may be refined at the sub-block level, for example, using the derived CU-level MV as a starting point. The current CU may be divided into one or more sub-blocks (e.g., M×M sub-blocks). The value of M may be calculated, for example, according to Equation (5).

[0053]

number

[0054] w and h may represent the width and height of the current CU, respectively. The parameter D may be a predefined splitting depth, which may be set to, for example, 3, and may be signaled in a sequence parameter set (SPS).

[0055] Overlapped block motion compensation (OBMC) may be applied to remove disturbing artifacts in the motion compensation stage. OBMC may be performed on one or more (e.g., all) inter-block boundaries, except for the right and bottom boundaries of a block. In an embodiment, when a video block is coded in a sub-block mode (e.g., ATMVP, STMVP, and / or FRUC, etc.), OBMC may be performed on one or more of the sub-block boundaries (e.g., all four of the sub-block boundaries).

[0056] 7 is a diagram illustrating an example of an OBMC operation. OBMC may be applied to a subblock (e.g., subblock A in FIG. 7) and / or a motion vector associated with the current subblock. When OBMC is applied to a subblock, motion vectors associated with neighboring subblocks (e.g., up to four neighboring subblocks) may be used to derive a prediction signal for the current subblock. Predictions using motion vectors associated with neighboring subblocks may be averaged to generate a prediction signal for the current subblock.

[0057] A weighted average may be used in OBMC to generate a prediction signal for a block. A prediction signal using a motion vector of a neighboring subblock may be represented as PN. A prediction signal using a motion vector of the current subblock A may be represented as PC. Samples (e.g., when OBMC is applied) in a specific row and / or column (e.g., the first and / or last four rows / columns) of PN may be weighted-averaged with samples at the same position in PC. The samples to which the weighted average is applied may be determined for corresponding neighboring subblocks. For example, the samples to which the weighted average is applied may be determined for corresponding neighboring subblocks based on the positions of the neighboring subblocks.

[0058] In an embodiment, when a neighboring subblock is above the current subblock A (e.g., neighboring subblock b in FIG. 7), samples in the first X rows of the current subblock may be adjusted. In an embodiment, when a neighboring subblock is below the current subblock A (e.g., neighboring subblock d in FIG. 7), samples in the last X rows of the current subblock may be adjusted. In an embodiment, when a neighboring subblock is to the left of the current subblock (e.g., subblock a in FIG. 7), samples in the first X columns of the current subblock may be adjusted. In an embodiment, when a neighboring subblock is to the right of the current subblock (e.g., subblock c in FIG. 7), samples in the last X columns of the current subblock may be adjusted.

[0059] The values ​​of X and / or weights may be determined based on the coding mode used to code the current block. In some embodiments, when the current block coding is skipped in sub-block mode, weighting factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} may be used for the first four rows / columns of the PN, and weighting factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} may be used for the first four rows / columns of the PC. In some embodiments, when the current block is coded in sub-block mode, the first two rows / columns of the PN and PC (e.g., only the first two rows / columns) may be averaged. In these embodiments, weighting factors {1 / 4, 1 / 8} may be used for the PN, and weighting factors {3 / 4, 7 / 8} may be used for the PC.

[0060] The generalized bi-prediction (GBi) can improve the motion compensation prediction for the bi-prediction mode. In the bi-prediction mode, the prediction signal at sample x shown in Equation (6) may be calculated by Equation (6).

[0061]

number

[0062] P[x] may represent a predicted signal (e.g., a resulting predicted signal) of sample x located at picture position x. i [x+v i ] is the motion vector (MV) v for the i-th list (e.g., list 0, list 1). i(w0, w1) may indicate a motion compensated prediction signal of x using (w0, w1). The parameters w0 and w1 may indicate weight values ​​shared across one or more (e.g., all) samples in the block. One or more prediction signals may be obtained by adjusting weight values ​​such as w0 and w1 based on Equation (6). For example, the weight values ​​w0 and / or w1 may be configured to indicate the same prediction for uni-prediction and bi-prediction. The weight values ​​may be configured to be (w0, w1) = (1, 0) for uni-prediction with reference list L0, (w0, w1) = (0, 1) for uni-prediction with reference list L1, and (w0, w1) = (0.5, 0.5) for bi-prediction with two reference lists. Weights may be signaled per CU. The weight value may be set to w0 + w1 = 1. In such a case, one weight (e.g., only one weight) may be signaled, and Equation (6) may be simplified as shown in Equation (7).

[0063]

number

[0064] The weight value w1 may be discretized, for example, {-1 / 4, 1 / 4, 3 / 8, 1 / 2, 5 / 8, 3 / 4, 5 / 4}. The weight value may be represented by an index value within a range (e.g., a small, restricted range).

[0065] FIG. 15 shows an example block diagram of a video encoder with generic GBi support. Similar to the video encoder shown in FIG. 1, spatial prediction and temporal prediction may be two pixel-domain prediction modules in the video encoder shown in FIG. 15. The spatial prediction module may be similar (e.g., identical) to the one introduced in FIG. 1. The temporal prediction module for motion prediction in FIG. 1 may be extended with GBi support, for example, by enabling bi-prediction to combine two separate prediction signals in a weighted average manner. The selected weight index may be signaled in the bitstream. FIG. 16 shows an example block diagram of a bi-prediction module. As shown in FIG. 16, the GBi estimation module may include a process for generating an inter-prediction signal. The GBi estimation module may perform motion estimation on the reference picture(s). For example, the GBi estimation module may search for two motion vectors (MVs) (e.g., two optimal MVs) indicating two reference blocks and may search for a weight index (e.g., an optimal weight index) to minimize a weighted bi-prediction error between the current video block and the bi-prediction.

[0066] A generalized bi-predictive prediction signal may be calculated, for example, as a weighted average of two prediction blocks.

[0067] 17 shows an example block diagram of a video decoder that supports a GBi module and decodes a bitstream generated by a video encoder (e.g., shown in FIG. 15). The coding mode and prediction information may be used to derive a prediction signal, for example, using spatial prediction or motion-compensated prediction (e.g., with generalized bi-prediction support). For generalized bi-prediction, for example, block motion information and weight values ​​(e.g., in the form of indices indicating the weight values) may be received and decoded to generate a prediction block.

[0068] Using the block motion information and the weight values, a generalized bi-prediction module (e.g., shown in FIG. 16) may calculate a generalized bi-prediction prediction signal, for example, as a weighted average of two motion-compensated prediction blocks. FIG. 18 shows an example block diagram of a bi-prediction module. As shown in FIG. 18, the GBi estimation unit may include a weight value estimation unit and a motion estimation unit. The GBi estimation unit may generate an inter-prediction signal, such as a final inter-prediction signal. The weight value estimation unit may search for a weight index (e.g., an optimal weight index) to minimize a weighted bi-prediction error between the current video block and the bi-prediction signal.

[0069] The GBi described herein may be or include bi-predictive weighted averaging (BPWA).

[0070] As described herein, LIC can extend standard motion compensation prediction, for example, by addressing illumination changes between different pictures in the motion compensation stage. Figures 8A-8B illustrate exemplary motion compensation operations associated with generating a prediction signal for a coding block when LIC is applied. In the example of Figures 8A-8B, the coding block may be bi-predictive and may include multiple sub-blocks (e.g., the block may be coded in sub-block mode). The dotted text blocks in Figures 8A-8B can describe LIC-related operations.

[0071] In an embodiment, when a current block or sub-block is bi-predicted, LIC may be applied separately to prediction signals in reference lists L0 and L1, e.g., as shown in equation (4). LIC parameter estimation and / or LIC-based sample adjustment (e.g., as shown in equation (1)) may be performed twice to generate a prediction signal for the current block or sub-block. Such an approach (e.g., as shown in equations (2) and (3)) may derive a LIC scaling factor and offset (e.g., using a template-based approach). LIC parameters (e.g., LIC scaling factor and / or offset) may be combined with a sub-block coding mode (e.g., ATMVP, STMVP, and / or FRUC, etc.). Depending on the sub-block coding mode, a block may be divided into one or more (e.g., multiple) sub-blocks, and the sub-blocks may be associated with corresponding motion vectors (e.g., unique motion vectors). When LIC is applied to a block, LIC parameters may be derived for the prediction direction (e.g., L0 and L1) of the sub-block.

[0072] In an embodiment, when LIC is applied to a coding block (e.g., as shown in FIGS. 8A-8B), LIC may be enabled in the motion compensation stage (e.g., standard motion compensation stage) and / or the OBMC stage. LIC may be invoked on one or more (e.g., multiple) occasions. For example, LIC may be invoked in the motion compensation stage and / or the OBMC stage. The invoked LIC may generate a prediction signal for the current block. A sub-block within the coding block may be assigned a motion vector (e.g., its own motion vector). LIC operations (e.g., LIC parameter estimation and / or LIC-based sample adjustment) may be performed on the motion vector. FIG. 7 shows an example associated with the approach described herein. If sub-blocks A, a, b, c, and d in FIG. 7 are bi-predicted, eight LIC operations may be performed in the OBMC stage to generate a prediction signal for sub-block A. Two of the eight LIC operations may be related to LIC-based motion compensation using motion vectors associated with neighboring sub-blocks. If K represents the number of sub-blocks within the current block, a total of 10×K LIC calls (e.g., 2×K LIC calls associated with standard motion compensation and 2×4×K LIC calls associated with OBMC) may be performed to generate a prediction signal for the current block.

[0073] LIC may be employed to compensate for illumination changes between one or more temporal reference pictures and the current picture (e.g., in a motion compensation stage). LIC may be applied based on a linear model. As described herein, when bi-prediction is applied, scaling factors and / or weights may be estimated for the prediction directions (e.g., estimated separately).

[0074] When performing LIC operations, one or more of the following may be applied: LIC parameters may be estimated (e.g., estimated once) for a bi-predicted video coding unit (e.g., a block or sub-block). For example, LIC parameters may be estimated by considering bi-predictive reference template samples of template samples associated with the current block or sub-block (e.g., by averaging the reference template samples). By considering bi-predictive reference template samples of template samples for the current block or sub-block, fewer LIC operations (e.g., one LIC parameter estimation and one LIC base sample adjustment) can be performed to generate a prediction signal for the current block / sub-block.

[0075] An optimization approach may be used to derive and / or adjust the LIC parameters: scaling factors and offsets associated with lists L0 and L1 may be jointly optimized (e.g., jointly adjusted).

[0076] Some examples are provided herein in the context of sub-block mode. For example, a coding block may be divided into one or more sub-blocks, and the sub-blocks may be assigned motion vectors. Those skilled in the art will recognize that the approaches described herein may be applied to video blocks coded in different coding modes (e.g., coding blocks that may not be divided and / or may not have a single motion vector).

[0077] The current block and / or the current sub-block may be bi-predicted. In an embodiment, if the current block or sub-block is bi-predicted, LIC parameters may be derived (e.g., derived separately) and applied to reference lists L0 and L1. LIC prediction signals associated with the reference lists may be averaged to generate a prediction signal for the current block or sub-block. In some embodiments described herein, the LMSE-based LIC derivation may be performed one or more times.

[0078] A bi-predictive reference template sample (e.g., a bi-predicted reference template sample) may be generated for a template sample associated with a current block or sub-block. The reference template sample may be identified, for example, based on one or more motion vectors associated with the current block or sub-block. For example, the reference template sample may be an adjacent temporal reference CU of the current CU or may correspond to a template sample for the current CU. The reference template samples may be jointly considered (e.g., averaged) in LIC parameter derivation. For example, identified reference template samples for a template sample may be averaged to generate a bi-predicted reference template sample. In an embodiment, an LMSE-based approach (e.g., LMSE estimation or calculation) may be applied to derive an LIC parameter that can be used to adjust a prediction signal for the current block or sub-block. For example, the LMSE-based approach may be performed to determine an LIC parameter such that a difference between the bi-predicted reference template sample and the template sample for the current CU can be minimized.

[0079] In an embodiment, the LIC parameter may be calculated based on using a linear model approach. For example, the linear model approach may use a bi-predicted reference template sample and a minimum value and / or a maximum value associated with the template sample. The minimum value and / or the maximum value of the template sample and the bi-predicted reference template sample may be used to determine the LIC parameter.

[0080] 9 is a diagram illustrating an example of the LIC operation described herein (e.g., for a bi-predicted or bi-predicted CU). T(x, y) may represent a template sample for a current block or sub-block at coordinates (x, y).

[0081]

number

[0082] and

[0083]

number

[0084] may each represent a reference template sample adjacent to the temporal reference CU (e.g., in each of the L0 and L1 lists) that corresponds to the template sample for the current CU. The reference template sample may be a bidirectional motion vector of the current block or sub-block.

[0085]

number

[0086] and

[0087]

number

[0088] A bi-predicted reference template sample of the template sample may be generated. For example, the bi-predicted reference template sample may be generated based on Equation (8) by averaging identified reference template samples of the template sample for the current CU (e.g., associated with adjacent temporal reference CUs such as L0 and L1 shown in FIG. 9).

[0089]

number

[0090] The LMSE-based approach may be used to derive LIC parameters, such as a scaling factor and / or offset associated with the LIC. The scaling factor and / or offset associated with the LIC may be derived using the LMSE-based approach by minimizing the difference between a template sample (e.g., for a current CU) and a corresponding bi-predicted reference template sample (e.g., a bi-directional reference sample), for example, as shown in equations (9) and (10).

[0091]

number

[0092]

number

[0093] The parameter N may represent the number of template samples associated with the current block / sub-block.

[0094] Those skilled in the art will recognize that the LMSE approach described herein may be an example of deriving LIC parameters. As such, one or more approaches, such as the linear model approach described herein, may be used to derive LIC parameters using bi-predicted reference template samples.

[0095] The derived LIC parameters may be applied to the bi-predicted reference template sample signals of the current block or sub-block based on a linear model, for example, as shown in equation (11).

[0096]

number

[0097] P(x,y) may be a prediction signal for the current block or sub-block.

[0098]

number

[0099] and

[0100]

number

[0101] may be two reference blocks and / or sub-blocks of the current block or sub-block associated with L0 and L1, respectively.

[0102] As described herein, one LIC parameter estimation and one LIC base sample adjustment may be performed to derive a prediction signal for a bi-predicted block or sub-block. Figures 10A-10B show examples of motion compensation operations after bi-directional LIC is applied as described herein.

[0103] The parameter K may represent the total number of sub-blocks in the current coding block. Separately estimating LIC parameters for one or more prediction directions, as shown in FIGS. 8A-8B, can result in approximately 10×K LIC operations to generate a prediction signal for the current block. Using the approach shown in FIGS. 10A-10B, approximately 5×K LIC operations may be performed to generate a prediction signal for the current block. The 5×K LIC operations may include, for example, approximately K LIC operations associated with standard motion compensation and approximately 4×K LIC operations associated with OBMC.

[0104] 8A-8B, LIC may be enabled in the OBMC stage. For example, LIC may be enabled when LIC is applied to a coding block. Enabling LIC in the OBMC stage may result in one or more (e.g., multiple) LIC invocations, and a prediction signal for the current block may be generated. When LIC is applied in a sub-block mode (e.g., ATMVP, STMVP, and / or FRUC) that allows a block to be divided into sub-blocks with multiple sub-blocks and motion vectors, the number of LIC invocations may be further increased, for example, because LIC parameter derivation may be frequently invoked during OBMC-based motion compensation using motion vectors associated with spatial neighbors of the current sub-block.

[0105] Reference template samples may be combined to derive LIC parameters in the OBMC stage. LIC calculations in the OBMC stage may be simplified. Reference template samples associated with template samples for the current block or sub-block may be combined (e.g., by averaging the reference template samples). A scaling factor and offset pair may be estimated. For example, the scaling factor and offset pair may be estimated by minimizing the difference between the template sample and the combined reference template sample. The scaling factor and offset pair may be used for OBMC-based motion compensation of the current block or sub-block. Using the example shown in FIG. 7, a combined prediction signal for a template sample of the current sub-block A may be generated by averaging corresponding reference template samples using motion vectors of four neighboring sub-blocks, e.g., a, b, c, and d. Averaging reference template samples using motion vectors of neighboring sub-blocks can be shown by Equation (12).

[0106]

number

[0107] sample

[0108]

number

[0109] ,

[0110]

number

[0111] ,

[0112]

number

[0113] , and

[0114]

number

[0115] may be reference template samples generated using motion vectors associated with each of the neighboring sub-blocks a, b, c, and d. The set of LIC parameters may be, for example, T(x,y) and T(x,y) based on equations (9) and (10). ave (x, y) may be estimated by applying an LMSE-based optimization to minimize the difference between (x, y). The derived values ​​of the LIC parameters, such as the scaling factor α and the offset β, may be used for OBMC-based motion compensation from neighboring sub-blocks a, b, c, and d (e.g., for one or more OBMC-based motion compensations from neighboring sub-blocks a, b, c, and d).

[0116] 11A-11B illustrate an example of a motion compensation operation when LIC parameters are derived (e.g., derived once) at the OBMC stage. Using the example approach described herein, the total number of LIC derivations can be reduced to 2×K (e.g., K derivations for standard motion compensation and K derivations for OBMC).

[0117] LIC parameter derivation and sample adjustment may be performed, for example, for standard motion compensation and / or OBMC. LIC parameters derived during standard motion compensation may be reused for OBMC. For example, LIC parameters may be derived and stored for one or more sub-blocks within a current block in the standard motion compensation stage. In the OBMC stage, the stored LIC parameters may be reused (e.g., fetched) for OBMC-based motion compensation of the current sub-block.

[0118] 12A-12B illustrate an example of a motion compensation operation during which LIC parameters derived from standard motion compensation for OBMC can be reused. If the LIC parameters are reused, the total number of LIC derivations can be reduced to approximately K (e.g., from 2×K for K derivations for standard motion compensation and K derivations for OBMC).

[0119] Using the example approach shown in FIGS. 12A-12B, the number of IC parameter derivations can be reduced. The total number of LIC-based sample adjustments (e.g., according to Equation (1)) that can be performed in the standard motion compensation stage and / or the OBMC stage may be similar (e.g., identical) to that in JEM. For example, approximately 5×K LIC-based sample adjustments (e.g., K LIC-based sample adjustments may be performed in the standard motion compensation stage and 4×K LIC-based sample adjustments may be performed in the OBMC stage) may be performed before a prediction signal for the current block is generated. When LIC parameters derived from the standard motion compensation stage are reused in the OBMC stage, on-chip memory (e.g., additional on-chip memory) may be used to store the LIC parameters (e.g., scaling factor α and offset β). For example, the maximum coding tree unit (CTU) size may be 128×128, and the sub-block size may be 4×4. The scaling factor α and offset β may each be expressed with 5 bits of precision and the bit depth of the input video. For 10-bit input video, the size of the on-chip memory for caching the LIC parameters may be equal to approximately 3 kilobytes (e.g., (128 / 4) x (128 / 4) x (1 byte + 2 bytes) = 32 x 32 x 3 bytes).

[0120] As discussed herein, the LIC-based sample adjustment and the calculation of the weighted average of multiple prediction samples in OBMC may involve linear operations. The impact of rounding errors (e.g., caused by converting floating-point operations to fixed-point operations) may be small. The coding performance before and after changing the dimensions of the LIC-based sample adjustment and BMC-based sample averaging may be similar to each other.

[0121] After OBMC-based sample averaging is performed, one or more LIC operations may be performed. Figures 13A-13B show examples of motion compensation operations using the approach described herein. As shown, standard motion compensation may be applied to generate a motion-compensated prediction signal for a sub-block within a CU. OBMC may be applied to a sub-block (e.g., a sub-block within a CU) by combining the sub-block's prediction signal with one or more prediction signals generated using the motion vectors of one or more spatial neighbors.

[0122] If LIC is enabled for a CU, LIC parameters may be derived (e.g., calculated) for one or more sub-blocks of the CU using the bidirectional LIC parameter derivation approach described herein. Predicted samples of the sub-blocks may be adjusted using the derived LIC parameters. For example, as shown in Figures 13A-13B, the total number of LIC-based operations for LIC-based parameter derivation and / or LIC-based sample adjustment can be reduced to K. Storage of LIC parameters may be omitted (e.g., because LIC is performed after OBMC) to save memory space (e.g., memory buffers) used to cache LIC parameters.

[0123] 13A-13B, LIC parameter derivation and / or LIC-based sample adjustment may be performed after OBMC of a sub-block is completed. Using this approach, storage of LIC parameters may be omitted.

[0124] LIC parameter derivation may be performed in the standard motion compensation stage. The derived LIC parameters may be used for sample adjustment. For example, the derived LIC parameters may be used for sample adjustment after OBMC of a sub-block is completed and an OBMC prediction signal (e.g., a combined OBMC prediction signal) is generated. LIC-based sample adjustment may be disabled in the standard motion compensation stage. If LIC-based sample adjustment is disabled in the standard motion compensation stage, the LIC parameters derived in the standard motion compensation stage may not be used to adjust prediction samples generated from the standard motion compensation stage.

[0125] 14A-14B illustrate an example of a motion compensation operation using the approach described herein (e.g., motion compensation by deriving LIC parameters in a standard motion compensation stage and performing LIC-based sample adjustment after OBMC). LIC parameter derivation and OBMC may be performed in parallel (e.g., LIC parameter derivation and OBMC for a sub-block may be performed simultaneously and / or in parallel). LIC-adjusted predicted samples can be obtained with reduced latency (e.g., minimum latency).

[0126] Although the bidirectional LIC parameter derivation approach is described herein as facilitating LIC simplification in the OBMC stage, it should be noted that such simplification may be achieved when the bidirectional LIC parameter derivation approach is disabled. The LIC simplification process described herein may be implemented in combination with other LIC schemes. For example, the LIC simplification process described herein may be combined with LIC design. LIC parameters may be derived (e.g., derived separately) and applied to prediction lists L0 and L1, for example, as shown in Figures 13A-13B and 14A-14B. The LIC-adjusted prediction signals may be averaged to generate a prediction signal for a subblock. In those examples (e.g., the examples shown in Figures 14A-14B), two different sets of LIC parameters (e.g., which may include scaling factors and offsets) may be maintained in the standard motion compensation stage for each of the prediction directions L0 and L1.

[0127] LIC operations may be disabled during the OBMC stage. For example, LIC may be skipped during OBMC-based motion compensation of one or more sub-blocks within the current block (e.g., using motion vectors associated with the sub-block's neighbors). In an embodiment, LIC may be disabled regardless of whether LIC is applied to the current block during standard motion compensation.

[0128] LIC operation(s) may be disabled for sub-block coding modes (e.g., ATMVP, STMVP, and / or FRUC, etc.). For block(s) coded by sub-block mode(s), the block(s) may be divided (e.g., further divided) into one or more (e.g., multiple) sub-blocks, each having a motion vector (e.g., a unique motion vector), to generate a fine-granularity motion field. As described herein, LIC operation may be disabled for ATMVP sub-block modes.

[0129] LIC parameters (e.g., LIC scaling factors and offsets) associated with reference lists L0 and L1 may be optimized. For example, the LIC parameters may be jointly optimized (e.g., jointly adjusted). The template sample for the current CU and the motion-compensated reference template sample (e.g., associated with reference lists L0 and L1) may be used as inputs to jointly optimize (e.g., jointly adjust) the scaling factors and offsets in L0 and L1. The distortion (e.g., difference) between the template sample and the combined reference template sample may be minimized. For example, based on the same notation used in equations (1) to (4), LMSE estimation may be performed as in equation (13).

[0130]

number

[0131] As shown, the parameter N may represent the number of template samples associated with the current block or sub-block.

[0132]

number

[0133] and

[0134]

number

[0135] is the template sample T(x i , y i ) may represent the reference template sample.

[0136]

number

[0137] Each set of may be determined for reference lists L0 and L1 so as to minimize the difference between the template samples for L0 and L1 and the combined reference template samples. Such difference may be calculated by subtracting the LIC-adjusted reference samples for both L0 and L1 from multiple (e.g., two) template samples.

[0138] The LIC technique may be simplified for bi-predictive CUs by GBi. When GBi / BPWA is enabled, a weight index may be signaled for bi-predictive CUs. GBi may be performed on top of LIC. For example, LIC may be applied to reference lists L0 and L1 for two prediction blocks (e.g., each of the two prediction blocks). Generalized bi-predictive weights provided for two illumination-compensated prediction blocks for bi-prediction (e.g., each of the two illumination-compensated prediction blocks for bi-prediction) may be used to combine prediction templates.

[0139] Generalized bi-prediction may be performed on template samples from two prediction lists to generate a GBi-based bi-predictive template reference sample. LIC may be estimated. For example, LIC parameters may be estimated using the GBi-based bi-predictive template reference sample and the template sample of the current CU. GBi and LIC may be performed on the bi-predictive signal of the current block / sub-block (e.g., GBi followed by LIC).

[0140] The extension of GBi to Equation (8) may be mathematically shown in Equation (14). In LIC, a bi-predictive signal of the template may be generated. For example, the bi-predictive signal of the template may be generated by weighted averaging two reference samples of the template in L0 and L1.

[0141]

number

[0142] T w (x,y) may represent the bi-prediction signal of the template.

[0143]

number

[0144] and

[0145]

number

[0146] is the bidirectional motion vector of the current block / sub-block

[0147]

number

[0148] and

[0149]

number

[0150] The weights (1-w) and w may represent GBi weights applied to the L0 and reference samples.

[0151] A linear model parameter derivation such as the LMSE-based procedure described herein may be used to calculate the values ​​of the scaling factors and offsets used for LIC, for example, by minimizing the difference between the template samples and the bidirectional reference samples given by equations (9) and (10), and T calculated using equation (14). w (x,y) is T avg May be used instead of (x,y).

[0152] The derived LIC parameters are, for example, according to Eq. (15):

[0153]

number

[0154] It may be applied to the bidirectional prediction signal of the current block / sub-block based on a linear model, and P(x,y) may represent the prediction signal of the current block / sub-block.

[0155]

number

[0156] and

[0157]

number

[0158] may denote two references for the current block / sub-block in L0 and L1, respectively. The parameters α0 and β0 may denote LIC gradient and offset parameters. The scaling factors and offsets in lists L0 and L1 in equation (13) may be jointly optimized.

[0159] 19A illustrates an exemplary communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple-access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communication system 100 may enable the multiple wireless users to access such content through sharing of system resources, including wireless bandwidth. For example, the communication system 100 may utilize one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tailed unique word DFT spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, and filter bank multicarrier (FBMC).

[0160] 19A, communications system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RANs 104 / 113, CNs 106 / 115, public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, although it will be recognized that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d, any of which may be referred to as a “station” and / or “STA,” may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspot or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearables, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain situations), consumer electronics devices, and devices operating on commercial and / or industrial wireless networks. Any of the WTRUs 102a, 102b, 102c, 102d may be referred to interchangeably as a UE.

[0161] The communications system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communications networks, such as the CN 106 / 115, the Internet 110, and / or other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a NodeB, an eNodeB, a Home NodeB, a Home eNodeB, a gNB, an NR NodeB, a site controller, an access point (AP), a wireless router, etc. Although the base stations 114a, 114b are each depicted as a single element, it will be understood that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0162] The base station 114a may be part of the RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station 114a and / or base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be in the licensed spectrum, the unlicensed spectrum, or a combination of the licensed and unlicensed spectrum. A cell may provide coverage for wireless services in a particular geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, one for each sector of the cell. In an embodiment, the base station 114a may utilize multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.

[0163] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over the air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0164] More specifically, as described above, the communication system 100 may be a multiple-access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, and SC-FDMA. For example, the base station 114a and the WTRUs 102a, 102b, and 102c in the RAN 104 / 113 may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 115 / 116 / 117 using Wideband CDMA (WCDMA). WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High Speed ​​Uplink (UL) Packet Access (HSUPA).

[0165] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE), and / or LTE Advanced (LTE-A), and / or LTE Advanced Pro (LTE-A Pro).

[0166] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as New Radio (NR) radio access, which may establish the air interface 116 using NR.

[0167] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement both LTE radio access and NR radio access, e.g., using a dual connectivity (DC) principle. Thus, the air interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0168] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement a wireless technology such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rates for GSM Evolution (EDGE), and GSM EDGE (GERAN).

[0169] 19A may be, for example, a wireless router, a Home NodeB, a Home eNodeB, or an access point and may utilize any suitable RAT to facilitate wireless connectivity in a localized area, such as a business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., used by drones), and a roadway. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In an embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a picocell or femtocell. 19A, the base station 114b may have a direct connection to the Internet 110. Therefore, the base station 114b may not need to access the Internet 110 via the CN 106 / 115.

[0170] The RAN 104 / 113 may communicate with the CN 106 / 115, which may be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have various quality of service (QoS) requirements, such as different throughput, delay, error resilience, reliability, data throughput, and mobility requirements. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions, such as user authentication. Although not shown in FIG. 19A , it will be understood that the RAN 104 / 113 and / or the CN 106 / 115 may communicate directly or indirectly with other RANs that utilize the same RAT as the RAN 104 / 113 or a different RAT. For example, in addition to being connected to the RAN 104 / 113, which may utilize NR radio technology, the CN 106 / 115 may also communicate with another RAN (not shown) that utilizes GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0171] The CN 106 / 115 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network providing plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) in the TCP / IP Internet protocol suite. The network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the network 112 may include another CN connected to one or more RANs that may utilize the same RAT as the RAN 104 / 113 or a different RAT.

[0172] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links.) For example, the WTRU 102c shown in Figure 19A may be configured to communicate with a base station 114a, which may employ a cellular-based wireless technology, and with a base station 114b, which may utilize IEEE 802.11 wireless technology.

[0173] 19B is a system diagram illustrating an example WTRU 102. As shown in FIG. 19B, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be understood that the WTRU 102 may include any subcombination of the above elements while remaining consistent with an embodiment.

[0174] The processor 118 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors in conjunction with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. While FIG. 19B depicts the processor 118 and the transceiver 120 as separate components, it will be understood that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0175] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) over the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In an embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF and light signals. It will be understood that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0176] 19B, the transmit / receive element 122 is depicted as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may utilize MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0177] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As mentioned above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as, for example, NR and IEEE 802.11.

[0178] The processor 118 of the WTRU 102 may be coupled to and may receive user input data from a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit). The processor 118 may output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. Additionally, the processor 118 may obtain information from and store data in any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. The non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, etc. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as located on a server or home computer (not shown).

[0179] The processor 118 may receive power from the power source 134 and may be configured to distribute and / or control the power to other components within the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0180] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or instead of, information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 116 from base stations (e.g., base stations 114a, 114b) and / or may determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may obtain location information using any suitable location determination method while remaining consistent with an embodiment.

[0181] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, and the like. The peripheral device 138 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor, a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0182] The WTRU 102 may include a full-duplex radio where transmission and reception of some or all of the signals associated with a particular subframe (e.g., for both the UL (e.g., for transmission) and the downlink (e.g., for reception)) may be parallel and / or simultaneous. The full-duplex radio may include an interference management unit 139 to reduce and / or substantially eliminate self-interference via hardware (e.g., a choke) or via signal processing via a processor (e.g., a separate processor (not shown) or the processor 118). In an embodiment, the WTRU 102 may include a half-duplex radio for transmission and reception of some or all of the signals (e.g., associated with a particular subframe for either the UL (e.g., for transmission) or the downlink (e.g., for reception)).

[0183] 19C is a system diagram illustrating the RAN 104 and the CN 106, according to an embodiment. As described above, the RAN 104 may employ E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also communicate with the CN 106.

[0184] The RAN 104 may include eNodeBs 160a, 160b, and 160c, although it will be understood that the RAN 104 may include any number of eNodeBs while remaining consistent with an embodiment. The eNodeBs 160a, 160b, and 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and 102c over the air interface 116. In one embodiment, the eNodeBs 160a, 160b, and 160c may implement MIMO technology. Thus, the eNodeB 160a, for example, may use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a.

[0185] Each of the eNodeBs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, etc. As shown in Figure 19C, the eNodeBs 160a, 160b, 160c may communicate with each other over an X2 interface.

[0186] 19C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. While each of the above elements is depicted as part of the CN 106, it will be understood that any of these elements may be owned and / or operated by an entity different from the CN operator.

[0187] The MME 162 may be connected to each of the eNodeBs 160a, 160b, 160c in the RAN 104 via an S1 interface and may act as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation / deactivation, selecting a particular serving gateway during initial attach of the WTRUs 102a, 102b, 102c, etc. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0188] The SGW 164 may be connected to each of the eNodeBs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring the user plane during inter-eNodeB handover, triggering paging when DL data is available to the WTRUs 102a, 102b, 102c, and managing and storing the context of the WTRUs 102a, 102b, 102c.

[0189] The SGW 164 may be connected to a PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0190] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional landline communication devices. For example, the CN 106 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0191] Although in Figures 19A-19D the WTRU is described as a wireless terminal, it is contemplated that in certain representative embodiments such a terminal may use a wired communication interface (e.g., temporary or permanent) with the communication network.

[0192] In an exemplary embodiment, the other network 112 may be a WLAN.

[0193] A WLAN in infrastructure basic service set (BSS) mode may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access to or interface with a distribution system (DS) or another type of wired / wireless network that carries traffic within and / or outside the BSS. Traffic originating from outside the BSS to a STA may arrive through the AP and be delivered to the STA. Traffic originating from a STA to a destination outside the BSS may be sent to the AP for delivery to the respective destination. Traffic between STAs within the BSS may be sent through the AP; for example, a source STA may send traffic to the AP, and the AP may deliver the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic may be sent (e.g., directly) between a source STA and a destination STA using direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and STAs within or using an IBSS (e.g., all of the STAs) may communicate directly with each other. IBSS mode communication may sometimes be referred to herein as "ad hoc" mode communication.

[0194] When using the 802.11ac infrastructure mode of operation or a similar mode of operation, an AP may transmit beacons on a fixed channel, such as a primary channel. The primary channel may be a fixed width (e.g., a 20 MHz wide bandwidth) or a width dynamically configured via signaling. The primary channel may be the operating channel of the BSS and may be used by STAs to establish a connection with the AP. In one representative embodiment, for example, in an 802.11 system, carrier sense multiple access with collision avoidance (CSMA / CA) may be implemented. With CSMA / CA, STAs (e.g., every STA), including the AP, may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, the particular STA may back off. Within a given BSS, one STA (e.g., only one station) may transmit at any given time.

[0195] High-throughput (HT) STAs may use 40 MHz wide channels for communication, for example, by combining a primary 20 MHz channel with adjacent or non-adjacent 20 MHz channels to form a 40 MHz wide channel.

[0196] A very high throughput (VHT) STA can support channels of 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz width. A 40 MHz and / or 80 MHz channel may be formed by combining contiguous 20 MHz channels. A 160 MHz channel may be formed by combining eight contiguous 20 MHz channels or two non-contiguous 80 MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, after channel encoding, the data may be passed through a segment parser that may split the data into two streams. Inverse fast Fourier transform (IFFT) processing and time-domain processing may be performed separately for each stream. The streams may be mapped onto two 80 MHz channels, and the data may be transmitted by the transmitting STA. At the receiver of the receiving STA, the operations described above for the 80+80 configuration may be reversed, and the combined data may be transmitted to the medium access control (MAC).

[0197] Sub-1 GHz mode operation is supported by 802.11af and 802.11ah. Channel operating bandwidths and carriers are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV White Space (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah may support meter-type control / machine-type communication, such as MTC devices in macro coverage areas. MTC devices may have limited functionality, including, for example, support for certain bandwidths and / or limited bandwidths (e.g., only support for those bandwidths). MTC devices may include batteries with above-threshold battery life (e.g., to maintain very long battery life).

[0198] WLAN systems capable of supporting multiple channels and channel bandwidths, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include a channel that may be designated as a primary channel. The primary channel may have a bandwidth equal to the largest common operating bandwidth supported by all STAs in a BSS. The bandwidth of the primary channel may be set and / or limited by a STA that supports the smallest bandwidth operating mode among all STAs operating in the BSS. In the example of 802.11ah, for a STA (e.g., an MTC-type device) that supports (e.g., only supports) the 1 MHz mode, the primary channel may be 1 MHz wide, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or network allocation vector (NAV) setting may depend on the status of the primary channel. For example, if the primary channel is busy because a STA (that only supports 1 MHz operating mode) is transmitting to the AP, the entire available frequency band may be considered busy, even though most of the frequency band may remain idle and available for use.

[0199] In the United States, the available frequency bands that may be used by 802.11ah are 902 MHz to 928 MHz. In South Korea, the available frequency bands are 917.5 MHz to 923.5 MHz. In Japan, the available frequency bands are 916.5 MHz to 927.5 MHz. The total bandwidth available for 802.11ah is 6 MHz to 26 MHz, depending on country regulations.

[0200] 19D is a system diagram illustrating the RAN 113 and the CN 115, according to an embodiment. As described above, the RAN 113 may communicate with the WTRUs 102a, 102b, and 102c over the air interface 116 using NR radio technology. The RAN 113 may also communicate with the CN 115.

[0201] The RAN 113 may include gNBs 180a, 180b, and 180c, although it will be understood that the RAN 113 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, and 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, and 180c may implement MIMO technology. For example, the gNB 180a, 180b may utilize beamforming to transmit signals to and / or receive signals from the gNBs 180a, 180b, and 180c. Thus, the gNB 180a may, for example, transmit wireless signals to and / or receive wireless signals from the WTRU 102a using multiple antennas. In an embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit multiple component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on an unlicensed spectrum, while the remaining component carriers may be on a licensed spectrum. In an embodiment, the gNBs 180a, 180b, 180c may implement coordinated multipoint (CoMP) technology. For example, the WTRU 102a may receive coordinated transmissions from the gNBs 180a and 180b (and / or 180c).

[0202] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with a scalable numerology. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of different or scalable lengths (e.g., including different numbers of OFDM symbols and / or lasting for different lengths of absolute time).

[0203] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without accessing another RAN (e.g., eNodeBs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may utilize one or more of the gNBs 180a, 180b, 180c as mobility anchor points. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in unlicensed bands. In a non-standalone configuration, the WTRUs 102a, 102b, 102c may communicate with / connect to a gNB 180a, 180b, 180c while also communicating with / connecting to another RAN, such as an eNodeB 160a, 160b, 160c. For example, the WTRUs 102a, 102b, 102c may implement the DC principle to communicate with one or more gNBs 180a, 180b, 180c and one or more eNodeBs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the eNodeBs 160a, 160b, 160c may act as mobility anchors for the WTRUs 102a, 102b, 102c, and the gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for serving the WTRUs 102a, 102b, 102c.

[0204] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, support for network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data to User Plane Functions (UPFs) 184a, 184b and routing of control plane information to Access and Mobility Management Functions (AMFs) 182a, 182b, etc. As shown in FIG. 19D, the gNBs 180a, 180b, 180c may communicate with each other over the Xn interface.

[0205] The CN 115 shown in Figure 19D may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. While each of the above elements is depicted as part of the CN 115, it will be understood that any of these elements may be owned and / or operated by an entity different from the CN operator.

[0206] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via an N2 interface and may act as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting a particular SMF 183a, 183b, managing registration areas, terminating NAS signaling, and mobility management. Network slicing may be used by the AMF 182a, 182b to customize CN support for the WTRUs 102a, 102b, 102c based on the type of service utilized by the WTRUs 102a, 102b, 102c. Different network slices may be established for different use cases, such as services relying on Ultra-Reliable Low-Latency (URLLC) access, services relying on eMBB access, and / or services for Machine-Type Communication (MTC) access, etc. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as WiFi.

[0207] The SMFs 183a and 183b may be connected to the AMFs 182a and 182b in the CN 115 via an N11 interface. The SMFs 183a and 183b may also be connected to the UPFs 184a and 184b in the CN 115 via an N4 interface. The SMFs 183a and 183b may select and control the UPFs 184a and 184b and configure the routing of traffic through the UPFs 184a and 184b. The SMFs 183a and 183b may perform other functions, such as managing and assigning UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notification. PDU session types may be IP-based, non-IP-based, Ethernet-based, etc.

[0208] The UPFs 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via an N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPFs 184a, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multihoming PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0209] The CN 115 may facilitate communication with other networks. For example, the CN 115 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 115 and the PSTN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to local data networks (DNs) 185a, 185b through the UPFs 184a, 184b via an N3 interface to the UPFs 184a, 184b and an N6 interface between the UPFs 184a, 184b and the DNs 185a, 185b.

[0210] 19A-19D and the corresponding description thereof, one or more or all of the functions described herein with respect to one or more of the WTRUs 102a-d, base stations 114a-b, eNodeBs 160a-c, MME 162, SGW 164, PGW 166, gNBs 180a-c, AMFs 182a-b, UPFs 184a-b, SMFs 183a-b, DNs 185a-b, and / or any other devices described herein may be performed by one or more emulation devices (not shown). The emulation devices may be one or more devices configured to emulate one or more or all of the functions described herein. For example, the emulation devices may be used to test other devices and / or to simulate network and / or WTRU functionality.

[0211] The emulation device may be designed to perform one or more tests of other devices in a laboratory environment and / or in an operator network environment. For example, one or more emulation devices may perform one or more or all functions while fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices in the communication network. One or more emulation devices may perform one or more or all functions while temporarily implemented / deployed as part of a wired and / or wireless communication network. The emulation device may be directly coupled to another device for testing purposes and / or may perform tests using over-the-air wireless communication.

[0212] The one or more emulation devices may perform one or more functions, including all functions, without being implemented / deployed as part of a wired and / or wireless communication network. For example, the emulation devices may be utilized in a test lab and / or in a test scenario in an undeployed (e.g., test) wired and / or wireless communication network to perform tests of one or more components. The one or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (which may include, for example, one or more antennas) may be used by the emulation devices to transmit and / or receive data.

[0213] While features and elements have been described above in particular combinations, those skilled in the art will recognize that each feature or element may be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware embodied in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor in conjunction with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer. [Industrial Applicability]

[0214] The present invention can be used in video coding systems. [Explanation of symbols]

[0215] 100 Communication Systems 102 WTRU 104 RAN 106 Core Network 108 PSTN 110 Internet 114 Base Station

Claims

1. receiving a current coding unit (CU) in a video bitstream; determining whether the current CU is bi-predicted; identifying a first set of reference template samples adjacent to a first temporal reference CU and a second set of reference template samples adjacent to a second temporal reference CU, wherein the first set of reference template samples and the second set of reference template samples correspond to sets of template samples adjacent to the current CU; generating a set of bi-predicted reference template samples based on the identified first set of reference template samples and the identified second set of reference template samples; calculating local illumination compensation (LIC) parameters based on the generated set of bi-predicted reference template samples and the set of template samples for the current CU; applying the calculated LIC parameters to generate adjusted samples within the current CU; 10. A method for encoding comprising:

2. The method of claim 1 , wherein the current CU comprises a current block or a current sub-block.

3. 2. The method of claim 1 , wherein generating the set of bi-predicted reference template samples comprises averaging a first set of the reference template samples adjacent to the first temporal reference CU and a second set of the reference template samples adjacent to the second temporal reference CU.

4. 2. The method of claim 1 , wherein the first set of reference template samples adjacent to the first temporal reference CU and the second set of reference template samples adjacent to the second temporal reference CU are identified based on a motion vector of the current CU.

5. 2. The method of claim 1 , wherein the LIC parameter is calculated based on minimizing a difference between the set of bi-predicted reference template samples and the set of template samples for the current CU.

Images