Method and apparatus for predictive refinement using decoder-side motion vector refinement based on optical flow

Abstract

Methods, apparatuses, systems, architectures, and interfaces are provided for improving sub-block (SB) level motion compensation prediction based on motion vector (MV) refinement. A decoding method includes: receiving a bitstream of encoded video data, the bitstream including at least one video data block, the at least one video data block including a plurality of SBs; performing MV derivation including a decoder-based MV (DMVR) process for at least one SB in the block to generate a refined MV for each SB; performing SB-based motion compensation for the at least one sub-block to generate an SB-based prediction within each SB; obtaining a spatial gradient of the prediction within each SB; determining an MV offset for each pixel in each SB; obtaining an intensity change in each SB based on the spatial gradient and the MV offset using an optical flow equation; and refining the prediction within each SB based on the obtained intensity change.

Description

Background Technology

[0001] Video decoding systems are widely used to compress digital video signals to reduce the storage requirements and / or transmission bandwidth of such signals. Among the various types of video decoding systems (such as block-based, wavelet-based and object-based systems), block-based hybrid video decoding systems are currently the most widely used and deployed. Examples of such block-based video decoding systems include international video decoding standards such as MPEG-1 / 2 / 4 Part 2, H.264 / MPEG-4 Part 10 AVC[1][2], VC-1[3] and the latest video decoding standard called High Efficiency Video Decoding (HEVC)[4], which was developed by ITU-T / SG16 / Q.6 / VCEG and ISO / IEC / MPEG's JCT-VC (Joint Collaboration on Video Decoding). Attached Figure Description

[0002] A more detailed understanding can be obtained from the following detailed description given by way of example in conjunction with the accompanying drawings. As with the detailed description, the figures in these drawings are illustrative. Therefore, the drawings and detailed description should not be considered limiting, and other equivalent examples are possible and solicited. Furthermore, the same reference numerals in the figures denote the same elements, wherein:

[0003] Figure 1 This is a block diagram illustrating an example video encoder in which one or more embodiments may be performed and / or implemented;

[0004] Figure 2 It shows the relationship with Figure 1 A block diagram of an example video decoder used in video encoding and / or decoding systems, wherein one or more embodiments may be performed and / or implemented;

[0005] Figure 3A This is a diagram illustrating the affine motion field of a block described by two control point motion vectors according to an embodiment;

[0006] Figure 3B This is a diagram illustrating the sub-block-level motion derivation for affine blocks in motion prediction based on a 4-parameter affine pattern according to an embodiment.

[0007] Figure 4 This is a diagram illustrating an exemplary model for motion prediction based on a 6-parameter affine pattern according to an embodiment;

[0008] Figure 5 This is a diagram illustrating the motion vector refinement on the decoding side according to an embodiment;

[0009] Figure 6This is a diagram illustrating the difference between pixel-level motion vectors and sub-block-level motion vectors according to an embodiment;

[0010] Figure 7 This is a diagram illustrating exemplary blocks in a decoding unit for the purpose of illustrating the embodiments;

[0011] Figure 8A This is a system diagram illustrating an example communication system that can implement one or more embodiments of the disclosure;

[0012] Figure 8B This illustrates a method according to one embodiment. Figure 8A The diagram shows an example wireless transmit / receive unit (WTRU) used internally in a communication system.

[0013] Figure 8C This illustrates a method according to one embodiment. Figure 8A The diagram shows an example radio access network (RAN) and an example core network (CN) used within the communication system.

[0014] Figure 8D This illustrates a method according to one embodiment. Figure 8A The system diagram shown illustrates another example of a RAN and another example of a CN used internally in the communication system.

[0015] Figure 9 This is a system diagram illustrating the implementation of various aspects and operations according to the various embodiments. Detailed Implementation

[0016] introduce

[0017] In July 2018, the Joint Video Experts Group (JVET) launched a new project to develop a new generation of video decoding standards, which was named Universal Video Decoding (VVC) [6]. In the same month, a reference software codebase called the VVC Test Model (VTM) [7] was created to demonstrate a reference implementation of the VVC standard. For the initial VTM-1.0, most of the decoding modules, including intra-frame prediction, inter-frame prediction, transform / inverse transform and quantization / dequantization, as well as in-loop filters, followed the existing HEVC design. A key exception was the use of a block partitioning structure based on a multi-type tree in the VTM. Meanwhile, another reference software library called the Benchmark Set (BMS) [8] was also created to facilitate the evaluation of new decoding tools. In this BMS codebase, a list of decoding tools inherited from JEM

[10] provides higher decoding efficiency and moderate implementation complexity. This list was included on top of the VTM and used as a benchmark when evaluating similar decoding techniques during the VVC standardization process. Specifically, BMS-1.0 integrates nine JEM decoding tools, including 65-angle intra-frame prediction direction, modified coefficient decoding, Advanced Multiple Transform (AMT) + 4×4 Inseparable Secondary Transform (NSST), Affine Motion Model, Generalized Adaptive Loop Filter (GALF), Advanced Temporal Motion Vector Prediction (ATMVP), Adaptive Motion Vector Precision, Decoder-Side Motion Vector Refinement (DMVR), and Linear Model (LM) chroma mode.

[0018] Similar to HEVC, the VVC is built on a block-based hybrid video decoding framework. Figure 1 This illustrates a block diagram of a general block-based hybrid video coding system. The input video signal 102 is processed block by block. In HEVC, an extended block size (called a "decoding unit" or CU) is used to efficiently compress high-resolution (e.g., 1080p and above) video signals. In HEVC, a CU can be up to 64x64 pixels. A CU can be further partitioned into prediction units or PUs, for which a separate prediction method can be applied. See [link / reference] Figure 1This paper presents a general block diagram of a block-based video encoder, which performs spatial prediction (160) and / or temporal prediction (162) for each input video block (which may be a master block (MB) or CU). Spatial prediction (or “intra-frame prediction”) uses pixels from decoded neighboring blocks in the same video picture / slice to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also known as “inter-frame prediction” or “motion-compensated prediction”) uses pixels from decoded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal for a given video block is typically sent by one or more motion vectors indicating the amount and direction of motion between the current block and its reference blocks. In addition, if multiple reference pictures are supported (as is the case for many recent video decoding standards such as H.264 / AVC or HEVC), an additional reference picture index is sent for each video block; and this reference index is used to identify which reference picture in the reference picture repository (164) the temporal prediction signal is based on. Following spatial and / or temporal predictions, the mode decision box (180) in the encoder selects the optimal prediction mode, for example, based on a rate distortion optimization method. The prediction block (116) from spatial prediction 160 or temporal prediction 162 is then subtracted from the current video block; and the prediction residual is decorrelated and quantized (106) using a transform (104) to achieve the target bit rate. The quantized residual coefficients are inversely quantized (110) and inversely transformed (112) to form a reconstructed residual, which is then added back to the prediction block at 126 to form the reconstructed video block. Further in-loop filtering (166), such as a deblocking filter and an adaptive loop filter, may be applied to the reconstructed video block before it is placed in a reference image repository (164) for decoding future video blocks. In order to form the output video bitstream 120, the decoding mode (inter-frame or intra-frame), prediction mode information, motion information and quantized residual coefficients are sent to the entropy decoding unit (108) for further compression and packing to form the bitstream.

[0019] Figure 2A general block diagram of a block-based video decoder is given. The video bitstream 202 is first unpacked and entropy-decoded in entropy decoding unit 208. The decoding mode and prediction information are sent to a spatial prediction unit 260 (for intra-frame decoding mode) and / or a temporal prediction unit 262 (for inter-frame decoding mode) for their appropriate purposes to form prediction blocks. Residual transform coefficients are sent to inverse quantization unit 210 and inverse transform unit 212 to reconstruct the residual blocks. The prediction blocks (from spatial prediction 260 if it is intra-frame decoding mode, or from temporal prediction 262 if it is inter-frame decoding mode) are then added to the residual blocks at 226. The reconstructed blocks may be further processed by in-loop filtering 266 before being stored in a reference picture repository 264. In addition to being stored in the reference picture repository for predicting future video blocks, the reconstructed video 220 is also sent to drive a display device.

[0020] In modern video codecs, bidirectional motion-compensated prediction (MCP) is known for its high efficiency in removing temporal redundancy by utilizing the temporal correlation between images, and has been widely adopted in most existing video codecs [2][9]

[10] [7]. Moreover, a dual prediction signal is simply formed by combining two single prediction signals using a weight value equal to 0.5. However, combining the single prediction signals in this simple way is not optimal, especially under conditions where the brightness changes rapidly from one reference image to another. Therefore, several prediction techniques have been developed with the aim of compensating for brightness changes over time by applying some global or local weights and offset values ​​to each sample value in the reference image. Some of these techniques are discussed below.

[0021] Affine mode

[0022] In HEVC, only the translational motion model is applied to motion compensation prediction, while in the real world, there are many types of motion, such as zooming in / out, rotation, perspective motion, and other irregular motion. In VTM-2.0[7], affine motion compensation prediction is applied. The affine motion model is either 4-parameter or 6-parameter. A first flag is sent to each CU after inter-frame decoding to indicate whether the translational motion model or the affine motion model is applied to the inter-frame prediction. If it is an affine motion model, a second flag is sent to indicate whether it is a 4-parameter or 6-parameter model.

[0023] A four-parameter affine motion model has the following parameters: two parameters for translational motion in the horizontal and vertical directions, one parameter for scaling motion in both directions, and one parameter for rotational motion in both directions. The horizontal scaling parameter is equal to the vertical scaling parameter. The horizontal rotation parameter is equal to the vertical rotation parameter. This four-parameter affine motion model is decoded in the VTM using two motion vectors at two control point positions defined at the top left and top right corners of the current CU. Figure 3A As shown, the affine motion field of the block is described by two control point motion vectors (V0, V1). Based on the control point motion, the motion field (v0, V1) of an affine decoded block is... x ,v y It is described as:

[0024]

[0025] Where (v 0x ,v 0y (v) is the motion vector of the top left control point. 1x ,v 1y ) represents the motion vector of the upper right control point, such as Figure 3A As shown, w is the width of the CU. In VTM-2.0, the motion field of the affine decoded CU is derived at the 4×4 block level; that is, for each of the 4×4 blocks within the current CU, (v... x ,v y And apply it to the corresponding 4×4 block. Figure 3B The motion derivation of affine blocks in 4-parameter affine motion prediction at (e.g.) 4×4 block level is shown.

[0026] Iteratively estimate the four parameters of the 4-parameter affine model. Represent the motion vector (MV) pair for step k as follows: The original luminance signal is represented as I(i,j), and the predicted luminance signal is represented as I′. k (i,j). Spatial gradient g x (i, j) and g y (i, j) are respectively applied to the prediction signal I′ in the horizontal and vertical directions. k The Sobel filter

[13] for (i,j) is derived. The derivative of Eq(1) can be expressed as:

[0027]

[0028] Where (a, b) are the incremental translation parameters, and (c, d) are the incremental scaling and rotation parameters for step k. The incremental MV at the control point can be derived using Eq(3) and (4) from its coordinates. For example, (0, 0) and (w, 0) are the coordinates of the top left and top right control points, respectively.

[0029]

[0030]

[0031] Based on the optical flow equation, the relationship between brightness variation and spatial gradient and temporal shift is formulated as follows:

[0032]

[0033] Replace with Eq(2) and We obtain the equation for the parameters (a, b, c, d).

[0034]

[0035] Since all samples in CU satisfy Eq(6), the parameter set (a, b, c, d) can be solved using the least squares error method. The MV at the two control points at step (k+1) can be solved using Eq(3) and (4). Furthermore, they can be rounded to a specific precision (i.e., 1 / 4 pixel). Using iteration, the MV at the two control points can be refined until convergence occurs when all parameters (a, b, c, d) are zero or the iteration time meets a predetermined limit.

[0036] A 6-parameter affine motion model has the following parameters: two parameters for translational motion in the horizontal and vertical directions, one parameter for scaling motion, one parameter for rotational motion in the horizontal direction, and one parameter for scaling motion and rotational motion in the vertical direction. This 6-parameter affine motion model is decoded using three MVs at three control points. For example... Figure 4 As shown, the three control points of the 6-parameter affine decoding CU are defined at the top left, top right, and bottom left corners of the CU. The motion at the top left control point is related to translational motion, the motion at the top right control point is related to rotation and scaling motion in the horizontal direction, and the motion at the bottom left control point is related to rotation and scaling motion in the vertical direction. For the 6-parameter affine motion model, the rotation and scaling motion in the horizontal direction can differ from those in the vertical direction. The motion vector (vx, v) of each sub-block... y The result is derived using three MVs at the control points, as follows:

[0037]

[0038] Where (v 2x ,v 2y (x, y) is the motion vector of the lower left control point, (x, y) is the center position of the sub-block, and w and h are the width and height of the CU.

[0039] The six parameters of the 6-parameter affine model are estimated in a manner similar to that described above in conjunction with the 4-parameter affine motion model. Eq(2) is modified as follows:

[0040]

[0041] In step k, (a, b) are the incremental translation parameters, (c, d) are the incremental scaling and rotation parameters in the horizontal direction, and (e, f) are the incremental scaling and rotation parameters in the vertical direction. Equation (8) is also changed accordingly, as shown in Eq (9) below.

[0042]

[0043] By considering all samples within CU, the parameter set (a, b, c, d, e, f) can be solved using the least squares method. The MV of the upper left control point is calculated using Eq(3). Calculate the MV of the upper right control point using Eq(10) below. Calculate the MV of the upper right control point using Eq 11.

[0044]

[0045]

[0046] Decoder-side motion vector refinement (DMVR)

[0047] To increase the accuracy of the MV in the merged mode, a decoder-side motion vector thinning based on bidirectional matching (BM) is applied in VTM4

[11] . In the bidirectional prediction operation, a thinned MV is searched around the initial MV in the reference image list L0 and the reference image list L1. The BM method calculates the distortion between two candidate blocks in the reference image list L0 and list L1. Figure 9 As shown, the sum of absolute differences (SAD) between red blocks is calculated based on each MV candidate around the initial MV. The MV candidate with the lowest SAD becomes the refined MV and is used to generate the bidirectional prediction signal.

[0048] The refined motion vector (MV) derived from the DMVR is used to generate inter-frame prediction samples and also for predicting temporal motion vectors in future image decoding. The original MV is used during deblocking and also for predicting spatial motion vectors in future CU decoding.

[0049] like Figure 5 As shown, the search points around the initial MV and MV offset follow the MV difference mirror (i.e., symmetric) rule. In other words, any point checked by DMVR (represented by candidate MV pairs (MV0, MV1)) follows the following two equations:

[0050]

[0051] MV offset This represents the thinning offset between the initial MV and the thinned MV in one of the reference images. In VTM4, the thinning search range is two integer luminance samples from the initial MV. To reduce the complexity of the search, a fast search method with an early termination mechanism can be applied.

[0052] Predictive refinement (PROF) using optical flow for affine modes

[0053] To achieve finer granularity of motion compensation,

[12] proposed a method for refining sub-block-based affine motion compensation prediction using optical flow, which is called Prediction Refinement Using Optical Flow (PROF).

[0054] After performing sub-block-based affine motion compensation, the brightness prediction samples are refined by adding differences derived from the optical flow equation. The proposed PROF is described in the following four steps.

[0055] Step 1: Perform the sub-block-based affine motion compensation to generate sub-block prediction I(i,j).

[0056] Step 2: Using a 3-tap filter [-1, 0, 1] at each sample location, calculate the spatial gradient g of the sub-block prediction. x (i,j) and g y (i,j).

[0057] g x (i,j)=I(i+1,j)-I(i-1,j)

[0058] g y (i,j)=I(i,j+1)-I(i,j-1)

[0059] The sub-block prediction is extended by one pixel on each side for gradient calculation. To reduce memory bandwidth and complexity, pixels on the extended boundary are copied from the nearest integer pixel position in the reference image. Therefore, additional interpolation for filling regions is avoided.

[0060] Step 3: Brightness prediction refinement is calculated using the following optical flow equation.

[0061] ΔI(i,j)=g x (i,j)*Δv x (i,j)+g y (i,j)*Δv y (i,j) (13)

[0062] Where Δvx (i,j) is the difference between the pixel MV (denoted as v(i,j)) calculated for the sample pixel position (i,j) in the horizontal direction and the sub-block MV of the sub-block to which pixel (i,j) belongs, and Δv y (i,j) is the difference between the pixel MV (represented as v(i,j)) calculated for the sample pixel position (i,j) in the vertical direction and the sub-block MV of the sub-block to which pixel (i,j) belongs.

[0063] Figure 6 The help demonstrates how the MV of each individual pixel can differ from the MV of the sub-blocks.

[0064] Since the affine model parameters and the pixel position relative to the sub-block center do not change between sub-blocks, Δv(i,j) can be calculated for the first sub-block and reused for other sub-blocks in the same CU. Assuming x and y are the horizontal and vertical offsets from the pixel position to the sub-block center, Δv(x,y) can be derived by the following equation:

[0065]

[0066] For a 4-parameter affine model

[0067]

[0068] For a 6-parameter affine model

[0069]

[0070] Among them, (v 0x ,v 0y ),(v 1x ,v 1y ),(v 2x ,v 2y ) are the motion vectors of the upper left, upper right, and lower left control points, and w and h are the width and height of the CU.

[0071] Step 4: Finally, refine the brightness prediction and add it to the sub-block prediction I(i,j). Generate the final prediction I' using the following equation.

[0072] I'(i,j)=I(i,j)+ΔI(i,j) (14)

[0073] Predictive thinning based on motion vector thinning using optical flow

[0074] The following is an improved method for sub-block-level motion compensation prediction based on motion vector refinement. After deriving the refined motion vector (RMV) through the DMVR process and performing sub-block-based motion compensation based on the RMV, pixel intensity is refined by adding the difference derived from the optical flow equation (Eq(13)), which is referred to here as motion vector-based prediction refinement using optical flow (MVRPROF). This MVRPROF technique can achieve pixel-level granularity without significantly increasing complexity, and also maintains a worst-case memory access bandwidth comparable to that of sub-block-level motion compensation based on conventional DMVR.

[0075] In this invention, MVPROF is proposed to improve the granularity of DMVR-based sub-block-level motion compensation prediction by applying pixel intensity variations derived from the optical flow equation. This method requires only one motion compensation operation per sub-block, similar to existing affine motion compensation in VVC. According to an exemplary embodiment, in a first step, a refined motion vector for each sub-block in the CU is derived by performing a DMVR procedure (e.g., as described in section 1.2 above). This is followed by sub-block-based motion compensation, such as that described in section 0 above, to generate sub-block-based predictions.

[0076] In the second step, the spatial gradient g of the sub-block prediction at each sample location is calculated by following the process described in Part 0. x (i,j) and g y (i,j).

[0077] In the third step, the motion vector offset Δv(i,j) is calculated at each sample location by following the process described below.

[0078] like Figure 7 As shown, multiple motion vector offsets exist within the current CU. For each sub-block within the CU (e.g., Figure 7 The current sub-block (excluding the sub-block located at the left and / or top boundary of the current CU) has three adjacent sub-blocks whose MV offset values ​​are available after DMVR processing. Consider as follows: Figure 7 The motion of each of the four sub-blocks, represented by its four MV offsets, can be described by a six-parameter motion model as follows:

[0079]

[0080] Among them, MV X_offset and MV Y_offset It is a horizontal and vertical component of an MV offset, X sub_block and Y sub_blockThese are the X and Y coordinates of the center position of the corresponding sub-block. Based on the known center position and MV offset of each of the four adjacent sub-blocks (i.e., the current sub-block and the three aforementioned adjacent sub-blocks), the parameter a can be calculated for all four sub-blocks (a total of 8 equations) by performing linear regression modeling on Eq(15). xx ,a xy ,a yx ,a yy ,b x and b y .

[0081] In one embodiment, the center positions and MV offsets of the three adjacent sub-blocks can be used to calculate the six parameters, while the fourth (current) sub-block can be used to estimate and evaluate the model error. In one embodiment, if the model error is greater than a predefined threshold, MVPROF may not be applied to the current sub-block (i.e., the fourth sub-block).

[0082] In another embodiment, all four sub-blocks can be used to calculate the six parameters.

[0083] Given the affine motion model defined in Eq(15) and the model parameters calculated as described above, the motion vector offset Δv(i,j) at each sample position belonging to a sub-block can be calculated as follows:

[0084]

[0085] Where i and j are the X and Y coordinates of each sample / pixel, respectively.

[0086] In some cases, a 4-parameter affine model may better fit the motion than a 6-parameter affine model. Therefore, if other offsets are available, then it is possible to achieve this by using only the MV offsets from the two sub-blocks (e.g., Figure 7 The four available MV offsets (any two) and the remaining offsets can be used to derive a 4-parameter affine model for verification / evaluation purposes. In the 4-parameter affine model:

[0087]

[0088] For some sub-blocks, such as those located at the left or top boundary of the current CU, where there may be only one available neighboring sub-block, the motion vector offset at each sample location can be derived using a 4-parameter affine model as defined above.

[0089] In the fourth step, the brightness intensity change of each pixel in sub-block n of CU is calculated by the optical flow equation:

[0090] ΔI(i,j)=g x (i,j)*Δvx (i,j)+g y (i,j)*Δv y (i,j) (18)

[0091] Here, Δv(i,j) and g(i,j) are the MV offset and spatial gradient at each sample position (i,j), respectively, which have been calculated in the previous steps.

[0092] Finally, the predictions for each list of reference images can be refined by adding the aforementioned brightness intensity variations. The final prediction I' is generated by the following equation.

[0093] I'(i,j)=I(i,j)+ΔI(i,j) (19)

[0094] Many implementation variations of the proposed MVRPROF method are possible, including the following.

[0095] In an alternative embodiment, when the model parameter a is derived from Eq(15) xx ,a xy ,a yx ,a yy ,b x and b y When considering MV offset, it is not limited to Figure 7 The diagram shows only four sub-blocks, considering only the top-left, top-left, and top-upper neighboring sub-blocks. More candidate neighboring sub-blocks not too far from the current sub-block can also be considered. For example, MVPROF can use the MV offsets of all sub-blocks to derive an affine motion model for the CU, and then apply this affine model to derive the sample-based MV offset for each sample.

[0096] Alternatively, when deriving the model parameters from Eq(15) and evaluating model error, a configurable or adaptive error threshold can be set to evaluate model accuracy. In one example, multiple models, such as 6-parameter and 4-parameter models, can be evaluated, and the model with the lowest L1 or L2 error can be selected to derive Δv(i,j) at each sample location, where L1 is the weighted sum of absolute differences (SAD) and L2 is the weighted sum of squared differences (SSD).

[0097] In one exemplary implementation, when estimating model error, weighted L1 or L2 differences can be considered (e.g., the MV offsets of different neighboring sub-blocks can have different weights).

[0098] In another exemplary implementation, if only four MV offsets are used to derive the motion model, the model error can be evaluated before deriving the model to reduce computational complexity (because if the model error is large, motion model estimation can be skipped). For example, the sum of the two MV offsets in the two diagonal directions can be calculated first, and the difference between the two sums can be used for the model error. For example, in Figure 7 In the middle, MV can be calculated separately. off (x-1, y-1) and MV off The sum of (x,y) and MV off (x, y-1) and MV off The sum of (x-1, y). The difference between these two sums can be used to represent the model error.

[0099] In yet another exemplary embodiment, when the model parameters are calculated using multiple adjacent MV offsets, the aspect ratio of the sub-block can remain the same as that of the current sub-block. Alternatively, different aspect ratios can be allowed.

[0100] In a further variation, when multiple adjacent MV offsets are used to calculate the model parameters, the size of adjacent sub-blocks can be the same as the current sub-block. Alternatively, adjacent sub-blocks of different sizes can be allowed.

[0101] Furthermore, in the exemplary embodiments described above, the MVPPROF operates such that each sub-block within the current CU independently derives model parameters, and each sample within the same sub-block shares the same model parameters, but each sample from different sub-blocks within the current CU may not share the model parameters. However, in another embodiment, the model parameters may be derived only once, and then all samples within the current CU share the same parameters.

[0102] Furthermore, in a variant of Eq(19), the intensity difference derived from Eq(18) can be multiplied by a weighting factor w before being added to the prediction, as shown in the equation below:

[0103] I'(i,j)=I(i,j)+w·ΔI(i,j) (20)

[0104] Here, w is set to a value between 0 and 1, inclusive. The value w can be signaled at the CU level or the image level. For example, w can be signaled using a weight index.

[0105] The MVRPROF technique can be used after the L0 and L1 predictions based on DMVR are combined by weights. Alternatively, the MVRPROF technique can be applied to a single prediction, such as L0 or L1. For example, in one embodiment, the MVRPROF technique can be applied to the one of L0 or L1 that is closer to the current image in the temporal domain.

[0106] Example network for implementation of the embodiments

[0107] Figure 8A This is an illustration of an exemplary communication system 100 that can implement one or more of the disclosed embodiments. The communication system 100 can be a multiple access system providing voice, data, video, messaging, broadcasting, and other content to multiple wireless users. The communication system 100 enables multiple wireless users to access such content by sharing system resources, including wireless bandwidth. For example, the communication system 100 can use one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Extended OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtering OFDM, and Filter Bank Multicarrier (FBMC), etc.

[0108] like Figure 8A As shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110, and other networks 112. However, it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network components. Each WTRU 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. For example, any WTRU 102a, 102b, 102c, or 102d may be referred to as a “station” and / or “STA”, and may be configured to transmit and / or receive wireless signals. It may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, and devices operating on commercial and / or industrial wireless networks, etc. Any of WTRU 102a, 102b, 102c, or 102d may be interchangeably referred to as a UE.

[0109] The communication system 100 may also include base stations 114a and / or 114b. Each base station 114a and / or base station 114b may be any type of device configured to enable its access to one or more communication networks (e.g., CN106 / 115, Internet 110, and / or other networks 112) by wirelessly interfacing with at least one of WTRUs 102a, 102b, 102c, 102d. For example, base stations 114a and 114b may be base transceiver stations (BTS), node B, e-node B, home node B, home e-node B, gNB, NR node B, site controller, access point (AP), and wireless routers, etc. Although each base station 114a and 114b is described as a single component, it should be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network components.

[0110] Base station 114a may be part of RAN 104 / 113, and the RAN may also include other base stations and / or network components (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies called cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide radio service coverage for a specific geographic area that is relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, a cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, i.e., each transceiver corresponds to one sector of the cell. In one embodiment, base station 114a may use multiple-input multiple-output (MIMO) technology and may use multiple transceivers for each sector of the cell. For example, by using beamforming, signals can be transmitted and / or received in a desired spatial direction.

[0111] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116, wherein the air interface can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, millimeter wave, infrared (IR), ultraviolet (UV), visible light, etc.). Air interface 116 can be established using any suitable radio access technology (RAT).

[0112] More specifically, as described above, the communication system 100 can be a multiple access system and can use one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, and SC-FDMA, etc. For example, base station 114a in RAN 104 / 113 and WTRUs 102a, 102b, and 102c can implement a certain radio technology, such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), wherein the technology can use Wideband CDMA (WCDMA) to establish air interfaces 115 / 116 / 117. WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0113] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c may use some kind of radio technology, such as evolved UMTS terrestrial radio access (E-UTRA), wherein the technology may use Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTA Pro (LTE-A Pro) to establish air interface 116.

[0114] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c may implement a certain radio technology, such as NR radio access, wherein the radio technology can establish an air interface 116 using a novel radio (NR).

[0115] In one embodiment, base station 114a and WTRUs 102a, 102b, and 102c can implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can jointly implement LTE radio access and NR radio access (e.g., using the dual connectivity (DC) principle). Thus, the air interface used by WTRUs 102a, 102b, and 102c can be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0116] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c may implement the following radio technologies, such as IEEE 802.11 (i.e., WiFi), IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rate for GSM Evolution (EDGE), and GSM EDGE (GERAN), etc.

[0117] Figure 8A Base station 114b can be a wireless router, home node B, home e node B, or access point, and can use any suitable RAT to facilitate wireless connectivity in a local area, such as a business premises, residence, vehicle, campus, industrial facility, air corridor (e.g., for use by drones), and road, etc. In one embodiment, base station 114b and WTRUs 102c, 102d can establish a wireless local area network (WLAN) by implementing a radio technology such as IEEE 802.11. In one embodiment, base station 114b and WTRUs 102c, 102d can establish a wireless personal area network (WPAN) by implementing a radio technology such as IEEE 802.15. In yet another embodiment, base station 114b and WTRUs 102c, 102d can establish a picocell or femtocell by using a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.). Figure 8A As shown, base station 114b can be directly connected to the Internet 110. Therefore, base station 114b does not need to access the Internet 110 via CN106 / 115.

[0118] RAN 104 / 113 can communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more WTRUs 102a, 102b, 102c, 102d. This data can have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, and mobility requirements, etc. CN 106 / 115 can provide call control, billing services, location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or can perform advanced security functions such as user authentication. Although in Figure 8AWhile not shown, it should be understood that RAN 104 / 113 and / or CN 106 / 115 can communicate directly or indirectly with other RANs that use the same RAT or a different RAT as RAN 104 / 113. For example, in addition to connecting to RAN 104 / 113 which uses NR radio technology, CN 106 / 115 can also communicate with other RANs (not shown) that use GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technologies.

[0119] CN 106 / 115 may also act as a gateway for WTRU 102a, 102b, 102c, 102d to access PSTN 108, the Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Simple Old-Style Telephone Service (POTS). The Internet 110 may include a global network of interconnected computer equipment systems using common communication protocols (e.g., Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite). Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, wherein the one or more RANs may use the same RAT or a different RAT as RAN 104 / 113.

[0120] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multi-mode capability (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers communicating with different wireless networks on different wireless links). For example... Figure 8A The WTRU 102c shown can be configured to communicate with base station 114a using cellular-based radio technology, and with base station 114b using IEEE 802 radio technology.

[0121] Figure 8B This is a system diagram illustrating an example of WTRU 102. (See diagram below.) Figure 8B As shown, WTRU 102 may include a processor 118, a transceiver 120, a transmitter / receiver unit 122, a speaker / microphone 124, a numeric keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and other peripheral devices 138. It should be understood that, while remaining consistent with the embodiments, WTRU 102 may also include any sub-combination of the foregoing components.

[0122] Processor 118 can be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and a state machine, etc. Processor 118 can perform signal encoding, data processing, power control, input / output processing, and / or any other function that enables WTRU 102 to operate in a wireless environment. Processor 118 can be coupled to transceiver 120, and transceiver 120 can be coupled to transmitting / receiving unit 122. Although Figure 8B While the processor 118 and transceiver 120 are described as separate components, it should be understood that the processor 118 and transceiver 120 can also be integrated into a single electronic component or chip.

[0123] Transmit / receive component 122 may be configured to transmit or receive signals to or from a base station (e.g., base station 114a) via air interface 116. For example, in one embodiment, transmit / receive component 122 may be an antenna configured to transmit and / or receive RF signals. As an example, in another embodiment, transmit / receive component 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals. In yet another embodiment, transmit / receive component 122 may be configured to transmit and / or receive RF and optical signals. It should be understood that transmit / receive component 122 may be configured to transmit and / or receive any combination of wireless signals.

[0124] Although Figure 8B While the transmit / receive component 122 is described as a single component, the WTRU 102 may include any number of transmit / receive components 122. More specifically, the WTRU 102 may use MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive components 122 (e.g., multiple antennas) that transmit and receive radio signals via the air interface 116.

[0125] Transceiver 120 can be configured to modulate signals to be transmitted by transmitter / receiver 122 and demodulate signals received by transmitter / receiver 122. As described above, WTRU 102 can have multimode capability. Therefore, transceiver 120 can include multiple transceivers that allow WTRU 102 to communicate using various RATs (e.g., NR and IEEE 802.11).

[0126] The processor 118 of WTRU 102 can be coupled to a speaker / microphone 124, a numeric keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit), and can receive user input data from these components. The processor 118 can also output user data to the speaker / microphone 124, the numeric keypad 126, and / or the display / touchpad 128. Furthermore, the processor 118 can access and store information from any suitable memory, such as non-removable memory 130 and / or removable memory 132. Non-removable memory 130 can include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 can include a subscriber identity module (SIM) card, a memory stick, a secure digital card (SD) memory card, etc. In other embodiments, processor 118 may access information from and store data in memories that are not actually located in WTRU 102, such as those memories located in a server or home computer (not shown).

[0127] The processor 118 can receive power from the power supply 134 and can be configured to distribute and / or control power for other components in the WTRU 102. The power supply 134 can be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell battery packs (such as nickel-cadmium (Ni-Cd), nickel-zinc (Ni-Zn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, and fuel cells, etc.

[0128] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) related to the current location of the WTRU 102. As a supplement or replacement to the information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via the air interface 116, and / or determine its location based on signal timing received from two or more nearby base stations. It should be understood that, while remaining consistent with the embodiments, the WTRU 102 may acquire location information using any suitable positioning method.

[0129] The processor 118 can also be coupled to other peripheral devices 138, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, peripheral devices 138 may include accelerometers, electronic compasses, satellite transceivers, digital cameras (for photos and / or video), Universal Serial Bus (USB) ports, vibration devices, television transceivers, hands-free headsets, etc. Modules, FM radio units, digital music players, media players, video game console modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, and activity trackers, etc. Peripheral devices 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors, geolocation sensors, altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0130] WTRU 102 may include a full-duplex wireless device, wherein the reception or transmission of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous for the wireless device. The full-duplex wireless device may include an interference management unit that reduces and / or substantially eliminates self-interference by means of hardware (e.g., choke coils) or by means of a processor (e.g., a separate processor (not shown) or by means of processor 118) for signal processing. In one embodiment, WTRU 102 may include a half-duplex wireless device that transmits or receives some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)).

[0131] Figure 8C This is a system diagram illustrating RAN 104 and CN 106 according to one embodiment. As described above, RAN 104 can communicate with WTRUs 102a, 102b, and 102c using E-UTRA radio technology on air interface 116. RAN 104 can also communicate with CN 106.

[0132] RAN 104 may include eNodeBs 160a, 160b, and 160c; however, it should be understood that RAN 104 may include any number of eNodeBs while remaining consistent with the embodiments. Each eNodeB 160a, 160b, and 160c may include one or more transceivers communicating with WTRUs 102a, 102b, and 102c on air interface 116. In one embodiment, eNodeBs 160a, 160b, and 160c may implement MIMO technology. Thus, for example, eNodeB 140a may use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a.

[0133] Each eNodeB 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, etc. For example... Figure 8C As shown, nodes B160a, 160b, and 160c can communicate with each other via the X2 interface.

[0134] Figure 8C The CN 106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing components is described as part of the CN 106, it should be understood that any of these components may be owned and / or operated by an entity other than the CN operator.

[0135] MME 162 can connect to each eNode B160a, 160b, and 160c in RAN 104 via the S1 interface and can act as a control node. For example, MME 162 can be responsible for authenticating users of WTRUs 102a, 102b, and 102c, performing bearer activation / deactivation processes, and selecting a specific serving gateway during the initial attach process of WTRUs 102a, 102b, and 102c, etc. MME 162 can also provide control plane functionality for handover between RAN 104 and other RANs (not shown) using other radio technologies (e.g., GSM and / or WCDMA).

[0136] The SGW 164 can connect to each eNodeB 160a, 160b, and 160c in RAN 104 via the S1 interface. The SGW 164 typically routes and forwards user data packets to / from WTRUs 102a, 102b, and 102c. Furthermore, the SGW 164 can perform other functions, such as anchoring the user plane during handover between eNodeBs, triggering paging processing when DL data is available to WTRUs 102a, 102b, and 102c, and managing and storing the context of WTRUs 102a, 102b, and 102c, etc.

[0137] SGW 164 can be connected to PGW 166, which can provide packet-switched network (e.g., Internet 110) access for WTRUs 102a, 102b, and 102c to facilitate communication between WTRUs 102a, 102b, and 102c and IP-enabled devices.

[0138] CN 106 can facilitate communication with other networks. For example, CN 106 can provide circuit-switched network (e.g., PSTN 108) access for WTRUs 102a, 102b, and 102c to facilitate communication between WTRUs 102a, 102b, and 102c and conventional landline communication equipment. For example, CN 106 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server), and the IP gateway may act as an interface between CN 106 and PSTN 108. Furthermore, CN 106 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0139] Although Figure 8A-1 In D, WTRU is described as a wireless terminal; however, it should be understood that in some representative embodiments, such a terminal may use a wired communication interface (e.g., temporary or permanent) with the communication network.

[0140] In some representative embodiments, the other network 112 may be a WLAN.

[0141] A WLAN employing an Infrastructure Basic Services Set (BSS) model may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may access or interface with a distributed system (DS) or other types of wired / wireless networks that send traffic into and / or out of the BSS. Traffic originating outside the BSS and destined for a STA can be delivered to the STA via the AP. Traffic originating from a STA and destined for a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. Traffic between STAs within the BSS can be sent via the AP, for example, where a source STA can send traffic to the AP and the AP can deliver the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as point-to-point traffic. Point-to-point traffic can be sent between the source and destination STAs (e.g., directly therebetween) using Direct Link Establishment (DLS). In some representative embodiments, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). For example, a WLAN using the Standalone BSS (IBSS) mode may not have an access point (AP) and can communicate directly with each other either within the IBSS or with the STAs using the IBSS (e.g., all STAs). Here, the IBSS communication mode is sometimes referred to as a "self-organizing" communication mode.

[0142] When operating in 802.11ac infrastructure mode or a similar mode, the AP can transmit beacons on a fixed channel (e.g., the primary channel). The primary channel can have a fixed width (e.g., a 20 MHz bandwidth) or a width dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by STAs to establish connections with the AP. In some representative embodiments, carrier-sensing multiple access with collision avoidance (CSMA / CA) can be implemented (e.g., in an 802.11 system). For CSMA / CA, STAs, including the AP (e.g., each STA), can sense the primary channel. If a particular STA senses / detects and / or determines that the primary channel is busy, that particular STA can back off. In a given BSS, at any given time, there is only one STA (e.g., only one station) transmitting.

[0143] High-throughput (HT) STAs can communicate using channels with a width of 40 MHz (e.g., by combining a 20 MHz main channel with adjacent or non-adjacent 20 MHz channels to form a 40 MHz channel).

[0144] Very High Throughput (VHT) STAs can support channels with widths of 20MHz, 40MHz, 80MHz, and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining consecutive 20MHz channels. A 160MHz channel can be formed by combining eight consecutive 20MHz channels or by combining two non-consecutive 80MHz channels (this combination may be referred to as an 80+80 configuration). For the 80+80 configuration, after channel coding, data is transmitted and passed through a segmented parser that splits the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time-domain processing can be performed individually on each stream. The streams can be mapped onto two 80MHz channels, and the data can be transmitted by the STA performing the transmission. On the receiver of the STA performing the reception, the above operations for the 80+80 configuration can be reversed, and the combined data can be sent to the Media Access Control (MAC).

[0145] 802.11af and 802.11ah support sub-1 GHz operating modes. Compared to 802.11n and 802.11ac, the channel operating bandwidth and carrier used in 802.11af and 802.11ah are reduced. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV white space (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah can support instrument-type control / machine-type communication (e.g., MTC devices in macro coverage areas). MTCs may have certain capabilities, such as limited capabilities including supporting (e.g., only supporting) certain and / or limited bandwidths. MTC devices may include a battery with a battery life exceeding a threshold (e.g., for maintaining a very long battery life).

[0146] For WLAN systems that can support multiple channels and channel bandwidths (e.g., 802.11n, 802.11ac, 802.11af, and 802.11ah), these systems include a channel that can be designated as the primary channel. The bandwidth of the primary channel can be equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by a single STA, which is derived from all STAs operating in the BSS that support the minimum bandwidth operating mode. In the example of 802.11ah, even if the APs and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes, the width of the primary channel can be 1MHz for STAs that support (e.g., only support) the 1MHz mode (e.g., MTC type devices). Carrier sensing and / or Network Assignment Vector (NAV) settings can depend on the status of the primary channel. If the main channel is busy (e.g., because the STA (which only supports 1MHz operating mode) transmits to the AP), then the entire available band can be considered busy even if most of the band is free and available.

[0147] In the United States, the available frequency band for 802.11ah is 902MHz to 928MHz. In South Korea, the available frequency band is 917.5MHz to 923.5MHz. In Japan, the available frequency band is 916.5MHz to 927.5MHz. Depending on the country, the total bandwidth available for 802.11ah ranges from 6MHz to 26MHz.

[0148] Figure 8D This is a system diagram illustrating RAN 113 and CN 115 according to one embodiment. As described above, RAN 113 can communicate with WTRUs 102a, 102b, and 102c using NR radio technology on air interface 116. RAN 113 can also communicate with CN 115.

[0149] RAN 113 may include gNBs 180a, 180b, and 180c; however, it should be understood that RAN 113 may include any number of gNBs while maintaining conformity with the embodiments. Each gNB 180a, 180b, and 180c may include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 180b may use beamforming to transmit and / or receive signals to and / or from gNBs 180a, 180b, and 180c. Thus, for example, gNB 180a may use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a. In one embodiment, gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB 180a can transmit multiple component carriers (not shown) to WTRU 102a. A subset of these component carriers may be in unlicensed spectrum, while the remaining component carriers may be in licensed spectrum. In one embodiment, gNBs 180a, 180b, and 180c may implement Cooperative Multipoint (CoMP) technology. For example, WTRU 102a can receive cooperative transmissions from gNBs 180a and 180b (and / or gNB 180c).

[0150] WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using transmissions associated with scalable parameter configurations. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing can be different for different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using subframes or transmission time intervals (TTIs) of different or scalable lengths (e.g., containing different numbers of OFDM symbols and / or varying absolute durations).

[0151] gNBs 180a, 180b, and 180c can be configured to communicate with WTRUs 102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c without accessing other RANs (e.g., eNodeBs 160a, 160b, and 160c). In standalone configuration, WTRUs 102a, 102b, and 102c can use one or more of gNBs 180a, 180b, and 180c as mobile anchors. In standalone configuration, WTRUs 102a, 102b, and 102c can use signals in unlicensed frequency bands to communicate with gNBs 180a, 180b, and 180c. In a non-standalone configuration, WTRUs 102a, 102b, and 102c communicate / connect with gNBs 180a, 180b, and 180c simultaneously with other RANs (e.g., eNodeBs 160a, 160b, and 160c). For example, WTRUs 102a, 102b, and 102c can communicate substantially simultaneously with one or more gNBs 180a, 180b, and 180c, as well as one or more eNodeBs 160a, 160b, and 160c, by implementing DC principles. In a non-standalone configuration, eNodeBs 160a, 160b, and 160c can act as mobile anchors for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c can provide additional coverage and / or throughput to service WTRUs 102a, 102b, and 102c.

[0152] Each gNB 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, support network slicing, implement dual connectivity, implement interoperability processing between NR and E-UTRA, route user plane data to User Plane Functions (UPF) 184a and 184b, and route control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 8D As shown, gNB 180a, 180b, and 180c can communicate with each other via the Xn interface.

[0153] Figure 8DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and may include data network (DN) 185a, 185b. While each of the foregoing components is described as part of CN 115, it should be understood that any of these components may be owned and / or operated by an entity other than the CN operator.

[0154] AMF 182a and 182b can connect to one or more gNBs 180a, 180b, and 180c in RAN 113 via the N2 interface and can act as control nodes. For example, AMF 182a and 182b can be responsible for authenticating users of WTRU 102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different needs), selecting specific SMF 183a and 183b, managing registration areas, terminating NAS signaling, and mobility management, etc. AMF 182a and 1823b can use network slicing to customize the CN support provided to WTRU 102a, 102b, and 102c based on the type of service being used. As an example, different network slices can be established for different use cases, such as services relying on Ultra Reliable Low Latency (URLLC) access, services relying on Enhanced Massive Mobile Broadband (eMBB) access, and / or services for Machine Type Communication (MTC) access, etc. AMF 162 can provide control plane functionality for switching between RAN 113 and other RANs (not shown) that use other radio technologies (e.g., LTE, LTE-A, LTE-A Pro and / or non-3GPP access technologies such as WiFi).

[0155] SMFs 183a and 183b can connect to AMFs 182a and 182b in CN 115 via the N11 interface. SMFs 183a and 183b can also connect to UPFs 184a and 184b in CN 115 via the N4 interface. SMFs 183a and 183b can select and control UPFs 184a and 184b, and can configure traffic routing through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, and Ethernet-based, etc.

[0156] UPF 184a and 184b can be connected to one or more gNBs 180a, 180b, and 180c in RAN 113 via the N3 interface, thus providing packet-switched network (e.g., Internet 110) access for WTRU 102a, 102b, and 102c to facilitate communication between WTRU 102a, 102b, and 102c and IP-enabled devices. UPF 184a and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multihomed PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring processing, etc.

[0157] CN 115 can facilitate communication with other networks. For example, CN 115 may include or can communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN 115 and CN 108. Furthermore, CN 115 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRUs 102a, 102b, and 102c can be connected to local data networks (DNs) 185a and 185b via the N3 interface connected to UPFs 184a and 184b and the N6 interface between UPFs 184a and 184b and DNs 185a and 185b.

[0158] In view of Figures 8A-8D And about Figures 8A-8D The corresponding descriptions herein refer to one or more of the following functions, which can be performed by one or more emulation devices (not shown): WTRU 102a-d, Base Station 114a-b, eNodeB 160a-c, MME 162, SGW164, PGW 166, gNB 180a-c, AMF 182a-ab, UPF 184a-b, SMF 183a-b, DN 185a-b, and / or any other devices described herein. These emulation devices can be one or more devices configured to simulate one or more of the functions described herein. For example, these emulation devices can be used to test other devices and / or simulate network and / or WTRU functions.

[0159] The simulation device can be designed to perform one or more tests on other devices in a laboratory environment and / or a carrier network environment. For example, the one or more simulation devices can perform one or more functions while being implemented and / or deployed, wholly or partially, as part of a wired and / or wireless communication network, to test other devices within the communication network. The one or more simulation devices can perform one or more functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. The simulation device can be directly coupled to other devices to perform tests, and / or can use over-the-air wireless communication to perform tests.

[0160] One or more simulation devices can perform one or more functions, including all functionalities, without being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation devices can be used in test environments, such as test labs and / or test scenarios where wired and / or wireless communication networks are not deployed (e.g., under test), to perform tests on one or more components. The one or more simulation devices can be test equipment. The simulation devices can transmit and / or receive data using direct RF coupling and / or wireless communication via RF circuitry (which, as an example, may include one or more antennas).

[0161] Figure 9 This is a system diagram illustrating the implementation of various aspects and operations according to the various embodiments.

[0162] See Figure 9 System 1000 can be implemented as a device comprising the various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of System 1000 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of System 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, System 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, System 1000 is configured to implement one or more aspects described herein.

[0163] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing various aspects, such as those described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, the storage device 1040 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0164] System 1000 includes an encoder / decoder module 1030 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents one or more modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Alternatively, the encoder / decoder module 1030 may be implemented as a separate element of system 1000 or may be incorporated into processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0165] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to execute the various aspects described herein may be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. According to various embodiments, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of various items during the execution of the processes described herein. These stored items may include, but are not limited to, input video, decoded video or portions of the decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0166] In some embodiments, the memory within the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. External memory may be memory 1020 and / or storage device 1040, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for video decoding and decoding operations, such as working memory for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Decoding; also known as H.265 and MPEG-H Part 2), or VVC (Universal Video Decoding, a new standard developed by the Joint Video Team experts JVET).

[0167] As shown in box 1130, input to the components of system 1000 can be provided through various input devices. Such input devices include, but are not limited to: (i) an RF section that receives radio frequency (RF) signals, for example, transmitted over the air by a broadcaster; (ii) component input terminals (or a set of component input terminals); (iii) universal serial bus (USB) input terminals; and / or (iv) high-definition multimedia interface (HDMI) input terminals. Other examples not shown in Figure 10 include composite video.

[0168] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) downconverting the selected signal, (iii) again limiting the band to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the downconverted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements to perform these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various of these functions, including, for example, downconverting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection to a desired frequency band through filtering, down-conversion, and re-filtering. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0169] Additionally, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Solomon error correction) can be implemented as needed, for example, within a separate input processing IC or processor 1010. Similarly, various aspects of USB or HDMI interface processing can be implemented as needed within a separate interface IC or within processor 1010. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0170] Various components of system 1000 can be housed within an integrated housing. Within this integrated housing, various components can be interconnected and transmit data therebetween using a suitable connection arrangement 1140 (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards).

[0171] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 1060 may be implemented, for example, in a wired and / or wireless medium.

[0172] In various embodiments, a wireless network (e.g., a Wi-Fi network, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)) is used to stream or otherwise provide data to the system 1000. In these embodiments, the Wi-Fi signal is received via a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cloud-based communications. Other embodiments use a set-top box that transmits data via an HDMI connection to input box 1130 to provide streaming data to the system 1000. Still other embodiments use an RF connection to input box 1130 to provide streaming data to the system 1000. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0173] The system 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. The display 1100 in various embodiments includes one or more of the following: for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. The display 1100 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. The display 1100 can also be integrated with other components (e.g., in a smartphone) or stand alone (e.g., an external monitor for a laptop computer). In various examples of embodiments, the other peripheral devices 1120 include one or more of the following: a standalone digital video disc (or digital multifunction disc) (DVR, for both), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of the system 1000. For example, a disc player performs the function of playing the output of the system 1000.

[0174] In various embodiments, signaling (such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention) is used to transmit control signals between system 1000 and display 1100, speaker 1110, or other peripheral devices 1120. The output devices may be communicatively coupled to system 1000 via dedicated connections through corresponding interfaces 1070, 1080, and 1090. Alternatively, the output devices may be connected to system 1000 via communication interface 1050 using communication channel 1060. The display 1100 and speaker 1110 may be integrated into a single unit within an electronic device (e.g., a television set) along with other components of system 1000. In various embodiments, display interface 1070 includes a display driver, such as a timing controller ((TCon) chip).

[0175] For example, if the RF portion of input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separated from one or more other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection, such as an HDMI port, a USB port, or a COMP output.

[0176] These embodiments can be implemented by processor 1010 or by computer software implemented in hardware or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. The memory 1020 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 1010 can be of any type suitable for the technical environment and, as a non-limiting example, can include one or more of the following: microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0177] Various implementations involve decoding. As used herein, "decoding" can include, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by decoders of various implementations described herein, such as extracting an image from a tiled (packed) image, determining an upsampling filter to use and then upsampling the image, and flipping the image back to its predetermined orientation.

[0178] As a further example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process will be clear based on the specific context of the description and is believed to be fully understood by those skilled in the art.

[0179] Various implementations involve encoding. In a manner similar to the above discussion of “decoding,” “encoding,” as used herein, can include all or part of a process performed, for example, on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy decoding. In various embodiments, such processes also, or alternatively, include processes performed by encoders of the various implementations described herein.

[0180] As a further example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of differential encoding and entropy decoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process will become clear based on the specific context of the description and is believed to be fully understood by those skilled in the art. Note that the grammatical elements used herein are descriptive terms, and this disclosure is not limited thereto. Therefore, the use of other grammatical element names is not excluded.

[0181] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding devices. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0182] The implementations and aspects described herein can be implemented, for example, in a method or process, apparatus, software program, data stream, or signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), implementations of the discussed features can also be implemented in other forms (e.g., apparatus or program). For example, an apparatus can be implemented with appropriate hardware, software, and firmware. The method can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0183] The reference to "an embodiment," "an embodiment," "an implementation," or "an implementation," and other variations, means that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases "in an embodiment," "in one embodiment," "in one implementation," or "in one implementation," and any other variations appearing in various places throughout this application, do not necessarily refer to the same embodiment.

[0184] Additionally, this application may involve "determining" various types of information. Determining the information may include, for example, one or more of the following: estimating the information, calculating the information, predicting the information, or retrieving the information from memory.

[0185] Furthermore, this application may relate to "accessing" various types of information. Accessing such information may include, for example, one or more of the following: receiving the information, retrieving the information (e.g., retrieving the information from a memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0186] Additionally, this application can refer to "receiving" various types of information. Like "accessing," receiving is intended to be a broad term. Receiving said information may include, for example, one or more of the following: accessing the information or (e.g., retrieving the information from memory). Furthermore, "receiving" is generally referred to in one or more ways during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0187] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one of” is intended to cover the selection of only the first listed option (A), or only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to multiple listed items.

[0188] Furthermore, as used herein, the term "signal" specifically refers to indicating something to the corresponding decoder. For example, in some embodiments, the encoder signals a specific parameter among a plurality of parameters for selecting region-based filter parameters used for artifact removal filtering. This could be a specific one among a plurality of transforms, decoding modes, or flags. Thus, in one embodiment, the same parameter is used on both the encoder and decoder sides. Therefore, for example, the encoder can send (explicitly signal) a specific parameter to the decoder so that the decoder can use that same specific parameter. Conversely, if the decoder already has said specific parameter as well as other parameters, signaling can be used without sending (implicitly signal) to simply allow the decoder to know and select said specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. While the foregoing refers to the verb form of the term "signal," the term "signal" can also be used as a noun herein.

[0189] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry, for example, information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. The formatting may include, for example, encoding the data stream and modulating a carrier wave using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0190] We have described several embodiments. The features of these embodiments may be provided individually or in any combination across various claim classes and types. Furthermore, across various claim classes and types, embodiments may individually or in any combination include any of the following features, devices, or aspects: (1) modifying the prediction process applied in the decoder and / or encoder; (2) enabling several advanced prediction methods in the decoder and / or encoder; (3) inserting syntax elements into the signaling that enable the decoder to identify the prediction method to be used; (4) selecting a prediction method to be applied at the decoder based on these syntax elements; (5) applying the prediction method to derive the prediction at the decoder; (6) adapting the residual at the encoder according to any of the embodiments discussed; (7) a bitstream or signal including one or more of the described syntax elements or variations thereof; (8) a bitstream or signal including a syntax that conveys information generated according to any of the embodiments described; (9) inserting syntax elements into the signaling that enable the decoder to adapt the residual in a manner corresponding to that used by the encoder; and (10) creating and / or sending and / or receiving and / or decoding a bitstream or signal including one or more of the described syntax elements or variations thereof. (11) Creating and / or transmitting and / or receiving and / or decoding according to any of the described embodiments; (12) A method, process, apparatus, medium for storing instructions, medium for storing data, or signal according to any of the described embodiments; (13) A TV, set-top box, cellular phone, tablet computer, or other electronic device that performs filter parameter adaptation according to any of the described embodiments; (14) A TV, set-top box, cellular phone, tablet computer, or other electronic device that performs filter parameter adaptation and displays (e.g., using a monitor, screen, or other type of display) the resulting image according to any of the described embodiments; (15) A TV, set-top box, cellular phone, tablet computer, or other electronic device that selects (e.g., using a tuner) a channel to receive a signal including an encoded image according to any of the described embodiments and performs filter parameter adaptation; and (16) A TV, set-top box, cellular phone, tablet computer, or other electronic device that receives (e.g., using an antenna) a signal including an encoded image over the air according to any of the described embodiments and performs filter parameter adaptation.

[0191] While features and elements in specific combinations have been described above, those skilled in the art will recognize that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in computer programs, software, or firmware incorporated into a computer-readable medium for execution by a computer or processor. Examples of non-transitory computer-readable media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, buffer memory, semiconductor storage devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROM discs and digital multipurpose discs (DVDs)). The processor associated with the software can be used to implement radio frequency transceivers used in WTRU 102, UE, terminals, base stations, RNCs, or any computer host.

[0192] Furthermore, the above embodiments mention a processing platform, a computing system, a controller, and other devices including a processor. These devices may include at least one central processing unit (“CPU”) and memory. In accordance with the practice of those skilled in the art of computer programming, references to the behavior or symbolic representation of operations or instructions may be executed by different CPUs and memories. Such behavior and operations or instructions may be referred to as “running,” “computer running,” or “CPU running.”

[0193] Those skilled in the art will understand that actions and symbolically represented operations or instructions include the manipulation of electronic signals by the CPU. The electronic system represents data bits that may cause electronic signals to be transformed or reduced, and memory locations in the memory system that store the data bits, thereby reconfiguring or otherwise altering CPU operations and other signal processing. The memory location storing the data bits is a physical location having specific electrical, magnetic, optical, or organic properties corresponding to or representing the data bits. It should be understood that the exemplary embodiments described herein are not limited to the platforms or CPUs described above, and other platforms and CPUs can also support the methods provided.

[0194] Data bits can also be stored on computer-readable media, including disks, optical disks, and any other volatile (e.g., random access memory (“RAM”) or non-volatile (e.g., read-only memory (“ROM”)) mass storage systems readable by the CPU. Computer-readable media can include cooperative or interconnected computer-readable media, which can exist solely on the processing system or distributed across multiple interconnected processing systems located locally or remotely to the processing system. It should be understood that these representative embodiments are not limited to the aforementioned memories, and other platforms and memories can also support the described methods.

[0195] In one illustrative embodiment, any operations, processes, etc., described herein can be implemented as computer-readable instructions stored on a computer-readable medium. These computer-readable instructions can be executed by a processor of a mobile unit, network component, and / or any other computing device.

[0196] There is virtually no difference between the hardware and software implementations of various aspects of the system. The choice between hardware and software is typically (but not always, as the choice between hardware and software can be significant in certain contexts) a design choice representing a trade-off between cost and efficiency. The processes and / or systems and / or other technologies described herein can be implemented by various carriers (e.g., hardware, software, and / or firmware), and the preferred carrier can vary depending on the context in which the processes and / or systems and / or other technologies are deployed. For example, if the implementation determines that speed and accuracy are paramount, then the implementer may tend to use a primarily hardware and / or firmware carrier. If flexibility is paramount, then the implementer may tend to use a primarily software implementation. Alternatively, the implementer may choose some combination of hardware, software, and / or firmware.

[0197] The specific embodiments described above have illustrated various embodiments of the device and / or processing using block diagrams, flowcharts, and / or examples. Just as such block diagrams, flowcharts, and / or examples encompass one or more functions and / or operations, those skilled in the art will understand that each function and / or operation within such block diagrams, flowcharts, or examples can be implemented individually and / or collectively by a wide range of hardware, software, firmware, or virtually any combination thereof. As examples, suitable processors include general-purpose processors, special-purpose processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), field-programmable gate arrays (FPGAs), any other type of integrated circuit (IC), and / or state machines.

[0198] While features and elements in specific combinations have been described above, those skilled in the art will recognize that each feature or element can be used alone or in any combination with other features and elements. This disclosure is not intended to limit the scope to the embodiments described herein, which are intended to illustrate different aspects. Those skilled in the art will understand that numerous modifications and variations are possible without departing from the spirit and scope. Elements, actions, or instructions used in this specification should not be construed as essential to the invention unless explicitly provided otherwise. In addition to the methods and apparatuses enumerated herein, those skilled in the art will clearly understand from the above description functionally equivalent methods and apparatuses within the scope of this disclosure. Such modifications and variations should fall within the scope of the appended claims. This disclosure is limited only by the appended claims and the full scope of their equivalents. It should be understood that this disclosure is not limited to any particular method or system.

[0199] It should also be understood that the terminology used herein is intended to describe particular embodiments only and is not intended to be limiting. When referenced herein, the terms “station” and its abbreviation “STA”, “user equipment” and its abbreviation “UE” as used herein may refer to (i) a wireless transmit and / or receive unit (WTRU) as described below; (ii) any of the various embodiments of the WTRU as described below; (iii) a device having wireless and / or wired capabilities (e.g., connectable), particularly, the device being configured with some or all of the structures and functions of the WTRU as described above; (iii) a device having wireless and / or wired capabilities configured with relatively fewer structures and functions compared to all the structures and functions of the WTRU as described above; or (iv) a similar device.

[0200] In some representative embodiments, several portions of the subject matter described herein may be implemented using application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and / or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein can be implemented, in whole or in part, in an equivalent manner in an integrated circuit, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that circuit design and / or code writing relating to the software and / or firmware in accordance with this disclosure also fall within the skill level of those skilled in the art. Furthermore, those skilled in the art will understand that the mechanisms of the subject matter described herein can be distributed in various forms as program products, and the illustrative embodiments of the subject matter described herein are applicable regardless of the particular type of signal-bearing medium used to actually perform the distribution. Examples of signal-bearing media include, but are not limited to, the following: recordable media, such as floppy disks, hard disks, CDs, DVDs, digital magnetic tapes, computer memory, etc., and transmission-type media, such as digital and / or analog communication media (e.g., optical fiber, waveguides, wired communication links, wireless communication links, etc.).

[0201] The topics described herein sometimes show different components contained within or connected to other different components. It should be understood that the architectures described in this way are merely examples, and numerous other architectures for implementing the same functionality are indeed feasible. Conceptually, any arrangement of components that achieve the same function is effectively “associated” to achieve the desired functionality. Therefore, any two components combined here to achieve a particular function can be considered “associated” with each other, thereby achieving the desired functionality, regardless of the architecture or intermediate components. Similarly, any two components associated in this way can also be considered “operably connected” or “operably coupled” to each other to achieve the desired functionality, and any two components that can be associated in this way can also be considered “operably coupled” to each other to achieve the desired functionality. Specific examples of being operablely coupled include, but are not limited to, components that can be physically paired and / or physically interact and / or components that can interact wirelessly and / or wirelessly and / or components that interact logically and / or are logically interactive.

[0202] As for any substantially plural and / or singular terms used herein, those skilled in the art can appropriately convert them from plural to singular and / or from singular to plural depending on the context and / or application. For clarity, various singular / plural permutations may be explicitly described herein.

[0203] Those skilled in the art will understand that, in general, the terms used herein, particularly in the appended claims (e.g., the body of the appended claims), should be treated as “open-ended” terms (for example, the term “comprising” should be interpreted as “comprising but not limited to,” the term “having” should be interpreted as “having at least,” the term “including” should be interpreted as “comprising but not limited to,” etc.). Those skilled in the art will further understand that if the introduced claim refers to a specific quantity, then this intention should be explicitly stated in the claim, and without such a statement, such intention does not exist. For example, if only one item is intended, then the term “single” or similar language can be used. As an aid to understanding, subsequent appended claims and / or the description herein may include the use of introductory phrases such as “at least one” and “one or more” to introduce the statement of the claim. However, the use of such phrases should not be construed as a way of introducing a claim statement that limits any particular claim containing such a claim statement to an embodiment containing only one such statement by using the indefinite articles “a” or “an,” even when the same claim contains the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and / or “an” should be interpreted as “at least one” or “one or more”). The same applies to the use of definite articles used to introduce claim statements. Furthermore, even when a specific number of claim statements are explicitly stated, those skilled in the art will recognize that such a statement should be interpreted as referring to at least the number stated (e.g., an unmodified statement of “two statements” without other modifiers means at least two statements or two or more statements). Furthermore, in these instances, if a specification similar to "at least one of A, B, and C, etc." is used, then such a structure should generally have the meaning of that specification as understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" would include, but is not limited to, systems having only A, only B, only C, A and B, A and C, B and C, and / or A, B, and C, etc.). In instances where a specification similar to "at least one of A, B, or C, etc." is used, such a structure should generally have the meaning of said specification as understood by a person skilled in the art (for example, "a system having at least one of A, B, or C" includes, but is not limited to, systems having only A, only B, only C, A and B, A and C, B and C, and / or A, B, and C, etc.).Those skilled in the art will further understand that virtually any disjoint words and / or phrases proposing two or more alternatives, whether in the specification, claims, or drawings, should be understood to imply the possibility of including one, any, or both of these items. For example, the phrase "A or B" will be understood to include the possibility of including "A" or "B" or "A and B". Furthermore, the term "any one" as used herein, followed by a series of multiple items and / or multiple item categories, is intended to include "any one," "any combination," "any number," and / or "a combination of any number" of items and / or item categories, alone or in combination with other items and / or other item categories. Additionally, the terms "set" or "group" as used herein should include any number of items, including zero. Additionally, the term "quantity" as used herein is intended to include any number, including zero.

[0204] Furthermore, if any feature or aspect of this disclosure is described in accordance with the Markush group, then those skilled in the art will recognize that this disclosure is thus described in accordance with any single member or subgroup of members of the Markush group.

[0205] Those skilled in the art will understand that, for any and all purposes (e.g., in providing a written description), all scopes disclosed herein also include any and all possible subscopes and combinations thereof. Any scope listed can be readily considered to adequately describe and enable the same scope that is decomposed into at least two, three, four, five, ten, etc., equal parts. As a non-limiting example, each scope discussed herein can be readily decomposed into a lower third, a middle third, and an upper third. Those skilled in the art will understand that all language such as “at most,” “at least,” “greater than,” “less than,” etc., includes the numbers stated and refers to a scope that can subsequently be decomposed into subscopes as described above. Finally, as those skilled in the art will understand, a scope will include each individual member. Thus, by way of example, a group with 1-3 cells refers to a group with 1, 2, or 3 cells. Similarly, a group with 1-5 cells refers to a group with 1, 2, 3, 4, or 5 cells, and so on.

[0206] Furthermore, unless otherwise stated, the claims should not be mistakenly considered as limited to the order or elements described. Additionally, the term "means for..." used in any claim is intended to invoke 35 U.S.SC §112(f) or to mean the "means-plus-function" claim format, and any claim without the word "means" does not have this meaning.

[0207] Although the invention has been described and illustrated herein with reference to specific embodiments, it is not intended to be limited to the details shown. Rather, various modifications to the details may be made within the scope of the claims and without departing from the invention.

[0208] Throughout this disclosure, those skilled in the art will understand that certain representative embodiments may be used alternatively or in combination with other representative embodiments.

[0209] While features and elements in specific combinations have been described above, those skilled in the art will recognize that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in computer programs, software, or firmware incorporated into a computer-readable medium for execution by a computer or processor. Examples of non-transitory computer-readable media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, buffer memory, semiconductor storage devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROM discs and digital multipurpose discs (DVDs)). The processor associated with the software can be used to implement radio frequency transceivers used in WTRUs, UEs, terminals, base stations, RNCs, or any computer host.

[0210] Furthermore, the above embodiments mention a processing platform, a computing system, a controller, and other devices including a processor. These devices may include at least one central processing unit (“CPU”) and memory. In accordance with the practice of those skilled in the art of computer programming, references to the behavior or symbolic representation of operations or instructions may be executed by different CPUs and memories. Such behavior and operations or instructions may be referred to as “running,” “computer running,” or “CPU running.”

[0211] Those skilled in the art will understand that actions and symbolically represented operations or instructions include the manipulation of electronic signals by the CPU. Electronic systems represent data bits that may cause electronic signals to be transformed or reduced, and memory locations in a memory system that store data bits, thereby reconfiguring or otherwise altering data bits for CPU operations and other signal processing. A memory location that holds data bits is a physical location having specific electrical, magnetic, optical, or organic properties that correspond to or represent the data bits.

[0212] The data bits can also be stored on a computer-readable medium, including disks, optical disks, and any other volatile (e.g., random access memory (“RAM”)) or non-volatile (e.g., read-only memory (“ROM”) mass storage system readable by the CPU. The computer-readable medium can include cooperative or interconnected computer-readable media, which can exist solely on the processing system or distributed across multiple interconnected processing systems located locally or remotely to the processing system. It should be understood that these representative embodiments are not limited to the aforementioned memories, and other platforms and memories can also support the described methods.

[0213] Unless explicitly described herein, the elements, actions, or instructions used in the description of this application should not be construed as critical or necessary to the invention. Additionally, as used herein, the article “a” is intended to include one or more items. Where only one item is referred to, the term “a” or similar language is used. Furthermore, as used herein, the term “any” followed by a list of multiple items and / or multiple item categories is intended to include “any,” “any combination,” “any number,” and / or “any combination of multiple” of said items and / or item categories, and may be used alone or in combination with other items and / or other item categories. Furthermore, as used herein, the term “set” is intended to include any number of items, including zero. Furthermore, as used herein, the term “quantity” is intended to include any quantity, including zero.

[0214] Furthermore, the claims should not be construed as being limited to the order or elements described unless such a statement is made. Additionally, the use of the term "means for..." in any claim is intended to invoke 35 U.S.SC §112(f), and any claim without the phrase "means for..." does not have such an intent.

[0215] For example, suitable processors include general-purpose processors, special-purpose processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), field-programmable gate arrays (FPGAs), any other type of integrated circuit (IC) and / or state machines.

[0216] A software-associated processor can be used to implement a radio frequency transceiver for use in a Transmitter-Receiver Unit (WTRU), User Equipment (UE), terminal, base station, Mobility Management Entity (MME), or Evolved Packet Core (EPC), or any type of host computer. The WTRU can be used in conjunction with modules implemented in hardware and / or software, said modules including Software-Defined Radio (SDR) and other components such as cameras, video camera modules, videophones, speaker amplifiers, vibration devices, speakers, microphones, television transceivers, hands-free headsets, keyboards, etc. Modules, FM radio units, Near Field Communication (NFC) modules, Liquid Crystal Display (LCD) units, Organic Light Emitting Diode (OLED) units, digital music players, media players, video game console modules, Internet browsers, and / or any type of Wireless Local Area Network (WLAN) or Ultra Wideband (UWB) module.

[0217] Although the invention has been described in relation to a communication system, it is contemplated that the system can be implemented in software on a microprocessor / general-purpose computer (not shown). In some embodiments, the functionality of one or more of the various components can be implemented in software that controls the general-purpose computer.

[0218] Furthermore, although the invention has been shown and described herein with reference to specific embodiments, the invention is not limited to the details shown. Rather, various modifications to the details may be made within the scope of the claims and without departing from the invention.

Claims

1. A video decoding method, comprising: Receive video data blocks, wherein the video data blocks include multiple sub-blocks; as well as For one of the plurality of sub-blocks, the prediction of the sub-block is refined in the following manner: Derive motion vectors, each motion vector indicating the refined motion predicted for a corresponding sub-block among the plurality of sub-blocks. Calculate the affine motion model based on the derived motion vectors. Using the affine motion model, the corresponding motion vector offset is determined for the predicted pixels of the sub-block. The intensity change of the pixel is obtained through the optical flow equation, based on the corresponding spatial gradient and the corresponding motion vector offset. The pixels are refined based on the obtained intensity changes.

2. The method of claim 1, wherein the derived motion vector is obtained through a decoder-based motion vector refinement process.

3. The method of claim 1, wherein the model error is estimated based on the derived motion vector, and if the estimated model error exceeds a threshold, the calculation, determination, acquisition, and refinement for the sub-block are skipped.

4. The method of claim 1, wherein the plurality of sub-blocks includes sub-blocks adjacent to the sub-blocks, and wherein the calculation of the affine motion model is based on motion vectors derived for the adjacent sub-blocks, the method further comprising: The model error of the affine motion model is estimated using the motion vectors derived for the sub-block; as well as If the estimated model error exceeds a threshold, the determination, acquisition, and refinement for the sub-block are skipped.

5. The method of claim 1, wherein the affine motion model computed for the sub-block is used to refine the prediction of another sub-block of the video data block.

6. The method of claim 1, wherein the affine motion model is a four-parameter affine motion model for one or more of the plurality of sub-blocks, and wherein the one or more sub-blocks are located at the left boundary and / or top boundary of the video data block.

7. The method according to claim 1, wherein the calculation of the affine motion model further comprises: Calculate multiple models based on the derived motion vectors; Estimate the model errors of the multiple models; as well as The model with the smallest error among the multiple models is selected as the affine motion model.

8. The method according to claim 1, wherein the calculation of the affine motion model further comprises: Multiple models are computed based on the derived motion vectors, wherein the multiple models include one or more of a six-parameter affine motion model and a four-parameter affine motion model.

9. The method of claim 1, wherein refining the pixel based on the obtained intensity change comprises: The obtained intensity changes are weighted using weighting factors, and Add a weighted intensity change to the pixel.

10. The method of claim 9, further comprising: Receive a weight index, which indicates, at the video block level or image level, the weight factor used to weight the obtained intensity changes.

11. A video encoding method, comprising: Receive video data blocks, wherein the video data blocks include multiple sub-blocks; as well as For one of the plurality of sub-blocks, the prediction of the sub-block is refined in the following manner: Derive motion vectors, each motion vector indicating the refined motion predicted for a corresponding sub-block among the plurality of sub-blocks. Calculate the affine motion model based on the derived motion vectors. Using the affine motion model, the corresponding motion vector offset is determined for the predicted pixels of the sub-block. The intensity change of the pixel is obtained through the optical flow equation, based on the corresponding spatial gradient and the corresponding motion vector offset. The pixels are refined based on the obtained intensity changes.

12. A video decoding apparatus, comprising: At least one processor; as well as Memory, storing instructions that, when executed by the at least one processor, cause the device to: Receive video data blocks, wherein the video data blocks include multiple sub-blocks, and For one of the plurality of sub-blocks, the prediction of the sub-block is refined in the following manner: Derive motion vectors, each motion vector indicating the refined motion predicted for a corresponding sub-block among the plurality of sub-blocks. Calculate the affine motion model based on the derived motion vectors. Using the affine motion model, the corresponding motion vector offset is determined for the predicted pixels of the sub-block. The intensity change of the pixel is obtained through the optical flow equation, based on the corresponding spatial gradient and the corresponding motion vector offset. The pixels are refined based on the obtained intensity changes.

13. The apparatus of claim 12, wherein the derived motion vector is obtained through a decoder-based motion vector refinement process.

14. The apparatus of claim 12, wherein the plurality of sub-blocks includes sub-blocks adjacent to the sub-blocks, wherein the calculation of the affine motion model is based on motion vectors derived for the adjacent sub-blocks, and wherein the instructions further cause the apparatus to: The model error of the affine motion model is estimated using the motion vectors derived for the sub-block; and If the estimated model error exceeds a threshold, the determination, acquisition, and refinement for the sub-block are skipped.

15. The apparatus of claim 12, wherein the affine motion model calculated for the sub-block is used to refine the prediction of another sub-block of the video data block.

16. The apparatus of claim 12, wherein the calculation of the affine motion model further comprises: Calculate multiple models based on the derived motion vectors; Estimate the model errors of the multiple models; as well as The model with the smallest error among the multiple models is selected as the affine motion model.

17. The apparatus of claim 12, wherein the calculation of the affine motion model further comprises: Multiple models are computed based on the derived motion vectors, wherein the multiple models include one or more of a six-parameter affine motion model and a four-parameter affine motion model.

18. The apparatus of claim 12, wherein refining the pixel based on the obtained intensity change comprises: The obtained intensity changes are weighted using weighting factors, and Add a weighted intensity change to the pixel.

19. The apparatus of claim 18, wherein the instructions further cause the apparatus to: Receive a weight index, which indicates, at the video block level or image level, the weight factor used to weight the obtained intensity changes.

20. A video encoding apparatus, comprising: At least one processor; as well as Memory, storing instructions that, when executed by the at least one processor, cause the device to: Receive video data blocks, wherein the video data blocks include multiple sub-blocks, and For one of the plurality of sub-blocks, the prediction of the sub-block is refined in the following manner: Derive motion vectors, each motion vector indicating the refined motion predicted for a corresponding sub-block among the plurality of sub-blocks. Calculate the affine motion model based on the derived motion vectors. Using the affine motion model, the corresponding motion vector offset is determined for the predicted pixels of the sub-block. The intensity change of the pixel is obtained through the optical flow equation, based on the corresponding spatial gradient and the corresponding motion vector offset. The pixels are refined based on the obtained intensity changes.

Images