Methods for simplifying adaptive loop filter in video coding

Abstract

To provide systems, methods and means for adaptively selecting an adaptive loop filter (ALF) procedure for a frame based on which temporal layer the frame is in.SOLUTION: ALF procedures may vary in computational complexity. One or more frames including the current frame may be in a temporal layer of a coding scheme. The decoder may determine the current frame's temporal layer level within the coding scheme. The decoder may select an ALF procedure based on the current frame's temporal layer level. If the current frame's temporal layer level is higher within the coding scheme than some other temporal layer levels, an ALF procedure that is less computationally complex may be selected for the current frame. Then, the decoder may perform the selected ALF procedure on the current frame.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] This application relates to a method for simplifying an adaptive loop filter in video coding.

Background Art

[0002] Cross - reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 62 / 579,977, filed on November 1, 2017, and U.S. Provisional Patent Application No. 62 / 607,033, filed on December 18, 2017, the contents of which are incorporated herein by reference.

[0003] Video coding systems are widely used to compress such signals in order to reduce the storage requirements and / or transmission bandwidth of digital video signals. Among various types of video coding systems, such as block - based systems, wavelet - based systems, and object - based systems, block - based hybrid video coding systems can be most widely used and deployed. Examples of block - based video coding systems include international video coding standards such as H.261, MPEG - 1, MPEG - 2, H.263, H.264 / AVC, and H.265 / HEVC.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Provide a novel system, method, and means for simplifying an adaptive loop filter in video coding.

Means for Solving the Problems

[0005] Disclose a system, method, and means for adaptively selecting an adaptive loop filter (ALF) procedure for a frame based on in which temporal layer the frame is located. The ALF procedures can differ in terms of computational complexity.

[0006] The decoder may receive an indication as to whether the ALF is available. The decoder may apply the ALF to the current frame based on the indication that the ALF is available. The current frame may be in a temporal layer of a coding scheme. The coding scheme may include a plurality of temporal layers ranging from a low temporal level to a high temporal level. The decoder may determine the temporal layer level of the current frame within the coding scheme.

[0007] The decoder may select an ALF procedure based on the temporal layer level of the current frame. If the temporal layer level of the current frame is higher than some other temporal layer levels within the coding scheme, a first ALF procedure that is less computationally complex than a second ALF procedure may be selected for the current frame. If the temporal layer level is the lowest within the coding scheme, the second ALF procedure may be selected for the current frame. The decoder may then execute the ALF procedure selected for the current frame.

[0008] The current frame may include a current block having a plurality of pixels. An ALF procedure that is less computationally complex may classify the block based on a subset of the pixels. If the temporal layer level of the current frame is the highest within the coding scheme, a subset of the pixels may be selected from the pixels in the current block. One or more gradients may be calculated for the current block using the selected subset of the pixels, and the current block may be classified for the ALF based on the calculated gradients. The subset of the pixels may be selected from the pixels in the current block by skipping at least one pixel in the current block.

[0009] For example, a subset of pixels can be selected from the pixels in the current block by skipping at least one pixel vertically and / or skipping at least one pixel horizontally. If the temporal layer level of the current frame is the highest within the coding scheme, the subset of pixels can be selected from the pixels in the current block by skipping one or more pixels vertically and skipping one or more pixels horizontally. If the temporal layer level of the current frame is higher than the lowest level and lower than the highest level within the coding scheme, the subset of pixels can be selected from the pixels in the current block by skipping one or more pixels vertically, by skipping one or more pixels horizontally, or by skipping one or more pixels diagonally.

[0010] The ALF procedure may include a block classification based on each pixel of the current block. The ALF procedure may be computationally more complex than an ALF procedure that includes a block classification based on a subset of pixels. A gradient can be calculated for the current block using each pixel in the current block, and the current block can be classified for ALF based on the calculated gradient.

[0011] The decoder may receive an indication that a subset of pixels should be selected to calculate the gradient and / or an indication of how the subset of pixels should be selected.

[0012] The ALF procedure that is not computationally very complex may include block classification based on fewer gradient calculations than block classification based on each pixel in the block. For example, the ALF procedure may include block classification based on gradient calculation for a subset of pixels (e.g., based on fewer gradient calculations than for all pixels). If the temporal layer level of the current frame is higher within the coding scheme, the calculation of gradients in at least one of the vertical, horizontal, or diagonal directions of the pixels of the current block may be skipped. If the temporal layer level of the current frame is the lowest within the coding scheme, the gradients in each of the vertical, horizontal, or diagonal directions of the pixels of the current block may be calculated.

[0013] The ALF procedure that is not computationally very complex may include selectively skipping block classification. If the temporal layer level of the current frame is higher than some temporal layer levels within the coding scheme, block classification may be skipped for the current frame. If the temporal layer level of the current frame is the lowest within the coding scheme, block classification may be performed for the current frame.

[0014] The ALF procedure that is not computationally very complex may conditionally skip ALF filtering. One or more gradients may be calculated for the current block of the current frame using the pixels of the current block. The sum of the gradients is determined for the current block and may then be compared to a threshold to determine whether ALF should be made unavailable for the current block. If the sum of the multiple gradients is less than the threshold, ALF may be made unavailable for the current block, and the current block may be reconstructed with ALF made unavailable.

Brief Description of the Drawings

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14A

Figure 14B

Figure 14C

Figure 14D

DETAILED DESCRIPTION OF THE INVENTION

[0016] Next, a detailed description of exemplary embodiments will be given with reference to various figures. Although this description provides detailed examples of possible implementations, it should be noted that the details are exemplary and do not limit the scope of application at all.

[0017] FIG. 1 shows an example of a block-based hybrid video encoding system. The input video signal 302 can be processed block by block. For example, in order to efficiently compress a high-resolution (e.g., 1080p or higher) video signal, an extended block size (e.g., coding unit (CU)) can be used (e.g., in HEVC). The CU can be, for example, 64×64 pixels. The CU can be partitioned (e.g., into prediction units (PU)). Separate (e.g., the same or different) prediction procedures can be applied to the PU. Spatial prediction 360 and / or temporal prediction 362 can be performed, for example, for each input video block (e.g., macroblock (MB) or CU).

[0018] Spatial prediction (e.g., intra prediction) can predict a current video block by using pixels from samples of adjacent blocks (e.g., reference samples) that have already been coded in the same video picture / slice (e.g., the same). Spatial prediction can reduce spatial redundancy that may be inherent in the video signal. Temporal prediction (e.g., inter prediction or motion compensation prediction) can predict a current video block by using, for example, reconstructed pixels from a previously coded video picture. Temporal prediction can reduce temporal redundancy that may be inherent in the video signal. A temporal prediction signal for a given video block can be signaled, for example, by one or more motion vectors (MVs), which can indicate the amount and direction of motion between the current block and the reference block. A reference picture index (e.g., for each video block) can be sent, for example, when multiple reference pictures are supported (e.g., for H.264 / AVC or HEVC). The reference index can be used to identify the reference picture from which the temporal prediction signal came (e.g., in the reference picture store 364).

[0019] The mode decision block 380 in the encoder may select a prediction mode (e.g., the best) based on, for example, a rate distortion optimization procedure (e.g., after spatial and / or temporal prediction). The prediction block may be subtracted from the current video block 316. The prediction residue may be decorrelated (e.g., using the transform 304) and quantized 306. The quantized residue coefficients may be inverse quantized 310 and inverse transformed 312, for example, to form the reconstructed residue. The reconstructed residue may be added back to the prediction block 326, for example, to form the reconstructed video block. In-loop filter processing 366 (e.g., a deblocking filter and / or an adaptive loop filter) may be applied to the reconstructed video block, for example, before it is placed in the reference picture store 364 and used to code future video blocks. The entropy coding unit 308 may output a video bitstream 320 by compressing and packing, for example, coding mode (e.g., inter or intra), prediction mode information, motion information, and / or quantized residue coefficients. The encoder may include, for example, a WTRU, a processor of the WTRU, etc. as described herein.

[0020] Figure 2 shows an example of a block-based video decoder. The video bitstream 202 can be unpacked and entropy decoded in the entropy decoding unit 208. Coding mode and prediction information can be sent, for example, to the spatial prediction unit 260 to form a prediction block (e.g., when intra-coded) or to the temporal prediction unit 262 (e.g., when inter-coded). Residual transform coefficients can be provided, for example, to the inverse quantization unit 210 and the inverse transform unit 212 to reconstruct a residual block. The prediction block and the residual block can be summed, for example, at addition 226. In-loop filter processing can be applied, for example, to the reconstructed block before it is stored in the reference picture store 264. The video reconstructed in the reference picture store 264 can be sent out, for example, to drive a display device and / or to predict future video blocks. The decoder can include, for example, a WTRU, a processor of the WTRU, etc. (as described herein).

[0021] Motion information (e.g., MV and reference picture index) can be determined by the encoder and sent (e.g., explicitly) to the decoder. A significant amount of overhead can be spent coding motion parameters for inter-coded blocks. The overhead of signaling motion information can be reduced, for example, by the coding mode (e.g., FRUC). For example, when the FRUC coding mode is available for a CU, signaling of the MV and / or reference picture index can be skipped. Information can be derived (e.g., at the decoder side) by, for example, template matching techniques or bidirectional matching techniques.

[0022] The in-loop filter can be employed in an encoder and / or a decoder. The in-loop filter can include one or more of a deblocking filter, sample adaptive offset, or ALF. The ALF can be a Wiener filter that can be applied to a reconstructed block (e.g., to minimize the mean squared error between the original block and the reconstructed block). The ALF can be trained in the encoder. The ALF parameters can be signaled in the bitstream. The ALF can be applied to the luma and / or chroma components. The ALF can be applied to the luma component (e.g., luma ALF) and / or the chroma component (e.g., chroma ALF). The luma ALF and the chroma ALF can be trained separately. The luma component and the chroma component may not share a common ALF.

[0023] The chroma ALF can have fewer options compared to the luma ALF. For example, the luma ALF can be selected among three different filter shapes: 5×5 diamond, 7×7 diamond, and 9×9 diamond. FIG. 3 shows an example of the ALF shapes of (a) 5×5 diamond shape, (b) 7×7 diamond shape, and (c) 9×9 diamond shape. The chroma ALF can use a 5×5 diamond-shaped filter (e.g., can always use it). In the case of the luma ALF, the ALF can be applied to the entire frame using frame-level ALF or can be applied block-wise using block-level ALF. Frame-level ALF can be used for the chroma ALF. In the case of the luma ALF, block classification can be performed on each 2×2 block to categorize them into one of 25 classes. Each class can use a different ALF filter. Block classification may not be performed on the chroma component. The chroma samples in a frame can be considered to belong to a class (e.g., class 0).

[0024] Block classification can be performed on the reconstructed luma samples. FIG. 4 shows an example of the ALF procedure in the encoder. Block classification can be performed on the luma component of the reconstructed frame to classify one or more (e.g., each) 2×2 blocks. The encoder can train, for example, a 9×9 frame-level luma ALF for each class using the corresponding reconstructed pixels and the original pixels in the frame. The encoder can train the 9×9 block-level ALF by testing some (e.g., all possible) ALF block depths (

[0025]

Number

[0026] ). The 9×9 frame-level and / or block-level luma ALF can be diamond-shaped. For the ALF block depth, the encoder can perform one or more (e.g., two) iterations.

[0027] For example, in the first iteration, the frame filtered by the 9×9 frame-level ALF can be used as the starting point. The encoder can determine the block partitioning for the coding tree unit (CTU) for using the block-level ALF. The encoder starts in the CTU and can recursively divide the CTU into four equal sub-blocks, for example, if the current block depth is shallower than the related CU depth and (

[0028]

Number

[0029] ). If the current block depth is shallower than the related CU depth and (

[0030]

Number

[0031] ) If not shallower, for a given block, two sum of squared differences (SSD) values can be calculated: the SSD between the ALF-filtered block and the original uncompressed block (Filt_SSD), and the SSD between the unfiltered reconstructed block and the original block (unFilt_SSD). If Filt_SSD is less than unFilt_SSD, ALF may be available for the block. If Filt_SSD is not less than unFilt_SSD, ALF may be made unavailable for the block. When a CTU in a frame is processed, the rate distortion (RD) cost of the block-filtered frame can be calculated, and the relevant ALF parameters of the block-filtered frame can be saved.

[0032] In a second iteration, a different set of ALFs can be trained using the blocks for which ALF was selected during the first iteration. The different set of ALFs can be applied as frame-level ALF to the reconstructed frame (e.g., the entire reconstructed frame). The encoder can determine a block partition for one or more (e.g., each) CTUs for using the different set of ALFs. The encoder can start at a CTU. The encoder can, for example, recursively divide a CTU into four equal sub-blocks if the current block depth is shallower than the relevant CU depth and (

[0033]

Number

[0034] ) If shallower, the CTU can be recursively divided into four equal sub-blocks. If the current block depth is shallower than the relevant CU depth and (

[0035]

Number

[0036] ) If not shallower than, for a given block, two sum of squared differences (SSD) values can be calculated: the SSD between the ALF-filtered block and the original uncompressed block (Filt_SSD), and the SSD between the unfiltered reconstructed block and the original block (unFilt_SSD). If Filt_SSD is smaller than unFilt_SSD, ALF may be available for the block. If Filt_SSD is not smaller than unFilt_SSD, ALF may be made unavailable for the block. When the CTUs in a frame are processed, the rate distortion (RD) cost of the block-filtered frame can be calculated and the relevant ALF parameters of the block-filtered frame can be saved.

[0037] The candidate block-level ALF that results in the minimum RD cost can be selected as the block-level ALF. The RD cost of the frame-level ALF and the RD cost of the block-level ALF can be compared. The frame-level and / or block-level ALF that results in a lower RD cost (e.g., the shape of the filter) can be selected. The encoder can test for other filter shapes (e.g., 7×7 filter and 5×5 filter) and / or select a filter shape with a lower RD cost. If ALF is not used, the RD cost of the selected frame-level or block-level ALF in the selected filter shape can be compared to the RD cost. A method that results in a lower RD cost can be determined. Based on the determination, the picture-level ALF flag (e.g., alf_flag) can have a value of 1 (indicating, for example, on ALF) or 0 (indicating, for example, off ALF). When luma ALF is available, chroma ALF can be calculated. The decision on whether to use chroma ALF for a frame or picture can be made based on the RD cost.

[0038] (For example, for the current frame) The test can be performed to inspect the ALF generated for the previous frame. The previous frame may belong to the same time layer as the time layer of the current frame or a time layer lower than the time layer of the current frame. If using the ALF generated for the previous frame results in the lowest RD cost, a temporal prediction flag of 1 and an index to the previous ALF can be signaled.

[0039] Block classification can be performed (e.g., for each) on the luma component of each 2×2 reconstructed block to categorize the reconstructed block into one of 25 classes. The classification index C can be derived, for example, according to Equation 1.

[0040] [Number]

[0041] D can be the directionality,

[0042] [Number]

[0043] can be the quantization value of the activity. D and

[0044] [Number]

[0045] To calculate and, for example, the gradients in the horizontal, vertical, and two diagonal directions can be calculated using a 1-D Laplacian according to Equations 2 to 5. The gradients in the horizontal, vertical, and two diagonal directions may be referred to as directional gradients.

[0046] [Number]

[0047] i and j may refer to the coordinates of the top - left sample in a 2×2 reconstructed block, and R(i, j) may indicate the reconstructed sample at coordinates (i, j). The maximum and minimum values of the horizontal and vertical gradients may be set, for example, as in Equation 6.

[0048]

Number

[0049] The maximum and minimum values of the two diagonal gradients may be set, for example, as in Equation 7.

[0050]

Number

[0051] The value of the directionality D can be derived by comparing the above - mentioned values with each other and comparing them with two threshold values t1 and t2 as follows. Step 1.

[0052]

Number

[0053] and

[0054]

Number

[0055] If both and are true, D can be set to 0. Step 2.

[0056]

Number

[0057] If so, proceed to Step 3; otherwise, proceed to Step 4. Step 3.

[0058]

Number

[0059] If so, D can be set to 2; otherwise, D can be set to 1. Step 4.

[0060]

Number

[0061] If so, D can be set to 4; otherwise, D can be set to 3.

[0062] The activity value A can be calculated, for example, according to Equation 8.

[0063]

Number

[0064] A can be further comprehensively quantized in the range of 0 to 4, and the quantization value can be

[0065]

Number

[0066] shown as.

[0067] LIC can be used for inter-prediction of content with illumination changes over time. LIC may involve deriving a scale factor "a" and / or an offset "b" by a least squares method for an inter-coded CU. For example, adjacent reconstructed samples and corresponding temporal reference samples indicated by motion information can be used in the least squares method. The encoder can check, for example, whether LIC is available for a picture based on whether the illumination change is between the current picture and the reference picture of the current picture. Histograms of the current picture and the reference picture of the current picture can be calculated at the encoder. If the histogram difference between the current picture and the reference picture is smaller than a given threshold, LIC can be made unavailable for the current picture. If the histogram difference between the current picture and the reference picture is larger than a given threshold, LIC can be made available for the current picture.

[0068] ALF can be applied as an in-loop process to reconstructed luma and / or chroma samples (e.g., in an encoder and / or decoder). The ALF procedure may include one or more of block classification or filtering. Block classification may be based on the granularity of 2×2 blocks. Filtering may be based on one or more filter sizes from the set {9×9, 7×7, 5×5}.

[0069] Depending on different ALF procedures, the computational complexity of the encoder and / or decoder can be different. Some ALF procedures may be associated with a lower computational complexity than other ALF procedures. As an example, for block classification, the computational complexity of the encoder and / or decoder can be reduced by selecting a subset of the pixels of the block. The computational complexity of the encoder and / or decoder can be reduced by selectively disabling filtering.

[0070] Executing the ALF procedure may include, for example, applying block classification and / or filtering to the blocks reconstructed in an encoder and / or a decoder. Classifying a 2×2 block may include calculating four directional gradients for the luma component of the block. Any 2×2 block may be classified. Filtering may include applying a 5×5 diamond filter (e.g., 7 taps), a 7×7 diamond filter (e.g., 13 taps), or a 9×9 diamond filter (e.g., 21 taps) to the luma component and / or applying a 5×5 diamond filter to the chroma component. The minimum filter taps may be 7. Filter taps greater than 7 may be used for the luma component. The symmetry of the ALF may be considered.

[0071] The ALF procedure, which varies in terms of computational complexity, may be used based on the temporal layer and / or gradient calculation.

[0072] The ALF procedure can be applied to a frame based on in which time layer within the coding scheme the frame is located. The coding scheme can include a hierarchical coding structure. The coding scheme can include a plurality of time layers. A time layer can include one or more frames. Each time layer can be associated with a time layer level. In a non-limiting example, the coding scheme can include four time layers at time layer levels 0, 1, 2, and 3, respectively. Frames at time layer levels 3 and 2 can be in a higher time layer within the coding scheme. A frame at time layer level 3 can be in the highest time layer within the coding scheme. Frames at time layer levels 0 and 1 can be in a lower time layer within the coding scheme. A frame at time layer level 0 can be in the lowest time layer within the coding scheme. In another non-limiting example, the coding scheme can include five time layers at time layer levels 0, 1, 2, 3, and 4, respectively. A frame at time layer level 2 can be in an intermediate time layer within the coding scheme that includes five time layers. Frames with a higher time layer level can sometimes refer to frames with a lower time layer level. There can be multiple time levels with different quality settings. For example, a frame at a lower time layer level can have better quality using more coding bits compared to a frame at a higher time layer level.

[0073] In one example, block classification in one or more of the encoder or decoder can be skipped for frames in higher temporal layers during the coding mode. Samples in the temporal frames of the upper layer can be mapped to one class. The encoder can quantize one or more higher temporal layers using a larger QP. Frames in one or more higher temporal layers (e.g., reconstructed frames) can be smooth, for example, by heavy quantization. Frames in one or more higher temporal layers can include lower gradients than frames in lower or intermediate temporal layers. For frames in one or more higher temporal layers, block classification can result in fewer classes than block classification for lower or intermediate temporal layers.

[0074] In one example, different ALF procedures can include different block classification techniques in terms of computational complexity. Block classification with reduced computational complexity can be used for frames in one or more higher temporal layers. For example, for frames in one or more higher temporal layers, the blocks of the frame can be classified into two classes. For frames in intermediate or lower temporal layers, the blocks of the frame can be classified into 25 classes.

[0075] Fewer gradients can be computed for blocks in frames in one or more higher temporal layers for block classification. For frames in one or more higher temporal layers, gradient calculation in one or more of the vertical, horizontal, or diagonal directions of the pixels of the blocks in the frame can be skipped. For example, gradient calculation in the diagonal direction of the pixels of the block can be skipped. Gradients in the horizontal and vertical directions (e.g., only those) can be computed.

[0076] Gradient calculation can be performed for blocks in a frame, for example, for block classification based on a subset of pixels in the block. FIG. 5 shows an example of gradient calculation for (a) block classification where gradients can be calculated for each pixel in a 6×6 window for each 2×2 block, (b) a window sub-sampled by two in the horizontal direction, (c) a window sub-sampled by two in the vertical direction, and (d) a window sub-sampled by two in the horizontal direction and by two in the vertical direction.

[0077] In one example, for each 2×2 block, a 6×6 pixel window (for example, as shown in FIG. 5(a)) can be used. For each pixel in this window, four gradients can be calculated. The four gradients can include gradients in the horizontal direction, the vertical direction, and two diagonal directions.

[0078] The gradient can be calculated for a subset of the pixels of a block in a frame for block classification. In one example, the gradient can be calculated for a subset of the pixels in a 6×6 pixel window for one or more 2×2 blocks (e.g., as shown in FIGS. 5(b) - 5(d)). Calculating the gradient using a subset of the pixels can reduce the computational complexity (e.g., in an encoder or decoder). For example, the subset of pixels can be selected from the pixels in a block (e.g., the current block) by skipping at least one pixel horizontally among the pixels in the block. FIG. 5(b) can show an example of calculating the gradient for a subset of the pixels in a 6×6 pixel window for a 2×2 block. As shown in FIG. 5(b), the pixels in every other column can be skipped. The number of pixels used to calculate the gradient in the 6×6 pixel window can be reduced (e.g., by half). By using a subset of the pixels (e.g., sub - sampling), the 6×6 pixel window can become a 3×6 pixel window. The gradient can be calculated for one or more (e.g., each) pixel in the 3×6 window. The gradient calculation (e.g., the number of gradients to be calculated or calculated) can be reduced by half for one or more 2×2 blocks. The gradient calculation (e.g., of Eqs. 2 - 5) can be updated based on a subset of the pixels. For example, the sub - sampling operation can be incorporated into the gradient calculation of Eqs. 2 - 5. An updated set of gradient equations for a reduced number of pixels (e.g., a subset) by half horizontally can be given by Eqs. 9 - 12.

[0079]

Number

[0080] The gradient calculated in Eqs. 9 - 12 can be multiplied by a sub - sampling factor (e.g., 2) to hold the values of the gradient range before sub - sampling.

[0081] The gradient can be calculated for a subset of pixels that are vertically subsampled. FIG. 5(c) can show an example of calculating the gradient for a subset of pixels in a 6×6 pixel window for a 2×2 block. As shown in FIG. 5(c), pixels in every other row in the vertical direction can be skipped. By using a subset of pixels (e.g., subsampling), the 6×6 pixel window can become a 6×3 window as shown in FIG. 5(c).

[0082] The gradient can be calculated for a subset of pixels that are subsampled in both the vertical and horizontal directions. FIG. 5(d) can show an example of calculating the gradient for a subset of pixels in a 6×6 pixel window for a 2×2 block. As shown in FIG. 5(d), pixels in every other row and every other column in the vertical and horizontal directions can be skipped. By using a subset of pixels (e.g., subsampling), the 6×6 pixel window can become a 6×3 window as shown in FIG. 5(d).

[0083] A subset of pixels can be used for gradient calculation in different block classification methods. For example, a subset of pixels can be selected to calculate the gradient for block classification performed on a 4×4 block with an 8×8 pixel window.

[0084] The gradient can be calculated for a subset of pixels in a block of frames belonging to a higher temporal layer (e.g., the highest temporal layer or the next highest temporal layer) and / or an intermediate temporal layer. The gradient can be calculated for a subset of pixels in a block of frames belonging to a lower temporal layer. Blocks in frames belonging to a higher temporal layer can be smoother (e.g., can have weak edges). The gradient for blocks of frames belonging to a higher temporal layer, an intermediate temporal layer, and / or a lower temporal layer can be calculated using subsampled operations.

[0085] In one example, frames in the highest temporal layer can use a subset of subsampled pixels in the vertical and horizontal directions for gradient calculation. At least one pixel in the vertical direction and at least one pixel in the horizontal direction among the pixels in the block can be skipped. For example, if the temporal layer level of the current frame is the highest level within the coding mode, the ALF procedure for the current frame can be selected. In the ALF procedure for the current frame in the highest temporal layer, the subset of pixels can be selected from the pixels in the block of the current frame by skipping every other row of pixels in the vertical direction and every other column of pixels in the horizontal direction. The gradient for the block can be calculated using the selected subset of pixels. The block can be classified for ALF based on the calculated gradient. The block can be the current block.

[0086] Frames in a lower temporal layer, an intermediate temporal layer, or the second highest temporal layer can use subsampling of pixels in a block in one direction (e.g., only the vertical direction, only the horizontal direction, or only one of the diagonal directions). The subsampling can reduce the pixels in a 6×6 pixel window by half.

[0087] For example, if the temporal layer level of the current frame is higher than the lowest level and lower than the highest level within the coding scheme, an ALF procedure for the current frame can be selected. The selected ALF procedure can be different from the ALF procedure used for the frames in the highest temporal layer. The selected ALF procedure can be different from the ALF procedure used for the frames in the lowest temporal layer. In the ALF procedure, a subset of pixels can be selected from the pixels in a block of the current frame by skipping at least one pixel in at least one of the vertical, horizontal, or diagonal directions of the pixels in the block. The gradient for the block can be calculated using the selected subset of pixels. The block can be classified for ALF based on the calculated gradient.

[0088] If the temporal layer level of the current frame is the lowest level within the coding scheme, an ALF procedure for the current frame can be selected. In the ALF procedure for the current frame in the lowest temporal layer, the gradient for the block can be calculated using each pixel in the block. The block can be classified for ALF based on the calculated gradient.

[0089] An encoder or a decoder can determine or select an ALF procedure for a frame, for example, based on the temporal layer level of the frame (e.g., the current frame). The encoder or the decoder can determine whether a subset of pixels should be selected to calculate the gradient for block classification. The encoder or the decoder can determine how a subset of pixels should be selected.

[0090] The encoder may signal to the decoder whether subsampling should be used for a frame. The encoder may signal which subsampling should be used for a frame. In one example, an indication of whether subsampling should be used and / or an indication of which subsampling scheme should be used may be signaled for each frame. For example, the indication may be a syntax element signaled in the bitstream.

[0091] The encoder may determine or select the ALF procedure by comparing the RD cost. The encoder may perform ALF without subsampling for a frame (e.g., as described herein). If ALF is selected or made available for a frame, the encoder may perform different ALF procedures and / or calculate the RD cost associated with the ALF procedure. For example, the encoder may repeat the ALF process using different subsets of pixels for each of the subsampling schemes (e.g., as described herein) and / or compare the RD costs associated with each subsampling scheme.

[0092] If some conditions are met, the encoder may signal an indication. Equation 13 shows an example of the conditions. RD minSub <ωRD ref , where RD minSub =min(RD V ,RD H ,RD HV ) Equation 13

[0093] In Equation 13, RD ref and RD minSubcan represent, respectively, the RD cost of a method (e.g., a reference method) and the smallest RD cost among different subsampled methods. ω can represent a scale factor greater than 1. If one of the conditions of Equation 13 is satisfied, the encoder can signal one or more of a subsampling flag having a value of 1 or an index to a subsampling method. The signaling can be performed at the frame level in the picture parameter set or in the slice header. In one example, a prefix code can be constructed as an index for each subsampling method. FIG. 6 shows an example of constructing a prefix code for different subsampling methods. As shown in FIG. 6, a determination can be made as to whether vertical subsampling is used. If vertical subsampling is used, the index or a digit of the index can be set to 0. If vertical subsampling is not used, the index or a digit of the index can be set to 1. A determination can be made as to whether horizontal subsampling is used. If horizontal subsampling is used, the index or a digit of the index can be set to 0. If horizontal subsampling is not used, the index or a digit of the index can be set to 1. Table 1 can show the obtained index map. If none of the conditions in Equation 13 are satisfied, a subsampling flag having a value of 0 can be signaled in the bitstream.

[0094] Table 1 can illustrate an example of indices for different subsampling methods.

[0095]

Table 1

[0096] The decoder may receive an indication of whether sub - sampling should be used for a frame and / or an indication of which sub - sampling should be used for the frame. The decoder may determine, based on the indication, whether sub - sampling should be used for the frame and / or which sub - sampling should be used for the frame. The decoder may perform ALF based on the indicated sub - sampling scheme. The decoder may skip sub - sampling based on an indication that sub - sampling should be skipped for the frame.

[0097] In one example, the value of ω in Equation 13 may be set to 1. For example, a value of ω equal to 1 may indicate that no bias is given to the reference ALF scheme. Sub - sampling may be applied (e.g., only applied) to ALF block classification and / or may not necessarily increase the RD cost of the filtered frame. In one example, the value of ω in Equation 13 may be set to less than 1. A value of ω less than 1 may give a bias to the reference ALF scheme.

[0098] ALF procedures that differ in terms of computational complexity may include various filtering operations. For example, an ALF procedure with reduced computational complexity may skip one or more filtering operations.

[0099] To reduce computational complexity, frame adaptive ALF skip can be used. For example, as shown in FIG. 4, the encoder may determine and / or select the ALF (e.g., luma ALF) used for the luma component for a given frame. The luma ALF may be the luma ALF that generates the minimum RD cost (e.g., the best luma ALF) among a plurality of luma ALFs. The encoder may compare the RD cost of the filtered frame using the determined luma ALF with the RD cost of the unfiltered frame. The encoder may make a picture level decision as to whether to use the luma ALF for the frame.

[0100] If the luma ALF is not used (e.g., not selected) for the luma component of the frame, the encoder may not test the chroma ALF for the chroma component of the frame. The encoder may test for the temporally predicted ALF. The encoder may determine (e.g., select or choose) the temporally predicted ALF based on the test. The selected temporally predicted ALF may result in the minimum RD cost among the temporally predicted ALF and / or the ALF from the current frame.

[0101] FIG. 7 shows an example of an ALF procedure with reduced computational complexity using frame adaptive ALF skip. The ALF may be skipped or made unavailable for frames in higher temporal layers. The QP may be larger for frames in higher temporal layers. In one example, the ALF may be skipped or made unavailable for frames in the highest temporal layer (e.g., only that one).

[0102] As shown in FIG. 7, block classification can be performed for luma samples in the ALF procedure. Frame-level luma ALF can be performed (starting, for example, from a 9×9 diamond-shaped filter). Block-level luma ALF can be performed (starting, for example, from a 9×9 diamond-shaped filter). The luma filter shape can be determined. An appropriate luma method (for example, the luma method with the minimum RD cost) can be determined. Block classification can be performed on the luma component of the reconstructed frame for classifying one or more (for example, each) 2×2 blocks. The encoder can train, for example, a 9×9 frame-level luma ALF for each class using the corresponding reconstructed pixel and the original pixel in the frame. The encoder can train the 9×9 block-level ALF by testing several (for example, all possible) ALF block depths (

[0103]

Number

[0104] ). The 9×9 frame-level and / or block-level luma ALF can be diamond-shaped. The encoder can test other filter shapes (for example, 7×7 and 5×5 filters) and / or select a filter shape with a lower RD cost. If ALF is not used, the RD cost of the selected frame-level or block-level ALF in the selected filter shape can be compared with the RD cost. A method that results in a lower RD cost can be determined.

[0105] The sum of the squared errors (SSE) of the related filter-processed frame (ALF_SSE) can be calculated with respect to the original frame (for example, by the encoder). The sum of the squared errors (SSE) of the related filter-processed frame (ALF_SSE) can be compared with the SSE of the unfiltered frame (Orig_SSE) as shown in Equation 14. Orig_SSE ≤ T * ALF_SSE Equation 14 T can be a bias factor. In one example, T can be set to 1.003. If Orig_SSE is less than or equal to the result of the product of ALF_SSE and the bias factor T, ALF can be made unavailable for use for the luma and chroma components of a given frame. If Orig_SSE is greater than the result of the product of ALF_SSE and the bias factor, the encoder can enable the use of ALF for the luma component of the frame. A chroma filter can be derived and it can be determined whether to apply the chroma ALF to the chroma component of the frame. The temporally predicted ALF can be checked.

[0106] If the frame distortion of the unfiltered frame (Orig_SSE) is within a given tolerance from the frame distortion of the ALF-filtered frame (ALF_SSE), the bias factor can enable the encoder to make ALF unavailable. The bias factor can enable more frames to be processed (e.g., encoded or decoded) in a state where ALF is unavailable than when the bias factor is not used. The bias factor can be set to a value that chooses to skip ALF. For example, if the bias factor is set to a larger value, skipping ALF can be chosen. The decoder can skip or make ALF unavailable, for example, upon receiving an indication from the encoder.

[0107] The bias factor can be set differently for frames in different temporal layers. The bias factor can be set to a value that chooses to skip ALF for frames in a higher temporal layer. For example, the bias factor for frames in a higher temporal layer within a coding scheme can be set larger than the bias factor for frames in a lower temporal layer.

[0108] The bias factor can be set differently for frames related to different QPs used for picture coding. The bias factor can be set to a value that chooses to skip ALF for frames related to a larger coding QP. For example, the bias factor for frames related to a larger coding QP can be set to be larger than the bias factor for frames related to a smaller coding QP.

[0109] The bias factor can be set differently for frames related to different frame resolutions. The bias factor can be set to a value that chooses to skip ALF for frames with a smaller resolution. For example, the bias factor for frames with a larger resolution can be set to be relatively smaller than that for frames with a smaller resolution.

[0110] The bias factor can be set differently for frames related to different illumination changes. The bias factor can be set based on whether there is an illumination change between the current frame and some (e.g., all) of the reference frames of the current frame. If the current frame has no illumination change from all of the reference frames of the current frame, the bias factor can be set to a value that chooses to skip the ALF. For example, if no illumination change is detected between the current frame and all of the reference frames of the current frame, the bias factor can be set to a larger value. When the bias factor is set to a larger value, skipping the ALF may be chosen. If an illumination change is detected between the current frame and some (e.g., all) of the reference frames of the current frame, the bias factor can be set to a smaller value. The illumination change can be detected through a histogram-based method used by local illumination compensation (LIC). If LIC is made available by the codec, the bias factor can be determined based on whether LIC is available for the current frame. For example, if LIC is available for the current frame, the bias factor for the ALF can be set to a smaller value. If LIC is not available for the current frame, the bias factor for the ALF can be set to a larger value.

[0111] Pixel-adaptive ALF skip can be used for an ALF procedure with reduced computational complexity. The ALF can include a Wiener filter that minimizes the mean squared error between the reconstructed sample and the original sample. If the reconstructed block is smooth and / or has no gradient, the ALF can be made unavailable in an ALF procedure with reduced computational complexity.

[0112] As described herein, during block classification, the gradient g h , g v , g d0 , and g d1can be calculated for each (e.g., each) 2×2 block. In one example, this gradient information can be employed to make a decision as to whether to skip the ALF filtering for a 2×2 block.

[0113] FIG. 8 shows an example of block classification for a 2×2 block using an ALF on / off decision that uses the sum of gradients g sum and a threshold T G . For example, an encoder and / or a decoder can make a decision at a block classification stage. The sum of gradients (e.g., gradient sum) can be calculated for a 2×2 block as shown in Equation 15. g sum = g v + g h + g d0 + g d1 Equation 15 H and W can respectively indicate the height and width of the video. A buffer having dimensions (e.g., ALF_mask)

[0114] [Number]

[0115] can be maintained in the encoder and / or decoder. The buffer can be used to store information as to whether the reconstructed pixels (e.g., reconstructed luma component) of the luma in a 2×2 block have been ALF filtered. The sum of gradients can be compared with a threshold T G . If the sum of gradients is less than the threshold T G , the ALF-on indication (e.g., ALF_mask flag) associated with the 2×2 block can be set to 0. If the sum of gradients is not less than the threshold T G , the ALF-on indication associated with the 2×2 block can be set to 1. When the ALF-on indication associated with the 2×2 block is set to 1, ALF can be performed for the 2×2 block (e.g., by the decoder).

[0116] A determination can be made as to whether the ALF on indication related to a 2×2 block is equal to 1. For example, the determination can be made before executing the ALF filter processing step. FIG. 9 shows an example of applying ALF to a 2×2 block. As shown in FIG. 9, it can be checked whether the ALF on indication related to a given 2×2 block has a value of 1. The filter processing can be executed when certain conditions are met (for example, only then). If the conditions are not met, the filter processing can be skipped for the block (for example, the current block). As shown in FIG. 9, a determination can be made as to whether ALF_mask is equal to 1. If ALF_mask is not equal to 1, ALF may not be applied (for example, can be skipped). If ALF_mask is equal to 1, ALF can be applied to the 2×2 block.

[0117] Threshold T G can be determined in advance and / or can be fixed. In one example, threshold T G can be fixed and / or can be derived based on offline training. In the case of a coding method using time layers, the threshold can be offline trained for each time layer. For example, the threshold can be offline trained for each time layer. The threshold (for example, for each time layer) may or may not be signaled. The threshold can be determined in advance and fixed in both the encoder and / or the decoder.

[0118] Threshold T GThe value of sum can be derived, for example, based on the desired ratio of ALF skips. In one example, the threshold can be trained during the encoding process. In a coding scheme that uses time layers, the threshold can be trained for each time layer. For example, the first frame of each time layer can be used to train the threshold used by subsequent frames in the same time layer. This threshold can be determined based on the desired ratio of ALF skips. The desired ratio of ALF skips can be specified as an input parameter to the encoder. FIG. 10 shows the gradient sum (g sum ) value of the frame G using a histogram to train the threshold T

[0119] An example is shown. As shown in FIG. 10, the parameters of the histogram can be selected. The parameters of the histogram can include one or more of the bin size, the total number of bins, and the maximum value of the histogram. During the block classification of a frame (e.g., the frame under consideration), the histogram can be constructed using the gradient sum of 2×2 blocks in the frame. The resulting histogram can represent the count of occurrences for each (e.g., bin) of the gradient sum. The count related to the ALF skip ratio can be determined, for example, as shown in Equation 16, using the skip ratio (ALF_skip_percentage) and the total count of the histogram (total_count). ALF_skip_count = ALF_skip_percentage * total_count Equation 16

[0120] The histogram can be examined to determine at which minimum bin index the cumulative count becomes greater than or equal to ALF_skip_count. The examination can start from bin index 0. The value of the selected bin can be used as the threshold T G . The value of the selected bin can include the center value or the maximum value of the selected bin. The value of the selected bin (e.g., the threshold T G) can be used for ALF skip decision for subsequent frames in the same temporal layer. Threshold T G may or may not be signaled. Threshold T G can be derived in the encoder and / or decoder.

[0121] Threshold T G can be calculated for each (e.g., every) frame in the video sequence. The encoder signals the threshold T once per frame in the bitstream G The decoder can receive the threshold T and / or determine whether the ALF on indication associated with a 2×2 block has a value of 1. G

[0122] The encoder can signal the skip ratio (e.g., only once per sequence). In an example where the encoder can signal the skip ratio only once per sequence, histogram generation can be performed on a block or group of blocks basis in the encoder and / or decoder. The threshold can be calculated in the decoder based on the signaled skip ratio.

[0123] Whether to skip ALF can be determined based on the RD cost. The encoder can use the RD cost to select between applying ALF for a frame and using pixel-adaptive ALF skip. FIG. 11 shows an exemplary rate-distortion (RD)-based approach used to select between applying ALF and using pixel-adaptive ALF skip with a skip ratio of "A". The threshold can be signaled per frame for the skip method. If the threshold is to be derived in the decoder, signaling may not be performed.

[0124] The bias factor β can be used to enable more frames to be processed (e.g., encoded or decoded) using pixel-adaptive ALF skip than when the bias factor is not used. The bias factor can be set to a value that selects the one that enables pixel-adaptive ALF skip. For example, when the bias factor is set to a larger value, pixel-adaptive ALF skip can be selected.

[0125] As shown in FIG. 11, the encoder can perform ALF and / or calculate the associated RD cost (RD orig ). The encoder can apply ALF for a specified skip ratio “A” (e.g., execute the ALF process). Applying ALF for a specified skip ratio “A” can include calculating a threshold T G and / or performing pixel-adaptive ALF skip. The encoder can calculate the associated RD cost (RD skipA ). A bias factor β having a value of 1 or more can be determined (e.g., selected). The bias factor β can be multiplied by RD orig . The product of multiplying the bias factor β by RD orig can be compared with the associated RD cost RD skipA . If the product is greater than the associated RD cost RD skipA , a flag (e.g., alf_skip_flag) can have a value of 1 in the bitstream. The alf_skip_flag having a value of 1 can indicate that pixel-adaptive ALF skip can be applied to the current frame. The threshold T G for the specified skip ratio “A” can be signaled. If the product is less than or equal to the associated RD cost RD skipA , the alf_skip_flag can have a value of 0 in the bitstream. The alf_skip_flag having a value of 0 can indicate that ALF can be applied to the current frame.

[0126] The skip ratio of "A" and the skip ratio of "B" can be used (e.g., by an encoder) to select between applying ALF and using pixel-adaptive ALF skip. In one example, the technique described in FIG. 11 can be extended to select from two or more skip ratios. FIG. 12 shows an exemplary RD cost-based selection process when two skip ratios ("A" and "B") are specified for pixel-adaptive ALF skip. The skip ratios can be tested in ascending order, e.g., when "A" is smaller than "B", "A" is tested first. As shown in FIG. 11, the encoder can perform ALF and / or calculate the associated RD cost (RD orig ). The encoder can apply ALF for the specified skip ratio "A" (e.g., execute the ALF process). Applying ALF for the specified skip ratio "A" can include calculating a threshold T G and / or executing pixel-adaptive ALF skip. The encoder can calculate the associated RD cost (RD skipA ). A bias factor β having a value of 1 or more can be determined (e.g., selected). The bias factor β can be used such that more frames are processed (e.g., encoded or decoded) using pixel-adaptive ALF skip than when the bias factor is not used.

[0127] The bias factor β can be multiplied by RD orig . The product of multiplying the bias factor β by RD orig can be compared with the associated RD cost RD skipA . If the product is less than or equal to the associated RD cost RD skipA , the alf_skip_flag can have a value of 0 in the bitstream, indicating that ALF should be applied to the current frame. If the product β × RD orig is greater than RD skipA , the encoder can test the skip ratio "B" and / or calculate the associated RD cost (RD skipB ). The product β × RD origis RD skipB If it is greater than RD, alf_skip_flag may have a value of 1, indicating that pixel-adaptive ALF skip can be performed using a threshold related to skip ratio B. The threshold related to "B" may be signaled. The product β × RD orig is RD skipB If it is equal to or the same as RD, alf_skip_flag may have a value of 1, indicating that pixel-adaptive ALF skip can be performed using a threshold related to skip ratio A. The threshold related to "A" may be signaled.

[0128] Parameters related to pixel-adaptive ALF skip can be parsed, for example, in a decoder. FIG. 13 shows an example of parsing parameters related to pixel-adaptive ALF skip in a decoder. As shown in FIG. 13, the decoder can parse the bitstream to read the parameters related to ALF. (For example, when alf_flag has a value of 1) If ALF is available for a given frame, the bitstream can be parsed to read alf_skip_flag. An alf_skip_flag having a value of 1 may indicate that pixel-adaptive ALF skip can be applied to the current frame. If alf_skip_flag has a value of 1, the threshold T G is parsed from the bitstream and / or can be used for ALF skip determination. If this flag is not equal to 1, ALF can be applied to the current frame and / or the threshold T G can be set to a default value (for example, 0).

[0129] When multiple skip ratios are specified in the encoder and / or at that time, the decoder may derive a threshold (e.g., using the techniques herein). The encoder may signal, for example, once per video sequence, a part (e.g., all) of the candidate skip ratios. When pixel-adaptive ALF skip is selected for a frame and / or at that time, the encoder may signal an index corresponding to the selected skip ratio. The decoder may derive a threshold based on the candidate skip ratios and the index corresponding to the selected skip ratio.

[0130] ALF filter tap selection may be based on the temporal layer. For the luma component of a frame, 5×5, 7×7, and 9×9 ALFs may be used. For the two chroma components, a 5×5 ALF may be used. In a coding scheme using the temporal layer, a larger QP may be assigned to frames in a higher temporal layer than to frames in a lower temporal layer. Frames in a higher temporal layer may be smoother than frames in a lower temporal layer. For a higher temporal layer, a smaller size ALF may be used. In one example, the size of the ALF used for a frame may be restricted based on which temporal layer is in the frame in the coding scheme. For example, frames in a higher temporal layer may be restricted to use a smaller size filter. In one example, in a random access configuration where five temporal layers are used, frames in the fourth temporal layer may be restricted to use a 5×5 ALF (e.g., only a 5×5 ALF). Frames in the second and third layers may be restricted to use a 5×5 and / or 7×7 ALF. Frames in the remaining two lower layers may use any of the three filter sizes.

[0131] A block-level ALF that reduces computational complexity can be provided. A block-level ALF can be generated. The encoder can compare the SSDs for the filtered blocks and the unfiltered blocks. The encoder can determine whether to enable or disable the ALF for a given block. The SSD can be calculated between the reconstructed block and the original block. The bias in the SSD comparison can be used to choose to disable the ALF for the block. Using the bias in the SSD comparison can result in fewer blocks being filtered in the decoder. The bias factor γ can be multiplied by the SSD of the filtered block (Filt_SSD). The bias factor γ can have a value greater than 1. If the product of the bias factor γ multiplied by the SSD of the filtered block Filt_SSD is greater than the SSD of the unfiltered block, the ALF can be disabled for the block.

[0132] The computational complexity of the ALF can be reduced by combining one or more of the techniques described herein. In one example, block classification that reduces computational complexity and / or filtering that reduces computational complexity can be combined. For example, in the case of a frame in the highest temporal layer, vertical subsampling can be used during gradient calculation and pixel-adaptive ALF skip can be used during filtering.

[0133] FIG. 14A is a diagram showing an exemplary communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multi-connection system that provides content such as voice, data, video, messaging, broadcast, etc. to a plurality of wireless users. The communication system 100 may enable a plurality of wireless users to access such content through sharing of system resources including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero-tail unique-word DFT-spread OFDM (ZT UW DTS-s OFDM), unique-word OFDM (UW-OFDM), resource block-filtered OFDM, filter bank multicarrier (FBMC).

[0134] As shown in FIG. 14A, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, a RAN 104 / 113, a CN 106 / 115, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, it being understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d, which may sometimes be referred to as “stations” and / or “STAs,” may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular telephones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in an industrial and / or automated processing chain context), home electronic device devices, devices operating on commercial and / or industrial wireless networks, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may sometimes be interchangeably referred to as a UE.

[0135] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks such as CN106 / 115, the Internet 110, and / or other network 112. By way of example, base stations 114a, 114b may be a base transceiver station (BTS), Node B, eNode B, home Node B, home eNode B, gNB, NR Node B, site controller, access point (AP), wireless router, etc. Although base stations 114a, 114b are each shown as a single element, it will be understood that base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0136] Base station 114a may be part of RAN104 / 113 and may also include other base stations and / or network elements (not shown) such as a base station controller (BSC), a radio network controller (RNC), a relay node, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, sometimes referred to as a cell (not shown). These frequencies may be within an authorized spectrum, an unlicensed spectrum, or a combination of an authorized spectrum and an unlicensed spectrum. A cell may provide coverage for wireless services to a particular geographic area that may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, i.e., one for each sector of the cell. In one embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0137] Base stations 114a, 114b can communicate with one or more of WTRUs 102a, 102b, 102c, 102d via an air interface 116, which can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, millimeter wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 can be established using any suitable radio access technology (RAT).

[0138] More specifically, as described above, communication system 100 can be a multi-connection system and can employ one or more channel access methods such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 114a in RAN 104 / 113 and WTRUs 102a, 102b, 102c can implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA) that can establish air interfaces 115 / 116 / 117 using Wideband CDMA (WCDMA). WCDMA can include communication protocols such as High-Speed Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA can include High-Speed Downlink (DL) Packet Access (HSDPA) and / or High-Speed UL Packet Access (HSUPA).

[0139] In one embodiment, base stations 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA) that can establish air interface 116 using Long-Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-A Pro.

[0140] In one embodiment, base stations 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access that can establish air interface 116 using New Radio (NR).

[0141] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, for example, using the dual connectivity (DC) principle. Accordingly, the air interface utilized by WTRUs 102a, 102b, 102c may be characterized by transmissions sent between multiple types of radio access technologies and / or multiple types of base stations (e.g., eNBs and gNBs).

[0142] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c may implement wireless technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), GSM Evolution for Enhanced Data Rates (EDGE), GSM EDGE (GERAN), etc.

[0143] The base station 114b in FIG. 14A can be, for example, a wireless router, a home node B, a home e-node B, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in a local area such as a workplace, home, vehicle, campus, industrial facility, (for example, an aerial corridor for use by drones), a road, etc. In one embodiment, the base station 114b and the WTRUs 102c, 102d can implement a wireless technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d can implement a wireless technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d can utilize a cellular-based RAT (such as WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a pico cell or a femto cell. As shown in FIG. 14A, the base station 114b can have a direct connection to the Internet 110. Thus, the base station 114b may not need to access the Internet 110 via the CN 106 / 115.

[0144] RAN 104 / 113 may communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, applications, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRUs 102a, 102b, 102c, 102d. The data may have various Quality of Service (QoS) requirements such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid originating calls, Internet connectivity, video distribution, etc., and / or perform high-level security functions such as user authentication. Although not shown in FIG. 14A, it should be understood that RAN 104 / 113 and / or CN 106 / 115 may communicate directly or indirectly with other RANs that employ the same radio access technology (RAT) as RAN 104 / 113 or a different RAT. For example, in addition to being connected to RAN 104 / 113, which may utilize New Radio (NR) wireless technology, CN 106 / 115 may also communicate with another RAN (not shown) that employs GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi wireless technology.

[0145] CN106 / 115 can also act as a gateway for WTRU102a, 102b, 102c, 102d to access the PSTN108, the Internet 110, and / or other networks 112. The PSTN 108 can include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 can include a global system of interconnected computer networks and devices that use common communication protocols such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and / or the Internet Protocol (IP) in the TCP / IP Internet protocol suite. The network 112 can include wired and / or wireless communication networks that are owned and / or operated by other service providers. For example, the network 112 can include another CN connected to one or more RANs that may employ the same or a different RAT than the RAN 104 / 113.

[0146] Some or all of the WTRU102a, 102b, 102c, 102d in the communication system 100 may include multimode capabilities (e.g., the WTRU102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example, the WTRU102c shown in FIG. 14A can be configured to communicate with a base station 114a that may employ a cellular-based wireless technology and with a base station 114b that may employ IEEE802 wireless technology.

[0147] Figure 14B is a system diagram showing an exemplary WTRU 102. As shown in Figure 14B, the WTRU 102 can include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, a non-removable memory 130, a removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and / or other peripheral devices 138. It should be understood that the WTRU 102 can include any sub-combination of the above elements while remaining in accordance with an embodiment.

[0148] The processor 118 can be a general-purpose processor, a dedicated processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 can perform signal coding, data processing, power control, input / output processing, and / or any other function that enables the WTRU 102 to operate in a wireless environment. The processor 118 can be coupled to a transceiver 120 that can be coupled to a transmit / receive element 122. Although Figure 14B shows the processor 118 and the transceiver 120 as separate components, it should be understood that the processor 118 and the transceiver 120 can be integrated together in an electronic package or chip.

[0149] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF signals and optical signals. It will be appreciated that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0150] Although the transmit / receive element 122 is shown in FIG. 14B as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via the air interface 116.

[0151] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs such as, for example, NR and IEEE 802.11.

[0152] The processor 118 of the WTRU 102 may be coupled to the speaker / microphone 124, keypad 126, and / or display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit) and may receive user input data therefrom. The processor 118 may also output user data to the speaker / microphone 124, keypad 126, and / or display / touchpad 128. Further, the processor 118 may access information from and store data in any suitable type of memory, such as the non-removable memory 130 and / or the removable memory 132. The non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from and store data in a memory that is not physically located on the WTRU 102, such as on a server or home computer (not shown).

[0153] The processor 118 may receive power from the power source 134 and may be configured to distribute and / or control power to other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel cadmium (NiCd), nickel zinc (NiZn), nickel metal hydride (NiMH), lithium ion (Li-ion), etc.), a solar cell, a fuel cell, and the like.

[0154] Processor 118 may also be coupled to a GPS chipset 136 configured to provide location information (e.g., longitude and latitude) regarding the current location of WTRU 102. In addition to, or instead of, information from GPS chipset 136, WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) via air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It should be appreciated that WTRU 102 may capture location information by any suitable location determination method while remaining in accordance with the embodiments.

[0155] Processor 118 may also be further coupled to other peripheral devices 138 that may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, peripheral devices 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photos and / or video), a Universal Serial Bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth® module, a Frequency Modulation (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a Virtual Reality and / or Augmented Reality (VR / AR) device, an activity tracker, etc. Peripheral devices 138 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor, a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biosensor, and / or a humidity sensor.

[0156] WTRU102 may include full-duplex radio in which some or all of the transmission and reception of signals (associated with a particular subframe for both UL (e.g., for transmission) and downlink (e.g., for reception)) can be parallel and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference either via hardware (e.g., choke) or via signal processing through a processor (e.g., a separate processor (not shown) or a via-processor 118). In one embodiment, the WRTU102 may include half-duplex radio for the transmission and reception of some or all of the signals (associated with a particular subframe for either UL (e.g., for transmission) or downlink (e.g., for reception)).

[0157] Figure 14C is a system diagram showing RAN104 and CN106, according to one embodiment. As described above, RAN104 may employ E-UTRA radio technology to communicate with WTRU102a, 102b, 102c via air interface 116. RAN104 may also communicate with CN106 from time to time.

[0158] RAN104 may include eNode Bs 160a, 160b, 160c, although it will be appreciated that RAN104 may include any number of eNode Bs while remaining in accordance with the embodiment. Each of eNode Bs 160a, 160b, 160c may include one or more transceivers for communicating with WTRU102a, 102b, 102c via air interface 116. In one embodiment, eNode Bs 160a, 160b, 160c may implement MIMO technology. Thus, eNode B 160a may use multiple antennas, for example, to transmit and / or receive wireless signals from WTRU102a.

[0159] Each of the eNodeBs 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in the UL and / or DL, etc. As shown in Figure 14C, the eNodeBs 160a, 160b, and 160c can communicate with each other via the X2 interface.

[0160] The CN 106 shown in Figure 14C can include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. Although each of the above elements is shown as part of the CN 106, it should be understood that any of these elements can be owned and / or operated by an entity other than the CN operator.

[0161] The MME 162 can be connected to each of the eNodeBs 162a, 162b, and 162c in the RAN 104 via the S1 interface and can act as a control node. For example, the MME 162 can be responsible for authenticating users of the WTRUs 102a, 102b, and 102c, activating / deactivating bearers, selecting a specific serving gateway during the initial attach of the WTRUs 102a, 102b, and 102c, etc. The MME 162 can provide control plane functions for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies such as GSM and / or WCDMA.

[0162] SGW 164 can be connected to each of the eNodeBs 160a, 160b, and 160c in RAN 104 via the S1 interface. The SGW 164 can generally route and transfer user data packets between the WTRUs 102a, 102b, and 102c. The SGW 164 can perform other functions such as anchoring the user plane during handover between eNodeBs, triggering paging when DL data is available for the WTRUs 102a, 102b, and 102c, and managing and storing the contexts of the WTRUs 102a, 102b, and 102c.

[0163] The SGW 164 can be connected to a PGW 166 that can provide the WTRUs 102a, 102b, and 102c access to a packet switched network such as the Internet 110 to facilitate communication between the WTRUs 102a, 102b, and 102c and IP-enabled devices.

[0164] The CN 106 can facilitate communication with other networks. For example, the CN 106 can provide the WTRUs 102a, 102b, and 102c access to a circuit switched network such as the PSTN 108 to facilitate communication between the WTRUs 102a, 102b, and 102c and conventional fixed communication devices. For example, the CN 106 can include or communicate with an IP gateway (such as an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. Further, the CN 106 can provide the WTRUs 102a, 102b, and 102c access to other networks 112 that can include other wired and / or wireless networks owned and / or operated by other service providers.

[0165] Although the WTRU is described as a wireless terminal in FIGS. 14A - 14D, in some representative embodiments that such a terminal can use (e.g., temporarily or permanently), wired communication is contemplated to interface with the communication network.

[0166] In an exemplary embodiment, the other network 112 can be a WLAN.

[0167] A WLAN in infrastructure basic service set (BSS) mode can have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP can have access to or an interface to another type of wired / wireless network that carries traffic in and out of the distribution system (DS) or BSS. Traffic destined for an STA originating from outside the BSS can arrive through the AP and be sent to the STA. Traffic transmitted from an STA to a destination outside the BSS can be sent to the AP for delivery to each destination. Traffic between STAs within the BSS may or may not be sent through the AP. For example, here, the source STA can send traffic to the AP, and the AP can send the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic can be sent between the source STA and the destination STA (e.g., directly between them) using direct link setup (DLS). In some exemplary embodiments, the DLS can use 802.11e DLS or 802.11z tunneled DLS (TDLS). A WLAN using independent BSS (IBSS) mode may not have an AP, and STAs within or using the IBSS (e.g., all of the STAs) can communicate directly with each other. The IBSS communication mode may sometimes be referred to herein as the "ad hoc" communication mode.

[0168] When using the 802.11ac infrastructure operation mode or a similar operation mode, the AP may transmit beacons on a fixed channel such as a primary channel. The primary channel may be of a fixed width (e.g., a 20 MHz bandwidth) or a width dynamically set via signaling. The primary channel may be the operating channel of the BSS and may be used by the STA to establish a connection with the AP. In some representative embodiments, Carrier Sense Multiple Access / Collision Avoidance (CSMA / CA) may be implemented, for example, in an 802.11 system. In CSMA / CA, STAs including the AP (e.g., any STA) may sense the primary channel. If the primary channel is sensed / detected by a particular STA and / or determined to be busy, the particular STA may back off. One STA (e.g., just one station) may transmit at a given time within a given BSS.

[0169] A high throughput (HT) STA may use a 40 MHz width channel for communication, for example, via a combination of a primary 20 MHz channel with adjacent or non - adjacent 20 MHz channels to form a 40 MHz width channel.

[0170] Very High Throughput (VHT) STAs may support channels with widths of 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz. 40 MHz and / or 80 MHz channels may be formed by combining consecutive 20 MHz channels. 160 MHz channels may be formed by combining eight consecutive 20 MHz channels or by combining two non - consecutive 80 MHz channels, sometimes referred to as an 80 + 80 configuration. In the 80 + 80 configuration, data may be passed through a segment parser that can split the data into two streams after channel encoding. Inverse Fast Fourier Transform (IFFT) processing and time - domain processing may be performed separately for each stream. The streams may be mapped onto two 80 MHz channels and the data may be transmitted by the transmitting STA. At the receiver of the receiving STA, the operations described above for the 80 + 80 configuration may be reversed and the combined data may be sent to the Media Access Control (MAC).

[0171] Sub - 1 GHz operation modes are supported by 802.11af and 802.11ah. The channel operating bandwidth and carriers are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports bandwidths of 5 MHz, 10 MHz, and 20 MHz in the TV White Space (TVWS) spectrum, and 802.11ah supports bandwidths of 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz using the non - TVWS spectrum. According to an exemplary embodiment, 802.11ah may support meter - type control / machine - type communication such as MTC devices in a macro - coverage area. MTC devices may have limited capabilities including some, for example, support for some and / or limited bandwidths (e.g., only that support). MTC devices may include a battery with a battery life above a threshold (e.g., to maintain a very long battery life).

[0172] A WLAN system that can support multiple channels and channel bandwidths such as 802.11n, 802.11ac, 802.11af, and 802.11ah includes channels that can be designated as primary channels. The primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by the STA that supports the minimum bandwidth operating mode among all STAs when operating in the BSS. In the example of 802.11ah, even when the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes, the primary channel can be 1MHz wide for an STA (e.g., an MTC type device) that supports the 1MHz mode (e.g., supports only that). Carrier sensing and / or the setting of the Network Allocation Vector (NAV) can depend on the status of the primary channel. For example, if the primary channel is busy for an STA (that supports only the 1MHz operating mode), it can be considered busy to transmit across the entire available frequency band for the AP, even if most of the frequency band remains idle and available.

[0173] In the United States, the available frequency band that can be used by 802.11ah ranges from 902 MHz to 928 MHz. In South Korea, the available frequency band ranges from 917.5 MHz to 923.5 MHz. In Japan, the available frequency band ranges from 916.5 MHz to 927.5 MHz. The total available bandwidth for 802.11ah is from 6 MHz to 26 MHz depending on the country code.

[0174] FIG. 14D is a system diagram showing RAN 113 and CN 115 according to one embodiment. As described above, RAN 113 can employ NR radio technology to communicate with WTRUs 102a, 102b, 102c via air interface 116. RAN 113 may also communicate with CN 115.

[0175] RAN 113 may include gNBs 180a, 180b, and 180c, but it should be understood that RAN 113 may include any number of gNBs while remaining in accordance with the embodiments. Each of gNBs 180a, 180b, and 180c may include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 108b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Thus, gNB 180a may use multiple antennas, for example, to transmit wireless signals to and / or receive wireless signals from WTRU 102a. In one embodiment, gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB 180a may transmit multiple component carriers to WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In one embodiment, gNBs 180a, 180b, and 180c may implement coordinated multipoint (CoMP) technology. For example, WTRU 102a may receive coordinated transmission from gNB 180a and gNB 180b (and / or gNB 180c).

[0176] WTRUs 102a, 102b, and 102c may communicate with gNBs 180a, 180b, and 180c using scalable cryptographic-related transmissions. For example, the OFDM symbol interval and / or the OFDM subcarrier interval may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. WTRUs 102a, 102b, and 102c may communicate with gNBs 180a, 180b, and 180c using various or scalable length subframes or transmission time intervals (TTIs) (e.g., including a varying number of OFDM symbols and / or lasting for various lengths of absolute time).

[0177] gNBs 180a, 180b, and 180c may be configured to communicate with WTRUs 102a, 102b, and 102c in a stand-alone configuration and / or a non-stand-alone configuration. In a stand-alone configuration, WTRUs 102a, 102b, and 102c may communicate with gNBs 180a, 180b, and 180c without accessing another RAN (e.g., eNodeBs 160a, 160b, and 160c, etc.). In a stand-alone configuration, WTRUs 102a, 102b, and 102c may utilize one or more of gNBs 180a, 180b, and 180c as a mobility anchor point. In a stand-alone configuration, WTRUs 102a, 102b, and 102c may communicate with gNBs 180a, 180b, and 180c using signals in an unlicensed band. In a non-stand-alone configuration, WTRUs 102a, 102b, and 102c may communicate with / connect to gNBs 180a, 180b, and 180c while also communicating with / connecting to another RAN such as eNodeBs 160a, 160b, and 160c. For example, WTRUs 102a, 102b, and 102c may implement the DC principle to communicate with one or more gNBs 180a, 180b, and 180c and one or more eNodeBs 160a, 160b, and 160c substantially simultaneously. In a non-stand-alone configuration, eNodeBs 160a, 160b, and 160c may act as a mobility anchor for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c may provide additional coverage and / or throughput for serving WTRUs 102a, 102b, and 102c.

[0178] Each of gNBs 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle decisions for radio resource management, handover decisions, user scheduling in UL and / or DL, support for network slicing, dual connectivity, interconnection between NR and E-UTRA, routing of user plane data towards user plane functions (UPFs) 184a, 184b, routing of control plane information towards access and mobility management functions (AMFs) 182a, 182b, etc. As shown in FIG. 14D, gNBs 180a, 180b, and 180c can communicate with each other via the Xn interface.

[0179] CN 115 shown in FIG. 14D can include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and optionally, data networks (DNs) 185a, 185b. Although each of the above elements is shown as part of CN 115, it should be understood that any of these elements can be owned and / or operated by entities other than the CN operator.

[0180] AMF 182a and 182b can be connected to one or more of gNBs 180a, 180b, and 180c in RAN 113 via the N2 interface and can act as control nodes. For example, AMF 182a and 182b can authenticate users of WTRUs 102a, 102b, and 102c, support network slicing (e.g., handling of different PDU sessions with different requirements), select specific SMFs 183a and 183b, manage registration areas, terminate NAS signaling, perform mobility management, etc. Network slicing can be used by AMF 182a and 182b to customize the support of the CN for WTRUs 102a, 102b, and 102c based on the type of service utilized by WTRUs 102a, 102b, and 102c. For example, different network slices can be established for different use cases such as services relying on ultra-reliable low-latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services for machine-type communication (MTC) access, etc. AMF 162 can provide control plane functions for switching between RAN 113 and other RANs (not shown) employing other radio technologies such as non-3GPP access technologies like LTE, LTE-A, LTE-A Pro, and / or WiFi.

[0181] SMF 183a and 183b can be connected to AMF 182a and 182b in CN 115 via the N11 interface. SMF 183a and 183b can also be connected to UPFs 184a and 184b in CN 115 via the N4 interface. SMF 183a and 183b can select and control UPFs 184a and 184b and configure traffic routing through UPFs 184a and 184b. SMF 183a and 183b can perform other functions such as managing and allocating the UE's IP address, managing PDU sessions, enforcing policies and controlling QoS, and providing notifications for downlink data. The type of PDU session can be IP-based, non-IP-based, Ethernet-based, etc.

[0182] UPF 184a and 184b can be connected to one or more of gNBs 180a, 180b, and 180c in RAN 113 via an N3 interface that can provide WTRUs 102a, 102b, and 102c access to a packet switched network such as the Internet 110 to facilitate communication between the WTRUs 102a, 102b, 102c and an IP-enabled device. UPF 184a and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multi-home PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0183] CN 115 can facilitate communication with other networks. For example, CN 115 can include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between CN 115 and the PSTN 108. Further, CN 115 can provide WTRUs 102a, 102b, and 102c access to other networks 112 that can include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRUs 102a, 102b, and 102c can be connected to local data networks (DNs) 185a and 185b through UPF 184a and 184b via an N3 interface to UPF 184a and 184b and an N6 interface between UPF 184a and 184b and DNs 185a and 185b.

[0184] In view of FIGS. 14A-14D and the corresponding description, one or more of the functions described herein with respect to one or more of WTRUs 102a-d, base stations 114a-b, eNodeBs 160a-c, MME 162, SGW 164, PGW 166, gNBs 180a-c, AMFs 182a-b, UPFs 184a-b, SMFs 183a-b, DNs 185a-b, and / or any other device described herein may be performed by one or more emulation devices (not shown). An emulation device may be one or more devices configured to emulate one or more or all of the functions described herein. For example, an emulation device may be used to test other devices and / or to simulate network and / or WTRU functionality.

[0185] An emulation device may be designed to implement one or more tests of other devices in a laboratory environment and / or in an operator network environment. For example, one or more emulation devices may perform one or more or all of the functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices in the communication network. One or more emulation devices may perform one or more or all of the functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. An emulation device may be directly coupled to another device for testing and / or may perform tests using over-the-air wireless communication.

[0186] One or more emulation devices may perform one or more functions, including all, without being implemented / deployed as part of a wired and / or wireless communication network. For example, an emulation device may be utilized in a test scenario in a laboratory and / or in a wired and / or wireless communication network that is not deployed (e.g., for testing) in order to conduct tests on one or more components. One or more emulation devices may be test equipment. Wireless communication may be used by an emulation device to transmit and / or receive data, either directly via RF coupling and / or via an RF circuit (which may include one or more antennas).

[0187] The processes described above may be implemented in a computer program, software, and / or firmware incorporated in a computer-readable medium for execution by a computer and / or processor. Examples of computer-readable media include, but are not limited to, electronic signals (transmitted via wired and / or wireless connections) and / or computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and / or optical media such as CD-ROM disks and / or digital versatile disks (DVD). The processor associated with the software may be used to implement a radio frequency transceiver for use in a WTRU, terminal, base station, RNC, and / or any host computer.

Claims

1. 1. A video decoding device, comprising: Selecting a subset of pixels from the plurality of pixels; obtaining a sum of diagonal variations using said subset of pixels; determining a diagonal gradient based on said sum of diagonal variations; performing adaptive loop filtering (ALF) on the video block based on the diagonal gradients; Decode the picture that includes the video block. Processor configured for A video decoding device comprising:

2. 2. The video decoding device of claim 1 , wherein for the selection of the subset of pixels, pixels within the plurality of pixels are skipped in a diagonal direction, and the performing of ALF on the video block includes classifying the video block, the classification of the video block being based on the diagonal gradient.

3. 2. The video decoding device of claim 1 , wherein the processor is further configured to determine a diagonal variation using a first pixel of the subset of pixels and a second pixel of the subset of pixels, the second pixel being diagonally located from the first pixel, and the sum of diagonal variations is determined based on the diagonal variations.

4. 2. The video decoding device of claim 1 , wherein the diagonal gradient is a first diagonal gradient associated with a first diagonal direction, the processor being further configured to determine a second diagonal gradient associated with a second diagonal direction, and wherein the ALF is performed on the video block further based on the second diagonal gradient.

5. The processor, determining a first diagonal variation using a first pixel of the subset of pixels and a second pixel of the subset of pixels, the second pixel being located in a first diagonal direction from the first pixel, the sum of diagonal variations being a first sum of diagonal variations and obtained based on the first diagonal variation; determining a second diagonal variation using a third pixel of the subset of pixels and a fourth pixel of the subset of pixels, the fourth pixel being located in a second diagonal direction from the third pixel; The second diagonal variation is used to obtain a second sum of diagonal variations, and the ALF is performed further based on the second sum of diagonal variations.

2. A video decoding device according to claim 1, configured to:

6. 2. The video decoding device of claim 1 , wherein for the selection of the subset of pixels, a pixel of the plurality of pixels is skipped in a vertical direction and a pixel of the plurality of pixels is skipped in a horizontal direction, and the processor is further configured to determine a horizontal gradient and a vertical gradient using the subset of pixels, and the ALF is performed further based on the horizontal gradient and the vertical gradient.

7. 1. A video decoding method, comprising: selecting a subset of pixels from the plurality of pixels; obtaining a sum of diagonal variations using said subset of pixels; determining a diagonal gradient based on said sum of diagonal variations; performing adaptive loop filtering (ALF) on the video block based on the diagonal gradients; decoding a picture including the video block; A video decoding method comprising:

8. the selecting of the subset of pixels includes skipping pixels of the plurality of pixels in a diagonal direction; determining a diagonal variation using a first pixel of the subset of pixels and a second pixel of the subset of pixels, the second pixel being diagonally located from the first pixel, and the sum of diagonal variations being obtained based on the diagonal variations; The video decoding method of claim 7, further comprising:

9. 8. The video decoding method of claim 7, wherein the performing of ALF on the video block includes classifying the video block, the classifying the video block based on the diagonal gradient.

10. 1. A video encoding device comprising: Selecting a subset of pixels from the plurality of pixels; obtaining a sum of diagonal variations using said subset of pixels; determining a diagonal gradient based on said sum of diagonal variations; performing adaptive loop filtering (ALF) on the video block based on the diagonal gradients; Encoding a picture including the video block. Processor configured for A video encoding device comprising:

11. The processor, determining a first error associated with the picture; determining a second error based on an ALF associated with the picture; comparing the first error and the second error using a bias factor; and including in the video data an ALF indication indicative of an ALF usage status associated with the picture based on the comparison. The video encoding device of claim 10 further configured to:

12. 11. The video encoding device of claim 10, wherein for the selection of the subset of pixels, a pixel among the plurality of pixels in a diagonal direction is skipped, and the processor is further configured to determine a diagonal variation using a first pixel of the subset of pixels and a second pixel of the subset of pixels, the second pixel being diagonally located from the first pixel, and the sum of diagonal variations is obtained based on the diagonal variation.

13. 1. A video encoding method, comprising: selecting a subset of pixels from the plurality of pixels; obtaining a sum of diagonal variations using said subset of pixels; determining a diagonal gradient based on said sum of diagonal variations; performing adaptive loop filtering (ALF) on the video block based on the diagonal gradients; encoding a picture including the video block; A video encoding method comprising:

14. determining a first error associated with the picture; determining a second error based on an ALF associated with the picture; comparing the first error and the second error using a bias factor; including in the video data an ALF indication indicating an ALF usage status associated with the picture based on the comparison; The video encoding method of claim 13 comprising:

15. the diagonal gradient is a first diagonal gradient associated with a first diagonal direction; determining a second diagonal gradient associated with a second diagonal direction, wherein the ALF is performed on the video block further based on the second diagonal gradient. The video encoding method of claim 13 further comprising:

Images