Encoder, decoder, and corresponding non-blocking filter adaptation method

Abstract

The present invention provides a deblocking method for deblocking transform block boundaries in coding blocks in image encoding and / or decoding, wherein the coding block including transform blocks is coded in an inter-frame prediction mode, and the transform blocks include a first transform block and a second transform block adjacent to the first transform block, the method comprising: determining a value of a boundary strength (BS) parameter for the boundary between the first transform block and the second transform block to be a first value when the boundary between the first transform block and the second transform block is a transform block boundary (such as an SBT boundary) and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients (one or more non-zero residual transform coefficients); and performing a deblocking filtering process on the boundary between the first transform block and the second transform block based on at least the value of the boundary strength parameter.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This patent application claims priority based on U.S. Provisional Patent Application No. 62 / 797,163, filed on January 25, 2019. The content of the above - mentioned patent application is hereby incorporated by reference in its entirety into this specification.

[0002] [Technical Field] Multiple embodiments of the present disclosure generally relate to the field of video processing, and in particular, to encoders, decoders, and corresponding methods for non - blocking filter adaptation, and more particularly, to sub - block conversion and non - blocking filters for conversion block boundaries caused by SBT coding tools.

Background Art

[0003] Video coding (video encoding and decoding) is used in a wide range of digital video applications, such as broadcast digital TV, video transmission via the Internet and mobile networks, real - time interactive applications such as video chat and video conferencing, DVDs and Blu - ray discs, video content acquisition and editing systems, and video cameras for security applications.

[0004] The amount of video data required to depict a relatively short video can be quite large, and that amount of video data can be problematic when that data is streamed or otherwise communicated over a communication network with limited bandwidth capacity. In this way, video data is generally compressed before being communicated over modern communication networks. Since memory resources can be limited, the size of the video can also be an issue when storing the video in a storage device. Video compression devices typically use software and / or hardware at the source to code the video data before transmission or storage, thereby reducing the amount of data required to represent a digital video image. The compressed data is then received at the destination by a video decompression device that decodes the video data. Improved compression and decompression techniques that improve the compression ratio without sacrificing much of the image quality are desirable because network resources are limited and the demand for higher video quality is constantly increasing.

[0005] Block-based image coding schemes have in common that they may produce edge artifacts along multiple block edges. Those artifacts are due to the coding of multiple coding blocks being independent. Those edge artifacts are generally easily visible to the user. The goal of block-based image coding is to reduce edge artifacts below a visibility threshold. This goal is achieved by performing deblocking filtering. Such deblocking filtering is performed on the decoding side, on the one hand, to remove visible edge artifacts and also on the encoding side so that those edge artifacts are not coded at all in the image.

[0006] On the one hand, the conventional approach does not consider that discontinuities may occur when using inter-frame prediction between a plurality of transform / coding blocks (such as a plurality of transform / coding blocks having chroma samples or chroma components) where edges occur between the blocks. In this way, non-blocking filtering is difficult or may not even yield the expected results. Summary of the Invention

[0007] In view of the above problems, multiple embodiments of the present application provide a non-blocking filter device, an encoder, a decoder, and a corresponding method that can even reduce or remove blocking artifacts across the boundaries between a plurality of transform / coding blocks (such as a plurality of transform blocks having chroma samples) using inter-frame prediction, with the aim of improving coding efficiency.

[0008] In particular, in relation to inter-frame prediction, a sub-block transform (SBT) coding tool is introduced, and the SBT coding tool is applied to both luminance samples and chroma samples. Multiple embodiments of the present application also provide other non-blocking filter devices, other encoders, other decoders, and corresponding methods that can even reduce or remove blocking artifacts that the sub-block transform (SBT) coding tool may cause, with the aim of improving coding efficiency.

[0009] Multiple embodiments of the present invention are defined by further advantageous implementations of multiple embodiments according to the features of the independent claims and the features of the dependent claims.

[0010] Multiple specific embodiments are outlined in the appended independent claims, together with other embodiments in the dependent claims.

[0011] The above and other objects are achieved by the subject matter of the independent claims. Further implementations are apparent from the dependent claims, the specification, and the drawings.

[0012] According to a first aspect of the present disclosure, in image encoding and / or image decoding, a non-blocking method for non-blocking a transform block boundary (internal edge) in a coding block is provided. (The coding block is divided (distributed) into a plurality of transform blocks in an inter-frame prediction process, particularly when sub-block transform is enabled, the current coding unit is divided into a plurality of transform units, etc.) The coding block is coded (predicted) in an inter-frame prediction mode, the coding block includes a plurality of transform blocks, (for example, the transform blocks including p0 and q0 are adjacent in the vertical direction or the horizontal direction, etc.) the plurality of transform blocks include a first transform block and a second transform block adjacent to the first transform block, and the method includes: Determining (or setting) a value of a boundary strength (BS) parameter for the boundary between the first transform block (such as the first transform block using inter-frame prediction) and the second transform block (such as the second transform block using inter-frame prediction) to be a first value when the boundary between the first transform block and the second transform block is a transform block boundary and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients (one or more non-zero residual transform coefficients); Performing a non-blocking filtering process on the boundary between the first transform block and the second transform block based on at least the value of the boundary strength parameter.

[0013] It is possible to understand that the first conversion block and the second conversion block on both sides of the conversion block boundary use inter-frame prediction. In one example, the samples of the coding block are chroma samples. In other examples, the coding block has luminance samples and chroma samples. Accordingly, in one example, the samples of the first conversion block and the second conversion block are chroma samples. In other examples, both the first conversion block and the second conversion block have luminance samples and chroma samples. Specifically, according to the prior art, the deblocking filtering process is applied to the coding sub-block edges and conversion block edges of the video, but excludes the edges in the chroma components that use inter-frame prediction on both sides of the edge. However, according to the first aspect of the present disclosure, it is possible to deblock the edges in the chroma components that use inter-frame prediction on both sides of the edge.

[0014] In addition to the boundary strength (BS) parameter for the boundary between the first conversion block and the second conversion block, it is possible to understand that other parameters may be considered for the deblocking filtering process. That is, depending on the specific filtering decision result, in some cases, no samples may need to be modified, or in other cases, deblocking filtering may be performed where only one sample in each adjacent row or each adjacent column perpendicular to the boundary is modified.

[0015] In the present disclosure that may be applied to a prediction unit (PU), a coding unit (CU), etc., it should be noted that the terms "block", "coding block", or "image block" are used. In general VVC, the transform unit and the coding unit are substantially aligned, except for a few scenarios where sub-block transform (SBT) is used. In the present disclosure, it is possible to understand that the terms "block / image block / coding block" may be exchanged with each other. In the present disclosure, the terms "sample / pixel" may be exchanged with each other.

[0016] In the prior art, consideration has not been given to filtering the boundaries between transform blocks / coding blocks that use inter-frame prediction among the chroma components. On the other hand, according to the present invention, the filtering process is improved to reduce block artifacts at the boundaries between a plurality of transform blocks / coding blocks that use inter-frame prediction and have chroma samples.

[0017] In one possible implementation of the method according to the first aspect itself, the first transform block has residual data, and the second transform block does not have residual data, or the first transform block does not have residual data, and the second transform block has residual data.

[0018] of the first aspect itself In one possible implementation of the method according to any of the preceding implementations, the transform block is a sub-block transform SBT transform block.

[0019] In one possible implementation of the method according to any of the preceding implementations of the first aspect or the first aspect itself, the number of transform blocks is 2 or 3 or other values.

[0020] In one possible implementation of a method according to any preceding implementation of the first aspect or the first aspect itself, the boundary between the first conversion block and the second conversion block is a sub-block conversion SBT boundary.

[0021] In one possible implementation of a method according to any preceding implementation of the first aspect or the first aspect itself, when the boundary between the first conversion block and the second conversion block is a conversion block boundary and at least one of the first conversion block and the second conversion block has one or more non-zero conversion coefficients, the step of determining the value of the boundary strength parameter for the boundary between the first conversion block and the second conversion block to be a first value is including the step of determining the value of the boundary strength parameter for the boundary between the first conversion block and the second conversion block to be a first value when the boundary between the first conversion block and the second conversion block is a sub-block conversion SBT boundary and at least one of the first conversion block and the second conversion block has one or more non-zero conversion coefficients.

[0022] In one possible implementation of a method according to any preceding implementation of the first aspect or the first aspect itself, the conversion block further includes a third conversion block adjacent to the second conversion block, the method comprising the step of determining the value of the boundary strength parameter for the boundary between the second conversion block and the third conversion block to be a first value when the boundary between the second conversion block and the third conversion block is a sub-block conversion SBT boundary and at least one of the second conversion block and the third conversion block has one or more non-zero conversion coefficients, or The boundary between the second conversion block and the third conversion block is a sub-block conversion SBT boundary, and when both the second conversion block and the third conversion block have zero conversion coefficients (all zero conversion coefficients), determining a value of a boundary strength parameter for the boundary between the second conversion block and the third conversion block to be a second value. further includes.

[0023] In one possible implementation of the method according to any preceding implementation or the first aspect itself, the first value is 1.

[0024] In one possible implementation of the method according to any preceding implementation of the first aspect, the second value is 0.

[0025] In one possible implementation of the method according to any preceding implementation or the first aspect itself of the first aspect, the conversion block boundary between the first conversion block and the second conversion block is unblocked (filtered) only if it is aligned (overlaps) with an n×n sample grid, where n is an integer. As a result, it is possible to further reduce the computational load of the overall coding process.

[0026] In one possible implementation of the method according to any preceding implementation or the first aspect itself of the first aspect, even if the conversion block boundary between the first conversion block and the second conversion block is not aligned (does not overlap) with an n×n sample grid, the conversion block boundary between the first conversion block and the second conversion block is unblocked (filtered), where n is an integer. It is possible to unblock a target boundary that is not aligned with an n×n grid.

[0027] In one possible implementation of the method according to any preceding implementation of the first aspect, n is 4 or 8. The prior art only considers boundaries that overlap with an 8×8 grid. In the case of the present invention, even when the SBT internal boundary is not aligned with the 8×8 grid when applying the asymmetric division, that internal boundary would be considered a filtering candidate. Also, by filtering the SBT internal boundary, the block artifacts caused by the SBT are reduced.

[0028] In one possible implementation of the method according to any preceding implementation of the first aspect or the first aspect itself, the samples of the first conversion block and the second conversion block are luminance samples, or the first conversion block and the second conversion block have luminance samples and chrominance samples. Conversion edges with both sides using inter-frame prediction in chrominance components such as those caused by the SBT can also cause block artifacts. In particular, such block artifacts can be severe when the main information of the video sequence (such as the campfire sequence used under general test conditions) is represented by chrominance components. Therefore, the present invention proposes introducing a de-blocking filtering process for conversion edges with both sides using inter-frame prediction in chrominance components.

[0029] In one possible implementation of the method according to any preceding implementation of the first aspect, the n×n sample grid is a 4×4 sample grid for the samples of the first conversion block and the second conversion block that are luminance samples, or the n×n sample grid is an 8×8 sample grid for the samples of the first conversion block and the second conversion block that are chrominance samples.

[0030] In one possible implementation of the method according to any preceding implementation of the first aspect or the first aspect itself, the coding block is divided horizontally or vertically.

[0031] In one possible implementation of the method according to any preceding implementation of the first aspect or the first aspect itself, when the coding block is divided horizontally, the transform block boundary between the first transform block and the second transform block is a horizontal transform block boundary (horizontal sub-block transform, SBT boundary), or when the coding block is divided vertically, the transform block boundary between the first transform block and the second transform block is a vertical transform block boundary (vertical sub-block transform, SBT boundary). The present invention functions for both the vertical transform block boundary and the horizontal transform block boundary.

[0032] In one possible implementation of the method according to any preceding implementation of the first aspect or the first aspect itself, the current coding block is coded using the sub-block transform SBT tool, or the transform block boundary is executed by the sub-block transform SBT tool.

[0033] According to a second aspect of the present disclosure, in image encoding and / or image decoding, a non-blocking method for non-blocking a block boundary in a coding block (coding unit) is provided. (The coding block is coded (predicted) in an inter-frame prediction mode, as in the case of being coded in a sub-block transform SBT mode). The coding block (the coding block predicted inter-frame) includes a plurality of transform blocks. (For example, when enabling sub-block transform, the current coding unit is divided into a plurality of transform units, and the coding block is divided (distributed) into a plurality of transform blocks during the inter-frame prediction process or at the time of the inter-frame prediction process). For example, the transform blocks include adjacent in the vertical or horizontal direction p0 and q0, etc.). The plurality of transform blocks include a first transform block and a second transform block adjacent to the first transform block. The method includes In response to a determination that it is necessary to filter a transform block boundary between the first transform block and the second transform block, the boundary between the first transform block and the second transform block is a sub-block transform SBT boundary, and when at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, determining a value of a boundary strength parameter for the boundary between the first transform block and the second transform block to be a first value; Performing a non-blocking filtering process on the transform block boundary between the first transform block and the second transform block based on at least the value of the boundary strength parameter.

[0034] It is possible to understand that the first transformation block and the second transformation block on both sides of the transformation block boundary use inter-frame prediction. In one example, the samples of the coding block are chroma samples. In one example, the samples of the coding block are luma samples. In other examples, the coding block has luma samples and chroma samples. Accordingly, in one example, the samples of the first transformation block and the second transformation block are chroma samples, and in other examples, the samples of the first transformation block and the second transformation block are luma samples. In other examples, the first transformation block and the second transformation block have luma samples and chroma samples. Specifically, according to the prior art, the deblocking filtering process is applied to the coding sub-block edges and transformation block edges of the video, but excludes the edges in the chroma components that use inter-frame prediction on both sides of the edge, and thus may also exclude the internal SBT boundaries (because it is possible to apply the SBT tool to both luma component and chroma component ). On the other hand, according to the second aspect of the present disclosure, it is possible to deblock a plurality of internal SBT boundaries caused by the sub-block transform (SBT) coding tool, and in particular, it is possible to deblock the internal SBT boundaries in the chroma components caused by the sub-block transform (SBT) coding tool.

[0035] In addition to the boundary strength (BS) parameter for the boundary between the first transformation block and the second transformation block, it is possible to understand that other parameters may be considered for the deblocking filtering process. That is, depending on the specific filtering decision result, in some cases, no samples need to be modified, or in other cases, deblocking filtering may be performed where only one sample is modified in each row or each column that is perpendicular and adjacent to the boundary.

[0036] Perform partitioning of an inter - prediction block (i.e., an inter - coding block, which is an abbreviation of the current coding block coded in the inter - prediction mode) into internal transform blocks. Since (one transform block has residual data and the other blocks do not have residual data), the transform is performed only for one of the plurality of transform blocks and not for the other blocks. These transform blocks may be symmetric (i.e., two sub - blocks of the same size) or asymmetric (i.e., the sub - blocks are not sub - blocks of the same size). Such partial transforms may cause block artifacts along the boundaries between the plurality of internal transform blocks. In the prior art that degrades subjective quality when enabling sub - block transform (SBT), filtering these boundaries has not been considered. On the other hand, according to a second aspect of the present invention, the filtering process is improved to reduce the block artifacts of the SBT boundaries caused by the SBT coding tool. When detecting those boundaries that would be considered for filtering, consider the internal boundaries between the plurality of internal transform blocks caused by the SBT coding tool.

[0037] In one possible implementation of the method according to the second aspect itself, the first transform block has residual data and the second transform block has no residual data, or the first transform block has no residual data and the second transform block has residual data.

[0038] In any preceding implementation of the second aspect or in one possible implementation of the method according to the second aspect itself, the transform block is a sub - block transform SBT transform block.

[0039] In any preceding implementation of the second aspect or in one possible implementation of the method according to the second aspect itself, the number of transform blocks is 2 or 3 or other values.

[0040] In one possible implementation of a method according to any preceding implementation of the second aspect or the second aspect itself, the method comprises: further comprising determining whether it is necessary to filter the transform block boundary between the first transform block and the second transform block.

[0041] In one possible implementation of a method according to any preceding implementation of the second aspect or the second aspect itself, in response to a determination that it is necessary to filter the transform block boundary between the first transform block and the second transform block, when the boundary between the first transform block and the second transform block is a sub-block transform SBT boundary and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, the step of determining the value of the boundary strength parameter for the boundary between the first transform block and the second transform block to be a first value comprises: including, in response to a determination that the boundary of the transform block between the first transform block and the second transform block is aligned (overlapping) with an n×n sample grid, when the boundary between the first transform block and the second transform block is a sub-block transform SBT boundary and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, determining the value of the boundary strength parameter for the boundary between the first transform block and the second transform block to be a first value, where n is an integer.

[0042] In one possible implementation of a method according to any preceding implementation of the second aspect, n is 4 or 8.

[0043] In one possible implementation of the method according to any preceding implementation of the second aspect or the second aspect itself, the samples of the first conversion block and the second conversion block are luminance samples, or the samples of the first conversion block and the second conversion block are chrominance samples, or the first conversion block and the second conversion block have luminance samples and chrominance samples.

[0044] In one possible implementation of the method according to any preceding implementation of the second aspect, the n×n sample grid is a 4×4 sample grid for the samples of the first conversion block and the second conversion block that are luminance samples, or the n×n sample grid is an 8×8 sample grid for the samples of the first conversion block and the second conversion block that are chrominance samples. It is recognized that conversion edges having both sides that use inter-frame prediction in chrominance components such as those caused by SBT may also produce block artifacts. In particular, such block artifacts can be severe when the main information of a video sequence (such as the campfire sequence used under general test conditions) is represented by chrominance components. Therefore, the present invention proposes introducing a de-blocking filtering process for such conversion edges having both sides that use inter-frame prediction in chrominance components.

[0045] In one possible implementation of the method according to any preceding implementation of the second aspect or the second aspect itself, the conversion block further includes a third conversion block adjacent to the second conversion block, and the method In response to a determination that a transform block boundary between the second transform block and the third transform block is aligned (coincident) with an n×n sample grid, the boundary between the second transform block and the third transform block is a sub-block transform SBT boundary, and when at least one of the second transform block and the third transform block has one or more non-zero transform coefficients, determining a value of a boundary strength parameter for the boundary between the second transform block and the third transform block to be a first value, or, In response to a determination that a transform block boundary between the second transform block and the third transform block is aligned (coincident) with an n×n sample grid, the boundary between the second transform block and the third transform block is a sub-block transform SBT boundary, and when both the second transform block and the third transform block have zero transform coefficients, determining a value of a boundary strength parameter for the boundary between the second transform block and the third transform block to be a second value, further comprising.

[0046] In one possible implementation of a method according to any preceding implementation of the second aspect, n is 4 or 8. It has been recognized that 4×4 block edges occur more frequently in VVC than block edges in HEVC. In HEVC, only quadtree partitioning is allowed for coding blocks, i.e., the resulting coding units are always square. On the other hand, in VVC, quadtrees with multiple types of tree partitioning are allowed, i.e., the partitioning may result in narrow 4×N coding blocks or flat N×4 coding blocks. Further, sub-block partitioning tools such as SBT may further result in 4×N transform block edges or N×4 transform block edges. Therefore, the grid size is set to 4×4 to account for filtering edges that do not overlap with the 8×8 grid.

[0047] In one possible implementation of the method according to any previous implementation of the second aspect or the second aspect itself, the samples of the second conversion block and the third conversion block are luminance samples, or the samples of the second conversion block and the third conversion block are chrominance samples.

[0048] In one possible implementation of the method according to any previous implementation of the second aspect, the n×n sample grid is a 4×4 sample grid for the samples of the second conversion block and the third conversion block that are luminance samples, or the n×n sample grid is an 8×8 sample grid for the samples of the second conversion block and the third conversion block that are chrominance samples.

[0049] In one possible implementation of the method according to any previous implementation of the second aspect or the second aspect itself, the first value is 1.

[0050] In one possible implementation of the method according to any previous implementation of the second aspect, the second value is 0.

[0051] In one possible implementation of the method according to any previous implementation of the second aspect or the second aspect itself, the coding block is divided in a horizontal direction or a vertical direction.

[0052] In one possible implementation of the method according to any previous implementation of the second aspect or the second aspect itself, when the coding block is divided in a horizontal direction, the conversion block boundary between the first conversion block and the second conversion block is a horizontal conversion block boundary, or when the coding block is divided in a vertical direction, the conversion block boundary between the first conversion block and the second conversion block is a vertical conversion block boundary.

[0053] In one possible implementation of the method according to any preceding implementation of the second aspect or the second aspect itself, the current coding block is coded using a sub-block transform SBT tool, or the transform block boundary is caused by a sub-block transform SBT tool.

[0054] According to a third aspect of the present disclosure, there is provided a device for use in an image encoder and / or an image decoder to unblock a transform block boundary in a coding block, the coding block being coded (predicted) in an inter-frame prediction mode, (for example, when enabling sub-block transform, the current coding unit is split into two transform units) the coding block is split (distributed) into a plurality of transform blocks, (for example, the transform blocks include adjacent in the vertical or horizontal direction such as p0 and q0) the plurality of transform blocks include a first transform block and a second transform block adjacent to the first transform block, and the device includes an unblocking filter, the unblocking filter determines a value of a boundary strength parameter for the boundary between the first transform block and the second transform block to be a first value when the boundary between the first transform block and the second transform block is a transform unit boundary and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, and is configured to perform an unblocking filtering process on the boundary between the first transform block and the second transform block based at least on the value of the boundary strength parameter.

[0055] In one possible implementation of the device according to the third aspect itself, the first conversion block has residual data, and the second conversion block has no residual data, or the first conversion block has no residual data, and the second conversion block has residual data.

[0056] In one possible implementation of the device according to any preceding implementation of the third aspect or the third aspect itself, the conversion block is a sub-block transform (SBT) conversion block.

[0057] In one possible implementation of the device according to any preceding implementation of the third aspect or the third aspect itself, the number of conversion blocks is two or three.

[0058] In one possible implementation of the device according to any preceding implementation of the third aspect or the third aspect itself, the boundary between the first conversion block and the second conversion block is a sub-block transform (SBT) boundary.

[0059] In one possible implementation of the device according to any preceding implementation of the third aspect or the third aspect itself, the deblocking filter is configured to determine a value of a boundary strength parameter for the boundary between the first conversion block and the second conversion block to be a first value when the boundary between the first conversion block and the second conversion block is a sub-block transform (SBT) boundary and at least one of the first conversion block and the second conversion block has one or more non-zero conversion coefficients.

[0060] In one possible implementation of a device according to any preceding implementation of the third aspect or the third aspect itself, the conversion block further includes a third conversion block adjacent to the second conversion block, and the deblocking filter further includes: when the boundary between the second conversion block and the third conversion block is a sub-block transform (SBT) boundary and at least one of the second conversion block and the third conversion block has one or more non-zero conversion coefficients, determining a value of a boundary strength parameter for the boundary between the second conversion block and the third conversion block to be a first value; or when the boundary between the second conversion block and the third conversion block is an SBT boundary and both the second conversion block and the third conversion block have zero conversion coefficients, determining a value of a boundary strength parameter for the boundary between the second conversion block and the third conversion block to be a second value.

[0061] In one possible implementation of a device according to any preceding implementation of the third aspect, the first value is 1.

[0062] In one possible implementation of a device according to any preceding implementation of the third aspect, the second value is 0.

[0063] In one possible implementation of a device according to any preceding implementation of the third aspect or the third aspect itself, the conversion block boundary between the first conversion block and the second conversion block is deblocked only when it is aligned (overlapped) with an n×n sample grid, where n is an integer.

[0064] In one possible implementation of a device according to any preceding implementation of the third aspect or the third aspect itself, even if the transform block boundary between the first transform block and the second transform block is not aligned (not overlapping) with an n×n sample grid, the transform block boundary between the first transform block and the second transform block is unblocked, where n is an integer.

[0065] In one possible implementation of a device according to any preceding implementation of the third aspect, n is 4 or 8.

[0066] In one possible implementation of a device according to any preceding implementation of the third aspect or the third aspect itself, the samples of the first transform block and the second transform block are luminance samples, or the samples of the first transform block and the second transform block are chrominance samples.

[0067] In one possible implementation of a device according to any preceding implementation of the third aspect, the n×n sample grid is a 4×4 sample grid for the samples of the first transform block and the second transform block that are luminance samples, or the n×n sample grid is an 8×8 sample grid for the samples of the first transform block and the second transform block that are chrominance samples.

[0068] In one possible implementation of a device according to any preceding implementation of the third aspect or the third aspect itself, the coding block is divided in the horizontal or vertical direction.

[0069] In one possible implementation of a device according to any preceding implementation of the third aspect or the third aspect itself, when the coding block is divided horizontally, the transform block boundary between the first transform block and the second transform block is a horizontal transform block boundary, or when the coding block is divided vertically, the transform block boundary between the first transform block and the second transform block is a vertical transform block boundary.

[0070] In one possible implementation of a device according to any preceding implementation of the third aspect or the third aspect itself, the current coding block is coded using a sub-block transform SBT tool, or the transform block boundary is caused by a sub-block transform SBT tool.

[0071] According to a fourth aspect of the present disclosure, a device is provided for use in an image encoder and / or an image decoder to unblock a block boundary in a coding block (coding unit), (such as the coding block is coded in a sub-block transform SBT mode) the coding block is coded (predicted) in an inter-frame prediction mode, (such as when enabling sub-block transform, the current coding unit is divided into two transform units) the coding block (the coding block predicted inter-frame) is divided (distributed) into a plurality of transform blocks, (in the inter-frame prediction process, for example, the transform blocks include p0 and q0 and are adjacent vertically or horizontally) the plurality of transform blocks include a first transform block and a second transform block adjacent to the first transform block, and the device includes an unblocking filter, and the unblocking filter is In response to a determination that it is necessary to filter a transform block boundary between the first transform block and the second transform block, the boundary between the first transform block and the second transform block is a sub-block transform (SBT) boundary, and when at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, a value of a boundary strength parameter for the boundary between the first transform block and the second transform block is determined to be a first value, and configured to perform an in-loop filtering process on the transform block boundary between the first transform block and the second transform block based on at least the value of the boundary strength parameter.

[0072] In one possible implementation of a device according to any preceding implementation of the fourth aspect, the first transform block has residual data and the second transform block has no residual data, or the first transform block has no residual data and the second transform block has residual data.

[0073] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, the transform block is a sub-block transform (SBT) transform block.

[0074] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, the number of transform blocks is two or three.

[0075] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, said non-blocking filter is Furthermore to determine whether it is necessary to filter the transform block boundary between the first transform block and the second transform block configured as .

[0076] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, the non-blocking filter is configured to determine a value of a boundary strength parameter for a boundary between the first transformation block and the second transformation block to be a first value in response to a determination that a boundary of the transformation block between the first transformation block and the second transformation block is aligned (overlapping) with an n×n sample grid, the boundary between the first transformation block and the second transformation block is a sub-block transform SBT boundary, and at least one of the first transformation block and the second transformation block has one or more non-zero transformation coefficients, where n is an integer.

[0077] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, n is 4 or 8.

[0078] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, samples of the first transformation block and the second transformation block are luminance samples, or samples of the first transformation block and the second transformation block are chrominance samples, or the first transformation block and the second transformation block have luminance samples and chrominance samples. It has been recognized that conversion edges with both sides using inter-frame prediction in chrominance components such as those caused by SBT may also produce block artifacts. In particular, such block artifacts can be severe when the main information of a video sequence (such as a campfire sequence used under typical test conditions) is represented by chrominance components. Accordingly, the present invention proposes introducing a non-blocking filtering process for conversion edges with both sides using inter-frame prediction in chrominance components.

[0079] In one possible implementation of a device according to any of the preceding implementations of the fourth aspect, the n×n sample grid is a 4×4 sample grid for the samples of the first and second transform blocks that are luminance samples, or the n×n sample grid is an 8×8 sample grid for the samples of the first and second transform blocks that are chrominance samples.

[0080] It is recognized that 4×4 block edges occur more frequently in VVC than in HEVC block edges. In HEVC, only quadtree partitioning is allowed for coding blocks, i.e., the resulting coding units are always square. On the other hand, in VVC, quadtrees with multiple types of tree partitioning are allowed, i.e., the partitioning may result in narrow 4×N coding blocks or flat N×4 coding blocks. Furthermore, sub-block splitting tools such as SBT may further result in 4×N transform block edges or N×4 transform block edges. Therefore, the grid size is set to 4×4 to consider filtering edges that do not overlap with the 8×8 grid.

[0081] In one possible implementation of a device according to any of the preceding implementations of the fourth aspect or the fourth aspect itself, the transform block further includes a third transform block adjacent to the second transform block, the non-blocking filter further, In response to a determination that a transform block boundary between the second transform block and the third transform block is aligned (coincident) with an n×n sample grid, the boundary between the second transform block and the third transform block is a sub-block transform SBT boundary, and when at least one of the second transform block and the third transform block has one or more non-zero transform coefficients, determining a value of a boundary strength parameter for the boundary between the second transform block and the third transform block to be a first value, or, In response to a determination that a transform block boundary between the second transform block and the third transform block is aligned (coincident) with an n×n sample grid, the boundary between the second transform block and the third transform block is a sub-block transform SBT boundary, and when both the second transform block and the third transform block have zero transform coefficients, determining a value of a boundary strength parameter for the boundary between the second transform block and the third transform block to be a second value, is configured.

[0082] In one possible implementation of a device according to any preceding implementation of the fourth aspect, n is 4 or 8.

[0083] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, the samples of the second transform block and the third transform block are luminance samples, or the samples of the second transform block and the third transform block are chrominance samples.

[0084] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, the n×n sample grid is a 4×4 sample grid for the samples of the second and third conversion blocks that are luminance samples, or the n×n sample grid is an 8×8 sample grid for the samples of the second and third conversion blocks that are chrominance samples.

[0085] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, the first value is 1.

[0086] In one possible implementation of a device according to any preceding implementation of the fourth aspect, the second value is 0.

[0087] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, the coding block is divided in a horizontal or vertical direction.

[0088] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, when the coding block is divided horizontally, the conversion block boundary between the first and second conversion blocks is a horizontal conversion block boundary, or when the coding block is divided vertically, the conversion block boundary between the first and second conversion blocks is a vertical conversion block boundary.

[0089] In one possible implementation of a device according to any preceding implementation of the fourth aspect or the fourth aspect itself, the current coding block is coded using a sub-block transform SBT tool, or the conversion block boundary is caused by a sub-block transform SBT tool.

[0090] According to a fifth aspect of the present disclosure, an encoder is provided that includes a processing circuit for performing a method according to any preceding implementation of the first aspect or the first aspect itself, or a method according to any preceding implementation of the second aspect or the second aspect itself.

[0091] According to a sixth aspect of the present disclosure, a decoder is provided that includes a processing circuit for performing a method according to any preceding implementation of the first aspect or the first aspect itself, or a method according to any preceding implementation of the second aspect or the second aspect itself.

[0092] According to a seventh aspect of the present disclosure, a computer program product is provided that includes program code for performing a method according to any preceding implementation of the first aspect or the first aspect itself, or a method according to any preceding implementation of the second aspect or the second aspect itself.

[0093] According to an eighth aspect of the present disclosure, a non-transitory and computer-readable medium is provided that holds program code, and when the program code is executed by a computer device, the computer device is caused to perform a method according to any preceding implementation of the first aspect or the first aspect itself, or a method according to any preceding implementation of the second aspect or the second aspect itself.

[0094] According to a ninth aspect of the present disclosure, a decoder is provided that includes one or more processors and a non-transitory and computer-readable storage medium coupled to the processor and storing programming for execution by the processor, and when the programming is executed by the processor, the programming configures the decoder to perform a method according to any preceding implementation of the first aspect or the first aspect itself, or a method according to any preceding implementation of the second aspect or the second aspect itself.

[0095] According to a tenth aspect of the present disclosure, there is provided an encoder including one or more processors and a non-transitory and computer-readable storage medium coupled to the processor and storing programming for execution by the processor. When the programming is executed by the processor, the programming configures the encoder to execute a method according to any preceding implementation of the first aspect or the first aspect itself, or a method according to any preceding implementation of the second aspect or the second aspect itself.

[0096] According to an eleventh aspect of the present disclosure, there is provided a deblocking filter device for deblocking a transform block boundary in a coding block. (The coding block is divided (distributed) into a plurality of transform blocks during an inter-frame prediction process. For example, when sub-block transform is enabled, the current coding unit is divided into two transform units.) The coding block is coded (predicted) in an inter-frame prediction mode. The coding block includes a plurality of transform blocks, and the plurality of transform blocks include a first transform block and a second transform block adjacent to the first transform block. The deblocking filter determines a value of a boundary strength BS parameter for the boundary between the first transform block and the second transform block to be a first value when the boundary between the first transform block and the second transform block is a transform block boundary and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, and executes a deblocking filtering process for the boundary between the first transform block and the second transform block based at least on the first value of the BS parameter.

[0097] According to a twelfth aspect of the present disclosure, a deblocking filter device for deblocking a block boundary in a coding block (coding unit) is provided, the coding block being coded (predicted) in an inter-frame prediction mode (in particular, the coding block being coded in a sub-block transform SBT mode), (the coding block predicted inter-frame being divided (distributed) into a plurality of transform blocks in an inter-frame prediction process, for example, when enabling sub-block transform, the current coding unit being divided into two transform units), the coding block including a first transform block and a second transform block adjacent to the first transform block (for example, the transform blocks including p0 and q0 and being adjacent in the vertical or horizontal direction), the deblocking filter device determining a value of a boundary strength parameter for the boundary between the first transform block and the second transform block to be a first value in response to a determination that it is necessary to filter the transform block boundary between the first transform block and the second transform block, the boundary between the first transform block and the second transform block being a sub-block transform SBT boundary and at least one of the first transform block and the second transform block having one or more non-zero transform coefficients, and means for performing a deblocking filtering process on the transform block boundary between the first transform block and the second transform block based at least on the value of the boundary strength parameter.

[0098] A method according to a first aspect of the present invention, i.e., when a boundary between a first transform block and a second transform block is a transform block boundary and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, determining a value of a boundary strength BS parameter for the boundary between the first transform block and the second transform block to be a first value, and performing a deblocking filtering process on the boundary between the first transform block and the second transform block based at least on the first value of the BS parameter may be performed by an apparatus according to an eleventh aspect of the present invention. Further features and implementations of the apparatus according to the eleventh aspect of the present invention correspond to the features and implementations of the method according to the first aspect of the present invention.

[0099] A method according to a second aspect of the present invention, i.e., in response to a determination that it is necessary to filter a transform block boundary between the first transform block and the second transform block, when the boundary between the first transform block and the second transform block is a sub-block transform SBT boundary and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, determining a value of a boundary strength parameter for the boundary between the first transform block and the second transform block to be a first value, and performing a deblocking filtering process on the transform block boundary between the first transform block and the second transform block based at least on the value of the boundary strength parameter may be performed by an apparatus according to a twelfth aspect of the present invention. Further features and implementations of the apparatus according to the twelfth aspect of the present invention correspond to the features and implementations of the method according to the second aspect of the present invention.

[0100] The apparatus may be extended to an implementation form corresponding to the implementation form of the method according to any of the preceding aspects. Accordingly, the implementation form of the apparatus includes one or more features of the corresponding implementation form of the method according to any of the preceding aspects. The advantages of those plural apparatuses according to any of the preceding aspects are the same as the advantages in the case of the corresponding implementation form of the method according to any of the preceding aspects.

[0101] According to a further aspect, the present invention relates to an apparatus for decoding a video stream, including a processor and a memory. The memory stores instructions that cause the processor to execute a method according to a first aspect.

[0102] According to a further aspect, the present invention relates to an apparatus for decoding a video stream, including a processor and a memory. The memory stores instructions that cause the processor to execute a method according to a first aspect.

[0103] Details of one or more embodiments are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the detailed description of the invention, the drawings, and the claims.

Brief Description of the Drawings

[0104] In the following description, multiple embodiments of the present invention will be described in more detail with reference to the multiple accompanying drawings and charts.

[0105]

Figure 1A

Figure 1B

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

[0106] In the following description, unless otherwise specified, the same reference numerals refer to the same features or at least functionally equivalent features.

Embodiments for Carrying Out the Invention

[0107] In the following description, reference is made to a plurality of accompanying drawings, which form part of the present disclosure and, by way of example, show a plurality of specific aspects of a plurality of embodiments of the present invention or a plurality of specific aspects of those plurality of embodiments of the present invention. It is understood that the plurality of embodiments of the present invention may be used in other aspects and may include structural or logical changes not shown in the drawings. Accordingly, the following detailed description should not be construed in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0108] For example, it is understood that the disclosure related to the described method may also apply to a corresponding device or system configured to perform the method, and vice versa. For example, when describing one or more specific method steps, even if one or more such units are not explicitly described or illustrated in the drawings, the corresponding device may include one or more units such as functional units (e.g., one unit for performing one or more of those steps, or multiple units each performing one or more of a plurality of those steps, etc.) to perform the one or more described method steps. For example, on the one hand, when describing a particular device based on one or more units such as functional units, even if one or more such steps are not explicitly described or illustrated in the drawings, the corresponding method may include one step (e.g., one step for performing the functions of one or more of those units, or multiple steps each performing the functions of one or more of a plurality of those units, etc.) to perform the functions of the one or more units. Furthermore, it is understood that, unless otherwise specified, the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0109] Video coding typically refers to the processing of a sequence of images that form a video or video sequence. Instead of the term "image", the terms "frame" or "picture" may be used as synonyms in the field of video coding. Video coding (or coding in general) includes two parts: video encoding and video decoding. Video encoding is performed on the source side and typically involves processing the original video image (e.g., by compression) to reduce the amount of data required to represent the video image for more efficient storage and / or transmission. Video decoding is performed on the destination side and typically involves the reverse process compared to when the encoder reconstructs the video image. It should be understood that the various embodiments referring to the "coding" of a video image (or generally an image) are related to the "encoding" or "decoding" of the video image or each video sequence. The combination of the encoding part and the decoding part is also referred to as a CODEC (Coding and Decoding).

[0110] In the case of lossless video coding, it is possible to reconstruct the original video image, i.e., (assuming no transmission loss or other data loss during storage or transmission), the reconstructed video image has the same quality as the original video image. In the case of lossy video coding, for example, further compression by quantization is performed to reduce the amount of data representing the video image, and it is impossible to completely reconstruct the video image at the decoder, i.e., the quality of the reconstructed video image is lower or worse compared to the quality of the original video image.

[0111] Some of the video coding standards belong to the group of "hybrid video codecs with losses" (i.e., combining spatial and temporal prediction in the sample domain and 2D transform coding for applying quantization in the transform domain). Each video of a video sequence is typically partitioned into a set of non-overlapping blocks, and the coding is typically performed at the block level. In other words, in the encoder, typically, for example, using spatial (within-video) prediction and / or temporal (inter-video) prediction, a predicted block is generated, the predicted block is subtracted from the current block (the block being currently processed / to be processed), a residual block is obtained, the residual block is transformed, and the residual block is quantized in the transform domain to reduce the amount of data to be transmitted (compression), thereby processing, i.e., encoding, the video at the block (video block) level. On the other hand, in the decoder, the reverse process compared to the encoder is applied to the encoded or compressed block to reconstruct the current block for presentation. Further, the encoder replicates the decoder processing loop, whereby both the encoder and the decoder will generate the same prediction (such as within-video prediction and inter-video prediction, etc.) and / or generate subsequent block processing, i.e., reconstruction for coding.

[0112] In the following description, based on FIGS. 1 to 3, a plurality of embodiments of a video coding system 10, a video encoder 20, and a video decoder 30 will be described.

[0113] FIG. 1A is a schematic block diagram illustrating an exemplary coding system 10, such as, for example, a video coding system 10 (or, simply, coding system 10) that can utilize the technology of this application. The video encoder 20 (or, simply, encoder 20) and the video decoder 30 (or, simply, decoder 30) of the video coding system 10 represent a plurality of examples of devices, and those devices may be configured to execute the technology according to various examples described in this application.

[0114] As shown in FIG. 1A, the coding system 10 includes a source device 12, and the source device 12 is configured to provide encoded video data 21 to a destination device 14, for example, for decoding the encoded video data 13.

[0115] The source device 12 includes an encoder 20, and additionally, that is, optionally, may include a video source 16, a preprocessor (or, preprocessing unit) 18 such as a video preprocessor 18, for example, and a communication interface or communication unit 22.

[0116] The video source 16 may include any type of video capture device, such as a camera for capturing real-world video, and / or any type of video generation device, such as a computer graphics processor for generating computer-animated video, or real-world video, (e.g., screen content, virtual reality (VR) video, etc.) computer-generated video, and / or any combination of any of them (e.g., augmented reality (AR) video, etc.) for obtaining and / or providing, or may be those devices. The video source may be any type of memory or storage device for storing any of the above videos.

[0117] Distinguishing the processes executed by the preprocessor 18 and the preprocessing unit 18, the video or video data 17 may also be referred to as unprocessed video or unprocessed video data 17.

[0118] The preprocessor 18 is configured to receive the (unprocessed) video data 17 and perform preprocessing on the video data 17 to obtain preprocessed video 19 or preprocessed video data 19. The preprocessing executed by the preprocessor 18 may include, for example, trimming, color format conversion (e.g., from RGB to YCbCr), color correction, or noise removal. It is possible to understand that the preprocessing unit 18 may be an optional component.

[0119] The video encoder 20 is configured to receive the preprocessed video data 19 and provide encoded video data 21 (for example, based on FIG. 2, further details will be described below). The communication interface 22 of the source device 12 is configured to receive the encoded video data 21 and transmit the encoded video data 21 (or any further processed version of the encoded video data) to another device, such as the destination device 14 or any other device, via the communication channel 13 for storage or direct reconstruction.

[0120] The destination device 14 includes a decoder 30 (such as a video decoder 30 for example), and additionally, optionally, may include a communication interface or communication unit 28, a postprocessor 32 (or postprocessing unit 32), and a display device 34.

[0121] The communication interface 28 of the destination device 14 is configured to receive the encoded video data 21 (or a further processed version of any of the encoded video data) directly from the source device 12, or from any other source such as, for example, a memory device such as an encoded video data storage device, and to provide the encoded video data 21 to the decoder 30.

[0122] The communication interface 22 and the communication interface 28 may be configured to transmit or receive the encoded video data 21 or the encoded data 13 via, for example, a direct communication link between the source device 12 and the destination device 14 such as a direct wired connection or a direct wireless connection, or any type of network such as, for example, a wired network or a wireless network or any combination thereof, or any type of private network and public network, or any combination of any of these types.

[0123] For example, the communication interface 22 may be configured to process the encoded video data by packaging the encoded video data 21 into an appropriate format such as a packet and / or using any type of transmission encoding or processing for transmission via a communication link or a communication network.

[0124] The communication interface 28 paired with the communication interface 22 may be configured to receive the transmitted data and process the transmitted data using any type of corresponding transmission decoding or processing and / or unpackaging to obtain the encoded video data 21.

[0125] Both the communication interface 22 and the communication interface 28 may be configured as a unidirectional communication interface or as a bidirectional communication interface, as indicated by the arrow of the communication channel 13 in FIG. 1A going from the source device 12 to the destination device 14. For example, they may be configured to send and receive messages, for example, to set up a connection, thereby verifying and exchanging any other information related to a communication link and / or data transmission, such as encoded video data transmission.

[0126] The decoder 30 is configured to receive the encoded video data 21 and provide the decoded video data 31 or the decoded video 31 (for example, further details will be described below based on FIG. 3 or FIG. 5).

[0127] The post-processor 32 of the destination device 14 is configured to post-process the decoded video data 31 (also called reconstructed video data in some cases), such as the decoded video 31, to obtain post-processed video data 33, such as the post-processed video 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, trimming, or resampling, or any other processing for preparing the decoded video data 31 for display by, for example, the display device 34.

[0128] The display device 34 of the destination device 14 is configured to receive the post-processed video data 33, for example, to display video for a user or viewer. The display device 34 may be, for example, any type of display for presenting the reconstructed video such as an integrated display or an external display or monitor, or may include those displays. Those displays may include, for example, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0129] FIG. 1A shows the source device 12 and the destination device 14 as a plurality of individual devices, but multiple embodiments of the devices may include both or include the functions of both, and may include the source device 12 or corresponding function and the destination device 14 or corresponding function. In such embodiments, the same hardware and / or software may be used, or the source device 12 or corresponding function and the destination device 14 or corresponding function may be implemented by individual hardware and / or software or any combination thereof.

[0130] As will be apparent to those skilled in the art by description, the presence and (exact) partitioning of the functions of the plurality of different units or the plurality of different functions within the source device 12 and / or the destination device 14 shown in FIG. 1A may vary depending on the actual device and application.

[0131] As shown in FIG. 1B, an encoder 20 (such as video encoder 20) or a decoder 30 (such as video decoder 30), or both the encoder 20 and the decoder 30 may be implemented by a processing circuit such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, hardware, those dedicated to video coding, or any combination thereof. The encoder 20 may be implemented by a processing circuit 46 for implementing various modules, as described for the encoder 20 of FIG. 2 and / or any other encoder system or subsystem described herein. The decoder 30 may be implemented by a processing circuit 46 for implementing various modules, as described for the decoder 30 of FIG. 3 and / or any other decoder system or subsystem described herein. The processing circuit may be configured to perform various operations, as described later. As shown in FIG. 5, when the technology is implemented partially by software, the device may store instructions for the software in a suitable non-transitory computer-readable storage medium and execute the instructions by hardware using one or more processors to perform the technology of this disclosure. For example, as shown in FIG. 1B, either the video encoder 20 or the video decoder 30 may be integrated as part of a combined encoder / decoder (CODEC) in a single device.

[0132] The source device 12 and the destination device 14 may include any of a wide range of devices such as, for example, a notebook computer or a laptop computer, a mobile phone, a smartphone, a tablet or a tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content delivery server, etc.), a broadcast receiver device, or a broadcast transmitter device, etc., and may include any of a variety of handheld devices or fixed devices, may not use an operating system at all, or may use any kind of operating system. In some cases, the source device 12 and the destination device 14 may be equipped for wireless communication. Accordingly, the source device 12 and the destination device 14 may be wireless communication devices.

[0133] In some cases, the video coding system 10 illustrated in FIG. 1A is merely one example, and the technology of this application may be applied to video coding settings (such as, for example, video encoding or video decoding, etc.), and those video coding settings do not necessarily include data communication between an encoding device and a decoding device. In other examples, the data is retrieved from local memory or streamed via a network, etc. The video encoding device may encode the data and store it in memory, and / or the video decoding device may retrieve the data from memory and decode it. In some of the multiple examples, encoding and decoding are performed by multiple devices, and those multiple devices do not communicate with each other, but simply encode the data and store it in memory and / or retrieve the data from memory and decode it.

[0134] For the sake of convenience of explanation, in this specification, a plurality of embodiments of the present invention are described by referring to the next-generation video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG), such as the reference software of High Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC). Those skilled in the art will understand that the plurality of embodiments of the present invention are not limited to HEVC or VVC.

[0135] Encoder and encoding method

[0136] FIG. 2 shows a schematic block diagram of an exemplary video encoder 200 which is configured to implement the technology of this application. In the example of FIG. 2, the video encoder 200 includes an input 200 (or input interface 202 ), a residual calculation unit 204, a conversion processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse conversion processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy encoding unit 270, and an output unit 272 (or output interface 272). The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a partitioning unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The video encoder shown in FIG. 2 202 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec. 200 The residual calculation unit 204, the conversion processing unit 206, the quantization unit 208, and the mode selection unit 260 are the encoder

[0137] 200 ​It may be regarded as forming the forward signal path of the video encoder 20. On the other hand, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 244, and the intra-frame prediction unit 254 may be regarded as forming the reverse signal path of the video encoder 20. The reverse signal path of the video encoder 20 corresponds to the signal path of the decoder (refer to the video decoder in FIG. 3 300 ). The inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 244, and the intra-frame prediction unit 254 are also regarded as forming the "embedded decoder" of the video encoder 20.

[0138] Video and video segmentation (video and blocks)

[0139] The encoder 20 is configured to receive, for example, video 17 (or video data 17) such as a video in a sequence of videos forming a video or a video sequence via an input 202 . The received video or video data may also be pre-processed video 19 (or pre-processed video data 19). For simplicity, the following description refers to video 17. Video 17 may also be referred to as the video to be coded or the current video (especially, for example, in video coding to distinguish the current video from other videos such as previously coded videos and / or decoded videos in the same video sequence, i.e., also including the current video).

[0140] (Digital) video may be considered or may be considered as a two-dimensional array or matrix of samples having intensity values. The samples in that array may also be referred to as pixels or pels (abbreviations of video elements). The number of samples in the horizontal and vertical directions (or axes) of the array or video defines the size and / or resolution of the video. For color representation, typically, three color components are used, that is, the video may be represented by or may include three sample arrays. In the RGB format or color space, the video includes corresponding red, green, and blue sample arrays. On the other hand, in the case of video coding, each pixel is typically represented by a luminance format and a chrominance format or color space such as YCbCr, for example, and the YCbCr includes a luminance component indicated by Y (and sometimes L may be used instead), and two chrominance components indicated by Cb and Cr. The luminance (or, abbreviated, brightness) component Y represents the lightness or gray level intensity (such as in a grayscale video), while the two chrominance (or, abbreviated, chroma) components Cb and Cr represent chrominance information components or color information components. Therefore, a video in the YCbCr format includes a luminance sample array of luminance sample values (Y) and two chrominance sample arrays of chrominance values (Cb and Cr). A video in the RGB format may be transformed or converted to the YCbCr format, and vice versa, and the process is also known as color conversion or color transformation. If the video is monochrome, the video may include only a luminance sample array. Therefore, the video may be, for example, an array of luminance samples in a monochrome format, or two corresponding arrays of chroma samples and an array of luminance samples in color formats of 4:2:0, 4:2:2, and 4:4:4.

[0141] Multiple embodiments of the video encoder 20 may include a video segmentation unit (not shown in FIG. 2), and the video segmentation unit segments the video into a plurality of video blocks (typically non-overlapping). 201configured to segment the video 17 into them. These blocks may also be referred to as root blocks, macroblocks (H.264 / AVC), or coding tree blocks (CTB) or coding tree units (CTU) (H.265 / HEVC and VVC). The video segmentation unit may be configured to use the same block size for all of the video of a video sequence and to use a corresponding grid that defines the block size, or may be configured to vary the block size between multiple videos or between multiple subsets or groups of videos and to segment each video into corresponding blocks.

[0142] In a further embodiment, the video encoder may be configured to directly receive, for example, one block forming the video 17, some of the plurality of blocks, or all of the plurality of blocks, etc., of the video 17 201 blocks. The video block 201 may also be referred to as the current video block or the video block being coded.

[0143] Similar to the video 17, the video block 201 is also considered or may be considered to be a two-dimensional array or matrix of samples having intensity values (sample values), although of dimensions smaller than the video 17. In other words, for example, the block 201 is one sample array (e.g., a luminance array in the case of a monochrome video 17, or a luminance array or a chrominance array in the case of a color video), or three sample arrays (e.g., a luminance and two chrominance arrays in the case of a color video 17), or any other number of arrays and / or other types of arrays depending on the color format applied. The number of samples in the horizontal and vertical directions (or axes) of the block 201 defines the size of the block 201 . Thus, the block may be, for example, an M×N (M columns × N rows) array of samples or an M×N array of transform coefficients.

[0144] The multiple embodiments of the video encoder 20 shown in FIG. 2 may be configured to encode the video 17 block by block, for example, by performing encoding and prediction for each block. 201 For example, it may be configured to encode the video 17 block by block, such as performing encoding and prediction for each block.

[0145] Residual calculation

[0146] The residual calculation unit 204 may be configured to calculate a residual block 205 (also referred to as the residual 205) in the sample region and obtain the residual block 205 in the sample region by subtracting the sample values of the prediction block 265 from the sample values of the video block, for example, for each sample (for each pixel) of the video block. 201 Based on the video block and the prediction block 265 (further details about the prediction block 265 will be described later), the residual block 205 is calculated by subtracting the sample values of the prediction block 265 from the sample values of the video block. 201 and obtain the residual block 205 in the sample region.

[0147] Transformation

[0148] The transformation processing unit 206 may be configured to apply a transformation such as a discrete cosine transform (DCT) or a discrete sine transform (DST) to the sample values of the residual block 205 to obtain transformation coefficients 207 in the transformation region. The transformation coefficients 207 may also be referred to as transformation residual coefficients and represent the residual block 205 in the transformation region.

[0149] The conversion processing unit 206 may be configured to apply integer approximations of DCT / DST such as conversion specified for H.265 / HEVC. Compared with the orthogonal DCT transform, such integer approximations are typically scaled by a certain factor. To preserve the norm of the residual blocks processed by the forward and inverse transforms, a plurality of additional scaling factors are applied as part of the conversion process. Those scaling factors are typically selected based on specific constraints such as scaling factors that are powers of two for shift operations, the bit depth of the transform coefficients, the trade-off between accuracy and implementation cost, etc. Specific scaling factors are specified, for example, for the inverse transform by the inverse conversion processing unit 212 (and, for example, the corresponding inverse conversion by the inverse conversion processing unit 312 in the video decoder 30), and in the encoder 20, for example, the corresponding scaling factors for the forward transform may be specified accordingly by the conversion processing unit 206.

[0150] Multiple embodiments of the video encoder 20 (each, the conversion processing unit 206) may be configured to output conversion parameters such as, for example, one or more types of conversions, which are directly encoded or compressed by, for example, the entropy encoding unit 270, such that, for example, the video decoder 30 may receive and use those multiple conversion parameters for decoding.

[0151] Quantization

[0152] The quantization unit 208 may be configured to quantize the transform coefficients 207, for example, by applying scalar quantization or vector quantization, to obtain the quantized coefficients 209. Those quantized coefficients 209 may also be referred to as quantized transform coefficients 209 or quantized residual coefficients 209.

[0153] The quantization process can reduce the bit depth associated with some or all of the conversion coefficients 207. For example, during quantization, a rounding operation may be performed on an n-bit conversion coefficient to obtain an m-bit conversion coefficient, where n is greater than m. The degree of quantization may be modified by adjusting the quantization parameter (QP). For example, in the case of scalar quantization, it is possible to apply multiple different scalings to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. The applicable quantization step size may be indicated by the quantization parameter (QP). The quantization parameter may be, for example, an index into a predefined set of applicable quantization step sizes. For example, a small quantization parameter may correspond to fine quantization (a small quantization step size), a large quantization parameter may correspond to coarse quantization (a large quantization step size), or the reverse correspondence is also possible. Quantization may include division by the quantization step size. For example, the corresponding and / or inverse dequantization by the inverse quantization unit 210 may include multiplication by that quantization step size. For example, multiple embodiments according to some standards such as HEVC may be configured to determine the quantization step size using the quantization parameter. Generally, the quantization step size may be calculated based on the quantization parameter using a fixed-point approximation of an equation that includes division. An additional scaling factor may be introduced for quantization and dequantization to restore the norm of the residual block that may be modified due to the scaling used in the fixed-point approximation of the equation for the quantization step size and quantization parameter. In one exemplary implementation, the scaling of inverse transformation and dequantization may be combined. Alternatively, a customized quantization table may be used and sent, for example, by signaling in the bitstream from the encoder to the decoder.Quantization is an operation that involves loss, and that loss increases as the quantization step size increases.

[0154] Multiple embodiments of the video encoder 20 (each, quantization unit 208) may be configured to output a quantization parameter (QP) directly by, for example, the entropy encoding unit 270 or to output a quantization parameter (QP) encoded by the entropy encoding unit 270, such that, for example, the video decoder 30 may receive and apply the quantization parameter for decoding.

[0155] Inverse quantization

[0156] The inverse quantization unit 210 is configured to apply an inverse quantization of the quantization unit 208 to the quantized coefficients, for example, by applying a scheme inverse to the quantization scheme applied by the quantization unit 208 based on or using the same quantization step size as the quantization unit 208, to obtain dequantized coefficients 211. The dequantized coefficients 211 may also be referred to as dequantized residual coefficients 211 and typically correspond to the transform coefficients 207, although they are not the same as the transform coefficients due to loss by quantization.

[0157] Inverse transform

[0158] The inverse transform processing unit 212 is configured to apply an inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT), inverse discrete sine transform (DST), or other inverse transform, to obtain a residual block 213 (or, corresponding dequantized coefficients 213) reconstructed in the sample region. The reconstructed residual block 213 may also be referred to as the (reconstructed) transform block 213.

[0159] Reconstruction

[0160] The reconstruction unit 214 (e.g., an adder such as an adder 214 for obtaining a sum) is configured to obtain a reconstructed block 215 in the sample domain by adding, for example, the sample values of the reconstructed residual block 213 and the sample values of the prediction block 265 for each sample, thereby adding the (reconstructed) transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265.

[0161] Filtering

[0162] The loop filter unit 220 (or, simply, "loop filter" 220) is configured to filter the reconstructed block 215 to obtain a filtered block 221, or generally, is configured to filter the reconstructed samples to obtain filtered samples. The loop filter unit is configured, for example, to smooth pixel transitions or, if not, to improve video quality. The loop filter unit 220 may include a deblocking filter, a sample adaptive offset (SAO) filter, or one or more other filters such as, for example, a bidirectional filter, an adaptive loop filter (ALF), a sharpening filter , a smoothing filter, or a collaborative filter, or one or more loop filters such as a combination of any of them. The loop filter unit 220 is shown as an in-loop filter in FIG. 2, but in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 may also be referred to as a filtered and reconstructed block 221. In the present disclosure, an improved loop filter, particularly an improved deblocking filtering device, is provided and will be introduced in detail later.

[0163] Multiple embodiments of the video encoder 20 (each loop filter unit 220) may be configured to output loop filter parameters (such as sample adaptive offset information) directly by the entropy encoding unit 270 or to output loop filter parameters encoded by the entropy encoding unit 270, whereby, for example, the decoder 30 may receive and apply the same loop filter parameters or respective loop filters for decoding.

[0164] Decoded video buffer

[0165] The decoded video buffer (DPB) 230 may be a memory for storing reference video or, generally, reference video data for encoding video data by the video encoder 20. The DPB 230 may be composed of any of various memory devices such as a dynamic random access memory (DRAM) or other types of memory devices, and the dynamic random access memory (DRAM) includes synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), and resistive RAM (RRAM). The decoded video buffer (DPB) 230 may be configured to store one or more filtered blocks 221. The decoded video buffer 230 may further be configured to store other previously filtered blocks such as, for example, blocks 221 that have been previously reconstructed and filtered of the same current video or multiple different videos such as, for example, previously reconstructed video, i.e., decoded video (and corresponding reference blocks and samples) and / or partially reconstructed current video (and corresponding reference blocks and samples) for, for example, inter-frame prediction. The decoded video buffer (DPB) 230 may also be configured to store one or more reconstructed unfiltered blocks 215, or generally, reconstructed unfiltered samples, or any other further processed version of a reconstructed block or sample, for example, when the reconstructed block 215 has not been filtered by the loop filter unit 220.

[0166] Mode Selection (Segmentation and Prediction)

[0167] The mode selection unit 260 includes a segmentation unit 262, an inter-frame prediction unit 244, and an intra-frame prediction unit 254, and, for example, the original block 201 (the current block of the current video 17 201Original video data such as, and, for example, the decoded video buffer 230 or one or more previously decoded videos from other buffers (such as, for example, a line buffer 216 etc.), and / or reconstructed video data such as, for example, samples or blocks of the same (current) video that have been reconstructed and filtered and / or not filtered are received or acquired. The reconstructed video data is used, for example, as reference video data for prediction such as inter-frame prediction or intra-frame prediction to obtain the prediction block 265 or the predictor 265.

[0168] The mode selection unit 260 may be configured to determine or select a current block prediction mode (including the case where there is no segmentation) and a segmentation for a prediction mode (such as, for example, an intra-frame prediction mode or an inter-frame prediction mode, etc.), and generate a corresponding prediction block 265, and the corresponding prediction block 265 is used for calculating the residual block 205 and for reconstructing the reconstruction block 215.

[0169] Multiple embodiments of the mode selection unit 260 may be configured to select a partitioning and prediction mode (e.g., from among the partitioning and prediction modes supported by or available to the mode selection unit 260), where those partitioning and prediction modes provide the best match, i.e., the smallest residual (the smallest residual implies better compression for transmission or storage) or the smallest signaling overhead (the smallest signaling overhead implies better compression for transmission or storage), or consider or balance both the partitioning and prediction modes. The mode selection unit 260 may be configured to determine the partitioning and prediction modes based on rate-distortion optimization (RDO), i.e., select a prediction mode that provides the smallest rate distortion. The terms "best," "smallest," "optimal," etc. in this context do not necessarily refer to the overall "best," "smallest," "optimal," etc., and also refer to achieving an end criterion or selection criterion such that a value exceeds or falls below a threshold or other constraint that may lead to a "near-optimal selection" but reduces complexity and processing time.

[0170] In other words, the partitioning unit 262 may be configured to repeatedly use, for example, quadtree partitioning (QT), binary tree partitioning (BT), ternary tree partitioning (TT), or any combination thereof, to partition the block into smaller block partitions or sub-blocks (which form blocks again), and, for example, perform a prediction for each of those block partitions or sub-blocks, where the mode selection includes the selection of the tree structure of the block 201 to be partitioned, and the prediction mode is applied to each of those block partitions or sub-blocks. 201

[0171] In the following description, the partitioning and prediction processing (e.g., by the partitioning unit 260) performed by the exemplary video encoder 20 will be described in more detail.

[0172] compartmentalization

[0173] The partitioning unit 262 partitions the current block into smaller partitions, e.g., smaller blocks of square size or rectangular size. 201 may be partitioned (or distributed) into smaller blocks (which may also be referred to as subblocks). These smaller blocks may be further partitioned into smaller partitions. This partitioning may also be referred to as tree partitioning or hierarchical tree partitioning, where the partitioning may be recursive, such that the root block at root tree level 0 (hierarchical level 0, depth 0) is partitioned into two or more blocks at the next subtree level, such as a node at tree level 1 (hierarchical level 1, depth 1), which may then be partitioned again into two or more blocks at the next lower level, such as tree level 2 (hierarchical level 2, depth 2), until the partitioning is terminated, for example because a termination criterion has been achieved, such as reaching a maximum tree depth or a minimum block size. The blocks that are not further partitioned are also referred to as leaf blocks or leaf nodes of the tree. A tree that uses partitioning into two partitions is called a binary tree. segmentation A tree that uses a partitioning into three partitions, called a ternary tree (BT), is called a ternary tree. segmentation A tree that uses a partitioning into four partitions is called a quadtree. segmentation In this disclosure, when applying the SBT coding tool during inter-frame prediction, the coding block is divided into transform blocks.

[0174] As mentioned above, the term "block" as used herein may be a portion of a video, particularly a square or rectangular portion. For example, referring to HEVC and VVC, a block may be a Coding Tree Unit (CTU), a Coding Unit (CU), a Prediction Unit (PU), and a Transform Unit (TU), and / or a corresponding block such as, for example, a Coding Tree Block (CTB), a Coding Block (CB), a Transform Block (TB), or a Prediction Block (PB), or may correspond to those blocks.

[0175] For example, a Coding Tree Unit (CTU) may be a CTB of luminance samples, two corresponding CTBs of chrominance samples of a video having three sample arrays, or a CTB of a monochrome video or samples of a video coded using three individual color planes and syntax configurations used to code the samples, or may include these. Correspondingly, a Coding Tree Block (CTB) may be an N×N block of samples for some value of N, whereby the division of a plurality of CTBs of a component is a segmentation. A Coding Unit (CU) may be a Coding Block of luminance samples, two corresponding Coding Blocks of chrominance samples of a video having three sample arrays, or a Coding Block of a monochrome video or samples of a video coded using three individual color planes and syntax configurations used to code the samples, or may include these. Correspondingly, a Coding Block (CB) may be an M×N block of samples for some values of M and N, whereby the division of a plurality of Coding Blocks of a CTB is a segmentation.

[0176] In multiple embodiments, for example, according to HEVC, by using a quadtree structure shown as a coding tree, a coding tree unit (CTU) may be distributed into multiple CUs. The decision of whether to code a video region using inter-frame (temporal) prediction or intra-frame (spatial) prediction is made at the CU level. Each CU may further be distributed into one, two, or four PUs according to the PU distribution type. Inside a certain PU, the same prediction process is applied, and relevant information is sent to the decoder based on the PU. After obtaining a residual block by applying a prediction process based on the PU distribution type, the CU may be segmented into transform units (TUs) according to another quadtree structure similar to the coding tree for a certain CU.

[0177] In multiple embodiments, for example, according to the latest video coding standard under development, currently referred to as Versatile Video Coding (VVC), quadtree and binary tree (QTBT) partitioning are used to partition coding blocks. In the QTBT block structure, a CU may have either a square shape or a rectangular shape. For example, a coding tree unit (CTU) is first partitioned by a quadtree structure. The quadtree leaf nodes are further partitioned by a binary tree structure or a triple tree structure. The partitioned leaf nodes are called coding units (CUs), and the subdivision is used for prediction processing and transformation processing without any further subdivision. This means that CUs, PUs, and TUs have the same block size within the QTBT coding block structure. At the same time, for example, multiple partitions such as ternary tree partitioning have also been proposed to be used together with the QTBT block structure.

[0178] In one example, the mode selection unit 260 of the video encoder 20 may be configured to execute any combination of the multiple partitioning techniques described herein.

[0179] As described above, the video encoder 20 is configured to determine or select the best or optimal prediction mode from a set of (predetermined) multiple prediction modes. That set of prediction modes may include, for example, an intra prediction mode and / or an inter prediction mode.

[0180] Intra prediction

[0181] The set of intra prediction modes may include, for example, non - directional modes such as DC (or average) mode and planar mode, or 35 different intra prediction modes such as directional modes defined in HEVC, or, for example, non - directional modes such as DC (or average) mode and planar mode, or 67 different intra prediction modes such as directional modes defined in VVC.

[0182] The intra prediction unit 254 is configured to generate an intra prediction block 265 using a plurality of reconstructed samples of a plurality of adjacent blocks of the same current video according to a certain intra prediction mode from the set of intra prediction modes.

[0183] The intra prediction unit 254 (or generally the mode selection unit 260) is further configured to (corresponding to that of Figure 1A output an intra prediction parameter (or generally information indicating the intra prediction mode selected for that block) to the entropy encoding unit 270 in the form of a syntax element 266 for inclusion in the encoded video data 21, whereby, for example, the video decoder 30 may receive and use that prediction parameter for decoding. ) encoded video data 271

[0184] Inter prediction

[0185] ​The set of inter-frame prediction modes (or possible inter-frame prediction modes) depends on available reference video (i.e., for example, at least previously partially decoded video stored in DBP230), and whether only the whole or part of the reference video, such as the search window area around the area of the current block of the reference video, is used to search for the best matching reference block, and / or other inter-frame prediction parameters such as whether pixel interpolation completion such as half-pel / quarter-pel and / or quarter-pel interpolation completion is applied, etc.

[0186] In addition to the above prediction modes, a skip mode and / or a direct mode may be applied.

[0187] The inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (neither of which is shown in FIG. 2). The motion estimation unit receives or acquires, for motion estimation, a video block 201 (the current video block of the current video 17 201 ), and at least one or a plurality of previously reconstructed blocks such as the decoded video 231, or, for example, the reconstructed blocks of one or more other / different previously decoded videos 231. For example, the video sequence may include the current video and the previously decoded video 231. In other words, the current video and the previously decoded video 231 may be part of a sequence of videos forming the video sequence, or may form a sequence of videos forming the video sequence.

[0188] The encoder 20 may be configured to, for example, select a reference block from a plurality of reference blocks of the same or different videos among a plurality of other videos, and provide, as inter-frame prediction parameters to the motion estimation unit, a reference video (or a reference video index), and / or an offset (spatial offset) between the position (x coordinate, y coordinate) of the reference block and the position of the current block. This offset is also referred to as a motion vector (MV).

[0189] The motion compensation unit is configured to, for example, obtain inter-frame prediction parameters such as receiving the inter-frame prediction parameters, and perform inter-frame prediction based on or using the inter-frame prediction parameters, thereby obtaining an inter-frame prediction block 265. The motion compensation performed by the motion compensation unit may include extracting or generating a prediction block based on a motion vector / block vector determined by motion estimation, and it is possible to perform interpolation to sub-pixel accuracy. Interpolation filtering can generate additional pixel samples from known pixel samples, and thus potentially increase the number of candidate prediction blocks that may be used to encode a video block. When receiving a motion vector for a current video block's PU, the motion compensation unit can position the prediction block indicated by the motion vector in one of the reference video lists.

[0190] The motion compensation unit may also generate a plurality of blocks and syntax elements associated with the video slice for use when the video decoder 30 decodes the video blocks of the video slice. In the present disclosure, when sub-block transform (SBT) (such as when applying SBT coding tools) is enabled during inter-frame prediction, the coding block is divided into a plurality of transform blocks.

[0191] Entropy coding

[0192] The entropy encoding unit 270 applies, for example, an entropy encoding algorithm or scheme (such as a variable length coding (VLC) scheme, a context adaptive VLC scheme (CAVLC), an arithmetic coding scheme, binarization, a context adaptive binary arithmetic coding (CABAC), a syntax-based context adaptive binary arithmetic coding (SBAC), a probabilistic interval segmentation entropy (PIPE) coding, or other entropy encoding methods or techniques) to, for example, quantization coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters, and / or other syntax elements, or applies bypass (non-compression), and may output via output 272, for example, in the form of an encoded bitstream 21. ( Encoded video data 21 ) encoded video data 271 is configured to obtain, whereby, for example, the video decoder 30 can receive and use those parameters for decoding. The encoded bitstream 21 may be transmitted to the video decoder 30, or may be stored in memory for later transmission or retrieval by the video decoder 30. In the present disclosure, a plurality of syntax elements such as cu_sbt_flag and cu_sbt_horizontal_flag may be encoded as the bitstream 21.

[0193] Other structural variations of the video encoder 20 may be used to encode the video stream. For example, the non-transform-based encoder 20 may directly quantize the residual signal for a particular block or frame without using the transform processing unit 206. In other implementations, the encoder 20 may have a quantization unit 208 and an inverse quantization unit 210 that are combined to form a single unit.

[0194] Decoder and decoding method

[0195] FIG. 3 shows an example of a video decoder 30, which is configured to implement multiple techniques of this application. The video decoder 30 is configured to receive, for example, the encoded video data 21 (such as the encoded bitstream 21 etc.) encoded by the encoder 20 and obtain the decoded video 331. The encoded video data or bitstream includes information for decoding encoded video data such as data representing video blocks of an encoded video slice and related syntax elements. ( encoded video data 21 ) encoded video data 271 In the example of FIG. 3, the decoder

[0196] includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (such as an adder 314 for obtaining a sum etc.), a loop filter 320, a decoded video buffer 330, an inter-frame prediction unit 344, and an intra-frame prediction unit 354. The inter-frame prediction unit 344 may be a motion compensation unit or may include a motion compensation unit. The video decoder 300 may execute a decoding path that is generally reverse to the encoding path described with respect to the video encoder 300 from FIG. 2 in some of the multiple examples. 200

[0197] encoder 200As described with respect to, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 344, and the intra-frame prediction unit 354 are also considered to form the "embedded decoder" of the video encoder 20. Therefore, the inverse quantization unit 310 may be functionally the same as the inverse quantization unit 110, the inverse transform processing unit 312 may be functionally the same as the inverse transform processing unit 212, the reconstruction unit 314 may be functionally the same as the reconstruction unit 214, the loop filter 320 may be functionally the same as the loop filter 220, and the decoded picture buffer 330 may be functionally the same as the decoded picture buffer 230. Therefore, the video encoder 200 The descriptions provided for each unit and function of are correspondingly applicable to each unit and function of the video decoder 30.

[0198] Entropy decoding

[0199] The entropy decoding unit 304 analyzes the bitstream 21 (or, generally, ( the encoded video data 21 ) encoded video data 271 ) and, for example, ( the encoded video data 21 ) encoded video data 271to obtain quantized coefficients 309 and / or decoded coding parameters (not shown in FIG. 3 ), such as any or all of inter-frame prediction parameters (e.g., reference picture index and motion vectors), intra-frame prediction parameters (e.g., intra-frame prediction mode or index), transform parameters, quantization parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 may be configured to apply a decoding algorithm or scheme corresponding to the encoding scheme, as described with respect to the entropy encoding unit 270 of the encoder 20. The entropy decoding unit 304 may be further configured to provide the inter-frame prediction parameters, intra-frame prediction parameters, and / or other syntax elements to the mode selection unit 360, as well as provide other parameters to other units of the decoder 30. The video decoder 30 may receive syntax elements at a video slice level and / or a video block level.

[0200] inverse quantization

[0201] The inverse quantization unit 310 may be implemented (e.g., by parsing and / or decoding, e.g., by the entropy decoding unit 304). ( Encoded video data 21 ) encoded video data 271 quantization parameter (QP) (or, in general, information regarding inverse quantization) and the quantized coefficients from video encoder 20, and, based on the quantization parameters, apply inverse quantization to quantized decoded coefficients 309 to obtain dequantized coefficients 311, which may also be referred to as transform coefficients 311. The inverse quantization process may include using the quantization parameter determined by video encoder 20 for each video block in the video slice to determine the degree of quantization, and similarly, the degree of inverse quantization that needs to be applied.

[0202] Reverse transformation

[0203] The inverse transformation processing unit 312 may also be configured to receive the dequantized coefficients 311, referred to as the transformation coefficients 311, and apply a transformation to those dequantized coefficients 311 in order to obtain the reconstructed residual block 213 in the sample domain. Those reconstructed residual blocks 213 may also be referred to as transformation blocks 313. The transformation may be an inverse transformation such as, for example, an inverse DCT, an inverse DST, an inverse integer transformation, or a conceptually similar inverse transformation process, etc. The inverse transformation processing unit 312 further (e.g., by the entropy decoding unit 304, for example, by analyzing and / or decoding) ( encoded video data 21 ) encoded video data 271 may be configured to receive transformation parameters or corresponding information therefrom and determine the transformation to be applied to the dequantized coefficients 311.

[0204] Reconstruction

[0205] The reconstruction unit 314 (e.g., an adder or an adder 314 for obtaining a sum, etc.) may be configured to add the sample values of the reconstructed residual block 313 and the sample values of the prediction block 365 to the prediction block 365, for example, by adding the reconstructed residual block 313, so as to obtain the reconstructed block 315 in the sample domain.

[0206] Filtering

[0207] The loop filter unit 320 (present either in or after the coding loop) is configured to filter the reconstructed block 315 to obtain the filtered block 321, thereby, for example, smoothing pixel transitions or, if not, improving video quality. The loop filter unit 320 is a non-blocking filter, a sample adaptive offset (SAO) filter, or, for example, a bidirectional filter, an adaptive loop filter (ALF), sharpeningfilter One or more other filters, such as a smoothing filter or a collaborative filter, or one or more loop filters such as any combination thereof may be included. The loop filter unit 320 is shown as an in-loop filter in FIG. 3, but in other configurations, the loop filter unit 320 may be implemented as a post-loop filter. In the present disclosure, an improved loop filter, particularly an improved non-blocking filter device, will be provided and will be described in detail later.

[0208] Decoded video buffer

[0209] The decoded video block 321 of the video is then stored in the decoded video buffer 330, and the decoded video buffer 330 stores the decoded video 331 as a reference video for subsequent motion compensation for the display of each of the other videos and / or outputs.

[0210] The decoder 30 is configured to output the decoded video 311, for example, via the output 312, for presentation or viewing by the user.

[0211] Prediction

[0212] intra-frame The prediction unit 344 may be the same as the inter-frame prediction unit 244 (in particular, the motion compensation unit), and the intra-frame prediction unit 354 may be functionally the same as the inter-frame prediction unit 254 (for example, by the entropy decoding unit 304, for example, by analyzing and / or decoding) ( Encoded video data 21 ) encoded video data 271Based on the segmentation parameters and / or prediction parameters received from [[ID=]], and / or the respective information, perform the determination of allocation or the determination of segmentation and prediction. The mode selection unit 360 may be configured to perform prediction (intra-frame prediction or inter-frame prediction) for each block based on the (filtered or unfiltered) reconstructed video, block, or respective sample, to obtain the prediction block 365.

[0213] When the video slice is coded as an intra-coding (I) slice, the intra-frame prediction unit 354 of the mode selection unit 360 is configured to generate a prediction block 365 for the video block of the current video slice based on the data from the previously decoded blocks of the current video and the signaled intra-frame prediction mode. When the video is coded as an inter-coding (i.e., B or P) slice, the inter-frame prediction unit 344 (such as a motion compensation unit, etc.) of the mode selection unit 360 is configured to generate a prediction block 365 for the video block of the current video slice based on the motion vector and other syntax elements received from the entropy decoding unit 304. For inter-frame prediction, the prediction block may be generated from one of the plurality of reference videos in one of the plurality of reference video lists. The video decoder 30 may construct the reference frame lists, list 0 and list 1, using a default construction technique based on the reference videos stored in the DPB 330. In addition to or as an alternative to slices (such as video slices), multiple tile groups (such as video tile groups) and / or multiple tiles (such as video tiles) may be used for multiple embodiments, or the same or similar things may be applied to multiple embodiments thereof, such as being able to code video using I, P, or B tile groups and / or tiles.

[0214] The mode selection unit 360 is configured to determine prediction information about video blocks of a current video slice by analyzing motion vectors and other syntax elements, and use the prediction information to generate a prediction block for the currently decoded video block. For example, the mode selection unit 360 uses some of the plurality of received syntax elements to determine a prediction mode (e.g., intra prediction or inter prediction, etc.) used to code a plurality of video blocks of a video slice, an inter prediction slice type (e.g., B slice, P slice, or GPB slice, etc.), construction information for one or more of a plurality of reference picture lists for that slice, a motion vector for each inter-coded video block of that slice, an inter prediction state for each inter-coded video block of that slice, and other information for decoding video blocks in the current video slice. For example, in addition to a slice (e.g., a video slice) or as an alternative to a slice (e.g., a video slice), multiple tile groups (e.g., video tile groups) and / or tiles (e.g., video tiles) can be used, and for multiple embodiments or multiple of those embodiments may apply the same or similar things.

[0215] Multiple embodiments of the video decoder 30 shown in FIG. 3 may be configured to segment and / or decode video by using multiple slices (also referred to as video slices), and the video may be segmented into one or more slices (typically non-overlapping) or decoded using one or more of those slices, and each slice may include one or more blocks (e.g., CTUs, etc.).

[0216] The video decoder shown in FIG. 3 300Multiple embodiments may be configured to segment and / or decode video by using tile groups (also referred to as video tile groups) and / or multiple tiles (also referred to as video tiles), where the video may be segmented into one or more tile groups (typically non-overlapping) or decoded using one or more of those tile groups, and each tile group may include one or more blocks (such as one or more CTUs, etc.) or one or more tiles, and each tile may, for example, be rectangular in shape and may include one or more blocks (such as one or more CTUs, etc.) such as a complete block or a fragmented block.

[0217] Using multiple other variations of the video decoder 30, ( the encoded video data 21 ) encoded video data 271 may be decoded. For example, the decoder 30 may generate an output video stream without using the loop filtering unit 320. For example, a non-transform-based decoder 30 may directly inverse quantize the residual signal for a particular block or frame without using the inverse transform processing unit 312. In other implementations, the video decoder 30 may have an inverse quantization unit 310 and an inverse transform processing unit 312 combined in a single unit.

[0218] It should be understood that in the encoder 20 and the decoder 30, the processing result of the current step may be further processed and then, in turn, the processing result may be output to the next step. For example, after interpolation filtering, motion vector derivation, or loop filtering, further operations such as clip or shift may be performed on the processing result of interpolation filtering, motion vector derivation, or loop filtering.

[0219] (Including, but not limited to, the control point motion vector in affine mode, affine, plane, sub-block motion vectors in ATMVP mode, and temporal motion vectors, etc.) It should be noted that further operations may be applied to the derived motion vector of the current block. For example, the value of the motion vector is limited to a predefined range according to its representation bits. When the representation bits of the motion vector are bitDepth, the range is -2^(bitDepth - 1) to 2^(bitDepth - 1) - 1, where "^" means the exponential function. For example, when bitDepth is set equal to 16, the range is -32768 to 32767, and when bitDepth is set equal to 18, the range is -131072 to 131071. For example, the values of the derived motion vectors (such as the MVs of four 4×4 sub-blocks in one 8×8 block) are restricted such that the maximum difference among the integer parts of the MVs of those four 4×4 sub-blocks is no more than N pixels, such as no more than 1 pixel.

[0220] FIG. 4 is a schematic diagram of a video coding device 400 according to one embodiment of the present disclosure. The video coding device 400 is suitable for implementing a plurality of disclosed embodiments as described herein. In one embodiment, the video coding device 400 may be a decoder such as the video decoder 30 of FIG. 1A or an encoder such as the video encoder 20 of FIG. 1A.

[0221] Video coding device 400 includes an inlet port 410 (or input port 410) and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an outlet port 450 (or output port 450) for transmitting data, and a memory 460 for storing data. Video coding device 400 may also include optoelectronic (OE) components and electro-optical (EO) components coupled to inlet port 410, receiver unit 420, transmitter unit 440, and outlet port 450 for the exit or entry of optical or electrical signals.

[0222] Processor 430 is implemented by hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (such as a multi-core processor, etc.), FPGA, ASIC, and DSP. Processor 430 communicates with inlet port 410, receiver unit 420, transmitter unit 440, outlet port 450, and memory 460. Processor 430 includes a coding module 470. Coding module 470 implements a plurality of disclosed embodiments as described above. For example, coding module 470 implements, processes, prepares, or provides various coding operations. Therefore, including coding module 470 provides a substantial improvement to the functionality of video coding device 400 and results in the conversion of video coding device 400 to different states. Alternatively, coding module 470 is implemented as instructions stored in memory 460 and executed by processor 430.

[0223] Memory 460 may include one or more disks, tape drives, and solid state drives, and be used as an overflow data storage device to store such programs when selecting a program for execution and to store instructions and data read during program execution. Memory 460 may be, for example, volatile and / or non-volatile, and may be read only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), and / or static random access memory (SRAM).

[0224] FIG. 5 is a simplified block diagram of an apparatus 500 that may be used as either or both of the source device 12 and the destination device 14 of FIG. 1 according to one exemplary embodiment.

[0225] The processor 502 in apparatus 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or devices capable of manipulating or processing information, whether existing currently or developed later. The disclosed implementation may be realized, as shown, using a single processor, such as processor 502, for example, but more than one processor may be used to achieve advantages in speed and efficiency.

[0226] In one implementation, the memory 504 in the device 500 may be a read-only memory (ROM) device or a random access memory (RAM) device. As the memory 504, any other suitable type of storage device may be used. The memory 504 may include code and data 506 that the processor 502 accesses using the bus 512. The memory 504 may further include an operating system 508 and an application program 510. The application program 510 includes at least one program, and at least one of those programs enables the processor 502 to execute the methods described herein. For example, the application program 510 may include applications 1 through N, and those applications 1 through N further include a video coding application that executes the methods described herein.

[0227] The device 500 may also include one or more output devices such as a display 518. In one example, the display 518 may be a touch-sensitive display that combines a touch-sensitive element operable to detect touch input with a display. The display 518 may be coupled to the processor 502 via the bus 512.

[0228] Although shown as a single bus herein, the bus 512 of the device 500 may be composed of multiple buses. Further, the secondary storage device 514 may be directly coupled to other components of the device 500 or accessed via a network and may include a single integrated unit such as a memory card or multiple units such as multiple memory cards. In this way, the device 500 may be implemented in a variety of configurations.

[0229] Combined Inter-Frame Intra-Frame Prediction (CIIP)

[0230] Conventionally, a coding unit is either intra-frame prediction (i.e., using reference samples within the same video) or inter-frame prediction (i.e., using reference samples within multiple other videos). Combined inter-frame intra-frame prediction combines these two prediction approaches. Therefore, that prediction approach may also be referred to as combined inter-frame intra-frame prediction (CIIP). When enabling combined inter-frame intra-frame prediction, weights are applied to the intra-frame predicted samples and the inter-frame predicted samples, and the final prediction is derived as a weighted average sample.

[0231] A flag, the CIIP flag, is used to indicate the point in time when combined inter-frame intra-frame prediction is applied to a block.

[0232] The sub-block transform (SBT) coding tool divides (i.e., distributes or splits) an inter-frame prediction block (i.e., an inter-frame coding block, which is an abbreviation of the current coding block coded in the inter-frame prediction mode) into two transform blocks and performs a transform only on one of those transform blocks and not on the other. Those two transform blocks may be symmetric (i.e., two transform blocks of the same size) or asymmetric (i.e., for example, two transform blocks having the same width but a height ratio of 1:3, or, for example, two transform blocks having the same height but a width ratio of 1:3). Such partial transforms may cause block artifacts along the boundary between those two transform blocks. On the other hand, in the prior art that degrades subjective quality when enabling SBT, filtering those boundaries is not considered.

[0233] To reduce the block artifacts at the conversion block boundaries caused by the SBT, an improved filtering process is proposed. When detecting those boundaries that would be considered for filtering, the internal boundaries between multiple conversion blocks caused by the SBT are taken into account. Further, the prior art only considers the boundaries that overlap with the 8×8 grid. In the present invention, when applying asymmetric partitioning (i.e., distribution or splitting), even if the SBT internal boundary is not aligned with the 8×8 grid, that internal boundary will be considered as a filtering candidate. Also, by filtering the SBT internal boundary, the block artifacts caused by the SBT are reduced.

[0234] As shown in FIG. 6, further, the block 600 to which CIIP is applied may be divided into several sub-blocks. In FIG. 6, all of the sub-block boundaries within the CU are applied with combined inter-frame intra-frame prediction (CIIP). In one example, the sub-block 601 is derived by horizontally dividing the block, and each sub-block has the same width as the original block, but the height is 1 / 4 of the height of the original block.

[0235] In one example, the sub-block 602 is derived by vertically dividing the block, and each sub-block has the same height as the original block, but the width is 1 / 4 of the width of the original block. In the example shown in FIG. 6, the sub-division by the vertical division 602 and the corresponding boundaries are labeled. Here, the intra-frame block 600 is divided into four sub-divisions, namely sub0, sub1, sub2, and sub3. Three sub-division boundaries, i.e., the sub-division boundary A between sub-division 0 and sub-division 1, the sub-division boundary B between sub-division 1 and sub-division 2, and the sub-division boundary C between sub-division 2 and sub-division 3 are labeled, and a similar definition may be used in the example of the horizontal division 601.

[0236] Blocking artifacts usually cause the results associated with intra-frame prediction having more residual signals. Therefore, due to CIIP prediction, blocking artifacts may be incorporated. Those blocking artifacts occur not only at the boundaries of the CIIP blocks but also at the sub-block edges such as the vertical sub-block edges A, B, C, etc. inside the CIIP blocks in FIG. 6. Corresponding to the above cases, it is possible to identify the horizontal sub-block edges.

[0237] Blocking artifacts may occur at both the CIIP boundaries and the sub-block edges inside the CIIP blocks, but the distortions caused by those two boundaries may be different, and different boundary strengths may be required.

[0238] For example, when the intra-frame prediction mode of the CIIP block is the horizontal mode and the vertical division shown in FIG. 6 is applied, the CIIP itself may generate a plurality of sub-block edges, and three sub-blocks are generated.

[0239] As shown in FIG. 7, in order to reduce blocking artifacts, after horizontal division of the coding block 700 into the sub-block 701 or after vertical division of the coding block 700 into the sub-block 702, the sub-block boundaries are de-blocking filtered. FIG. 7 shows all the de-blocking of the sub-block edges in the CU to which combined inter-frame intra-frame prediction (CIIP) is applied.

[0240] Figure 8 shows all the non-blocking of sub-block TU boundaries in the CUs that overlap (are aligned with) an 8×8 sample grid starting from a sample in the upper left of a certain CU. As shown in Figure 8, after horizontal partitioning of the coding block 800 into sub-block 801, or after vertical partitioning of the coding block 800 into sub-block 802, only the sub-block boundaries that overlap the 8×8 sample grid are non-blocked, and the remaining sub-block edges are not non-blocked. This non-blocking has the advantage of reducing the computational complexity because only a few edges are non-blocked.

[0241] Figure 9 shows another alternative. Figure 9 shows all the non-blocking of sub-block edges in the CUs that overlap a 4×4 sample grid. In this case, after horizontal partitioning of the coding block 900 into sub-block 901, or after vertical partitioning of the coding block 900 into sub-block 902, all the sub-block boundaries that overlap the 4×4 sample grid are non-blocked.

[0242] FIG. 10 shows a weak filter that uses only 3 samples for decision-making and modifies 1 sample when the sub-block size is smaller than 8 samples in a direction orthogonal to the non-blocking direction. In the example of FIG. 6 using vertical divisions, when W is 16 samples, each sub-block has a width of 4 samples. In this case, as shown in FIG. 10, a weak filter that modifies at most 1 sample 10314 or 10331 along the sub-block boundary 1032 between sub-block 1031 and sub-block 1033 may be used. In the example shown in FIG. 10, filtering is performed, for example, orthogonal to the sub-block boundary 1032 between sub-block 1031 and sub-block 1033 and in each row of sub-blocks 1031 and 1033 adjacent to that sub-block boundary 1032. As shown in FIG. 10, a weak filter that modifies at most 1 sample 1108 or 10311 along the edge 1020 between adjacent block 1010 and current block 1030 may be used. In another example shown in FIG. 10, filtering is performed, for example, orthogonal to the edge 1020 between sub-block 1031 of block 1030 and adjacent block 1010 and in each row of sub-block 1031 or adjacent block 1010 adjacent to that edge 1020.

[0243] On the other hand, sub-block edges may also be caused by TU size limitations. In VTM3.0, the maximum TU size is 64×64 samples. As shown in FIG. 11, when CU1100 is 128×128 samples, CU1100 is divided into 4 TUs 1101, resulting in 4 TU boundaries 1102. When the maximum TU size is 64, CUs using combined inter-frame intra prediction are 128×128, and those CUs are divided into 4 TUs 1101, and the transformation is applied at a granularity of 64×64. It is necessary to non-block the TU boundary 1102 emphasized by the dashed line.

[0244] FIG. 12 shows a coding unit 1200, to which CIIP is applied and which is further divided into a plurality of transform units 1201. It is necessary to deblock a TU boundary 1202 highlighted by a dashed line.

[0245] FIG. 13 illustrates all deblocking of sub-block TU boundaries 1302 among a plurality of TUs 1301 in a CU 1300 that overlaps (is aligned with) an 8×8 sample grid starting from the top-left sample of the CU 1300.

[0246] TU boundaries inside a coding unit

[0247] When applying certain coding tools (such as sub-block transform, SBT, etc.), as shown in FIG. 14, TU edges 1402 among a plurality of TUs 1401 may occur inside a CU 1400. The coding unit 1400 is further divided into a plurality of transform units 1401 according to a sub-block transform tool. In such a case, it may also be necessary to deblock those internal TU edges 1402 inside the coding unit 1400.

[0248] When using SBT for a CU 1400 coded inter-frame, in the bitstream, SBT type and SBT position information are sent by signaling. As shown in FIG. 14, there are two SBT types and two SBT positions. For SBT-V (or SBT-H), the TU width (or height) may be equal to half of the CU width (or height) or equal to 1 / 4 of the CU width (or height), resulting in a 2:2 distribution or a 1:3 / 3:1 distribution. The 2:2 distribution is similar to a binary tree (BT) distribution, while the 1:3 / 3:1 distribution is similar to an asymmetric binary tree (ABT) distribution. In the ABT distribution, only a small area contains non-zero residuals. When one dimension of a CU is 8 in luminance samples, a 1:3 / 3:1 distribution along that dimension is not permitted. There are up to eight SBT modes for a certain CU.

[0249] For SBT-V and SBT-H, apply position-dependent transform core selection to the luminance conversion block (the chroma TB always uses DCT-2). The two positions of SBT-H and SBT-V are associated with multiple different core transforms. More specifically, the horizontal transform and the vertical transform for each SBT position are specified in Figure 14. For example, the horizontal transform and the vertical transform for SBT-V position 0 are DCT-8 and DST-7, respectively. When one side of the residual TU1401 is larger than 32, the transforms for both dimensions are set to DCT-2. Therefore, the sub-block transform jointly specifies the TU tiling, cbf, and the horizontal and vertical core transform types of the residual block.

[0250] The variable maxSbtSize is signaled by the SPS to specify the maximum CU size to which SBT can be applied. In the VTM7 reference software, for HD sequences and 4K sequences, the encoder sets maxSbtSize to 64, and for other smaller resolution sequences, maxSbtSize is set to 32.

[0251] SBT is not applied to CUs coded using the combined inter-intra mode or the TPM mode.

[0252] Similarly, Figure 15 shows the coding unit 1500, which is further divided into a plurality of transform units 1501 (A, B, C) according to a residual quad-tree (RQT) similar to the sub-block transform.

[0253] In the remaining part of this application, CIIP block: A coding block predicted by applying CIIP prediction; Intra block: A coding block predicted by applying intra prediction instead of CIIP prediction; Inter-frame block: A coding block predicted by applying inter-frame prediction instead of CIIP prediction; uses the word of.

[0254] Non-blocking filter and boundary strength

[0255] As described above, the present invention, particularly in the first and second aspects, includes performing a non-blocking filtering process on the conversion block boundary between the first conversion block and the second conversion block, at least based on the value of the boundary strength parameter. The boundary strength parameter is further described and defined in the following description (refer to Table 1).

[0256] Video coding schemes such as HEVC and VVC are designed in accordance with the principle that has achieved success in block-based hybrid video coding. Using this principle, a video is first divided into a plurality of blocks, and then each block is predicted by using intra-frame prediction or inter-frame prediction. Those multiple blocks are relatively coded from a plurality of adjacent blocks and approximated to the original signal so as to have a certain degree of similarity. Since the coded blocks only approximate the original signal, the differences between those approximated signals may cause discontinuities at the prediction and conversion block boundaries. These discontinuities are attenuated by a non-blocking filter.

[0257] The decision of whether to filter a certain block boundary uses bitstream information such as the prediction mode and motion vector. Some of the multiple coding conditions are more likely to generate strong block artifacts, and those strong block artifacts are represented by a so-called boundary strength (Bs or BS) variable, and the boundary strength variable is assigned for each block boundary and determined as shown in Table 1.

Table 1

[0258] Non-blocking is applied only to block boundaries where Bs is greater than 0 for the luminance component, and only to block boundaries where Bs is greater than 1 for the chrominance component. A larger value of Bs enables stronger filtering by using a larger clipping parameter value. The derivation condition of Bs reflects the probability that the strongest block artifacts appear at block boundaries predicted within the frame.

[0259] Normally, two adjacent blocks 1601 and 1602 of a boundary are labeled with P and Q as shown in FIG. 16. FIG. 16 shows the case of a vertical boundary. When considering a horizontal boundary, it is necessary to rotate FIG. 16 by 90 degrees clockwise. In that case, P will be on the upper side and Q will be on the lower side.

[0260] The method according to the first aspect of the present invention is illustrated in the flowchart of FIG. 17. The method is a non-blocking method for non-blocking a transform block boundary in a coding block in image encoding and / or image decoding. The coding block is coded in an inter-frame prediction mode. The coding block is divided into a transform block including a first transform block and a second transform block adjacent to the first transform block. In the method, the boundary between the first transform block and the second transform block is a transform block boundary, and when at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, step 1701 of determining a value of a boundary strength (BS) parameter for the boundary between the first transform block and the second transform block so as to be a first value, and a further step 1702 of performing a non-blocking filtering process on the boundary between the first transform block and the second transform block based at least on the value of the boundary strength parameter.

[0261] The method according to the second aspect of the present invention is illustrated in the flowchart of FIG. 18. The method is a non-blocking method for non-blocking block boundaries in coding blocks in image coding and / or image decoding, where the coding block is coded in an inter-frame prediction mode, the coding block is divided into a transform block including a first transform block and a second transform block adjacent to the first transform block, and the method includes, in response to a determination that it is necessary to filter a transform block boundary between the first transform block and the second transform block, a step 1801 of determining a value of a boundary strength parameter for the boundary between the first transform block and the second transform block such that it becomes a sub-block transform SBT boundary, and when at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, and a step 1802 of performing a non-blocking filtering process on the transform block boundary between the first transform block and the second transform block based at least on the value of the boundary strength parameter.

[0262] FIG. 19 illustrates a device according to a third aspect. The device 1900 includes a deblocking filter 1901 configured to deblock a transform block boundary within a coding block, where (the coding block is divided (distributed) into a plurality of transform blocks during an inter-frame prediction process, for example, when sub-block transform is enabled, the current coding unit is divided into two transform units, etc.) the coding block is coded (predicted) in an inter-frame prediction mode, the coding block includes a plurality of transform blocks, and the plurality of transform blocks include a first transform block and a second transform block adjacent to the first transform block. The deblocking filter 1901 has a boundary between the first transform block and the second transform block as a transform block boundary, and when at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, a determination module 1902 configured to determine a value of a boundary strength parameter for the boundary between the first transform block and the second transform block to be a first value, and a deblocking filtering module 1903 configured to perform a deblocking filtering process on the boundary between the first transform block and the second transform block based at least on the first value of the boundary strength parameter.

[0263] In one example, the deblocking filter 1901 may correspond to the loop filter 220 of FIG. 2. In another example, the deblocking filter 1901 may correspond to the loop filter 320 of FIG. 3. Correspondingly, in one example, an exemplary configuration of the device 1900 may correspond to the encoder 200 in FIG. 2. In another example, an exemplary configuration of the device 1900 may correspond to the decoder 300 in FIG. 3.

[0264] FIG. 20 illustrates a device according to a fourth aspect. The device 2000 includes a deblocking filter 2001 configured to deblock block boundaries within a coding block (coding unit), and the coding block is coded (predicted) in an inter-frame prediction mode (in particular, the coding block is coded in a sub-block transform SBT mode), (for example, when enabling sub-block transform, the current coding unit is split into two or more transform units as such, the coding block predicted inter-frame is split (distributed) into a plurality of transform blocks in the inter-frame prediction process) (for example, the transform blocks include adjacent in the vertical or horizontal direction p0 and q0 as such) the coding block includes a first transform block and a second transform block adjacent to the first transform block. The deblocking filter 2001 determines a value of a boundary strength parameter for the boundary between the first transform block and the second transform block to be a first value (for example, in response to a determination that it is necessary to filter the transform block boundary between the first transform block and the second transform block, as such, the boundary between the first transform block and the second transform block is a sub-block transform SBT boundary, and when at least one of the first transform block and the second transform block has one or more non-zero transform coefficients), and a determination module 2002 configured to, and at least based on the value of the boundary strength parameter, a deblocking module 2003 configured to perform a deblocking filtering process on the transform block boundary between the first transform block and the second transform block.

[0265] In one example, the deblocking filter 2001 may correspond to the loop filter 220 in FIG. 2. In other examples, the deblocking filter 2001 may correspond to the loop filter 320 in FIG. 3. Correspondingly, in one example, the exemplary configuration of the device 2000 may correspond to the encoder 200 in FIG. 2. In other examples, the exemplary configuration of the device 2000 may correspond to the decoder 300 in FIG. 3.

[0266] References Multipurpose Video Coding (Draft 3) is defined as VVC Draft 3.0 and can be found via the link: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 12_Macao / wg11 / JVET-L1001-v13.zip is discoverable.

[0267] In one example, according to 8.6.2.5 of VVC Draft 3.0 v9, 8.6.2.5 Derivation Process of Boundary Filtering Strength The input to this process is - the video sample array recPicture, - the position (xCb, yCb) specifying the top-left sample of the current coding block relative to the top-left sample of the current video, - the variable nCbW specifying the width of the current coding block, - the variable nCbH specifying the height of the current coding block, - the variable edgeType specifying whether to filter vertical (EDGE_VER) edges or horizontal edges (EDGE_HOR), - the 2D (nCbW) × (nCbH) array edgeFlags, and is The output of this process is the 2D (nCbW) × (nCbH) array bS specifying the boundary filtering strength. The variable xD i , yD j, xN, yN are - When edgeType is equal to EDGE_VER, xD i is set to (i << 3), yD j is set to (j << 2), xN is set to Max(0, (nCbW / 8) - 1), and yN is set to (nCbH / 4) - 1, - In other cases (when edgeType is equal to EDGE_HOR), xD i is set to (i << 2), yD j is set to (j << 3), xN is set to (nCbW / 4) - 1, and yN is set to Max(0, (nCbH / 8) - 1), is derived as. xD i (i = 0…xN) and yD j (j = 0…yN) for - When edgeFlags[xD i [yD j is 0, the variable bS[xD i [yD j is set to 0, such a derivation process is applied. - In other cases, it is as follows. - The sample values p0 and q0 are - When edgeType is equal to EDGE_VER, p0 is recPicture [xCb + xD i - 1][yCb + yD j , and q0 is recPicture [xCb + xD i [yCb + yD j , - In other cases (when edgeType is equal to EDGE_HOR), p0 is recPicture [xCb + xD i [yCb + yD j - 1], and q0 is recPicture [xCb + xD i [yCb + yD j , is derived as. - The variable bS[xD i [yD j is derived as follows. - When sample p0 or q0 is present in the coding block of a coding unit coded using an intra-frame prediction mode, bS[xD i [yD j is set to 2. - Otherwise, when the block edge is also a transform block edge and sample p0 or q0 is present in a transform block containing one or more non-zero transform coefficient levels, bS[xD i [yD j is set to 1. - Otherwise, when one or more of the following conditions are true, bS[xD i [yD j is set to 1. - For the prediction of the coding sub-block containing sample p0, a reference picture or a different number of motion vectors different from the reference picture or motion vectors for the prediction of the coding sub-block containing sample q0 is used. Note 1 - The determination of whether the reference pictures used for two coding sub-blocks are the same or different depends only on which pictures are referenced, regardless of whether the prediction is formed using an index to reference picture list 0 or an index to reference picture list 1, and regardless of whether the index positions in the reference picture list are different. Note 2 - The number of motion vectors used for the prediction of the coding sub-block having the top-left sample occupying (xSb, ySb) is equal to PredFlagL0[xSb][ySb] + PredFlagL1[xSb][ySb]. ​- Predict the coding sub-block containing sample p0 using one motion vector, predict the coding sub-block containing sample q0 using one motion vector, and the absolute difference between the horizontal or vertical components of the used motion vectors is 4 or more in units of 1 / 4 luminance samples. - Use two motion vectors and two different reference pictures to predict the coding sub-block containing sample p0, use two motion vectors for the same two reference pictures to predict the coding sub-block containing sample q0, and the absolute difference between the horizontal or vertical components of the two motion vectors used for predicting the two coding sub-blocks for the same reference picture is 4 or more in units of 1 / 4 luminance samples. - Use two motion vectors of the same reference picture to predict the coding sub-block containing sample p0, use two motion vectors of the same reference picture to predict the coding sub-block containing sample q0, and both of the following conditions are true. - The absolute difference between the horizontal or vertical components of the list 0 motion vectors used for predicting the two coding sub-blocks is 4 or more in 1 / 4 luminance samples, or the absolute difference between the horizontal or vertical components of the list 1 motion vectors used for predicting the two coding sub-blocks is 4 or more in 1 / 4 luminance samples. - The absolute difference between the horizontal or vertical component of the list 0 motion vector used for predicting the coding sub-block containing sample p0 and the horizontal or vertical component of the list 1 motion vector used for predicting the coding sub-block containing sample q0 is 4 or more in units of 1 / 4 luminance samples, or the absolute difference between the horizontal or vertical component of the list 1 motion vector used for predicting the coding sub-block containing sample p0 and the horizontal or vertical component of the list 0 motion vector used for predicting the coding sub-block containing sample q0 is 4 or more in units of 1 / 4 luminance samples. - Otherwise, the variable bS[xDi][yDj] is set to 0.

[0268] Furthermore, the VVC document refers to "coding tree semantics" and "sub-block transform (SBT)" as follows. 7.4.9.4 Coding tree semantics … … A cu_sbt_flag equal to 1 specifies that sub-block transform is used for the current coding unit. A cu_sbt_flag equal to 0 specifies that sub-block transform is not used for the current coding unit. When the cu_sbt_flag does not exist, its value is presumed to be equal to 0. Note - When using sub-block transform, the coding unit is split into two transform units, one of which has residual data and the other does not. A cu_sbt_horizontal_flag equal to 1 specifies that the current coding unit is split horizontally into two transform units. A cu_sbt_horizontal_flag[x0][y0] equal to 0 specifies that the current coding unit is split vertically into two transform units.

[0269] According to one embodiment of this application, when a CU is split into a plurality of sub-blocks and the transform is applied at the sub-block granularity, it is necessary to unblock the sub-block TU boundary inside the CU. This embodiment proposes to unblock the sub-block TU boundary inside the CU by an appropriate method.

[0270] In this embodiment, for a boundary with both sides as shown in FIG. 16 (the spatially adjacent plurality of blocks on each side are shown as block P and block Q), the boundary strength is derived as follows. ● When both blocks P and Q are within the same CU and the boundary between block P and block Q is a sub-block TU boundary, as shown in FIG. 14 or FIG. 15, the boundary strength parameter of that boundary is set according to the following conditions. ■ When at least one of the adjacent blocks P and Q has at least one non-zero transform coefficient, the boundary strength parameter of that boundary is set to a non-zero value such as 1, for example. ■ In other cases, when neither block P nor block Q has a non-zero transform coefficient, the boundary strength parameter of this boundary is set to 0. ● In other cases, the boundary strength is derived as in the above example, that is, in the same way as the boundary strength derivation process defined in Section 8.6.2.5 of VVC Draft 3.0 v9. ● The pixel samples included in block Q and block P are filtered by applying a non-blocking filter according to the determined boundary strength.

[0271] In one example, when the sub-block TU boundary is aligned with the N×M grid, as defined in the above embodiment, those sub-block TU boundaries are non-blocked. In one example, N is 8 and M is 8. In other examples, N is 4 and M is 4. In other cases (when those sub-block TU boundaries are not aligned with the N×M grid), those sub-block TU boundaries are not non-blocked.

[0272] In one example, for a CU whose upper left position is not aligned with the 8×8 grid (as shown in FIG. 13), as defined in the above embodiment, the sub-block TU boundary aligned with the 8×8 grid is non-blocked. In other cases (when those sub-block TU boundaries are not aligned with the 8×8 grid), those sub-block TU boundaries are not non-blocked.

[0273] In one example, for a CU whose upper left position is aligned with an 8×8 grid (as shown in FIG. 13), as defined in the above embodiment, the sub-block TU boundaries aligned with the 8×8 grid are de-blocked. In other cases (when those sub-block TU boundaries are not aligned with the 8×8 grid), those sub-block TU boundaries are not de-blocked.

[0274] In one example, as defined in the above embodiment, regardless of the position of the sub-block TU boundary, all of the sub-block TU boundaries inside the CU are de-blocked.

[0275] The present invention provides the following further embodiments.

[0276] A coding method, wherein the coding includes decoding or encoding, and the method includes: dividing a coding unit or a coding block into at least two sub-blocks including a first sub-block and a second sub-block; and when a boundary between the first sub-block and the second sub-block is aligned with a boundary of a sub-block transform unit, setting a value of a boundary strength parameter corresponding to the boundary between the first sub-block and the second sub-block according to one or more transform coefficients of the first sub-block or one or more transform coefficients of the second sub-block, wherein the first sub-block and the second sub-block are transform blocks.

[0277] The coding unit or the coding block may be divided horizontally or vertically.

[0278] When a value of one or more transform coefficients of the first sub-block is not equal to 0, or when a value of one or more transform coefficients of the second sub-block is not equal to 0, the value of the boundary strength parameter may be set to a first value. The first value may not be equal to 0, and in particular, the first value may be 1 or 2.

[0279] When all values of the conversion coefficients of the first sub-block are equal to 0 and all values of the conversion coefficients of the second sub-block are equal to 0, the value of the boundary strength parameter may be set to a second value. The second value may be 0.

[0280] The encoder (20) may include a processing circuit for executing the above method. The decoder (30) may include a processing circuit for executing the above method.

[0281] The computer program may include program code for executing the above method.

[0282] According to one aspect, the decoder may include one or more processors and a non-transitory and computer-readable storage medium coupled to the processors and storing programming for execution by the processors, and when the processors execute, the programming configures the decoder to execute the method described above.

[0283] According to one aspect, the encoder may include one or more processors and a non-transitory and computer-readable storage medium coupled to the processors and storing programming for execution by the processors, and when the processors execute, the programming configures the encoder to execute the method described above.

[0284] The following description is an explanation of a decoding method, a plurality of application examples of a system using them, and an encoding method shown by the plurality of embodiments mentioned above.

[0285] FIG. 21 is a block diagram showing a content supply system 3100 for realizing a content delivery service. This content supply system 3100 includes a capture device 3102 and a terminal device 3106, and optionally includes a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination of these types, etc.

[0286] The capture device 3102 may generate data and encode the data by the encoding method shown by the above-described multiple embodiments. Alternatively, the capture device 3102 may deliver the data to a streaming server (not shown in the figure), and the server encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, a camera, a smartphone or a Pad, a computer or a laptop, a video conferencing system, a PDA, an in-vehicle device, or any combination of these, etc. For example, the capture device 3102 may include the source device 12 described above. When the data includes video, the video encoder 20 included in the capture device 3102 may actually perform video encoding processing. When the data includes audio (i.e., voice), the audio encoder included in the capture device 3102 may actually perform audio encoding processing. In some practical scenarios, the capture device 3102 distributes the encoded video and audio data by multiplexing them together. In other practical scenarios, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106 individually.

[0287] In the content supply system 3100, the terminal device 3106 receives and plays the encoded data. The terminal device 3106 may be a smartphone or Pad 3108 capable of decrypting the encoded data mentioned above, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or a device having data reception and decryption capabilities such as any combination thereof. For example, the terminal device 3106 may include the destination device 14 described above. When the encoded data includes video, the video decoder 30 included in the terminal device is prioritized to perform video decoding. When the encoded data includes audio, the audio decoder included in the terminal device is prioritized to perform audio decoding processing.

[0288] For example, in the case of a terminal device having its own display such as a smartphone or Pad 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device may supply the decrypted data to its own display. For example, in the case of a terminal device not equipped with a display such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, it contacts an external display 3126 to receive and display the decrypted data in the terminal device.

[0289] When each device in this system performs encoding or decoding, the video encoding device or video decoding device shown in the above-mentioned multiple embodiments may be used.

[0290] FIG. 22 is a diagram showing the configuration of one example of the terminal device 3106. After the terminal device 3106 receives a stream from the capture device 3102, the protocol processing unit 3202 analyzes the transmission protocol of the stream. The protocol includes, but is not limited to, Real-Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real-Time Transport Protocol (RTP), Real-Time Messaging Protocol (RTMP), or any combination of these types.

[0291] After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 may separate the multiplexed data into encoded audio data and encoded video data. As described above, in some cases among a plurality of actual scenarios, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. In this situation, the encoded data is transmitted to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.

[0292] Inverse multiplexing generates a video elementary stream (ES), an audio ES, and optionally subtitles. As described in the above-mentioned multiple embodiments, a video decoder 3206 including a video decoder 30 decodes the video ES by a decoding method to generate a video frame as shown in the above-mentioned multiple embodiments, and supplies this data to a synchronization unit 3212. An audio decoder 3208 decodes the audio ES to generate an audio frame and supplies this data to the synchronization unit 3212. Alternatively, the video frame may be stored in a buffer (not shown in FIG. 22 before being supplied to the synchronization unit 3212. Similarly, the audio frame may be stored in a buffer (not shown in FIG. 22 before being supplied to the synchronization unit 3212.

[0293] The synchronization unit 3212 synchronizes the video frame and the audio frame and supplies video / audio to a video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of video information and audio information. The information may be coded according to syntax using a time stamp related to the presentation of the coded audio data and visual data and a time stamp related to the delivery of the data stream itself.

[0294] When subtitles are included in the stream, a subtitle decoder 3210 decodes the subtitles, synchronizes the video frame, the audio frame, and the decoded subtitles, and supplies video / audio / subtitles to a video / audio / subtitle display 3216.

[0295] The present invention is not limited to the above-mentioned system. For example, any one of the video encoding devices or video decoding devices in the above-mentioned multiple embodiments may be incorporated into other systems such as a vehicle system.

[0296] The present invention has been described in connection with various embodiments herein. However, other variations to the disclosed embodiments may be understood and achieved by those skilled in the art when implementing the invention recited in the claims by considering the drawings, the disclosure, and the appended claims. In those claims, the term "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may be capable of realizing the functions of several items recited in those claims. The mere fact that several means are recited in several different dependent claims does not normally indicate that a combination of those several means cannot be used to advantage. A computer program may be stored / distributed in a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, or distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0297] Those skilled in the art will understand that the "(method and apparatus) 'blocks' ('units')" in the various drawings represent or explain the functions of the embodiments of the present invention (not necessarily individual "units" in hardware or software), and thus explain the functions or features of not only the apparatus embodiments but also the method embodiments (unit = step).

[0298] The term "unit" is used only for the purpose of explaining the functions of the embodiments of the encoder / decoder and is not intended to limit the present disclosure.

[0299] In some of the multiple embodiments provided by this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the described device embodiments are merely illustrative. For example, the division of units is only a logical function division, and in actual implementation, other divisions may be possible. For example, multiple units or components may be combined or integrated into other systems, or some features may be ignored or not executed. In addition, by using some interfaces, the shown or described couplings, direct couplings, or communication connections may be implemented. Indirect couplings or communication connections between multiple devices or multiple units may be implemented in electronic form, mechanical form, or other forms.

[0300] The multiple units described as multiple individual parts may or may not be physically separated, and the multiple parts shown as multiple units may or may not be multiple physical units, may be located in one place, or may be distributed among multiple network units. According to actual requirements, some or all of these multiple units may be selected to achieve the objectives of the multiple technical solution methods of these multiple embodiments.

[0301] In addition, the multiple functional units in the multiple embodiments of the present invention may be integrated into one processing unit, or each of these multiple units may physically exist alone, or two or more units may be integrated into one unit.

[0302] Although multiple embodiments of the present invention have been mainly described based on video coding, embodiments of the coding system 10, the encoder 20, and the decoder 30 (and corresponding system 10), and a plurality of other embodiments described herein may also be configured for still image processing or coding, i.e., processing or coding of individual images independent of any preceding or subsequent images as in video coding. It should be noted that generally, when video processing coding is limited to a single image 17, only the inter-frame prediction units 244 (encoder) and 344 (decoder) may not be available. All of the other functions (also referred to as tools or techniques) of the video encoder 20 and the video decoder 30, such as residual calculation 204 / 304, transformation 206, quantization 208, inverse quantization 210 / 310, (inverse) transformation 212 / 312, segmentation 262 / 362, intra-frame prediction 254 / 354, and / or loop filtering 220, 320, and entropy coding 270 and entropy decoding 304, etc., may be equally used for still image processing.

[0303] By hardware, software, firmware, or any combination thereof, for example, multiple embodiments of encoder 20 and decoder 30, and, for example, multiple functions described herein with reference to encoder 20 and decoder 30 may be implemented. When implemented by software, those multiple functions may be stored in a computer-readable medium, or transmitted as one or more instructions or codes via a communication medium, and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium, or the computer-readable medium may include a communication medium, which includes any medium that facilitates the transfer of a computer program from one location to another, for example, in accordance with a communication protocol. In this way, the computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or a carrier wave. The data storage medium may be any available medium, which may be accessed by one or more computers or one or more processors, and those one or more computers or one or more processors search for instructions, codes, and / or data configurations for implementing the multiple techniques described by this disclosure. A computer program product may include a computer-readable medium.

[0304] By way of example, and not limitation, such computer-readable storage media can be RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store the program code required in the form of instructions or data structures and that can also be accessed by a computer. Also, any connection is not strictly speaking called a computer-readable medium. For example, when transmitting instructions from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, wireless, and microwave, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, wireless, and microwave are included in the definition of the medium. On the other hand, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather relate to non-transient and tangible storage media. As used herein, magnetic disks and discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy discs, and Blu-ray discs, where magnetic disks typically read data magnetically, while discs optically read data by laser. The above combinations should also be included within the scope of computer-readable media.

[0305] The commands may be executed by one or more processors such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated circuits or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the above configurations or other structures suitable for implementing the techniques described herein. Additionally, in some of the various aspects, the functions described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated within a combined codec. Also, the technology may be implemented entirely by one or more circuits or logic elements.

[0306] The techniques of the present disclosure may be implemented by a variety of devices or apparatuses, which include wireless handsets, integrated circuits (ICs), or sets of ICs (such as chip sets). In the present disclosure, various components, modules, or units are described to emphasize the functional aspects of the devices configured to execute the disclosed techniques, but in implementation, it is not necessarily required to have multiple different hardware units. Rather, as described above, the various units may be combined within a codec hardware unit, or may be provided by an assembly of interoperable hardware units including one or more of the processors described above together with appropriate software and / or firmware.

Claims

1. A non-blocking method for non-blocking a transform block boundary in a coding block in image coding and / or image decoding, wherein the coding block including the transform block is coded in an inter-frame prediction mode, the transform block includes a first transform block and a second transform block adjacent to the first transform block, the non-blocking method includes: determining a value of a boundary strength (BS) parameter for the boundary between the first transform block and the second transform block to be a first value when the boundary between the first transform block and the second transform block is a sub-block transform (SBT) boundary and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, the first value being 1, and the boundary between the first transform block and the second transform block being non-blocked only when the sub-block transform (SBT) boundary between the first transform block and the second transform block, which are luminance samples, is aligned with a 4×4 sample grid; performing a non-blocking filtering process on the boundary between the first transform block and the second transform block based at least on the first value of the BS parameter. A method.

2. The transform block further includes a third transform block adjacent to the second transform block, the method includes: determining a value of a boundary strength parameter for the boundary between the second transform block and the third transform block to be the first value when the boundary between the second transform block and the third transform block is an SBT boundary and at least one of the second transform block and the third transform block has one or more non-zero transform coefficients, or The step of determining a value of a boundary strength parameter for a boundary between the second conversion block and the third conversion block, where the boundary between the second conversion block and the third conversion block is a sub-block transform (SBT) boundary, and when both the second conversion block and the third conversion block have all-zero conversion coefficients, the second value is 0, The method according to claim 1, further comprising.

3. When the coding block is divided horizontally, the boundary between the first conversion block and the second conversion block is a horizontal sub-block transform (SBT) boundary, or When the coding block is divided vertically, the boundary between the first conversion block and the second conversion block is a vertical sub-block transform (SBT) boundary, the method according to claim 1 or 2.

4. The current coding block is coded using a sub-block transform (SBT) tool, or the boundary is caused by a sub-block transform (SBT) tool, the method according to any one of claims 1 to 3.

5. A de-blocking method for de-blocking a block boundary in a coding block in image encoding and / or image decoding, wherein the coding block including a conversion block is coded in an inter-frame prediction mode, and the conversion block includes a first conversion block and a second conversion block adjacent to the first conversion block, The de-blocking method is In response to a determination that it is necessary to filter a boundary between the first conversion block and the second conversion block, when the boundary between the first conversion block and the second conversion block is a sub-block transform (SBT) boundary and at least one of the first conversion block and the second conversion block has one or more non-zero conversion coefficients, determining a value of a boundary strength parameter for the boundary between the first conversion block and the second conversion block to be a first value, wherein the first value is 1, and the boundary between the first conversion block and the second conversion block, which is a luminance sample, is non-blocked only when aligned with a 4×4 sample grid, step; performing a non-blocking filtering process on the boundary between the first conversion block and the second conversion block based at least on the value of the boundary strength parameter; Method. **Claim 6** In response to a determination that it is necessary to filter a boundary between the first conversion block and the second conversion block, when the boundary between the first conversion block and the second conversion block is a sub-block transform (SBT) boundary and at least one of the first conversion block and the second conversion block has one or more non-zero conversion coefficients, the step of determining a value of a boundary strength parameter for the boundary between the first conversion block and the second conversion block to be a first value is including determining a value of a boundary strength parameter for the boundary between the first conversion block and the second conversion block to be a first value in response to a determination that the boundary between the first conversion block and the second conversion block is aligned with a 4×4 sample grid, when the boundary between the first conversion block and the second conversion block is a sub-block transform (SBT) boundary and at least one of the first conversion block and the second conversion block has one or more non-zero conversion coefficients, where n is an integer, the method according to claim 5.

7. The conversion block further includes a third conversion block adjacent to the second conversion block, The method includes: in response to a determination that a boundary between the second conversion block and the third conversion block is aligned with a 4×4 sample grid, the boundary between the second conversion block and the third conversion block is a sub-block transform (SBT) boundary, and when at least one of the second conversion block and the third conversion block has one or more non-zero conversion coefficients, determining a value of a boundary strength parameter for the boundary between the second conversion block and the third conversion block to be a first value; or in response to a determination that a boundary between the second conversion block and the third conversion block is aligned with a 4×4 sample grid, the boundary between the second conversion block and the third conversion block is a sub-block transform (SBT) boundary, and when both the second conversion block and the third conversion block have all zero conversion coefficients, determining a value of a boundary strength parameter for the boundary between the second conversion block and the third conversion block to be a second value, wherein the second value is 0; The method according to claim 5 or 6, further comprising:

8. when the coding block is divided horizontally, the boundary between the first conversion block and the second conversion block is a horizontal sub-block transform (SBT) boundary, or when the coding block is divided vertically, the boundary between the first conversion block and the second conversion block is a vertical sub-block transform (SBT) boundary. The method according to any one of claims 5 to 7.

9. The current coding block is coded using a sub-block transform (SBT) tool, or the boundary is caused by a sub-block transform (SBT) tool. The method according to any one of claims 5 to 8.

10. A device for use in an image encoder and / or an image decoder to de-block a transform block boundary within a coding block, wherein the coding block including the transform block is coded in an inter-frame prediction mode, the transform block includes a first transform block and a second transform block adjacent to the first transform block, the device includes a de-blocking filter, and the de-blocking filter determines a value of a boundary strength parameter for a boundary between the first transform block and the second transform block to be a first value when the boundary between the first transform block and the second transform block is a sub-block transform (SBT) boundary and at least one of the first transform block and the second transform block has one or more non-zero transform coefficients, the first value is 1, and the boundary between the first transform block and the second transform block is de-blocked only when the sub-block transform (SBT) boundary between the first transform block and the second transform block, which are luminance samples, is aligned with a 4×4 sample grid, and is configured to perform a de-blocking filtering process on the boundary between the first transform block and the second transform block based at least on the first value of the boundary strength parameter. Device.

11. the transform block further includes a third transform block adjacent to the second transform block, and the de-blocking filter further determines a value of a boundary strength parameter for a boundary between the second transform block and the third transform block to be the first value when the boundary between the second transform block and the third transform block is a sub-block transform (SBT) boundary and at least one of the second transform block and the third transform block has one or more non-zero transform coefficients, or The boundary between the second conversion block and the third conversion block is a sub-block transform (SBT) boundary, and when both the second conversion block and the third conversion block have all-zero conversion coefficients, the value of the boundary strength parameter for the boundary between the second conversion block and the third conversion block is determined to be a second value, the second value being 0, the device according to claim 10.

12. When the coding block is divided horizontally, the boundary between the first conversion block and the second conversion block is a horizontal sub-block transform (SBT) boundary, or When the coding block is divided vertically, the boundary between the first conversion block and the second conversion block is a vertical sub-block transform (SBT) boundary, the device according to claim 10 or 11.

13. The current coding block is coded using a sub-block transform (SBT) tool, or the boundary is caused by a sub-block transform (SBT) tool, the device according to any one of claims 10 to 12.

14. A device used in an image encoder and / or an image decoder for deblocking a block boundary in a coding block, the coding block including a conversion block being coded in an inter-frame prediction mode, the conversion block including a first conversion block and a second conversion block adjacent to the first conversion block, The device includes a deblocking filter, and the deblocking filter is In response to a determination that it is necessary to filter a boundary between the first transformation block and the second transformation block, the boundary between the first transformation block and the second transformation block is a sub-block transformation (SBT) boundary, and when at least one of the first transformation block and the second transformation block has one or more non-zero transformation coefficients, a value of a boundary strength parameter for the boundary between the first transformation block and the second transformation block is determined to be a first value, the first value is 1, and the sub-block transformation (SBT) boundary between the first transformation block, which is a luminance sample, and the second transformation block is aligned with a 4×4 sample grid, only then the boundary between the first transformation block and the second transformation block is de-blocked, and configured to perform a de-blocking filtering process on the boundary between the first transformation block and the second transformation block based at least on the value of the boundary strength parameter, a device. **Claim 15** The de-blocking filter is configured to determine a value of a boundary strength parameter for a boundary between the first transformation block and the second transformation block to be a first value in response to a determination that the boundary between the first transformation block and the second transformation block is aligned with a 4×4 sample grid, the boundary between the first transformation block and the second transformation block is a sub-block transformation (SBT) boundary, and when at least one of the first transformation block and the second transformation block has one or more non-zero transformation coefficients, n is an integer, the device according to claim 14. **Claim 16** The transformation block further includes a third transformation block adjacent to the second transformation block, the de-blocking filter further In response to a determination that a boundary between the second transform block and the third transform block is aligned with a 4×4 sample grid, the boundary between the second transform block and the third transform block is a sub-block transform (SBT) boundary, and when at least one of the second transform block and the third transform block has one or more non-zero transform coefficients, determining a value of a boundary strength parameter for the boundary between the second transform block and the third transform block to be a first value, or, configured to determine a value of a boundary strength parameter for the boundary between the second transform block and the third transform block to be a second value in response to a determination that the boundary between the second transform block and the third transform block is aligned with a 4×4 sample grid, the boundary between the second transform block and the third transform block is a sub-block transform (SBT) boundary, and both the second transform block and the third transform block have all zero transform coefficients, the second value being 0, the device according to claim 14 or 15.

17. When the coding block is divided horizontally, the boundary between the first transform block and the second transform block is a horizontal sub-block transform (SBT) boundary, or, When the coding block is divided vertically, the boundary between the first transform block and the second transform block is a vertical sub-block transform (SBT) boundary, the device according to any one of claims 14 to 16.

18. The current coding block is coded using a sub-block transform (SBT) tool, or the boundary is caused by a sub-block transform (SBT) tool, the device according to any one of claims 14 to 17.

19. An encoder (20) comprising a processing circuit for performing the method according to any one of claims 1 to 9.

20. A decoder (30) comprising a processing circuit for performing the method according to any one of claims 1 to 9.

21. A computer program comprising program code for performing the method according to any one of claims 1 to 9.

22. A non-transitory computer-readable storage medium that holds program code, wherein when executed by a computer device, the program code causes the computer device to execute the method according to any one of claims 1 to 9.

23. A decoder, comprising one or more processors and a non-transitory computer-readable storage medium coupled to the processor and storing programming for execution by the processor, wherein the programming configures the decoder to execute the method according to any one of claims 1 to 9 when executed by the processor. Decoder.

24. An encoder, comprising one or more processors and a non-transitory computer-readable storage medium coupled to the processor and storing programming for execution by the processor, wherein the programming configures the encoder to execute the method according to any one of claims 1 to 9 when executed by the processor. Encoder.

Images