Loop filtering method based on adaptive pixel classification criteria

Abstract

Disclosed is a loop filter method based on an adaptive pixel classification criterion. The loop filter method based on an adaptive pixel classification criterion in an image decoding device includes a step of classifying a restored sample according to an absolute classification criterion or a relative classification criterion, a step of obtaining offset information according to a result of classifying the restored sample, a step of adding the offset value to the restored sample with reference to the obtained offset information, and a step of outputting the restored sample to which the offset value is added. Thus, errors of the restored sample can be corrected.

Description

[0001] This application is a divisional application of application number 201880033604.2, filed on March 22, 2018, entitled "Loop Filtering Method Based on Adaptive Pixel Classification Benchmark". Technical Field

[0002] This invention relates to a loop filtering method based on an adaptive pixel classification criterion, and more specifically to a method for classifying restored samples according to an absolute classification criterion or a relative classification criterion, and adding an offset value adaptively set according to the classification result to the restored samples to filter the restored samples. Background Technology

[0003] The various organizations known as ISO / ISE MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group) jointly established the JCV-VC (Joint Collaborative Team on Video Coding) in January 2013, which developed the ISO / IEC MPEG-H HEVC (High Efficiency Video Coding) / ITU-T H.265 video compression standard. Furthermore, to adapt to the increasing popularity of high-quality video with the rapid development of current information and communication technologies, ISO / ISE MPEG and ITU-T VCEG organized the JVET (Joint Video Exploration Team) at the 22nd JCT-VC Geneva meeting. They devoted considerable effort to developing a new generation of video compression technology standards for compressing UHD (Ultra High Definition) video, which offers even greater clarity than HD (High Definition) video.

[0004] In addition, a sample adaptive offset is proposed for the loop filter based on the existing video compression standard technology (HEVC), which adds an offset value for the restored pixel (or sample) to minimize the error between the restored image and the original image.

[0005] The existing sample adaptive offset is divided into edge offset and band offset. The adaptive offset value is determined based on the restored sample. Specifically, the offset value is adaptively used based on the edge formed with the restored pixel as the center and the pixel band to which the restored pixel belongs.

[0006] However, due to the many characteristics of recent images, there are limitations in eliminating errors in the restored images by relying solely on existing edge offsets and band offsets to determine the offset values. Summary of the Invention

[0007] Technical issues

[0008] The purpose of this invention, which aims to solve the problems described above, is to provide a loop filtering method based on an adaptive pixel classification benchmark.

[0009] Technical solution

[0010] According to one aspect of the present invention for achieving the above-mentioned objectives, a loop filtering method based on an adaptive pixel classification benchmark is provided.

[0011] The loop filtering method based on adaptive pixel classification criterion in the image decoding device may include the steps of classifying the restored sample based on an absolute classification criterion or a relative classification criterion, obtaining offset information based on the classification result of the restored sample, adding an offset value to the restored sample with reference to the obtained offset information, and outputting the restored sample with the offset value added.

[0012] Here, the step of classifying the restored sample may include classifying the restored sample according to the band range to which the brightness value of the restored sample belongs when the classification criterion is the absolute classification.

[0013] Here, the step of classifying the restored sample may include, when the classification criterion is the relative classification, classifying the restored sample based on at least one of edge information and slope information derived by comparing the pixel values ​​of the restored sample with the pixel values ​​of the adjacent samples adjacent to the restored sample.

[0014] Technical effect

[0015] When using the sample adaptive offset method based on absolute or relative classification of the present invention as described above, errors in the restored image can be corrected more precisely.

[0016] Furthermore, since it applies to multiple classification criteria, it has the advantage of being able to adapt to offsets that conform to the characteristics of each restored image. Attached Figure Description

[0017] Figure 1 This is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention;

[0018] Figure 2 This is a block diagram of an image encoding apparatus according to an embodiment of the present invention;

[0019] Figure 3This is a structural diagram of an image decoding apparatus according to an embodiment of the present invention;

[0020] Figures 4a to 4c This is an illustrative diagram showing blocks, segments, and sections of a sub-region used by an image encoding / decoding apparatus as an embodiment of the present invention;

[0021] Figure 5a and Figure 5b This is an illustrative diagram illustrating the generation of segments from consecutive blocks in a scanning sequence, according to an embodiment of the present invention.

[0022] Figures 6a to 6d This is an example diagram showing the blocks and basic coding units within an image;

[0023] Figure 7 It is the source code that explicitly processes and sets information when encoding or decoding is performed on a block-by-block basis;

[0024] Figure 8 This is an illustrative diagram of sample pixels based on relative classification applicable sample adaptive offset, used to illustrate an embodiment of the present invention;

[0025] Figure 9 This is an illustrative diagram illustrating an embodiment of the present invention where relative classification is based on edge classification;

[0026] Figure 10 This is an illustrative diagram illustrating a method for assigning offset values ​​to blocks that are subject to adaptive offset of samples based on relative classification, according to an embodiment of the present invention;

[0027] Figure 11 This is an illustrative diagram illustrating a method for performing adaptive sample shifting based on absolute classification, according to an embodiment of the present invention;

[0028] Figure 12 This is an illustrative diagram illustrating various methods for generating domain information according to an embodiment of the present invention;

[0029] Figure 13 This is the source code for the syntax used in the sample adaptive offset based on relative and absolute classification, which is used to illustrate an embodiment of the present invention.

[0030] Figure 14 This is a flowchart of a method for performing adaptive sample offset based on absolute or relative classification, according to an embodiment of the present invention. Detailed Implementation

[0031] This invention can be modified in many ways and has many embodiments. Specific embodiments are shown in the accompanying drawings and described in detail in the specification. However, this invention is not intended to limit it to a specific implementation; therefore, it should be understood to include all modifications, equivalents, and substitutions that fall within the scope of the invention's ideas and techniques. In describing the drawings, similar constituent elements are labeled with similar reference numerals.

[0032] The terms "first," "second," "A," and "B," etc., can be used to describe multiple constituent elements, but the constituent elements shall not be limited by these terms. The purpose of using these terms is to distinguish one constituent element from others. For example, without departing from the scope of the invention, a first constituent element may be named a second constituent element, and similarly, a second constituent element may be named a first constituent element. The term "and / or" includes a combination of multiple related described items or a single item among multiple related described items.

[0033] When a constituent element is mentioned as being "connected to" or "accessed to" other constituent elements, it should be understood that while it may be directly connected to or accessed by other constituent elements, there may be other constituent elements in between. Conversely, when a constituent element is mentioned as being "directly connected to" or "directly accessed to" other constituent elements, it should be understood that there are no other constituent elements in between.

[0034] The terminology used in this application is for illustrative purposes only and is not intended to limit the invention. The singular form also includes the plural unless otherwise specified. Terms such as "comprising" or "having" in this application should be understood as the presence of the features, numbers, steps, actions, constituent elements, parts, or combinations thereof described in the specification, and should not be construed as pre-excluding the presence or additional possibilities of one or more other features or numbers, steps, actions, constituent elements, parts, or combinations thereof.

[0035] Unless otherwise defined, all terms used herein, including technical and scientific terms, shall have the same meaning as commonly understood by one of ordinary skill in the art. Terms defined in commonly used dictionaries shall be interpreted in a meaning consistent with the context of the relevant technical text and shall not be interpreted as strange or overly formal unless explicitly defined in this application.

[0036] Generally, an image can be composed of a series of still images, which can be distinguished by a Group of Pictures (GOP). Each still image can be called a picture or a frame. Higher-level concepts can include GOPs, sequences, etc., and each image can be divided into specified regions such as slices, tiles, and blocks. Furthermore, a GOP can include units such as I-images, P-images, and B-images. An I-image can represent an image that is self-encoded / decoded without using a reference image, while P-images and B-images can represent images that are encoded / decoded using reference images to perform processes such as motion estimation and motion compensation. Generally, in the case of P-images, I-images and P-images can be used as reference images, and in the case of B-images, I-images and P-images can also be used as reference images, but these definitions can be changed through encoding / decoding settings.

[0037] Here, the image referenced during encoding / decoding is called the Reference Picture, and the reference block or pixel is called the Reference Block or Reference Pixel. Furthermore, the reference data can be not only pixel values ​​in the spatial domain, but also coefficient values ​​in the frequency domain, as well as various encoding / decoding information generated and determined during the encoding / decoding process.

[0038] The smallest unit constituting an image can be a pixel, and the number of bits used to represent a pixel is called the bit depth. Generally, the bit depth can be 8 bits, but other bit depths can be supported depending on the encoding settings. Regarding bit depth, at least one bit depth can be supported based on the color space. Furthermore, the image can be composed of at least one color space depending on its color format. The image can be composed of more than one image of a certain size or more than one image of other sizes, depending on the color format. For example, in the case of YCbCr 4:2:0, it can be composed of one grayscale component (Y in this example) and two color difference components (Cb / Cr in this example), where the ratio of the color difference component to the grayscale component can be 1:2 horizontally and vertically. According to another example, in the case of 4:4:4, the horizontal and vertical ratios can be the same. In the case of more than one color space as in the above examples, the image can be segmented into each color space.

[0039] This invention is described based on a partial color space (Y in this example) of a partial color format (YCbCr). The same or similar applications can be made to other color spaces based on the color format (Cb, Cr in this example) (depending on the specific color space settings). However, it is also possible to have partial differences for each color space (depending on the specific color space settings). That is, settings dependent on each color space can represent settings that are proportional to or dependent on the composition ratio of each component (e.g., determined according to 4:2:0, 4:2:2, 4:4:4, etc.), while settings independent of each color space can represent settings that are unrelated to the composition ratio of each component or independently possess only that color space. In this invention, depending on the encoder / decoder, partial compositions can have independent or dependent settings.

[0040] The setup information or syntax elements required for image encoding can be determined at the unit level, such as video, sequence, image, segment, block, etc. These can be recorded in the bitstream and transmitted to the decoder in units such as VPS (Video Parameter Set), SPS (Sequence Parameter Set), PPS (Picture Parameter Set), Slice Header, Tile Header, and Block Header. The decoder can then parse the setup information transmitted from the encoder at the same unit level for use in the image decoding process. Each parameter set has an inherent ID value, and a lower-level parameter set can have the ID value of a higher-level parameter set to reference. For example, a lower-level parameter set can reference information from more than one higher-level parameter set with consistent ID values. In the above examples of various units, when a unit includes more than one other unit, the corresponding unit can be called the higher-level unit, and the included units can be called the lower-level unit.

[0041] In the case of setting information occurring in the aforementioned units, each corresponding unit may include content regarding independent settings or content regarding settings dependent on previous, subsequent, or higher-level units. Here, dependent settings can be understood as flag information indicating the setting information of the corresponding unit, used to comply with settings of previous, subsequent, or higher-level units (e.g., a 1-bit flag indicating compliance and a 0-bit indicating non-compliance). In this invention, setting information is described primarily with examples of independent settings, but examples may also include content added to or replaced with information regarding the dependency relationship of setting information of previous, subsequent, or higher-level units on the current unit.

[0042] The following is a detailed description with reference to preferred embodiments of the present invention.

[0043] Figure 1 This is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

[0044] See Figure 1 The image encoding device 105 and the decoding device 100 can be user terminals such as personal computers (PCs), laptops, PDAs, portable media players (PMPs), handheld game consoles (PSPs), wireless communication terminals, smartphones, TVs, etc., or server terminals such as application servers and service servers. They can include various devices such as communication modems for communicating with various devices or wireless communication networks, various programs for inter-frame or intra-frame prediction for encoding or decoding images, memory 120, 125 for storing data, and processors 110, 115 for running programs for calculation and control. Furthermore, the image encoded into a bitstream by the image encoding device 105 can be transmitted in real-time or non-real-time to the image decoding device 100 via various communication interfaces such as networks, short-range wireless communication networks, wireless local area networks, wireless broadband networks, and mobile communication networks, or via cables and Universal Serial Bus (USB), and then decoded by the image decoding device 100 to restore the image for playback. Additionally, the image encoded into a bitstream by the image encoding device 105 can be transmitted from the image encoding device 105 to the image decoding device 100 via a computer-readable storage medium.

[0045] Figure 2 This is a block diagram of an image encoding apparatus according to an embodiment of the present invention.

[0046] The image encoding device 20 in this embodiment is as follows: Figure 2 As shown, it may include a prediction unit 200, a subtraction unit 205, a transformation unit 210, a quantization unit 215, an antiquantization unit 220, an inverse transformation unit 225, an addition unit 230, a filter unit 235, an encoded image buffer 240, and an entropy encoding unit 245.

[0047] The prediction unit 200 may include an intra-frame prediction unit that performs intra-frame prediction and an inter-frame prediction unit that performs inter-frame prediction. Intra-frame prediction can determine an intra-frame prediction mode by using pixels from adjacent blocks as reference pixels, and generate a prediction block using the intra-frame prediction mode. Inter-frame prediction can determine motion information of the current block using one or more reference images, and perform motion compensation using the motion information to generate a prediction block. It determines whether to perform intra-frame or inter-frame prediction on the current block (encoding unit or prediction unit), and determines specific information related to each prediction method (e.g., intra-frame prediction mode, motion vector, reference image, etc.). Here, the processing unit and prediction method to be performed, as well as the processing unit for specific content, can be determined according to the encoding / decoding settings. For example, the prediction method, prediction mode, etc., are determined by prediction unit (or encoding unit), and the prediction is performed by prediction block unit (or encoding unit, transform unit).

[0048] The subtraction unit 205 subtracts the prediction block from the current block to generate a residual block. That is, the subtraction unit 205 calculates the difference between the pixel value of each pixel in the current block to be encoded and the predicted pixel value of each pixel in the prediction block generated by the prediction unit to generate a residual block as a residual signal in block form.

[0049] The transformation unit 210 transforms the residual block into a frequency band to transform each pixel value of the residual block into frequency coefficients. Here, the transformation unit 210 can use various transformation methods such as Hadamard Transform, Discrete Cosine Transform (DCT-based Transform), Discrete Sine Transform (DST-based Transform), and Cartesian Transform (KLT-based Transform) to transform the residual signal into a frequency band, and the residual signal transformed into a frequency band becomes the frequency coefficients.

[0050] The quantization unit 215 quantizes the residual block having frequency coefficients that have been transformed into a frequency band by the transformation unit 210. Here, the quantization unit 215 can quantize the transformed residual block using methods such as dead zone uniform threshold quantization, quantization weighted matrix, or improved quantization techniques. It can use one or more quantization techniques as a complement, which can be determined through encoding mode, prediction mode information, etc.

[0051] The entropy coding unit 245 scans the generated quantized frequency coefficient sequence using various scanning methods to generate a quantized coefficient sequence. This involves various binarization methods for the encoded information generated during the coding process (fixed-length binarization).<fixedlength binariation> Univariate binarization<unary binarization> Rice binarization<truncated rice> Syntax elements are generated using methods such as k-th order exp-golomb, and encoded and output using various entropy coding techniques, including Context Adaptive Binary Arithmetic Coding (CABAC) and Context Adaptive Variable Length Coding (CAVLC). The scan pattern can be set to one of several patterns, such as zigzag, diagonal, or raster.

[0052] The dequantization unit 220 dequantizes the residual block quantized by the quantization unit 215. That is, the quantization unit 220 dequantizes the quantized frequency coefficient sequence to generate a residual block with frequency coefficients.

[0053] The inverse transform unit 225 performs an inverse transform on the residual block dequantized by the dequantization unit 220. That is, the inverse transform unit 225 performs an inverse transform on the frequency coefficients of the dequantized residual block to generate a residual block with pixel values, i.e., a restored residual block. Here, the inverse transform unit 225 can perform the inverse transform in reverse using the transformation method used by the transform unit 210.

[0054] The addition unit 230 adds the predicted block predicted by the prediction unit 200 to the residual block restored by the inverse transform unit 225 to restore the current block. The restored current block can be stored as a reference image (or reference block) in the decoded image buffer 240 and referenced during the encoding of other blocks and images.

[0055] The filter unit 235 may include one or more post-processing filtering procedures such as a deblocking filter, a SAO (Self-Avoiding Filter), and an Adaptive Loop Filter (ALF). The deblocking filter removes block distortion generated at the boundaries between blocks in the restored image. The ALF filters by comparing the values ​​of the restored image after filtering blocks through the deblocking filter with the values ​​of the original image. The SAO restores the offset difference between the residual block to which the deblocking filter was applied, pixel by pixel, and can be applied in forms such as band offset and edge offset. This post-processing filter can be applied to restored images or blocks.

[0056] The encoded image buffer 240 can store blocks or images restored by the filter unit 235. The restored blocks or images stored in the decoded image buffer 240 can be provided to the prediction unit 200 that performs intra-frame prediction or inter-frame prediction.

[0057] Although not shown in the diagram, it may also include segmentation units, which can be divided into coding units of various sizes by means of segmentation units (more specifically, block segmentation units). Here, the coding unit may consist of multiple coding blocks depending on the color format (e.g., one grayscale coding block, two color difference coding blocks, etc.). For ease of explanation, a color component unit is assumed for illustration. The coding block may have a variable size such as M×M (e.g., M is 4, 8, 16, 32, 64, 128, etc.). Alternatively, depending on the segmentation method (e.g., tree-based segmentation, quadtree segmentation, binary tree segmentation, etc.), the coding block may have a variable size such as M×N (e.g., M and N are 4, 8, 16, 32, 64, 128, etc.). Here, the coding block can be the basic unit for intra-frame prediction, inter-frame prediction, transformation, quantization, entropy coding, etc.

[0058] This invention is described based on the assumption that multiple sub-blocks of the same size and shape are obtained according to the segmentation method. However, it can also be applied to cases with asymmetric sub-blocks (e.g., in the case of a binary tree, dividing 4M×4N into 3M×4N / M×4N or 4M×3N / 4M×N, etc.). Here, asymmetric sub-blocks can be supported by information that further determines whether they are supported based on the segmentation method for obtaining symmetric sub-blocks, according to the encoding / decoding settings.

[0059] The partitioning of a coding block (M×N) can be based on a recursive tree structure. Here, partitioning can be indicated by a partition flag (e.g., a quadtree partition flag, a binary tree partition flag). For example, if the partition flag of a coding block with a partition depth of k is 0, the coding of the coding block is performed within the coding block at partition depth k; if the partition flag of a coding block with a partition depth of k is 1, the coding of the coding block is performed in four sub-coding blocks (quadtree partitioning) or two sub-coding blocks (binary tree partitioning) at partition depth k+1, depending on the partitioning method. Here, the block size is (M>>1)×(N>>1) in the case of four coding blocks, and (M>>1)×N or M×(N>>1) in the case of two coding blocks. The sub-coding block can be reset as coding block (k+1) and partitioned into sub-coding blocks (k+2) through the aforementioned process. Here, in the case of quadtree partitioning, one partitioning flag (e.g., whether to partition) can be supported, and in the case of binary tree partitioning, at least one (or more than two) flags can be supported (e.g., a partitioning direction flag <horizontal or vertical, which can be omitted in some cases based on the preceding superior or previous partitioning results>).

[0060] Block partitioning can be performed from the largest coded block to the smallest coded block. Alternatively, it can be performed from the minimum partition depth (0) to the maximum partition depth. That is, partitioning is performed recursively until the block size reaches the minimum coded block size or the partition depth reaches the maximum partition depth. Here, partitioning can be performed according to the encoding / decoding settings (e.g., image <fragment, block> type). Encoding mode <intra inter>Color difference components <y cb cr>(etc.) Adaptively set the size of the maximum and minimum coding blocks, and the maximum segmentation depth.

[0061] For example, when the maximum coding block size is 128×128, quadtree segmentation can be performed in the range of 8×8 to 128×128, and binary tree segmentation can be performed in the range of 4×4 to 32×32 with a maximum segmentation depth of 3. Alternatively, quadtree segmentation can be performed in the range of 8×8 to 128×128, and binary tree segmentation can be performed in the range of 4×4 to 128×128 with a maximum segmentation depth of 3. The former case can be a setting in I-type image (e.g., segment), and the latter case can be a setting in P-type or B-type image. As illustrated in the examples above, segmentation settings such as the maximum coding block size, minimum coding block size, and maximum segmentation depth can be supported together with or individually with the above encoding / decoding settings, depending on the segmentation method.

[0062] When multiple segmentation methods are supported, segmentation is performed within the block support range of each method. If the block support ranges of different segmentation methods overlap, a priority order may exist between them. For example, quadtree segmentation can precede binary tree segmentation. Furthermore, when multiple segmentation methods are supported, the result of the preceding segmentation can determine whether to execute the subsequent segmentation. For example, if the result of the preceding segmentation indicates that segmentation should be performed, the subsequent segmentation may not be executed; instead, the sub-coded blocks segmented by the preceding segmentation can be re-established as coded blocks and then segmented again.

[0063] Alternatively, if the result of the preceding segmentation indicates that segmentation will not be performed, segmentation can be performed based on the result of the subsequent segmentation. Here, if the result of the subsequent segmentation indicates that segmentation will be performed, the segmented sub-coded blocks can be reset as coded blocks for further segmentation; if the result of the subsequent segmentation indicates that segmentation will not be performed, segmentation will not be performed. Here, if the result of the subsequent segmentation indicates that segmentation will be performed and the segmented sub-coded blocks are reset as coded blocks, and multiple segmentation methods are supported (e.g., where the block support ranges of each segmentation method overlap), the preceding segmentation may be omitted, and only the subsequent segmentation may be supported. That is, if the result of the preceding segmentation indicates that segmentation will not be performed when multiple segmentation methods are supported, it means that the preceding segmentation will not be performed.

[0064] For example, in the case of an M×N coded block that can be partitioned into a quadtree or a binary tree, the quadtree partitioning flag can be confirmed first. If the flag is 1, the block is partitioned into four sub-coded blocks of size (M>>1)×(N>>1). These sub-coded blocks can then be reconfigured as coded blocks for further partitioning (quadtree or binary tree partitioning). If the flag is 0, the binary tree partitioning flag can be confirmed. If the flag is 1, the block is partitioned into two sub-coded blocks of size (M>>1)×N or M×(N>>1). These sub-coded blocks can then be reconfigured as coded blocks for further partitioning (binary tree partitioning). If the flag is 0, the partitioning process ends and encoding begins.

[0065] The example illustrates the scenario of executing multiple segmentation methods, but is not limited to this; various combinations of segmentation methods can be supported. For example, segmentation methods such as quadtree / binary tree / quadtree+binary tree can be used. Here, the basic segmentation method can be set to quadtree, and the additional segmentation method to binary tree. Information regarding the support for additional segmentation methods can be implicitly determined or explicitly included in the unit of sequence, image, fragment, block, etc.

[0066] In the example described, information related to segmentation, such as the size of the coded block, the supported range of the coded block, and the maximum segmentation depth, can be included in units such as sequences, images, segments, and blocks, or implicitly determined. In summary, the range of permissible blocks can be determined by the maximum coded block size, the supported block range, and the maximum segmentation depth.

[0067] The coded blocks obtained through segmentation in the process described above can be set to the maximum size for intra-frame or inter-frame prediction. That is, the coded block at the end of segmentation can be the starting size for the prediction block segmentation, for intra-frame or inter-frame prediction. For example, if the coded block is 2M×2N, the prediction block can have a size equal to or smaller than 2M×2N, or M×N. Alternatively, it can have sizes of 2M×2N, 2M×N, M×2N, or M×N. Or, it can have the same size as the coded block, 2M×2N. Here, having the same size as the prediction block indicates that prediction block segmentation is not performed, but prediction is performed at the size obtained through coded block segmentation. That is, it indicates that no segmentation information for the prediction block is generated. This setting can also be applied to transform blocks, and transforms can be performed in units of segmented coded blocks.

[0068] It can have multiple configurations according to the following encoding / decoding settings. For example, (after determining the coding block), at least one prediction block and at least one transform block can be obtained based on the coding block. Or, a prediction block of the same size as the coding block can be obtained, and at least one transform block can be obtained based on the coding block. Or, a prediction block and a transform block of the same size as the coding block can be obtained. In the case where at least one block is obtained in the above example, the segmentation information of each block can occur (be generated), and in the case where one block is obtained, the segmentation information of each block does not occur.

[0069] The blocks of various sizes in the form of squares or rectangles obtained according to the above results can be blocks for intra-picture prediction and inter-picture prediction, can be blocks for transformation and quantization of residual components, and can be blocks for filtering restored pixels.

[0070] In addition, the unit (e.g., SAO, etc.) of the block to which the filtering of restored pixels is applied can be based on the largest coding block (e.g., MxM), but it can also be based on coding blocks with various block sizes and forms (e.g., having MxN, M / 2xM / 2, NxN / 2, etc.), (such as coding blocks obtained according to block segmentation, or a unit called a filter block can be supported separately), and the filtering is applied in units of these blocks. This means that filtering-related information occurs in units of the above blocks.

[0071] And, in addition to the block segmentation unit that performs block segmentation as described above, the segmentation unit can also include an image segmentation unit. The image segmentation unit can divide an image into at least one processing unit (e.g., color space <YCbCr, RGB, XYZ, etc.>, segment, block, etc.), and the block segmentation unit can divide the largest (or basic) coding unit into at least one processing unit (e.g., coding, prediction, transformation, quantization, entropy, loop filter unit, etc.). Here, a block is a collection of coding blocks, representing a quadrilateral area obtained by dividing an image horizontally and vertically, and a segment represents an area formed by a collection of coding blocks that are consecutive according to the scanning order of coding blocks.

[0072] It can be composed of at least one color space according to the color format of the video. For example, in the case of YCbCr, it can be composed of one luminance component and two chrominance components. And, the horizontal and vertical length ratios of the color components can be determined according to the color format. For example, in the case of YCbCr 4:2:0, the horizontal and vertical lengths of the chrominance components can be 1 / 2 of the horizontal and vertical lengths of the luminance component, and in the case of YCbCr 4:4:4, the horizontal and vertical lengths of the chrominance components can have the same length as the luminance component. When composed of more than one color space as above, the image can be segmented into each color space. And, each color space can be divided into the largest coding block.

[0073] Furthermore, the image can be segmented into at least one block. Specifically, at least one block can be obtained by segmenting the image into horizontal (or vertical) columns, and by segmenting it into vertical (or horizontal) columns. Each block can be further divided into at least one segment. And each segment can be segmented using the largest possible encoding unit.

[0074] Furthermore, the image can be segmented into at least one segment. Each segment can be further divided into at least one sub-segment. And each sub-segment can be segmented using the largest possible encoding unit.

[0075] Some of the units may not be included, some or all may be selectively included depending on the encoding / decoding settings, and additional units may be included.

[0076] The basic coding units obtained through image segmentation can be divided into basic coding blocks according to the color space, and their size and shape can be determined based on image characteristics and resolution. The supported block size or shape can be defined by using horizontal and vertical lengths raised to the power of 2 (2...). n The input image can be represented as an N×N square (2n×2n, 256×256, 128×128, 64×64, 32×32, 16×16, 8×8, 4×4, etc., where N is an integer between 2 and 8) or an M×N rectangle (2m×2n). For example, in the case of high-resolution 8k UHD images, the input image can be divided into 256×256 pixels; in the case of 1080p HD images, the input image can be divided into 128×128 pixels; and in the case of WVGA images, the input image can be divided into 16×16 pixels, etc.

[0077] Information about the size or shape of the block can be recorded in the bitstream transmission in units such as sequences, images, fragments, and blocks, which can be parsed by the decoder to recover the relevant information.

[0078] Figure 3 This is a configuration diagram of an image decoding apparatus according to an embodiment of the present invention.

[0079] See Figure 3 The image decoding device 30 may be configured to include an entropy decoding unit 305, a prediction unit 310, an anti-quantization unit 315, an inverse transformation unit 320, an adder / subtractor 325, a filter 330, and a decoded image buffer 335.

[0080] Furthermore, the prediction unit 310 may be configured to include an in-frame prediction module and an inter-frame prediction module.

[0081] The entropy decoding unit 305 can receive quantization coefficient sequences, transform coefficient sequences, or signal sequences from the bit stream of the image encoding device 20, and decode them using entropy decoding methods (CABAC, CAVLC, etc.). It can then transmit the data obtained from the grammatical elements of the received decoding information to the prediction unit 310.

[0082] The prediction unit 310 can generate prediction blocks based on data from the entropy decoding unit 305. The prediction unit 310 performs the same process as the prediction unit 200 of the image encoding apparatus 20 described above.

[0083] The dequantization unit 315 can dequantize the quantized transformation coefficients provided by the bit stream and decoded by the entropy decoding unit 305.

[0084] The inverse transform unit 320 can apply inverse DCT, inverse integer transform, or similar inverse transform methods to the transform coefficients to generate residual blocks.

[0085] Here, the dequantization unit 315 and the detransformation unit 320 reverse the processes executed by the transformation unit 210 and the quantization unit 215 of the image encoding apparatus 20 described above, and this can be achieved in various ways. For example, the same process and detransformation shared with the transformation unit 210 and the quantization unit 215 can be used, and the transformation and quantization processes can be reversed based on information from the image encoding apparatus 20 regarding the transformation and quantization processes (e.g., transformation size, transformation pattern, quantization type, etc.).

[0086] The residual block, which has undergone dequantization and inverse transformation, can be added to the prediction block derived from the prediction unit 310 to generate a restored image block. This addition operation can be performed by the adder / subtractor 325.

[0087] Filter 330 can apply a deblocking filter to the restored image blocks as needed to remove blocking phenomena. In order to improve video quality, other loop filters can be used before and after the decoding process.

[0088] The restored and filtered image blocks can be stored in the decoded image buffer 335.

[0089] Although not shown in the figure, the image decoding device 30 may also include a segmentation unit, which may be composed of an image segmentation unit and a block segmentation unit. Regarding the segmentation unit, those skilled in the art can easily understand it as... Figure 2 The configuration is the same or corresponding to that of the image encoding device, so a detailed description is omitted.

[0090] During image encoding / decoding, discrepancies may occur between the input and output pixel values. To prevent distortion caused by computational errors, a pixel value adjustment process can be performed. Pixel value adjustment is the process of adjusting pixel values ​​that are outside the acceptable range to within that range; this process can be called clipping.

[0091] [Table 1]

[0092]

[0093] Table 1 shows example code for the Clip_x function, which performs pixel value adjustments. Refer to Table 1 for the input pixel value (pixel_val) and the minimum allowed pixel value range (min). I With the maximum value max I This can be used as an input parameter to the clipping function Clip_x. Here, the minimum value min is explained based on the bit depth (bit_depth). I =0, maximum value max I It can be 2 bit_depth -1. When the clipping function Clip_x is executed, it is compared to the minimum value min. I (Parameter B) Smaller input pixel value pixel_val (Parameter A) Changes to the minimum value min I , compared to the maximum value max I (Parameter C) A large input pixel value can be changed to the maximum value max. I Therefore, the output value can be returned as the output pixel value 'pixel_val' after the pixel value adjustment is completed.

[0094] Here, the range of pixel values ​​depends on the bit depth, and the pixel values ​​constituting an image (e.g., an image, a segment, a block, a chunk, etc.) vary depending on the type and characteristics of the image, and therefore do not necessarily occur within all pixel value ranges. According to one embodiment of the invention, the range of pixel values ​​constituting an actual image can be used in the image encoding / decoding process.

[0095] For example, in the pixel value adjustment methods in Table 1, the minimum value of the clipping function can also be mined. I The minimum value among the pixel values ​​used to constitute the actual image, and the maximum value of the cropping function. I The largest value among the pixel values ​​that make up the actual image can be used.

[0096] In summary, the video encoding / decoding device may include a pixel value adjustment method based on bit depth and / or a pixel value adjustment method based on the range of pixel values constituting the video. The encoder / decoder may support flag information for determining whether to support an adaptive pixel value adjustment method. When the flag information is '1', pixel value adjustment method selection information may occur. When the flag information is '0', a preset pixel value adjustment method (in this example, the method based on bit depth) may be used as the basic pixel value adjustment method. When the pixel value adjustment method selection information indicates a pixel value adjustment method based on the range of pixel values constituting the video, it may include information related to the pixel values of the video. For example, according to the color components, information such as the minimum and maximum values and the following central value of each video may be an example. Information generated related to pixel value adjustment may be collected and transmitted in units such as video, sequence, image, segment, block, and block of the encoder. The decoder may analyze the collected information and restore the related information in the same unit.

[0097] In addition, through the above process, the range of pixel values including the minimum and maximum pixel values may be changed (determined or defined) by pixel value adjustment based on bit depth or pixel value adjustment based on the range of pixel values constituting the video, and additional pixel value range information may also be changed (determined or defined). For example, the maximum and minimum pixel values constituting the actual video may be changed, and the central value of the pixel values constituting the actual video may also be changed.

[0098] That is, in the pixel value adjustment process based on bit depth, minI may represent the minimum pixel value of the video, maxI may represent the maximum pixel value of the video, I may represent the color component, and medianI may represent the central pixel value of the video. minI may be 0, maxI may be (1 << bit_depth) - 1, medianI may be 1 << (bit_depth - 1), and median may be obtained in other forms according to the encoding / decoding settings including the above example. The central value is just a term used for illustration in this invention and may be an information representing the pixel value range information that may be changed (determined or defined) during the encoding / decoding process of the video along with the pixel value adjustment process.

[0099] For example, in the pixel value adjustment process based on the range of pixel values constituting the video, minI may be the minimum pixel value of the video, maxI may be the maximum pixel value of the video, and medianI may be the central pixel value of the video. medianI may be the average of the pixel values in the video, may be the value located in the middle when the pixels of the video are aligned, and may be a value obtained according to the pixel value range information of the video. medianI may be derived from at least one of minI and maxI. That is, medianI may be a pixel value within the range of pixel values of the video.

[0100] Specifically, medianI can be (minI+maxI) / 2 or (minI+maxI)>>1, (minI+maxI+1) / 2, (minI+maxI+1)>>1, etc., values ​​obtained based on the pixel value range information of the image (minI, maxI in this example). median can be obtained in other forms according to the encoding / decoding settings, including the example described above.

[0101] The following is an example of the pixel value adjustment process (in this case, the center value).

[0102] As an example, a pixel value adjustment process is selected based on a basic bit depth of 8 bits (0-255) and a range of pixel values ​​constituting the image {in this example, the minimum value is 10 and the maximum value is 190. The center value is 100 under the average setting derived from the minimum and maximum values}. If the current block is the first block within the image (in this example, the picture), since there are no adjacent blocks to be used for encoding / decoding (in this example, left, bottom left, top left, top, and top right), the reference pixel can be filled with a center value of 100. This reference pixel can then be used to perform an in-frame prediction process according to the prediction mode.

[0103] As an example, if a pixel value adjustment process is selected based on a basic bit depth of 10 bits (0-1023) and a range of pixel values ​​constituting the image (in this case, a center value of 600; relevant syntax elements exist), and the current block is the first block within the image (in this case, a segment or block), since there are no adjacent blocks to be used for encoding / decoding (in this case, left, lower left, upper left, upper, and upper right), the reference pixel can be filled with a center value of 600. The in-frame prediction process can then be performed using this reference pixel according to the prediction mode.

[0104] As an example, a pixel value adjustment process is selected with a basic bit depth of 10 bits and based on the range of pixel values ​​constituting the image (in this case, the center value is 112; relevant syntax elements exist). The setting that determines whether pixels of the current block can be used in the prediction of the current block depends on the encoding mode of adjacent blocks (intra-frame prediction / inter-frame prediction), is activated. In this example, if the encoding mode of the block is intra-frame prediction, it can be used as a reference pixel for the current block; otherwise, it cannot be used. Without activating this setting, pixels can be used as reference pixels for the current block regardless of the encoding mode of that block. The relevant syntax element is `constrained_intra_pred_flag` and can occur in P or B image types. If the current block is located on the left side of the image, there are no adjacent blocks to be used for encoding / decoding (left, lower left, upper left in this example). There are adjacent blocks to be used for encoding / decoding (upper, upper right in this example), but because the encoding mode of this block is inter-frame prediction, it is prohibited by the aforementioned setting. Therefore, if no usable reference pixel exists, the reference pixel can be filled with the center value (112 in this example). That is, since no usable reference pixel exists, it can be filled with the center value of the image pixel value range. The reference pixel can be used to perform an intra-frame prediction process according to the prediction mode.

[0105] The embodiments described illustrate various cases related to the central value in the prediction unit, but these can be configured as other structures included in image encoding / decoding. Furthermore, the embodiments are not limited to these examples and can be modified and expanded into various other cases.

[0106] The pixel value adjustment process in this invention is applicable to the encoding / decoding processes of prediction units, transformation units, quantization units, dequantization units, inverse transformation units, filter units, and memory. For example, the input pixel in the pixel value adjustment method can be a reference sample or a prediction sample in the prediction process, or a reconstructed sample in the transformation, quantization, inverse transformation, or dequantization process. Furthermore, it can be a reconstructed pixel in the loop filtering process or a storage sample in the memory. Here, the reconstructed pixel in the transformation and quantization, and their inverse processes, can represent the reconstructed pixel before the application of the loop filter. The reconstructed pixel in the loop filter can represent the reconstructed pixel after the application of the loop filter. The reconstructed pixel in the deblocking filter process can represent the reconstructed pixel after the application of the deblocking filter. The reconstructed pixel in the SAO process can represent the reconstructed pixel after the application of SAO. The reconstructed pixel in the ALF process can represent the reconstructed pixel after the application of ALF. The above illustrates various scenarios, but is not limited to them; it applies to the input, intermediate, and output steps of all encoding / decoding processes that call the pixel value adjustment process.

[0107] Figures 4a to 4c This is an illustrative diagram showing blocks, segments, and sections of a sub-region used by an image encoding / decoding apparatus as an embodiment of the present invention.

[0108] See Figure 4a This confirms that the image is divided into blocks in both the vertical and horizontal directions at intervals of a certain length (B_W and B_H in this example). Here, a block can be the basic coding unit (or the largest coding unit) obtained by the image segmentation unit, and it can also be a unit applicable to blocks, segments, etc.

[0109] See Figure 4b This allows for the identification of blocks obtained by segmenting the image in at least one of the vertical and horizontal directions. These blocks can be encoded / decoded independently of other regions (other blocks, etc.) or partially encoded / decoded dependently. Blocks can be like... Figure 4b A block is formed by a cluster of spatially adjacent blocks (in this example, the width T_W0 and height T_H0 of the first block, and the width T_W1 and height T_H1 of the second block). A block can be a single image if it is not divided in either the vertical or horizontal direction.

[0110] See Figure 4c This allows us to identify segments of an image obtained by bundling consecutive blocks. These segments can be encoded / decoded independently of or partially dependent on other regions (or other segments). The bundle of consecutive blocks can be determined based on the scan order, generally based on the raster scan order, but it can be defined by the encoder / decoder settings. A segment can be a single image if all blocks present in an image are comprised of a single bundle.

[0111] Figure 5a and Figure 5b This is an example diagram illustrating how a segment is generated from a cluster of consecutive blocks according to a scanning sequence, according to an embodiment of the present invention.

[0112] Encoding / decoding in an image can be based on a raster scan sequence, but can be chosen from at least one scan sequence complement, which can be defined according to the encoding / decoding settings. This scan sequence can be determined by steps including: determining the starting point of the scan; determining a primary scan sequence based on the starting point in either the left-right or up-down direction; and determining a secondary scan sequence based on a direction not determined in the primary scan sequence (either left-right or up-down). The starting point of the scan can be one of the top left, bottom left, top right, or bottom right edges of a reference area such as the image.

[0113] See Figure 5a This example demonstrates setting the top left corner of the image as the scan start point, setting the first scan sequence to move from left to right, and setting the second scan sequence to move from top to bottom (or downwards). Raster scanning can also be... Figure 5a The scanning order, when consecutive blocks are bundled in this order, allows the acquisition of the first segment S0, the second segment S1, the third segment S2, and the fourth segment S3.

[0114] See Figure 5b This example demonstrates setting the top leftmost point of the image as the scan start point, setting the first scan sequence to move from top to bottom, and setting the second scan sequence to move from left to right. Figure 5b When the scan order is bundled with consecutive blocks, a configuration different from the previous one can be obtained. Figure 5a The first segment S0, the second segment S1, the third segment S2, and the fourth segment S3 of the form.

[0115] Figures 6a to 6d This is an example diagram showing the blocks and basic coding units within an image.

[0116] According to one embodiment of the present invention, an image is divided into at least one vertical column and at least one horizontal column to form blocks, and encoding / decoding can be performed on a block-by-block basis. Figure 6a The image can be divided into blocks by vertical boundaries (inner boundaries b1, b2) and horizontal boundaries (inner boundary b5). A block is a region enclosed by one or more of the vertical boundaries (inner boundaries b1, b2) and one or more of the horizontal boundaries (inner boundary b5). In the case of a region located outside the image, vertical boundaries (outer boundaries b0, b3) and horizontal boundaries (outer boundaries b4, b6) can be further considered. Blocks obtained through this process can have a quadrilateral shape, or a square shape depending on the encoder / decoder settings for the image's characteristics, format, etc.

[0117] Segmenting an image into blocks formed by vertical and horizontal boundaries can include multiple blocks. The vertical and horizontal boundaries of the image segmentation follow the boundaries of adjacent blocks, thus not dividing individual blocks. Therefore, each block can include an integer number of blocks. If a block does not consist of an integer number of blocks, it can be expanded so that an image or block serving as a higher-level unit can constitute an integer number of blocks. Therefore, processing can be performed on a block-by-block basis, with each block undergoing encoding / decoding on a block-by-block basis.

[0118] When an image is segmented into at least one block, the segmentation information about the block (e.g., information about the positions of vertical and horizontal boundaries <or the horizontal and vertical lengths of each block caused by them> or information about equal / unequal segmentation, etc.) can be recorded in a bitstream for transmission in units such as sequences or images. During image decoding, the segmentation information about the block can be parsed from the sequence or image units, and the image regions can be restored by decoding the block. Finally, the segmentation information about the block can be used to reconstruct an image from each region.

[0119] In the case of blocks, for the purpose of real-time processing of large amounts of high-resolution imagery, encoding / decoding can be performed by dividing the image into more than one block. In this case, a large image is simply divided into multiple images of varying sizes, and the setup information required for the encoding / decoding process of each block can be allocated from the higher-level unit (e.g., image, PPS). For example, instead of generating and transmitting header information separately for each block unit, encoding / decoding setup information can be referenced from the PPS.

[0120] In addition to the block segmentation information, supplementary information can also be recorded and transmitted in higher-level units such as video, sequences, and images. Here, the supplementary information can be at least one encoding / decoding setting information required during the encoding / decoding process of the block unit.

[0121] Alternatively, information about block segmentation and additional data can be recorded and transmitted in block units. This differs from encoding / decoding settings determined at a higher level, where encoding / decoding can be performed at block units. Specifically, it differs from an encoding / decoding setting determined at a higher level.

[0122] For example, header information can be generated and transmitted separately for each block unit, or more than one encoding / decoding setting information can be referenced in the PPS. Here, the PPS may include more than one supplementary group of encoding / decoding setting information for block units.

[0123] As mentioned above, the configuration information for block unit encoding / decoding may include the block type I / P / B, information about the image list referenced by the block, the block quantization parameter (QP) information, the block unit loop filter control, scan order, whether to encode / decode, and other information required for block encoding / decoding.

[0124] See Figure 6a This confirms an example of applying an independent scan order on a block-by-block basis. Therefore, the initial blocks used to start encoding / decoding may differ depending on the scan order determined on a block-by-block basis. Figure 6a The numbers marked in each block indicate the scanning order of the blocks within that block, i.e., the order in which they are processed for encoding / decoding. Furthermore, Figure 6a This indicates an example where more than one scan order applies to each block. When the scan order within a block is determined according to <scan start point / first scan order / second scan order>, the first block 60 can have a scan order determined by <top left / left->right / top->bottom>, the second block 61 can have a scan order determined by <top left / top->bottom / left->right>, the third block 62 can have a scan order determined by <top right / top->bottom / right->left>, the fourth block 63 can have a scan order determined by <bottom right / bottom->top / right->left>, the fifth block 64 can have a scan order determined by <bottom right / right->left / bottom->top>, and the sixth block 65 can have a scan order (or encoding / decoding order) determined by <top right / right->left / top->bottom>.

[0125] The segmentation of blocks and block segments is performed according to a first scan order (e.g., the scan order of the image, Z-scan order, etc.), and the encoding / decoding within blocks and block segments can be performed according to the first scan order (e.g., the encoding / decoding of blocks, chunks, etc., is performed according to the scan order of the image). Alternatively, the segmentation of blocks and block segments can be performed according to the first scan order, and the encoding / decoding within blocks and block segments can be performed according to a second scan order (e.g., an independent scan order of block units). Here, the second scan order can be different depending on whether the blocks and block segments are the same or different.

[0126] See Figure 6b It can be confirmed that in cases where encoding / decoding is selectively performed on a block-by-block basis, it can be confirmed that an instruction has been assigned to determine whether... Figure 6a Each block is encoded / decoded using 0s or 1s. Blocks not encoded / decoded can be filled with arbitrary pixels or data already obtained from the encoded / decoded region. An arbitrary pixel is a pixel belonging to a range of pixels that can be represented by the bit depth transmitted from the bitstream, and can be a preset pixel (e.g., the Min, Median, or Max range of pixels) determined according to the common settings of the encoder / decoder.

[0127] Furthermore, in the encoding / decoding settings where partial dependencies exist between blocks, data obtained from adjacent blocks after encoding / decoding can be referenced in the encoding / decoding of the corresponding block.

[0128] For example, data of at least one pixel located at the boundary of an adjacent block where encoding / decoding ends can be included in temporary storage for the purpose of reference when encoding / decoding a portion of the current block.

[0129] Alternatively, based on the format and characteristics of the image determined by the higher-level unit (e.g., when transforming a three-dimensional image such as a panoramic image (or 360-degree virtual reality image) into a two-dimensional image for encoding / decoding, it can be divided into multiple units (or surfaces) according to the encoding / decoding settings such as the projection format). <face>In the case of (or segmentation), adjacent regions in three-dimensional space can be considered to have spatial correlation. However, when performing spatial transformation (3D -> 2D), the settings of the 2D space configured on each unit (or surface) cannot be used to assume that adjacent units in 2D space necessarily have spatial correlation. That is, adjacent units in 2D space may or may not have spatial correlation, and non-adjacent units may or may not have spatial correlation. This can be confirmed by checking 360-degree image encoding / decoding settings, etc., to determine if spatial correlation exists. The data at the end of encoding / decoding where spatial correlation exists can be referenced. Here, the referenced data can be obtained using a memory copy method by directly copying a certain area, or by using a method obtained through a series of transformation processes.

[0130] See Figure 6c It can be confirmed that, for example, an independent QP (Quantization Parameter) can be applied to each block, and quantization can be performed using independent quantization parameters such as QP0 to QP5. These quantization parameters can also be represented by information such as the difference between the QP set at the higher-level unit (image, etc.) of the block.

[0131] See Figure 6d This can be used to identify an example consisting of two blocks within an image (forming a first block comprising T0, T1, and T2, and a second block comprising T3, T4, and T5) and six block segments (T0 to T5). Blocks can be distinguished by boundary line b7, and block segments can be distinguished by boundary lines b1 to b4 and b7. As mentioned above, blocks and block segments can be obtained using boundary lines b0 to b8. In the case of block segments, segmentation is performed within the block, and its segmentation information is generated. For example, b1 and b2, which serve as vertical column boundaries, can be boundaries that are continuous with b3 and b4, or they can be boundaries that are discontinuous.

[0132] Blocks T0 through T5 can be categorized into dependent blocks T1, T2, T4, and T5, and independent blocks T0 and T3. Regarding dependent blocks, information used or generated during texture and entropy encoding of a specified block can be referenced for the texture and entropy encoding of other blocks. The same applies during decoding; information parsed during entropy decoding of a specified block within a dependent block, and information used or recovered during texture decoding, can be referenced for entropy decoding and source decoding of other blocks. The block description assumes that encoding / decoding is performed on a block-by-block basis.

[0133] In independent blocks, the information used or generated in texture and entropy encoding is completely independent of each other and encoded independently. The same applies to decoding; for entropy and texture decoding of independent blocks, the parsing and reconstruction information of other blocks are not used at all.

[0134] Information about whether a block segment is a subordinate or independent block segment can be included in the block segment header. When decoding an image, the information about the block segment type can be parsed from the block segment header, and the current block segment can be restored by referring to other block segments based on the block segment type, or it can be determined whether to decode it independently from other block segments.

[0135] In particular, the values ​​of the syntax elements in the header of an independent block segment, i.e., the header information, can be determined by referring to the header information of the preceding block segment. Conversely, the header information in the header of a dependent block segment can be determined by referring to the header information of the preceding block segment.

[0136] A block may include at least one block segment. If a block includes only one block segment, it may include a separate block segment. Furthermore, in addition to a separate block segment and a separate segment, a block may also include at least one subordinate block segment. At least one block segment contained in a block can be sent / received through the same Access Unit.

[0137] Figure 7 It is the source code that explicitly determines the information set when encoding or decoding is performed in blocks.

[0138] See Figure 7 `tile_enabled_flag` represents a syntax element regarding whether or not the image is segmented. When `tile_enabled_flag` is active (assumed to be 1), it indicates that the image is segmented into two or more blocks for encoding / decoding, and additional block-related information can be confirmed. When inactive (assumed to be 0), the image can be treated as a single block and encoded / decoded at the image unit (or fragment unit). Incrementing `num_tile_rows_minus1` and `num_tile_columns_minus1` by 1 indicates the number of blocks segmented according to the horizontal and vertical reference axes of the image. Although not shown in this example, information on the horizontal and vertical lengths of each block can be generated based on whether the blocks are equally or unequally segmented (indicating signals). The horizontal and vertical lengths of each block can be generated using the number of basic encoding units.

[0139] `loop_filter_across_tile_enabled_flag` is a syntax element that determines whether a loop filter is applied to a block boundary. When `loop_filter_across_tile_enabled_flag` is active (assuming it's 1), deblocking filters and loop filters supported by the codecs such as SAO and ALF are applied to the block boundary. When inactive (assuming it's 0), deblocking filters, SAO, ALF, and other loop filters are not applied to the block boundary. In this example, `loop_filter_across_tile_enabled_flag` being active means that all loop filters, including deblocking filters, SAO, and ALF, are active. However, this is not the only possibility; the application of a loop filter can be set individually for each loop filter, or additional information (signals indicating whether each loop filter is applied) can be generated.

[0140] `independent_tile_coding_setting_enabled_flag` is a syntax element concerning whether tile-level encoding / decoding settings are supported. When `independent_tile_coding_setting_enabled_flag` is active (assuming it's 1), encoding / decoding can be performed with independent encoding / decoding settings at the tile level. For example, encoding / decoding can be performed by generating tile segmentation information or necessary settings for the encoding / decoding process at the tile level. Alternatively, at least one encoding / decoding setting determined at the higher-level unit can be referenced. When inactive (assuming it's 0), the necessary settings for the tile encoding / decoding process can be allocated from the higher-level unit. Specifically, an encoding / decoding setting determined at the higher-level unit (e.g., an image) can be referenced.

[0141] In addition to allowing encoding / decoding settings for block units, it also supports `tile_qp_offset_enabled_flag`, `tile_coding_skip_enabled_flag`, and `tile_adaptive_scan_enabled_flag` to determine the encoding / decoding settings for block units. These syntax elements respectively represent the QP settings for block units, the applicability of encoding / decoding for block units, and the applicability of the scan order for block units. Additional information related to each block unit can be generated based on the activation or deactivation of each syntax element (assuming it's 1).

[0142] `tile_coding_skip_flag` is a syntax element indicating whether a block is encoded or decoded. When activated, encoding / decoding is not performed; when deactivated, encoding / decoding can be performed. Whether or not additional confirmation of block unit encoding / decoding settings is required depends on whether block unit encoding / decoding is performed. When `tile_coding_skip_flag` is activated (assuming it's 1), block unit encoding / decoding settings are not confirmed; when deactivated (assuming it's 0), block unit encoding / decoding settings are confirmed. `tile_type` indicates the block type, which can be determined by one of I / P / B. `tile_scan_idx` indicates the block scan order, which can be determined by one of the complement groups regarding more than one scan order. `tile_qp_offset` indicates QP-related information determined by the block unit, which can be composed of the difference value between the QP and the higher-level unit. Furthermore, syntax elements such as `end_of_tile_flag` and `end_of_tile_segment_flag` can also be constructed.

[0143] The examples described are partial examples of block-level encoding / decoding settings, which can determine whether the encoding / decoding settings determined by the higher-level unit are applied directly to the block or partially supported independently of the block. The above examples illustrate how to determine whether the higher-level unit supports block-level encoding / decoding settings, but it is also possible to generate header information at the block level to include relevant information for transmission. Furthermore, the examples described are partial examples of block-level encoding / decoding settings, and may include necessary setting information in other encoding / decoding processes. The syntactic elements described in the examples can be encoded and included in the bitstream transmission using various binarization methods (fixed-length binarization, single-item binarization, Rice binarization, Exp-Golomb binarization, etc.), which can be recovered by the decoder parsing the relevant information.

[0144] Information related to the encoding / decoding settings of the block unit, including the information mentioned above, can be explicitly generated or implicitly determined based on the format, characteristics, etc. of the image determined at the higher-level unit.

[0145] Figure 8 This is an illustrative diagram illustrating an embodiment of the present invention, showing the adaptive offset of sample pixels based on relative classification. Figure 9 This is an example diagram illustrating a classification method based on edge classification in a relative classification according to an embodiment of the present invention.

[0146] SAO (Simulation Offset Optimization) is a technique designed to reduce image quality degradation caused by encoding / decoding settings such as QP (Quadrature Processing) during encoding / decoding. SAO can perform an additive offset value process (or an offset compensation process) on a sample (e.g., pixel) basis. Here, the offset value can be determined based on the sample's color composition, classification criteria (explained later), specific classification criteria (explained later), the sample's (x, y) coordinates, etc. SAO can be referred to as a post-processing filter or a loop filter, or a filtering method included in such a filter. The sample pixel to which SAO is applied can represent the restored pixel generated by adding prediction information to the residual signal. Specifically, it refers to applying a loop filter (e.g., SAO, ALF, etc.) to the restored image obtained by adding the residual signal of the current image after inverse quantization and inverse transformation to the prediction signal of the current image after intra-frame prediction or inter-frame prediction processes. Here, the offset value can be obtained based on the distortion between the input image and the restored image.

[0147] Generating offset information at the sample pixel level requires a large amount of data; therefore, it is possible to generate offset information at the set level of samples. Thus, a classification criterion needs to be established to constitute the set of samples to which a single offset information is applied. This classification criterion can be further subdivided into at least one specific classification criterion.

[0148] For example, classification criteria may include edge offset (EO) as a classification of whether a sample pixel has an edge, and band offset (BO) as a classification of the band to which the sample pixel belongs, and may include additional offsets. Here, the edge offset can be additionally set according to the edge direction and edge classification of the specific classification criteria. Furthermore, the band offset can be additionally set according to the band position of the specific classification criteria. The setting may include determining the number and type of specific classification criteria, additional lower-level specific classifications, etc., in the encoder / decoder using the information mentioned above. The offset setting can be determined based on factors such as segment / block type (I / P / B), encoding mode, color components / space, block size and shape, etc.

[0149] One embodiment of the sample classification method of the present invention can be divided into relative classification and absolute classification. Relative classification can be a method of classifying samples based on the relative (or correlation) between the sample to which the offset is to be applied and at least one neighboring sample, while absolute classification can be a method of classifying samples based on the characteristics of the sample to which the offset is to be applied itself.

[0150] Relative classification can be performed based on the judgment results obtained from two or more samples, taking into account characteristics such as slope information and edge information between samples. Specifically, relative classification can be based on the current sample after encoding / decoding has ended and at least one neighboring sample after encoding / decoding has ended.

[0151] See Figure 8 This allows you to confirm the method for setting the current sample (C_sample) and its neighboring samples (N_sample1, N_sample2, ..., N_sample8).

[0152] Specifically, Figure 8 The system can classify the current sample (C_sample) based on the relationships between three samples centered on it, defined by horizontal axes 86, vertical axes 82, and diagonal axes 80 and 84. This also allows for... Figure 8 The direction shown in the diagram is called the edge direction.

[0153] Here, it is shown that relative classification is applied based on the relationship between three consecutive samples, with the current sample as the center. It can also be done based on... Figure 8 The diagram illustrates how the current sample is centered on a set direction of 80, 82, 84, 86, which selects three or more consecutive odd-numbered samples (5, 7, 9, etc.).

[0154] according to Figure 8 The relationships between the three selected samples in multiple directions (80, 82, 84, 86) are classified as follows: Figure 9 As shown.

[0155] See Figure 9 This allows you to confirm the category determined by comparing the pixel values ​​of three samples. The height of this graph represents the pixel value of the sample; the sample in the middle of the graph is the current sample (C_sample), and the samples to its left and right can be neighboring samples (N_sample). Figure 9 The method of classifying categories by comparing the pixel values ​​of three samples can be called category classification based on the edges formed by the three samples. Therefore, the classification can be based on... Figure 9 The method of setting each category's offset information (or offset value) is called edge offset.

[0156] [Mathematical Expression 1]

[0157] (C_sample<N_sampleA)and(C_sample<N_sampleB)

[0158] Specifically, mathematical formula 1 indicates that the current sample (C_sample) has a smaller pixel value than its neighboring samples (N_sampleA, N_sampleB). When mathematical formula 1 is satisfied, the current sample can be classified into the first category (Category1).

[0159] [Mathematical Expression 2]

[0160] (C_sample<N_sampleA)and(C_sample=N_sampleB)

[0161] [Mathematical Expression 3]

[0162] (C_sample=N_sampleA)and(C_sample<N_sampleB)

[0163] Mathematical expressions 2 and 3 represent cases where the current sample (C_sample) has pixel values ​​less than or equal to those of its neighboring samples (N_sampleA, N_sampleB). If either mathematical expression 2 or 3 is satisfied, the current sample can be classified as Category 2.

[0164] [Mathematical Expression 4]

[0165] (C_sample>N_sampleA)and(C_sample=N_sampleB)

[0166] [Mathematical Expression 5]

[0167] (C_Sample=N_SampleA)and(C_sample>N_sampleB)

[0168] Mathematical expressions 4 and 5 represent cases where the current sample (C_sample) has pixel values ​​less than or equal to those of its neighboring samples (N_sampleA, N_sampleB). If either mathematical expression 4 or 5 is satisfied, the current sample can be classified as Category 3.

[0169] [Mathematical Expression 6]

[0170] (C_sample>N_sampleA)and(C_sample>N_sampleB)

[0171] Mathematical formula 6 indicates that the current sample (C_sample) has a pixel value greater than that of its neighboring samples (N_sampleA, N_sampleB). When mathematical formula 6 is satisfied, the current sample can be classified into the fourth category (Category4).

[0172] Adjacent samples in mathematical formulas 1 to 6 may include Figure 8 The direction selects two adjacent samples (referred to as N_sampleA and N_sampleB respectively) as the object, or more than two adjacent samples can be selected.

[0173] See Figure 9 It can be confirmed that the relative size relationship of pixel values ​​is represented by the first category (Category 1), the second category (Category 2), the third category (Category 3), and the fourth category (Category 4) as described above.

[0174] Here, offset information can be obtained and generated at the classification unit (e.g., all or part of the classification can be considered as objects, or additional classifications not shown can be considered as objects). Samples not classified into categories one through four can be classified as samples for which no offset is applied. The criteria for classifying categories one through four can be pre-set in the encoding and decoding devices, such as... Figure 8 The orientation (also known as edge orientation) of three sample pixels is determined by the encoding device, which generates information indicating this orientation and transmits it to the decoding device, which receives the orientation information. Figure 9 Taking the category as an example, instructions can be generated. Figure 8 The direction information (or edge direction information) of any one of the first direction 80, the third direction 82, the fifth direction 84, and the seventh direction 86.

[0175] Here, the offset information assumes that the corrected current sample is close to the average of the neighboring samples. In categories 1 and 2, the sign of the offset should be positive (+), and in categories 3 and 4, the sign of the offset should be negative (-). Therefore, the offset information can omit the sign of the offset value and consist only of the absolute value of the offset value.

[0176] According to Figure 8 and Figure 9 The description (defined as relative classification based on edge classification) illustrates a relative classification of an embodiment of the present invention.

[0177] First, the first embodiment of the relative classification according to one embodiment of the present invention is not based on... Figure 8 Instead of selecting three samples in any given direction, multiple selected adjacent samples can be used regardless of the direction.

[0178] See you again Figure 8 In the case of the description, the category can be classified by comparing the pixel values ​​of the current sample (C_sample) with those of its neighboring samples (N_sample1, N_sample2, ..., N_sample8).

[0179] For example, by comparing the pixel value of the current sample with the eight neighboring samples (N_sample1, N_sample2, ..., N_sample8), if the current sample is larger than the pre-set threshold number (or the current sample is larger in all cases), the current sample can be classified as the first category.

[0180] Furthermore, the pixel value of the current sample can be compared with the eight neighboring samples (N_sample1, N_sample2, ..., N_sample8) around the current sample. If the current sample is less than or equal to a certain value for more than a certain number of pre-set thresholds (or the current sample is less than or equal to a certain value for all cases), the current sample can be classified into the second category.

[0181] Furthermore, the pixel value of the current sample can be compared with the eight neighboring samples (N_sample1, N_sample2, ..., N_sample8) around the current sample. If the current sample is greater than or equal to a pre-set threshold number (or the current sample is greater than or equal to a certain value in all cases), the current sample can be classified into the third category.

[0182] Furthermore, the pixel value of the current sample can be compared with the eight neighboring samples (N_sample1, N_sample2, ..., N_sample8) around the current sample. If the current sample is smaller than the preset threshold number, the current sample can be classified into the fourth category.

[0183] Here, a comparison is given between the current sample and its eight neighboring samples, but this is not the only example. For instance, the current sample can be compared with its four adjacent samples in the horizontal and vertical directions (based on...). Figure 8 The comparison between N_sample2, N_sample4, N_sample5, and N_sample7 can also be performed on the current sample and its four diagonally adjacent samples (based on...). Figure 8 The comparison between N_sample1, N_sample3, N_sample6, and N_sample8.

[0184] Therefore, in the first embodiment of relative classification according to an embodiment of the present invention, the edge information of the current sample can be determined by comparing the neighboring samples around the current sample (e.g., the 4 or 8 mentioned above).

[0185] Furthermore, in one embodiment of the present invention, the classification information regarding the first embodiment of relative classification can be implicitly configured for processing. Offset information can be generated as signed or unsigned offset information, which can be determined according to the encoder / decoder settings. For example, unsigned offset information can be generated if the current sample being corrected is close to its neighboring sample; otherwise, signed offset information is generated. When generating signed offset information, the probability of partial sign occurrences (where the current sample is close to one of the + or - signs of a neighboring sample) during entropy encoding / decoding can be set high, while the probability of opposite sign occurrences can be set low.

[0186] Furthermore, when the difference between adjacent samples is greater than or equal to a preset value th_val based on the current sample, it can be determined that the pulse is classified as either the stated category or an additional category.

[0187] A second embodiment of the relative classification according to one embodiment of the present invention is not based on... Figure 8 Instead of classifying based on any one of the shown directions, further consideration is given to... Figure 8 The categories are classified in multiple parallel directions.

[0188] For example, assuming according to Figure 8 In the case of classifying the current sample by comparing its pixel values ​​with those of its neighboring samples in the horizontal direction 86, further comparisons of pixel values ​​between samples can be performed based on multiple directions parallel to the horizontal direction 86. Specifically, directions parallel to the horizontal direction 86 can be considered, passing through the first sample (N_sample1), the second sample (N_sample2), the third sample (N_sample3), and the sixth sample (N_sample6), the seventh sample (N_sample7), and the eighth sample (N_sample8). Therefore, comparisons can be performed on samples including... Figure 8 If a value in one of the three horizontal directions (e.g., two directions) is above a preset value in the horizontal direction 86 and two directions parallel to the horizontal direction 86, and satisfies the comparison conditions of mathematical formulas 1 to 6 and the pixel values ​​described herein, it can be classified into one of the first to fourth categories. Here, if a direction does not satisfy the comparison conditions of mathematical formulas 1 to 6 and the pixel values ​​described herein, the current sample may not be classified into the aforementioned category. According to the second embodiment of relative classification, since it is possible to determine whether there is an edge in multiple directions, it can be called classification based on two-dimensional edge information.

[0189] A second embodiment of relative classification according to one embodiment of the present invention can be used to classify samples, and offset information can be obtained and generated in units of the classification categories. The classification information can be implicitly or explicitly set; in the case of an explicit setting, relevant syntax elements (binarization correlation) can be generated based on the number of complement groups for all categories. This example illustrates cases where the number of all pixels used in the judgment process for relative classification is 5 or 9, but this can be determined according to the encoder / decoder settings.

[0190] A third embodiment of the relative classification according to an embodiment of the present invention may be performed considering the pixel value slope between the current sample and the neighboring samples, as shown by the pixel value slope between the current sample and the neighboring samples.

[0191] according to Figure 8 If two adjacent samples (N_sampleA, N_sampleB) are selected with the current pixel (C_sample) as the center, the categories can be classified according to the slope of the pixel value as follows.

[0192] [Mathematical Expression 7]

[0193] (N_sampleB<C_sample<N_sampleA)

[0194] [Mathematical Expression 8]

[0195] (N_sampleB=C_sample<N_sampleA)

[0196] [Mathematical Expression 9]

[0197] (N_sampleB<C_sample=N_sampleA)

[0198] Referring to the relationships in mathematical formulas 7 to 9, it can be confirmed that they are based on... Figure 8 The relationship between the slope of the pixel values ​​of the three samples in the direction of increasing morphology is determined by the formula. Therefore, if the relationship between the current sample and its neighboring samples satisfies mathematical formulas 7, 8, or 9, it can be classified as the first category.

[0199] [Mathematical Expression 10]

[0200] (N_sampleB>C_sample>N_sampleA)

[0201] [Mathematical Expression 11]

[0202] (N_sampleB=C_sample>N_sampleA)

[0203] [Mathematical Expression 12]

[0204] (N_sampleB>C_sample=N_sampleA)

[0205] Referring to the mathematical expressions 10 to 12, it can be confirmed that it is based on... Figure 8 The relationship between the slope of the pixel values ​​of the three samples in the direction of decreasing shape is determined by the formula. Therefore, if the relationship between the current sample and its neighboring samples satisfies mathematical formula 10, 11, or 12, it can be classified as the second category.

[0206] Furthermore, in a third embodiment of the relative classification of the present invention, not only the shape of the pixel value slope is considered, but also the magnitude of the slope is further considered.

[0207] For example, if the slope of the pixel values ​​between the current sample and its neighboring samples increases, and the magnitude of this slope (e.g., defined as the difference between the pixel values ​​of the current sample and those of its neighboring samples) exceeds a predetermined threshold, it can be classified into the first category. Similarly, if the slope of the pixel values ​​between the current sample and its neighboring samples decreases, and the magnitude of this slope exceeds a predetermined threshold, it can be classified into the second category. That is, not only the relative size of the current sample and its neighboring samples, but also the difference between the pixel values ​​of the current sample and those of its neighboring samples can serve as a basis for relative classification. This example can be a case of the second and third categories in place of the first embodiment, or an additional consideration.

[0208] Information regarding classification can be processed implicitly, while slope direction information can be processed explicitly. Samples not classified according to the stated conditions can be composed of samples for which no offset was applied. That is, samples classified as edge samples can be classified as samples for which no offset was applied, without being classified according to the stated conditions.

[0209] A fourth embodiment of relative classification according to an embodiment of the present invention, Figure 8 and Figure 9 In edge classification, the magnitude of the pixel value slope can be further considered when setting the category.

[0210] For example, the first category of mathematical formulas 1 to 6 can be set to the case where the current sample (C_sample) has a smaller pixel value than the neighboring samples (N_sampleA, N_sampleB), and the pixel value difference between the current sample and the neighboring samples is above a pre-set threshold.

[0211] Furthermore, the second category of mathematical formulas 1 to 6 can be set to the case where the current sample (C_sample) has a pixel value less than or equal to that of the neighboring samples (N_sampleA, N_sampleB), and the pixel value difference between the current sample and the neighboring samples is above a pre-set threshold.

[0212] Furthermore, the third category of mathematical formulas 1 to 6 can be set to the case where the current sample (C_sample) has a pixel value greater than or equal to that of the neighboring samples (N_sampleA, N_sampleB), and the pixel value difference between the current sample and the neighboring samples is above a pre-set threshold.

[0213] Furthermore, the fourth category in mathematical formulas 1 to 6 can be set such that the current sample (C_sample) has a pixel value greater than that of its neighboring samples (N_sampleA, N_sampleB), and the pixel value difference between the current sample and its neighboring samples is above a pre-set threshold. In this fourth embodiment, samples classified according to edge classification are divided into samples with fewer errors and samples with more errors, the purpose of which is to apply different offset values ​​to the distinguished samples respectively.

[0214] The relative classification of one embodiment of the present invention may be combined with two or more of the first to third embodiments to further include specific classifications of categories. For example, the relative classification of one embodiment of the present invention may be based on... Figure 8 and Figure 9 A primary classification is performed based on the categories defined in the edge classification. A secondary classification is then performed on each category of the primary classification, using the slope shape or magnitude between samples as a benchmark. The fourth embodiment of the relative classification described above can be one such variation.

[0215] Category information can be processed through implicit or explicit settings, while edge direction information can be processed through explicit settings. Offset information can be generated as signed or unsigned offsets depending on the situation, which can be determined by the codec settings. For example, Figure 9 In categories 1 and 4, unsigned offset information can be generated, while in categories 2 and 3, signed offset information can be generated.

[0216] Figure 10 This is an illustrative diagram illustrating a method for assigning offset values ​​to objects based on a relative classification and an adaptive offset of the applicable samples, according to an embodiment of the present invention.

[0217] See Figure 10 This allows for adaptive sample offsetting of all samples located within the diagonally marked block. Here, the current sample and its neighboring samples, considering relative relationships, are shown as shown in Figures A through F. Figure 10 A is like Figure 8 Five samples were set up diagonally for samples 80 and 84. Figure 10 B is like Figure 8 Five samples were arranged vertically in the 82-point configuration. Figure 10 C is a set of 9 samples centered on the current sample. Figure 10 D is like Figure 8 Three samples were arranged horizontally in the 86th sample group. Figure 10 E is like Figure 8 Five samples were set up in the horizontal and vertical directions for samples 82 and 86. Figure 10 F is like Figure 8 Three samples were set up in a diagonal direction for 80.

[0218] Encoding / decoding ends when the location of an adjacent sample is within a block marked with a diagonal line (e.g., inside the image or within the same segment or block as the current block), but sample adaptive offset is performed using samples before applicable filtering. When the location of an adjacent sample is outside the boundary of a block marked with a diagonal line (e.g., outside the image boundary or in a segment or block different from the current block), validity verification of the region to which the adjacent sample belongs can be performed first.

[0219] The relative classification of the adjacent samples can be used based on the validity confirmation result. As mentioned above, if the validity confirmation result indicates that there are unusable samples, samples within the current block can be used to fill the unusable sample positions. For example, samples outside the image can be used by copying or linearly extrapolating boundary samples that belong to the current block and are adjacent to samples outside the image.

[0220] Furthermore, even if adjacent samples are located outside the boundaries of a block marked with a diagonal line, they can still be obtained and used from a portion of the current image. For example, in the case of a 360-degree image, there may be regions within the image that are not adjacent in two-dimensional space but are related. Therefore, data from adjacent samples can be obtained and used from regions that are related to the current pixel (or block) (in this example, assumed to be regions that are not spatially adjacent) through processes such as copying or transformation.

[0221] Here, if there are many samples outside the block and the validity confirmation result is unusable (two in case B), the image with a maximum interval of 2 pixels (E2 region) can be filled. In general (A, C, D, E, F other than B), the image with an interval of 1 pixel can be filled.

[0222] Figure 11 This is an illustrative diagram illustrating a method for performing adaptive sample offset based on absolute classification according to an embodiment of the present invention.

[0223] One embodiment of the present invention provides a method for setting an offset value based on characteristics inherent in the sample itself, such as the brightness value and bandgap information of the applicable sample, according to the absolute classification-based sample adaptive offset. Here, the target sample for which absolute classification is performed for adaptive offset can be the current sample after encoding / decoding.

[0224] See Figure 11 It can be confirmed that the pixel value range determined by the bit depth is divided into pixel value bands with certain intervals. Specifically, with a bit depth of 8 bits (pixel values ​​ranging from 0 to 255), it is equally divided into 2... 5 There are 32 bands. Here, the sample adaptive offset based on absolute classification can set offset values ​​for samples belonging to some bands among the multiple segmented bands. Here, the band to which a sample belongs can be determined based on the brightness value of each sample.

[0225] The start position information of the band with the set sample adaptive offset value is transmitted from the encoding device to the decoding device. The offset information can be applied to a continuous portion of the band including the band to which the start position information belongs (within...). Figure 11 In the example case, four bands (including the starting band K) are generated. Here, the band-related settings (number of bands, spacing, etc.) can be predetermined by the encoding / decoding device, generating start position information for the bands with applicable offset values. Furthermore, this start position information can be padded (fixed length 0.5 bits) from all bands (e.g., 32 bands). The offset information consists of the absolute value of the offset and sign information.

[0226] In one embodiment of the present invention, absolute classification can be performed (or classified) based on the brightness information of the sample. The sample used in the determination of a specific classification can be at least one sample. Furthermore, the specific classification (e.g., the length of the band, the number of segmented bands, etc., as band-related settings) can be based on fixed settings (e.g., fixed band length, fixed number of bands, etc.) or adaptive settings (in this example, settings differentiated by the encoder / decoder). In the case of adaptive settings, relevant information can be implicitly determined or explicitly processed. Furthermore, information about the bands to which the offset applies (e.g., band position information) can be generated based on fixed or adaptive band complement information. Furthermore, the number of complements that can be present can be determined according to the band settings (in this example, the number of bands, etc.). The band settings can be determined based on one or more factors (in this example, quantization parameters, band segmentation parameters, pixel value range information, etc.). Additional settings, including the aforementioned information, can be determined by the encoder / decoder. Information regarding these settings can be recorded in the bitstream transmission in units such as video, sequence, image, segment, block, etc., and the decoder can parse this to recover the relevant information. This is not limited to the example described above and can be varied and expanded into many other cases.

[0227] The length and number of bands for absolute classification in one embodiment of the present invention can be adaptively determined. For example, it can be divided into 2 based on variable k. k Each band has a length that can be 2 in relation to the bit depth. (bit _ depth)-k The length of the bands that can be effectively offset based on the image can exist, and adaptive settings can be supported for this purpose. The variable k can be implicitly determined based on encoding / decoding settings (e.g., block size, shape, image type, etc.) or explicitly include relevant information, and the number of bands, the length of the bands, etc. can be determined by the variable k.

[0228] Figure 12 This is an illustrative diagram illustrating various methods for generating domain information according to an embodiment of the present invention.

[0229] An embodiment of the present invention performs sample adaptive offset based on absolute classification to generate adaptive band information (e.g., band start position information). Information can be generated based on preset conditions (e.g., the condition that offset information occurs in n consecutive bands, i.e., n is preset). Two or more position information can be generated based on the offset bands, and position information can be generated based on the restored samples of the current image.

[0230] Refer to 12a to confirm that more than two offset application location information are generated (in this example, two band information; that is, assuming the bands where the offset occurs may be discontinuous). Specifically, to indicate the application of sample adaptive offset to band m, location information k1 for band m can be generated, and to indicate the application of sample adaptive offset to band m+3, location information k2 for band m+3 can be generated. The location information for these bands (the bands where the offset information occurs) can be generated sequentially based on the first band or in reverse order based on the last band. The location information for the bands where the offset occurs can be generated independently or dependently.

[0231] When generating more than two band field information, the position information of the first band field can be based on the total number of band fields (2 in this example). k The syntax elements are generated (fixed-length binarized .k bits in this example). The position information of the first band can be generated based on the total number of bands, just like the position information of the previously encoded / decoded bands (in the case of independent bands) or based on the position information of the previously encoded / decoded bands (in the case of dependent bands).

[0232] Taking the latter case as an example, such as 12a where the first band position information is k1 and the second band position information is k2, k1 is first encoded / decoded, and k1 is used as the predicted value of the second band position information, i.e., k2. Therefore, the second band position information k2 can generate a difference value with the first band position information (i.e., the predicted value). The decoding device can add k1, which is the predicted value, and the difference value between k2 and k1 to recover k2. Here, the difference between the total number of bands and k1 can be used as the maximum value to generate the syntax elements about it (in this example, a binarization method considering the maximum value is applied). The example described is about the first and second band position information, and it can also be applied in the same or similar way when there are more than two band position information (for example, the case of encoding / decoding the difference value between the band position information to be encoded / decoded and the previously encoded / decoded band position information).

[0233] Furthermore, the area marked with a diagonal line in 12a indicates the existence of a band of pixel values ​​constituting the image. Here, the encoding / decoding device identifies the range of pixel values ​​constituting the image (e.g., image, segment, block, etc.) and can generate or obtain the position information (or start position information of the band) of the band based on the range of pixel values ​​of the image. Moreover, the length of the band can be determined based on the range of pixel values ​​of the image, and offset information can be generated or obtained accordingly. Referring to 12b, it can be confirmed that the range of pixel values ​​constituting the image in 12a is divided into 2... k Each band field. In this case, the length of each band field can be obtained that is 2 times longer than that obtained based on the bit depth. k A narrower band length.

[0234] For example, if the lowest and highest pixel values ​​of the current image are set to the entire pixel value range, the set pixel value range can be divided into 2. k This is because it divides the range of pixel values ​​based on bit depth into 2. k The case of a smaller band is one band length (width), thus enabling refined correction through offset, which can be generated based on the band length (in this example, a binarization method considering the maximum value is used). That is, it can be determined based on k and the band length. For example, the range of pixel values ​​(e.g., the range of pixel values ​​is 118 if the maximum value of a pixel is 128 and the minimum value is 10) can be divided into 2 k In the case of a single band, when k is 5, the maximum length of a band is 4 (8 when k is 5 and the pixel value range is 0 to 255), and the grammatical elements for the offset can be generated based on this. In this example, the band can be based on the offset information obtained by segmenting the image according to the pixel value range and obtaining the band length.

[0235] Furthermore, according to 12c, it can be confirmed that the range of pixel values ​​is determined based on the bit depth, and the range of pixel values ​​determined based on the bit depth is divided into 2 bands. k Since the entire range of pixel values ​​(from 0 to a 255.8-bit reference) is determined based on the bit depth, the length of each band is fixed according to the number of bands, and the offset information can be generated based on this band length. Here, if the lowest pixel value (minc) and the highest pixel value (maxc) of an image exist in bands P1 and Pmax respectively, the bands where offset information can occur exist between bands P1 and Pmax. Therefore, Pmax-P1+1 bands can be considered the number of bands where offset information can occur, and thus the position information of the bands where offset information occurs can be generated with Pmax-P1+1 bands as the maximum value.

[0236] Referring to 12a, when the band is divided according to the entire range of pixel values ​​based on bit depth, the total number of bands is 32 (assuming k is 5). The band where actual pixel values ​​occur is the band between band m and m+4, therefore the total number of bands is 5. With a total of 32 bands, syntax elements regarding band location information can be generated using 5-bit fixed-length binarization (5 bits are needed when only the start position information of the band is sent). However, with 5 bands, syntax elements can be generated using variable-length binarization with a maximum value of 5 (requiring 2 and 3 bits in this example). That is, maintaining the length of the bands while reducing the information generated by band information improves coding performance. In this example, the bands can be divided according to bit depth, and the band location information is generated based on the range of pixel values ​​in the image.

[0237] The above example could be a case where the number of offset information is fixed (assuming four). The number of offset information can be adaptively set according to the encoding / decoding, but it's also possible for the number of offsets to be adaptively determined based on the characteristics of the image. For example, when segmenting bands according to bit depth as in 12a, if the pixel values ​​of the image (assumed to be blocks in this example) occur in three or fewer bands, four offsets are not required, so only three offset information may be generated. This case could be an example where the start position of the band to which the offset is applied is also implicitly obtained. In this example, the number of offsets is adaptively determined based on the range of pixel values ​​in the image, implicitly determining the band position information.

[0238] pass Figure 12 Various embodiments based on the pixel value range of an image have been described, but are not limited to these. Various variations and combinations are possible, and the image can be combined with additional elements not described in the examples. Here, the image can be either a restored image or the original image. In the case of a restored image, the pixel value range information can be implicitly obtained; in the case of an original image, the pixel value range information can be explicitly obtained. The restored image may represent the image before the application of a filter (i.e., the image after encoding / decoding but before the application of a loop filter). The pixel value range information of the original image has been explained above through a pixel value adjustment process based on the pixel value range. Since this information is explicitly included in the bitstream, not only the encoder but also the decoder can confirm the pixel value range information of the original image.

[0239] Figure 13 This is the source code for a syntax element used to illustrate an embodiment of the present invention for adaptive offset of samples based on relative and absolute classification.

[0240] See Figure 13 In SPS, `sps_sample_adaptive_offset_enabled_flag` can be a syntax element regarding whether SAO is supported; in PPS, `pps_sample_adaptive_offset_enabled_flag` can be a syntax element regarding whether SAO is supported in PPS; and `slice_sample_adaptive_offset_enabled_flag` can be a syntax element regarding whether SAO is supported in a slice. Other units (blocks, etc.) can also define syntax elements regarding whether SAO is supported. When the syntax element of the higher-level unit is active, it determines whether to attach and generate syntax elements of the lower-level unit. In this example, when the syntax element described in the slice is active (assumed to be 1 in this example), `slice_sao_luma_flag` and `slice_sao_chroma_flag` can be generated as syntax elements regarding whether SAO is applicable based on the color components. When inactive (assumed to be 0 in this example), SAO may not be applicable in the corresponding image.

[0241] If at least one of the grayscale and chroma components is subject to SAO, a syntax element regarding `offset_type1_enabled_flag` can be defined. This can be a syntax element of a pre-defined type regarding whether SAO is applicable. In this example, it can be an offset application method by absolute classification (or by relative classification), and `offset_type2_enabled_flag` can be an offset application method by relative classification (or by absolute classification). If `offset_type1_enabled_flag` is active, a syntax element regarding `offset_type2_enabled_flag` can be additionally defined. If inactive, `offset_type2_enabled_flag` is not additionally defined, but type2 can be implicitly activated. type1 and type2 can be activated, or only one of type1 and type2 can be activated. It can be a syntax element defined in a fragment, or a syntax element defined in other units (sequence, image, block, etc.).

[0242] Offset-related information can be referenced from at least one adjacent block at the end of encoding / decoding. This referenced block can be the left, top-left, bottom-left, top, or top-right block, etc. The referenced complement group can consist of two or more blocks. The priority order of the referenced blocks (also related to syntax element generation) can be determined according to the encoding / decoding settings. It can use the same or similar order (block position or sub-block position within a block) as when referencing prediction information from adjacent blocks during intra-frame or inter-frame prediction, or it can be a separate order. `sao_merge_flag` can be a syntax element indicating whether to obtain offset-related information from adjacent blocks, and `sao_merge_idx` can be a syntax element representing information about adjacent blocks. All or part of the offset-related information can be confirmed from adjacent blocks. If part of the offset-related information is obtained, additional syntax elements can be defined.

[0243] `sao_type_idx` represents information about the selection of the offset application method; it can be a syntax element generated when more than two offset application methods are supported. `sao_offset_abs` can be the absolute value information about the offset, which can generate the equivalent of `k`. The number of `k` can be determined by the encoding / decoding settings. `sao_offset_sign` can be the sign information about the offset information; it can be a syntax element generated only when the absolute value information about the offset is not zero. The sign information about the offset can be supported according to the offset application method settings. If the offset method is set to support the offset information only with unsigned absolute value information, it is not generated; if it supports the offset information with signed absolute value information, it is generated. `sao_type_info` can be information about the specific classification of the offset method.

[0244] The offset (relative or absolute classification) setting information can be determined based on the encoding / decoding settings (in this example, block size and shape, image type, encoding mode, quantization parameters, etc.), and there can be at least one or more combinations formed by the process.

[0245] Figure 14 This is a flowchart of a method for performing adaptive sample offset based on absolute or relative classification, according to an embodiment of the present invention.

[0246] See Figure 14 The method for performing sample adaptive offset based on absolute or relative classification in an image decoding device may include: step S100 of classifying the restored sample according to an absolute or relative classification criterion, step S110 of obtaining offset information based on the result of classifying the restored sample, step S120 of adding an offset value to the restored sample with reference to the obtained offset information, and step S130 of outputting the restored sample with the offset value added.

[0247] Here, the step S100 of classifying the restored sample may include: when the classification criterion is the absolute classification, classifying the restored sample according to the band to which the brightness value of the restored sample belongs.

[0248] Here, the step S100 of classifying the restored sample may include: when the classification criterion is the relative classification, classifying the restored sample based on at least one of edge information and slope information derived by comparing the pixel values ​​of the restored sample and the pixel values ​​of adjacent samples adjacent to the restored sample.

[0249] The method of this invention can be stored in a computer-readable medium in the form of program commands executable by various computer devices. The computer-readable medium may include, alone or in combination, program commands, data files, data structures, etc. The program commands stored in the storage medium may be specifically designed and constructed for this invention, or may be known to those skilled in the art of computer software.

[0250] Examples of computer-readable media may include read-only memory (ROM), random access memory (RAM), flash memory, and other hardware devices specifically configured for storing and executing program instructions. Examples of program instructions may include not only machine code produced by a compiler, but also high-level language code that can be run by a computer using an interpreter. The aforementioned hardware devices may be configured to function as at least one software module to perform the actions of this invention, and vice versa.

[0251] Furthermore, all or part of the structure or function of the above-mentioned method or apparatus can be implemented in combination or separately.

[0252] The preferred embodiments of the present invention have been described above. Those skilled in the art should understand that various modifications and changes can be made to the present invention without departing from the spirit and scope of the invention as set forth in the claims.< / face> < / y> < / intra>

Claims

1. A loop filtering method based on sample classification benchmarks, wherein, The method is performed in an image decoding device and includes: The segmentation information based on bitstream signaling divides the image including the current block into blocks, one of which is divided into block segments only in one of the vertical or horizontal directions, independent of other blocks, by at least one segmentation boundary line, and each block segment consists of an integer number of consecutive rows of complete coded tree units and has a rectangular shape; The current block is restored by obtaining the current block based on at least one segmented coding block of quadtree segmentation and binary tree segmentation; The restored samples in the current block are classified based on the sample classification criterion, wherein the sample classification criterion is an absolute classification criterion or a relative classification criterion; Offset information is obtained based on the results of the reconstructed samples classified; Based on the obtained offset information, an offset value is added to the restored sample; and The output includes the restored sample with the offset value added. Specifically, when the size of the encoded block is greater than or equal to a predefined threshold, only the quadtree segmentation is performed for segmenting the encoded block. The predefined threshold size is 128x128. Specifically, the binary tree segmentation is performed based on a binary tree segmentation flag indicating whether the binary tree segmentation should be performed and a binary tree segmentation direction flag indicating whether the segmentation is horizontal or vertical. Specifically, the binary tree split is performed only when the quadtree split is no longer required, and Wherein, in response to the sample classification benchmark being the relative classification benchmark, two adjacent samples of the restored sample are used as the restored sample.

2. The method according to claim 1, wherein, The classification restoration samples include: When the sample classification criterion is the absolute classification criterion, the restored sample is classified according to the band to which the brightness value of the restored sample belongs.

3. The method according to claim 1, wherein, The classification restoration samples include: When the sample classification benchmark is the relative classification benchmark, the restored sample is classified based on at least one of the edge information and slope information derived by comparing the pixel values ​​of the restored sample with the pixel values ​​of the adjacent samples adjacent to the restored sample.

4. The method according to claim 1, in, The first dividing boundary line of the first block used for the block and the second dividing boundary line of the second block adjacent to the first block are discontinuous at the boundary between the first block and the second block.

5. A loop filtering method based on sample classification benchmarks, wherein, The method is performed in an image encoding apparatus and includes: The image including the current block is segmented into blocks, one of which is divided into block segments only in one of the directions, either vertical or horizontal, independent of the other blocks, by at least one segmentation boundary line, and each block segment consists of an integer number of consecutive rows of complete coding tree units and has a rectangular shape; The current block is restored by obtaining the current block based on at least one segmented coding block of quadtree segmentation and binary tree segmentation; The restored samples in the current block are classified based on the sample classification criterion, wherein the sample classification criterion is an absolute classification criterion or a relative classification criterion; Offset information is obtained based on the results of the reconstructed samples classified; Based on the obtained offset information, an offset value is added to the restored sample; and The output includes the restored sample with the offset value added. The segmentation information of the image is encoded in a bitstream. Specifically, when the size of the encoded block is greater than or equal to a predefined threshold, only the quadtree segmentation is performed for segmenting the encoded block. The predefined threshold size is 128x128. Specifically, the binary tree split is performed only when the quadtree split is no longer required. Specifically, when performing the binary tree segmentation, a binary tree segmentation flag indicating whether the segmentation is performed and a binary tree segmentation direction flag indicating horizontal or vertical segmentation are encoded in the bitstream. Wherein, in response to the sample classification benchmark being the relative classification benchmark, two adjacent samples of the restored sample are used as the restored sample.

6. A non-transitory computer-readable medium storing instructions thereon, wherein when a processor executes the instructions, the instructions execute a decoding method, the decoding method comprising: The segmentation information based on bitstream signaling divides the image including the current block into blocks, one of which is divided into block segments only in one of the vertical or horizontal directions, independent of other blocks, by at least one segmentation boundary line, and each block segment consists of an integer number of consecutive rows of complete coded tree units and has a rectangular shape; The current block is restored by obtaining the current block based on at least one segmented coding block of quadtree segmentation and binary tree segmentation; The restored samples in the current block are classified based on a sample classification criterion, which can be an absolute classification criterion or a relative classification criterion. Offset information is obtained based on the results of the reconstructed samples classified; An offset value is added to the restored sample based on the obtained offset information; as well as The output includes the restored sample with the offset value added. Specifically, when the size of the encoded block is greater than or equal to a predefined threshold, only the quadtree segmentation is performed for segmenting the encoded block. The predefined threshold size is 128x128. Specifically, the binary tree segmentation is performed based on a binary tree segmentation flag indicating whether the binary tree segmentation should be performed and a binary tree segmentation direction flag indicating whether the segmentation is horizontal or vertical. Specifically, the binary tree split is performed only when the quadtree split is no longer required, and Wherein, in response to the sample classification benchmark being the relative classification benchmark, two adjacent samples of the restored sample are used as the restored sample.

7. A method for transmitting a bit stream, comprising: The bit stream is generated by performing an encoding method; and The bit stream is transmitted, and the bit stream includes segmentation information. The encoding method includes: The image including the current block is segmented into blocks, one of which is divided into block segments only in one of the directions, either vertical or horizontal, independent of the other blocks, by at least one segmentation boundary line, and each block segment consists of an integer number of consecutive rows of complete coding tree units and has a rectangular shape; The current block is restored by obtaining the current block based on at least one segmented coding block of quadtree segmentation and binary tree segmentation; The restored samples in the current block are classified based on a sample classification criterion, which can be an absolute classification criterion or a relative classification criterion. Offset information is obtained based on the results of the reconstructed samples classified; Based on the obtained offset information, an offset value is added to the restored sample; and The output includes the restored sample with the offset value added. The segmentation information of the image is encoded in a bitstream. Specifically, when the size of the encoded block is greater than or equal to a predefined threshold, only the quadtree segmentation is performed for segmenting the encoded block. The predefined threshold size is 128x128. Specifically, the binary tree split is performed only when the quadtree split is no longer required. Specifically, when performing the binary tree segmentation, a binary tree segmentation flag indicating whether the segmentation is performed and a binary tree segmentation direction flag indicating horizontal or vertical segmentation are encoded in the bitstream. Wherein, in response to the sample classification benchmark being the relative classification benchmark, two adjacent samples of the restored sample are used as the restored sample.

Images