Video decoding method, video encoding method, and bitstream transmission method

Abstract

The present invention provides a video coding and decoding method and apparatus based on limited offset compensation and loop filtering. [Solution] The video decoding method receives a first indicator from an encoder indicating whether at least one of the sequences, pictures, frames, slices, coding units (CUs), prediction units (PUs), and transform units (TUs) supports limited offset compensation; receives a second indicator from the encoder indicating whether constrained sample adaptive offset (SAO) compensation or adaptive loop filter (ALF) can be applied; receives SAO parameters from the encoder; and applies SAO compensation or ALF to the pixels of the image restored based on the second indicator and the parameters.

Description

Technical Field

[0001] The present invention relates to digital video, and more particularly, to a video encoding and decoding method based on limited offset compensation and loop filter, and an apparatus therefor.

Background Art

[0002] Recently, as broadcast services having HD (High Definition) resolution have expanded not only in Korea but also worldwide, many users have become accustomed to high-resolution and high-quality video, and as a result, many institutions are accelerating the development of next-generation video devices. In addition, while interest in UHD (Ultra High Definition) having a resolution four times or more that of HDTV has been increasing along with HDTV, compression techniques for higher-resolution and high-quality video are required.

[0003] Video compression techniques include an inter prediction technique for predicting pixel values included in a current picture from pictures before and / or after in time, an intra prediction technique for predicting pixel values included in a current picture using pixel information within the current picture, a weighted prediction technique for preventing image quality degradation due to illumination changes, etc., and an entropy encoding technique for assigning short codes to symbols that appear frequently and long codes to symbols that appear rarely. In particular, when prediction for a current block is performed in skip mode, a prediction block is generated using only the value predicted from a previously encoded area, and separate motion information and residual signals are not transmitted from an encoder to a decoder. Video data can be efficiently compressed by such video compression techniques.

[0004] During video encoding and decoding, offset compensation or loop filtering can be applied to minimize the difference between the original video and the reconstructed video. In the case of offset compensation, the error in pixel values ​​between the original video and the reconstructed video is calculated to determine the offset, and this is applied to the reconstructed video to minimize distortion from the original video. In the case of loop filtering, filter coefficients based on a Wiener filter that minimizes the error between the original video and the reconstructed video are derived, and then applied to the reconstructed video to minimize distortion from the original video.

[0005] On the other hand, compressed video bitstreams can be transmitted over network channels prone to errors. However, conventional offset compensation or loop filters have no way to address errors that occur within the compressed video bitstream, allowing errors to propagate temporally or spatially. Consequently, conventional offset compensation or loop filters can significantly degrade the image quality of the restored video and may even make decoding the compressed video bitstream impossible.

[0006] Therefore, the application of error-resistant offset compensation or loop filtering is required. [Overview of the project] [Problems that the invention aims to solve]

[0007] The technical problem of the present invention is to provide a video coding and decoding method and apparatus based on limited offset compensation and filtering. The present invention provides a method for limiting the application of offset compensation or loop filtering during video coding and decoding by utilizing the coding parameters of at least one block from among the pixel-adaptive offset compensation or loop filtering target block and peripheral blocks. [Means for solving the problem]

[0008] In one embodiment, a video decoding method is provided. The video decoding method includes the steps of: receiving a restricted offset compensation indicator from an encoder indicating whether at least one of a sequence, picture, frame, slice, coding unit (CU), prediction unit (PU), and transform unit (TU) supports restricted offset compensation; receiving an SAO compensation indicator from the encoder indicating whether sample adaptive offset (SAO) compensation can be performed; receiving SAO parameters from the encoder; and performing sample adaptive offset compensation on pixels of the image recovered based on the SAO compensation indicator and the SAO parameters.

[0009] In another embodiment, a video decoding method is provided. The video decoding method includes the steps of: sending a restricted offset compensation indicator to a decoder indicating whether at least one of a sequence, picture, frame, slice, coding unit (CU), prediction unit (PU), and transform unit (TU) supports restricted offset compensation; sending an SAO compensation indicator to the decoder indicating whether sample adaptive offset (SAO) compensation can be performed; sending SAO parameters to the decoder; and performing sample adaptive offset compensation on pixels of the image recovered based on the SAO compensation indicator and the SAO parameters.

[0010] In another embodiment, a video encoding method is provided. The video encoding method includes the steps of: sending a restricted loop filter indicator to a decoder indicating whether at least one of a sequence, picture, frame, slice, coding unit (CU), prediction unit (PU), and transform unit (TU) supports the application of a restricted loop filter; sending an ALF application indicator to the decoder indicating whether an adaptive loop filter (ALF) is applicable; sending ALF parameters to the decoder; and applying the ALF to pixels of the image restored based on the ALF application indicator and the ALF parameters. [Effects of the Invention]

[0011] During video encoding and decoding, error-resistant offset compensation or loop filtering can be applied. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the configuration of one embodiment of a video encoding device. [Figure 2] This is a block diagram showing the configuration of one embodiment of a video decoding device. [Figure 3] An example of the proposed video encoding method is shown. [Figure 4] This indicates the type of edge offset determined by the angle. [Figure 5] This demonstrates how the offset type is determined by the edge offset type using encoding parameters in the proposed video encoding method. [Figure 6] An embodiment of the proposed video decoding method is shown. [Figure 7] Other embodiments of the proposed video encoding method are shown below. [Figure 8]An example of the filter shape determined by an encoder in the proposed video encoding method is shown. [Figure 9] A case of classifying a filter based on the BA method using encoding parameters in the proposed video encoding method is shown. [Figure 10] An example of a case where ALF is applied using encoding parameters in the proposed video encoding method is shown. [Figure 11] An embodiment of the proposed video decoding method is shown. [Figure 12] An example of the filter shape used in the proposed video decoding method is shown.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In describing the embodiments of this specification, when it is determined that a specific description of a related known configuration or function obscures the gist of this specification, the detailed description thereof will be omitted.

[0014] When one component is referred to as "connected" or "coupled" to another component, it should be understood that it may be directly connected or coupled to the other component in question, but there may also be other components in between. Also, in the present invention, the description of including a specific configuration does not exclude configurations other than the corresponding configuration, meaning that additional configurations can be included within the scope of the implementation of the present invention or the technical idea of the present invention.

[0015] Terms such as first, second, etc. can be used to describe various components, but the components should not be limited by the terms. The terms are only used for the purpose of distinguishing one component from another. For example, unless it goes beyond the scope of the rights of the present invention, the first component can be named the second component, and similarly, the second component can also be named the first component.

[0016] In addition, the components disclosed in the embodiments of the present invention are independently illustrated to show different characteristic functions, and it does not mean that each component is constituted by separated hardware or a single software component unit. That is, each component is listed and included as each component for convenience of explanation, and at least two of the components can be integrated to form one component, or one component can be divided into multiple components to perform functions. Such integrated embodiments and separated embodiments of each component are also included in the scope of the rights of the present invention as long as they do not deviate from the essence of the present invention.

[0017] In addition, some components are not essential components for performing the essential functions in the present invention, but are merely optional components for improving performance. The present invention can be implemented by including only the essential components for the essential implementation of the present invention excluding the components used merely for performance improvement, and the structure including only the essential components excluding the optional components used merely for performance improvement is also included in the scope of the rights of the present invention.

[0018] FIG. 1 is a block diagram showing a video encoding apparatus according to an embodiment of the present invention.

[0019] Referring to FIG. 1, the video encoding apparatus 100 includes a motion prediction unit 110, a motion compensation unit 115, an intra prediction unit 120, a subtractor 125, a conversion unit 130, a quantization unit 135, an entropy encoding unit 140, an inverse quantization unit 145, an inverse conversion unit 150, an adder 155, a filter unit 160, and a reference video buffer 165.

[0020] The video encoding device 100 can encode the input video in either intra mode or inter mode and output a bitstream. In intra mode, prediction is performed by the intra prediction unit 120, and in inter mode, prediction can be performed via the motion prediction unit 110, motion compensation unit 115, etc. After generating prediction blocks for the input blocks of the input video, the video encoding device 100 can encode the difference between the input blocks and the prediction blocks.

[0021] In intra mode, the intra prediction unit 120 can generate predicted blocks by performing spatial predictions using the pixel values ​​of already encoded blocks surrounding the current block.

[0022] In intermode mode, the motion prediction unit 110 can find the region in the reference video stored in the reference video buffer 165 that best matches the input block during the motion prediction process and obtain a motion vector. The motion compensation unit 115 can generate a predicted block by performing motion compensation using the motion vector and the reference video stored in the reference video buffer 165.

[0023] The subtractor 125 can generate a residual block by the difference between the input block and the generated predicted block. The transformer 130 can output a transform coefficient by performing a transformation on the residual block. A residual signal means the difference between the original signal and the predicted signal, or a signal in the transformed form of the difference between the original signal and the predicted signal, or a signal in the transformed and quantized form of the difference between the original signal and the predicted signal. A residual signal is called a residual block in block units.

[0024] The quantization unit 135 can output quantized coefficients obtained by quantizing the conversion coefficients using quantization parameters.

[0025] The entropy coding unit 140 can output a bitstream by entropy coding symbols corresponding to values ​​calculated by the quantization unit 135 or coding parameter values ​​calculated during the coding process using a probability distribution.

[0026] When entropy coding is applied, the compression performance of video coding can be improved by assigning fewer bits to symbols with a high probability of occurrence and more bits to symbols with a low probability of occurrence.

[0027] For entropy coding, coding methods such as CAVLC (Context-Adaptive Variable Length Coding) and CABAC (Context-Adaptive Binary Arithmetic Coding) can be used. For example, the entropy coding unit 140 can perform entropy coding using a Variable Length Coding (VLC) table. The entropy coding unit 140 can also perform entropy coding after deriving a binary method for the target symbol and a probability model for the target symbol / bin.

[0028] The quantized coefficients can be dequantized by the dequantization unit 145 and then inversely transformed by the inverse transformation unit 150. The adder 155 can generate a restored block using the predicted block and the inversely transformed quantized coefficients.

[0029] The filter unit 160 can apply at least one of the following filters to a restored block or restored picture: a deblocking filter, SAO (Sample Adaptive Offset), or ALF (Adaptive Loop Filter). The restored block that has passed through the filter unit 160 can be stored in the reference video buffer 165.

[0030] Figure 2 is a block diagram showing an image decoding device according to one embodiment of the present invention.

[0031] Referring to Figure 2, the video decoding device 200 includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transformation unit 230, an intra prediction unit 240, a motion compensation unit 250, a filter unit 260, a reference video buffer 270, and an adder 280.

[0032] The video decoding device 200 can receive the bitstream output from the encoder, perform decoding in intra-mode or inter-mode, and output the reconstructed video, i.e., the restored video. In intra-mode, prediction is performed by the intra-prediction unit 240, and in inter-mode, prediction can be performed via the motion compensation unit 250. The video decoding device 200 can generate a predicted block by obtaining a restored residual block from the input bitstream, and then generate a reconstructed block, i.e., the restored block, by adding the restored residual block and the predicted block.

[0033] The entropy decoding unit 210 can entropy decode the input bitstream using a probability distribution to generate symbols in the form of quantized coefficients. The entropy decoding method can be performed in accordance with the entropy coding method described above.

[0034] The quantized coefficients are inversely quantized in the inverse quantization unit 220 and inversely transformed in the inverse transformation unit 230. As a result of the inverse quantization / inverse transformation of the quantized coefficients, a residual block can be generated.

[0035] In intra mode, the intra prediction unit 240 can generate predicted blocks by performing spatial prediction using the pixel values ​​of already encoded blocks surrounding the current block. In inter mode, the motion compensation unit 250 can generate predicted blocks by performing motion compensation using the motion vector and the reference video stored in the reference video buffer 270.

[0036] The adder 280 can generate a reconstructed block based on the residual block and the prediction block. The filter unit 260 can apply at least one of the following to the reconstructed block: a deblocking filter, SAO, or ALF. The filter unit 260 outputs the reconstructed image, i.e., the reconstructed image. The reconstructed image can be stored in the reference image buffer 270 and used for inter-frame prediction.

[0037] Constrained intra-prediction (CIP) is a technique for making video encoding or decoding more robust to errors. CIP technology, when used for intra-prediction, does not utilize the restored pixel region surrounding the target block if it is inter-screen encoded. However, when the restored pixel region surrounding the target block is intra-screen encoded, it uses the restored surrounding pixels to generate reference pixels using interpolation or extrapolation. Intra-prediction can then be performed based on these generated reference pixels. Therefore, even if the picture referenced by the surrounding inter-screen encoded block is lost, the target block remains unaffected. However, conventional deblocking filtering processes always perform filtering on the restored video regardless of the applicability of restricted intra-prediction or encoding parameters, allowing errors to propagate to areas of the restored video where errors do not occur. For example, an error occurring in an inter-screen encoded block can be propagated to an intra-screen encoded block. Therefore, conventional deblocking filtering processes have the problem of significantly degrading the subjective image quality of the restored video.

[0038] To solve the aforementioned problems, a method can be proposed in which a flag is transmitted to indicate whether or not to apply limited sample adaptive offset (SAO) compensation or adaptive loop filter (ALF). If the transmitted flag determines that the pixel adaptive offset compensation or adaptive loop filter should be applied restrictively, then the pixel adaptive offset compensation or adaptive loop filter can be applied restrictively based on the encoding parameters of the current block and surrounding blocks. This allows the intra-screen encoded block to be decoded correctly even if the inter-screen encoded block cannot be restored correctly. In other words, it is possible to prevent errors in the inter-screen encoded block from being propagated to the intra-screen encoded block, and the restoration result of the intra-screen encoded block can be maintained identically in the encoder and decoder.

[0039] The proposed video coding and decoding methods are described below through examples. First, a video coding and decoding method based on limited pixel-adaptive offset compensation is described. Pixel-adaptive offset compensation can be included in in-loop filtering, and in-loop filtering may include a deblocking filter in addition to pixel-adaptive offset compensation.

[0040] Figure 3 shows one example of the proposed video encoding method.

[0041] In step S100, the encoder sends a limited offset compensation indicator to the decoder. In step S110, the encoder sends an SAO compensation indicator to the decoder indicating whether or not to perform sample adaptive offset (SAO) compensation. In step S120, the encoder sends SAO parameters to the decoder. In step S130, the encoder performs pixel adaptive offset compensation on the recovered image based on the SAO compensation indicator and SAO parameters.

[0042] The limited offset compensation indicator transmitted in step S100 will be described below.

[0043] The decoder can determine, based on the restricted offset compensation indicator sent from the encoder, whether at least one of the following elements—sequence, picture, frame, field, slice, coding unit (CU), prediction unit (PU), or transform unit (TU)—supports restricted offset compensation.

[0044] The encoder can insert a restricted offset compensation indicator into the bitstream and transmit it to the decoder. The restricted offset compensation indicator can be inserted into the bitstream via an entropy coding process such as arithmetic coding or variable length coding (VLC). The restricted offset compensation indicator can be transmitted using a sequence parameter set (SPS), picture parameter set (PPS), adaptation parameter set (APS), or slice header within the bitstream. The decoder can obtain the transmitted restricted offset compensation indicator by parsing the bitstream through an entropy decoding process.

[0045] Table 1 shows an example of a restricted offset compensation indicator inserted into the bitstream. In Table 1, the offset compensation indicator is inserted into the sequence parameter set.

[0046] [Table 1]

[0047] In Table 1, `constrained_offset_flag` indicates the restricted offset compensation indicator. A value of `constrained_offset_flag` of 0 indicates that restricted offset compensation is not supported, while a value of 1 indicates that restricted offset compensation is supported. Alternatively, a value of 1 for `constrained_intra_pred_flag`, a parameter for error-resistant in-screen prediction, indicates that restricted offset compensation is supported without the insertion of a separate offset compensation indicator.

[0048] Table 2 shows other examples of restricted offset compensation indicators inserted into a bitstream. In Table 2, the restricted offset compensation indicator is inserted into the picture parameter set.

[0049] [Table 2]

[0050] In Table 2, `constrained_offset_flag` indicates the restricted offset compensation indicator. A value of `constrained_offset_flag` of 0 indicates that restricted offset compensation is not supported, while a value of 1 indicates that restricted offset compensation is supported.

[0051] Table 3 shows other examples of restricted offset compensation indicators inserted into a bitstream. In Table 3, the restricted offset compensation indicator is inserted into the picture parameter set.

[0052] [Table 3]

[0053] Table 3 indicates that loop_filter_across_tiles_enabled_flag or loop_filter_across_slices_enabled_flag are restricted offset compensation indicators. A value of 0 for loop_filter_across_tiles_enabled_flag indicates that restricted offset compensation is supported. Alternatively, a value of 0 for loop_filter_across_slices_enabled_flag indicates that restricted offset compensation is supported.

[0054] Alternatively, the encoder and decoder can always support limited offset compensation without the need to insert a separate offset compensation indicator.

[0055] On the other hand, if the restricted offset compensation indicator is set to 1 and the encoder performs restricted offset compensation, the encoder can utilize coding parameters. These coding parameters include at least one of the following: coding mode (intra-coded or inter-coded), intra-prediction mode, inter-prediction mode, coded block flag (CBF), quantization parameter, motion vector, motion vector predictor, reference picture index, and whether or not a slice / tile boundary is enabled.

[0056] For example, the encoding parameters can include tile boundaries, and if the value of the restricted offset compensation indicator is 0, offset compensation can be restricted so that it does not extend beyond the tile boundaries. In this case, the restricted offset compensation indicator is loop_filter_across_tiles_enabled_flag in Table 3. Tile boundaries can be determined based on the tile identifier (ID). Alternatively, the encoding parameters can include slice boundaries, and if the value of the restricted offset compensation indicator is 0, offset compensation can be restricted so that it does not extend beyond the slice boundaries. In this case, the restricted offset compensation indicator is loop_filter_across_slices_enabled_flag in Table 3. Slice boundaries can be determined based on the slice identifier.

[0057] For example, encoding parameters can be used to determine whether a block or surrounding block to which limited offset compensation is applied is encoded in-screen or inter-screen. In this case, if a block is encoded in-screen, it can be said that the block is encoded in intra-screen mode, and if a block is encoded inter-screen, it can be said that the block is encoded in inter-screen mode. Also, for example, if a block is encoded in PCM (pulse coded modulation) mode, it can be determined that the block is encoded in-screen.

[0058] When an encoder performs limited offset compensation using coding parameters, the coding parameters determine the degree of reliability, and this determined degree of reliability can be applied when performing limited offset compensation. For example, as shown in Table 4, the degree of reliability can be determined by each coding parameter, or by a combination of one or more coding parameters.

[0059] [Table 4]

[0060] Referring to Table 4, blocks encoded within a screen can be considered highly reliable because predictions are currently performed within the slice, while blocks encoded between screens can be considered less reliable because predictions were previously performed via the slice. Furthermore, if CBF=0 or the inter-screen mode is skip mode, there is no residual signal, resulting in relatively greater distortion and lower reliability compared to other blocks. Blocks within a slice / tile boundary can be considered highly reliable, while blocks outside the boundary can be considered less reliable. In Table 3, if the value of the restricted offset compensation indicator, loop_filter_across_tiles_enabled_flag or loop_filter_across_slices_enabled_flag, is 0, unreliable out-of-bounds areas are not permitted.

[0061] When limited offset compensation is performed, in particular, pixel-adaptive offset compensation can be implemented. The encoder can improve performance by calculating the pixel value error between the original image and the reconstructed image to determine the offset, and applying this to the reconstructed image to minimize distortion from the original image.

[0062] The SAO compensation indicator transmitted in step S110 can be included in the sequence parameter set, picture parameter set, adaptive parameter set, or slice header. The SAO compensation indicator is sample_adaptive_offset_enabled_flag. Additionally, signaling whether to perform pixel adaptive offset compensation for the luminance and chrominance components can be included in each bitstream.

[0063] The SAO parameters sent in step S120 will be explained.

[0064] An SAO parameter can include at least one of the following: offset compensation block structure, quad tree depth, offset type, offset category, and offset value. The SAO parameter can include the offset compensation block structure within the bitstream. The offset compensation block structure within the SAO parameter is sao_split_flag. In this case, a single slice can be split into a quad tree, and information regarding the offset compensation block structure can be signaled. Information regarding the depth of the quad tree split can also be included in the bitstream, and the smallest unit of the split region is the LCU (largest coding unit).

[0065] Alternatively, the SAO parameters may include the offset type, offset type, offset sign, and offset value. Table 5 shows the number of offset types and resulting offset types in pixel adaptive offset compensation.

[0066] [Table 5]

[0067] Referring to Table 5, there are a total of 7 offset types. However, the offset types in Table 5 are merely examples, and the number of offset types can vary. Each offset type can have a different number of each other and different offset values. Edge offsets (EO) can be classified into four offset types based on their angle. Each offset type in edge offsets can have four offset types depending on the condition. The offset type and offset code in edge offsets can be determined by comparing the offset-compensated pixel with the surrounding pixels. That is, in the case of edge offsets, the offset type and offset code can be determined by the decoder without additional signaling. Band offsets (BO) can be classified into two offset types based on the position of the band, each of which can have 16 offset types. In band offsets, the offset type can be determined by dividing the range of pixel values ​​that the offset-compensated pixel can have into 16 intervals, and then determining which of these intervals it falls into. The determined offset type can be used to encode an offset type index, which can then be signaled to the decoder, and the offset type and offset code can be classified by the encoder and decoder respectively based on the condition without signaling. The determined offset type and offset code can correspond to the parsed offset values, respectively. If the offset type is determined to be an edge offset, four offset values ​​can be signaled to the decoder; if it is determined to be a band offset, sixteen offset values ​​can be signaled to the decoder.

[0068] On the other hand, SAO parameters can be determined based on the coding parameters of at least one block from among the block targeted for pixel adaptive offset compensation or the surrounding blocks of the target block. For example, when determining the offset type with edge offset type, the coding parameters of at least one block from among the block targeted for pixel adaptive offset compensation and the surrounding blocks can be used. For example, the coding parameters may include tile boundaries, which can be determined based on tile identifiers. Alternatively, the coding parameters may include slice boundaries, which can be determined based on slice identifiers.

[0069] Figure 4 shows the types of edge offsets determined by angle. Edge offsets can be classified into four offset types based on angle. In Figure 4, C represents the pixel subject to pixel adaptive offset compensation, and N represents the peripheral pixel.

[0070] Figure 5 shows how the offset type and offset code are determined by the edge offset type using the coding parameters in the proposed video coding method.

[0071] Referring to Figure 5, the target block and the left-hand block for pixel adaptive offset compensation are encoded within the screen, while the target block and the upper block are encoded across screens. That is, in Figure 5, C and N1 are in-screen block pixels, and N2 is an inter-screen block pixel. Table 6 shows the conditions for determining the offset type, where N can be either N1 or N2. If the offset type is determined to be 1 or 2, the offset sign can be a positive number, and if the offset type is determined to be 3 or 4, the offset sign can be a negative number.

[0072] [Table 6]

[0073] Assume that the target pixel for pixel-adaptive offset compensation is contained within a block encoded in the screen, and an error occurs in a pixel in a surrounding block encoded between screens. In this case, when determining the offset type, the pixels in the surrounding block encoded between screens are not used, and only the pixels encoded in the screen within the target block are used to determine the offset type. This is to prevent the error from propagating to the pixels in the screen-encoded block. Alternatively, when determining the offset type, the pixels in the surrounding block encoded between screens are not used, and the pixels in the surrounding block encoded between screens are replaced with pixels in the screen-encoded block to determine the offset type. For example, in Figure 5, the pixel value of N2 can be changed to the pixel value of D to determine the offset type. Alternatively, the offset type may not be determined at all.

[0074] The encoder can restore the offset-compensated pixel value by adding the offset value calculated based on the SAO compensation indicator and SAO parameters to the pixel value. After decoding each offset value, the decoder can perform pixel-adaptive offset compensation using the offset value corresponding to the offset type, which is classified conditionally for each pixel within each block.

[0075] Pixel-adaptive offset compensation can be performed based on the coding parameters of at least one block, either the target block or its surrounding blocks. The coding parameters may include tile boundaries, and pixel-adaptive offset compensation can be performed based on tile boundaries. For example, pixel-adaptive offset compensation may not be performed across tile boundaries. Alternatively, the coding parameters may include slice boundaries, and pixel-adaptive offset compensation can be performed based on slice boundaries. Pixel-adaptive offset compensation may not be performed across slice boundaries.

[0076] Alternatively, assuming that the target pixel for pixel-adaptive offset compensation is contained within a block encoded in the screen, and an error occurs in a pixel of a surrounding block encoded between screens, pixel-adaptive offset compensation can be performed using only the pixels encoded within the target block, without using the pixels of the surrounding blocks encoded between screens. This is to prevent the error from propagating to the pixels of the blocks encoded within the screen. Alternatively, pixel-adaptive offset compensation can be performed by replacing the pixels of the surrounding blocks encoded between screens with pixels of the blocks encoded within the screen, without using the pixels of the surrounding blocks encoded between screens. Or, pixel-adaptive offset compensation may not be performed at all.

[0077] In summary, the encoder divides a single slice into blocks of varying sizes within a quad-tree structure, determines the optimal type of edge offset or band offset for each block using rate-distortion optimization (RDO), and then determines the offset type and offset value for the determined optimal type. Such SAO parameters can then be entropy-encoded and transmitted to the decoder.

[0078] The video encoding method based on limited offset compensation described above can also be directly applied to video decoding methods. That is, the decoder receives and parses the limited offset compensation indicator, SAO compensation indicator, SAO parameters, etc., transmitted from the encoder, and performs pixel-adaptive offset compensation based on this.

[0079] Figure 6 shows one embodiment of the proposed video decoding method.

[0080] In step S200, the decoder receives a limited offset compensation indicator from the encoder. Table 7 shows an example of a limited offset compensation indicator inserted into the picture parameter set.

[0081] [Table 7]

[0082] In Table 7, if the value of constrained_intra_pred_flag is 1, i.e., if restricted intra-predation is performed, the decoder can parse constrained_in_loop_filter_flag to determine whether to apply the restricted in-loop filter. If the value of constrained_in_loop_filter_flag is 1, it can be instructed to apply the restricted in-loop filter, and if the value of constrained_in_loop_filter_flag is 0, it can be instructed not to apply the restricted in-loop filter. The application of the restricted in-loop filter is limited to at least one of the following: deblocking filter, offset compensation, and ALF.

[0083] In step S210, the decoder receives an SAO compensation indicator from the encoder that indicates whether or not SAO compensation should be performed. The decoder can determine whether or not SAO compensation should be performed by parsing the SAO compensation indicator sample_adaptive_offset_enabled_flag, which is transmitted in the bitstream as part of the sequence parameter set, picture parameter set, adaptive parameter set, or slice header. The decoder can also parse information from the bitstream to determine whether or not SAO compensation should be performed for each of the luminance and chrominance components.

[0084] In step S220, the decoder receives the SAO parameters from the encoder. The decoder can parse the SAO parameters sent from the encoder. For example, if the SAO parameter contains sao_split_flag, which is information about the offset-compensated block structure within the bitstream, the decoder can parse this to determine the block structure that performs pixel-adaptive offset compensation. The decoder can also parse information about the depth to which the bitstream is split into quad trees.

[0085] If the SAO parameters include an offset type and offset type, the offset type and the resulting offset type are based on Table 5 described above. There are a total of seven offset types. Each offset type can have a different number of offsets and different offset values. If the offset type is determined to be an edge offset, the decoder can parse four offset values ​​from the bitstream, and if it is determined to be a band offset, it can parse sixteen offset values ​​from the bitstream. Furthermore, each offset type can correspond to the parsed offset values. For example, with an edge offset, the offset type and offset code can be determined by comparing the offset-compensated pixel with the surrounding pixels, and with a band offset, the offset type can be determined by dividing the range of pixel values ​​that the offset-compensated pixel can have into 16 intervals and then determining which of these intervals it falls into.

[0086] On the other hand, when determining the offset type based on the offset type, if the target pixel belongs to a block encoded within the screen and the surrounding pixels belong to a block encoded between screens, the offset type for the target pixel may not be determined. That is, the value of the offset type can be set to 0 so that offset compensation is not performed. For example, if the value of constrained_in_loop_filter_flag in the offset compensation indicator is 1, and the pixel located at (x,y) belongs to a block encoded within the screen, and one or more pixels located at (x+hPos[k],y+vPos[k]) belong to a block encoded between screens, the value of the offset type can be set to 0. In this case, hPos[k] and vPos[k] are values ​​that indicate the position of the surrounding pixels according to the offset type, and can be determined by Table 8. k=0 or 1.

[0087] [Table 8]

[0088] Referring to Table 8, for example, if the offset type is 2, the value of constrained_in_loop_filter_flag in the offset compensation indicator is 1, and the pixel located at (x,y) belongs to a block encoded within the screen, and one or more pixels located at (x,y+1) or (x,y-1) belong to a block encoded between screens, then the offset type value can be set to '0'.

[0089] On the other hand, if the value of the restricted offset compensation indicator is 1, and the pixel located at (x,y) and one or more pixels located at (x+hPos[k],y+vPos[k]) belong to different slices / tiles, that is, if one or more pixels located at (x+hPos[k],y+vPos[k]) are located outside the slice / tile to which the pixel located at (x,y) belongs, the offset type value can be set to 0. Also, if the boundary of the slice / tile is the boundary of the picture, then outside the boundary of the slice / tile is outside the picture where there are no pixels.

[0090] On the other hand, the SAO parameters can be determined based on the coding parameters of at least one block, either the block targeted for pixel adaptive offset compensation or the surrounding blocks of the target block.

[0091] In step S230, the decoder performs pixel-adaptive offset compensation based on the SAO compensation indicator and SAO parameters. The decoder can restore the offset-compensated pixel value by adding the offset value calculated based on the SAO compensation indicator and SAO parameters to the pixel value. Pixel-adaptive offset compensation can be performed based on the coding parameters of at least one block of the target block or surrounding blocks of the target block. If the offset type value is set to 0, pixel-adaptive offset compensation may not be performed on the target pixel. That is, RecSaoPicture[x,y] = RecPicture[x,y]. RecSaoPicture[x,y] shows the pixel value after performing pixel-adaptive offset compensation on the pixel located at (x,y), and RecPicture[x,y] shows the restored pixel value before performing pixel-adaptive offset compensation.

[0092] The following describes the video encoding and decoding method based on ALF.

[0093] Figure 7 shows another example of the proposed video encoding method.

[0094] In step S300, the encoder sends a restricted loop filter indicator to the decoder. In step S310, the encoder sends an ALF application indicator to the decoder indicating whether ALF can be applied. In step S320, the encoder sends ALF parameters to the decoder. In step S330, the encoder applies ALF to the restored video based on the ALF application indicator and ALF parameters.

[0095] The restricted loop filter indicator sent in step S300 will be explained below.

[0096] The decoder can determine whether to apply a restricted loop filter to at least one of the sequences, pictures, frames, fields, slices, CUs, PUs, or TUs to be encoded, based on the restricted loop filter directive sent from the encoder.

[0097] The encoder can insert restricted loop filter indicators into the bitstream and transmit them to the decoder. Restricted loop filter indicators can be inserted into the bitstream via arithmetic coding or entropy coding processes such as VLC. Restricted offset compensation indicators can be transmitted using SPS, PPS, APS, or slice headers within the bitstream. The decoder can obtain the transmitted restricted offset compensation indicators by parsing the bitstream through an entropy decoding process.

[0098] Table 9 shows an example of a restricted loop filter directive inserted into a bitstream. In Table 9, the loop filter directive is inserted into the sequence parameter set.

[0099] [Table 9]

[0100] In Table 9, `constrained_filter_flag` indicates the restricted loop filter directive. A value of `constrained_filter_flag` of 0 indicates that the restricted loop filter will not be applied, while a value of 1 indicates that the restricted loop filter will be supported. Alternatively, a value of 1 for `constrained_intra_pred_flag`, a parameter for error-resistant in-screen prediction, indicates that the restricted loop filter will be applied without the insertion of a separate loop filter directive.

[0101] Table 10 shows other examples of restricted loop filter directives inserted into a bitstream. In Table 10, the restricted loop filter directive is inserted into the picture parameter set.

[0102] [Table 10]

[0103] Table 10 shows the constrained_filter_flag indicator for restricted loop filtering. A value of 0 for constrained_filter_flag indicates that the restricted loop filter should not be applied, while a value of 1 for constrained_filter_flag indicates that the restricted loop filter should be applied.

[0104] Alternatively, as shown in Table 3, loop_filter_across_tiles_enabled_flag or loop_filter_across_slices_enabled_flag can indicate a restricted loop filter indicator. A value of 0 for loop_filter_across_tiles_enabled_flag can indicate that a restricted loop filter should be applied. Or, a value of 0 for loop_filter_across_slices_enabled_flag can indicate that a restricted loop filter should be applied.

[0105] Alternatively, the encoder and decoder can always apply a restricted loop filter without the need to insert a separate loop filter directive.

[0106] On the other hand, if the restricted loop filter indicator is set to 1 and the encoder applies a restricted loop filter, the encoder can utilize encoding parameters. These encoding parameters include at least one of the following: encoding mode (in-screen or cross-screen encoding), in-screen prediction mode, cross-screen prediction mode, CBF, quantization parameters, motion vector, motion vector predictor, reference image index, and slice / tile boundary eligibility.

[0107] For example, the encoding parameters can include tile boundaries, and if the value of the restricted loop filter indicator is 0, the loop filter can be restricted so that it is not applied beyond tile boundaries. In this case, the restricted loop filter indicator is loop_filter_across_tiles_enabled_flag in Table 3. Tile boundaries can be determined based on the tile identifier (ID). Alternatively, the encoding parameters can include slice boundaries, and if the value of the restricted loop filter indicator is 0, the loop filter can be restricted so that it is not applied beyond slice boundaries. In this case, the restricted loop filter indicator is loop_filter_across_slices_enabled_flag in Table 3. Slice boundaries can be determined based on the slice identifier.

[0108] For example, encoding parameters can be used to determine whether a block or surrounding block to which a restricted loop filter is applied is encoded in-screen or cross-screen. In this case, if a block is encoded in-screen, it can be said that the block is encoded in in-screen mode, and if a block is encoded cross-screen, it can be said that the block is encoded in cross-screen mode. Also, for example, if a block is encoded in PCM mode, it can be determined that the block is encoded in-screen.

[0109] When an encoder performs limited offset compensation using coding parameters, the coding parameters can determine the degree of confidence, and this determined degree of confidence can be applied when performing limited offset compensation. For example, as shown in Table 4, the degree of confidence can be determined by each coding parameter, or by a combination of one or more coding parameters. In Table 3, if the value of the limited loop filter indicator loop_filter_across_tiles_enabled_flag or loop_filter_across_slices_enabled_flag is 0, weak out-of-bounds areas are not allowed.

[0110] When a restricted loop filter is applied, an ALF (Alternate Layer Filter) can be applied in particular. The encoder can then derive filter coefficients based on a Wiener filter that minimizes the error between the original and reconstructed images, and apply these to the reconstructed image to minimize distortion from the original image.

[0111] The ALF application indicator transmitted in step S310 can be included in the sequence parameter set, picture parameter set, adaptive parameter set, or slice header. The ALF application indicator is adaptive_loop_filter_flag. Furthermore, the applicability of ALF to the luminance and chrominance components can be included in the respective bitstreams for signaling. ALF applicability can also be determined on a per-CU or per-image basis.

[0112] The ALF parameters transmitted in step S320 will be explained below.

[0113] The ALF parameters may include at least one of the following: filter shape, filter coefficient, filter classification method, filter index, filter prediction method, and maximum filter execution depth.

[0114] The encoder can determine the optimal filter shape from among several filter shapes. The encoder can also determine the filter coefficients to be used to apply ALF. In this case, there can be one or more filter coefficients, which can be encoded into exponential Golomb codes of different orders. To efficiently encode the filter coefficients, predictive coding can be performed between them using methods such as differential pulse code modulation (DPCM), and some filter coefficients can be predictively coded from the sum of other filter coefficients. Furthermore, if it is decided to apply ALF, the encoder can select a filter using either region-based adaptation (RA) or block-based adaptation (BA) as the filter classification method. For example, if the filter classification method is determined to be RA, the value of alf_region_adaptation_flag is set to 1, and if the filter classification method is determined to be BA, the value of alf_region_adaptation_flag is set to 0. When the RA method is used, one filter can be selected from among several filters for each divided video region. When the BA method is used, one filter can be selected from among several filters considering the amount and directionality of pixel changes. In this case, the filter index in the ALF parameter can be used to indicate which filter has been selected. In addition, information about the maximum depth to which ALF is applied can be inserted into the bitstream so that ALF is applied only up to a specific depth of CU.

[0115] Figure 8 shows an example of a filter shape determined by the encoder using the proposed video encoding method. Referring to Figure 8, the numbers within each filter represent the filter coefficient index. The encoder sends information about the filter shape and filter classification method to the decoder in the ALF parameters, and the filter is selected according to the determined filter classification method. Up to 16 filters can exist. When filtering is performed based on the selected filters, when filtering pixel values ​​located in the center of the filter shape, filtering can be performed by summing the products of each filter coefficient and the pixel value corresponding to each position.

[0116] On the other hand, when classifying filters based on the BA method, the coding parameters of at least one block among the target block and surrounding blocks to which ALF is applied can be used. For example, the coding parameters may include tile boundaries, which can be determined based on the tile identifier. Alternatively, the coding parameters may include slice boundaries, which can be determined based on the slice identifier.

[0117] Figure 9 shows how the proposed video encoding method uses encoding parameters to classify filters based on the BA method. For example, if the target block for ALF application is encoded within the screen and the surrounding blocks are encoded between screens, and the horizontal or vertical orientation is determined in 4x4 block units, then in Figure 9, pixels within a 4x4 block without diagonal lines can be pixels of the in-screen block, and pixels with diagonal lines can be pixels of the inter-screen block. Also, 'R' indicates a restored pixel, VA indicates a vertical orientation, and HA indicates a horizontal orientation.

[0118] For example, consider a scenario where the target pixels for ALF application based on the BA method are included in an encoded block within the screen, and an error occurs in the pixels of a surrounding encoded block between screens. In this case, the filter can be classified using only the pixels encoded within the target block, without using the pixels of the surrounding blocks encoded between screens. This is to prevent errors from propagating to the pixels of the blocks encoded within the screen. Alternatively, the filter can be classified by replacing the pixels of the surrounding blocks encoded between screens with pixels of the blocks encoded within the screen, without using the pixels of the surrounding blocks encoded between screens. For example, in Figure 9, 'R (0,0) 'When determining horizontal or vertical orientation by position, 'R' is included in the inter-screen block. (-1,0) 'or 'R (0,-1) After changing the value to the value of the block on the screen, the direction can be determined. Alternatively, filters may not be categorized at all.

[0119] Figure 10 shows an example of applying ALF using encoding parameters with the proposed video encoding method.

[0120] When ALF is applied, it can be determined based on the coding parameters of at least one block among the block to which ALF is applied or the surrounding blocks of the block to which ALF is applied. The coding parameters include tile boundaries, and ALF can be applied based on tile boundaries. For example, ALF may not be applied across tile boundaries. Alternatively, the coding parameters include slice boundaries, and ALF can be applied based on slice boundaries. ALF may not be applied across slice boundaries.

[0121] Alternatively, assuming that the target pixels for ALF application are contained within a block encoded on the screen, and an error occurs in the pixels of a surrounding block encoded between screens, ALF can be applied using only the pixels within the target block or the pixels of the surrounding blocks encoded on the screen, without using the pixels of the surrounding blocks encoded between screens. This is to prevent the error from propagating to the pixels of the blocks encoded on the screen. When applying the filter shape in Figure 8(a) to each pixel value of the 4x4 block in Figure 10(a), the position of the pixel to be filtered is '9' in the center, and the filter is applied using the surrounding pixel values ​​and the filter coefficient at the corresponding position. In this case, the filter is applied only if the filter coefficient is included in a block encoded within the screen, as shown in Figure 10(b). That is, the filter can be applied only to pixel values ​​i, j, k, l, m, n, o, and p. Alternatively, ALF can be applied by not using pixels from blocks encoded between screens among the surrounding blocks, and replacing pixels from blocks encoded between screens with pixels from blocks encoded within the screen. Or, ALF may not be applied at all.

[0122] The encoder can apply ALF based on the ALF application indicator and ALF parameters. ALF can be applied based on the encoding parameters of at least one block, either the block to which ALF is applied or the surrounding blocks of the block to which it is applied.

[0123] In summary, the encoder synchronizes a slice into a coding tree block structure, determines the feasibility of filtering, maximum filter depth, filter prediction method, filter classification method, filter shape, and filter coefficients for each CU using RDO, and applies ALF using the determined optimal ALF parameters. These ALF parameters can then be entropy encoded and transmitted to the decoder.

[0124] The video encoding method based on the restricted loop filter described above can also be directly applied to the video decoding method. That is, the decoder receives and parses the restricted loop filter indicator, ALF application indicator, ALF parameters, etc., transmitted from the encoder, and applies the ALF based on this.

[0125] Figure 11 shows one embodiment of the proposed video decoding method.

[0126] In step S400, the decoder receives a restricted loop filter indicator from the encoder. Table 11 shows an example of a restricted loop filter indicator inserted into the picture parameter set.

[0127] [Table 11]

[0128] In Table 11, if the value of constrained_intra_pred_flag is 1, i.e., if restricted intra-predation is performed, the decoder can parse constrained_in_loop_filter_flag to determine whether to apply the restricted in-loop filter. If the value of constrained_in_loop_filter_flag is 1, it can be instructed to apply the restricted in-loop filter, and if the value of constrained_in_loop_filter_flag is 0, it can be instructed not to apply the restricted in-loop filter. The restricted in-loop filter can be applied to at least one of the following: deblocking filter, offset compensation, and ALF.

[0129] In step S410, the decoder receives an ALF application indicator from the encoder that indicates whether ALF is applicable or not. The decoder can determine whether ALF is applicable or not by parsing the ALF application indicator adaptive_loop_filter_enabled_flag, which is transmitted in the bitstream as part of the sequence parameter set, picture parameter set, adaptive parameter set, or slice header. The decoder can also parse information from the bitstream that determines whether ALF is applicable or not for each of the luminance and chrominance components, or information regarding ALF applicability on a CU basis.

[0130] In step S420, the decoder receives the ALF parameters from the encoder.

[0131] The decoder can parse the ALF parameters transmitted from the encoder. The ALF parameters may include at least one of the following: filter shape, filter coefficients, filter classification method, filter index, filter prediction method, and maximum filter execution depth. For example, the decoder can parse the bitstream to determine the filter shape and / or filter coefficients. In this case, there may be one or more filter coefficients, which can be decoded into exponential Golomb codes of different orders. Furthermore, to efficiently decode the filter coefficients, they can be predicted and decoded using methods such as DPCM, and some filter coefficients can be predicted and decoded from the sum of other filter coefficients. Additionally, filters can be selected differently using either the RA method or the BA method as a filter classification means. For example, if the alf_region_adaptation_flag transmitted by the encoder is parsed and the result is '1', the filter can be classified using the RA method; if the result is '0', the filter can be classified using the BA method. When the RA method is used, one filter can be selected from among several filters for each divided image region. When the BA method is used, one filter can be selected from among several filters considering the amount and directionality of pixel changes. In this case, the filter index in the ALF parameter can be used to indicate which filter has been selected.

[0132] When the block to which the ALF is applied is encoded within the screen, and the blocks to which the surrounding pixels belong are encoded across screens, when determining the horizontal or vertical orientation of a block, it is possible to determine the corresponding filter using only the pixels of the in-screen block. For example, in Figure 9, R is used for orientation determination. (0,0) and R (0,2) , R (2,0) and R (2,2)The formula in Figure 9 can be applied to the position, and in Figure 9, if the shaded area is encoded between screens and the unshaded area is encoded within a screen, the decoder will determine that both the ALF target pixel and the surrounding pixels belong within the screen. (2,2) Direction can be determined solely by location, allowing the application of the appropriate filter to be decided.

[0133] On the other hand, the ALF parameters can be determined based on the encoding parameters of at least one block among the block to which ALF is applied or the surrounding blocks of the block.

[0134] In step S430, the decoder applies ALF based on the ALF application indicator and ALF parameters. The decoder can apply ALF based on the ALF application indicator and ALF parameters. ALF can be applied based on the encoding parameters of at least one block from the block to which ALF is applied or the surrounding blocks of the block to which the block to which ALF is applied. If the block to which the pixel to which ALF is applied belongs is encoded within a screen and the surrounding pixels belong to a block encoded between screens, ALF may not be applied to the pixel to which ALF is applied.

[0135] Figure 12 shows an example of a filter shape used in the proposed video decoding method. When applying ALF to the pixel at position '9' with a filter shape like that shown in Figure 12, if one or more pixels surrounding the target pixel belong to a block encoded between screens, ALF may not be applied to the target pixel.

[0136] The present invention can be embodied in hardware, software, or a combination thereof. In hardware implementation, it can be embodied in an ASIC (application-specific integrated circuit), DSP (digital signal processing), PLD (programmable logic device), FPGA (field programmable gate array), processor, controller, microprocessor, other electronic unit, or a combination thereof, designed to perform the functions described above. In software implementation, it can be embodied in a module that performs the functions described above. The software can be stored in a memory unit and executed by a processor. The memory unit and processor can employ a variety of means well known to those skilled in the art.

[0137] In the exemplary system described above, the method is explained based on a sequence diagram of a series of steps or blocks, but the present invention is not limited to the order of steps, and some steps may occur with other steps, in a different order, or simultaneously. Furthermore, those skilled in the art will understand that the steps shown in the sequence diagram are not exclusive, that other steps may be included, or that one or more steps in the sequence diagram can be deleted without affecting the scope of the present invention.

Claims

1. In video decoding methods, A step of receiving at least one restricted offset compensation indicator indicating whether restricted offset compensation is supported by at least one of the tiles and slices, an SAO compensation indicator indicating whether SAO compensation is feasible, and SAO parameters including an offset value, The step includes performing SAO compensation on target pixels within a target block of the restored image based on the SAO compensation indicator and the SAO parameters, The SAO parameter includes an offset value and offset type information for identifying whether an edge offset or a band offset is applied to the target pixel. When the edge offset is applied to the offset-compensated pixel within the target block, the offset category is determined based on the encoding parameters of the target block and at least one surrounding block of the target block for SAO compensation. The method for decoding video is characterized in that the offset sign of the offset value is determined by comparing the value of the target pixel in the target block with the value of an adjacent pixel of the target pixel.

2. The method according to claim 1, characterized in that the SAO compensation indicator is received with respect to each of the luminance component and the chrominance component.

3. In video encoding methods, A step of determining SAO parameters, comprising: at least one restricted offset compensation indicator indicating whether restricted offset compensation is supported by at least one of the tiles and slices; an SAO compensation indicator indicating whether SAO compensation is feasible; and at least one restricted offset compensation indicator indicating whether restricted offset compensation is supported by at least one of the tiles and slices; an SAO compensation indicator indicating whether SAO compensation is feasible; and The step includes performing SAO compensation on target pixels within a target block of the restored image based on the SAO compensation indicator and the SAO parameters, The SAO parameter includes an offset value and offset type information for identifying whether an edge offset or a band offset is applied to the target pixel. When the edge offset is applied to the offset-compensated pixel within the target block, the offset category is determined based on the encoding parameters of the target block and at least one surrounding block of the target block for SAO compensation. A video encoding method characterized in that the offset code of the offset value is determined by comparing the value of the target pixel in the target block with the value of an adjacent pixel of the target pixel.

4. The method according to claim 3, characterized in that the SAO compensation indicator is received with respect to each of the luminance component and the chrominance component.

5. A method for transmitting a bitstream generated by a video encoding method, The bitstream is transmitted, The aforementioned video encoding method is A step of determining SAO parameters, comprising: at least one restricted offset compensation indicator indicating whether restricted offset compensation is supported by at least one of the tiles and slices; an SAO compensation indicator indicating whether SAO compensation is feasible; and at least one restricted offset compensation indicator indicating whether restricted offset compensation is supported by at least one of the tiles and slices; an SAO compensation indicator indicating whether SAO compensation is feasible; and The step includes performing SAO compensation on target pixels within a target block of the restored image based on the SAO compensation indicator and the SAO parameters, The SAO parameter includes an offset value and offset type information for identifying whether an edge offset or a band offset is applied to the target pixel. When the edge offset is applied to the offset-compensated pixel within the target block, the offset category is determined based on the encoding parameters of the target block and at least one surrounding block of the target block for SAO compensation. A bitstream transmission method characterized in that the offset sign of the offset value is determined by comparing the value of the target pixel in the target block with the value of an adjacent pixel of the target pixel.

Images