Method and device for encoding and decoding images using adaptive deblocking filtering

Abstract

Disclosed is an encoding / decoding method and apparatus related to adaptive deblocking filtering. Provided is an image decoding method for performing adaptive filtering, the method comprising: determining a boundary strength; and performing deblocking filtering on a current block and an adjacent block adjacent to the current block based on the boundary strength, wherein the boundary strength is determined based on whether both the current block and the adjacent block use a current picture as a reference.

Description

[0001] This application is a divisional application of an invention patent application with an application date of June 13, 2016, application number 201680047024.X, and invention name “Method and device for encoding and decoding images using adaptive deblocking filtering”. Technical Field

[0002] The present invention relates to image encoding and decoding technology. More specifically, the present invention relates to a method and apparatus for encoding and decoding an image using adaptive deblocking filtering according to a prediction mode. Background Art

[0003] With the popularization of the Internet and mobile terminals and the development of information and communication technology, the use of multimedia data has increased. Therefore, in order to perform various services and operations in various systems through image prediction, the demand for improving the performance and efficiency of image processing systems has rapidly increased.

[0004] Meanwhile, in the existing image encoding / decoding technology, in the inter-frame prediction method, motion information of neighboring blocks of a current block is predicted in at least one reference picture located before or after the current picture, or in the intra-frame prediction method, a motion vector of the current block is estimated by obtaining motion information from a reference block in the current picture.

[0005] However, since the prediction block is generated by using a temporal prediction mode between pictures, the existing inter-frame prediction method has a problem in that its calculation is complex. The problem with the intra-frame prediction method is that its encoding is complex.

[0006] In addition, image coding includes prediction, transformation, quantization and entropy coding processes. In the quantization process, due to quantization errors, block artifacts and ringing phenomena may appear in the reconstructed picture, and thus, the subjective definition of the reconstructed image is degraded.

[0007] Therefore, in conventional image encoding and decoding techniques using quantization, filtering is performed on the reconstructed picture to improve the subjective definition. Conventional techniques related to filtering of the reconstructed picture are classified into a post-filter method and a loop filter method according to whether the filtered picture is used as a reference picture in an inter-frame prediction mode.

[0008] In the post-filter approach, filtering is performed outside the image decoder just before the reconstructed image is output. In the loop filter approach, after filtering is applied to the reconstructed picture, the filtered picture is inserted into the decoded picture buffer (DPB) and used as a reference picture in the inter-prediction mode.

[0009] Meanwhile, in the loop filter method, that is, in the deblocking filter method, when filtering is performed, the reconstructed pixels stored in the memory are loaded, and then filtering is performed. The filtered pixels are stored in the memory, and thus frequent memory access is caused. In addition, in the deblocking filter method, the filtering calculation process is complicated. Due to the computational complexity and the overhead for memory access, the decoder has a large complexity of 20 to 30%. Therefore, in the image encoding and decoding technology, an effective method for filtering the reconstructed picture is needed. Summary of the invention

[0010] Technical issues

[0011] Accordingly, the present invention has been proposed in view of the above problems, and an object of the present invention is to provide a method for encoding and decoding an image using adaptive deblocking filtering and an apparatus thereof.

[0012] Another object of the present invention is to provide a method and apparatus for encoding and decoding images, thereby improving the performance and efficiency of an image processing system.

[0013] Technical Solution

[0014] According to one aspect for achieving the above-mentioned purpose, there is provided an image decoding method for performing adaptive filtering, the method comprising: determining a boundary strength; and performing deblocking filtering on a current block and a neighboring block adjacent to the current block based on the boundary strength, wherein the boundary strength is determined based on whether both the current block and the neighboring block use a current picture as a reference.

[0015] Here, the boundary strength is determined based on the color components of the current block.

[0016] Here, when the color component of the current block is a luminance component and both the current block and the neighboring block use the current picture as a reference, the boundary strength is determined to be 1.

[0017] According to one aspect for achieving the above-mentioned purpose, there is provided an image encoding method for performing adaptive filtering, the method comprising: determining a boundary strength; and performing deblocking filtering on a current block and a neighboring block adjacent to the current block based on the boundary strength, wherein the boundary strength is determined based on whether both the current block and the neighboring block use a current picture as a reference.

[0018] According to one aspect to achieve the above-mentioned purpose, a non-transitory computer-readable medium for transmitting video data is provided, the video data including information indicating whether the corresponding block is encoded using a picture including the corresponding block as a reference, in a decoding process, information about a current block and a neighboring block adjacent to the current block is used to determine whether both the current block and the neighboring block use the current picture as a reference, wherein a boundary strength of a deblocking filter is determined based on whether both the current block and the neighboring block use the current picture as a reference.

[0019] According to one aspect for achieving the above-mentioned purpose, there is provided an image decoding method for performing adaptive filtering in inter-frame prediction, the method comprising: searching for a reference block by performing inter-frame prediction on a current block of an encoding target; generating a flag indicating whether the found reference block exists in a current picture in which the current block is located; reconstructing the current block by using the found reference block; adaptively applying a loop filter to the reconstructed current block based on the flag; and storing the current block with or without the loop filter applied in a decoded picture buffer (DPB).

[0020] Here, the loop filter may include at least one of a deblocking filter and a sample adaptive offset (SAO).

[0021] Here, in the case of adaptively applying the loop filter, when the flag indicates that the reference block exists within the current picture, the loop filter is not applied to the reconstructed current block.

[0022] Here, in the case of adaptively applying the loop filter, when the flag indicates that the reference block does not exist in the current picture, the loop filter is applied to the reconstructed current block.

[0023] According to another aspect for achieving the above-mentioned purpose, there is provided an image decoding method for performing inter-frame prediction, the method comprising: encoding and reconstructing an image signal including a reference block located in the same picture as a current block, wherein block matching is performed on the reference block in the inter-frame prediction of the current block; storing the reconstructed image signal in a decoded picture buffer (DPB) and an independent temporary memory; and performing inter-frame prediction of the current block by block matching using the reconstructed image signal.

[0024] Here, in the case where the reconstructed image signal is stored in the temporary memory, the reconstructed image signal is stored in the temporary memory after passing through the loop filter and the independent filter.

[0025] Here, independent filters perform filtering on the left and upper boundaries of the reference block included in the reconstructed image signal.

[0026] Here, the independent filter performs filtering on at least one of a transform block boundary, a prediction block boundary, and a common block boundary of a transform block and a prediction block in the reconstructed image signal.

[0027] According to another aspect for achieving the above-mentioned purpose, there is provided an image decoding method for performing adaptive filtering in inter-frame prediction, the method comprising: reconstructing an image signal including a reference block from a bitstream, wherein block matching is performed on the reference block in the inter-frame prediction of a current block to be encoded; obtaining a flag indicating whether the reference block exists in a current picture in which the current block is located from the bitstream; reconstructing the current block by using the reference block; adaptively applying a loop filter to the reconstructed current block based on the obtained flag; and storing the current block with or without the loop filter applied in a decoded picture buffer (DPB).

[0028] Here, the loop filter may include at least one of a deblocking filter and a sample adaptive offset (SAO).

[0029] Here, in the case of adaptively applying the loop filter, when the flag indicates that the reference block exists within the current picture, the loop filter is not applied to the reconstructed current block.

[0030] Here, in the case of adaptively applying the loop filter, when the flag indicates that the reference block does not exist in the current picture, the loop filter is applied to the reconstructed current block.

[0031] According to another aspect for achieving the above-mentioned purpose, there is provided an image decoding method for performing inter-frame prediction, the method comprising: reconstructing an image signal including a reference block located in the same picture as a current block from a bit stream, wherein block matching is performed on the reference block in the inter-frame prediction of the current block; storing the reconstructed image signal in a decoded picture buffer (DPB) and an independent temporary memory; and performing inter-frame prediction of the current block by block matching using the reconstructed image signal.

[0032] Here, in the case where the reconstructed image signal is stored in the temporary memory, the reconstructed image signal is stored in the temporary memory after passing through the loop filter and the independent filter.

[0033] Here, independent filters perform filtering on the left and upper boundaries of the reference block included in the reconstructed image signal.

[0034] Here, the independent filter performs filtering on at least one of a transform block boundary, a prediction block boundary, and a common block boundary of a transform block and a prediction block in the reconstructed image signal.

[0035] According to another aspect for achieving the above-mentioned purpose, there is provided an image decoding device for performing adaptive filtering in inter-frame prediction, the device comprising at least one processor, wherein the processor is configured to: reconstruct an image signal including a reference block from a bitstream, wherein block matching is performed on the reference block in the inter-frame prediction of a current block to be encoded; obtain a flag indicating whether the reference block exists in a current picture in which the current block is located from the bitstream; reconstruct the current block by using the reference block; adaptively apply a loop filter to the reconstructed current block based on the obtained flag; and store the current block with or without the loop filter applied in a decoded picture buffer (DPB).

[0036] Here, the loop filter may include at least one of a deblocking filter and a sample adaptive offset (SAO).

[0037] Here, in the case of adaptively applying the loop filter, when the flag indicates that the reference block exists within the current picture, the loop filter is not applied to the reconstructed current block.

[0038] Here, in the case of adaptively applying the loop filter, when the flag indicates that the reference block does not exist in the current picture, the loop filter is applied to the reconstructed current block.

[0039] According to another aspect for achieving the above-mentioned purpose, an image encoding method is provided, the method comprising: generating a reconstructed signal based on a prediction signal that refers to a decoded image and a residual signal generated by inverse quantization and inverse transformation; determining whether to skip filtering of the reconstructed signal; and when it is determined that filtering is not skipped, performing deblocking filtering.

[0040] According to another aspect for achieving the above-mentioned purpose, an image decoding method is provided, the method comprising: generating a reconstructed signal based on a prediction signal that refers to a decoded image and a residual signal generated by inverse quantization and inverse transformation; determining whether to skip filtering of the reconstructed signal; and when it is determined that filtering is not skipped, performing deblocking filtering.

[0041] According to another aspect to achieve the above-mentioned purpose, an image decoding device is provided, which includes an entropy decoder, an inverse quantization unit, an inverse transform unit, an addition and subtraction unit, a prediction unit, a filtering skip confirmation unit, a skip selection circuit, a filter unit and a decoded picture buffer, wherein the filtering skip confirmation unit is located between the addition and subtraction unit and the filter unit, the skip selection circuit is located in the space between the filtering skip confirmation unit and the filter unit and in the space between the filtering skip confirmation unit and the decoded picture buffer, and the filtering skip confirmation unit controls the skip selection circuit based on selection information from a filtering skip flag.

[0042] According to another aspect for achieving the above-mentioned purpose, an image encoding method is provided, the method comprising: generating a reconstructed signal based on a prediction signal that refers to a decoded image and a residual signal generated by inverse quantization and inverse transformation; determining whether to skip filtering of the reconstructed signal; and when it is determined that filtering is not skipped, performing deblocking filtering, wherein the performing of deblocking filtering comprises a first sub-step of confirming block boundaries, a second sub-step of setting filtering strength, and a third sub-step of determining whether to apply filtering.

[0043] According to another aspect for achieving the above-mentioned purpose, an image decoding method is provided, the method comprising: generating a reconstructed signal based on a prediction signal that refers to a decoded image and a residual signal generated by inverse quantization and inverse transformation; determining whether to skip filtering of the reconstructed signal; and when it is determined that filtering is not skipped, performing deblocking filtering, wherein the performing of deblocking filtering comprises a first sub-step of confirming block boundaries, a second sub-step of setting filtering strength, and a third sub-step of determining whether to apply filtering.

[0044] According to another aspect for achieving the above-mentioned purpose, an image decoding device is provided, the device comprising: a memory storing a decoding program for generating a reconstructed signal based on a prediction signal referring to a decoded image and a residual signal generated by inverse quantization and inverse transformation, wherein the memory stores a filtering program for skipping or performing filtering after determining whether to skip filtering the reconstructed signal; and a processor coupled to the memory to execute a program code of the decoding program or the filtering program, wherein through the program code, the processor is configured to: generate a reconstructed signal based on a prediction signal referring to a decoded image and a residual signal generated by inverse quantization and inverse transformation; determine whether to skip filtering the reconstructed signal; and when it is determined that filtering is not skipped, perform deblocking filtering, wherein when performing deblocking filtering, block boundaries are confirmed, filtering strength is set, and it is determined whether to apply filtering.

[0045] According to one aspect for achieving the above-mentioned purpose, there is provided an image encoding method involving configuring reference pixels in intra-frame prediction, the method comprising: obtaining reference pixels of a current block from adjacent blocks in intra-frame prediction of the current block; adaptively performing filtering on the reference pixels; generating a prediction block of the current block by using the reference pixels to which the adaptive filtering is applied as input values ​​according to a prediction mode of the current block; and applying an adaptive post-filter to the prediction block.

[0046] Here, when obtaining the reference pixel, the reference pixel of the current block may be obtained from the adjacent block.

[0047] Here, the acquisition of reference pixels may be determined according to the availability of neighboring blocks.

[0048] Here, the availability of the neighboring block may be determined based on the position of the neighboring block and / or a specific flag (constrained_intra_pred_flag). For example, when the neighboring block is available, the specific flag may have a value of 1. When the prediction mode of the neighboring block is an inter mode, the reference pixels of the block may not be available in the prediction of the current block.

[0049] Here, a specific flag (constrained_intra_pred_flag) may be determined based on a prediction mode of a neighboring block, and the prediction mode may be an intra prediction mode or an inter prediction mode.

[0050] Here, when a specific flag (constrained_intra_pred_flag) has a value of 0, regardless of the prediction mode of the neighboring block, the availability of the neighboring block may be "yes". When the specific flag has a value of 1 and the prediction mode of the neighboring block is an intra-frame prediction mode, the availability of the neighboring block may be "yes", and when the prediction mode of the neighboring block is an inter-frame prediction mode, the availability of the neighboring block may be "no".

[0051] Here, in the inter prediction, a prediction block may be generated by referring to at least one reference picture.

[0052] Here, reference pictures may be managed by reference picture list 0 (list 0) and reference picture list 1 (list 1), and one or more past pictures, future pictures, and current pictures may be included in list 0 and list 1.

[0053] Here, it may be adaptively determined whether the current picture is included in the reference picture lists (list 0 and list 1).

[0054] Here, information for determining whether the current picture is included in the reference picture list may be included in a sequence parameter set, a reference picture parameter set, or the like.

[0055] According to one aspect to achieve the above-mentioned purpose, there is provided an image decoding method performed by a computing device, the method comprising: obtaining a flag in a sequence or picture unit indicating the availability of reference pixels of a neighboring block from an input bit stream; determining the availability of reference pixels of the neighboring block when intra-frame prediction is performed based on the flag; and when the flag has a value of 0, regardless of the prediction mode of the neighboring block, the reference pixels of the neighboring block are used in the prediction of the current block, and when the flag has a value of 1 and the prediction mode of the neighboring block is an intra-frame prediction mode, the reference pixels of the neighboring block are used in the prediction of the current block, and when the prediction mode of the neighboring block is an inter-frame prediction mode, the reference pixels of the neighboring block are not used in the prediction of the current block.

[0056] Here, in the inter prediction mode, the prediction block may be generated based on block matching in a reference picture.

[0057] Here, the reference picture may be managed by list 0 in the P picture, and the reference picture may be managed by list 0 and list 1 in the B picture.

[0058] Here, in the inter prediction mode, list 0 may include the current picture.

[0059] Here, in the inter prediction mode, list 1 may include the current picture.

[0060] Here, it may be determined that the current picture is included in list 0 and list 1 based on a flag transmitted from a sequence parameter.

[0061] Here, it may be determined that the current picture is included in list 0 and list 1 based on a flag transmitted from a picture parameter.

[0062] Beneficial Effects

[0063] When the method and apparatus for encoding and decoding an image using adaptive deblocking filtering according to an embodiment of the present invention are used, the performance of deblocking filtering can be improved, thereby improving the performance of an image processing system or an image prediction system.

[0064] Furthermore, according to the present invention, the blocking effect of an intra-prediction block or an inter-prediction block can be reduced, so that the subjective definition can be improved, and the encoding efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a view illustrating a system using an apparatus for encoding an image and / or an apparatus for decoding an image according to the present invention.

[0066] Figure 2 is a block diagram illustrating an apparatus for encoding an image according to an embodiment of the present invention.

[0067] Figure 3 is a block diagram illustrating an apparatus for decoding an image according to an embodiment of the present invention.

[0068] Figure 4 is an exemplary view illustrating inter prediction of a P slice in a method for encoding and decoding an image according to an embodiment of the present invention.

[0069] Figure 5 is an exemplary view illustrating inter prediction of a B slice in a method for encoding and decoding an image according to an embodiment of the present invention.

[0070] Figure 6is an exemplary view illustrating a case where a prediction block is generated in a single direction in a method for encoding and decoding an image according to an embodiment of the present invention.

[0071] Figure 7 is an exemplary view illustrating a reference picture list in a method for encoding and decoding an image according to an embodiment of the present invention.

[0072] Figure 8 is an exemplary view illustrating another example of performing inter prediction from a reference picture list in a method for encoding and decoding an image according to an embodiment of the present invention.

[0073] Fig. 9 is an exemplary view illustrating intra prediction in a method for encoding an image according to an embodiment of the present invention.

[0074] Fig.10 is an exemplary view illustrating a prediction principle in a P slice or a B slice in a method for encoding an image according to an embodiment of the present invention.

[0075] Fig.11 It is shown in Fig.10 An exemplary view of a case where interpolation is performed in a method for encoding an image.

[0076] Fig.12 is a view illustrating a main process of a method for encoding an image according to an embodiment of the present invention as a syntax in a coding unit.

[0077] Fig.13 Is the diagram when through Fig.12 An exemplary view of an example of supporting symmetric type partitioning or asymmetric type partitioning as in inter-frame prediction when block matching in a current block generates a prediction block.

[0078] Fig.14 It is the positive image Fig. 9 As in the intra prediction of FIG. 1 , an exemplary view of examples of supporting 2N×2N and N×N in inter prediction is provided.

[0079] Fig.15 1 is a view illustrating a process of performing 1-D horizontal filtering on pixels at positions a, b, and c (assume x) of an image in a method for encoding an image according to an embodiment of the present invention.

[0080] Fig.16 is an exemplary view illustrating a current block and neighboring blocks that can be applied to a method for encoding an image according to an embodiment of the present invention.

[0081] Fig.17 is an exemplary view illustrating generating a prediction block through block matching in a current block in a method for encoding an image according to an embodiment of the present invention.

[0082] Fig.18 is an exemplary view illustrating a reference block used as a prediction block through at least one block matching in a coding area in a method for encoding an image according to an embodiment of the present invention.

[0083] Fig.19 is a block diagram illustrating an apparatus for encoding an image according to an embodiment of the present invention.

[0084] Fig. 20 is a block diagram illustrating an apparatus for decoding an image according to an embodiment of the present invention.

[0085] Figures 21a to 21d is an exemplary view illustrating transmission in block units of various sizes in a method for encoding an image according to an embodiment of the present invention.

[0086] Fig. 22 is an exemplary view illustrating transmission of a flag in a method for encoding an image according to an embodiment of the present invention.

[0087] Fig.23 is a flowchart illustrating a filtering process in a method for encoding an image and a method for decoding an image according to an embodiment of the present invention.

[0088] Fig.24 The diagram can be applied to Fig.23 An exemplary view of a prediction block or a transform block of a method for encoding an image.

[0089] Fig.25 It is shown in Fig.23 An exemplary view of setting a boundary of a filtering strength in a method for encoding an image.

[0090] Figures 26a to 26c It is shown in Fig.25 An exemplary view of a pattern of pixel values ​​within an image in a method for encoding an image.

[0091] Fig. 27 It is shown in Fig.23 Flowchart of a filter strength setting process in a method for encoding / decoding an image.

[0092] Fig.28 It is shown in Fig.23 A flowchart of another example of a filter strength setting process in a method for encoding an image.

[0093] Fig.29 It is shown in Fig.23 A flowchart of still another example of a filter strength setting process in a method for encoding / decoding an image.

[0094] Fig.30 It is shown in Fig.23 A flowchart of still another example of a filter strength setting process in a method for encoding / decoding an image.

[0095] Fig.31 It is shown in Fig.23 An exemplary view of a filter application determination process in a method for encoding an image.

[0096] Fig.32 It is shown in Fig.23 An exemplary view of another example of a filter application determination process in a method for encoding an image.

[0097] Fig.33 It is shown in Fig.23 An exemplary view of still another example of a filter application determination process in a method for encoding an image. DETAILED DESCRIPTION

[0098] Various modifications can be made to the present invention, and various embodiments of the present invention are possible, and examples of the present invention will now be provided and described in detail with reference to the accompanying drawings. However, the present invention is not limited thereto, and the exemplary embodiments can be interpreted as including all modifications, equivalents or substitutes within the technical concept and technical scope of the present invention.

[0099] The terms "first", "second", etc. used in the specification can be used to describe various components, but these components should not be interpreted as being limited to these terms. These terms are only used to distinguish one component from other components. For example, a "first" component can be renamed as a "second" component without departing from the scope of the present invention, and the "second" component can also be similarly renamed as a "first" component. The term "and / or" includes a combination of multiple items or any one of the multiple items.

[0100] It should be understood that in this specification, when an element is simply referred to as being “connected to” or “coupled to” another element rather than being “directly connected to” or “directly coupled to” another element, the element may be “directly connected to” or “directly coupled to” another element, or may be connected to or coupled to another element with other elements interposed therebetween. Conversely, it should be understood that when an element is referred to as being “directly coupled to” or “directly connected to” another element, there are no intervening elements.

[0101] The terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. Unless it is obvious that there is a different meaning in the context, expressions used in the singular form include expressions in the plural form. In this specification, it should be understood that terms such as "including", "having", etc. are intended to indicate the presence of features, quantities, steps, actions, elements, parts or combinations thereof disclosed in the specification, and are not intended to exclude the possibility that one or more other features, quantities, steps, actions, elements, parts or combinations thereof may exist or may be added.

[0102] Unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which the invention belongs. It should be understood that the terms defined in the dictionary are the same as those in the prior art context, and unless otherwise clearly indicated in the context of this specification, these terms should not be defined in an ideal or overly formal manner.

[0103] Typically, a video may consist of a series of pictures, and each picture may be divided into predetermined areas such as frames or blocks. In addition, the predetermined area, as a block, may be referred to as a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU) of different sizes. Each unit may be composed of a luminance block and two chrominance blocks, and may be constructed differently according to the color format. In addition, the size of the luminance block and the chrominance block may be determined according to the color format. For example, in the case of 4:2:0, the size of the chrominance block may be 1 / 2 of the width and length of the luminance block. For these units and terms, reference may be made to the existing HEVC (High Efficiency Video Coding) or H.264 / AVC (Advanced Video Coding).

[0104] In addition, the picture, block or pixel referenced to encode or decode the current block or current pixel is a reference picture, reference block or reference pixel. In addition, those skilled in the art will appreciate that the term "picture" used hereinafter may be replaced by another term having the same meaning, such as "image", "frame", etc.

[0105] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing the present invention, the same constituent elements in the drawings are represented by the same reference numerals to facilitate overall understanding of the present invention, and repeated description of the same elements is omitted.

[0106] Figure 1 is a view illustrating a system using an apparatus for encoding an image and / or an apparatus for decoding an image according to the present invention.

[0107] refer to Figure 1, a system using the apparatus for encoding an image and / or the apparatus for decoding an image may be a user terminal 11 (such as a personal computer (PC), a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a PlayStation portable (PSP), a wireless communication terminal, a smart phone, a television (TV), etc.) or a server terminal 12 (such as an application server, a service server, etc.). The system may be referred to as a computing device.

[0108] In addition, the computing device may include various devices, including a communication device such as a communication modem for communicating with various devices or a wired / wireless communication network, a memory 18 for storing various programs and data for inter-frame or intra-frame prediction of encoding or decoding images, and a processor 14 for executing programs to perform calculations and controls.

[0109] In addition, the computing device transmits the image encoded in the bit stream by using the device for encoding the image to the device for decoding the image in real time or non-real time through a wired / wireless communication network (such as the Internet, a wireless LAN, a WiBro network, and a mobile network) or through various communication interfaces (such as a cable, a universal serial bus (USB), etc.), so that the device for decoding the image decodes the image to reconstruct the image. In addition, the image encoded in the bit stream by the device for encoding the image can be transmitted to the device for decoding the image through a computer-readable recording medium.

[0110] Figure 2 is a block diagram illustrating an apparatus for encoding an image according to an embodiment of the present invention. Figure 3 is a block diagram illustrating an apparatus for decoding an image according to an embodiment of the present invention.

[0111] like Figure 2 As shown, the apparatus 20 for encoding an image according to an embodiment may include: a prediction unit 200, a subtractor 205, a transform unit 210, a quantization unit 215, an inverse quantization unit 220, an inverse transform unit 225, an adder 230, a filter unit 235, a decoded picture buffer (DPB) 240 and an entropy encoder 245. In addition, the apparatus 20 for encoding an image may further include a segmentation unit 190.

[0112] In addition, if Figure 3 As shown, the apparatus 30 for decoding an image according to an embodiment may include: an entropy decoder 305 , a prediction unit 310 , an inverse quantization unit 315 , an inverse transform unit 320 , an adder 325 , a filter unit 330 and a decoded picture buffer 335 .

[0113] The apparatus 20 for encoding an image and the apparatus 30 for decoding an image may be independent apparatuses, or may be provided as a single apparatus for encoding and decoding an image. In this case, the prediction unit 200, the inverse quantization unit 220, the inverse transform unit 225, the adder 230, the filter unit 235, and the decoded picture buffer 240 in the apparatus 20 for encoding an image are respectively the same technical elements as the prediction unit 310, the inverse quantization unit 315, the inverse transform unit 320, the adder 325, the filter unit 330, and the decoded picture buffer 335 in the apparatus 30 for decoding an image. The apparatus 20 for encoding an image and the apparatus 30 for decoding an image may be implemented to have at least the same structure or to perform at least the same function. In addition, when the entropy encoder 245 performs its function inversely, it is also the entropy decoder 305. Therefore, in the detailed description of the technical elements and their operating principles, the description of the technical elements corresponding to each other will be omitted.

[0114] The apparatus for decoding an image corresponds to a computing device that applies the method of encoding an image performed by the apparatus for encoding an image to decoding, and therefore in the following description, the description will focus on the apparatus for encoding an image.

[0115] The computing device may include a memory storing a program and a software module for implementing the method for encoding an image and / or the method for decoding an image, and a processor coupled to the memory to execute the program. A device for encoding an image may be referred to as an encoder, and a device for decoding an image may be referred to as a decoder.

[0116] Components of an apparatus for encoding an image according to an embodiment will be described in detail below.

[0117] The segmentation unit 190 segments the input image into blocks having a predetermined size (M×N). Here, M or N is an arbitrary natural number equal to or greater than 1.

[0118] Specifically, the segmentation unit 190 may be composed of a picture segmentation unit and a block segmentation unit. The size or shape of the block may be determined according to the characteristics of the image, the resolution, etc. The size or shape of the block supported by the picture segmentation unit may be an M×N square (256×256, 128×128, 64×64, 32×32, 16×16, 8×8, 4×4, etc.) or an M×N rectangle whose width and length are indicated as powers of 2. For example, in the case of an 8k UHD image with high resolution, the input image may be segmented in a size of 256×256. In the case of a 1080p HD image, the input image may be segmented in a size of 128×128, and in the case of a WVGA image, the input image may be segmented in a size of 16×16.

[0119] Information about the size or shape of a block may be set in units such as a sequence, a picture, a slice, etc., and the relevant information may be sent to a decoder. That is, the relevant information may be set in a sequence parameter set, a picture parameter set, a slice header, or a combination thereof.

[0120] Here, "sequence" refers to a unit consisting of multiple related scenes. Picture is a term used for a series of luminance (Y) components or luminance and chrominance (Y, Cb, and Cr) components in a scene or picture. Depending on the situation, the scope of a picture can be a frame or a field.

[0121] The term "slice" may refer to an independent slice segment and multiple dependent slice segments present in the same access unit. An access unit may represent a set of NAL (Network Abstraction Layer) units associated with a coded picture. The NAL unit is a syntactic structure in which a video compression bitstream is in a network-friendly H.264 / AVC and HEVC standard format. A slice unit is usually composed of one NAL unit, and in the system standard, the NAL or NAL set that constitutes a frame is usually considered to be an access unit.

[0122] Continuing with the description of the picture segmentation unit, information about the size (M×N) or shape of the block may be formed by an explicit flag. Specifically, when the block is square, the information about the size of the block may include information about a length, and when the block is rectangular, information about each length, or information about the difference between the width and the height. For example, when M and N are powers of k (assuming k is 2) (M=2 m ,N=2 n ), information about m and n can be encoded in various methods such as unary binarization, truncated unary binarization, etc., and the relevant information can be sent to the decoder.

[0123] In addition, the “minimum available partition size” (Minblksize) supported by the picture partition unit can provide a minimum partition size of I×J (for ease of explanation, assume that I=J, I=2 i ,J=2 j ), mi or nj. As another example, when M is different from N, the difference between m and n (|mn|) can be provided. Optionally, the "maximum available size of the partition" (Maxblksize) supported by the picture partitioning unit can provide information about I×J (for ease of explanation, assume I=J, I=2 i ,J=2 j ), im or nj information.

[0124] This implies that, for example, when the syntax of the relevant information exists but cannot be verified in the encoder / decoder, the encoder or decoder can follow the preset default setting. For example, when the relevant syntax cannot be verified in the case of identifying the information about the shape of the block, the shape of the block can be set to a square as a default setting. In addition, when identifying the information about the size of the block, specifically, in the case of identifying the information about the size of the block by the difference from the "minimum segmentation available size" (Minblksize), when the syntax related to the difference is verified but the syntax related to the "minimum segmentation available size" (Minblksize) cannot be verified, the information can be obtained from the default setting value related to the "minimum segmentation available size" (Minblksize).

[0125] As described above, in the picture segmentation unit, information about the size or shape of the block may be explicitly transmitted from the encoder and / or the decoder, or the size or shape of the block may be implicitly determined according to characteristics and resolution of the image.

[0126] As described above, the blocks divided and determined by the picture segmentation unit can be used as default coding units. In addition, the blocks divided and determined by the picture segmentation unit can be the smallest units constituting the upper units such as pictures, strips, tiles, etc., or can be the largest units of coding blocks, prediction blocks, transform blocks, quantization blocks, entropy blocks, loop filter blocks, etc. However, some blocks are not limited to this, and exceptions are possible. For example, some blocks (such as loop filter blocks) can be applied as units larger than the size of the above blocks.

[0127] The block segmentation unit segments the coding, prediction, transform, quantization, entropy, and loop filter blocks. The segmentation unit 190 may be included in each component to perform the function. For example, the transform unit 210 may include a transform unit segmentation unit, and the quantization unit 215 may include a quantization block segmentation unit. The size or shape of the initial block of the block segmentation unit may be determined based on the segmentation result of the previous stage or the upper block.

[0128] For example, in the case of the coding block, a block obtained by the picture segmentation unit of the previous stage may be set as the initial block. Alternatively, in the case of the prediction block, a block obtained by segmenting the coding block of the previous level as the prediction block may be set as the initial block. Alternatively, in the case of the transform block, a block obtained by segmenting the coding block of the previous level as the transform block may be set as the initial block.

[0129] The conditions for determining the size or shape of the initial block are not always fixed, and some of the conditions may change or exceptions may exist. In addition, the segmentation operation (segmentation possibility, segmentation available block shape, etc.) of the current level may be affected by a combination of at least one factor in the segmentation state of the previous stage or upper block (e.g., the size of the coding block, the shape of the coding block, etc.) and the setting conditions of the current level (e.g., the size of the transform block, the shape of the supported transform block, etc.).

[0130] The block segmentation unit can support a quadtree-based segmentation method. That is, the pre-segmented block can be segmented into four blocks with 1 / 2 width and 1 / 2 length. The segmentation can be repeated based on the initial block (dep_0) until the "segmentation available depth limit" (dep_k, k is the number of times the segmentation is available, and the block size of the "segmentation available depth limit" (dep_k) satisfies M>>k, N>>k).

[0131] In addition, a binary tree-based segmentation method can be supported. The pre-segmented block can be segmented into two blocks with 1 / 2 width or 1 / 2 length. The segmentation based on the quadtree and the segmentation based on the binary tree can be symmetrical segmentation or asymmetrical segmentation. Based on the setting of the encoder / decoder, it can be determined which segmentation method to use. The symmetrical segmentation method will be mainly described in the method for encoding an image of the present invention.

[0132] The split flag (div_flag) may indicate whether splitting is performed on each block. When the value is 1, splitting is performed, and when the value is 0, splitting is not performed. Alternatively, when the value is 1, splitting is performed and additional splitting is possible, and when the value is 0, splitting is not performed and additional splitting is not allowed. Depending on conditions such as "minimum split available size", "split available depth limit", etc., this flag may only consider whether to perform splitting, and may not consider whether to perform additional splitting.

[0133] The split flag can be used for quadtree-based splitting and binary tree-based splitting. In binary tree-based splitting, the split direction can be determined based on split depth, coding mode, prediction mode, size, shape, block type (one of coding, prediction, transformation, quantization, entropy, loop filtering, etc. or one of brightness and chrominance), slice type, "split available depth limit", "minimum / maximum split available size" or a combination thereof. In addition, according to the split flag and / or split direction, only the width of the block can be split into 1 / 2 or only its length can be split into 1 / 2.

[0134] For example, assuming that when the block is M×N (M>N) and M is greater than N, horizontal segmentation is supported, and assuming that the "current segmentation depth" (dep_curr) is less than the "segmentation available depth limit" so that additional segmentation is possible, 1 bit is allocated to the segmentation flag. When the value is 1, horizontal segmentation is performed, and when the value is 0, no additional segmentation is performed. One segmentation depth may be used in quadtree-based segmentation and binary tree-based segmentation, or the segmentation depth may be used in each of quadtree-based segmentation and binary tree-based segmentation. In addition, one "segmentation available depth limit" may be used in quadtree-based segmentation and binary tree-based segmentation, or the "segmentation available depth limit" may be used in each of quadtree-based segmentation and binary tree-based segmentation.

[0135] As another example, when the block is M×N (M>N), and N is equal to the preset "minimum segmentation available size" so that horizontal segmentation is not supported, 1 bit is allocated to the segmentation flag. When the value is 1, vertical segmentation is performed, and when the value is 0, segmentation is not performed.

[0136] In addition, a flag (div_h_flag, div_v_flag) for horizontal segmentation or vertical segmentation may be supported, and binary segmentation may be supported based on the flag. The horizontal segmentation flag (div_h_flag) or the vertical segmentation flag (div_v_flag) may indicate whether horizontal or vertical segmentation is performed on each block. When the horizontal segmentation flag (div_h_flag) or the vertical segmentation flag (div_v_flag) has a value of 1, horizontal or vertical segmentation is performed, and when the value is 0, horizontal or vertical segmentation is not performed.

[0137] In addition, when the flag has a value of 1, the horizontal or vertical segmentation is performed, and an additional horizontal or vertical segmentation may be performed. When the value is 0, the horizontal or vertical segmentation is not performed, and an additional horizontal or vertical segmentation may not be performed. According to conditions such as "minimum segmentation available size", "segmentation available depth limit", etc., the flag may only consider whether to perform segmentation, and may not consider whether to perform additional segmentation.

[0138] In addition, a flag (div_flag / h_v_flag) for horizontal segmentation or vertical segmentation can be supported, and binary segmentation can be supported based on the flag. The segmentation flag (div_flag) can indicate whether horizontal or vertical segmentation is performed on each block, and the segmentation direction flag (h_v_flag) can indicate horizontal or vertical segmentation.

[0139] When the segmentation flag (div_flag) has a value of 1, segmentation is performed, and segmentation in the horizontal or vertical direction is performed based on the segmentation direction flag (h_v_flag). When the value is 0, segmentation in the horizontal or vertical direction is not performed. In addition, when the value is 1, segmentation in the horizontal or vertical direction is performed based on the segmentation direction flag (h_v_flag), and additional segmentation in the horizontal or vertical direction may be performed. When the value is 0, segmentation in the horizontal or vertical direction is not performed, and additional segmentation in the horizontal or vertical direction may not be performed.

[0140] Depending on conditions such as "minimum split available size", "split available depth limit", etc., this flag may only consider whether to perform a split, or may not consider whether to perform an additional split.

[0141] A segmentation flag for each of the segmentation in the horizontal and vertical directions may be supported, and based on the flag, a binary tree based segmentation may be supported. In addition, as described above, when the segmentation direction is predetermined, only one of the two segmentation flags may be used or both segmentation flags may be used.

[0142] For example, when all flags are indicated as available, the shape of the block can be divided into one of M×N, M / 2×N, M×N / 2, and M / 2×N / 2. In this case, these flags can be encoded as 00, 10, 01, and 11 in the order of horizontal division flag or vertical division flag (div_h_flag / div_v_flag).

[0143] The above case is an example of a setting in which the segmentation flag can be used in an overlapping manner, but a setting in which the segmentation flag cannot be used in an overlapping manner is also possible. For example, the shape of the segmentation block can be segmented into M×N, M / 2×N, and M×N / 2. In this case, the flag can be encoded as 0, 10, and 11 in the order of the segmentation flag (div_flag) and the "horizontal vertical flag" (h_v_flag, which is a flag indicating whether the segmentation direction is horizontal or vertical). Here, overlapping can mean that the segmentation in the horizontal direction and the segmentation in the vertical direction are performed simultaneously.

[0144] Depending on the settings of the encoder and / or decoder, quadtree-based segmentation and binary tree-based segmentation can be used alone or in combination. For example, quadtree-based or binary tree-based segmentation can be determined based on the size or shape of the block. That is, when the shape of the block is M×N and M is greater than N, horizontal segmentation is performed. When the shape of the block is M×N and N is greater than M, segmentation based on the vertical direction can support binary tree-based segmentation. When the shape of the block is M×N and N is equal to M, quadtree-based segmentation can be supported.

[0145] As another example, when the block size (M×M) is equal to or greater than the "block partition boundary value" (thrblksize), binary tree-based partitioning may be supported. When the block size (M×M) is less than the "block partition boundary value", quadtree-based partitioning may be supported.

[0146] As another example, when M or N of the block (M×N) is equal to or less than the “first maximum available partition size” (Maxblksize1) and equal to or greater than the “first minimum available partition size” (Minblksize1), quadtree-based partitioning may be supported. When M or N of the block (M×N) is equal to or less than the “second maximum available partition size” (Maxblksize2) and equal to or greater than the “second minimum available partition size” (Minblksize2), binary tree-based partitioning may be supported.

[0147] When the first segmentation support range and the second segmentation support range, which can be defined by the "maximum segmentation available size" and the "minimum segmentation available size", overlap each other, the first or second segmentation method can be given priority according to the setting of the encoder / decoder. In an embodiment, the first segmentation method may be a quadtree-based segmentation, and the second segmentation method may be a binary tree-based segmentation. For example, when the "first minimum segmentation available size" (Minblksize1) is 16 and the "second maximum segmentation available size" (Maxblksize2) is 64 and the pre-segmentation block is 64×64, it falls within the first segmentation support range and the second segmentation support range, and therefore quadtree-based segmentation or binary tree-based segmentation can be used.

[0148] According to the presetting, in the case of giving priority to the first segmentation method (quadtree-based segmentation in the embodiment), when the segmentation flag (div_flag) has a value of 1, the quadtree-based segmentation can be performed, and the additional quadtree-based segmentation can be performed. When the segmentation flag has a value of 0, the quadtree-based segmentation may not be performed, and the additional quadtree-based segmentation may not be performed. Optionally, when the segmentation flag (div_flag) has a value of 1, the quadtree-based segmentation may be performed, and the additional quadtree-based or binary-based segmentation may be performed. When the segmentation flag has a value of 0, the quadtree-based segmentation may not be performed, and the additional quadtree-based segmentation may not be performed, but the binary-based segmentation may be performed. According to conditions such as "minimum segmentation available size", "segmentation available depth limit", etc., the flag may only consider whether to perform the segmentation, and may not consider whether to perform the additional segmentation.

[0149] When the split flag (div_flag) has a value of 1, splitting is performed to obtain four 32×32-sized blocks. The block is larger than the "first minimum split available size" (Minblksize1), and therefore additional quadtree-based splitting can be performed. When the split flag has a value of 0, additional quadtree-based splitting may not be performed. The size of the current block (64×64) is within the second split support range, and therefore binary tree-based splitting may be performed. When the split flag (in the order of div_flag / h_v_flag) has a value of 0, splitting may no longer be performed. When the split flag has a value of 10 or 11, horizontal splitting or vertical splitting may be performed.

[0150] When the pre-partitioned block is 32×32 and no additional quadtree-based partitioning is performed because the value of the partition flag (div_flag) is 0, and the "second maximum partition available size" (Maxblksize2) is 16, the size of the current block (32×32) may fall within the second partition support range, and thus no additional partitioning may be performed. In the above description, the priority of the partitioning method may be determined according to at least one factor or a combination of factors among a slice type, a coding mode, a luminance / chrominance component, etc.

[0151] As another example, various settings can be supported based on brightness and chrominance components. For example, a quadtree or binary tree segmentation structure determined in the brightness component can be used for the chrominance component without encoding / decoding additional information. Alternatively, when independent segmentation of brightness and chrominance components is supported, quadtrees and binary trees can be supported in the brightness component, and quadtree-based segmentation can be supported in the chrominance component.

[0152] In addition, in the luminance and chrominance components, quadtree-based segmentation and binary tree-based segmentation can be supported, but the segmentation support ranges thereof may be the same or different for the luminance and chrominance components, or may be proportional or not proportional to the luminance and chrominance components. For example, when the color format is 4:2:0, the segmentation support range of the chrominance component may be N / 2 of the segmentation support range of the luminance component.

[0153] As another example, the settings may be different depending on the slice type. For example, in I slices, quadtree-based partitioning may be supported, and in P slices, binary tree-based partitioning may be supported, and in B slices, both quadtree-based partitioning and binary tree-based partitioning may be supported.

[0154] As described above, quadtree-based segmentation and binary tree-based segmentation can be set and supported based on various conditions. These examples are not limited to this, and may include a case where the condition is switched, one or more factors mentioned in the example, or a combination thereof, and the example may be modified. The "segmentation available depth limit" may be determined based on at least one factor or a combination thereof in the segmentation method (quadtree, binary tree), slice type, luminance / chrominance component, coding mode, etc.

[0155] In addition, the segmentation support range may be determined according to at least one factor or a combination thereof of a segmentation method (quadtree, binary tree), a slice type, a luminance / chrominance component, a coding mode, etc. The related information may indicate the maximum and minimum values ​​of the segmentation support range. When the information is configured as an explicit flag, the maximum / minimum value information of the length or the information about the difference between the minimum and maximum values ​​of the length may be indicated.

[0156] For example, when the maximum value and the minimum value are power operations of k (assuming k is 2), the exponential information of the maximum value and the minimum value can be encoded by various binarizations to be provided to the decoding device. Alternatively, the difference between the exponents of the maximum value and the minimum value can be provided. Here, the information sent can be the exponential information of the minimum value and the information about the difference in exponents.

[0157] According to the above description, information related to the flag can be generated and sent in units such as sequence, picture, slice, tile, block, etc.

[0158] The above-mentioned segmentation flag can indicate the block segmentation information through a quadtree or a binary tree or a combination thereof. The segmentation flag can be encoded in various methods such as unary binarization, truncated unary binarization, etc., so that relevant information can be provided to a decoding device. At least one scanning method can be performed on the bit stream structure of the segmentation flag to indicate the segmentation information of the block.

[0159] For example, the bitstream of the segmentation flag may be configured based on the segmentation depth order (from dep0 to dep_k), or the bitstream of the segmentation flag may be configured based on whether the segmentation is performed. In the method based on the segmentation depth order, the segmentation information of the depth of the current level is obtained based on the initial block, and then the segmentation information of the depth of the next level is obtained. In the method based on whether the segmentation is performed, the segmentation information of the blocks that are segmented based on the initial block is obtained preferentially. Other scanning methods not disclosed in the above examples may be selected.

[0160] In addition, according to an embodiment, the block segmentation unit may generate and indicate index information of a predefined block candidate group instead of the above-mentioned segmentation flag. The shape of the block candidate group may include, for example, M×N, M / 2×N, M×N / 2, M / 4×N, 3M / 4×N, M×N / 4, M×3N / 4, M / 2×N / 2, etc., which are segmentation block shapes that the pre-segmented blocks may have.

[0161] As described above, when the candidate group of the partition block is determined, the index information about the partition block shape may be encoded by various methods such as fixed length binarization, truncated unary binarization, truncated binarization, etc. Just like the above-mentioned partition flag, the candidate group of the partition block may be determined based on at least one factor or a combination of factors among the partition depth, coding mode, prediction mode, size, shape, type of block, slice type, "partition available depth limit", maximum / minimum partition available size, etc. of the block.

[0162] For the following explanation, it is assumed that (M×N, M×N / 2) is candidate list 1 (list1), (M×N, M / 2×N, M×N / 2, M / 2×N / 2) is candidate list 2 (list2), (M×N, M / 2×N, M×N / 2) is candidate list 3 (list3), and (M×N, M / 2×N, M×N / 2, M / 4×N, 3M / 4×N, M×N / 4, M×3N / 4, M / 2×N / 2) is candidate list 4 (list4). For example, in the description based on M×N, in the case of (M=N), the partition block candidates of candidate list 2 can be supported, and in the case of (M≠N), the partition block candidates of candidate list 3 can be supported.

[0163] As another example, when M or N in M×N is equal to or greater than the boundary value (blk_th), the partition block candidates of candidate list 2 may be supported. When M or N in M×N is less than the boundary value, the partition block candidates of candidate list 4 may be supported. In addition, when M or N is equal to or greater than the first boundary value (blk_th_1), the partition block candidates of candidate list 1 may be supported. When M or N is less than the first boundary value (blk_th_1) but equal to or greater than the second boundary value (blk_th_2), the partition block candidates of candidate list 2 may be supported. When M or N is less than the second boundary value (blk_th_2), the partition block candidates of candidate list 4 may be supported.

[0164] As another example, when the coding mode is intra prediction, the partition block candidates of candidate list 2 may be supported. When the coding mode is inter prediction, the partition block candidates of candidate list 4 may be supported.

[0165] Even if the same partition block candidates are supported, the bit configuration according to the binarization in each block may be the same or different. For example, as applied in the partition flag, when the supported partition block candidates are restricted according to the size or shape of the block, the bit configuration according to the binarization of the relevant block candidates may be different. For example, in the case of (M>N), block shapes resulting from horizontal partitioning, namely M×N, M×N / 2, and M / 2×N / 2, may be supported. The binary bits of the index of M×N / 2 according to the partition block candidate group (M×N, M / 2×N, M×N / 2, and M / 2×N / 2) may be different from the binary bits of the index of M×N / 2 according to the current condition.

[0166] Depending on the type of block (e.g., coding, prediction, transform, quantization, entropy, loop filter block), information about the partitioning and shape of the block can be indicated by using one of the partitioning flags or the partitioning index. In addition, depending on the type of block, the block size limit for partitioning, the shape of the block, and the "partition available depth limit" can be different.

[0167] The encoding and decoding process in the block unit includes encoding block determination, prediction block determination, transform block determination, quantization block determination, entropy block determination, loop filter determination, etc. The order of the encoding and decoding process is not always fixed, and some sequences may be changed or excluded. Based on the encoding cost of each candidate according to the size and shape of the block, the size and shape of each block are determined, and the image data of the determined block and the information related to the segmentation about the size and shape of the determined block may be encoded.

[0168] The prediction unit 200 can be implemented by using a prediction module as a software module, and a prediction block of a coding target block can be generated by an intra prediction method or an inter prediction method. Here, the prediction block almost matches the coding target block in terms of pixel difference, and the pixel difference can be determined by various methods including SAD (sum of absolute difference) and SSD (sum of square difference). In addition, various syntaxes that can be used to decode image blocks can be generated here. The prediction block can be classified into intra blocks and inter blocks according to the coding mode.

[0169] Intra prediction is a prediction technique using spatial correlation, which is a prediction method that predicts the current block by using reference pixels of a reconstructed block previously encoded and decoded in the current picture. That is, the luminance value reconstructed by intra prediction and reconfiguration can be used as a reference pixel by the encoder and decoder. Intra prediction can be effective for flat areas with continuity and flat areas with constant directionality, and can use spatial correlation to ensure random access and avoid error diffusion.

[0170] In inter-frame prediction, a compression technique that removes data redundancy by using time correlation with reference to an image encoded in at least one past and future picture is used. That is, in inter-frame prediction, a prediction signal with high similarity can be generated by referring to at least one past and future picture. The encoder using inter-frame prediction can search for a block with high correlation with the current block to be encoded in the reference picture, and can send the position information and residual signal of the selected block to the decoder. The decoder can generate the same prediction block as the encoder by using the selection information of the transmitted image, and can configure the reconstructed image by compensating the transmitted residual signal.

[0171] Figure 4 is an exemplary view illustrating inter prediction of a P slice in a method for encoding and decoding an image according to an embodiment of the present invention. Figure 5 is an exemplary view illustrating inter prediction of a B slice in a method for encoding and decoding an image according to an embodiment of the present invention.

[0172] In the method for encoding an image of an embodiment, since the prediction block is generated from a previously encoded encoded picture with high temporal correlation, inter-frame prediction can improve encoding efficiency. Current (t) may represent the current picture to be encoded, and based on the temporal flow of the picture or POC (picture order count), may include a first reference picture having a first temporal distance (t-1) before the POC of the current picture and a second reference picture having a second temporal distance (t-2) before the first temporal distance.

[0173] That is, Figure 4 As shown, in the inter-frame prediction that can be applied to the method for encoding an image according to the embodiment, by block matching between the current block of the current picture (current(t)) and the reference block of the reference picture (t-1, t-2), motion estimation for searching the optimal prediction block from the previously encoded reference picture (t-1, t-2) can be performed. For accurate estimation, after performing interpolation processing based on a structure in which at least one sub-pixel is arranged between two adjacent pixels, the optimal prediction block is obtained, and then motion compensation is performed to obtain the final prediction block.

[0174] In addition, if Figure 5 As shown, in the inter-frame prediction that can be applied to the method for encoding an image according to an embodiment, a prediction block can be generated from a reference picture (t-1, t+1) encoded bidirectionally existing in time based on a current picture (current(t)). In addition, two prediction blocks can be generated from at least one reference picture.

[0175] When an image is encoded by inter-frame prediction, information about a motion vector of an optimal prediction block and information about a reference picture may be encoded. In an embodiment, when a prediction block is generated unidirectionally or bidirectionally, a reference picture list is constructed differently, and a prediction block may be generated from the reference picture list. Basically, a reference picture that temporally exists before a current picture may be allocated to list 0 (L0) for management, and a reference picture that exists after the current picture may be allocated to list 1 (L1) for management.

[0176] When constructing reference picture list 0, when reference picture list 0 is not padded to the allowed number of reference pictures, a reference picture existing after the current picture may be allocated. Similarly, when constructing reference picture list 1, when reference picture list 1 is not padded to the allowed number of reference pictures, a reference picture existing before the current picture may be allocated.

[0177] Figure 6 is an exemplary view illustrating a case in which a prediction block is generated in a single direction in a method for encoding and decoding an image according to an embodiment of the present invention.

[0178] refer to Figure 6 In the method for encoding and decoding an image according to an embodiment, just like the conventional method, the prediction block can be searched from the previously encoded reference picture (t-1, t-2), and the prediction block can also be searched from the already encoded area in the current picture (current(t)).

[0179] That is, in the method for encoding and decoding an image according to an embodiment, a prediction block can be generated from a previously encoded picture (t-1, t-2) having a high temporal correlation and a prediction block having a high spatial correlation can be searched. Searching for a prediction block having a high spatial correlation may correspond to searching for a prediction block in an intra-frame prediction manner. In order to perform block matching from an already encoded area in a current picture, the method for encoding an image according to an embodiment may configure a syntax for information about prediction candidates in an intra-frame prediction mode.

[0180] For example, when n (n is an arbitrary natural number) intra prediction modes are supported, one mode is added to the intra prediction candidate group to support n+1 modes. M-1 ≤n+1<2 M The M fixed bits of can be used to encode the prediction mode. In addition, just like the MPM (most probable mode) of HEVC, a prediction mode with a high probability can be selected from the candidate group. In addition, the prediction mode can be preferentially encoded in the previous stage of encoding the prediction mode.

[0181] When a prediction block is generated by block matching in the current picture, the method for encoding an image according to an embodiment may configure the syntax of information about the inter-frame prediction mode. Information about motion or displacement may be used as additional information about the prediction mode. Information about motion or displacement may include information about the best candidate of multiple vector candidates, the difference between the best candidate vector and the actual vector, information about the reference direction, a reference picture, etc.

[0182] Figure 7 is an exemplary view illustrating a reference picture list in a method for encoding and decoding an image according to an embodiment of the present invention. Figure 8 is an exemplary view illustrating another example of performing inter prediction from a reference picture list in a method for encoding and decoding an image according to an embodiment of the present invention.

[0183] refer to Figure 7 In a method for encoding an image according to an embodiment, inter-frame prediction can be performed on a current block of a current picture (current(t)) from a first reference picture list (reference list 0, L0) and a second reference picture list (reference list 1, L1).

[0184] refer to Figure 7 and Figure 8 , reference picture list 0 may include reference pictures before the current picture (t). Reference pictures having a first temporal distance (t-1) and a second temporal distance (t-2) before the POC of the current picture (t) may be indicated as t-1 and t-2. In addition, reference picture list 1 may include reference pictures after the current picture (t). Reference pictures having a first temporal distance (t+1) and a second temporal distance (t+2) after the POC of the current picture (t) may be indicated as t+1 and t+2.

[0185] In the above example of the reference picture list configuration, the reference picture list is configured to have reference pictures with a difference of 1 in the temporal distance (based on the POC in the embodiment), but the difference in the temporal distance between the reference pictures may be configured differently. That is, the index difference between the reference pictures and the temporal distance difference between the reference pictures may not be proportional. In addition, the list configuration order may not be based on the temporal distance. This will be described in the example of the reference picture list configuration described later.

[0186] Depending on the slice type (I, P or B), prediction can be performed from the reference pictures in the list. When the prediction block is generated by block matching in the current picture (current(t)), encoding can be performed in an inter-prediction manner by adding the current picture to the reference picture list (reference list 0 and / or reference list 1).

[0187] like Figure 8As shown, the current picture (t) may be added to the reference picture list 0 (reference list 0), or the current picture (t) may be added to the reference picture list 1 (reference list 1). That is, the reference picture list 0 may be configured by adding the reference picture of the time distance (t) to the reference picture before the current picture (t), and the reference picture list 1 may be configured by adding the reference picture of the time distance (t) to the reference picture after the current picture (t).

[0188] For example, when configuring reference picture list 0, a reference picture before the current picture may be allocated to reference picture list 0, and then the current picture (t) may be allocated. When configuring reference picture list 1, a reference picture after the current picture may be allocated to reference picture list 1, and then the current picture (t) may be allocated. Alternatively, when configuring reference picture list 0, the current picture (t) may be allocated, and then the reference picture before the current picture may be allocated. When configuring reference picture list 1, the current picture (t) may be allocated, and then the reference picture after the current picture may be allocated.

[0189] In addition, when the reference picture list 0 is configured, the reference picture before the current picture may be allocated, and then the reference picture after the current picture and the current picture (t) may be allocated. Similarly, when the reference picture list 1 is configured, the reference picture after the current picture may be allocated, and then the reference picture before the current picture and the current picture (t) may be allocated. These examples are not limited thereto, and may include a case where a condition is switched, or may be modified.

[0190] Whether the current picture is included in each reference picture list (for example, not added to any list, or added only to list 0, or added only to list 1, or added to lists 0 and 1) can be set equally in the encoder / decoder, and the information thereon can be transmitted in units of sequence, picture, slice, etc. The information thereon can be encoded by methods such as fixed-length binarization, truncated unary binarization, truncated binarization, etc.

[0191] Not like Figure 7 In the method for encoding and decoding an image according to an embodiment, a prediction block is selected by performing block matching in a current picture (t), a reference picture list including information about the prediction block is configured, and the reference picture list is used for encoding and decoding the image.

[0192] When configuring the reference picture list, the settings of each list configuration order and rule and the allowed number of reference pictures for each list may be different and may be determined based on whether the current picture is included in the list (whether the current picture is included as a reference picture in inter-frame prediction), slice type, list reconfiguration parameters (which may be applied to lists 0 and 1 separately, or may be applied to lists 0 and 1 together), position within GOP (picture group), temporal layer information (temporal id) or a combination thereof. Explicitly, the relevant information may be sent in units such as sequences and pictures.

[0193] For example, in the case of a P slice, regardless of whether the current picture is included in the list, the reference picture list 0 may follow the list configuration rule A. In the case of a B slice, the reference picture list 0 including the current picture may follow the list configuration rule B, and the reference picture list 1 may follow the list configuration rule C. The reference picture list 0 not including the current picture may follow the list configuration rule D, and the reference picture list 1 may follow the list configuration rule E. In the list configuration rule, B may be equal to D, and C may be equal to E.

[0194] The list configuration rule may be the same as the example of the reference picture list configuration, or may be changed. As another example, when the current picture is included in the list, the allowed number of the first reference picture may be set. Otherwise, the allowed number of the second reference picture may be set. The allowed number of the first reference picture may be the same as or different from the allowed number of the second reference picture. The default setting may be that the difference between the allowed number of the first reference picture and the allowed number of the second reference picture is 1.

[0195] As another example, when the current picture is included in the list and the list reconfiguration parameter is applied, in slice A, all reference pictures may be included in the list reconfiguration candidate group, and in slice B, some reference pictures may be included in the list reconfiguration candidate group. Here, slice A or B may be distinguished based on whether the current picture is included in the list, temporal layer information, slice type, and position within the GOP. Whether to be included in the candidate group may be determined based on the POC or reference picture index of the reference picture, the reference prediction direction (before / after the current picture), and the current picture.

[0196] According to the above configuration, a reference block encoded in inter prediction is used for a current picture, and thus inter prediction is allowed or used in motion prediction of an I slice.

[0197] In addition, when configuring the reference picture list, the index assignment or list configuration order may be different according to the slice type. In the case of an I slice, as an example of reference picture list configuration, a lower index (e.g., idx=0, 1, 2) is used by increasing the priority in the current picture (current(t)). By binarization (fixed length binarization, truncated unary binarization, truncated binarization, etc.) in which the allowed number (C) of reference pictures of the reference picture list is the maximum value, the number of bits used to encode the image can be reduced.

[0198] In addition, in the case of a P or B slice, block matching is performed on the current picture. The possibility of selecting the reference picture of the current block as a prediction candidate is lower than the possibility of selecting another reference picture as a prediction candidate, and the number of bits used to encode the image can be reduced by various binarization methods in which the priority of block matching of the current picture is set low to use a higher index (e.g., idx=C, C-1) and the allowed number of reference pictures of the reference picture list is the maximum value.

[0199] Setting the priority of the current picture may be the same as the example of the reference picture list configuration, or may be configured in a modified manner. In addition, according to the slice type (e.g., I slice), the reference picture list is not configured so that information about the reference picture can be omitted. For example, the prediction block is generated by the existing inter-frame prediction, but in addition to the reference picture information, the inter-frame prediction information can be indicated using the motion information of the inter-frame prediction mode.

[0200] Depending on the slice type, the method for performing block matching in the current picture may be supported or not supported. For example, it is possible to set the block matching in the current block to support I slices but not P slices or B slices, but it is not limited to this. In addition, the method for supporting block matching in the current picture can be determined in units such as pictures, slices, tiles, etc., or determined by the position within the GOP, time layer information (time ID), etc. The setting information can be sent in the process of encoding the image, or can be sent by the encoder to the decoder in units such as sequences, pictures, slices, etc.

[0201] In addition, even if setting information or syntax exists in an upper unit and an operation related to the setting is in an on state, when the same setting information or syntax exists in a lower-level unit, the setting information in the lower-level unit has a higher priority than the setting information in the upper unit. For example, when the same or similar setting information is processed in a sequence, a picture, and a slice, the picture unit rather than the sequence unit may have priority, and the slice unit rather than the picture unit may have priority. For example, in a sequence parameter, a flag for supporting block matching in the current picture may be sps_curr_pic_BM_enabled_flag. In a picture parameter, a flag for supporting block matching in the current picture may be pps_curr_pic_BM_enabled_flag. When sps_curr_pic_BM_enabled_flag is on and pps_curr_pic_BM_enabled_flag is off, block matching in the current picture may not be supported. Based on the flag, it may be determined whether the current picture is included in reference picture list 0, and it may be determined whether the current picture is included in reference picture list 1.

[0202] Fig. 9 is an exemplary view illustrating intra prediction in a method for encoding an image according to an embodiment of the present invention.

[0203] refer to Fig. 9 , the intra prediction method according to the embodiment may include a series of reference pixel filling (reference sample filling), reference pixel filtering (reference sample filtering), intra prediction and boundary filtering.

[0204] The reference pixel filling step may be an example of the reference pixel configuration step, and the reference pixel filtering may be performed by a reference pixel filter unit. In intra prediction, a prediction block generation step and a prediction mode encoding step may be included, and boundary filtering may be an example of a post filtering step.

[0205] That is, in the intra-frame prediction performed in the method for encoding an image according to the embodiment, a reference pixel configuration step, a reference pixel filtering step, a prediction block generation step, a prediction mode encoding step and a post-filtering step may be included. Depending on various factors, such as the size of the block, the shape of the block, the position of the block, the prediction mode, the prediction method, the quantization parameter, etc., one or some of the above processes may be omitted, another process may be added, or the above order may be changed.

[0206] The reference pixel configuration step, the reference pixel filtering step, the prediction block generation step, the prediction mode encoding step, and the post-filtering step may be performed by a processor coupled to a memory in which the software module is stored. Therefore, hereinafter, for ease of explanation, as functional parts and / or components for performing the functions of the functional parts generated by a combination of the software modules implementing these steps and their processors, the reference pixel configuration unit, the reference pixel filter unit, the prediction block generation unit, the prediction mode encoding unit, and the post-filter unit may be referred to as executors of the respective steps.

[0207] Describing each component in detail, the reference pixel configuration unit configures the reference pixels to be used in the prediction of the current block by reference pixel padding. When the reference pixel does not exist or is unavailable, the reference pixel padding can be used for the reference pixel by copying the value from the adjacent pixel. The reconstructed picture buffer or the decoded picture buffer (DPB) can be used to copy the value.

[0208] That is, in intra prediction, prediction is performed by using reference pixels of a block encoded before the current picture. To this end, in the reference pixel configuration step, neighboring pixels such as neighboring blocks of the current block (i.e., left, upper left, lower left, upper, upper right blocks) can be used as reference pixels.

[0209] However, the candidate group of neighboring blocks for reference pixels is only an example of the result of raster scanning or z-scanning as the coding order of the blocks. When the inverse Z-scan is used as the coding order scanning method, in addition to the above-mentioned blocks, neighboring pixels such as the right, lower right, lower blocks, etc. can be used as reference pixels.

[0210] Furthermore, according to an embodiment, additional pixels other than neighboring pixels may be used in replacement or in combination with existing reference pixels according to phase configuration in intra prediction.

[0211] In addition, when prediction is performed in an angle mode in an intra prediction mode, a reference pixel decimal unit may be generated using reference pixels of integer units by linear interpolation. The mode in which prediction is performed by reference pixels present at integer unit positions may include some modes having vertical, horizontal, 45 degree angle, and 135 degree angle modes. For the above prediction modes, the process of generating reference pixels of decimal units may not be required.

[0212] In a prediction mode having a different angle other than the prediction mode, the interpolated reference pixel may have an interpolation accuracy of a power of 1 / 2 such as 1 / 2, 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64, or may have an accuracy of a multiple of 1 / 2.

[0213] The interpolation precision may be determined according to the number of supported prediction modes, the prediction direction of the prediction mode, etc. Fixed interpolation precision may be supported in a picture, a slice, a tile, a block, etc., or adaptive interpolation precision may be supported according to the size of the block, the shape of the block, the prediction direction of the supported mode, etc. Here, based on a specific line (e.g., the positive x-axis on the coordinate plane), the prediction direction of the mode may be indicated as tilt information or angle information of the direction indicated by the mode.

[0214] As an interpolation method, linear interpolation can be performed by adjacent integer pixels, but other interpolation methods can also be supported. For interpolation, at least one filter type and at least one number of taps can be supported, such as a 6-tap Wiener filter, an 8-tap Kalman filter, etc. According to the size of the block, the prediction direction for performing interpolation can be determined. In addition, relevant information can be sent through units such as sequences, pictures, strips, blocks, etc.

[0215] The reference pixel filter unit can perform filtering on the reference pixels to improve prediction efficiency by reducing residual degradation in the encoding process after configuring the reference pixels. The reference pixel filter unit can implicitly or explicitly determine the type of filter and whether to apply filtering based on the size of the block, the shape of the block, and the prediction mode. That is, even for filters with the same taps, the filter coefficients can be determined differently depending on the type of filter. For example, 3-tap filters such as [1,2,1] / 4 and [1,6,1] / 8 can be used.

[0216] In addition, the reference pixel filter unit can determine whether to send additional bits so that it can determine whether to apply filtering. For example, implicitly, the reference pixel filter unit can determine whether to apply filtering based on the characteristics of pixels in neighboring reference blocks (variance, standard deviation, etc.).

[0217] In addition, the reference pixel filter unit can determine whether to apply filtering when the relevant flag meets the preset concealment conditions of the residual coefficient, the intra prediction mode, etc. For example, the number of taps of the filter can be set to 3 taps such as [1,2,1] / 4 in a small block (blk), and can be set to 5 taps such as [2,3,6,3,2] / 16 in a large block (blk). The number of applications can be determined based on whether filtering is performed, whether filtering is performed once, whether filtering is performed twice, etc.

[0218] In addition, the reference pixel filter unit may apply filtering to reference pixels immediately adjacent to the current block by default. In addition to the immediately adjacent reference pixels, additional reference pixels may also be considered in the filtering process. For example, filtering may be applied to additional reference pixels by replacing the immediately adjacent reference pixels, or filtering may be applied to a combination of immediately adjacent reference pixels and additional reference pixels.

[0219] Filtering can be applied fixedly or adaptively. Filtering can be determined based on at least one factor or a combination of the size of the current block, the size of the adjacent block, the coding mode of the current block or the adjacent block, the block boundary characteristics of the current block and the adjacent block (for example, the boundary of the coding unit or the boundary of the transform unit), the prediction mode or direction of the current block or the adjacent block, the prediction method of the current block or the adjacent block, the quantization parameter, etc. The determination can have the same encoder / decoder setting (implicit), or can be performed based on the coding cost, etc. (explicit). The default filter is a low-pass filter, and the number of filter taps, filter coefficients, whether to encode the filter flag, the number of times the filter is applied, etc. can be determined based on the above factors. The information thereon can be set in units such as sequences, pictures, strips, blocks, etc., and the information can be sent to the decoder.

[0220] In intra prediction, the prediction block generation unit may generate a prediction block by an extrapolation method through reference pixels or by an interpolation method such as a DC or planar mode of reference pixels, or a method of copying reference pixels.

[0221] In the method of duplicating reference pixels, one reference pixel may be duplicated to generate at least one predicted pixel, or at least one reference pixel may be duplicated to generate at least one predicted pixel. The number of duplicated reference pixels may be equal to or less than the number of duplicated predicted pixels.

[0222] In addition, the prediction method can be classified into a directional prediction method and a non-directional prediction method. Specifically, the directional prediction method can be classified into a linear directional method and a curved directional method. The linear directional method uses an extrapolation method, but the pixels of the prediction block are generated by reference pixels placed on the prediction direction line. The curved directional method uses an extrapolation method, but the pixels of the prediction block are generated by reference pixels placed on the prediction direction line. The partial prediction direction of the pixel unit can be changed based on the specific direction of the block (e.g., edge).

[0223] In the method for encoding and decoding an image according to an embodiment, a linear directional method will be mainly described with respect to a directional prediction mode.

[0224] In addition, in the directional prediction method, the intervals between adjacent prediction modes are equal or unequal, and can be determined according to the size or shape of the block. For example, when a current block having a size and shape of M×N is obtained by the block partitioning unit, when M is equal to N, the intervals between the prediction modes may be equal. When M is different from N, the intervals between the prediction modes may be unequal.

[0225] As another example, when M is greater than N, a mode having a vertical directivity may allocate dense intervals between prediction modes close to the vertical mode (90°), and may allocate wider intervals between prediction modes far from the vertical mode. When N is greater than M, a mode having a horizontal directivity may allocate dense intervals between prediction modes close to the horizontal mode (180°), and may allocate wider intervals between prediction modes far from the horizontal mode.

[0226] These examples are not limited thereto and may include cases where conditions are switched or may be modified. Here, the interval between prediction modes may be calculated based on a numerical value indicating the directionality of each mode, and the directionality of the prediction mode may be numerically expressed by tilt information or angle information of the direction.

[0227] In addition, in addition to the above method, a method using spatial correlation may be used to generate a prediction block. For example, a reference block as a prediction block may be generated by using a current picture as a reference picture and using an inter-prediction method such as motion search and compensation.

[0228] In the prediction block generation step, the prediction block may be generated by using reference pixels according to the prediction method. That is, according to the prediction method, the prediction block may be generated by a directional prediction method or a non-directional prediction method such as extrapolation, interpolation, replication, and averaging of an existing intra-frame prediction method. Alternatively, the prediction block may be generated by using an inter-frame prediction method or other additional methods.

[0229] The intra prediction method can be supported under the same encoder / decoder setting and can be determined according to the slice type, block size, block shape, etc. The intra prediction method can be supported according to at least one of the above prediction methods or a combination thereof. The intra prediction mode can be configured according to the supported prediction method. The number of supported intra prediction modes can be determined according to the prediction method, slice type, block size, block shape, etc. The relevant information can be set and sent in units such as sequence, picture, slice, block, etc.

[0230] In the prediction mode encoding step, a mode in which the encoding cost according to each prediction mode is optimal may be determined as the prediction mode of the current block in terms of encoding cost.

[0231] For example, the prediction mode encoding unit may use the mode of at least one neighboring block to predict the current block mode so as to reduce the bits used for the prediction mode. The prediction mode may be included in a candidate group of the same most probable mode (MPM) as the candidate block mode, and the mode of the neighboring block may be included in the candidate group. For example, the candidate group may include prediction modes of the top left, bottom left, top, top right, etc. blocks of the current block.

[0232] The candidate group of prediction mode can be configured according to at least one factor or a combination thereof, including the position of adjacent blocks, the priority of adjacent blocks, the priority of partitioned blocks, the size or shape of adjacent blocks, a preset specific mode (in the case of chrominance blocks), and the prediction mode of luminance blocks. The relevant information can be sent in units such as sequences, pictures, stripes, and blocks.

[0233] For example, when the current block and the adjacent block are partitioned into at least two blocks, it can be determined under the same encoder / decoder setting which block's mode in the partitioned blocks is included as a mode prediction candidate for the current block. In addition, for example, the block partitioning unit performs quadtree-based partitioning on the left block of the adjacent block of the current block (M×M). When three partitioned blocks are composed and M / 2×M / 2, M / 4×M / 4, and M / 4×M / 4 blocks are included in the direction from top to bottom, based on the size of the block, the prediction mode of the M / 2×M / 2 block can be included as a mode prediction candidate for the current block.

[0234] As another example, the block partitioning unit performs binary tree-based partitioning on an upper block in a neighboring block of a current block (N×N). When three partitioned blocks are formed and blocks of N / 4×N, N / 4×N, and N / 2×N are included in the direction from left to right, the prediction mode of the first N / 4×N block from the left can be included as a mode prediction candidate of the current block according to a preset order (priority assigned from left to right).

[0235] As another example, when the prediction mode of the neighboring block of the current block is a directional prediction mode, a prediction mode adjacent to the prediction direction of the current mode (according to the tilt information or angle information of the mode direction) may be included in the mode prediction candidate group of the current block. In addition, according to the prediction mode configuration or the combination of neighboring blocks, a preset mode (plane, DC, vertical, horizontal, etc.) may be preferentially included.

[0236] In addition, the prediction mode with a high frequency of occurrence in the prediction mode of the adjacent block may be preferentially included. The priority not only indicates the possibility of being included in the mode prediction candidate group of the current block, but also indicates the possibility of being assigned a higher priority or index in the candidate group configuration (i.e., a higher possibility of being assigned fewer bits in the binarization process).

[0237] As another example, in the case where the maximum number of mode prediction candidate groups for the current block is k, where the left block consists of m blocks whose length is less than the height of the current block and the upper block consists of n blocks whose length is less than the width of the current block, when the sum of the partitioned blocks of the adjacent blocks (m+n) is greater than k, it is possible to fill the candidate group according to a preset order (from left to right, from top to bottom). When the sum of the partitioned blocks of the adjacent blocks (m+n) is greater than the maximum value k of the candidate group, the prediction modes of the adjacent blocks (left block and upper block) and the prediction modes of other adjacent blocks (e.g., lower left, upper left, upper right blocks, etc.) may be included in the mode prediction candidate group for the current block. These examples are not limited to this, and may include cases where the condition is switched, or may be modified.

[0238] As described above, the candidate blocks for mode prediction of the current block are not limited to a specific block position, and prediction mode information can be utilized from at least one block located in the left, upper left, lower left, upper, and lower right blocks. As shown in the above example, various factors can be considered to configure the prediction mode candidate group of the current block.

[0239] The prediction mode encoding unit may distinguish a candidate group of the most probable mode (MPM) (referred to as candidate group 1 in an embodiment) from a candidate group of the opposite mode (referred to as candidate group 2 in an embodiment). Depending on which candidate group the prediction mode of the current block belongs to, the process of encoding the prediction mode may be different.

[0240] The overall prediction mode may be composed of the sum of the prediction modes of candidate group 1 and the prediction modes of candidate group 2. The number of prediction modes of candidate group 1 and the number of prediction modes of candidate group 2 may be determined based on at least one factor or a combination of the total number of prediction modes, slice type, block size, block shape, etc. The same binarization or other binarization may be applied according to the candidate group.

[0241] For example, fixed-length binarization may be applied to candidate group 1, and truncated unary binarization may be applied to candidate group 2. In the above description, the number of candidate groups is 2 as an example, but it is possible to expand to a first candidate group having a pattern with the same high probability as the current block pattern, a second candidate group having a pattern with the same high probability as the current block pattern, and a candidate group of the opposite pattern, and changes are also allowed.

[0242] In the post-filtering step performed by the post-filter unit, some predicted pixels of the prediction block generated in the previous process may be replaced with values ​​generated by filtering at least one predicted pixel and at least one reference pixel adjacent to the boundary, taking into account the high correlation between the pixels in the current block and the reference pixels adjacent to the boundary of the current block and the adjacent blocks. Optionally, the predicted pixels may also be replaced with values ​​generated by filtering the numerical values ​​of the characteristics of the reference pixels adjacent to the boundary of the block (e.g., the difference in pixel values, tilt information, etc.). Another method other than the above method and having a similar purpose (correcting some predicted pixels of the prediction block by reference pixels) may be added.

[0243] In the post-filter unit, the type of filter and whether to apply filtering can be determined implicitly or explicitly. The encoder / decoder can set the reference pixels used by the post-filter unit, the position and number of current pixels, and the type of prediction mode. The relevant information can be sent in units such as sequence, picture, and strip.

[0244] In addition, in the post-filtering step, after the prediction block is generated, additional processing such as block boundary filtering may be performed. In addition, similar to boundary filtering, post-filtering may be performed on the current block reconstructed by acquiring a residual signal and then adding the residual signal and the prediction signal obtained by the transform / quantization process and its inverse process, taking into account the characteristics of the pixels of the adjacent reference blocks.

[0245] As a result, the prediction block is selected or obtained through the above-mentioned process. The information obtained from the process may include information related to the prediction mode, and it may be sent to the transform unit 210 for encoding the residual signal after obtaining the prediction block.

[0246] Fig.10 is an exemplary view illustrating a prediction principle in a P slice or a B slice in a method for encoding an image according to an embodiment of the present invention. Fig.11 It is shown in Fig.10 An exemplary view of a case where interpolation is performed in a method for encoding an image.

[0247] refer to Fig.10 , the method for encoding an image according to an embodiment may include motion estimation and interpolation steps. Information about a motion vector, a reference picture index, and a reference direction generated in the motion estimation step may be provided to the interpolation step. In the motion estimation step and the interpolation step, a value stored in a decoded picture buffer (DPB) may be used.

[0248] That is, the apparatus for encoding an image may perform motion estimation to find a block similar to the current block in a previously encoded picture. In addition, the apparatus for encoding an image may perform interpolation of a reference picture for accurate prediction. As a result, the apparatus for encoding an image may obtain a predicted block through a predictor, and information about the process may include a motion vector, a reference picture index (reference picture index or reference index), a reference direction, etc., and then residual signal encoding may be performed.

[0249] In an embodiment, since intra prediction is performed on a P slice or a B slice, the following can be achieved: Fig.11 A combined approach is shown that supports both inter-frame prediction and intra-frame prediction.

[0250] like Fig.11 As shown, the method for encoding an image according to an embodiment may include the steps of reference pixel filling (reference sample filling), reference pixel filtering (reference sample filtering), intra prediction, boundary filtering, motion estimation and interpolation.

[0251] When the apparatus for encoding an image supports block matching in the current picture, the prediction method in the I slice can be Fig.11 Rather than Fig. 9 That is, the apparatus for encoding an image may generate a prediction block using the prediction mode in the I slice and information about a motion vector, a reference picture index, a reference direction, etc. that only appear in the P slice or the B slice. However, due to the characteristic that the reference picture is the current picture, there may be information that can be partially omitted. For example, when the reference picture is the current picture, the reference picture index and the reference direction may be omitted.

[0252] In addition, in a device for encoding an image, when interpolation is applied, due to the characteristics of artificial images such as computer graphics, it may not be necessary to achieve block matching in fractional units, so whether to perform block matching can be set by the encoder and can be set in units such as sequences, pictures, and strips.

[0253] For example, the apparatus for encoding an image may not perform interpolation on a reference picture used in inter-frame prediction according to the setting of the encoder, and may not perform interpolation only when block matching is performed in the current picture. That is, the apparatus for encoding an image according to an embodiment may set whether to perform interpolation on a reference picture. Here, it may be determined whether to perform interpolation on all reference pictures or some reference pictures in the reference picture list.

[0254] For example, when the device for encoding an image does not need to perform block matching in decimal units due to the characteristics of the image in the reference block being an artificial image, it may not perform interpolation, and when it needs to perform block matching in decimal units due to a natural image, it may perform interpolation.

[0255] In addition, the apparatus for encoding an image may set whether block matching is applied to a reference picture for performing interpolation in block units. For example, when a natural image and an artificial image are mixed, interpolation may be performed on the reference picture. When the optimal motion vector can be obtained by searching a portion of the artificial image, the motion vector may be represented in a predetermined unit (here, assumed to be an integer unit). When the optimal motion vector can be obtained by selectively searching a portion of the natural image, the motion vector may be represented in another specific unit (here, assumed to be a 1 / 4 unit).

[0256] Fig.12 is a view illustrating a main process of a method for encoding an image according to an embodiment of the present invention as a syntax in a coding unit.

[0257] refer to Fig.12 , curr_pic_BM_enabled_flag indicates a flag that allows block matching in the current picture, and can be defined and transmitted in sequence and picture units. Here, the process of generating a prediction block by performing block matching on the current picture can represent an operation through inter-frame prediction. It can be assumed that cu_skip_flag, which is an inter-frame technique for not encoding a residual signal, is a flag supported only by P slices or B slices other than I slices. In this case, when curr_pic_BM_enabled_flag is turned on, I slices can also support block matching (BM) in inter-frame prediction mode.

[0258] When a prediction block is generated by block matching in the current picture, the method for encoding an image according to an embodiment may support skipping, and may support skipping in the case of intra-frame techniques other than block matching. Depending on the conditions, I-slices may not support skipping. Whether to perform skipping may be determined according to the setting of the encoder.

[0259] For example, when skipping is supported in an I slice, a prediction block can be directly reconstructed as a reconstructed block by block matching without encoding a residual signal by linking with prediction_unit() as a prediction unit. In addition, the apparatus for encoding an image classifies a method of using a prediction block by block matching in a current picture into an inter-frame prediction technique, and such classification can be handled by pred_mode_flag as a special flag.

[0260] In addition, the apparatus for encoding an image according to an embodiment may set the prediction mode to an inter-frame prediction mode (MODE_INTER) when pred_mode_flag has a value of 0, and may set the prediction mode to an intra-frame prediction mode (MODE_INTRA) when pred_mode_flag has a value of 1. This is an intra-frame technique similar to the prior art, but it may be classified as an inter-frame technique or an intra-frame technique in an I strip to distinguish it from an existing structure. That is, the apparatus for encoding an image according to an embodiment may not use temporal correlation in an I strip, but may use a structure of temporal correlation. part_mode represents information about the size and shape of a block divided in a coding unit.

[0261] Here, in the sequence parameters, the flag for supporting block matching in the current picture may be sps_curr_pic_BM_enabled_flag, and in the picture parameters, the flag for supporting block matching in the current picture may be pps_curr_pic_BM_enabled_flag. When sps_curr_pic_BM_enabled_flag is turned on and pps_curr_pic_BM_enabled_flag is turned off, block matching may not be supported in the current picture. According to the flag, it may be determined whether the current picture is included in the reference picture list 0, and it may be determined whether the current picture is included in the reference picture list 1.

[0262] Fig.13 Is the diagram when through Fig.12 An exemplary view of an example of supporting symmetric type partitioning or asymmetric type partitioning as in inter-frame prediction when block matching in a current block generates a prediction block.

[0263] refer to Fig.13 In the method for encoding an image according to an embodiment, when a prediction block is generated by block matching in a current picture, just as in inter-frame prediction, symmetric partitioning such as 2N×2N, 2N×N, N×2N and N×N can be supported or asymmetric partitioning such as nL×2N, nR×2N, 2N×nU and 2N×nD can be supported.

[0264] Fig.14 It is the positive image Fig. 9 As in the intra prediction of the block, an exemplary view of examples of 2N×2N and N×N are supported in the inter prediction. The size and shape of the block may be determined according to the partitioning method of the block partitioning unit.

[0265] refer to Fig.14In the method for encoding an image according to an embodiment, 2N×2N and N×N may be supported like the prediction block shape used in the existing intra prediction. This is an example of supporting a square shape by a quadtree partitioning method in a block partitioning unit or a partitioning method of a predefined block candidate group. In intra prediction, by adding a rectangular shape to a binary tree partitioning method or a predefined block candidate group, other block shapes may be supported, and their settings may be set by the encoder.

[0266] In addition, in intra prediction, the encoder can set whether to apply skipping only when performing block matching (ref_idx=curr) on the current picture, whether to apply skipping in existing intra prediction, and whether to apply skipping in other new intra predictions. The above information can be sent in units such as sequence, picture, slice, etc.

[0267] Subtractor 205 (reference Figure 2 ) subtracts the pixel values ​​of the prediction block generated by the prediction unit 200 from the pixel values ​​of the current block to be encoded, thereby deriving pixel difference values ​​and generating a residual block.

[0268] Transformation unit 210 (reference Figure 2 ) receives a residual block as a difference between a current block and a prediction block generated by intra prediction or inter prediction from a subtractor 205, and transforms the residual block in a frequency domain. Through the transformation, each pixel of the residual block corresponds to a transformation coefficient of the transformation block. The size and shape of the transformation block may be equal to or smaller than the coding unit. In addition, the size and shape of the transformation block may be equal to or smaller than the size and shape of the prediction unit. The apparatus for encoding an image may perform transformation on a plurality of prediction units at the same time.

[0269] The size or shape of the transform block can be determined by the block partitioning unit, and square or rectangular transforms can be supported according to the block partitioning. The transform-related settings supported in the encoder / decoder (supported transform block size, shape, etc.) can affect the block partitioning.

[0270] According to the encoding cost of each candidate, the size and shape of the transform block may be determined, and image data of the determined transform block and partition information about the determined size and shape of the transform block may be encoded.

[0271] The transformation may be performed by a one-dimensional transformation matrix. For example, each transformation matrix may be used adaptively in discrete cosine transform (DCT), discrete sine transform (DST), horizontal and vertical units. Adaptive use may be determined based on factors such as block size, block shape, block type (luminance / chrominance), coding mode, prediction mode information, quantization parameter, coding information of neighboring blocks, and the like.

[0272] For example, in intra prediction, when the prediction mode is horizontal mode, a DCT-based transform matrix may be used in the vertical direction, and a DST-based transform matrix may be used in the horizontal direction. In addition, when the prediction mode is vertical mode, a DCT-based transform matrix may be used in the horizontal direction, and a DST-based transform matrix may be used in the vertical direction.

[0273] The transformation matrix is ​​not limited to the above description. The information thereon can be determined by using an implicit or explicit method, or can be determined according to at least one factor or a combination thereof of the size of the block, the shape of the block, the encoding method, the prediction mode, the quantization parameter, the encoding information of the adjacent blocks, etc. The related information can be sent in units such as sequences, pictures, stripes, blocks, etc.

[0274] Here, in the case of using the explicit method, when at least two transformation matrices for the horizontal and vertical directions are a candidate group, information about which transformation matrix to use for each direction may be transmitted. Optionally, which transformation matrix to use for the horizontal and vertical directions is grouped into a pair, at least two pairs form a candidate group, and information about which transformation matrix to use in the horizontal and vertical directions may be transmitted.

[0275] In addition, taking into account the characteristics of the image, partial or overall transformations may be omitted. For example, one or both of the horizontal and vertical components may be omitted. When intra-frame prediction or inter-frame prediction cannot be performed well and the difference between the current block and the predicted block is large, that is, when the residual component is large, the coding loss in the transformation of the residual component may be large. This can be determined based on at least one factor or a combination thereof, such as the coding mode, the prediction mode, the size of the block, the shape of the block, the type of the block (luminance / chrominance), the quantization parameter, the coding information of the adjacent blocks, etc. It can be indicated by using an implicit or explicit method according to the above conditions, and the information thereon can be sent in units such as sequences, pictures, and strips.

[0276] Quantization unit 215 (reference Figure 2 ) may perform quantization on the residual component transformed by the transform unit 210. A quantization parameter may be determined in a block unit, and the quantization parameter may be set in a sequence, a picture, a slice, a block, or the like.

[0277] For example, the quantization unit 215 may predict the current quantization parameter by using one or at least two quantization parameters derived from neighboring blocks such as left, upper left, above, upper right, lower left, etc. of the current block.

[0278] In addition, when there is no quantization parameter predicted from a neighboring block, that is, when the block is located at the boundary of a picture, a slice, etc., the quantization unit 215 may output or transmit a difference value from a default parameter transmitted in units of a sequence, a picture, a slice, etc. When there is a quantization parameter predicted from a neighboring block, the difference value may be transmitted by using the quantization parameter of the block.

[0279] The priority of the block used to derive the quantization parameter may be preset and may be sent in units of sequence, picture, slice, etc. The residual block may be quantized by dead zone uniform threshold quantization (DZUTQ), a quantization weighting matrix, or a technique improved therefrom. At least one quantization technique may be set as a candidate and may be determined based on information about a coding mode, a prediction mode, etc.

[0280] For example, the quantization unit 215 can set the quantization weighting matrix to be applied to inter-frame coding, intra-frame coding, etc., and can have other weighting matrices based on the intra-frame prediction mode. Assuming that the size of the block is equal to the size of the quantization block of size M×N, the quantization weighting matrix can be configured by having different quantization coefficients for the position of each frequency component. In addition, the quantization unit 215 can select one of multiple existing quantization methods, or can be used under the same encoder / decoder settings. The information thereon can be sent in units such as sequences, pictures, and stripes.

[0281] at the same time, Figure 2 and Figure 3 The inverse quantization units 220 and 315 and the inverse transform units 225 and 320 shown in FIG. 2 may be implemented by inversely performing the processes of the transform unit 210 and the quantization unit 215. That is, the inverse quantization unit 220 may perform inverse quantization on the quantized transform coefficients generated by the quantization unit 215, and the inverse transform unit 225 may generate a reconstructed residual block by inversely transforming the inverse quantized transform coefficients.

[0282] Figure 2 and Figure 3 The adders 230 and 324 shown in FIG. 2 can add the pixel values ​​of the prediction block generated by the prediction unit to the pixel values ​​of the reconstructed residual block to generate a reconstructed block. The reconstructed block can be stored in the encoding and decoding picture buffers 240 and 335 to be provided to the prediction unit and the filter unit.

[0283] Filter unit 235 (reference Figure 2) can apply loop filters such as a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), etc. to the reconstructed block. The deblocking filter can perform filtering on the reconstructed block to remove distortion between block boundaries that occur during encoding and decoding. SAO is a filter process that reconstructs the difference between the original image and the reconstructed image offset by the pixel unit of the residual block. ALF can perform filtering to minimize the difference between the predicted block and the reconstructed block. ALF can perform filtering based on a comparison value between the reconstructed block and the current block that passed through the deblocking filter.

[0284] Entropy encoder 245 (reference Figure 2 ) may perform entropy encoding on the transform coefficients quantized by the quantization unit 215. For example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), and encoding other than probability interval partitioning entropy encoding (PIPE) implemented by using other encoding methods may be performed.

[0285] The entropy encoder 245 may include a bit string in which the quantization coefficients are encoded and information required to decode the encoded bit string into the encoded data. The encoded data may include the shape of the coding block, the quantization coefficients, the bit string in which the quantization block is encoded, and the information required for prediction. In the case of the quantization coefficients, the two-dimensional quantization coefficients may be scanned in one dimension. The distribution of the quantization coefficients may be different depending on the characteristics of the image. In particular, in the case of intra-frame prediction, since the distribution of the coefficients may be different depending on the prediction mode, the scanning method may be set differently.

[0286] In addition, the entropy encoder 245 may be set differently depending on the size of the block to be encoded. The scanning pattern may be preset to at least one of various patterns such as zigzag, diagonal, raster, etc., or as a candidate, may be determined based on information about the encoding mode, prediction mode, etc., and the same setting may be used on the encoder and decoder. The information thereon may be transmitted in units such as sequences, pictures, and strips.

[0287] The size of the quantized block (hereinafter quantized block) input to the entropy encoder 245 may be equal to or smaller than the size of the transform block. In addition, the quantized block may be partitioned into at least two sub-blocks, in which case the scanning mode in the partitioned block may be set to be the same as the scanning mode of the existing quantized block, or may be set differently.

[0288] For example, when the scanning pattern of the existing quantized block is a zigzag pattern, the zigzag pattern can be applied to all sub-blocks. Alternatively, the zigzag pattern can be applied to the sub-block located at the upper left of the block with the average value (DC) component, and the diagonal pattern can be applied to the other blocks. This can also be determined based on information about the coding mode, prediction mode, etc.

[0289] In addition, the start position of the scanning pattern in the entropy encoder 245 basically starts from the upper left, but may start from the upper right, lower right, or lower left according to the characteristics of the image. Information on which of the at least two candidate groups is selected may be transmitted in units of a sequence, a picture, a slice, etc. As an encoding technique, an entropy encoding technique may be used, but is not limited thereto.

[0290] at the same time, Figure 2 and Figure 3 The inverse quantization of the inverse quantization unit 220 and the inverse transformation of the inverse transformation unit 225 shown in can be realized by inversely configuring the quantization of the quantization unit 215 and the transformation of the transformation unit 210 and by combining the basic filter units 235 and 330 .

[0291] Next, interpolation that can be applied to the apparatus for encoding an image according to the present invention will be described as follows.

[0292] In order to improve the accuracy of prediction by block matching, interpolation is performed with a resolution of decimal units rather than integer units. As an interpolation method, there is DCT-IF (interpolation filter based on discrete cosine transform) and the like. DCT-IF technology is used as an interpolation method in HEVC (High Efficiency Video Coding). For example, pixels are generated in 1 / 2 and 1 / 4 units between integers, and interpolation is performed on a reference picture. Block matching is performed with reference to it to generate a prediction block.

[0293] [Table 1]

[0294] Pixel location Filter coefficient (fi) 0 0，0，0，64，0，0，0，0 1 / 4 -1，4，-10，58，17，-5，1，0 1 / 2 -1，4，-11，40，40，-11，4，-1 3 / 4 0，1，-5，17，58，-10，4，-1

[0295] [Table 2]

[0296] Pixel location Filter coefficient (fi) 0 0，64，0，0 1 / 8 -2，58，10，-2 1 / 4 -4，54，16，-2 3 / 8 -6，46，28，-4 1 / 2 -4，36，36，-4 5 / 8 -4，28，46，-6 3 / 4 -2，16，54，-4 7 / 8 -2，10，58，-2

[0297] Tables 1 and 2 show filter coefficients used in the luminance component and the chrominance component, respectively. For the luminance component, 8 taps are used, and for the chrominance component, 4 taps of DCT-IF are used. For the chrominance component, filters may be applied differently depending on the color format. In the case of 4:2:0 of ​​YCbCr, filters as shown in Table 2 may be applied. In the case of 4:4:4, filters as shown in Table 1 or other filters may be applied instead of the filters as shown in Table 2. In the case of 4:2:2, a horizontal 1-D 4 tap filter as shown in Table 2 and a vertical 1-D 8 tap filter as shown in Table 1 may be applied.

[0298] Fig.15 1 is a view illustrating a process of performing 1-D horizontal filtering on pixels at positions a, b, and c (assume x) of an image in a method for encoding an image according to an embodiment of the present invention.

[0299] like Fig.15 As shown, 1-D horizontal filtering can be applied to sub-pixels (assuming x) at positions a, b, and c between a first pixel (G) and a second pixel (H) adjacent thereto. This is indicated in the following equation.

[0300] x=(f1*E+f2*F+f3*G+f4*H+f5*I+f6*J+32) / 64

[0301] Next, 1-D vertical filtering can be applied to sub-pixels at positions d, h, and n (assuming y). This is indicated in the following equation.

[0302] y=(f1*A+f2*C+f3*G+f4*M+f5*R+f6*T+32) / 64

[0303] 2D separable filtering can be applied to sub-pixels e, f, g, i, j, k, p, q, and r at the center. For example, in the case of sub-pixel e, interpolation is performed on sub-pixel a and pixels in the vertical direction, and then, interpolation is performed by using these pixels. 1-D horizontal filtering is performed as interpolation of sub-pixel a between G and H, and 1-D vertical filtering is performed on sub-pixels obtained therefrom, so that sub-pixel e can be obtained. Similar operations can be performed on chrominance signals.

[0304] The above description is only a partial description of interpolation. In addition to DCT-IF, other filters can be used, and the filter type and the number of taps can be different according to the decimal unit. For example, an 8-tap Kalman filter is applied to 1 / 2, a 6-tap Wiener filter is applied to 1 / 4, and a 2-tap linear filter is applied to 1 / 8, so that fixed coefficients or filter coefficients are calculated like DCT-IF, and the filter coefficients can be encoded. As described above, one interpolation filter can be used for one picture, or different interpolation filters can be used for different areas according to the characteristics of the image. Optionally, at least two reference pictures to which multiple interpolation filters are applied are generated, and one of the at least two reference pictures can be selected.

[0305] Different filters may be applied according to coding information such as the type of reference picture, the temporal layer, the state of the reference picture (e.g., whether it is the current picture), etc. The above information may be set and transmitted in units of sequence, picture, slice, etc.

[0306] Next, before describing in detail improved techniques regarding motion estimation, motion compensation, and motion prediction that may be applied to a method for encoding an image according to an embodiment, basic meanings of these terms will be defined below.

[0307] In the motion prediction of the current block to be encoded, motion estimation is a process of dividing an image frame into small blocks and estimating displacement from which block on the encoding frame (reference frame) before or after in time. That is, motion estimation is a process of finding a block that is most similar to a target block when encoding the current block. Block-based motion estimation is a process of estimating the position to which a block (macroblock, etc.) of a video object or screen processing unit has moved in time.

[0308] In motion compensation, the current image is encoded using at least a partial area of ​​a previously encoded reference image, and a prediction block of the current block is generated based on the motion information (motion vector, reference picture index) of the optimal prediction block obtained in motion estimation to predict the current image. That is, motion compensation is a process of generating an error block as a difference between the current block to be encoded and the reference block identified as the most similar block.

[0309] "Motion prediction" means finding a motion vector for motion compensation in encoding. As the main techniques for motion prediction, there are skip, temporal prediction, spatial prediction, etc. Skipping means skipping the encoding of an image block when the motion of the screen is constant and the size of the motion vector predicted by the encoder is zero or the residual is small enough to be ignored. Temporal prediction can be mainly used for inter-frame prediction, and spatial prediction can be mainly used for intra-frame prediction.

[0310] The information obtained by the inter prediction may include information for identifying the direction of the reference picture list (unidirectional (L0, L1) and bidirectional), an index for identifying the reference picture in the reference picture list, a motion vector, etc. Since the temporal correlation is used, the motion information can be efficiently encoded when the characteristic that the current block and the adjacent block have the same or similar motion vector is utilized.

[0311] Fig.16 is an exemplary view illustrating a current block and neighboring blocks that can be applied to a method for encoding an image according to an embodiment of the present invention.

[0312] like Fig.16 As shown, whether to set a reference to a candidate group of neighboring blocks of the current block can be determined based on information about the type of the current picture, a temporal id, etc., and the information thereon can be fixed or can be sent in units such as sequences, pictures, and strips.

[0313] Here, the setting of whether to refer to the candidate group may include using only spatially adjacent blocks (A, B, C, D, E), or using spatially adjacent blocks (A, B, C, D, E) and temporally adjacent blocks (H, I, J), or using spatially adjacent blocks (A, B, C, D, E) and blocks (F, G) that are spatially far apart.

[0314] Next, the setting of “block matching can be applied to the current picture” will be described as follows.

[0315] First, in the case of an I picture, for example, spatially adjacent blocks have priority, and the remaining blocks can be set as a candidate group. For example, the availability of the reference blocks can be verified in the order of E→D→C→B→A→H→I→J. The availability can be compared with a preset reference value, or can be compared with other availability check values. The availability can be determined based on the encoding mode of the candidate block, motion information, the position of the candidate block, etc. The motion information may include a motion vector, a reference direction, a reference picture index, etc.

[0316] Since the current picture is an I picture, motion information exists only in an embodiment in which the coding mode is inter-frame prediction (INTER in this case). Therefore, it is first checked whether it is INTER in order of priority. For example, when n is 3 and E is encoded in INTER, E is excluded from the candidate group and D is checked. When D is encoded in INTER, motion information exists because block matching is performed in the current picture. Based on the motion information, D is added to the candidate group. Subsequently, n is 2. Next, the device for encoding an image can check the priority again. As a result, when three final candidates are obtained, the search for the candidate group is stopped.

[0317] The availability may be determined based on the coding mode and the boundary of the picture, slice, tile, etc. In the case of the boundary, the availability is checked as unavailable. When it is determined that the block is the same as or similar to the already filled candidate, the block is excluded from the candidate, and the reference pixel configuration unit may check the availability of the subsequent candidate.

[0318] Here, INTER is different from the existing inter-frame prediction (inter). That is, the INTER mode of the embodiment uses the structure of the inter-frame prediction (inter), but generates the prediction block in the current picture, and therefore the INTER mode of the embodiment is different from the inter-frame prediction in which the prediction block is generated in the reference picture. That is, in the encoding mode of the embodiment, the method of block matching in the current picture can be classified into the INTER mode and the intra mode (the same as the existing intra).

[0319] Since the scaling process may or may not be included, motion vector copying (MVC) and motion vector prediction (MVP) will be described separately hereinafter.

[0320] First, motion vector prediction (MVP) will be described.

[0321] In the case of P-picture or B-picture

[0322] In addition to the above candidates (A, B, C, D, E, F, G, H, I, J), the temporal candidates (F, G) will also be described. In an embodiment, the candidates are searched spatially, temporally, by configuring a mixed list, and then a constant candidate is found.

[0323] First, the priority of the candidates is determined, and the availability is checked according to the priority. The number of candidates (n) of the motion vector may be set to 2, and the priority may be the same as described in the brackets.

[0324] For example, when performing a spatial search, blocks may be classified into the following groups.

[0325] Group 1_1 = {A, B, C, I, J}, (C→B→A→I→J)

[0326] Group 1_2 = {D, E, H}, (D → E → H)

[0327] In an embodiment, based on the current block, group 1_1 of the two groups includes the immediately adjacent upper block, upper left block and upper right block. Based on the current block, group 1_2 includes the left block adjacent to the current block and the left block and lower left block not adjacent to the current block.

[0328] In another embodiment, the candidate blocks of the motion vector can be spatially searched by being classified into three groups. The three groups can be classified as follows.

[0329] Group 1_1 = {A, B, C}, (C → B → A)

[0330] Group 1_2 = {D, E}, (D→E)

[0331] Group 1_3 = {H, I, J}, (J→I→H)

[0332] In the three groups of the embodiment, based on the current block, group 1_1 may include a left block next to the current block, a left block next to it, and an upper right block next to it. Based on the current block, group 1_2 may include a left block next to the current block, and a block below the next left block. Group 1_3 may include blocks that are not next to the current block, with at least one block space between them.

[0333] In another embodiment, the candidate blocks of the motion vector can be spatially searched by being classified into three groups. The three groups can be classified as follows.

[0334] Group 1_1 = {B}

[0335] Group 1_2 = {D}

[0336] Group 1_3 = {A, C, E}, (E → C → A)

[0337] Among the three groups of the embodiment, group 1_1 may include blocks located in the vertical direction based on the current block, group 1_2 may include adjacent blocks located in the horizontal direction based on the current block, and group 1_3 may include the remaining adjacent blocks based on the current block.

[0338] As described above, a P picture or a B picture has reference information such as a reference direction, a reference picture, and the like, and thus a candidate group can be set according to the information. Based on the current block, a candidate block having a reference picture different from the reference picture of the current block can be included in the candidate group. Optionally, considering the temporal distance (picture count (POC)) between the reference picture of the current block and the reference picture of the candidate block, the vector of the block can be scaled and added to the candidate group. In addition, the scaled candidate group can be added according to which picture the reference picture of the current block is. In addition, when the temporal distance between the reference picture of the current block and the reference picture of the candidate block exceeds a preset distance, the scaled block can be excluded from the candidate group, otherwise, the scaled block can be included in the candidate group.

[0339] The above similarity check is a process of comparing and determining the similarity between the motion vectors already included in the prediction candidate group and the motion vector to be newly added thereto. By definition, "yes" can be set when the x and y components match completely, or when the difference is within a preset threshold range.

[0340] In the embodiment, the condition that the reference picture is the current picture is used as an example, but the condition that the reference picture is not the current picture may also be used. For example, a setting such as "exclude blocks that use pictures farther away than the picture indicated by the current block" may be used. In the embodiment, even if the current block is different from the reference picture, the block may be added to the candidate group by scaling.

[0341] Mixed List

[0342] When bidirectional prediction of the current block is performed and motion information exists for each reference picture in the reference picture list (L0, L1), availability can be checked according to the priority of the preset candidate group. In this case, the block encoded in bidirectional prediction is checked first according to the priority.

[0343] When the reference pictures are different, scaling is performed. When only bidirectional prediction blocks are added to the candidate group when searching for candidates spatially or temporally, and when the maximum number of candidates is not exceeded, blocks encoded in unidirectional prediction among the candidate blocks are added to the preliminary candidate group, and a combination of candidates can be used to generate candidates for bidirectional prediction.

[0344] [Table 3]

[0345]

[0346]

[0347] (a)

[0348] Candidate index L0 L1 0 <![CDATA[mvA1,ref1]]> <![CDATA[mvA2,ref0]]> 1 <![CDATA[mvB1’,ref1]]> <![CDATA[mvB2’,ref0]]> 2 mvC',ref1 mvD,ref0 3 mvC',ref1 mvE',ref0 4

[0349] (b)

[0350] First, it is assumed that the motion information of the bidirectional prediction of the current block is referenced in the reference pictures numbered 1 in L0 and numbered 0 in L1. In Table 3(a), the motion information of the first candidate block is (mvA1, ref1) and (mvA2, ref0), and the motion information of the second candidate block is (mvB1', ref1) and (mvB2', ref0). Here, the apostrophe (') represents a scaled vector. After performing spatial and temporal searches, when the number of candidates is 2 and n is 5, the unidirectional prediction blocks in the previous step can be added to the preliminary candidate group according to a preset priority.

[0351] In Table 3(a), the maximum number of candidates has not been found, and thus new candidates may be added by combining the scaled unidirectional candidates using the remaining motion vectors mvC, mvD, and mvE.

[0352] In Table 3(b), the motion information of the unidirectional prediction block is scaled according to the reference picture of the current block. Here, an example of generating a new combination by using a unidirectional candidate has been described, but a combination of new candidates is possible by using the motion information of the bidirectional reference pictures (L0 and L1). This part is not performed in cases such as unidirectional prediction. In addition, when the reference picture of the current block is the current picture, this part may not be performed.

[0353] Constant Candidate

[0354] Through the above processing, when the maximum number n of candidate blocks cannot be configured (2 in the embodiment), constant candidates with preset fixed coordinates can be added. Constant candidates with fixed coordinates such as (0,0), (-a,0), (-2*a,0) and (0,-b) can be used, and the number of constant candidates can be set to the maximum number of candidates.

[0355] The above-mentioned fixed coordinates may be set, or a constant candidate may be added using an average value, a weighted average value, a median value, etc. of at least two motion vectors included in the candidate group up to that time point. When n is 5 and three candidates {(mvA_x, mvA_y), (mvB_x, mvB_y), (mvC_x, mvC_y)} are obtained up to that time point, in order to find the remaining two candidates, a candidate group including a constant candidate with a preset priority is generated, and the constant candidate is added according to the priority. The constant candidate group may include constant candidates such as ((mvA_x+mvB_x) / 2, (mvA_y+mvB_y) / 2), ((mvA_x+mvB_x+mvC_x) / 3, (mvA_y+mvB_y+mvC_y) / 3), (median(mvA_x, mvB_x, mvC_x), median(mvA_y, mvB_y, mvC_y)), etc.

[0356] In addition, constant candidates may be set differently depending on the reference picture of the current block. For example, when the current picture is a reference picture, constant candidates such as (-a, 0), (0, -b), and (-2*a, 0) may be set. When the current picture is not a reference picture, constant candidates such as (0, 0), (-a, 0), (average(mvA_x, ...), average(mvA_y, ...)) may be set. The information thereon may be preset in the encoder or decoder, or may be sent in units such as sequences, pictures, and slices.

[0357] Hereinafter, motion vector copying (MVC) will be described in detail.

[0358] Explanation of P-picture or B-picture

[0359] In an embodiment, temporal candidates (F and G) are included. The candidate group may include A, B, C, D, E, F, G, H, I, and J. The search order is not preset, but here, the MVC candidates are searched spatially, temporally, and by configuring a hybrid list, and then constant candidates are added.

[0360] That is, the above part also uses an arbitrarily set search order instead of a preset order. The priority is determined, and availability is checked according to the priority. Assume that n is 5 and the priority is the same as in the brackets.

[0361] In the following, only the differences from the above-mentioned motion vector prediction (MVP) will be described. MVP can provide the following steps without scaling. For spatial candidates, availability can be checked without scaling processing. However, similar to MVC, the type of reference picture, the distance of the reference picture of the current picture to the current block, etc. can be excluded from the candidate group.

[0362] When a mixed list exists, candidates for bidirectional prediction may be generated according to a combination of candidates added up to that point in time shown in Table 4.

[0363] [Table 4]

[0364] Candidate index L0 L1 0 mvA,ref0 1 mvB,ref1 2 mvC,ref0 3 4

[0365] (a)

[0366] Candidate index L0 L1 0 mvA,ref0 1 mvB,ref1 2 mvC,ref0 3 mvA,ref0 mvC,ref0 4 mvB,ref1 mvC,ref0

[0367] (b)

[0368] As shown in Table 4(a), by combining the candidate using reference list L0 and the candidate using reference list L1, a new candidate can be added to the motion vector candidate group. As shown in Table 4(b), when a preset number (i.e., 5) of motion vectors cannot be found, a new candidate can be added by combining the subsequent candidate of L0 and the candidate using L1.

[0369] Encoding may be performed according to a mode such as MVP, MVC, etc. to find the optimal motion information candidate.

[0370] In skip mode, MVC can be used to perform encoding. That is, after processing the skip flag, information about the optimal motion vector candidate can be encoded. When there is only one candidate, this processing can be omitted. Encoding can be performed by performing transformation, quantization, etc. on the residual component that is the difference between the current block and the predicted block, without encoding the motion vector difference.

[0371] When not in skip mode, determine whether to process motion information by MVC with priority. When it is determined that motion information is processed by MVC, information about a candidate group of optimal motion vectors can be encoded. When it is determined not to process motion information by MVC, motion information can be processed by MVP. In the case of MVP, information about the optimal motion vector candidate can be encoded. Here, when there is only one candidate, the processing of motion information can be omitted. Information about the difference with the motion vector of the current block, the reference direction, the reference picture index, etc. can be encoded, and a residual component can be obtained, and then transformation and quantization can be performed thereon.

[0372] The description of the codec such as subsequent entropy and post-filtering etc. will be omitted to avoid redundancy.

[0373] Meanwhile, in the method for encoding or decoding image data, data-level parallelism is a method of dividing the data to be processed in the parallelized program into multiple units and allocating the data to different cores or threads so that the same operation is performed in parallel. In theory, since the image processing speed becomes faster when parallelism is configured in consideration of the performance limit of the core or thread, parallelism is one of the important factors of performance.

[0374] The data unit processed in parallel may be a frame, a slice, or a block. Hereinafter, inter-frame prediction is performed via parallel processing.

[0375] Fig.17 is an exemplary view illustrating generating a prediction block through block matching in a current block in a method for encoding an image according to an embodiment of the present invention. Fig.18 is an exemplary view illustrating a reference block used as a prediction block through at least one block matching in a coding area in a method for encoding an image according to an embodiment of the present invention.

[0376] The embodiment can be applied to the case where encoding is performed by an inter-coding structure when block matching is performed in the current block.

[0377] refer to Fig.17 , four blocks including the block (B_t) at the center are encoding target current blocks, and arrows indicate reference blocks and indicate a case where the reference blocks exist in the current picture.

[0378] exist Fig.18 In the , block (rfb) represents a reference block in a coding region that is used as a prediction block through at least one block matching.

[0379] In the case of an area that has been encoded before the current block, since the block matching in the current picture is still before the loop filter such as the deblocking filter and the sample adaptive offset (SAO) is applied, the block matching in the current picture may cause problems in parallel processing. Therefore, before the filtering of the block matching is applied, additional memory may be required. In addition, in the case of an area that has already been encoded, since the area has been encoded before the loop filter is applied, it may be difficult to generate an optimal prediction block due to degradation between blocks.

[0380] According to an embodiment, when referencing a memory for a current picture in a decoded picture buffer (DPB) and other blocks in the current picture, additional memory may be required. Here, the additional memory may be a current picture memory different from an existing memory for the current picture, or may be a temporary current picture memory used in an encoding or decoding process. In addition, additional large-capacity memory for parallel processing may be required.

[0381] In addition, when a loop filter such as a deblocking filter and SAO is applied, the memory for the current picture may be used, and when the loop filter is not applied, an additional memory may be used. In addition, according to the operation setting of the loop filter, additional memory may not be required in units such as a sequence parameter set, a picture parameter set, a slice header, etc. For example, in a loop filter operation setting where the deblocking filter is turned off, SAO is turned off, and ALF is turned off, additional memory may not be required, and the loop filter may be configured to include some or another of the above configurations.

[0382] As described above, the reference block that the current block refers to in another block may be stored in another memory. This additionally reduces the capacity of the available memory, and it is therefore necessary to avoid this reduction. In the following, it will be described how to prevent unnecessary storage consumption by adaptive filtering.

[0383] Fig.19 is a block diagram illustrating an apparatus for encoding an image according to an embodiment of the present invention.

[0384] refer to Fig.19 According to an embodiment, the device for encoding an image may include a prediction unit 200, an adder 205, a transform unit 210, a quantization unit 215, an inverse quantization unit 220, an inverse transform unit 225, a subtractor 230, a decoded picture buffer (DPB) 240, an entropy encoder 245, a filtering skip confirmation unit 250, a skip selection circuit 260 and a filter unit 290.

[0385] Here, the prediction unit 200 may include a first prediction device 201 for intra prediction and a second prediction device 202 for inter prediction. The first prediction device may be referred to as an intra prediction unit, and the second prediction device may be referred to as an inter prediction unit. In addition, the adder 205 and the subtractor 230 may be referred to as a first addition and subtraction unit and a second addition and subtraction unit, respectively.

[0386] In addition, the filtering skip confirmation unit 250 is located between the second addition and subtraction unit 230 and the filter unit 290, and the skip selection circuit 260 is located in the space between the filtering skip confirmation unit 250 and the filter unit 290 and in the space between the filtering skip confirmation unit 250 and the decoded picture buffer 240. The filtering skip confirmation unit 250 controls the skip selection circuit 260 based on the selection information from the filtering skip flag, thereby adaptively performing deblocking filtering.

[0387] In addition, the filter unit 290 may be referred to as a loop filter unit, and may include at least one of the deblocking filter 270, a sample adaptive offset (SAO) 280, and an adaptive loop filter (ALF). The filter unit 290 may perform filtering on the reconstructed image.

[0388] The deblocking filter 270 will be described in detail as follows. That is, during the prediction, transformation, quantization and entropy coding processes, a quantization error may occur in the quantization step. It is adjusted using a quantization parameter value. When the quantization parameter value is small, dense quantization is performed on the transform coefficients, and therefore the quantization error may be relatively small. When the quantization parameter value is large, the quantization error may be relatively large. In order to solve this problem, the reconstructed picture is filtered, and thus the image quality degradation can be reduced. To this end, in the method and apparatus for encoding an image according to an embodiment, post filtering or loop filtering using a deblocking filter may be applied. The loop filter can act as a basic codec for HEVC.

[0389] refer to Fig.19 , a filtering skip confirmation unit 250 is added. The filtering skip confirmation unit 250 can control the skip selection circuit 260 to apply the filter unit 290 and select the skip mode based on the flag indicating the use of the reference block. Accordingly, parallel processing can be performed by using one decoded picture buffer (DPB) 240 without additional memory.

[0390] Therefore, the setting information can be the same in the encoder and the decoder, and can be sent between the encoder and the decoder in units of sequence, picture, slice, etc.

[0391] Fig. 20 is a block diagram illustrating an apparatus for decoding an image according to an embodiment of the present invention.

[0392] refer to Fig. 20 According to an embodiment, the device for decoding an image may include: an entropy decoder 305, an inverse quantization unit 315, an inverse transform unit 320, a second addition and subtraction unit, a prediction unit 310, a filter unit 330, a decoded picture buffer (DPB) 335a, a filtering skip confirmation unit 350 and a skip selection circuit 360.

[0393] Here, the prediction unit 310 may include an intra prediction unit 311 for intra prediction and an inter prediction unit 312 for inter prediction. The filter unit 330 may include a deblocking filter 370 and a sample adaptive offset (SAO) 380 .

[0394] The filtering skip confirmation unit 350 is located between the second addition and subtraction unit and the filter unit 330. The skip selection circuit 360 is located in the space between the filtering skip confirmation unit 350 and the filter unit 330 and in the space between the filtering skip confirmation unit 350 and the decoded picture buffer 335a. The filtering skip confirmation unit 350 controls the skip selection circuit 360 based on the selection information from the filtering skip flag, thereby adaptively performing deblocking filtering on the coding block, the prediction block or the transform block.

[0395] That is, the decoder of the embodiment can indicate a switching operation based on a flag sent from the encoder, which indicates whether to skip filtering. When filtering is skipped, the data is stored in DPB 335a, otherwise, filtering is performed on the data, and the data is output on a display device, etc., or the data is stored in DPB 335a. That is, after obtaining a reconstructed picture and filtering the reconstructed picture, the device for decoding an image can output the reconstructed picture to a display device, and the reconstructed picture can be stored in DPB 335a for use as a reference picture in inter-frame prediction.

[0396] Figures 21a to 21d is an exemplary view illustrating transmission in block units of various sizes in a method for encoding an image according to an embodiment of the present invention.

[0397] The flags used for the above filtering can be sent in blocks of different sizes. Fig.21a As shown, when the maximum coding unit is 64×64, a flag may be transmitted to the maximum coding unit. The maximum coding unit may correspond to a coding tree unit (CTU).

[0398] In addition, if Figure 21bAs shown, the maximum coding unit can be sent in segmented coding units. When the flag is generated in the segmented unit, tree-based segmentation is possible. In an embodiment, the maximum coding unit can be segmented into four sub-coding units, or at least one of the four sub-coding units can be further segmented into four sub-coding units.

[0399] In addition, if Fig.21c As shown, at least one maximum coding unit may be transmitted in a bundle unit. In an embodiment, four maximum coding units may be transmitted in one bundle unit.

[0400] In addition, if Fig.21d As shown, a combination of sending at least one maximum coding unit in a bundling unit and sending the maximum coding unit in a plurality of sub coding units may be possible.

[0401] In the above case, the segmentation method is quadtree-based segmentation as an example, but segmentation may be performed in various block sizes and shapes according to a segmentation method for segmenting a unit.

[0402] Information on the maximum size of a block and the minimum size of a block may be equally set in the encoder and the decoder, and the information thereon may be transmitted in units of a sequence, a picture, a slice, and the like.

[0403] Fig. 22 is an exemplary view illustrating transmission of a flag in a method for encoding an image according to an embodiment of the present invention.

[0404] refer to Fig. 22 , in the method for encoding or decoding an image according to an embodiment, a flag for adaptive filtering may be transmitted.

[0405] For example, segmentation is performed when at least one segmented block is referenced, that is, when there is any block to which a loop filter is not applied. When additional segmentation cannot be performed, an on / off flag for adaptively applying a loop filter may be sent. The depth for segmenting the block may be determined based on the maximum size of the block, the picture type, and the like. However, when the supported depth is 0, segmentation may not be performed. The information thereon may be sent in units such as sequences, pictures, and strips. In an embodiment, the maximum size of a block is 64×64, and the supported depth is up to 3 (8×8). Here, in the case of sending the segmentation information of a block, when the relevant flag has a value of 1, this indicates that the block is segmented, and when the relevant flag has a value of 0, this indicates that the block is not segmented. The segmentation information may be generated based on the Z scanning direction.

[0406] First, a flag 1 (64×64 partition) indicating partitioning for a 64×64 block (top of the tree structure) is sent, a flag 1 (32×32 partition) for the upper left block (32×32) is sent, and partitioning information of 4 sub-blocks (16×16) of flag 0 (16×16 determination), flag 0 (16×16 determination), flag 0 (16×16 determination), and flag 0 (16×16 determination) may be sent. Next, a flag 0 (32×32 determination) for the upper right block is sent, indicating that no additional partitioning is performed, and a flag 0 (32×32 determination) for the lower left block may be sent. A flag 1 (32×32 partition) for the lower right block (32×32) is sent, and a flag 1 for the upper left block (16×16) is sent. In the case of the upper left block (16×16), the size of its sub-block (8×8) is equal to the supported depth (8×8), and therefore the split flag for the sub-block (8×8) is not transmitted. Flag 0 (16×16 determination), flag 0 (16×16 determination), and flag 0 (16×16 determination) may be transmitted for the remaining blocks (16×16). That is, 1100000011000 may be transmitted as data indicating whether to perform splitting.

[0407] Through the above processing, information indicating whether to perform segmentation can be generated, and then information on whether to apply filtering can be generated. Through the above processing, flags 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1 and 0 indicating whether filtering is applied to 13 blocks can be sent in the Z scanning direction.

[0408] The bitstream structure of the segmentation flag for indicating a flag related to the segmentation information of the block can select one of the one or more scanning methods. As described above, the use of Z scanning can preferentially obtain additional segmentation information about the block segmented based on the initial block. Optionally, there is a Z-scan-based method that follows the depth order, in which the segmentation information at the current level depth based on the initial block is obtained, and then the segmentation information at the subsequent level depth is obtained. When the above method is used, flag 1 (64×64 segmentation), flag 1 (32×32 segmentation), flag 0 (32×32 determination), flag 0 (32×32 determination), flag 1 (32×32 segmentation), flag 0 (16×16 determination), flag 0 (16×16 determination), flag 0 (16×16 determination), flag 0 (16×16 determination), flag 1 (16×16 segmentation), flag 0 (16×16 determination), flag 0 (16×16 determination), and flag 0 (16×16 determination) can be sent. That is, 1100100001000 may be transmitted as data indicating whether to perform segmentation. Other scanning methods not shown in the above example may be used.

[0409] exist Fig. 22In the figure, a circle may represent a segmentation mark, a square may represent a filter application mark, a slash may represent application of segmentation or filtering, and no color may represent no application of segmentation or filtering.

[0410] In an embodiment, since no loop filter is applied, there is degradation between blocks in the data of the encoded current picture. The memory for storing data may be a temporary memory for encoding or decoding the current picture. When the degraded data is referenced by block matching, the accuracy of the prediction may be reduced due to the degradation between blocks. In this case, before the encoding of the block is completed and before proceeding to the subsequent block, filtering for reducing block degradation is applied at the boundary between the left and upper sides of the current block, thereby reducing block degradation.

[0411] In addition, the boundary between the right side and the bottom side can be processed in the subsequent block to be encoded. The block boundary to which the filter is to be applied can be determined according to preset conditions such as the type of block boundary, the minimum size of the block, etc. The filter can be performed on the boundary of the transform block, the boundary of the prediction block, or the block boundary common to the transform block and the prediction block. The minimum size of the block to perform the filter can be set, and the information thereon can be generated or transmitted in units such as sequences, pictures, and slices.

[0412] In addition, by analyzing the characteristics of the block at the block boundary, for example, the coding mode, block boundary characteristics, prediction information, coding coefficients, etc., it is possible to set whether to apply filtering, filtering application pixels, filter types, etc. The setting information related thereto can be set equally in the encoder and the decoder, and can be sent in units such as sequences, pictures, and slices. The filtering used in the above embodiments can use the same configuration as the existing deblocking filter or some other configuration.

[0413] Fig.23 is a flowchart illustrating a filtering process in a method for encoding an image and a method for decoding an image according to an embodiment of the present invention.

[0414] refer to Fig.23 , the method for encoding or decoding an image according to an embodiment may include a filtering step.

[0415] That is, the filtering step of the embodiment may include a block boundary confirmation step in S231 , a filtering strength setting step in S233 , a filtering application determination step in S235 , and a filtering application step in S237 .

[0416] The block boundary confirmation step in S231, the filter strength setting step in S233, the filter application determination step in S235, and the filter application step in S237 may be performed by a memory (refer to Figure 1 18) or a processor (reference Figure 1 The memory stores its program, program code or software module, and the processor is coupled to the memory to execute the program.

[0417] That is, in the device for encoding or decoding an image according to the embodiment, the filtering controller (which is a processor that executes a program code stored in an adjacent memory or sent from a remote memory, or a functional or structural part of the processor) may include: a block boundary confirmation unit implemented in the block boundary confirmation step in S231; a filtering strength setting unit implemented in the filtering strength setting step in S233; a filtering application determination unit implemented in the filtering application determination step in S235; and a filtering application unit implemented in the filtering application step in S237.

[0418] Hereinafter, the above filtering steps will be described in detail.

[0419] Fig.24 The diagram can be applied to Fig.23 An exemplary view of a prediction block or a transform block of a method for encoding an image.

[0420] refer to Fig.24 In the apparatus for encoding / decoding an image according to an embodiment, a block boundary confirmation unit implemented in the block boundary confirmation step in S231 compares a preset condition with a block boundary where filtering is to be applied. When the preset condition is met, filtering may be applied to the boundary, otherwise, filtering may not be applied. The preset condition may include setting information such as which block boundary, the minimum size of the block, etc.

[0421] In addition, the block boundary confirmation unit can perform filtering on the boundary between blocks performing transformation or on the boundary between blocks performing prediction. Setting information on whether to perform filtering on the transformation block boundary or filtering on the prediction block boundary or perform both in parallel can be set and sent in units such as sequences, pictures, and slices.

[0422] In addition, in the block boundary confirmation step S231, the minimum size of the block (M×N) for performing filtering can be set. This setting can be sent in units such as sequences, pictures, and strips. Through the block segmentation unit, the size and shape of the block of the loop filter (deblocking filter in an embodiment) can be determined. Optionally, it can be determined based on the size and shape of another type of block. For example, the minimum size of the block can be equal to or greater than the minimum size of the prediction unit or the transform unit. The prediction block or transform block is provided in sizes of 64×64, 32×32, 16×16, 8×8, 4×4, etc., and can be provided in a rectangular shape of M×N.

[0423] exist Fig.24In , the block boundaries (VEB1, VEB2, VEB3, VEB4) indicate the vertical boundaries to which filtering is applied. Fig.24 In , the block boundary may further include a horizontal boundary to which filtering is applied. Fig.24 In , when the maximum size of a block is 64×64, the minimum size of a block to which filtering is applied may be 16×16. In this case, the minimum size of a block may be set according to a color format of a chroma component or a luminance component or for each chroma component.

[0424] exist Fig.24 In the case where filtering is applied only to block boundaries of vertical edge boundaries, the first coding unit (CU#1) may be a 64×64 CU (coding unit) partitioned into two 32×64 prediction units (PUs), the second coding unit (CU#2) may be a 64×64 CU as one 64×64 prediction unit (PU), the third coding unit (CU#3) may be a 64×64 CU partitioned into two 32×64 prediction units (PUs), and the fourth coding unit (CU#4) may be a 64×64 CU partitioned into two 32×64 prediction units (PUs). When filtering is applied to horizontal edge boundaries, each coding unit except the second coding unit may be partitioned into a greater number of prediction units.

[0425] Fig.25 It is shown in Fig.23 An exemplary view of setting a boundary of a filtering strength in a method for encoding an image.

[0426] refer to Fig.25 In the apparatus for encoding / decoding an image according to an embodiment, the filter strength setting unit of the filter strength setting step implemented in S233 may set which filter to use by analyzing the characteristics of the blocks located on opposite sides of the block boundary. That is, the filter strength setting unit may determine the filter strength based on which filter to use.

[0427] That is, the filtering strength setting unit may use information such as a coding mode of a block, a block boundary characteristic, prediction information, a coding coefficient, etc. for setting the filtering strength. The block boundary characteristic may include information about whether a boundary between blocks is in a coding unit or a transform unit, and the prediction information may include information about an intra prediction mode, a motion vector, a reference picture, etc.

[0428] Based on the above information, the filter strength setting unit can determine which filter strength to apply to the block boundary or not apply filtering. That is, the filter strength setting unit can determine the filter strength based on the pixel value, but this is not restrictive, and the filter strength can be determined based on the information of the block.

[0429] When the filter strength is determined based on the pixel value, the pixel unit of the boundary of the filter strength can be at least one row or one column. Due to the block boundary characteristics, the filter strength can be determined proportionally according to the width and height of the smallest block to which the filter is to be applied. According to the implementation picture, the filter strength can be set in a fixed pixel unit in the strip. The information related thereto can be set equally in the encoder and decoder, and can be sent in units such as sequences, pictures, and strips.

[0430] In the case of chrominance components, such as Fig.25 As shown, by using the encoding information of two opposite side blocks (e.g., P block and Q block) with a block boundary (B5) therebetween, a boundary strength (BS) or filtering strength can be determined. For example, it is possible to refer to the luminance component of the P block pixels, that is, the filtering strength at the pixel corresponding to the element in the set {p(3,0), p(2,0), p(1,0), p(0,0), p(3,1), p(2,1), p(1,1), p(0,1), p(3,2), p(2,2), p(1,2), p(0,2), p(3,3), p(2,3), p(1,3), and p(0,3)}. Furthermore, it is possible to refer to the luminance components of the pixels of the Q block, i.e., the filter intensities at the pixels corresponding to elements in the set {q(3,0), q(2,0), q(1,0), q(0,0), q(3,1), q(2,1), q(1,1), q(3,2), q(2,2), q(1,2), q(0,2), q(3,3), q(2,3), q(1,3), and q(0,3)}.

[0431] For example, when the size of the two blocks is equal to or greater than the preset size, the luminance component is referenced, otherwise, the chrominance component is referenced to set the filter strength. In the case of the chrominance component, the type and number of filters supported may be the same or different from the type and number of filters supported by the luminance component. For example, in the luminance component, filtering is applied with intensities of 0, 1, and 2, and in the chrominance component, filtering is applied with intensities of 0 and 1.

[0432] Figures 26a to 26c It is shown in Fig.25 An exemplary view of a pattern of pixel values ​​within an image in a method for encoding an image.

[0433] In the apparatus for encoding / decoding an image according to the embodiment, the filtering strength setting unit sets the image based on two blocks (reference Fig.25After determining the primary filter strength based on information of the block on the opposite side of the block boundary B5), the filter application determination unit can determine whether to apply the filter strength obtained by the filter strength setting unit based on the reconstructed pixels at the block boundary.

[0434] like Fig.26a and 26b As shown, when there is a change or no change between adjacent pixels, the change between adjacent pixels is linear, and therefore filtering is applied to reduce quantization error.

[0435] At the same time, if Fig.26c As shown, in the case of a step edge shape between adjacent pixels (p2,0 and p1,0), filtering may distort the actual edge, and thus filtering may hinder coding efficiency. Therefore, in an embodiment, by checking the amount of change in pixels at the boundary of two blocks, it can be determined whether to apply filtering or it can be determined that filtering is applied to pixels.

[0436] Fig. 27 It is shown in Fig.23 Flowchart of a filter strength setting process in a method for encoding / decoding an image.

[0437] In the method for encoding / decoding an image according to an embodiment, it is assumed that a filter strength is set for a vertical boundary. The filter strength is assumed to range from 0 to 2.

[0438] refer to Fig. 27 In the apparatus for encoding / decoding an image that implements the method for encoding / decoding an image according to an embodiment, the filter controller may determine whether any one of the two blocks is intra-coded in step S271. When the determination result is "yes", the boundary strength (BS) or the filter strength may be set to 2 in step S272. Here, when the filter strength has a high number, it means that the filter strength is strong.

[0439] As a result of the determination in step S271, when two blocks are inter-coded, the filter controller compares the coding coefficients of the two blocks to determine the filter strength. That is, in step S273, the filter controller can determine whether a block (P or Q) has at least one non-zero coefficient to be encoded. When the determination result is "yes", the filter controller can set the boundary strength (BS) (i.e., the filter strength) to 1 in step S277.

[0440] Meanwhile, as a result of the determination in step S273, when the two blocks do not have coefficients to be encoded, the filter controller may determine whether the reference pictures (reference or ref) are identical or different from each other in step S274. When the determination result is "yes", the filter controller may set the boundary strength (BS) (ie, filtering strength) to 1 in step S277.

[0441] Meanwhile, as a result of the determination in step S274, when the two blocks do not have different reference pictures, the filter controller may determine whether the numbers of motion vectors of the two blocks are different from each other in step S275. When the determination result is "yes", the filter controller may set the boundary strength (BS) (ie, filtering strength) to 1 at step S277.

[0442] Meanwhile, as a result of the determination in step S275, when the number of motion vectors of the two blocks is the same, the filter controller may determine in step S276 whether the difference of the x and y components between the motion vectors is equal to or greater than a preset level. When the determination result is "yes", the filter controller may set the boundary strength (BS) (i.e., the filtering strength) to 1 in step S277. Assuming that the motion vector accuracy is 1 / 4, the number 4 in units of 1 / 4 may correspond to 1 in units of integers. Therefore, depending on the accuracy of the motion vector, the boundary strength or boundary value may be different. For example, when the motion vector accuracy of one of the two blocks is an integer and the motion vector accuracy of the other block is 1 / 4, step S276 may be performed by changing the motion vector accuracy of the block having integer units to 1 / 4. Alternatively, when the motion vector accuracy of one of the two blocks is 1 / 4 and the motion vector accuracy of the other block is 1 / 8, the difference setting of the x and y components at step S276 may be changed from 4 to 8. Alternatively, when the motion vector precisions of the two blocks are integers, step S276 may be performed by changing the motion vector precisions of the two blocks to 1 / 4.

[0443] Meanwhile, when the determination result in step S276 is "No", the filter controller may determine that the two blocks have high similarity and may set the boundary strength (BS) or the filter strength to 0. Here, the filter strength of 0 means that filtering is not performed.

[0444] In an embodiment, a series of steps from step S271 to S278 may be modified. For example, the determination of whether to use another reference picture in step S274 may be omitted by performing scaling according to the distance of the reference picture. The difference between the number of determined motion vectors in step S275 may be modified according to image features. In addition, unlike step S273, even if the coding coefficients of the two blocks have non-zero coefficients, they may be compared or determined under the same conditions as the case where the frequency coefficients are located in a specific part of the frequency band and are equal to or less than a preset level. Here, the case of being located in a specific part of the frequency band may include: a case of being located in the upper left corner of a low-frequency component including a DC (direct current) component in an N×N frequency block; and a case where a non-zero coefficient appears at a position such as <0, 0>, <1, 0>, and <0, 1> and does not appear at other positions, and modifications thereof are also possible.

[0445] Fig.28 It is shown in Fig.23 A flowchart of another example of a filter strength setting process in a method for encoding an image.

[0446] and Fig. 27 Compared with the embodiment of , this embodiment is an example in which the current picture is included as a reference picture and the inter-frame encoding method is used. Fig. 27 Descriptions of overlapping parts of the embodiments will be omitted.

[0447] refer to Fig.28 , the filter controller may determine whether any one of the two blocks (P and Q) is intra-coded at step S271. When the determination result is "yes", the boundary strength (BS) or the filter strength may be set to 2. When the result is "no", it may be determined in step S271a whether any one of the two blocks uses the current picture as a reference picture.

[0448] When the result of the determination in step S271a is "yes", the filter controller may set the boundary strength (BS) to 2. Meanwhile, when the result is "no", the filter controller may determine whether a block (P or Q) has at least one non-zero coefficient to be encoded in step S273. When the result of the determination at step S273 is "yes", the filter controller may set the boundary strength (BS) to 2. When the result of the determination in step S273 is "no", the filter controller may determine whether a block (P or Q) has at least one non-zero coefficient to be encoded in step S273. Fig. 27 Execute steps S274 to S278 in the order shown.

[0449] As described above, in the method for encoding / decoding an image according to an embodiment, the filtering strength may be different depending on which picture the reference picture of the block is. Meanwhile, in an embodiment, when two blocks are inter-coded, it may be determined whether a block uses the current picture as a reference picture, but the present invention is not limited thereto and modifications are possible.

[0450] Fig.29 It is shown in Fig.23 A flowchart of still another example of a filter strength setting process in a method for encoding / decoding an image.

[0451] refer to Fig.29 In an apparatus for encoding / decoding an image that implements a method for encoding / decoding an image according to an embodiment, when two blocks have a current picture as a reference picture, the filtering controller may set a boundary strength (BS) or a filtering strength to 1 in step S276a.

[0452] The filtering process of this embodiment can be basically equivalent to Fig. 27In an embodiment, after determining in step S276 whether the difference between the x and y components of the motion vectors is equal to or greater than a preset level of at least one of the two blocks and the result is "no", determining in step S276a whether the two blocks (P and Q) use the current picture as a reference picture (ref) is performed.

[0453] Fig.30 It is shown in Fig.23 A flowchart of still another example of a filter strength setting process in a method for encoding / decoding an image.

[0454] The present embodiment is an example in which the current picture is included as a reference picture and the intra encoding method is used.

[0455] refer to Fig.30 , in the apparatus for encoding / decoding an image that implements the method for encoding / decoding an image according to the embodiment, the filtering controller may determine whether either of the two blocks is intra-coded in step S271.

[0456] When the determination result in step S271 is "No", the filter controller may determine whether a block (P or Q) has at least one non-zero coefficient in step S273. When the determination result in step S273 is "No", the filter controller may determine whether the reference pictures (reference or ref) are the same or different from each other in step S274. When the determination result in step S274 is "No", the filter controller may determine whether the numbers of motion vectors of the two blocks are different from each other in step S275. When the determination result in step S275 is "No", the filter controller may determine whether the difference between the x and y components of the motion vectors is equal to or greater than a preset level in step S276. When the determination result in step S276 is "No", the filter controller may determine that the two blocks have high similarity, and may set the boundary strength (BS) or the filter strength to 0 in step S278. Meanwhile, when the determination results in steps S273 , S274 , S275 , and S276 are “Yes,” the filter controller may set the boundary strength (BS) (ie, filter strength) to 1 in step S277 .

[0457] Meanwhile, when the determination result in step S271 is "yes", the filter controller may determine whether the two blocks (P and Q) perform intra-block matching (intraMB) in step S271b. When the determination result in step S271b is "no", the filter controller may set the boundary strength (BS) or the filter strength to 2 at step S272.

[0458] Meanwhile, when the determination result in step S271b is "yes", the filter controller may determine whether one block (P or Q) has at least one non-zero coefficient to be encoded in step S273b. When the determination result in step S273b is "no", the filter controller may determine whether the numbers of motion vectors of the two blocks are different from each other in step S275b. When the determination result in step S275b is "no", the filter controller may determine whether the difference of the x and y components between the motion vectors is equal to or greater than a preset level in step S276b. When the determination result in step S276b is "no", the filter controller may determine that the two blocks have high similarity, and may set the boundary strength (BS) or the filtering strength to 0 in step S278. When the determination results in steps S273b, S275b, and S276b are "yes", the filter controller may set the boundary strength (BS) (i.e., filtering strength) to 1 in step S277.

[0459] According to an embodiment, when either of the two blocks is intra-coded and the two blocks are not coded by block matching in the current picture, the filtering strength may be set to 2. In addition, when the two blocks are coded by block matching in the current picture and either of the two blocks has a non-zero coefficient to be coded, the filtering strength is set to 1. In addition, when the two blocks do not have coefficients to be coded and the numbers of motion vectors of the two blocks are different from each other, the filtering strength may be set to 1. In addition, when the numbers of motion vectors of the two blocks are the same and the difference between the x and y components of each motion vector is equal to or greater than a preset level, the filtering strength may be set to 1. When the difference is less than the preset level, the similarity between the two blocks is high and the filtering strength is set to 0. That is, in an embodiment, the filtering strength may be set differently depending on which reference picture is used.

[0460] Figure 31 to Figure 33 It is shown in Fig.23 An exemplary view of a filter application determination process in a method for encoding an image.

[0461] refer to Figure 31 to Figure 33 , the filter controller can calculate the amount of change in each block and the amount of change at the two block boundaries to determine the filter application. The left block of the two blocks on opposite sides of the block boundary (B5) is called a P block, and the right block is called a Q block.

[0462] The changes in P blocks are shown below.

[0463] dp0=|p(0,0)-p(1,0)x 2+p(2,0)|

[0464] dp1=|p(0,1)-p(1,1)x 2+p(2,1)|

[0465] dp2=|p(0,2)-p(1,2)x 2+p(2,2)|

[0466] dp3=|p(0,3)-p(1,3)x 2+p(2,3)|

[0467] In the variation, dpx (x includes 0, 1, 2, or 3) indicates the variation in the corresponding pixel.

[0468] The changes in the Q block are shown below.

[0469] dq0=|q(0,0)-q(1,0)x 2+q(2,0)|

[0470] dq1=|q(0,1)-q(1,1)x 2+q(2,1)|

[0471] dq2=|q(0,2)-q(1,2)x 2+q(2,2)|

[0472] dq3=|q(0,3)-q(1,3)x 2+q(2,3)|

[0473] In the above-mentioned variation, dqx (x includes 0, 1, 2 or 3) indicates the variation in the corresponding pixel.

[0474] The amount of change in the block boundary (B5) between the P block and the Q block is as follows.

[0475] dpq0=|p(0,0)-q(0,0)|

[0476] dpq1=|p(0,1)-q(0,1)|

[0477] dpq2=|p(0,2)-q(0,2)|

[0478] dpq3=|p(0,3)-q(0,3)|

[0479] In the above-mentioned variation, dpqx (x includes 0, 1, 2 or 3) indicates the variation in the corresponding pixel.

[0480] As described above, the variation can be measured in units of each pixel line, or can be measured in units of at least two pixel lines. In the case of whether filtering is applied in units of bundled pixel lines, in subsequent steps, the filtering strength, filtering application pixels, filter coefficients, etc. can be determined.

[0481] At least one variation may be considered, and according to an embodiment, in addition to the above three variations, further variations may be considered. When the above three variations are considered, it may be determined whether to apply filtering by comparing the variations with respective boundary values.

[0482] For example, when limited to the first line for illustration, the change in the P block, the change in the Q block, and the change at the PQ block boundary (B5) are less than the boundary values ​​(threshold_P, threshold_Q, and threshold_PQ), filtering is applied, and when they are greater than the boundary values, filtering may not be applied.

[0483] As another example, when the amount of change in the block boundary is greater than the boundary value and the amount of change in at least one of the blocks is less than the boundary value, filtering may not be applied.

[0484] As another example, when the change amount of the P block is greater than the boundary value and the change amount of the Q block and the change amount of the PQ block boundary are less than the boundary value, the step edge portion where the change amount of the P block occurs is detected, and therefore filtering is performed only on the portion at the step edge in the detected Q block and P block.

[0485] As described above, a limited filter may be applied in the filter application unit. In this case, the boundary value may be set by a quantization parameter, a bit depth, etc. When the quantization parameters of two blocks are different, one of the quantization parameters of the two blocks may be selected or the average value thereof may be used. When the boundary value is set by the average value, rounding may be used to process the decimal unit.

[0486] In addition, whether to apply filtering can be determined based on pixel values. In this case, whether to apply filtering can be determined by using the encoding information of the two blocks. In this case, Fig.31 As shown, the encoding modes of the two blocks may be intra modes, and the prediction mode information of the two blocks may be the same. The boundary to which filtering is applied may be a horizontal edge boundary, or may be a boundary obtained by two blocks being included in one prediction unit but having different transform units.

[0487] In addition, if Fig.32 As shown, through the pattern of the two blocks, the filter controller can determine that the two blocks are likely to be blocks with continuous correlation in the horizontal direction. In this case, different from Fig.31 As shown in Figure 2, the filter controller can determine the application filter without measuring the change.

[0488] In addition, if Fig.33 As shown, when the encoding mode of two blocks is intra mode and the prediction mode information of the two blocks is the same, the filter controller can measure the above-mentioned change amount and determine to apply filtering in pixel units due to the high possibility of the step edge occurring.

[0489] According to an embodiment, a filter controller or an apparatus for encoding / decoding an image including the filter controller may determine whether to apply filtering by considering the correlation between the two blocks based on the encoding modes of the two blocks, prediction mode information, etc. Since the encoding coefficient may affect the filter application, the filter controller may consider the encoding coefficient.

[0490] In addition, when the coding mode of the two blocks is an intra-frame mode, and in the prediction mode of the two blocks, the left block is a DC mode and the right block is a horizontal mode, and when the left block has at least one non-zero coefficient and the right block does not have a non-zero coefficient, the left block can be determined as a complex area and the amount of change of the block may be equal to or greater than the boundary value set above. However, in this case, since the image characteristics indicate a complex area rather than an edge area, the amount of change may be higher. Therefore, filtering can be applied even when the amount of change exceeds the boundary value. As described above, in addition to measuring the amount of change, prediction mode information, coding coefficients, etc. can also be checked to determine whether filtering is applied.

[0491] Meanwhile, in the above embodiments, the filtering strengths of 0, 1 and 2 are used as examples, but smaller or larger strengths may also be supported. This may be modified according to the settings of the encoder / decoder and may be sent in units such as sequences, pictures, and strips.

[0492] Reference again Fig.23 , the filter application unit implementing the filter application step at S237 can apply the filter according to the determined filter strength and the result of applying the filter. The filter application unit can determine the filter and the pixels to be applied for filtering according to the determined filter strength and through pixel-based analysis in addition. This determination can be performed after the advancement of the pixel line applied in the previous stage.

[0493] In addition, the filter application unit can apply filtering by analyzing the amount of change of pixels at the block boundary. For example, the filter strength and the applied pixels can be determined based on the pixel change amounts in the following 1) to 3) (the amount of change at the P block and the Q block, and the amount of change at the PQ boundary). According to the result of whether the amount of change is greater than or less than the boundary value, at least one filter is selected for the application of filtering. Here, different boundary values ​​can be applied to respective situations. Here, the boundary value can be determined based on the quantization parameters, bit depth, etc. of the two blocks. The filter may include, for example, a strong filter, a weak filter, etc.

[0494] 1)|p(0,0)-p(1,0)x 2+p(2,0)|+|q(0,0)-q(1,0)x 2+q(2,0)|

[0495] 2)|p(0,0)-p(3,0)|+|q(0,0)-q(3,0)|

[0496] 3)|p(0,0)-q(0,0)|

[0497] Here, the filter application unit can be determined based on the number of pixels applied in the filtering or the above results. That is, the filter application unit can apply filtering to preset candidate pixels, or can set pixels that do not perform filtering based on the above results. The filter can have the characteristics of a low-pass filter in the form of FIR (finite impulse response). The input pixel of the filter can be adaptively determined according to the filter strength and / or the position of the pixel.

[0498] In addition, the filter application unit can determine the application pixels of the filter according to the coding mode of the two blocks. Usually, at the horizontal boundary of the block, the filter is applied to the pixels in the vertical direction, and at the vertical boundary, the filter is applied to the pixels in the horizontal direction. However, in an embodiment, when the coding mode of the two blocks is an intra-frame mode and the prediction mode of the two blocks is a vertical mode, and when the boundary of the current block is a vertical boundary, the horizontal pixels at the horizontal boundary can be filtered to reduce the quantization error.

[0499] In addition, due to the high vertical correlation between pixels in two blocks, when filtering is performed on pixels in the vertical direction, the quantization error can be effectively reduced compared to the case where filtering is performed in the horizontal direction. In this case, as shown in the following 1) and 2), the pixel variation of the pixels in the vertical direction of the vertical boundary can be obtained. According to the result of whether the variation is greater than or less than the boundary value, at least one filter is selected and applied. In addition, the number of pixels applied in filtering can be determined based on the above results.

[0500] 1)|p(0,0)-p(0,1)|+|p(0,1)-p(0,2)|+|p(0,2)-p(0,3)|

[0501] 2)|q(0,0)-q(0,1)|+|q(0,1)-q(0,2)|+|q(0,2)-q(0,3)|

[0502] Furthermore, in order to reduce block degradation between two blocks, the filter application unit may apply filtering by measuring an amount of change between pixels at a block boundary.

[0503] According to an embodiment, when a prediction block is generated by performing block matching on a current picture, its strength may be determined according to whether the deblocking filter is processed in an existing intra prediction mode or in an inter prediction mode. When processing in the inter prediction mode, in addition to conditions that affect the determination of the deblocking filter strength such as the coding modes of the two blocks, whether there is the same reference picture, the size of the motion vector, etc., the filtering strength may also be determined based on the reference picture index.

[0504] In an embodiment, when adaptive deblocking filtering is used in a process of decoding a coded image, the method for encoding an image may be replaced by a method for decoding an image. In addition, the method for encoding / decoding an image may be performed by an image processing device or a device for encoding and decoding an image, wherein the device has at least one component for encoding and decoding or a component for performing a function corresponding to the component.

[0505] According to the embodiment, a high-performance and high-efficiency image encoding / decoding technology is provided, which can be generally used for international codecs such as MPEG-2, MPEG-4, H.264, etc., other codecs, media using these codecs, and image industries using intra-frame prediction technology. In addition, in the future, the present invention is expected to be applied to the field of image processing using standard codecs such as existing high-efficiency video coding (HEVC) technology, and H.264 / AVC and intra-frame prediction.

[0506] Although exemplary embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Claims

1. A method for image decoding that performs adaptive filtering, the method comprising: Determine boundary strength; and performing deblocking filtering on the current block and adjacent blocks adjacent to the current block based on the boundary strength, wherein, when neither the current block nor the neighboring block is predicted using intra prediction, the boundary strength is determined based on whether both the current block and the neighboring block use a current picture as a reference picture and whether the number of motion vectors of the current block is different from the number of motion vectors of the neighboring block, wherein, when both the current block and the neighboring block use the current picture as the reference picture and the number of motion vectors of the current block is different from the number of motion vectors of the neighboring block, the boundary strength is determined to be 1, The boundary strength being 1 indicates that the deblocking filtering is performed on the current block and the neighboring block. 2 . The method of claim 1 , wherein the boundary strength is further determined based on a color component of the current block. 3 . The method according to claim 2 , wherein when the color component of the current block is a luminance component and both the current block and the neighboring block use the current picture as the reference picture, the boundary strength is determined to be 1.

4. A method for image encoding performing adaptive filtering, the method comprising: Determine boundary strength; and performing deblocking filtering on the current block and adjacent blocks adjacent to the current block based on the boundary strength, wherein, when neither the current block nor the neighboring block is predicted using intra prediction, the boundary strength is determined based on whether both the current block and the neighboring block use a current picture as a reference picture and whether the number of motion vectors of the current block is different from the number of motion vectors of the neighboring block, wherein, when both the current block and the neighboring block use the current picture as the reference picture and the number of motion vectors of the current block is different from the number of motion vectors of the neighboring block, the boundary strength is determined to be 1, The boundary strength being 1 indicates that the deblocking filtering is performed on the current block and the neighboring block.

5. A non-transitory computer-readable medium for transmitting video data, the video data comprising information indicating whether to encode the corresponding block using a picture including the corresponding block as a reference, During decoding, when neither the current block nor a neighboring block adjacent to the current block is predicted using intra prediction, information about the current block and the neighboring blocks is used to determine whether both the current block and the neighboring blocks use the current picture as a reference picture. in, The boundary strength for deblocking filtering is determined based on whether both the current block and the neighboring block use the current picture as the reference picture and whether the number of motion vectors of the current block is different from the number of motion vectors of the neighboring block, wherein, when both the current block and the neighboring block use the current picture as the reference picture and the number of motion vectors of the current block is different from the number of motion vectors of the neighboring block, the boundary strength is determined to be 1, The boundary strength being 1 indicates that the deblocking filtering is performed on the current block and the neighboring block.

Images