An image encoding / decoding method for processing a virtual boundary as a picture boundary, a method for transmitting a bitstream, and a recording medium storing the bitstream

Abstract

An image encoding / decoding method, a bitstream transmission method, and a computer-readable recording medium for storing a bitstream are provided. An image decoding method according to the present disclosure is an image decoding method performed by an image decoding apparatus, and includes steps of: obtaining first information regarding a virtual boundary from a bitstream; determining, based on the first information, whether a virtual boundary of a reference picture is to be processed as a picture boundary; and padding a partial region within the reference picture based on the virtual boundary of the reference picture being processed as a picture boundary.

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method, a method for transmitting a bitstream, and a recording medium storing the bitstream, and more particularly, to an image encoding / decoding method that processes a virtual boundary as a picture boundary, a method for transmitting a bitstream, and a recording medium storing the bitstream.

Background Art

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, has been increasing in various fields. As the image data becomes higher in resolution and quality, the amount of information or bits to be transmitted relatively increases compared to conventional image data. The increase in the amount of information or bits to be transmitted results in an increase in transmission costs and storage costs.

[0003] Accordingly, there is a need for a highly efficient image compression technique for effectively transmitting, storing, and reproducing information of high-resolution, high-quality images.

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0005] Another object of the present disclosure is to process a virtual boundary as a picture boundary.

[0006] Another object of the present disclosure is to propose a method of processing a virtual boundary as a picture boundary according to the direction of the virtual boundary.

[0007] Another object of the present disclosure is to propose a syntax element for processing a virtual boundary as a picture boundary.

[0008] In addition, an object of the present disclosure is to provide a non-transitory computer-readable recording medium that stores a bitstream generated by the image encoding method according to the present disclosure.

[0009] In addition, an object of the present disclosure is to provide a non-transitory computer-readable recording medium that stores a bitstream received by the image decoding apparatus according to the present disclosure, decoded, and used for restoring an image.

[0010] In addition, an object of the present disclosure is to provide a method for transmitting a bitstream generated by the image encoding method according to the present disclosure.

[0011] The technical problems to be solved in the present disclosure are not limited to the above-described technical problems, and other technical problems not described above will be clearly understood by those of ordinary skill in the technical field to which the present disclosure pertains from the following description.

Means for Solving the Problems

[0012] An image decoding method according to an aspect of the present disclosure is an image decoding method performed by an image decoding apparatus, the method including: obtaining first information regarding a virtual boundary from a bitstream; determining, based on the first information, whether a virtual boundary of a reference picture is to be processed as a picture boundary; and padding a partial region within the reference picture based on the virtual boundary of the reference picture being processed as the picture boundary.

[0013] An image encoding method according to another aspect of the present disclosure is an image encoding method performed by an image encoding apparatus, the method including: determining whether to process a virtual boundary of a reference picture as a picture boundary; and padding a partial area in the reference picture based on the virtual boundary of the reference picture being processed as a picture boundary. First information indicating whether to process the virtual boundary of the reference picture as a picture boundary may be encoded in a bitstream.

[0014] A computer-readable recording medium according to another aspect of the present disclosure can store a bitstream generated by the image encoding method or apparatus of the present disclosure.

[0015] A transmission method according to another aspect of the present disclosure can transmit a bitstream generated by the image encoding method or apparatus of the present disclosure.

[0016] The features briefly summarized and described above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure to be described later, and do not limit the scope of the present disclosure.

Advantages of the Invention

[0017] According to the present disclosure, it is possible to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0018] Also, according to the present disclosure, it is possible to provide a method of processing a virtual boundary as a picture boundary.

[0019] Also, according to the present disclosure, it is possible to expand a search area for motion estimation and compensation and improve the accuracy of inter prediction.

[0020] Also, according to the present disclosure, it is possible to provide a non-transitory computer-readable recording medium that stores a bitstream generated by the image encoding method according to the present disclosure.

[0021] Further, according to the present disclosure, there can be provided a non-transitory computer-readable recording medium that stores a bitstream received by the image decoding apparatus according to the present disclosure, decoded, and used for restoring an image.

[0022] Also, according to the present disclosure, there can be provided a method of transmitting a bitstream generated by an image encoding method.

[0023] The effects obtained by the present disclosure are not limited to the above-described effects, and other effects not described above will be clearly understood by those having ordinary knowledge in the technical field to which the present disclosure pertains from the following description.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Best Mode for Carrying Out the Invention

[0025] Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those having ordinary knowledge in the technical field to which the present disclosure pertains can easily implement them. However, the present disclosure can be realized in various different forms and is not limited to the embodiments described herein.

[0026] In describing the embodiments of the present disclosure, when it is determined that a specific description of a known configuration or function may obscure the gist of the present disclosure, the detailed description thereof will be omitted. And in the drawings, parts not related to the description of the present disclosure are omitted, and the same reference numerals are given to the same parts.

[0027] In the present disclosure, when a certain component is “connected”, “coupled” or “connected” to another component, this can include not only a direct connection relationship but also an indirect connection relationship in which another component exists between them. Also, when a certain component “includes” or “has” another component, this means that, unless otherwise stated to the contrary, it does not exclude other components but can further include other components.

[0028] In the present disclosure, terms such as “first” and “second” are used only for the purpose of distinguishing one component from another and do not limit the order or importance between components unless otherwise specified. Therefore, within the scope of the present disclosure, the first component of one embodiment may be called the second component in another embodiment, and similarly, the second component of one embodiment may be called the first component in another embodiment.

[0029] In the present disclosure, the components that are distinguished from each other are for clearly explaining their respective features, and do not necessarily mean that the components are separated. That is to say, a plurality of components may be integrated and constituted by one hardware or software unit, or one component may be distributed and constituted by a plurality of hardware or software units. Therefore, even without separate mention, such integrated or distributed embodiments are also included in the scope of the present disclosure.

[0030] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some may be optional components. Therefore, embodiments constituted by a subset of the components described in one embodiment are also included in the scope of the present disclosure. In addition, embodiments that further include other components in the components described in various embodiments are also included in the scope of the present disclosure.

[0031] The present disclosure relates to image encoding and decoding, and the terms used in the present disclosure can have the ordinary meanings in the technical field to which the present disclosure belongs unless newly defined in the present disclosure.

[0032] In the present disclosure, "picture" generally means a unit indicating any one image in a specific time period, and a slice / tile is an encoding unit constituting a part of a picture, and one picture can be constituted by one or more slices / tiles. In addition, a slice / tile can include one or more CTUs (coding tree units).

[0033] In the present disclosure, "pixel" or "pel" can mean the smallest unit that constitutes a picture (or image). Also, the term "sample" can be used as a term corresponding to a pixel. A sample can generally indicate a pixel or a pixel value, and can also indicate only the pixel / pixel value of the luma component, or can also indicate only the pixel / pixel value of the chroma component.

[0034] In the present disclosure, "unit" can indicate the basic unit of image processing. A unit can include at least one of a specific region of a picture and information related to the region. A unit can, in some cases, be used interchangeably with terms such as "sample array", "block", or "area". In general, an M×N block can include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0035] In the present disclosure, "current block" can mean any one of "current coding block", "current coding unit", "block to be coded", "block to be decoded", or "block to be processed". When prediction is performed, "current block" can mean "current prediction block" or "block to be predicted". When transformation (inverse transformation) / quantization (inverse quantization) is performed, "current block" can mean "current transformation block" or "block to be transformed". When filtering is performed, "current block" can mean "block to be filtered".

[0036] In the present disclosure, unless explicitly stated as a chroma block, the "current block" can mean a block that includes all luma component blocks and chroma component blocks or the "luma block of the current block". The luma component block of the current block can be explicitly expressed as including an explicit description of the luma component block, such as "luma block" or "current luma block". Also, the chroma component block of the current block can be explicitly expressed as including an explicit description of the chroma component block, such as "chroma block" or "current chroma block".

[0037] In the present disclosure, " / " and "," can be interpreted as "and / or". For example, "A / B" and "A, B" can be interpreted as "A and / or B". Also, "A / B / C" and "A, B, C" can mean "at least one of A, B, and / or C".

[0038] In the present disclosure, "or" can be interpreted as "and / or". For example, "A or B" can mean 1) only "A", 2) only "B", or 3) "A and B". Alternatively, in the present disclosure, "or" can mean "additionally or alternatively".

[0039] Overview of Video Coding System

[0040] FIG. 1 schematically shows a video coding system to which an embodiment according to the present disclosure can be applied.

[0041] A video coding system according to an embodiment can include an encoding device 10 and a decoding device 20. The encoding device 10 can transmit encoded video and / or image information or data to the decoding device 20 in a file or streaming format via a digital storage medium or a network.

[0042] The encoding device 10 according to one embodiment can include a video source generation unit 11, an encoding unit 12, and a transmission unit 13. The decoding device 20 according to one embodiment can include a reception unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 can be referred to as a video / image encoding unit, and the decoding unit 22 can be referred to as a video / image decoding unit. The transmission unit 13 can be included in the encoding unit 12. The reception unit 21 can be included in the decoding unit 22. The rendering unit 23 can also include a display unit, and the display unit can be configured as a separate device or an external component.

[0043] The video source generation unit 11 can obtain a video / image through processes such as capture, synthesis, or generation of the video / image. The video source generation unit 11 can include a video / image capture device and / or a video / image generation device. The video / image capture device can include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generation device can include, for example, a computer, a tablet, and a smartphone, etc., and can (electronically) generate a video / image. For example, a virtual video / image can be generated through a computer or the like, and in this case, the video / image capture process can be replaced by a process in which related data is generated.

[0044] The encoding unit 12 can encode the input video / image. The encoding unit 12 can perform a series of procedures such as prediction, transformation, quantization, etc. for compression and encoding efficiency. The encoding unit 12 can output the encoded data (encoded video / image information) in the form of a bitstream.

[0045] The transmission unit 13 can acquire the encoded video / image information or data output in bitstream format, and transmit this to the receiving unit 21 of the decoding device 20 or other external objects via a digital storage medium or network in file or streaming format. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray (registered trademark), HDD, SSD, etc. The transmission unit 13 can include elements for generating a media file via a predetermined file format, and can include elements for transmission via a broadcast / communication network. The transmission unit 13 can be provided as a transmission device separate from the encoding device 12. In this case, the transmission device can include at least one processor that acquires the encoded video / image information or data output in bitstream format, and a transmission unit that transmits this in file or streaming format. The receiving unit 21 can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit 22.

[0046] The decoding unit 22 can decode the video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit 12.

[0047] The rendering unit 23 can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0048] Overview of Image Encoding Apparatus

[0049] FIG. 2 is a diagram schematically showing an image encoding device to which an embodiment according to the present disclosure can be applied.

[0050] As shown in FIG. 2, the image encoding apparatus 100 can include an image division unit 110, a subtraction unit 115, a conversion unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse conversion unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190. The inter prediction unit 180 and the intra prediction unit 185 can be collectively referred to as a "prediction unit". The conversion unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse conversion unit 150 can be included in a residual processing unit. The residual processing unit can further include the subtraction unit 115.

[0051] All or at least a part of the plurality of components constituting the image encoding apparatus 100 can be realized by one hardware component (for example, an encoder or a processor) according to an embodiment. Further, the memory 170 can include a DPB (decoded picture buffer) and can be realized by a digital storage medium.

[0052] The image segmentation unit 110 can divide an input image (or picture, frame) input to the image encoding apparatus 100 into one or more processing units. As an example, the processing unit can be called a coding unit (CU). The coding unit can be obtained by recursively dividing a coding tree unit (CTU) or a largest coding unit (LCU) according to a QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structure. For example, one coding unit can be divided into a plurality of coding units at a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the division of the coding unit, the quadtree structure can be applied first, and the binary tree structure and / or the ternary tree structure can be applied later. Based on the final coding unit that cannot be divided further, the coding procedure according to the present disclosure can be performed. The largest coding unit can be used as the final coding unit, and the coding units at a lower depth obtained by dividing the largest coding unit can also be used as the final coding unit. Here, the coding procedure can include procedures such as prediction, transformation, and / or restoration described later. As another example, the processing unit of the coding procedure can be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit can be divided or partitioned from the final coding unit, respectively. The prediction unit can be a unit of sample prediction, and the transformation unit can be a unit for deriving transformation coefficients and / or a unit for deriving a residual signal from the transformation coefficients.

[0053] The prediction unit (inter prediction unit 180 or intra prediction unit 185) can perform a prediction on a processing target block (current block) and generate a predicted block that includes prediction samples for the current block. The prediction unit can determine whether intra prediction is applied in units of the current block or CU, or whether inter prediction is applied. The prediction unit can generate various information related to the prediction of the current block and transmit it to the entropy encoding unit 190. The information related to the prediction can be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0054] The intra prediction unit 185 can predict the current block by referring to samples within the current picture. The samples to be referred to can be located in the neighborhood of the current block or at a distance according to the intra prediction mode and / or intra prediction technique. The intra prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes can include, for example, the DC mode and the Planar mode. The directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes according to the degree of fineness of the prediction direction. However, this is only an example, and more or fewer directional prediction modes can be used based on the setting. The intra prediction unit 185 can also determine the prediction mode to be applied to the current block using the prediction mode applied to the neighboring blocks.

[0055] The inter prediction unit 180 can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, neighboring blocks can include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same as or different from each other. The temporal neighboring block can be called by names such as a collocated reference block, a collocated CU (colCU), etc. The reference picture including the temporal neighboring block can be called a collocated picture (colPic). For example, the inter prediction unit 180 can construct a motion information candidate list based on neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction can be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter prediction unit 180 can use the motion information of neighboring blocks as the motion information of the current block. In the case of skip mode, unlike merge mode, the residual signal cannot be transmitted.In the case of the motion information prediction (motion vector prediction, MVP) mode, the motion vectors of neighboring blocks are used as motion vector predictors, and the motion vector difference and an indicator for the motion vector predictor are encoded to signal the motion vector of the current block. The motion vector difference can mean the difference between the motion vector of the current block and the motion vector predictor.

[0056] The prediction unit can generate a prediction signal based on various prediction methods and / or prediction techniques described below. For example, the prediction unit can not only apply intra prediction or inter prediction for predicting the current block, but also apply intra prediction and inter prediction simultaneously. A prediction method that applies intra prediction and inter prediction simultaneously for predicting the current block can be called CIIP (combined inter and intra prediction). Also, the prediction unit can perform intra block copy (IBC) for predicting the current block. Intra block copy can be used for content image / video coding such as games, for example, like SCC (screen content coding). IBC is a method of predicting the current block using a restored reference block within the current picture at a position a predetermined distance away from the current block. When IBC is applied, the position of the reference block within the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction within the current picture, but can be performed in the same way as inter prediction in terms of deriving a reference block within the current picture. That is, IBC can use at least one of the inter prediction techniques described in the present disclosure.

[0057] The prediction signal generated by the prediction unit can be used to generate a restored signal or can be used to generate a residual signal. The subtraction unit 115 can subtract the prediction signal (predicted block, predicted sample array) output from the prediction unit from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual block, residual sample array). The generated residual signal can be transmitted to the conversion unit 120.

[0058] The conversion unit 120 can apply a conversion technique to the residual signal to generate transform coefficients. For example, the conversion technique can include at least one of DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT means the conversion obtained from this graph when the relationship information between pixels is represented by a graph. CNT means the conversion obtained based on generating a prediction signal using all previously reconstructed pixels. The conversion process can also be applied to a pixel block having the same size of a square, and can also be applied to a non-square, variable-size block.

[0059] The quantization unit 130 can quantize the transform coefficients and transmit them to the entropy encoding unit 190. The entropy encoding unit 190 can encode the quantized signal (information regarding the quantized transform coefficients) and output it in the form of a bitstream. The information regarding the quantized transform coefficients can be called residual information. The quantization unit 130 can reorder the quantized transform coefficients in block form into a one-dimensional vector form based on the coefficient scan order, and can also generate the information regarding the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0060] The entropy encoding unit 190 can perform various encoding methods such as, for example, exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding), etc. The entropy encoding unit 190 can also encode, together or separately, information necessary for video / image restoration (such as the values of syntax elements, etc.) in addition to the quantized transform coefficients. The encoded information (such as the encoded video / image information) can be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information can further include information regarding various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Also, the video / image information can further include general constraint information. The signaling information, transmitted information, and / or syntax elements referred to in the present disclosure can be encoded through the above-described encoding procedure and included in the bitstream.

[0061] The bitstream can be transmitted via a network or stored in a digital storage medium. Here, the network can include a broadcast network and / or a communication network, etc., and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray (registered trademark), HDD, SSD, etc. A transmission unit (not shown) for transmitting and / or a storage unit (not shown) for storing the signal output from the entropy encoding unit 190 can be provided as internal / external elements of the image encoding apparatus 100, or the transmission unit can also be provided as a component of the entropy encoding unit 190.

[0062] The quantized transform coefficients output from the quantization unit 130 can be used to generate a residual signal. For example, by applying inverse quantization and inverse transformation to the quantized transform coefficients via the inverse quantization unit 140 and the inverse transformation unit 150, a residual signal (residual block or residual sample) can be restored.

[0063] The addition unit 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the restored residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185. When there is no residual for the block to be processed as in the case where the skip mode is applied, the predicted block can be used as the reconstructed block. The addition unit 155 can be called a restoration unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next block to be processed within the current picture and can also be used for inter prediction of the next picture after passing through filtering as described later.

[0064] The filtering unit 160 can apply filtering to the restored signal to improve the subjective / objective image quality. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and can store the modified restored picture in the memory 170, specifically in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, and the like. The filtering unit 160 can generate various information related to filtering as described later in the description of each filtering method and transmit it to the entropy encoding unit 190. The information related to filtering can be encoded by the entropy encoding unit 190 and output in the form of a bit stream.

[0065] The modified restored picture transmitted to the memory 170 can be used as a reference picture in the inter prediction unit 180. When inter prediction is applied through this, the image encoding device 100 can avoid prediction mismatches between the image encoding device 100 and the image decoding device, and can also improve the encoding efficiency.

[0066] The DPB in the memory 170 can store the modified restored picture for use as a reference picture in the inter prediction unit 180. The memory 170 can store the motion information of the blocks for which the motion information in the current picture has been derived (or encoded) and / or the motion information of the blocks in the already restored picture. The stored motion information can be transmitted to the inter prediction unit 180 for utilization as the motion information of the spatial neighboring blocks or the motion information of the temporal neighboring blocks. The memory 170 can store the restored samples of the restored blocks in the current picture and transmit them to the intra prediction unit 185.

[0067] Overview of Image Decoding Apparatus

[0068] FIG. 3 is a diagram schematically showing an image decoding apparatus to which an embodiment according to the present disclosure can be applied.

[0069] As shown in FIG. 3, the image decoding apparatus 200 can be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an addition unit 235, a filtering unit 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 can be collectively referred to as a "prediction unit". The inverse quantization unit 220 and the inverse transform unit 230 can be included in a residual processing unit.

[0070] All or at least a part of a plurality of components constituting the image decoding apparatus 200 can be realized by one hardware component (for example, a decoder or a processor) according to an embodiment. Further, the memory 170 can include a DPB and can be realized by a digital storage medium.

[0071] The image decoding apparatus 200 that has received a bitstream including video / image information can execute a process corresponding to the process performed by the image encoding apparatus 100 of FIG. 2 to restore an image. For example, the image decoding apparatus 200 can perform decoding using the processing unit applied in the image encoding apparatus. Therefore, the decoding processing unit can be, for example, a coding unit. The coding unit can be obtained by dividing a coding tree unit or a maximum coding unit. Then, the restored image signal decoded and output via the image decoding apparatus 200 can be reproduced via a reproducing apparatus (not shown).

[0072] The image decoding device 200 can receive the signal output from the image encoding device of FIG. 2 in the form of a bitstream. The received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bitstream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). The video / image information can further include information regarding various parameter sets such as an Adaptive Parameter Set (APS), a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), or a Video Parameter Set (VPS). Also, the video / image information can further include general constraint information. The image decoding device can further use the information regarding the parameter set and / or the general constraint information for decoding an image. The signaling information, the received information, and / or the syntax elements referred to in the present disclosure can be obtained from the bitstream by being decoded through the decoding procedure. For example, the entropy decoding unit 210 can decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the value of the syntax element necessary for image restoration and the quantized value of the transform coefficient regarding the residual. More specifically, the CABAC entropy decoding method receives the bin corresponding to each syntax element from the bitstream, determines a context model using the syntax element information to be decoded, the decoding information of the surrounding blocks and the block to be decoded, or the information of the symbol / bin decoded in the previous step, predicts the occurrence probability of the bin based on the determined context model, and performs arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element. At this time, the CABAC entropy decoding method can update the context model using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model.Of the information decoded by the entropy decoding unit 210, the information related to prediction is provided to the prediction units (inter prediction unit 260 and intra prediction unit 265), and the residual values entropy decoded by the entropy decoding unit 210, that is, the quantized transform coefficients and related parameter information, can be input to the inverse quantization unit 220. Also, of the information decoded by the entropy decoding unit 210, the information related to filtering can be provided to the filtering unit 240. On the other hand, a receiving unit (not shown) that receives a signal output from the image encoding device can be further provided as an internal / external element of the image decoding device 200, or the receiving unit can be provided as a component of the entropy decoding unit 210.

[0073] On the other hand, the image decoding device according to the present disclosure can be called a video / image / picture decoding device. The image decoding device can also include an information decoder (video / image / picture information decoder) and / or a sample decoder (video / image / picture sample decoder). The information decoder can include the entropy decoding unit 210, and the sample decoder can include at least one of the inverse quantization unit 220, the inverse transform unit 230, the addition unit 235, the filtering unit 240, the memory 250, the inter prediction unit 260, and the intra prediction unit 265.

[0074] In the inverse quantization unit 220, the quantized transform coefficients can be inverse quantized to output transform coefficients. The inverse quantization unit 220 can reorder the quantized transform coefficients in a two-dimensional block format. In this case, the reordering can be performed based on the coefficient scan order performed by the image encoding device. The inverse quantization unit 220 can perform inverse quantization on the quantized transform coefficients using a quantization parameter (for example, quantization step size information) to obtain transform coefficients.

[0075] In the inverse conversion unit 230, the conversion coefficients can be inversely converted to obtain a residual signal (residual block, residual sample array).

[0076] The prediction unit can perform prediction on the current block and generate a predicted block including predicted samples for the current block. The prediction unit can determine whether intra prediction or inter prediction is applied to the current block based on the information regarding the prediction output from the entropy decoding unit 210, and can determine a specific intra / inter prediction mode (prediction technique).

[0077] The prediction unit can generate a prediction signal based on various prediction methods (techniques) described later, which is the same as that described in the explanation of the prediction unit of the image encoding apparatus 100.

[0078] The intra prediction unit 265 can predict the current block by referring to samples within the current picture. The explanation of the intra prediction unit 185 can be similarly applied to the intra prediction unit 265.

[0079] The inter prediction unit 260 can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit 260 can construct a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction can be performed based on various prediction modes (techniques), and the information related to the prediction can include information indicating the mode (technique) of the inter prediction for the current block.

[0080] The addition unit 235 can generate a restored signal (restored picture, restored block, restored sample array) by adding the obtained residual signal to the prediction signal (predicted block, predicted sample array) output from the prediction unit (including the inter prediction unit 260 and / or the intra prediction unit 265). When there is no residual for the processing target block as in the case where the skip mode is applied, the predicted block can be used as the restored block. The description of the addition unit 155 can be similarly applied to the addition unit 235. The addition unit 235 may also be referred to as a restoration unit or a restored block generation unit. The generated restored signal can be used for the intra prediction of the next processing target block in the current picture, and can also be used for the inter prediction of the next picture through filtering as described later.

[0081] Filtering unit 240 can apply filtering to the restored signal to improve subjective / objective image quality. For example, filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in memory 250, specifically in the DPB of memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, and the like.

[0082] The (modified) restored picture stored in the DPB of memory 250 can be used as a reference picture by inter prediction unit 260. Memory 250 can store the motion information of the blocks for which the motion information in the current picture has been derived (or decoded) and / or the motion information of the blocks in the already restored pictures. The stored motion information can be transmitted to inter prediction unit 260 for utilization as the motion information of spatial neighboring blocks or temporal neighboring blocks. Memory 250 can store the restored samples of the restored blocks in the current picture and transmit them to intra prediction unit 265.

[0083] In this specification, the embodiments described in filtering unit 160, inter prediction unit 180, and intra prediction unit 185 of image encoding apparatus 100 can be applied to filtering unit 240, inter prediction unit 260, and intra prediction unit 265 of image decoding apparatus 200 in the same or corresponding manner.

[0084] General Image / Video Coding Procedure

[0085] In image / video coding, pictures constituting an image / video can be encoded / decoded according to a series of decoding orders. The output order of decoded pictures can be set to be different from the decoding order, and based on this, not only forward prediction but also backward prediction can be performed during inter prediction.

[0086] FIG. 4 shows an example of a schematic picture decoding procedure applicable to an embodiment of the present disclosure. In FIG. 4, S400 can be performed by the entropy decoding unit 210 of the image decoding apparatus 200 described above with reference to FIG. 3, S410 can be performed by the prediction unit, S420 can be performed by the residual processing unit, S430 can be performed by the addition unit 235, and S440 can be performed by the filtering unit 240. S400 can include the information decoding procedure described in the present disclosure, S410 can include the inter / intra prediction procedure described in the present disclosure, S420 can include the residual processing procedure described in the present disclosure, S430 can include the block / picture restoration procedure described in the present disclosure, and S440 can include the in-loop filtering procedure described in the present disclosure.

[0087] Referring to FIG. 4, the picture decoding procedure can generally include an image / video information acquisition procedure (S400) for obtaining from a bitstream (by decoding), a picture restoration procedure (S410 - S430), and an in-loop filtering procedure (S440) for the restored picture. The picture restoration procedure can be performed based on the predicted samples and residual samples obtained through the inter / intra prediction (S410) and residual processing (S420, inverse quantization and inverse transform for the quantized transform coefficients) processes described in the present disclosure. Through the in-loop filtering procedure for the restored picture generated by the picture restoration procedure, a modified restored picture can be generated, and the modified restored picture can be output as a decoded picture, and can also be stored in the decoded picture buffer or memory 250 of the image decoding apparatus 200 and used as a reference picture in the inter prediction procedure during the decoding of subsequent pictures. In some cases, the in-loop filtering procedure can be omitted. In this case, the restored picture can be output as a decoded picture, and can also be stored in the decoded picture buffer or memory 250 of the image decoding apparatus 200 and used as a reference picture in the inter prediction procedure during the decoding of subsequent pictures. The in-loop filtering procedure (S440) can include, as described above, a deblocking filtering procedure, a SAO (sample adaptive offset) procedure, an ALF (adaptive loop filter) procedure, and / or a bilateral filter procedure, etc., and some or all of them can be omitted. Also, one or some of the deblocking filtering procedure, SAO (sample adaptive offset) procedure, ALF (adaptive loop filter) procedure, and bilateral filter procedure can be sequentially applied, or all of them can be sequentially applied. For example, after the deblocking filtering procedure is applied to the restored picture, the SAO procedure can be performed.Alternatively, for example, after a deblocking filtering procedure is applied to the restored picture, the ALF procedure can be performed. This can be similarly performed in the image encoding apparatus 100 as well.

[0088] FIG. 5 shows an example of a picture encoding procedure applicable to an embodiment of the present disclosure. In FIG. 5, S500 can be performed by the prediction unit of the image encoding apparatus 100 described above with reference to FIG. 2, S510 can be performed by the residual processing unit, and S520 can be performed by the entropy encoding unit 190. S500 can include the inter / intra prediction procedure described in the present disclosure, S510 can include the residual processing procedure described in the present disclosure, and S520 can include the information encoding procedure described in the present disclosure.

[0089] Referring to FIG. 5, the picture encoding procedure can generally include not only a procedure of encoding information for picture restoration (e.g., prediction information, residual information, partitioning information, etc.) and outputting it in the form of a bitstream, but also a procedure of generating a restored picture for the current picture, and a procedure (optional) of applying in-loop filtering to the restored picture. The image encoding apparatus 100 can derive (corrected) residual samples from the quantized transform coefficients via the inverse quantization unit 140 and the inverse transform unit 150, and can generate a restored picture based on the prediction samples that are the output of S500 and the (corrected) residual samples. The restored picture generated in this way can be the same as the restored picture generated by the above-described image decoding apparatus 200. A restored picture corrected by the in-loop filtering procedure for the restored picture can be generated, which can be stored in the decoded picture buffer or memory 170, and can be used as a reference picture in the inter prediction procedure during the encoding of subsequent pictures in the same manner as in the case of the image decoding apparatus 200. As described above, in some cases, part or all of the in-loop filtering procedure can be omitted. When the in-loop filtering procedure is performed, (in-loop) filtering-related information (parameters) can be encoded by the entropy encoding unit 190 and output in the form of a bitstream, and the image decoding apparatus 200 can perform the in-loop filtering procedure in the same manner as the image encoding apparatus 100 based on the filtering-related information.

[0090] Through such in-loop filtering procedures, noise generated during image / video coding, such as blocking artifacts and ringing artifacts, can be reduced, and subjective / objective visual quality can be improved. Also, by performing the in-loop filtering procedures in both the image encoding device 100 and the image decoding device 200, the image encoding device 100 and the image decoding device 200 can derive the same prediction results, enhance the reliability of picture coding, and reduce the amount of data to be transmitted for picture coding.

[0091] As described above, picture restoration procedures can be performed not only in the image decoding device 200 but also in the image encoding device 100. Restoration blocks can be generated based on intra prediction / inter prediction for each block unit, and a restored picture including the restoration blocks can be generated. When the current picture / slice / tile group is an I picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based only on intra prediction. On the other hand, when the current picture / slice / tile group is a P or B picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based on intra prediction or inter prediction. In this case, inter prediction can be applied to some of the blocks within the current picture / slice / tile group, and intra prediction can also be applied to some of the remaining blocks. The color components of a picture can include a luma component and a chroma component, and unless explicitly limited in the present disclosure, the methods and examples proposed in the present disclosure can be applied to the luma component and the chroma component.

[0092] GDR (Gradual Decoding Refresh)

[0093] In order to prevent the generation of a high bitrate while providing random access to a bitstream, a solution has been proposed to use PIR (progressive intra refresh) technology instead of using an IRAP (intra random access point) picture. The PIR technology is called the GDR function.

[0094] The GDR function can be described using GDR pictures, one or more trailing pictures, and recovery point pictures within the coded video sequence (CVS) of the bitstream. A GDR picture is a picture that allows random access, and each VCL NAL unit can be a picture having an NAL unit type such as GDR_NUT. The CVS is a series of pictures starting from a GDR picture and can include all pictures up to the next GDR picture or the end of the bitstream.

[0095] The GDR function can operate over a series of pictures starting from the GDR picture and ending at the recovery point picture. The GDR picture can include a refreshed area and an unrefreshed area. Here, the refreshed area may be called a clean area or a correctly decoded area, and the unrefreshed area may be called a dirty area or an incorrectly decoded area. The trailing picture immediately adjacent to the GDR picture can also include a refreshed area and an unrefreshed area. The refreshed area can be encoded by referring to the refreshed area of the leading picture in the CVS. The refreshed area of the trailing picture is expanded when the coding process moves or progresses in a consistent direction (e.g., from the GDR picture towards the recovery point picture), and correspondingly, the unrefreshed area can be shrunk.

[0096] Virtual Boundary

[0097] Using virtual boundary signaling, in-loop filtering can be turned off at the signaled positions within the encoded picture, and these positions do not need to be aligned with the CTU boundary. Also, the virtual boundary does not introduce additional in-picture prediction breaks such as those introduced by slices and tiles when used for such purposes.

[0098] The virtual boundary can be useful in at least two functions. First, in the case of 360-degree video coding, when using a specific projection format that introduces discontinuities such as the misaligned face boundaries of a cube map projection for a 360-degree video, the virtual boundary can be used to deactivate in-loop filtering at such boundaries without the need to adjust the size of the content to align the projection discontinuities with the CTU boundaries. Second, when using the GDR function, the boundary between the refreshed region (i.e., the correctly decoded region) and the non-refreshed region in the restored image is signaled as a virtual boundary, and in-loop filtering can be deactivated at the virtual boundary, so that decoding mismatches of some samples at or near the virtual boundary can be prevented. This function can be useful when the application decides to display the correctly decoded region during the GDR process.

[0099] Problems of the Prior Art

[0100] FIG. 6 is a diagram showing a conventional method of processing a virtual boundary.

[0101] Referring to FIG. 6, the virtual boundary can be used to deactivate the loop filter on both sides of the boundary. Also, the virtual boundary can be used as a boundary to distinguish between a refreshed area and an unrefreshed area in the realization of the GDR function. For such a realization, encoder restrictions can be applied so that the encoded blocks of the current picture (solid rectangles included in the current picture) located in the refreshed area can only use inter prediction from samples (dashed rectangles in the reference picture) reconstructed in the refreshed area of the reference picture. The image encoding device 100 must prevent blocks within the referenced area of the current picture from referring to reconstructed samples outside the refreshed area of the reference picture. As a result, the search area from motion estimation and compensation in inter prediction needs to be further reduced.

[0102] Using a smaller area for motion estimation and compensation can potentially degrade the performance of m inter prediction, especially when the motion flow progresses in the direction in which the virtual boundary moves with respect to GDR.

[0103] FIG. 7 is a diagram showing a virtual boundary processing method according to an embodiment of the present disclosure that can solve the problems of the conventional method.

[0104] One possible solution to improve inter prediction performance is that the virtual boundary is processed in the same way as the picture boundary is processed for the inter prediction process. That is, padding is applied to the virtual boundary in the same way as padding is applied to the picture boundary.

[0105] By processing the virtual boundary in the same way as the picture boundary, the search area can be maximized and the entire area refreshed in the reference image can be used as the search area for motion estimation and compensation.

[0106] Embodiment

[0107] The present disclosure proposes various embodiments to solve the problems of the prior art. The following is a summary of the various embodiments proposed by the present disclosure, and the embodiments proposed by the present disclosure can be applied individually or in combination of two or more.

[0108] 1. To assist a specific function (GDR function), each encoded image can include a vertical virtual boundary or a horizontal virtual boundary that divides the image into two regions (i.e., upper and lower regions, or left and right regions). Hereinafter, the vertical virtual boundary will be referred to as the "vertical virtual boundary", and the horizontal virtual boundary will be referred to as the "horizontal virtual boundary".

[0109] 2. The two regions can constitute different regions from each other, such as a GDR-refreshed region and a non-GDR-refreshed region.

[0110] a. The GDR-refreshed region can be the region of the encoded picture where the refreshed region of the current picture can be correctly decoded when the decoding process starts from the GDR picture preceding the current picture.

[0111] b. The non-GDR-refreshed region can be the region of the encoded picture where the non-refreshed region of the current picture cannot be correctly decoded when the decoding process starts from the GDR picture preceding the current picture.

[0112] 3. The division of the two regions (i.e., the refreshed region and the non-refreshed region) can be consistent throughout the next coded video sequence (CVS).

[0113] a. When the virtual boundary used is a vertical virtual boundary:

[0114] - The refreshed area is located to the left of the virtual boundary in all pictures of the CVS, while the non-refreshed area can be located to the right of the virtual boundary.

[0115] - The refreshed area is located to the right of the virtual boundary in all pictures of the CVS, while the non-refreshed area can be located to the left of the virtual boundary.

[0116] b. When the virtual boundary used is a horizontal virtual boundary:

[0117] - The refreshed area is located above the virtual boundary in all pictures of the CVS, while the non-refreshed area can be located below the virtual boundary.

[0118] - The refreshed area is located below the virtual boundary in all pictures of the CVS, while the non-refreshed area can be located above the virtual boundary.

[0119] 4. When decoding the coded block of the current picture that is part of the refreshed area, the following applies.

[0120] a. If the block is an inter-predicted block, the area of the reference picture that is part of the refreshed area in the reference picture must be referred to.

[0121] b. The virtual boundary of the reference picture can be processed in a way similar to the picture boundary.

[0122] 5. When the virtual boundary is a vertical virtual boundary, the method of treating the virtual boundary as a picture boundary is as follows.

[0123] a. If there is a refreshed area to the left of the vertical virtual boundary, the area from the vertical virtual boundary to the right can be padded in a way similar to when padding is applied to the picture boundary.

[0124] b. Otherwise, if there is a refreshed area on the right side of the vertical virtual boundary, the area from the vertical virtual boundary to the left side can be padded in a manner similar to that in which padding is applied to the picture boundary.

[0125] 6. When the virtual boundary is a horizontal virtual boundary, the method of treating the virtual boundary as a picture boundary is as follows.

[0126] a. If there is a refreshed area above the horizontal virtual boundary, the area from the vertical virtual boundary to the bottom side can be padded in a manner similar to that in which padding is applied to the picture boundary.

[0127] b. Otherwise, if there is a refreshed area below the vertical virtual boundary, the area from the vertical virtual boundary to the top side can be padded in a manner similar to that in which padding is applied to the picture boundary.

[0128] 7. A flag can be signaled to indicate whether the virtual boundary is treated as a picture boundary for decoding the refreshed area. This flag can be signaled in the SPS or other parameter sets and may be called sps_treat_virtualboundary_as pictureboundary_flag.

[0129] 8. The flag can exist or can be signaled only when the GDR function is activated.

[0130] FIG. 8 is a flowchart showing an image encoding method according to an embodiment of the present disclosure, and FIG. 9 is a flowchart showing an image decoding method according to an embodiment of the present disclosure.

[0131] Referring to FIG. 8, the image encoding device 100 can determine whether to process the virtual boundary of the reference picture as a picture boundary (S810). The first information can be encoded and signaled in the bitstream. The first information is information regarding the virtual boundary and can indicate whether to process the virtual boundary of the reference picture as a picture boundary.

[0132] When the image encoding device 100 determines to process the virtual boundary of the reference picture as a picture boundary, it can pad a part of the area within the reference picture (S820). The position of the part of the area to be padded can be determined according to the direction or type of the virtual boundary.

[0133] Referring to FIG. 9, the image decoding device 200 can obtain the first information from the bitstream (S910). The first information is information regarding the virtual boundary and can indicate whether to process the virtual boundary of the reference picture as a picture boundary.

[0134] The image decoding device 200 can determine whether the virtual boundary of the reference picture is processed as a picture boundary based on the first information (S920). When it is determined that the virtual boundary of the reference picture is processed as a picture boundary, the image decoding device 200 can pad a part of the area within the reference picture (S930). The position of the part of the area to be padded can be determined according to the direction or type of the virtual boundary.

[0135] FIG. 10 is a flowchart showing a method for determining the position of a part of the area to be padded according to the direction (or type) of the virtual boundary.

[0136] Referring to FIG. 10, the image encoding device 100 and the image decoding device 200 can determine the direction or type of the virtual boundary (S1010). For example, the image encoding device 100 and the image decoding device 200 can determine whether the virtual boundary is a vertical virtual boundary or a horizontal virtual boundary.

[0137] When the virtual boundary is a vertical virtual boundary, the image encoding device 100 and the image decoding device 200 can pad the right region or the left region from the virtual boundary (S1030, S1040). That is, when the virtual boundary is a vertical virtual boundary, the right region located on the right side of the virtual boundary or the left region located on the left side of the virtual boundary can be padded.

[0138] The region to be padded among the right region and the left region can be determined based on the position of the refreshed region (or the non-refreshed region). For example, when the refreshed region is located on the left side of the virtual boundary (S1020), the image encoding device 100 and the image decoding device 200 pad the right region (S1030), and when the refreshed region is located on the right side of the virtual boundary (S1020), the left region can be padded (S1040). Stated otherwise, when the non-refreshed region is located on the left side of the virtual boundary (S1020), the image encoding device 100 and the image decoding device 200 pad the left region (S1040), and when the non-refreshed region is located on the right side of the virtual boundary (S1020), the right region can be padded (S1030).

[0139] Returning to the S1010 process, when the virtual boundary is a horizontal virtual boundary, the image encoding device 100 and the image decoding device 200 can pad the upper region or the lower region from the virtual boundary (S1060, S1070). That is, when the virtual boundary is a horizontal virtual boundary, the upper region located above the virtual boundary or the lower region located below the virtual boundary can be padded.

[0140] The padded area among the upper area and the lower area can be determined based on the position of the refreshed area (or the non-refreshed area). For example, when the refreshed area is located above the virtual boundary (S1050), the image encoding device 100 and the image decoding device 200 pad the lower area (S1060), and when the refreshed area is located below the virtual boundary (S1050), the upper area can be padded (S1070). Stated otherwise, when the non-refreshed area is located above the virtual boundary (S1050), the image encoding device 100 and the image decoding device 200 pad the upper area (S1070), and when the non-refreshed area is located below the virtual boundary (S1050), the lower area can be padded (S1060).

[0141] FIG. 11 is a flowchart showing a signaling method of the first information, and FIG. 12 is a flowchart showing an acquisition method of the first information.

[0142] The syntax structure for the signaling of the first information is shown in Table 1.

[0143] [Table 1]

[0144] The sps_treat_virtualboundary_as_pictureboundary_flag is a syntax element corresponding to the first piece of information. A value of 0 for sps_treat_virtualboundary_as_pictureboundary_flag indicates that when decoding the coded blocks of the refreshed area, the virtual boundary in the reference picture is not treated as a picture boundary, and a value of 1 for sps_treat_virtualboundary_as_pictureboundary_flag can indicate that when decoding the coded blocks of the refreshed area, the virtual boundary in the reference picture is treated as a picture boundary. If the sps_treat_virtualboundary_as_pictureboundary_flag does not exist, the value of the sps_treat_virtualboundary_as_pictureboundary_flag can be inferred as 0.

[0145] The sps_gdr_enabled_flag is a syntax element indicating whether the GDR function (GDR picture) is activated. A value of 0 for sps_gdr_enabled_flag indicates that the GDR function (GDR picture) is not activated, and a value of 1 for sps_gdr_enabled_flag can indicate that the GDR function (GDR picture) is activated. The sps_gdr_enabled_flag may be referred to as the "first flag".

[0146] Referring to FIG. 11, the image encoding apparatus 100 determines whether the GDR picture is activated (S1110), and when the GDR picture is activated, the sps_treat_virtualboundary_as_pictureboundary_flag can be encoded (S1120). That is, the sps_treat_virtualboundary_as_pictureboundary_flag can be encoded only when the GDR function is activated. The sps_gdr_enabled_flag indicating whether the GDR picture is activated can also be encoded.

[0147] Referring to FIG. 12, the image decoding apparatus 200 can obtain the sps_gdr_enabled_flag (first flag) from the bitstream (S1210), and can determine whether the GDR picture is activated based on the first flag (S1220). When the GDR picture is activated, the image decoding apparatus 200 can obtain the sps_treat_virtualboundary_as_pictureboundary_flag from the bitstream (S1230). That is, the sps_treat_virtualboundary_as_pictureboundary_flag can be obtained only when the GDR function is activated.

[0148] FIG. 13 is a diagram exemplarily showing a content streaming system to which an embodiment according to the present disclosure can be applied.

[0149] As shown in FIG. 13, the content streaming system to which the embodiment of the present disclosure is applied can generally include an encoding server, a streaming server, a Web server, a media storage, a user device, and a multimedia input device.

[0150] The encoding server compresses the content input from a multimedia input device such as a smartphone, a camera, or a camcorder into digital data to generate a bitstream, and plays a role of transmitting this to the streaming server. As another example, when a multimedia input device such as a smartphone, a camera, or a video camera directly generates a bitstream, the encoding server can be omitted.

[0151] The bitstream can be generated by an image encoding method and / or an image encoding apparatus to which the embodiment of the present disclosure is applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0152] The streaming server transmits multimedia data to a user device based on a user's request via a web server, and the web server can serve as a medium to inform the user of what services are available. When the user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system can include a separate control server, and in this case, the control server can play a role in controlling commands / responses between each device within the content streaming system.

[0153] The streaming server can receive content from a media storage and / or encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0154] Examples of the user device can include a mobile phone, a smart phone, a laptop computer, a digital broadcast terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, a slate PC, a tablet PC, an ultrabook, a wearable device, for example, a smartwatch, smart glass, an HMD (head mounted display), a digital TV, a desktop computer, a digital signage, and the like.

[0155] Each server within the content streaming system can be operated as a distributed server, and in this case, the data received from each server can be distributedly processed.

[0156] The scope of the present disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) that enable the operations of various example methods to be executed on a device or computer, and non-transitory computer-readable media on which such software or commands are stored and can be executed on a device or computer.

Industrial Applicability

[0157] Examples according to the present disclosure can be used for encoding / decoding images.

Claims

1. An image decoding method performed by an image decoding device, The first step is to obtain the virtual boundary information from the bitstream, Based on the first information, the step of determining whether the virtual boundary of the reference picture is treated as a picture boundary, An image decoding method comprising the step of padding a portion of a reference picture based on the fact that the virtual boundary of the reference picture is treated as a picture boundary.

2. The image decoding method according to claim 1, wherein, based on the fact that the virtual boundary of the reference picture is a vertical virtual boundary, a region to the right or left of the virtual boundary of the reference picture is padded.

3. The image decoding method according to claim 2, wherein the right-side region is padded based on the fact that the refreshed region is located to the left of the virtual boundary of the reference picture.

4. The image decoding method according to claim 2, wherein the left region is padded based on the fact that the refreshed region is located to the right of the virtual boundary of the reference picture.

5. The image decoding method according to claim 1, wherein, based on the fact that the virtual boundary of the reference picture is a horizontal virtual boundary, an upper or lower region of the reference picture from the virtual boundary is padded.

6. The image decoding method according to claim 5, wherein the lower region is padded based on the fact that the refreshed region is located above the virtual boundary of the reference picture.

7. The image decoding method according to claim 5, wherein the upper region is padded based on the fact that the refreshed region is located below the virtual boundary of the reference picture.

8. The image decoding method according to claim 1, wherein the first information is obtained based on the activation of a GDR (gradual decoding refresh) picture.

9. An image encoding method performed by an image encoding device, A step to determine whether to treat the virtual boundary of the reference picture as the picture boundary, The step of padding a portion of a region within a reference picture, based on the fact that the virtual boundary of the reference picture is treated as the picture boundary, Image encoding method wherein first information indicating whether the virtual boundary of the reference picture is processed as the picture boundary is encoded into a bitstream.

10. A method for transmitting a bitstream generated by an image encoding method, The aforementioned image encoding method is A step to determine whether to treat the virtual boundary of the reference picture as the picture boundary, The step of padding a portion of a region within a reference picture, based on the fact that the virtual boundary of the reference picture is treated as the picture boundary, A method for encoding first information indicating whether the virtual boundary of the reference picture is treated as the picture boundary into a bitstream.

11. A computer-readable recording medium storing a bitstream generated by the image encoding method described in Claim 9.

Images