Intra prediction using multiple reference lines

Abstract

To provide intra prediction using a plurality of reference lines.SOLUTION: A video coding device receives a bitstream including video data. The device determines an intra-prediction mode subset. The intra-prediction mode subset includes intra-prediction modes that correlate to a plurality of reference lines for a current image block, and excludes intra-prediction modes that correlate to a primary reference line for the current image block. When a first intra-prediction mode is included in the intra-prediction mode subset, the device decodes the first intra-prediction mode by an alternative intra-prediction mode index. When the first intra-prediction mode is not included in the intra-prediction mode subset, the device decodes the first intra-prediction mode by an intra-prediction mode index. The device presents the video data including an image block decoded based on the first intra-prediction mode.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This patent application claims priority to U.S. Non-provisional Patent Application No. 15 / 972,870, filed May 7, 2018, entitled "Intra Prediction Using Multiple Reference Lines," which in turn claims priority to U.S. Provisional Patent Application No. 62 / 503,884, filed May 9, 2017, by Shan Liu et al., entitled "Method and Apparatus for Intra Prediction Using Multiple Reference Lines," and U.S. Provisional Patent Application No. 62 / 511,757, filed May 26, 2017, by Xiang Ma et al., entitled "Method and Apparatus for Intra Prediction Using Multiple Reference Lines," the teachings and disclosures of which are incorporated herein by reference in their entireties.

[0002] STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT Not applicable.

[0003] Reference to Microfiche Appendix Not applicable. [Background technology]

[0004] The amount of video data required to render even a relatively short video can be substantial, which can create difficulties when the data is streamed or otherwise communicated over communications networks with limited bandwidth capacity. Therefore, video data is typically compressed before being communicated over modern telecommunications networks. Video size can also be an issue when the video is stored on a storage device, where storage resources may be limited. Video compression devices often use software and / or hardware at the source to encode video data prior to transmission or storage, thereby reducing the amount of data required to represent a digital video image. The compressed data is then received at the destination by a video decompression device, which decodes the video data. With limited network resources and ever-increasing demands for higher video quality, improved compression and decompression techniques that improve compression ratios with little or no sacrifice in image quality are desirable. Summary of the Invention

[0005] In one embodiment, the present disclosure provides a video encoding method including: a receiver configured to receive a bitstream; a processor coupled to the receiver, the processor configured to perform the steps of: determining an intra-prediction mode subset, the intra-prediction mode subset including intra-prediction modes correlated to a plurality of reference lines for a current image block and excluding an intra-prediction mode correlated to a primary reference line for the current image block; decoding a first intra-prediction mode using an alternative intra-prediction mode index when the first intra-prediction mode is included in the intra-prediction mode subset; and decoding the first intra-prediction mode using an intra-prediction mode index when the first intra-prediction mode is not included in the intra-prediction mode subset; and a display coupled to the processor, the display configured to present video data including the image block decoded based on the first intra-prediction mode.

[0006] Optionally, in any of the above aspects, another implementation of the aspect provides that the processor is further configured to: decode the reference line index when the first intra prediction mode is included in the intra prediction mode subset, the reference line index indicating a first reference line from the plurality of reference lines for the first intra prediction mode; and not decode the reference line index when the first intra prediction mode is not included in the intra prediction mode subset.

[0007] Optionally, in any of the above aspects, another implementation of the aspect provides that the reference line index is located after the first intra prediction mode in the bitstream.

[0008] Optionally, in any of the above aspects, another implementation of the aspect provides that the intra-prediction mode subset includes a starting directional intra-prediction mode (DirS), an ending directional intra-prediction mode (DirE), and every Nth directional intra-prediction mode between DirS and DirE, where N is a predetermined integer value.

[0009] Optionally, in any of the above aspects, another implementation of the aspect provides that the intra-prediction mode subset further includes a planar prediction mode and a direct current (DC) prediction mode.

[0010] Optionally, in any of the above aspects, another embodiment of the aspect provides that the intra-prediction mode subset includes a starting directional intra-prediction mode (DirS), an ending directional intra-prediction mode (DirE), a middle directional intra-prediction mode (DirD), a horizontal directional intra-prediction mode (DirH), a vertical directional intra-prediction mode (DirV), and valid directional intra-prediction modes in directions plus or minus N of DirS, DirE, DirD, DirH, and DirV, where N is a predetermined integer value.

[0011] Optionally, in any of the above aspects, another implementation of the aspect provides that the intra-prediction mode subset further includes a planar prediction mode and a direct current (DC) prediction mode.

[0012] Optionally, in any of the above aspects, another implementation of the aspect provides that the intra prediction mode subset includes intra prediction modes selected for decoded neighboring blocks, the decoded neighboring blocks being positioned in a predetermined neighborhood to the current image block.

[0013] Optionally, in any of the above aspects, another implementation of the aspect provides that the intra-prediction mode subset includes modes associated with a most probable mode (MPM) list for the current image block.

[0014] In one embodiment, the present disclosure includes a method including: storing in a memory a bitstream including an image block coded as a predictive block; obtaining, by a processor coupled to the memory, a current predictive block encoded using a Direct Current (DC) intra-prediction mode; determining a DC predicted value that approximates a current image block corresponding to the current predictive block by determining an average of all reference samples in at least two of a plurality of reference lines associated with the current predictive block; reconstructing, by the processor, the current image block based on the DC predicted value; and displaying on a display a video frame including the current image block.

[0015] Optionally, in any of the above aspects, another implementation of the aspect provides that determining the DC predicted value includes determining an average of all reference samples in N adjacent reference lines to the current predicted block, where N is a predetermined integer.

[0016] Optionally, in any of the above aspects, another implementation of the aspect provides that determining the DC predicted value includes determining an average of all reference samples within the selected reference line and the corresponding reference line.

[0017] Optionally, in any of the above aspects, another implementation of the aspect provides that determining the DC predicted value includes determining an average of all reference samples in adjacent reference lines and the selected reference line.

[0018] In an embodiment, the present disclosure includes a non-transitory computer-readable medium including a computer program product for use by a video coding apparatus, the computer program product including computer-executable instructions stored on the non-transitory computer-readable medium that, when executed by a processor, cause the video coding apparatus to perform the following steps: receive a bitstream via a receiver; receive, by the processor, an intra-prediction mode from the bitstream, the intra-prediction mode indicating a relationship between a current block and a selected reference line, the current block being associated with a plurality of reference lines including the selected reference line; decode, by the processor, the selected reference line based on a selected codeword indicating the selected reference line, the selected codeword including a length based on a selection probability of the selected reference line; and present video data including the decoded image block based on the intra-prediction mode and the selected reference line on a display.

[0019] Optionally, in any of the above aspects, another implementation of the aspect provides that the multiple reference lines are represented by multiple code words, and the reference line farthest from the current block is represented by the code word with the second shortest length.

[0020] Optionally, in any of the above aspects, another implementation of the aspect provides that the multiple reference lines are represented by multiple code words, and the reference line second farthest from the current block is represented by the code word having the second shortest length.

[0021] Optionally, in any of the above aspects, another implementation of the aspect provides that the plurality of reference lines are represented by a plurality of code words, and a predefined reference line other than an adjacent reference line is represented by a code word having a second shortest length.

[0022] Optionally, in any of the above aspects, another implementation of the aspect provides that the plurality of reference lines are represented by a plurality of code words, the plurality of code words being classified into a class A group and a class B group, the class A group including code words having lengths shorter than lengths of code words in the class B group.

[0023] Optionally, in any of the above aspects, another implementation of the aspect provides that the plurality of reference lines includes reference rows and reference columns, and the number of reference rows stored for the current block is half the number of reference columns stored for the current block.

[0024] Optionally, in any of the above aspects, another implementation of the aspect provides that the plurality of reference lines includes reference rows and reference columns, and the number of reference rows stored for the current block is equal to the number of reference columns stored for the current block minus one.

[0025] Optionally, in any of the above aspects, another implementation of the aspect provides that the plurality of reference lines includes a reference row, and the number of reference rows stored for the current block is selected based on the number of reference rows used by a deblocking filter operation.

[0026] For clarity, any of the above-described embodiments can be combined with any one or more of the other above-described embodiments to create new embodiments within the scope of the present disclosure.

[0027] These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. [Brief explanation of the drawings]

[0028] For a more complete understanding of this disclosure, reference is now made to the following brief description, taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0029] [Figure 1] 1 is a flowchart of an exemplary method for encoding a video signal.

[0030] [Figure 2] 1 is a schematic diagram of an exemplary encoding and decoding (codec) system for video encoding.

[0031] [Figure 3] FIG. 1 is a block diagram illustrating an example video encoder that may implement intra prediction.

[0032] [Figure 4] FIG. 1 is a block diagram illustrating an example video decoder that may implement intra prediction.

[0033] [Figure 5] FIG. 1 is a schematic diagram illustrating exemplary intra-prediction modes used in video encoding.

[0034] [Figure 6] FIG. 1 is a schematic diagram illustrating an example of block directional relationships in video coding.

[0035] [Figure 7]FIG. 1 is a schematic diagram illustrating an example of a dominant reference line scheme for coding a block with intra prediction.

[0036] [Figure 8] FIG. 1 is a schematic diagram illustrating an example of an alternative reference line scheme for coding a block with intra prediction.

[0037] [Figure 9] FIG. 1 is a schematic diagram illustrating an example intra-prediction mode subset.

[0038] [Figure 10] FIG. 1 is a schematic diagram illustrating an example conditional signaling representation for an alternative reference line in a video coding bitstream.

[0039] [Figure 11] FIG. 1 is a schematic diagram illustrating an example conditional signaling representation for key reference lines in a video coding bitstream.

[0040] [Figure 12] FIG. 1 is a schematic diagram illustrating an exemplary mechanism for DC mode intra prediction using alternate reference lines.

[0041] [Figure 13] FIG. 1 is a schematic diagram illustrating an example mechanism for encoding alternative reference lines in a codeword.

[0042] [Figure 14] FIG. 10 is a schematic diagram illustrating an exemplary mechanism for encoding alternative reference lines with different numbers of rows and columns.

[0043] [Figure 15] FIG. 10 is a schematic diagram illustrating another exemplary mechanism for encoding alternative reference lines with different numbers of rows and columns.

[0044] [Figure 16] 1 is a schematic diagram of an exemplary video encoding device;

[0045] [Figure 17] 1 is a flowchart of an exemplary method for video encoding using intra-prediction mode subsets with alternative reference lines.

[0046] [Figure 18] 10 is a flowchart of an exemplary method for video encoding using DC mode intra prediction with alternate reference lines.

[0047] [Figure 19] 1 is a flowchart of an exemplary method for video encoding using reference lines coded with codewords based on selection probabilities. DETAILED DESCRIPTION OF THE INVENTION

[0048] First, while example implementations of one or more embodiments are provided below, it should be understood that the disclosed systems and / or methods may be implemented using any number of technologies, whether currently known or in existence. The present disclosure should in no way be limited to the example implementations, drawings, and technologies shown below, including the example designs and implementations shown and described herein, but may be modified within the full scope of the appended claims and equivalents thereof.

[0049] Many techniques are used in tandem to compress video data during the video encoding process. For example, a video sequence is divided into image frames. The image frames are then divided into image blocks. The image blocks can then be compressed using inter-prediction (correlation between blocks in different frames) or intra-prediction (correlation between blocks within the same frame). In intra-prediction, the current image block is predicted from a reference line of samples. The reference line contains samples from adjacent image blocks, also called neighboring blocks. Samples from the current block are matched with samples from the reference line with the closest luma (brightness) or chroma (color) values. The current block is coded as a prediction mode that indicates the matching samples. Prediction modes include angular prediction mode, direct current (DC) mode, and planar mode. The difference between the values ​​predicted by these prediction modes and the actual values ​​is coded as a residual value in a residual block. Matching may be improved by using multiple reference lines. Improved matching results in fewer residual values, thereby increasing compression. However, increasing the number of reference lines may increase the number of bins (binary values) required to uniquely identify a matching reference line. If multiple reference lines are not required to determine the best match, the increased signaling overhead associated with identifying multiple reference lines may outweigh the compression gain typically associated with multiple reference lines, and thus may increase the overall bitstream file size. This results in reduced coding efficiency in such cases.

[0050] This application discloses mechanisms to support the video encoding process that reduce the signaling overhead associated with intra-prediction based on multiple reference lines, thereby increasing compression in video encoding systems. For example, mechanisms for creating intra-prediction mode subsets are disclosed. Allowing all intra-prediction modes to have access to the full set of multiple reference lines would increase signaling overhead and ultimately result in larger file sizes to encode. Thus, the intra-prediction mode subset includes a subset of intra-prediction modes that have access to alternative reference lines, and modes excluded from the intra-prediction mode subset are limited to accessing the primary reference line. As used herein, a primary reference line is a reference line located closest (e.g., immediately adjacent) to the current block. Alternate reference lines include both the primary reference line and a set of reference lines located farther from the current block than the primary reference line. The encoder may use modes from the intra-prediction mode subset when an alternative reference line is useful, and may use modes excluded from the intra-prediction mode subset when the primary reference line is sufficient. Because modes outside the intra-prediction mode subset are limited to accessing the primary reference line, reference line indices may be omitted for intra-prediction modes not included in the intra-prediction mode subset. This may be achieved by encoding the reference line indices after the intra-prediction mode information. When the intra-prediction mode information indicates that an intra-prediction mode is not included in the intra-prediction mode subset, the decoder can contextually recognize that a reference line index is not included. The intra-prediction modes included in the intra-prediction mode subset may be predetermined (e.g., stored in a table and / or hard-coded) and / or contextually estimated based on the intra-prediction mode subsets of neighboring blocks, etc. In some cases, the intra-prediction mode subset may be selected to include modes related to a most probable mode (MPM) list. This allows the most commonly selected modes to have access to alternative reference lines. Furthermore, the intra-prediction modes in the intra-prediction mode subset may be signaled based on an intra-prediction mode subset index. Because the intra-prediction mode subset includes fewer prediction modes than the complete intra-prediction mode set, the intra-prediction mode subset index may generally be signaled with fewer bins. Furthermore, an extension to DC intra-prediction modes is disclosed. The disclosed DC intra-prediction modes may allow DC prediction values ​​to be determined based on alternative reference lines. Also disclosed is a mechanism for condensing the codeword size representing a particular reference line. In this mechanism, reference lines are indexed based on selection probability rather than based on distance from the current image sample. For example, the reference line index most likely to be selected receives the shortest index, and the reference line least likely to be selected receives the longest index. This results in a small encoding of the reference line index in most cases. The codeword for the reference line index may be determined in advance and used consistently (e.g., stored in a table and / or hard-coded). Furthermore, certain hardware designs require more storage to store reference line rows than reference line columns. Thus, a mechanism is disclosed to support storing fewer reference rows than reference columns when encoding an image sample, thereby supporting reduced storage requirements during encoding.

[0051] FIG. 1 is a flowchart of an exemplary method 100 for encoding a video signal. Specifically, the video signal is encoded in an encoder. The encoding process compresses the video signal using various mechanisms to reduce the video file size. The smaller file size allows the compressed video file to be transmitted to a user while reducing the associated bandwidth overhead. A decoder then decodes the compressed video file to reconstruct the original video signal for display to the end user. The decoding process generally mirrors the encoding process to allow the decoder to consistently reconstruct the video signal.

[0052] In step 101, a video signal is input to an encoder. For example, the video signal may be an uncompressed video file stored in memory. As another example, the video file may be captured by a video capture device, such as a video camera, and encoded to support live streaming of the video. The video file may include both an audio component and a video component. The video component includes a series of image frames that, when viewed in sequence, create the impression of visual movement. The frames include pixels represented using brightness, referred to herein as the luma component, and color, referred to herein as the chroma component. In some examples, the frames may also include depth values ​​to support three-dimensional viewing.

[0053] In step 103, the video is divided into blocks. Partitioning involves subdividing pixels within each frame into square and / or rectangular blocks for compression. For example, a coding tree can be used to divide the blocks and then recursively subdivide the blocks until a configuration that supports further encoding is achieved. Thus, blocks are sometimes referred to as coding tree units in High Efficiency Video Coding (HEVC) (also known as H.265 and MPEG-H Part 2). For example, the luma component of a frame may be subdivided until each block contains relatively uniform illumination values. Further, the chroma component of a frame may be subdivided until each block contains relatively uniform color values. Thus, the partitioning scheme varies depending on the content of the video frame.

[0054] In step 105, various compression mechanisms are used to compress the image blocks partitioned in step 103. For example, inter-prediction and / or intra-prediction may be used. Inter-prediction is designed to take advantage of the fact that objects in a common scene tend to appear in consecutive frames. Thus, a block depicting an object in a reference frame need not be repeatedly described in subsequent frames. Specifically, an object such as a table may remain in a constant position across multiple frames. Thus, the table may be described once, and subsequent frames may reference the reference frame. Pattern matching mechanisms can be used to match objects across multiple frames. Furthermore, moving objects may be represented across multiple frames, for example, due to object motion or camera motion. As a specific example, a video may show a car moving across the screen over multiple frames. Motion vectors can be used to describe such motion. A motion vector is a two-dimensional vector that provides an offset from the object's coordinates in a frame to the object's coordinates in a reference frame. In this manner, inter-prediction can encode an image block in a current frame as a set of motion vectors that indicate its offset from a corresponding block in a reference frame.

[0055] Intra prediction encodes blocks within a common frame. Intra prediction exploits the fact that luma and chroma components tend to cluster in a frame. For example, a green patch in a tree tends to be located adjacent to similar green patches. Intra prediction uses multiple directional prediction modes (e.g., 33 in HEVC), planar mode, and DC mode. Directional mode indicates that the current block is similar / the same as samples of neighboring blocks in the corresponding direction. Planar mode indicates that a series of blocks along a row / column (e.g., a plane) can be interpolated based on neighboring blocks at the end of the row. Planar mode effectively indicates a smooth transition in brightness / color through the row / column by using a relatively constant slope in changing values. DC mode is used for boundary smoothing and indicates that the block is similar / the same as the average value associated with samples of all neighboring blocks related to the angular direction of the directional prediction mode. Thus, intra-predicted blocks can represent image blocks as various associated prediction mode values ​​instead of actual values. Furthermore, inter-predicted blocks can represent image blocks as motion vector values ​​instead of actual values. In either case, the prediction block may not exactly represent the image block. Any differences are stored in a residual block. To further compress the file, a transform may be applied to the residual block.

[0056] Various filtering techniques may be applied in step 107. In HEVC, filters are applied according to an in-loop filtering scheme. The block-based prediction discussed above can lead to the generation of blocky images at the decoder. Furthermore, block-based prediction schemes may encode blocks and then reconstruct the encoded blocks for later use as reference blocks. In-loop filtering schemes sequentially apply noise suppression filters, deblocking filters, adaptive loop filters, and sample adaptive offset (SAO) filters to blocks / frames. These filters mitigate such block artifacts so that the encoded file can be accurately reconstructed. Furthermore, these filters mitigate artifacts in the reconstructed reference blocks, thereby reducing the likelihood that artifacts will cause additional artifacts in subsequent blocks that are encoded based on the reconstructed reference blocks.

[0057] Once the video signal has been segmented, compressed, and filtered, the resulting data is encoded into a bitstream in step 109. The bitstream includes the data discussed above as well as any signaling data desired to support proper video signal reconstruction at the decoder. For example, such data may include partition data, prediction data, residual blocks, and various flags that provide coding instructions to the decoder. The bitstream may be stored in memory for transmission to the decoder upon request. The bitstream may also be broadcast and / or multicast to multiple decoders. Generating the bitstream is an iterative process. Thus, steps 101, 103, 105, 107, and 109 can occur sequentially and / or simultaneously across many frames and blocks. The order shown in FIG. 1 is presented for clarity and simplicity of discussion and is not intended to limit the video encoding process to any particular order.

[0058] The decoder receives the bitstream and begins the decoding process in step 111. Specifically, the decoder uses an entropy decoding scheme to convert the bitstream into corresponding syntax and video data. Using syntax data from the bitstream, the decoder determines partitions for frames in step 111. The partitioning should match the results of the block partitioning in step 103. The entropy encoding / decoding used in step 111 is now described. The encoder makes many choices during the compression process, such as selecting a block partitioning scheme from several possible options based on the spatial location of values ​​within the input image(s). Signaling the exact selection may involve the use of multiple bins. As used herein, a bin is a binary value (e.g., a bit value that can vary depending on the context) that is treated as a variable. Entropy encoding allows the encoder to discard any options that are clearly not feasible in a particular case, retaining a set of acceptable options. Each acceptable option is then assigned a codeword. The length of the codeword is based on the number of allowable options (e.g., one bin for two options, two bins for three to four options, etc.). The encoder then encodes a codeword for the selected option. This scheme reduces the size of the codeword because the codeword is only as large as desired to uniquely indicate a selection from a small subset of allowable options, rather than uniquely indicating a selection from a potentially large set of all possible options. The decoder then decodes the selection by determining the set of allowable options in a similar manner as the encoder. By determining the set of allowable options, the decoder can read the codeword and determine the selection made by the encoder.

[0059] In step 113, the decoder performs block decoding. Specifically, the decoder generates a residual block using an inverse transform. The decoder then uses the residual block and a corresponding prediction block to reconstruct an image block according to the partitioning. The prediction block may include both intra-predicted blocks and inter-predicted blocks, as generated by the encoder in step 105. The reconstructed image block is then positioned within a frame of the reconstructed video signal according to the partitioning data determined in step 111. The syntax for step 113 may also be signaled in the bitstream by entropy coding, as discussed above.

[0060] In step 115, filtering is performed on the frames of the reconstructed video signal in a manner similar to step 107 in the encoder. For example, a noise suppression filter, a deblocking filter, an adaptive loop filter, and an SAO filter may be applied to the frames to remove blocking artifacts. Once the frames have been filtered, the video signal may be output to a display in step 117 for viewing by an end user.

[0061] FIG. 2 is a schematic diagram of an exemplary encoding and decoding (codec) system 200 for video encoding. Specifically, codec system 200 provides functionality to support the implementation of method 100. Codec system 200 is generalized to depict components used in both an encoder and a decoder. Codec system 200 receives and partitions a video signal, as discussed with respect to steps 101 and 103 of method 100, resulting in a partitioned video signal 201. When acting as an encoder, codec system 200 then compresses the partitioned video signal 201 into a coded bitstream, as discussed with respect to steps 105, 107, and 109 of method 100. When acting as a decoder, codec system 200 generates an output video signal from the bitstream, as discussed with respect to steps 111, 113, 115, and 117 of method 100. Codec system 200 includes a general encoder control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, an intra-picture prediction component 217, a motion compensation component 219, a motion estimation component 221, a scaling and inverse transform component 229, a filter control analysis component 227, an in-loop filter component 225, a decoded picture buffer component 223, and a header formatting and context adaptive binary arithmetic coding (CABAC) component 231. Such components are coupled as shown. In FIG. 2, black lines indicate the movement of data to be encoded / decoded, and dashed lines indicate the movement of control data that controls the operation of other components. In an encoder, all of the components of codec system 200 may be present. A decoder may include a subset of the components of codec system 200.For example, the decoder may include an intra-picture prediction component 217, a motion compensation component 219, a scaling and inverse transform component 229, an in-loop filter component 225, and a decoded picture buffer component 223. These components are now described.

[0062] The segmented video signal 201 is a captured video stream that has been divided into blocks of pixels by a coding tree. The coding tree divides the blocks of pixels into smaller blocks of pixels using various partitioning modes. These blocks can then be further subdivided into smaller blocks. The blocks may be referred to as nodes of the coding tree. Larger parent nodes are divided into smaller child nodes. The number of times a node is subdivided is referred to as the depth of the node / coding tree. The divided blocks are sometimes referred to as coding units (CUs). Partitioning modes may include a binary tree (BT), a triple tree (TT), and a quad tree (QT), each of which is used to divide a node into two, three, or four child nodes of different shapes, depending on the partitioning mode used. The segmented video signal 201 is forwarded to a general coder control component 211, a transform scaling and quantization component 213, an intra picture estimation component 215, a filter control analysis component 227, and a motion estimation component 221 for compression.

[0063] The general encoder control component 211 is configured to make decisions related to encoding images of a video sequence into a bitstream according to application constraints. For example, the general encoder control component 211 manages the optimization of bitrate / bitstream size versus reconstruction quality. Such decisions may be made based on storage space / bandwidth availability and image resolution requirements. The general encoder control component 211 also manages buffer utilization in relation to transmission rate to mitigate buffer underrun and overrun issues. To manage these issues, the general encoder control component 211 manages segmentation, prediction, and filtering by other components. For example, the general encoder control component 211 can dynamically increase compression complexity to increase resolution and increase bandwidth usage, or decrease compression complexity to decrease resolution and bandwidth usage. Thus, the general encoder control component 211 controls other components of the codec system 200 to balance bitrate concerns with video signal reconstruction quality. The general encoder control component 211 generates control data that controls the operation of other components. The control data is also forwarded to the Header Formatting and CABAC component 231 and encoded in the bitstream to signal parameters for decoding at the decoder.

[0064] The segmented video signal 201 is also sent to a motion estimation component 221 and a motion compensation component 219 for inter prediction. A frame or slice of the segmented video signal 201 may be divided into multiple video blocks. The motion estimation component 221 and the motion compensation component 219 perform inter predictive coding of the received video blocks relative to one or more blocks in one or more reference frames to provide temporal prediction. The codec system 200 may perform multiple coding passes, for example, to select an appropriate coding mode for each block of video data.

[0065] The motion estimation component 221 and the motion compensation component 219 may be highly integrated but are illustrated separately for conceptual purposes. Motion estimation, performed by the motion estimation component 221, is the process of generating motion vectors, which estimate motion for a video block. A motion vector may indicate, for example, the displacement of a prediction unit (PU) of a video block relative to a predictive block in a reference frame (or other coding unit) with respect to a current block being coded in a current frame (or other coding unit). A predictive block is a block that is found to closely match a block to be coded in terms of pixel difference, which may be determined by sum of absolute difference (SAD), sum of square difference (SSD), or other difference metric. In some examples, the codec system 200 can calculate values ​​for sub-integer pixel locations of a reference picture stored in the decoded picture buffer 223. For example, the video codec system 200 may interpolate values ​​for quarter-pixel, eighth-pixel, or other fractional pixel locations of a reference picture. Thus, the motion estimation component 221 can perform motion searches for full-pixel and fractional-pixel locations and output motion vectors with fractional-pixel precision. The motion estimation component 221 calculates motion vectors for PUs of video blocks in inter-coded slices by comparing the location of the PUs with the location of the prediction blocks in the reference images. The motion estimation component 221 outputs the calculated motion vectors as motion data to the header formatting and CABAC component 231 for encoding, and outputs motion to the motion compensation component 219.

[0066] The motion compensation performed by the motion compensation component 219 may involve fetching or generating a prediction block based on a motion vector determined by the motion estimation component 221. Again, in some examples, the motion estimation component 221 and the motion compensation component 219 may be functionally integrated. Upon receiving a motion vector for the PU of the current video block, the motion compensation component 219 may locate the prediction block to which the motion vector points in the reference picture list. A residual video block is then formed by subtracting pixel values ​​of the prediction block from pixel values ​​of the current video block being coded to form pixel difference values. Generally, the motion estimation component 221 performs motion estimation on the luma component, and the motion compensation component 219 uses the motion vector calculated based on the luma component for both the chroma and luma components. The prediction block and the residual block are forwarded to the transform scaling and quantization component 213.

[0067] The split video signal 201 is also sent to an intra picture estimation component 215 and an intra picture prediction component 217. Like the motion estimation component 221 and motion compensation component 219, the intra picture estimation component 215 and intra picture prediction component 217 may be highly integrated but are illustrated separately for conceptual purposes. The intra picture estimation component 215 and intra picture prediction component 217 intra-predict the current block relative to blocks within the current frame, instead of the inter-prediction performed by the inter-frame motion estimation component 221 and motion compensation component 219 described above. In particular, the intra picture estimation component 215 determines the intra-prediction mode to use to encode the current block. In some examples, the intra picture estimation component 215 selects an appropriate intra-prediction mode to encode the current block from multiple tested intra picture prediction modes. The selected intra-prediction mode is then forwarded to the header formatting and CABAC component 231 for encoding.

[0068] For example, the intra picture estimation component 215 may calculate rate-distortion values ​​for various tested intra prediction modes using a rate-distortion analysis and select the intra prediction mode with the best rate-distortion characteristics among the tested modes. The rate-distortion analysis generally determines the amount of distortion (or error) between an encoded block and the original unencoded block that was encoded to generate the encoded block, as well as the bitrate (e.g., number of bits) used to generate the encoded block. The intra picture estimation component 215 may calculate a ratio from the distortion and rate for the various encoded blocks and determine which intra prediction mode exhibits the best rate-distortion value for the block. Additionally, the intra picture estimation component 215 may be configured to encode depth blocks of the depth map using a depth modeling mode (DMM) based on rate-distortion optimization (RDO).

[0069] The intra picture prediction component 217, when implemented in an encoder, generates a residual block from the predicted block based on the selected intra prediction mode determined by the intra picture estimation component 215, and, when implemented in a decoder, can read the residual block from the bitstream. The residual block contains the difference in values ​​between the predicted block and the original block, represented as a matrix. The residual block is then forwarded to the transform scaling and quantization component 213. The intra picture estimation component 215 and the intra picture prediction component 217 may operate on both the luma component and the chroma component.

[0070] The transform scaling and quantization component 213 is configured to further compress the residual block. The transform scaling and quantization component 213 applies a transform, such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a conceptually similar transform, to the residual block to generate a video block containing residual transform coefficient values. Wavelet transforms, integer transforms, subband transforms, or other types of transforms may also be used. The transform may convert the residual information from the pixel value domain to a transform domain, such as the frequency domain. The transform scaling and quantization component 213 is also configured to scale the transformed residual information, for example, based on frequency. Such scaling involves applying a scale factor to the residual information so that different frequency information is quantized with different granularity, which may affect the final visual quality of the reconstructed video. The transform scaling and quantization component 213 is also configured to quantize the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the transform scaling and quantization component 213 may then perform a scan of a matrix containing the quantized transform coefficients. The quantized transform coefficients are forwarded to the header formatting and CABAC component 231 and encoded within the bitstream.

[0071] The scaling and inverse transform component 229 applies the inverse operations of the transform scaling and quantization component 213 to support motion estimation. The scaling and inverse transform component 229 applies inverse scaling, transform, and / or quantization to reconstruct residual blocks in the pixel domain, for later use as reference blocks that may serve as predictive blocks for another current block, for example. The motion estimation component 221 and / or motion compensation component 219 can calculate reference blocks by adding the residual blocks to the corresponding predictive blocks for use in motion estimation of subsequent blocks / frames. A filter is applied to the reconstructed reference blocks to mitigate artifacts generated during scaling, quantization, and transform. Otherwise, such artifacts may cause inaccurate predictions (and result in further artifacts) when subsequent blocks are predicted.

[0072] The filter control analysis component 227 and the in-loop filter component 225 apply the filters to residual blocks and / or reconstructed image blocks. For example, a transformed residual block from the scaling and inverse transform component 229 may be combined with a corresponding prediction block from the intra-picture prediction component 217 and / or the motion compensation component 219 to reconstruct an original image block. The filters may then be applied to the reconstructed image block. In some examples, the filters may instead be applied to the residual block. Like the other components in FIG. 2, the filter control analysis component 227 and the in-loop filter component 225 are highly integrated and may be implemented together, but are depicted separately for conceptual purposes. The filters applied to reconstructed reference blocks are applied to specific spatial regions and include multiple parameters for adjusting how such filters are applied. The filter control analysis component 227 analyzes the reconstructed reference blocks to determine where such filters should be applied and set the corresponding parameters. Such data is forwarded as filter control data to the header formatting and CABAC component 231 for encoding. The in-loop filter component 225 applies such filters based on the filter control data. The filters may include a deblocking filter, a noise suppression filter, a signal-over-error (SAO) filter, and an adaptive loop filter. Such filters may be applied in the spatial / pixel domain (e.g., to reconstructed pixel blocks) or in the frequency domain, depending on the example.

[0073] When operating as an encoder, the filtered reconstructed image blocks, residual blocks, and / or prediction blocks are stored in decoded picture buffer 223 for later use in motion estimation as discussed above. When operating as a decoder, decoded picture buffer 223 stores the reconstructed and filtered blocks and transfers them to the display as part of the output video signal. Decoded picture buffer 223 may be any memory device capable of storing prediction blocks, residual blocks, and / or reconstructed image blocks.

[0074] The header formatting and CABAC component 231 receives data from various components of the codec system 200 and encodes such data into a coded bitstream for transmission to a decoder. Specifically, the header formatting and CABAC component 231 generates various headers for encoding control data, such as general control data and filter control data. Additionally, prediction data, including intra-prediction and motion data, and residual data in the form of quantized transform coefficient data are all encoded in the bitstream. The final bitstream contains all information desired by the decoder to reconstruct the original segmented video signal 201. Such information may also include an intra-prediction mode index table (also referred to as a codeword mapping table), definitions of encoding contexts for various blocks, indications of the most likely intra-prediction mode, indications of partition information, and so on. Such data may be encoded using entropy coding. For example, the information may be encoded using context adaptive variable length coding (CAVLC), CABAC, syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding technique. Following entropy coding, the encoded bitstream may be transmitted to another device (e.g., a video decoder) or archived for later transmission or retrieval.

[0075] 3 is a block diagram illustrating an example video encoder 300 capable of implementing intra-prediction. Video encoder 300 may be used to implement the encoding functionality of codec system 200 and / or to implement steps 101, 103, 105, 107, and / or 109 of method 100. Video encoder 300 partitions an input video signal, resulting in a partitioned video signal 301 that is substantially similar to partitioned video signal 201. Partitioned video signal 301 is then compressed and encoded into a bitstream by components of encoder 300.

[0076] Specifically, the segmented video signal 301 is forwarded to an intra-picture prediction component 317 for inter prediction. The intra-picture prediction component 317 may be substantially similar to the intra-picture estimation component 215 and the intra-picture prediction component 217. The segmented video signal 301 is also forwarded to a motion compensation component 321 for inter prediction based on reference blocks in a decoded picture buffer 323. The motion compensation component 321 may be substantially similar to the motion estimation component 221 and the motion compensation component 219. The prediction block and residual block from the intra-picture prediction component 317 and the motion compensation component 321 are forwarded to a transform and quantization component 313 for transforming and quantizing the residual block. The transform and quantization component 313 may be substantially similar to the transform scaling and quantization component 213. The transformed and quantized residual block and the corresponding prediction block (together with associated control data) are forwarded to an entropy coding component 331 for encoding into a bitstream. The entropy coding component 331 may be substantially similar to the header formatting and CABAC component 231.

[0077] The transformed and quantized residual block and / or the corresponding prediction block are also transferred from the transform and quantization component 313 to the inverse transform and quantization component 329 for reconstruction into a reference block for use by the motion compensation component 321. The inverse transform and quantization component 329 may be substantially similar to the scaling and inverse transform component 229. An in-loop filter in the in-loop filter component 325 is also applied to the residual block and / or the reconstructed reference block, depending on the example. The in-loop filter component 325 may be substantially similar to the filter control analysis component 227 and the in-loop filter component 225. The in-loop filter component 325 may include multiple filters, such as a noise suppression filter, a deblocking filter, an SAO filter, and / or an adaptive loop filter. The filtered block is then stored in the decoded picture buffer 323 for use as a reference block by the motion compensation component 321. The decoded picture buffer 323 may be substantially similar to the decoded picture buffer 223.

[0078] As discussed below, the intra picture prediction component 317 may perform intra prediction by selecting an intra prediction mode that uses an alternative reference line related to a neighboring block. To reduce signaling overhead, the intra picture prediction component 317 can determine an intra prediction mode subset that includes a subset of intra prediction modes that have access to the alternative reference line. Modes excluded from the intra prediction mode subset have access to the primary reference line. The intra picture prediction component 317 then has the option of selecting an intra prediction mode that uses the alternative reference line to obtain a better match, or an intra prediction mode that uses the primary reference line to support lower signaling overhead. The intra picture prediction component 317 can also use various other mechanisms to support increased coding efficiency when using alternative reference lines, as discussed below.

[0079] 4 is a block diagram illustrating an example video decoder 400 capable of implementing intra prediction. Video decoder 400 may be used to implement the decoding functionality of codec system 200 and / or to implement steps 111, 113, 115, and / or 117 of method 100. Decoder 400 receives a bitstream, for example, from encoder 300, and generates a reconstructed output video signal based on the bitstream for display to an end user.

[0080] The bitstream is received by the entropy decoding component 433. The entropy decoding component 433 performs the inverse function of the entropy encoding component 331. The entropy decoding component 433 is configured to implement an entropy decoding scheme such as CAVLC, CABAC, SBAC, PIPE coding, or other entropy coding techniques. For example, the entropy decoding component 433 can use header information to provide context for interpreting additional data encoded as codewords in the bitstream. The decoded information includes any desired information for decoding the video signal, such as general control data, filter control data, partition information, motion data, prediction data, and quantized transform coefficients from the residual block. The quantized transform coefficients are forwarded to the inverse transform and quantization component 429 for reconstruction into the residual block. The inverse transform and quantization component 429 can be substantially similar to the inverse transform and quantization component 329.

[0081] The reconstructed residual block and / or prediction block are forwarded to the intra picture prediction component 417 for reconstructing into an image block based on an intra prediction operation. The intra picture prediction component 417 may be substantially similar to the intra picture prediction component 317, but operates in reverse. Specifically, the intra picture prediction component 417 uses the prediction mode to locate a reference block within a frame and applies the residual block to the result to reconstruct an intra predicted image block. The reconstructed intra predicted image block and / or residual block and corresponding inter prediction data are forwarded to the decoded picture buffer component 423 via the in-loop filter component 425, which may be substantially similar to the in-loop filter component 325 and the decoded picture buffer component 323, respectively. The in-loop filter component 425 filters the reconstructed image block, residual block, and / or prediction block, and such information is stored in the decoded picture buffer component 423. The reconstructed image blocks from the decoded picture buffer component 423 are forwarded to the motion compensation component 421 for inter prediction. The motion compensation component 421 may be substantially similar to the motion compensation component 321, but may also operate in reverse. Specifically, the motion compensation component 421 generates a prediction block using a motion vector from a reference block and applies a residual block to the result to reconstruct an image block. The resulting reconstructed block may be forwarded to the decoded picture buffer component 423 via an in-loop filter component 425. The decoded picture buffer component 423 continues to store additional reconstructed image blocks. These can be reconstructed into frames via partition information. Such frames may be arranged in a sequence. This sequence is output to a display as a reconstructed output video signal.

[0082] Similar to the intra picture prediction component 317, the intra picture prediction component 417 may perform intra prediction based on an intra prediction mode using an alternative reference line. Specifically, the intra picture prediction component 417 recognizes the modes assigned to the intra prediction mode subset. For example, the intra prediction mode subset can correspond to a determined MPM list, can be predefined in memory, and / or can be determined based on the intra prediction mode of a neighboring block. In this manner, if the intra prediction mode for the current block is in the intra prediction mode subset, the intra picture prediction component 417 can obtain a reference line index from the bitstream. Otherwise, the intra picture prediction component 417 can speculatively determine that a primary reference line is intended by the encoder. The intra picture prediction component 417 may use various other mechanisms to support increased coding efficiency when using alternative reference lines, as discussed below.

[0083] 5 is a schematic diagram illustrating an example intra-prediction mode 500 used in video coding. For example, the intra-prediction mode 500 may be used by steps 105 and 113 of the method 100, the intra-picture estimation component 215 and the intra-picture prediction component 217 of the codec system 200, the intra-picture prediction component 317 of the encoder 300, and / or the intra-picture prediction component 417 of the decoder 400. Specifically, the intra-prediction mode 500 may be used to compress an image block into a prediction block comprising a selected prediction mode and a remaining residual block.

[0084] As mentioned above, intra prediction involves matching a current image block to corresponding sample(s) of one or more neighboring blocks. The current image block can then be represented as a selected prediction mode index and a residual block, which is much smaller than the total luma / chroma values ​​contained in the current image block. Intra prediction can be used when no reference frame is available or when inter-predictive coding is not used for the current block or frame. Reference samples for intra prediction may be derived from previously coded (or reconstructed) neighboring blocks within the same frame. Advanced Video Coding (AVC), also known as H.264 and H.265 / HEVC, uses reference lines of boundary samples of neighboring blocks as reference samples for intra prediction. Many different intra prediction modes are used to cover different texture or structural characteristics. H.265 / HEVC supports a total of 35 intra prediction modes 500, which spatially correlate the current block to one or more reference samples. Specifically, the intra prediction modes 500 include 33 directional prediction modes indexed as modes 2-34, a DC mode indexed as mode 1, and a planar mode indexed as mode 0.

[0085] During encoding, the encoder matches the luma / chroma values ​​of the current block with the luma / chroma values ​​of corresponding reference samples in reference lines that pass through the edges of neighboring blocks. Once a best match with one of the reference lines is found, the encoder selects one of the directional intra-prediction modes 500 that points to the best-matching reference line. For clarity of discussion, acronyms are used below to refer to specific directional intra-prediction modes 500. DirS represents the starting directional intra-prediction mode (e.g., mode 2 in HEVC) when counting clockwise from the bottom left. DirE represents the ending directional intra-prediction mode (e.g., mode 34 in HEVC) when counting clockwise from the bottom left. DirD represents the intermediate neutral intra-coding mode (e.g., mode 18 in HEVC) when counting clockwise from the bottom left. DirH represents the horizontal directional intra-prediction mode (e.g., mode 10 in HEVC). DirV represents the vertical intra-prediction mode (e.g., mode 26 in HEVC).

[0086] As discussed above, DC mode acts as a smoothing function, deriving a prediction for the current block as the average of all reference samples within a reference line that crosses neighboring blocks, and as discussed above, planar mode returns a prediction that exhibits a smooth transition (e.g., a constant gradient of values) between samples below and to the upper left or above and to the upper right of the reference line of reference samples.

[0087] For prediction modes with a prediction direction from DirS to DirH (including DirS and DirH), the reference samples of the previously coded and reconstructed neighboring blocks in the column to the left of the reference line are used as reference samples. For prediction modes with a prediction direction from DirV to DirE (including DirV and DirE), the reference samples of the previously coded and reconstructed neighboring blocks in the row above the reference line are used as reference samples.

[0088] Although there are many intra-prediction modes 500, not all intra-prediction modes 500 are selected with equal probability during video encoding. Furthermore, the intra-prediction modes 500 selected by neighboring blocks are statistically highly correlated with the intra-prediction mode 500 selected for the current block. Therefore, in some examples, an MPM list may be used. An MPM list is a list containing a subset of the intra-prediction modes 500 that are most likely to be selected. If the intra-prediction mode of the current block is included in an MPM list, the selected mode can be signaled in the bitstream by an MPM list index, which can use a codeword with fewer bins than the number of bins used to uniquely identify all of the intra-prediction modes 500.

[0089] The MPM list may be constructed with the intra-prediction modes 500 of several neighboring decoded blocks and several default intra-prediction modes that generally have a high selection probability. For example, in H.265 / HEVC, an MPM list of length 3 is constructed with the intra-prediction modes of two neighboring blocks (one above and one to the left of the current block). In the case of overlap mode, the MPM list is assigned by default as planar mode, DC mode, or DirV mode, in that order. If the MPM list has a longer length, the MPM list may also include the intra-prediction modes 500 of additional neighboring blocks and / or additional default intra-prediction modes 500. The length and construction scheme of the MPM list may be predefined.

[0090] 6 is a schematic diagram illustrating an example of the directional relationships of blocks 600 in video coding. For example, blocks 600 may be used when selecting intra-prediction modes 500. Thus, blocks 600 may be used by steps 105 and 113 of method 100, intra-picture estimation component 215 and intra-picture prediction component 217 of codec system 200, intra-picture prediction component 317 of encoder 300, and / or intra-picture prediction component 417 of decoder 400. In video coding, blocks 600 are partitioned based on video content and thus may include many rectangles and squares of various shapes and sizes. Blocks 600 are depicted as squares for purposes of illustration and are thus simplified from actual video coding blocks to support clarity of discussion.

[0091] Block 600 includes a current block 601 and a neighboring block 610. The current block 610 is any block being coded at a specified time. The neighboring block 610 is any block immediately adjacent to the left edge or top edge of the current block 601. Video coding generally proceeds from top left to bottom right. Thus, the neighboring block 610 may be encoded and reconstructed prior to coding of the current block 601. When coding the current block 601, the encoder matches the luma / chroma values ​​of the current block 601 with reference sample(s) from a reference line that follows the edge of the neighboring block 610. The match is then used to select an intra-prediction mode, for example, from the intra-prediction modes 500, to point to the matched sample (or samples when DC or planar mode is selected). The selected intra-prediction mode then indicates that the luma / chroma values ​​of the current block 601 are substantially similar to the reference sample corresponding to the selected intra-prediction mode. Any differences can be retained in the residual block. The selected intra-prediction mode is then encoded in the bitstream. At the decoder, the current block 601 can be reconstructed by using the luma / chroma values ​​of reference samples in a selected reference line in a neighboring block 610 that corresponds to the selected intra-prediction mode (together with residual information from the residual block, if any).

[0092] 7 is a schematic diagram illustrating an example of a primary reference line scheme 700 for encoding a block with intra prediction. The primary reference line scheme 700 may be used when selecting an intra prediction mode 500. Thus, the primary reference line scheme 700 may be used by steps 105 and 113 of the method 100, the intra picture estimation component 215 and the intra picture prediction component 217 of the codec system 200, the intra picture prediction component 317 of the encoder 300, and / or the intra picture prediction component 417 of the decoder 400.

[0093] The primary reference line scheme 700 uses a primary reference line 711 of reference samples 712. The primary reference line 711 includes edge samples (e.g., pixels) of neighboring blocks as reference samples 712 for the current block 701. As used herein, a reference sample 712 is a value, such as a chroma or luma value, of a pixel or a subportion thereof. The current block 701 is substantially similar to the current block 601. For purposes of discussion, the primary reference line 711 includes a reference row 713 that includes reference samples 712 above the current block 701. The primary reference line 711 also includes a reference column 714 that includes reference samples 712 to the left of the current block 701. In the primary reference line scheme 700, the primary reference line 711 is used, and the primary reference line 711 is the reference line immediately adjacent to the current block 701. Thus, the current block 701 is matched as closely as possible to the reference samples 712 included in the primary reference line 711 during intra prediction.

[0094] 8 is a schematic diagram illustrating an example of an alternative reference line scheme 800 for encoding a block with intra prediction. The alternative reference line scheme 800 may be used when selecting the intra prediction mode 500. Thus, the alternative reference line scheme 800 may be used by steps 105 and 113 of the method 100, the intra picture estimation component 215 and the intra picture prediction component 217 of the codec system 200, the intra picture prediction component 317 of the encoder 300, and / or the intra picture prediction component 417 of the decoder 400.

[0095] The alternate reference line scheme 800 uses a current block 801 that is substantially similar to the current block 701. Multiple reference lines 811 extend from the current block 801. The reference lines 811 include reference samples 812 that are similar to the reference samples 712. The reference lines 811 are substantially similar to the primary reference line 711, but extend further away from the current block 801. By using the alternate reference lines 811, the matching algorithm has access to more reference samples 812. The presence of more reference samples may, in some cases, result in a better match for the current block 801, which in turn may lead to fewer residual samples after a prediction mode is selected. The alternative reference line scheme 800 may be referred to as multiple lines intra prediction (MLIP) and is discussed in detail in Joint Video Experts Team (JVET) documents JVET-C0043, JVET-C0071, JVET-D0099, JVET-D0131, and JVET-D0149. The reference lines 811 may be numbered from 0 to M, where M is any predetermined constant value. The reference lines 811 may include reference rows 813 and reference columns 814 as shown, which are similar to reference rows 713 and reference columns 714, respectively.

[0096] During encoding, the encoder can select the best match from the reference lines 811 based on the RDO. Specifically, M+1 reference lines are used, from the closest reference line (RefLine0) to the farthest reference line (RefLineM), where M is greater than zero. The encoder selects the reference line with the best rate-distortion cost. The index of the selected reference line is signaled to the decoder in the bitstream. Here, RefLine0 may be referred to as the original reference line (e.g., the primary reference line 711), and RefLine1 through RefLineM may be referred to as additional or alternate reference lines. The additional reference line can be selected for use by any of the intra-prediction modes 500. In some cases, the additional reference line can be selected for use by a subset of the intra-prediction modes 500 (e.g., DirS through DirE may use the additional reference line).

[0097] Once a reference line 812 is selected, the index of the selected reference line from among the reference lines 811 is signaled to the decoder in the bitstream. By using different reference lines, a more accurate prediction signal can be derived, which in some cases can increase coding efficiency by reducing residual samples. However, using alternate reference lines 811 increases signaling overhead as the selected reference line is signaled to the decoder. As the number of reference lines 811 used increases, more bins are used during signaling to uniquely identify the selected reference line. Thus, using alternate reference lines 811 may actually reduce coding efficiency when the matching sample is in a reference line immediately adjacent to the current block 801.

[0098] Despite the compression advantages supported by MLIP, some areas can be improved to achieve higher coding gain. For example, to achieve increased compression, an intra-prediction mode subset may be adopted. Specifically, some intra-prediction modes may be selected to use the alternate reference line 811 to support increased matching accuracy. Other intra-prediction modes may be selected to use the primary reference line 711 and avoid the signaling overhead associated with using the alternate reference line 811. Intra-prediction modes that use the alternate reference line 811 may be included in the intra-prediction mode subset.

[0099] FIG. 9 is a schematic diagram 900 illustrating an example intra-prediction mode subset 930 for use in video coding of intra-prediction modes at an encoder or decoder. The alternative reference line scheme 800 and the primary reference line scheme 700 may be combined / modified to use both the intra-prediction mode list 920 and the intra-prediction mode subset 930. The intra-prediction mode list 920 includes all of the intra-prediction modes 923 (e.g., the intra-prediction modes 500) that can be used in the intra-prediction scheme. Each such intra-prediction mode 923 may be indexed by a corresponding intra-prediction mode index 921. In this example, some of the intra-prediction modes have access to alternative reference lines. The intra-prediction modes with access to alternative reference lines are stored in the intra-prediction mode subset 930 as alternative reference line prediction modes 933. The alternative reference line prediction modes 933 in the intra-prediction mode subset 930 may be indexed by an alternative intra-prediction mode index 931. The alternative intra-prediction mode index 931 is an index value used to number and indicate the alternative reference line prediction mode 933 in the intra-prediction mode subset 930. Because fewer intra-prediction modes are included in the intra-prediction mode subset 930, the alternative intra-prediction mode index 931 may include fewer bins than the intra-prediction mode index 921. Thus, if an intra-prediction mode is included in the intra-prediction mode subset 930, that intra-prediction mode can be matched to an alternative reference line. If an intra-prediction mode is not included in the intra-prediction mode subset 930, that intra-prediction mode can be matched to a primary reference line. This scheme takes advantage of the fact that the intra-prediction mode list 920 includes many intra-prediction modes 923 to cover texture and structural characteristics in detail. However, the likelihood of using an alternative reference line is relatively low. Therefore, a rough direction is sufficient for the alternative reference line.Thus, the intra-prediction modes 923 can be sampled to construct an intra-prediction mode subset 930 using a scheme as discussed below.

[0100] The use of both the intra-prediction mode list 920 and the intra-prediction mode subset 930 allows for a significant increase in coding efficiency. Furthermore, the selection of which intra-prediction modes to include as alternative reference line prediction modes 933 affects coding efficiency. For example, a larger mode range uses more bits to represent the alternative intra-prediction mode index 931, while a smaller mode range uses fewer bits. In one embodiment, the intra-prediction mode subset 930 includes every second intra-prediction mode in [DirS, DirE], where [A, B] denotes the set containing integer element x, and B≧x≧A. Specifically, the intra-prediction mode subset 930 may be associated with prediction modes {DirS, DirS+2, DirS+4, DirS+6, ..., DirE}, where {A, B, C, D} denotes the set containing all elements listed between the braces. Additionally, the number of bits representing the selected intra-prediction mode is reduced in the mode range of the alternative intra-prediction mode index 931. In such a case, the alternative intra-prediction mode index 931 may be derived by dividing the intra-prediction mode index 921 by two.

[0101] In another embodiment, the intra-prediction mode subset 930 includes intra-prediction modes in the MPM list 935. As described above, the MPM list 935 includes a subset of intra-prediction modes that are most likely to be selected. In this case, the intra-prediction mode subset 930 may be configured to include intra-prediction modes of decoded and / or reconstructed neighboring blocks of the current block. A flag may be used in the bitstream to indicate whether the selected intra-prediction mode is included in the MPM list 935. If the selected intra-prediction mode is in the MPM list 935, an MPM list 935 index is signaled. Otherwise, an alternative intra-prediction mode index 931 is signaled. If the selected intra-prediction mode is a primary reference line mode and is not included in the MPM list 935, an intra-prediction mode index 921 may be signaled. In some examples, a binary representation scheme may be used in which the alternative intra-prediction mode index 931 and / or the MPM list 935 index are fixed length. In some examples, the alternative intra-prediction mode index 931 and / or the MPM list 935 index are coded via context-based adaptive binary arithmetic coding (CABAC). Other binary representation and / or entropy coding mechanisms may be used. The binary representation / entropy coding mechanism used is predefined. The mode range of the intra-prediction mode subset 930 includes fewer intra-prediction modes (e.g., coarse modes) than the intra-prediction mode list 920. Thus, if an intra-prediction mode in the MPM list 935 (e.g., the mode of a neighboring block) is not included in the intra-prediction mode subset 930 mode range, the intra-prediction mode index 921 can be divided (e.g., by 2) and rounded to the closest alternative intra-prediction mode index 931 value, for example, by adding or subtracting 1. The rounding mechanism may be predefined.The relative positions of neighboring blocks, the scanning order, and / or the size of the MPM list 935 may also be predefined.

[0102] As a specific example, an encoder operating according to H.265 may use a mode range of size [DirS, DirE] of 33. If every second mode in the intra-prediction mode subset 930 is used, the size of the mode range for the alternative intra-prediction mode index 931 is 17. The size of the MPM list 935 may be 1, and the intra-prediction mode of the upper neighboring decoded block is used to construct the MPM list 935. If the mode of the neighboring block is not in the mode range of the alternative intra-prediction mode index 931 (e.g., mode 3), the mode of the neighboring block is rounded to mode 4 (or 2). As a specific example, if the selected intra-prediction mode is mode 16 according to the intra-prediction mode index 921, and if the selected mode is not in the MPM list 935, the selected intra-prediction mode will have an alternative intra-prediction mode index 931 of 8 (16 / 2=8). If a fixed length binary representation is used, then in this example, 1000 can be used to indicate the selected intra prediction mode.

[0103] The above example / embodiment assumes that the intra-prediction mode subset 930 includes every odd (or every even) of the intra-prediction modes 923. However, while using the mechanism described above, other mechanisms can also be used to populate the intra-prediction mode subset 930. In one example, the intra-prediction mode subset 930 includes every Nth intra-prediction mode in [DirS, DirE], where N is an integer equal to 0, 1, 2, 3, 4, etc. This example can also be written as {DirS, DirS+N, DirS+2N, ..., DirE}. As another example, the intra-prediction mode subset 930 may include every Nth intra-prediction mode in [DirS, DirE] as well as the planar and DC intra-prediction modes.

[0104] In another example, the intra-prediction mode subset 930 includes intra-prediction modes with a high general selection probability. Thus, intra-prediction mode signaling costs are reduced and coding efficiency is improved. In some examples, intra-prediction modes with a dominant direction are generally selected with a higher general selection probability (e.g., completely vertical, horizontal, etc.). Thus, the intra-prediction mode subset 930 may include such modes. For example, the intra-prediction mode subset 930 may include dominant directional intra-prediction modes such as DirS, DirE, DirD, DirV, and DirH. Neighboring modes to the dominant modes may also be included in some examples. As a specific example, the intra-prediction modes DirS, DirE, DirD, DirV, and DirH are included in the intra-prediction mode subset 930 along with neighboring intra-prediction modes with indexes plus or minus N. This may be represented as [DirS,DirS+N], [DirE-N,DirE], [DirD-N,DirD+N], [DirH-N,DirH+N], [DirV-N,DirV+N]. In another example, the intra-prediction mode subset 930 includes the intra-prediction modes DirS, DirE, DirD, DirV, DirH as well as neighboring intra-prediction modes with indices of plus or minus N and DC and planar modes.

[0105] In another example, the intra-prediction mode subset 930 is adaptively constructed and thus is not defined using predefined intra-prediction modes. For example, the intra-prediction mode subset 930 may include intra-prediction modes of decoded blocks (e.g., neighboring blocks 610) neighboring the current block (e.g., current block 601). The neighboring blocks may be located to the left and above the current block. Additional neighboring blocks may also be used. The size and configuration of the intra-prediction mode subset 930 are predefined. In another example, the intra-prediction mode subset 930 includes the intra-prediction modes of the neighboring blocks and certain predefined default intra-prediction modes, such as DC and planar modes. In another example, the intra-prediction mode subset 930 includes the intra-prediction modes in the MPM list 935.

[0106] Upon receiving a reference line index in the bitstream, the decoder determines that an intra-prediction mode list 920 is implied when the reference line index points to a primary reference line. In such a case, the decoder can read the subsequent intra-prediction mode index 921 to determine a corresponding intra-prediction mode 923. When the reference line index points to an additional reference line, an intra-mode subset 930 is implied. In such a case, the decoder can read the subsequent alternative intra-prediction mode index 931 to determine a corresponding alternative reference line prediction mode 933. In some examples, a flag may be used to indicate when the indicated alternative reference line prediction mode 933 is in the MPM list 935. In such a case, the alternative reference line prediction mode 933 may be signaled according to the index used by the MPM list 935.

[0107] As described above, drawing 900 may be used when encoding intra-prediction modes with alternate or primary reference lines. A signaling scheme for encoding such data is discussed below. Specifically, a reference line index may be signaled after the index of the corresponding intra-prediction mode. Furthermore, the reference line index may be conditionally signaled. For example, a reference line index may be signaled if the intra-prediction mode is associated with an alternate reference line, and may be omitted if the intra-prediction mode is associated with a primary reference line. This allows the reference line index to be omitted whenever the intra-prediction mode is not included in the intra-prediction mode subset 930, since there is no need to indicate the reference line index in the case of a primary reference line. This approach significantly improves coding efficiency by reducing signaling overhead. The conditional signaling scheme discussed below is an exemplary implementation of this concept.

[0108] 10 is a schematic diagram illustrating an example conditional signaling representation 1000 for an alternative reference line in a video coding bitstream. For example, the conditional signaling representation 1000 may be used in a bitstream during video coding using an intra-prediction mode such as intra-prediction mode 500 at an encoder or decoder when intra-prediction mode subset 930 is used as part of alternative reference line scheme 800. Specifically, the conditional signaling representation 1000 is used when an encoding device encodes or decodes a bitstream with intra-prediction data that includes a selected intra-prediction mode associated with an alternative reference line. Alternatively, the conditional signaling representation 1100 is used instead when an encoding device encodes or decodes a bitstream with intra-prediction data that includes a selected intra-prediction mode associated with a primary reference line, as discussed below.

[0109] The conditional signaling representation 1000 may include a coding unit 1041 field containing associated partitioning information. Such partitioning information indicates block boundaries to a decoder, allowing the decoder to fill decoded image blocks to generate a frame. The coding unit 1041 field is included for context and may or may not be located adjacent to other fields discussed with respect to the conditional signaling representation 1000. The conditional signaling representation 1000 further includes a flag 1042. The flag 1042 indicates to the decoder that the following information is for an intra-prediction mode associated with an alternative reference line. For example, the flag 1042 may indicate whether the following information is coded as an alternative intra-prediction mode index 931 or an MPM list 935 index. Following the flag 1042, an intra-prediction mode subset index 1043 field is used, for example, to code the alternative intra-prediction mode index 931. In some examples, the intra-prediction mode subset index 1043 field is substituted for the MPM list index field, which holds an MPM list 935 index, as discussed above. In either case, a decoder can decode the selected intra-prediction mode for the associated coding unit 1041 based on the flag 1042 and the index. Because the intra-prediction mode subset index 1043 field indicated that the selected intra-prediction mode relates to an alternative reference line, a reference line index 1044 is also included to indicate to the decoder which reference line contains the matching sample. Thus, the intra-prediction mode subset index 1043 provides the direction of the matching sample in a neighboring block, and the reference line index 1044 provides the distance to the matching sample. Based on this information, the decoder can determine the matching sample, use the matching sample to generate a predictive block, and optionally combine the predictive block with a residual block coded elsewhere in the bitstream to reconstruct a block of pixels.

[0110] 11 is a schematic diagram illustrating an example conditional signaling representation 1100 for a primary reference line in a video coding bitstream. For example, the conditional signaling representation 1100 may be used in a bitstream during video coding using an intra-prediction mode, such as intra-prediction mode 500, in an encoder or decoder when intra-prediction mode list 920 is used as part of primary reference line scheme 700. The conditional signaling representation 1100 may include a field for coding unit 1141 with partition information in a manner similar to that of coding unit 1041. The conditional signaling representation 1100 also includes a flag 1142 substantially similar to flag 1042, but in this case, the flag indicates that the selected intra-prediction mode is not associated with intra-prediction mode subset 930. Based on the information in flag 1142, an intra-prediction mode index 1145 can then be interpreted as a mode range index for intra-prediction mode index 921. Furthermore, since it is determined that the selected intra-prediction mode is associated with the primary reference line, the reference line index is omitted.

[0111] Thus, the conditional signaling representation 1000 and the conditional signaling representation 1100 can be used to signal either an intra-prediction mode using an alternative reference line or an intra-prediction mode using a primary reference line, respectively. These schemes may be used with any of the examples / embodiments discussed above. In one example, an intra-prediction mode in [DirS,DirE] may use an alternative reference line. In such a case, if the selected intra-prediction mode is not included in [DirS,DirE] (e.g., DC or planar mode), there is no need to signal a reference line index. Thus, the reference index is estimated to be equal to 0, where 0 indicates the primary reference line immediately adjacent to the current block. In such a case, the intra-prediction mode subset 930 may not be used, since all directional references use an alternative reference line in such a case. In summary, whenever the intra-prediction mode is not DC or planar mode, a reference line index is signaled in such an example. Furthermore, in such a case, whenever the intra-prediction mode is DC or planar mode, a reference line index is not signaled. In another example, a reference line index is signaled whenever an intra-prediction mode is in the intra-prediction mode subset 930. Thus, whenever an intra-prediction mode is not included in the intra-prediction mode subset 930, a reference line index is not signaled.

[0112] Table 1 below is an exemplary syntax table describing when the reference index is omitted when the selected mode is DC or planar mode. [Table 1] As shown in Table 1 above, if the intra prediction mode is non-planar and non-DC, the reference line index is signaled, in this example, along with the x and y position of the current block.

[0113] Table 2 below is an example syntax table describing when the reference index is signaled if the current intra-prediction mode is included in the intra-prediction mode subset, and is omitted otherwise. The intra-prediction mode subset can be determined according to any of the mechanisms discussed with respect to FIG. [Table 2] As shown in Table 2 above, if the intra prediction mode is in the intra prediction subset, a reference line index is signaled.

[0114] Further modifications can be made to the MLIP scheme, either independently or in combination with the examples / embodiments discussed above. For example, in many MLIP schemes, the DC intra prediction mode is limited to a primary reference line. In such cases, the DC intra prediction mode generates a predicted value that is the average of all reference samples in the reference line. This produces a smoothing effect between blocks. As will be discussed later, the DC intra prediction mode can be extended in the MLIP scheme to generate a predicted value that is the average of all reference samples in multiple reference lines.

[0115] 12 is a schematic diagram illustrating an example mechanism for DC mode intra prediction 1200 using an alternative reference line 1211. DC mode intra prediction 1200 can be used in an encoder or decoder, such as encoder 300 or decoder 400, when performing DC intra prediction according to intra prediction mode 500. The DC mode intra prediction 1200 scheme can be used in conjunction with intra prediction mode subset 930, conditional signaling representations 1000 and 1100, or can be used independently of such examples.

[0116] DC mode intra prediction 1200 uses the current block with alternate reference lines 1211, labeled 0-M. As such, any number of reference lines 1211 can be used. Such reference lines 1211 include reference samples 1212 that are substantially similar to reference line 811 and substantially similar to reference sample 812. The DC predicted value 1201 is then calculated as the average of the reference samples 1212 within the multiple reference lines 1211. In one example, the DC predicted value 1201 is calculated as the average of all reference samples 1212 within all reference lines 1211 0-M, regardless of which of the reference lines 1211 is selected during intra-prediction mode selection. This provides a very robust DC predicted value 1201 for the current block. For example, if there are four reference lines 1211, the average of all reference samples 1212 within the reference lines 1211, indexed 0 through 3, is determined and set as the DC predicted value 1201 for the current block. In such an example, the reference line index may possibly not be signaled in the bitstream.

[0117] In another example, the DC predicted value 1201 is determined based on the selected reference line and the corresponding reference line, thereby allowing the DC predicted value 1201 to be determined based on two correlated reference lines 1211 rather than all or only the reference line with index 0. For example, if four reference lines 1211 are used, the reference lines 1211 may be indexed as 0 through 4. In such a case, Table 3 below shows an example of the corresponding reference line 1211 that is used when a reference line is selected. [Table 3] As shown, correlated reference lines 1211 can be used to determine DC prediction value 1201. In such a case, the decoder can determine both reference lines 1211 once the selected reference line index is signaled in the bitstream.

[0118] In another example, DC mode intra prediction 1200 can always use the reference line with index 0 (the primary reference line) and may also select an additional reference line 1211, if such a selection would more accurately match the DC predicted value 1201 to the value of the pixel block being predicted. Table 4 below shows such an example. In such a case, the decoder may determine both reference lines 1211 once the selected reference line index is signaled in the bitstream. [Table 4]

[0119] Further modifications can be made to the MLIP scheme, either independently or in combination with the examples / embodiments discussed above. For example, in many MLIP schemes, reference lines are indexed based on their distance from the current block. For example, in JVET-C0071, codewords 0, 10, 110, and 111 are used to indicate RefLine0, RefLine1, RefLine2, and RefLine3, respectively. This approach can be improved by assigning shorter codewords to reference lines with a higher statistical probability of being selected. Because many reference lines are signaled in the bitstream, the average codeword length for reference line signaling is reduced when using this mechanism. This results in improved coding efficiency. An exemplary scheme for alternative reference line signaling using codewords is discussed below.

[0120] 13 is a schematic diagram illustrating an example mechanism 1300 for encoding an alternative reference line using a codeword. The mechanism 1300 can be used in an encoder or decoder, such as the encoder 300 or the decoder 400, when performing intra prediction according to the intra prediction mode 500. The mechanism 1300 can be used together with the intra prediction mode subset 930, the conditional signaling representations 1000 and 1100, the DC mode intra prediction 1200, or independently of such examples.

[0121] The mechanism 1300 uses a current block 1301 that is coded with intra-prediction and a corresponding reference line 1311 that includes reference samples. The current block 1301 and the reference line 1311 may be substantially similar to the current block 801 and the reference line 811, respectively. The current block 1301 is matched with one or more reference samples in the reference line 1311. Based on the match, an intra-prediction mode and a reference line 1311 are selected to indicate the matching reference samples. The selected intra-prediction mode is coded in the bitstream as discussed above. Furthermore, an index of the selected reference line can be coded in the bitstream by using a reference line codeword 1315. The reference line codeword 1315 is a binary value that indicates the index of the reference line. Furthermore, the reference line codeword 1315 omits leading zeros to further compress data. Thus, the reference line 1311 can be signaled using a single binary value in some cases.

[0122] In mechanism 1300, the length of the codeword 1315 for a reference line 1311 index may not be related to the distance from the reference line 1311 to the current block 1301. Instead, a codeword 1315 is assigned to a reference line 1311 based on the length of the codeword 1315 and the probability that the corresponding reference line 1311 will be selected as the reference line 1311 used for the current block 1301. In some cases, reference lines 1311 that are farther away from the current block 1301 may be given shorter codewords than reference lines 1311 that are immediately adjacent to the current block 1301. For example, codewords 1315 of 0, 10, 110, and 111 may be used to represent RefLine0, RefLine3, RefLine1, and RefLine2, respectively. Additionally, the first bin or first two bins of codeword 1315 may be context coded, and the last bin(s) may be bypass coded without context. The context may be derived from spatial (e.g., above and left) neighboring blocks. Table 5 shows an example encoding scheme for codeword 1315 when four reference lines 1311 are used. [Table 5] As shown in Table 5, the reference lines 1311 may be indexed as 0 through 3. In one exemplary MLIP scheme, such indices are represented by codewords 1315 based on distance from the current block 1301, with the closest reference line 1311 assigned the smallest codeword and the farthest reference line 1311 assigned the largest codeword 1315. In exemplary schemes 1 and 2, the most likely reference line 1311 is index 0, which receives the shortest codeword 1315. The second most likely reference line 1311 is the farthest reference line (index 3), which receives the second shortest codeword 1315. The remaining codewords 1315 are assigned to indices 1 and 2 in descending or ascending order, depending on the example.

[0123] In another example, the reference line 1311 with index 0 may remain the most likely reference line 1311. Furthermore, the second most likely reference line 1311 may be the reference line 1311 with index 2. This results in Table 6. [Table 6] As shown in Table 6, the most likely reference line 1311 receives index 0, which receives the shortest codeword 1315. The second most likely reference line 1311 is the second furthest reference line (index 2), which receives the second shortest codeword 1315. The remaining codewords 1315 are assigned to indices 1 and 3 in descending or ascending order, depending on the example.

[0124] The above example can be expanded if more reference lines are used. For example, if five reference lines are used, the example in Table 5 may be expanded as shown in Table 7. [Table 7] As shown in Table 7, the most likely reference line 1311 is index 0, which receives the shortest codeword 1315. The second most likely reference line 1311 is the reference line 1311 associated with index 2, which receives the second shortest codeword 1315. The remaining codewords 1315 are assigned to indices 1, 2, 4, and 5 in descending or ascending order, depending on the example.

[0125] In another example, the indexes of the reference lines 1311 can be sorted into classes, and each class can be assigned to a different group of code words 1315, which can be arranged differently. For example, the class A group may receive the shortest code words 1315, and class B may receive the longest code words 1315. The class A code words 1315 and class B code words 1315 may be assigned in ascending or descending order, or any combination thereof. The class construction scheme may be predefined. To clarify this scheme, the following example is provided. The example below uses six reference lines 1311 indexed from 0 to 5, as shown in Table 8 below. The reference lines 1311 can be assigned to any class as desired. For example, the reference lines 1311 indexed as 1, 3, and 5 may be assigned to class A, and the reference lines 1311 indexed as 2 and 4 may be assigned to class B. The reference line 1311 indexed as 0 may always be the most probable selection and therefore may always have the shortest codeword 1315. Class A may be assigned the shortest codeword 1315 other than the codeword 1315 for index 0. Class B may then be assigned the longest codeword 1315. The results of such a scheme are shown in Table 8. [Table 8] In the first example of Table 8, the class A and class B codewords 1315 are both incremented (descending) independently. In the second example of Table 8, the class A codewords 1315 are incremented (descending) while the class B codewords 1315 are decremented (ascending). In the third example of Table 8, the class A codewords 1315 are decremented (ascending) while the class B codewords 1315 are incremented (descending). In the fourth example of Table 9, the class A and class B codewords 1315 are both decremented (ascending) independently. Any number of reference lines and any number of classes may be used in this scheme to generate codewords 1315 as desired.

[0126] Further modifications can be made to the MLIP scheme, either independently or in combination with the examples / embodiments discussed above. For example, the MLIP scheme uses an alternative reference line having both rows and columns. In many MLIP schemes, the same number of rows and columns are used. However, when the MLIP scheme is implemented in hardware, the entire set of rows is stored in memory during encoding, for example, in a line buffer on the central processing unit (CPU) chip. Meanwhile, columns can be stored in a cache and pulled onto the CPU buffer as desired. Thus, using reference line rows is computationally more expensive in terms of system resources than using reference line columns. Because it may be desirable to reduce line buffer memory usage, the MLIP schemes described below use different numbers of reference rows and reference columns. Specifically, the MLIP schemes described below use fewer reference rows than reference columns.

[0127] 14 is a schematic diagram illustrating an example mechanism 1400 for encoding alternative reference lines with different numbers of rows and columns. The mechanism 1400 can be used in an encoder or decoder, such as the encoder 300 or the decoder 400, when performing DC intra prediction according to the intra prediction mode 500. The mechanism 1400 can be used together with the intra prediction mode subset 930, the conditional signaling representations 1000 and 1100, the DC mode intra prediction 1200, the mechanism 1300, or independently of such examples.

[0128] The mechanism 1400 uses a current block 1401 with alternate reference lines 1411 labeled 0-M. As such, any number of reference lines 1411 can be used. Such reference lines 1411 include reference samples that are substantially similar to reference lines 811 and / or 1211 and that are substantially similar to reference samples 812 and / or 1212. The reference lines 1411 include reference rows 1413 that include reference samples located above the current block 1401. The reference line 1411 also includes a reference column 1414 containing reference samples located to the left of the current block 1401. As described above, the MLIP scheme stores all of the reference lines 1413 in on-chip memory during intra prediction, which is resource-intensive. Therefore, to reduce memory usage and / or to achieve a particular tradeoff between coding efficiency and memory usage, the number of reference lines 1413 may be less than the number of reference columns 1414. As shown in FIG. 14, the number of reference lines 1413 may be half the number of reference columns 1414. For example, this can be achieved by removing half of the reference lines 1413 between the first and last reference lines 1413 (e.g., keeping RefLine0 and RefLine3 but removing RefLine1 and RefLine2). In another example, odd or even lines 1413 can be removed (e.g., keeping RefLine0 and RefLine2 but removing RefLine1 and RefLine3, or vice versa). The mechanism 1400 for removing rows 1413 can be predefined.

[0129] 15 is a schematic diagram illustrating another example mechanism 1500 for encoding alternative reference lines having different numbers of rows and columns. The mechanism 1500 can be used in an encoder or decoder, such as the encoder 300 or the decoder 400, when performing DC intra prediction according to the intra prediction mode 500. The mechanism 1500 can be used together with the intra prediction mode subset 930, the conditional signaling representations 1000 and 1100, the DC mode intra prediction 1200, the mechanism 1300, or independently of such examples.

[0130] Mechanism 1500 is substantially similar to mechanism 1400 but uses a different number of reference rows 1513. Specifically, mechanism 1500 uses a current block 1501 with alternate reference lines 1511 labeled 0-M. As such, any number of reference lines 1511 can be used. Such reference lines 1511 are substantially similar to reference lines 811 and / or 1211 and include reference samples that are substantially similar to reference samples 812 and / or 1212. Reference lines 1511 include reference rows 1513 that include reference samples located above current block 1501. Reference lines 1511 also include reference columns 1514 that include reference samples located to the left of current block 1501.

[0131] In this case, the number of reference rows 1513 is the number of reference columns 1514 minus K, where K is a positive integer less than the number of reference columns 1514. K may be predefined or may be signaled in the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), or slice header in the bitstream. In the example shown, the number of reference rows 1513 is the number of reference columns 1514 minus 1. However, any value of K may be used. The reference rows 1513 are removed using any predefined mechanism. In FIG. 15, the reference column 1513 associated with the reference line 1511 indexed as 1 is removed. In other examples, the reference columns 1513 associated with the reference lines 1511 indexed as 2, 3, etc. are removed.

[0132] When using mechanisms 1400 and / or 1500, the reference index of the reference row may be different from the reference index of the reference column. Several mechanisms for indicating or deriving the reference row index when the number of reference rows and columns differ as discussed above are presented below. In one mechanism, the reference column index and the left row index are signaled separately. For example, one syntax element is used to signal a reference row index between 0 and NumTop, inclusive, where NumTop is the number of reference rows. Another syntax element is used to signal a reference column index between 0 and NumLeft, inclusive, where NumLeft is the number of reference columns, inclusive.

[0133] In another mechanism, one syntax element is used to signal the reference line index for both the reference row and the reference column. In most instances, NumLeft is greater than NumTop. However, the mechanism described below also applies to situations where NumTop is greater than NumLeft. If NumLeft > NumTop, in one mechanism the reference column index is signaled using the mechanism described above. In such cases, the reference line index is between 0 and NumLeft inclusive. If a reference line or reference pixel from the reference row is used and the reference line index is greater than NumTop, the reference line associated with the index in NumTop is signaled.

[0134] Alternatively, if NumLeft > NumTop, the reference row index is signaled using the mechanism described above. In such a case, the reference index is between 0 and NumTop (inclusive). If a reference line or pixel from the reference row is used, the signaled (or parsed) reference line index indicates the selected reference line. When a reference sample from the reference column is selected because NumLeft is greater than NumTop, an index mapping table is used to map the signaled reference line index to the selected reference column. Exemplary mapping tables 9-11 are shown below. [Table 9] [Table 10] [Table 11] In some instances, the table mapping as shown above may be replaced by a calculation, depending on the implementation.

[0135] Furthermore, some intra-prediction modes use reference columns as reference lines, while others use reference rows as reference lines, while some intra-prediction modes use both reference rows and reference columns as reference lines. For example, the index range definition may depend on the intra-prediction mode, because the phase process and phase results are determined by the index range.

[0136] As a specific example, an intra-prediction mode that uses a reference column (e.g., [DirS,DirH]) has an index range of [0,NumLeft]. An intra-prediction mode that uses a reference row ([DirV,DirE]) has an index range of [0,NumTop]. There are two possible cases for an intra-prediction mode that uses both a reference column and a reference row. In the first case, the index range is between [0,NumTop] (denoted as Modi1). In this case, NumLeft>NumTop and the index range is [1,NumTop]. Therefore, from the NumLeft reference columns, some reference columns are selected, where the number of selected reference columns is less than or equal to Numtop. The selection mechanism is predefined in the encoder and decoder. In the second case, the index range is between [0,NumLeft] (denoted as Modi2). In this case, a mapping mechanism is used to determine the reference column index and the index reference row, for example, based on the signaled reference line index. The mapping mechanism is predefined. For example, any of the mapping mechanisms in Tables 9 to 11 above may be used.

[0137] As another example, when the number of reference lines is four, the reference rows of reference lines RefLine1 and RefLine3 may be removed to reduce on-chip memory usage. In this case, four reference columns and two reference rows are used. When an intra-prediction mode uses both reference columns and reference rows, the index range is redefined to uniquely signal the selected reference sample(s). In this case, the intra-prediction mode [DirS,DirH] has an index range of [0,3]. For the intra-prediction mode [DirV,DirE], the index range is [0,1].

[0138] For intra-prediction modes that use both reference columns and reference rows (e.g., intra-prediction modes in Planar, DC, or (DirH,DirV)), the index range is [0,1] (Modi1) or the index range is [0,3] (Modi2). When the index range is [0,1] (denoted as Modi1), two of the reference columns are used (e.g., the left portions of RefLine0 and RefLine2 are used). In this case, the number of reference rows and the number of reference columns are both 2. The remaining two reference columns (e.g., RefLine1 and RefLine3) may also be selected for use. The selection mechanism is predefined. When the index range is [0,3] (denoted as Modi2), the number of reference columns is 4, the number of reference rows is 2, and the index range is [0,3]. In this case, an index mapping mechanism is adopted (e.g., a reference column index in [0,1] corresponds to reference row index 0, and a reference column index in [2,3] corresponds to reference row index 1). Other mapping mechanisms can also be used (e.g., column index in [0,2] corresponds to row index 0, column 3 corresponds to row index 1, etc.), however the mapping mechanism used is predefined.

[0139] The reference index encoding scheme and reference line construction scheme can be combined to improve coding efficiency and reduce on-chip memory usage. Below is an example combination of such schemes. In this case, the furthest reference line index is assigned the shortest codeword, and NumTop is half of NumLeft. Furthermore, the number of reference lines is 4, and Ex1 in Table 8 is used. For the index encoding scheme, if NumTop is less than NumLeft, Mod1 is used, and the RefLine0 and RefLine3 columns are retained. This results in the reference line signaling scheme shown in Table 12 below. [Table 12]

[0140] In this example, when the intra prediction mode is between [DirS, DirH], the index range is [0, 3], and the mechanism for representing the reference line index is shown in Table 12. The reference column can be selected as the reference line according to the reference line index. Furthermore, when the intra prediction mode is between [DirV, DirE], the index range is [0, 1], and the mechanism for representing the reference line index is defined in Table 12. The reference row can be selected as the reference line according to the reference line index. In this case, RefLine0 and RefLine3 are taken into account, but RefLine1 and RefLine2 are not taken into account. When the intra prediction mode is between [Planar, DC] and (DirH, DirV), the index range is [0, 1], and the mechanism for representing the reference line index is defined in Table 12. Then, the reference column and the reference row are selected as the reference line according to the reference line index. In this case, RefLine0 and RefLine3 are taken into account, but RefLine1 and RefLine2 are not taken into account. Similarly, the binary representation tables discussed above may be applied to further increase coding efficiency.

[0141] FIG. 16 is a schematic diagram of a video encoding device 1600 according to an embodiment of the present disclosure. The video encoding device 1600 is suitable for implementing the disclosed embodiments as described herein. The video encoding device 1600 includes a downstream port 1620, an upstream port 1650, and / or a transceiver unit (Tx / Rx) 1610 for communicating data upstream and / or downstream over a network. The video encoding device 1600 also includes a processor 1630 including a logic unit and / or central processing unit (CPU) for processing data and a memory 1632 for storing data. The video encoding device 1600 may also include optical-to-electrical (OE), electrical-to-optical (EO), and / or wireless communication components coupled to the upstream port 1650 and / or downstream port 1620 for communicating data over an optical or wireless communication network. The video encoding device 1600 may also include input and / or output (I / O) devices 1660 for communicating data to and from a user. The I / O devices 1660 may include output devices such as a display for displaying video data, speakers for outputting audio data, etc. The I / O devices 1660 may also include input devices such as a keyboard, mouse, trackball, etc. and / or corresponding interfaces for interacting with such output devices.

[0142] The processor 1630 is implemented in hardware and software. The processor 1630 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 1630 communicates with the downstream port 1620, the Tx / Rx 1610, the upstream port 1650, and the memory 1632. The processor 1630 includes an encoding module 1614. The encoding module 1614 implements the disclosed embodiments, such as methods 1700, 1800, 1900, and / or any other mechanisms described above. Additionally, encoding module 1614 may implement codec system 200, encoder 300, decoder 400, perform intra prediction using intra prediction mode 500 with blocks 600, use primary reference line scheme 700, use alternate reference line scheme 800, use intra prediction mode subset 930, use representation 1000 and / or 1100, use DC mode intra prediction 1200, use schemes 1300, 1400, and / or 1500, and any combination thereof. Thus, the inclusion of encoding module 1614 significantly improves the functionality of video encoding device 1600 and implements transformations of video encoding device 1600 to different states. Alternatively, encoding module 1614 may be implemented as instructions stored in memory 1632 and executed by processor 1630 (e.g., as a computer program product stored on a non-transitory medium).

[0143] Memory 1632 may include one or more memory types such as a disk, tape drive, solid state drive, read only memory (ROM), random access memory (RAM), flash memory, ternary content-addressable memory (TCAM), static random access memory (SRAM), etc. Memory 1632 may be used as overflow data storage, to store programs when such programs are selected for execution, and to store instructions and data retrieved during program execution.

[0144] 17 is a flowchart of an example method 1700 of video encoding using an intra-prediction mode subset with an alternative reference line. For example, the method 1700 may operate on the video encoding device 1600 configured to function as the decoder 400. The method 1700 may also use the intra-prediction modes 500 in conjunction with the alternative reference line scheme 800 and the intra-prediction mode subset 930.

[0145] In step 1701, a bitstream is received. The bitstream includes an encoding of compressed video data that has been compressed by a decoder. In step 1703, an intra-prediction mode subset, such as intra-prediction mode subset 930, is determined. The intra-prediction mode subset includes intra-prediction modes that correlate to multiple reference lines for the current image block and excludes intra-prediction modes that correlate to a primary reference line for the current image block. The intra-prediction mode subset may include various groups of intra-prediction modes, as discussed above. In some examples, the intra-prediction mode subset includes DirS, DirE, and every Nth directional intra-prediction mode between DirS and DirE, where N is a predetermined integer value. In other examples, the intra-prediction mode subset may further include a planar prediction mode and a DC prediction mode. In a further example, the intra-prediction mode subset includes DirS, DirE, DirD, DirH, DirV, and valid directional intra-prediction modes in a direction plus or minus N of DirS, DirE, DirD, DirH, and DirV, where N is a predetermined integer value. Such an intra-prediction mode subset may further include a planar prediction mode and a DC prediction mode. In yet another example, the intra-prediction mode subset includes intra-prediction modes selected for decoded neighboring blocks located in a predetermined neighborhood to the current image block. In yet another example, the intra-prediction mode subset includes modes associated with the MPM list for the block. Furthermore, the intra-prediction mode subset may include any other combination of the intra-prediction modes discussed above.

[0146] In optional step 1705, if the selected intra-prediction mode is included in the intra-prediction mode subset, the selected intra-prediction mode is decoded using the intra-prediction mode subset index. In optional step 1707, if the selected intra-prediction mode is not included in the intra-prediction mode subset, the selected intra-prediction mode is decoded using the intra-prediction mode index. The selected intra-prediction mode may, in some cases, be decoded based on the MPM index, as discussed with respect to Figure 9. As discussed with respect to Figures 10-11, flags may be used to provide context for determining the indexing scheme.

[0147] In step 1709, when the selected intra-prediction mode is included in the intra-prediction mode subset, a reference line index is decoded from the encoding. The reference line index indicates a selected reference line from the plurality of reference lines for the selected intra-prediction mode. As discussed above, when the selected intra-prediction mode is not included in the intra-prediction mode subset, the reference line index is not decoded to reduce extra bits in the bitstream. To further support such conditional signaling in the bitstream, the reference line index may be coded after the selected intra-prediction mode in the encoding (e.g., as discussed with respect to Figures 10-11).

[0148] In step 1711, the video data is presented to the user via a display. The video data includes image blocks decoded based on the selected intra-prediction mode and corresponding reference line(s).

[0149] 18 is a flowchart of an example method 1800 of video encoding using DC mode intra prediction with an alternate reference line. For example, the method 1800 may function on a video encoding device 1600 configured to act as a decoder 400. The method 1800 may use the intra-prediction modes 500 in conjunction with the intra-prediction mode subset 930 with the alternate reference line scheme 800 and DC mode intra prediction 1200.

[0150] In step 1801, a bitstream is stored in a memory. The bitstream includes compressed video data. The compressed video data includes an image block coded as a predictive block according to an intra-prediction scheme. In step 1803, a current predictive block encoded using a DC intra-prediction mode is obtained. In step 1805, a DC predicted value is determined for the current predictive block. The DC predicted value approximates a current image block corresponding to the current predictive block by determining an average of all reference samples in at least two of a plurality of reference lines associated with the current predictive block. Thus, step 1805 extends the DC prediction mode to an alternative reference line context. In some examples, determining the DC predicted value may include determining an average of all reference samples in N reference lines adjacent to the current predictive block, where N is a predetermined integer. In some examples, determining the DC predicted value includes determining an average of all reference samples in a selected reference line and a corresponding (e.g., predefined) reference line. In yet another example, determining the DC predicted value includes determining an average of all reference samples in an adjacent reference line (e.g., a reference line with index 0) and a selected reference line signaled in the bitstream. Additionally, the DC predicted value may be determined using any combination of schemes such as those discussed above with respect to Figure 12. In step 1807, the current image block is reconstructed based on the DC predicted value. In block 1809, a frame containing the current image block is displayed to the user.

[0151] 19 is a flowchart of an example method 1900 of video encoding using reference lines coded with codewords based on selection probabilities. For example, method 1900 may function on video encoding device 1600 configured to act as decoder 400. Method 1900 may also use mechanism 1300 to use intra-prediction modes 500 in conjunction with alternative reference line scheme 800 and intra-prediction mode subset 930. Furthermore, as discussed with respect to FIGS. 14 and 15, different numbers of reference rows and columns may be employed by method 1900.

[0152] In step 1901, a bitstream including an encoding is received. The encoding includes video data compressed by an encoder. In step 1903, an intra-prediction mode is decoded from the encoding. The intra-prediction mode indicates a relationship between the current block and reference samples in a selected reference line. Furthermore, the current block is associated with multiple reference lines, including the selected reference line (e.g., an alternative reference line scheme). In step 1905, the selected reference line is decoded based on a selected codeword indicating the selected reference line. The selected codeword includes a length based on the selection probability of the selected reference line, as discussed with reference to FIG. 13 . For example, the multiple reference lines may be represented by multiple codewords. Furthermore, the reference line farthest from the current block may be represented by a codeword with the second shortest length. In another example, the reference line second farthest from the current block may be represented by a codeword with the second shortest length. In another example, a predefined reference line other than an adjacent reference line may be represented by a codeword with the second shortest length. In yet another example, the multiple codewords may be sorted into a class A group and a class B group. The class A group may include codewords of a length shorter than the length of the codewords in the class B group. Furthermore, the class A group and the class B group may be incremented and decremented independently of each other. In a further example, the reference lines may include different numbers of reference rows and reference columns, as discussed with respect to FIGS. 14 and 15. For example, multiple reference lines may include reference rows and reference columns. Furthermore, the number of reference rows stored for the current block may be half the number of reference columns stored for the current block. In another example, the number of reference rows stored for the current block may be equal to the number of reference columns stored for the current block minus one. In another example, the number of reference rows stored for the current block may be selected based on the number of reference rows used by the deblocking filter operation. In step 1907, the video data is presented to the user via a display.The video data includes image blocks decoded based on an intra-prediction mode and a selected reference line. Thus, methods 1700, 1800, and 1900 may be applied alone or in any combination to improve the effectiveness of alternative reference line schemes when encoding video via intra-prediction.

[0153] 1. A video encoding apparatus comprising: receiving means for receiving a bitstream; processing means configured to perform the steps of: determining an intra-prediction mode subset, the intra-prediction mode subset including intra-prediction modes correlated to a plurality of reference lines for a current image block and excluding an intra-prediction mode correlated to a primary reference line for the current image block; decoding a first intra-prediction mode using an alternative intra-prediction mode index if the first intra-prediction mode is included in the intra-prediction mode subset; and decoding the first intra-prediction mode using an intra-prediction mode index if the first intra-prediction mode is not included in the intra-prediction mode subset; and display means for presenting video data including the image block decoded based on the first intra-prediction mode.

[0154] Storing a bitstream including an image block coded as a predictive block in a memory means; obtaining a current predictive block encoded in a direct current (DC) intra-prediction mode by a processing means; determining a DC predicted value for approximating a current image block corresponding to the current predictive block by determining an average of all reference samples in at least two reference lines among a plurality of reference lines associated with the current predictive block; reconstructing the current image block based on the DC predicted value by the processor; and displaying a video frame including the current image block on a display means.

[0155] 1. A video encoding apparatus comprising: receiving means for receiving a bitstream; processing means configured to perform the steps of: decoding an intra-prediction mode from the bitstream, the intra-prediction mode indicating a relationship between a current block and a selected reference line, the current block being associated with a plurality of reference lines including the selected reference line; and decoding the selected reference line based on a selected codeword indicating the selected reference line, the selected codeword including a length based on a selection probability of the selected reference line; and display means for presenting video data including an image block decoded based on the intra-prediction mode and the selected reference line.

[0156] A first component is directly coupled to a second component when there are no intervening components between them, other than a line, trace, or another medium. A first component is indirectly coupled to a second component when there are intervening components between them, other than a line, trace, or another medium. The term "coupled" and variations thereof include both directly coupled and indirectly coupled. The use of the term "about" means a range including ±10% of the subsequent number, unless otherwise specified.

[0157] While several embodiments have been provided in this disclosure, it will be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the disclosure. The examples herein are intended to be illustrative and not limiting, and the intention is not to be limited to the details provided herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0158] Furthermore, the techniques, systems, subsystems, and methods described and illustrated as discrete or separate in various embodiments may be combined or integrated with other systems, components, techniques, or methods without departing from the scope of the present disclosure. Other examples of changes, substitutions, and alterations will be apparent to those skilled in the art and may be made without departing from the spirit and scope disclosed herein.

Claims

1. 1. A method implemented by a video coding device, comprising: receiving a bitstream; determining a value of a reference line index of the coding unit; if the value of the reference line index indicates an additional reference line, determining an intra-prediction mode signaled by a most probable mode list index from a list of most probable modes; and decoding the coding unit based on the intra-prediction mode. method.

2. If the value of the reference line index indicates a primary reference line, the method further includes determining the intra prediction mode from a group of modes excluding a list of most probable modes. The method of claim 1.

3. The method of claim 2 , wherein the group of modes excluding the list of most probable modes is indexed by an intra-prediction mode index.

4. The method of claim 1 , wherein the list of most probable modes includes intra-prediction modes used by neighboring coding units.

5. 4. The method of claim 1, wherein the value of the reference line index, if absent, is assumed to be equal to zero.

6. 1. A method implemented by a video coding device, comprising: determining a value of a reference line index of the coding unit; if the value of the reference line index indicates an additional reference line, determining an intra-prediction mode signaled by a most probable mode list index from a list of most probable modes; encoding the coding unit into a bitstream based on the intra-prediction mode; storing the bitstream in a memory of the video coding device; method.

7. If the value of the reference line index indicates a primary reference line, the method further includes determining the intra prediction mode from a group of modes excluding a list of most probable modes. The method of claim 6.

8. The method of claim 7 , wherein the group of modes excluding the list of most probable modes is indexed by an intra-prediction mode index.

9. The method of claim 6 , wherein the list of most probable modes includes intra-prediction modes used by neighboring coding units.

10. 9. The method of claim 6, wherein if the value of the reference line index is equal to zero, the value of the reference line index is omitted from the bitstream.

11. a memory containing instructions; one or more processors in communication with said instructions; A video decoding device comprising: The one or more processors execute the instructions to perform the method of any one of claims 1 to 3. Video decoding device.

12. a memory containing instructions; one or more processors in communication with said instructions; 1. A video encoding device comprising: The one or more processors execute the instructions to perform the method of any one of claims 6 to 8. Video encoding device.

13. 1. A video coding apparatus comprising: a transceiver configured to receive or transmit a bitstream, the bitstream including a reference line index, wherein if the reference line index indicates a primary reference line, a first intra-prediction mode for a coding unit is determined from an intra-prediction mode list; and if the reference line index indicates one of additional reference lines, a second intra-prediction mode for the coding unit is determined from a subset of the intra-prediction mode list, the subset of the intra-prediction mode list including only intra-prediction modes associated with a most probable mode (MPM) list; a storage device configured to store the bitstream; Device.

14. 1. A video coding method comprising: receiving or transmitting a bitstream, the bitstream including a reference line index, wherein if the reference line index indicates a primary reference line, a first intra-prediction mode for a coding unit is determined from an intra-prediction mode list; and if the reference line index indicates one of additional reference lines, a second intra-prediction mode for the coding unit is determined from a subset of the intra-prediction mode list, the subset of the intra-prediction mode list including only intra-prediction modes associated with a most probable mode (MPM) list; storing the bitstream. method.

Images