Method for generating mode list for multi-line intra prediction, and apparatus and computer program therefor

Abstract

To provide a method for signaling an intra prediction mode used to encode a current block.SOLUTION: The method includes generating a first most probable mode list (MPM list) corresponding to a zero reference line of the current block, wherein the first MPM list includes a first plurality of intra prediction modes, generating a second MPM list corresponding to one or more non-zero reference lines of the current block, wherein the second MPM list includes a second plurality of intra prediction modes, wherein the second plurality of intra prediction modes includes a subset of the first plurality of intra prediction modes, signaling a reference line index indicating a reference line used to encode the current block, and signaling an intra mode index indicating an intra prediction mode among the first MPM list and the second MPM list.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority under 35 U.S.C. §119 in the United States Patent and Trademark Office to U.S. Provisional Application No. 62 / 742,252, filed October 5, 2018, and U.S. Patent Application No. 16 / 511,626, filed July 15, 2019, which are incorporated herein by reference in their entireties.

[0002] FIELD OF THE DISCLOSURE This disclosure relates to advanced video coding techniques, and more particularly to a mode list generation scheme for multi-line intra prediction. [Background technology]

[0003] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) published the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4) [1]. In 2015, these two standards organizations jointly formed the Joint Video Exploration Team (JVET) to explore the possibility of developing the next video coding standard beyond HEVC. In October 2017, these standards organizations issued a joint call for proposals (CfP) for video compression with performance beyond HEVC. By February 15, 2018, a total of 22 CfP responses had been submitted for standard dynamic range (SDR), 12 for high dynamic range (HDR), and 12 for 360 video categories. In April 2018, all received CfP responses were evaluated at the 122 MPEG / 10th JVET Meeting. As a result of this meeting, JVET officially launched the standardization process for next-generation video coding beyond HEVC. The new standard was named VVC (Versatile Video Coding), and JVET was renamed the Joint Video Expert Team.

[0004] The intra prediction modes used in HEVC are shown in Figure 1. In HEVC, there are a total of 35 intra prediction modes, of which mode 10 is the horizontal mode, mode 26 is the vertical mode, and mode 2, mode 18, and mode 34 are diagonal modes. The intra prediction modes are signaled by three most probable modes (MPMs) and the remaining 32 modes.

[0005] To encode an intra mode, a Most Probable Mode (MPM) list of size 3 is created based on the intra modes of neighboring blocks. This MPM list is called the MPM list or primary MPM list. If the intra mode is not from the MPM list, a flag is signaled to indicate whether the intra mode belongs to the selected mode.

[0006] An example of the MPM list generation process for HEVC is shown below. ·If(leftIntraDir==aboveIntraDir&&leftIntraDir>DC_IDX) MPM[0]=leftIntraDir; MPM[1]=((leftIntraDir+offset)%mod)+2; MPM[2]=((leftIntraDir-1)%mod)+2; ·Else if(leftIntraDir==aboveIntraDir) MPM[0]=PLANAR_IDX; MPM[1]=DC_IDX; MPM[2]=VER_IDX; ·Else if(leftIntraDir!=aboveIntraDir) MPM[0]=leftIntraDir; MPM[1]=aboveIntraDir; If(leftIntraDir>0&&aboveIntraDir>0) ·MPM[2]=PLANAR_IDX; Else ·MPM[2]=(leftIntraDir+aboveIntraDir)<2?VER_IDX:DC_IDX;

[0007] Here, leftIntraDir is used to indicate the mode of the left block, and aboveIntraDir is used to indicate the mode of the above block. If the left or above block is not currently available, leftIntraDir or aboveIntraDir will be DC_IDX. Also, the variables "offset" and "mod" are constant values ​​set to 29 and 32, respectively.

[0008] Multi-line intra prediction is proposed to use more reference lines for intra prediction, and the encoder determines and signals the reference lines used to generate the intra predictor. The reference line index is signaled before the intra prediction mode, and if a non-zero reference line index is signaled, the planar mode / DC mode is excluded from the intra prediction mode. Figure 2 shows an example of four reference lines, each of which consists of six segments, i.e., segments A to F, with a reference sample at the top left. Furthermore, segments A and F are filled with the nearest samples from segments B and E, respectively.

[0009] For multi-line intra prediction, if the modes available for the non-zero reference lines are the same as those for the zero reference line, the coding complexity of multi-line intra prediction becomes very high. Therefore, it is necessary to reduce the number of intra prediction modes available for the non-zero reference lines. Summary of the Invention [Means for solving the problem]

[0010] In one embodiment, a method performed by at least one processor for signaling (also referred to herein as "indicating") an intra-prediction mode to be used to encode a current block in an encoded video bitstream includes the steps of: generating a first Most Probable Mode (MPM) list (MPM list) corresponding to a zero reference line of the current block, the first MPM list including a first plurality of intra-prediction modes; and generating a second MPM list corresponding to one or more non-zero reference lines of the current block, the second MPM list including a second plurality of intra-prediction modes, the second plurality of intra-prediction modes including a first plurality of intra-prediction modes. a step of signaling a reference line index indicating a reference line to be used for encoding the current block from a zero reference line and one or more non-zero reference lines, including a subset of the prediction modes; a step of signaling a reference line index indicating an intra prediction mode from among a zero reference line and one or more non-zero reference lines; and a step of signaling an intra mode index indicating an intra prediction mode, wherein the intra mode index indicates an intra prediction mode in a first MPM list based on the reference line index indicating that the reference line is the zero reference line, or the intra mode index indicates an intra prediction mode in a second MPM list based on the reference line index indicating that the reference line is one of the one or more non-zero reference lines.

[0011] In one embodiment, a device for signaling an intra-prediction mode to be used to encode a current block in an encoded video bitstream comprises: at least one memory configured to store program code; and at least one processor configured to read and operate as instructed by the program code, the program code comprising: first generation code configured to cause the processor to generate a first Most Probable Mode List (MPM list) corresponding to a zero reference line of the current block, the first MPM list including a first plurality of intra-prediction modes; and second generation code configured to cause the processor to generate a second MPM list corresponding to one or more non-zero reference lines of the current block, the second MPM list including a second plurality of intra-prediction modes, A device is provided that includes: a second generation code, wherein the plurality of intra prediction modes includes a subset of the first plurality of intra prediction modes; a first notification code configured to cause a processor to signal a reference line index indicating a reference line from among a zero reference line and one or more non-zero reference lines to be used to encode the current block; and a second notification code configured to cause the processor to signal an intra mode index indicating the intra prediction mode, wherein the intra mode index indicates an intra prediction mode in a first MPM list based on the reference line index indicating that the reference line is the zero reference line, or the intra mode index indicates an intra prediction mode in a second MPM list based on the reference line index indicating that the reference line is one of the one or more non-zero reference lines.

[0012] In one embodiment, a non-transitory computer-readable medium storing one or more instructions, the one or more instructions, when executed by one or more processors of a device that signals an intra-prediction mode to be used to encode a current block in an encoded video bitstream, causes the one or more processors to: generate a first Most Probable Mode List (MPM list) corresponding to a zero reference line of the current block, the first MPM list including a first plurality of intra-prediction modes; and generate a second MPM list corresponding to one or more non-zero reference lines of the current block, the second MPM list including a second plurality of intra-prediction modes, the second plurality of intra-prediction modes including a first plurality of intra-prediction modes. A non-transitory computer-readable medium is provided that generates a second MPM list including a subset of a plurality of intra-prediction modes, signals a reference line index indicating a reference line from among a zero reference line and one or more non-zero reference lines to be used to encode a current block, and signals an intra-mode index indicating an intra-prediction mode, wherein the intra-mode index indicates an intra-prediction mode in the first MPM list based on the reference line index indicating that the reference line is the zero reference line, and the intra-mode index indicates an intra-prediction mode in the second MPM list based on the reference line index indicating that the reference line is one of the one or more non-zero reference lines.

[0013] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram illustrating an example of an intra prediction mode in HEVC. [Figure 2] FIG. 10 is a diagram showing an example of a reference line adjacent to a coding block unit. [Figure 3]FIG. 10 is a diagram illustrating an example of an intra prediction mode in VVC. [Figure 4] 1 is a simplified block diagram of a communication system according to one embodiment. [Figure 5] FIG. 1 is a diagram of an arrangement of a video encoder and a video decoder in a streaming environment according to one embodiment. [Figure 6] FIG. 2 is a functional block diagram of a video decoder according to one embodiment. [Figure 7] FIG. 1 is a functional block diagram of a video encoder according to an embodiment. [Figure 8] 1 is a flowchart of an exemplary process for signaling an intra-prediction mode used to encode a current block in a coded video bitstream, according to one embodiment. [Figure 9] FIG. 1 is a diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] In VVC, there are a total of 95 intra-prediction modes, as shown in Figure 3, where mode 18 is the horizontal mode, mode 50 is the vertical mode, and modes 2, 34, and 66 are the diagonal modes. Modes 1 to 10 and modes 67 to 76 are sometimes called Wide-Angle Intra Prediction (WAIP) modes.

[0016] In VTM2.0.1, the size of the MPM list is 3, and the MPM list generation process is the same as in HEVC. One difference is that VTM2.0.1 has 67 signaled modes, so "offset" is changed to 61 and "mod" is changed to 64.

[0017] In the next section, we describe the encoding process of the luma intra mode, i.e., IntraPredModeY[xPb][yPb] can be derived by the following ordered steps: 1. Set the adjacent positions (xNbA, yNbA) and (xNbB, yNbB) to be equal to (xPb - 1, yPb) and (xPb, yPb - 1), respectively. 2. When X is replaced by either A or B, derive the variable candIntraPredModeX as follows. - 6.4. As specified in Section X [Ed.(BB): Neighbouring blocks availability checking process tbd], call the block availability derivation process with the location (xCurr, yCurr) set equal to (xPb, yPb) and the adjacent location (xNbY, yNbY) set equal to (xNbX, yNbX), and assign the output to availableX. - Derive the candidate intra prediction mode candIntraPredModeX as follows. - If one or more of the following conditions are met, set candIntraPredModeX equal to INTRA_DC. - The variable availableX is equal to FALSE. - CuPredMode[xNbX][yNbX] is not equal to MODE_INTRA. - X is equal to B and yPb - 1 is less than ((yPb >> CtbLog2SizeY) << CtbLog2SizeY). - Otherwise, set candIntraPredModeX equal to IntraPredModeY[xNbX][yNbX]. 3. Derive candModeList[x] for x = 0..2 as follows. - If candIntraPredModeB is equal to candIntraPredModeA, apply the following. - If candIntraPredModeA is less than 2 (i.e., equal to INTRA_PLANAR or INTRA_DC), derive candModeList[x] for x = 0..2 as follows. candModeList[0]=INTRA_PLANAR candModeList[1]=INTRA_DC candModeList[2]=INTRA_ANGULAR50 - Otherwise, derive candModeList[x] for x=0..2 as follows: candModeList[0]=candIntraPredModeA candModeList[1]=2+((candIntraPredModeA+61)%64) candModeList[2]=2+((candIntraPredModeA-1)%64) Otherwise (candIntraPredModeB is not equal to candIntraPredModeA), then apply the following: - Derive candModeList[0] and candModeList[1] as follows: candModeList[0]=candIntraPredModeA candModeList[1]=candIntraPredModeB - If neither candModeList[0] nor candModeList[1] is equal to INTRA_PLANAR, then set candModeList[2] equal to INTRA_PLANAR. - Else, if neither candModeList[0] nor candModeList[1] is equal to INTRA_DC, then set candModeList[2] equal to INTRA_DC. - Otherwise, set candModeList[2] equal to INTRA_ANGULAR50. 4. Derive IntraPredModeY[xPb][yPb] by applying the following procedure: - If intra_luma_mpm_flag[xPb][yPb] is equal to 1, set IntraPredModeY[xPb][yPb] equal to candModeList[intra_luma_mpm_idx[xPb][yPb]]. Otherwise, derive IntraPredModeY[xPb][yPb] by applying the following ordered steps: 1. Modify the array candModeList[x], x=0..2, by the following ordered steps: If i.candModeList[0] is greater than candModeList[1], swap both values ​​as follows: (candModeList[0],candModeList[1])=Swap(candModeList[0],candModeList[1]) ii.If candModeList[0] is greater than candModeList[2], swap both values ​​as follows: (candModeList[0],candModeList[2])=Swap(candModeList[0],candModeList[2]) iii.If candModeList[1] is greater than candModeList[2], swap both values ​​as follows: (candModeList[1],candModeList[2])=Swap(candModeList[1],candModeList[2]) 2. Derive IntraPredModeY[xPb][yPb] by the following ordered steps: i. Set IntraPredModeY[xPb][yPb] equal to intra_luma_mpm_remainder[xPb][yPb]. ii. For i between 0 and 2, if IntraPredModeY[xPb][yPb] is greater than or equal to candModeList[i], increment the value of IntraPredModeY[xPb][yPb] by 1.

[0018] Set the variable IntraPredModeY[x][y] equal to IntraPredModeY[xPb][yPb] where x=xPb..xPb+cbWidth-1 and y=yPb..yPb+cbHeight-1.

[0019] In the development of VVC, an MPM list of size 6 is proposed. The planar mode and the DC mode may always be included in the MPM list. Two adjacent modes, namely the left-hand mode and the upper-hand mode, may be used to generate the remaining four MPMs.

[0020] In VTM4.0, the size of the MPM list has been expanded to 6. If intra_luma_mpm_flag is true, it indicates that the current mode belongs to the candidate MPM list. Consider Table 1 below. [Table 1]

[0021] FIG. 4 shows a simplified block diagram of a communication system (300) according to one embodiment of the present disclosure. The communication system (300) may include at least two terminals (410-420) interconnected via a network (450). In the case of one-way data transmission, a first terminal (410) may locally encode video data for transmission to another terminal (420) via the network (450). The second terminal (420) may receive the encoded video data of the other terminal from the network (450), decode the encoded data, and display the recovered video data. One-way data transmission may be common in media delivery applications, etc.

[0022] 4 illustrates a second pair of terminals (430, 440) provided to support two-way transmission of encoded video, such as might occur during a video conference. For two-way transmission of data, each terminal (430, 440) may encode video data captured at its local location for transmission to the other terminal over the network (450). Each terminal (430, 440) may also receive encoded video data transmitted by the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[0023] In FIG. 4 , the terminals (410-440) may be depicted as servers, personal computers, and smartphones, although the principles of the present disclosure are not so limited. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. The network (450) represents any number of networks that convey coded video data between the terminals (410-440), including, for example, wired and / or wireless communication networks. The communication network (450) may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of the network (450) may not be important to the operation of the present disclosure, unless otherwise described herein.

[0024] 5 shows the arrangement of a video encoder and a video decoder in a streaming environment as an example application for the disclosed subject matter, which may be equally applicable to other video-enabled applications including, for example, video conferencing, digital television, and storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0025] The streaming system may include a capture subsystem (513), which may include a video source (501), such as a digital camera, that creates an uncompressed video sample stream (502). The sample stream (502), shown with a bold line to emphasize its high data volume compared to an encoded video bitstream, may be processed by an encoder (503) coupled to the camera (501). The encoder (503) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream (504), shown with a thin line to emphasize its low data volume compared to the sample stream, may be stored on a streaming server (505) for future use. One or more streaming clients (506, 508) may access the streaming server (505) to obtain copies (507, 509) of the encoded video bitstream (504). The client (506) may include a video decoder (510) that decodes a copy of an input, encoded video bitstream (507) and creates an output video sample stream (511) that can be rendered on a display (512) or other rendering device (not shown). In some streaming systems, the video bitstreams (504, 507, 509) may be encoded according to a particular video encoding / compression standard. Examples of these standards include ITU-T Recommendation H.265. Video coding standards under development are informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

[0026] FIG. 6 may be a functional block diagram of a video decoder (510) according to one embodiment of the present invention.

[0027] The receiver (610) can receive one or more codec video sequences to be decoded by the decoder (510), and in the same or another embodiment, can receive one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel (612), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (610) may receive the video data along with other data, such as coded audio data and / or auxiliary data streams, which may be forwarded to a respective using entity (not shown). The receiver (610) may separate the coded video sequences from the other data. To combat network jitter, a buffer memory (615) may be coupled between the receiver (610) and the entropy decoder / parser (620) (hereinafter "parser"). When the receiver 610 is receiving data from a store-and-forward device with sufficient bandwidth and controllability, or from an isochronous network, the buffer 615 may not be needed or may be small. If a buffer 615 is needed for use with a best-effort packet network such as the Internet, the buffer 615 may be relatively large and advantageously adaptively sized.

[0028] The video decoder (510) may include a parser (620) for reconstructing symbols (621) from the entropy-encoded video sequence. These symbol categories include information used to manage the operation of the decoder (510) and may include information for controlling a rendering device, such as a display (512), which is not an integral part of the decoder but may be coupled to the decoder, as shown in FIG. 6. The rendering device control information may be in the form of a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (620) may parse and entropy decode the received encoded video sequence. The encoding of the encoded video sequence may conform to a video coding technology or standard and may follow principles well known to those skilled in the art, including variable-length coding, Huffman coding, context-sensitive or non-context-sensitive arithmetic coding, etc. The parser (620) may extract a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroups may include Groups of Pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. The entropy decoder / parser may also extract transform coefficients, quantization parameter (QP) values, motion vectors, etc. from the coded video sequence information.

[0029] The parser (620) may perform entropy decoding / parsing operations on the video sequence received from the buffer (615) to create symbols (621). The parser (620) may receive the encoded data and selectively decode particular symbols (621). Additionally, the parser (620) may determine whether a particular symbol (621) should be provided to the motion compensation prediction unit (653), the scaler / inverse transform unit (651), the intra prediction unit (652), or the loop filter (656).

[0030] The reconstruction of the symbols (621) can include several different units, depending on the type of coded video picture or portion thereof (interpicture and intrapicture, interblock and intrablock, etc.), and other factors. Which units are included and how can be controlled by subgroup control information parsed by the parser (620) from the coded video sequence. The flow of such subgroup control information between the parser (620) and the following units is not shown for clarity.

[0031] In addition to the functional blocks already mentioned, the decoder (510) may be conceptually subdivided into several functional units, as described below. In an actual implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

[0032] The first unit is the scalar / inverse transform unit (651), which receives the quantized transform coefficients as symbols (621) from the parser (620), along with control information including which transform to use, block size, quantization factor, quantization scaling matrix, etc. This allows it to output blocks containing sample values ​​that can be input to the aggregator (655).

[0033] In some cases, the output samples of the scaler / inverse transform (651) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (652). In some cases, the intra-picture prediction unit (652) uses surrounding, already reconstructed information fetched from the current (partially reconstructed) picture (656) to generate blocks of the same size and shape as the block being reconstructed. The aggregator (655) optionally adds, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (652) to the output sample information provided by the scaler / inverse transform unit (651).

[0034] In other cases, the output samples of the scalar / inverse transform unit (651) may relate to an inter-coded, possibly motion-compensated, block. In such cases, the motion-compensated prediction unit (653) can access the reference picture memory (657) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (621) related to the block, these samples can be added by the aggregator (655) to the output of the scalar / inverse transform unit (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory from which the motion compensation unit fetches prediction samples can be controlled by motion vectors, which are available to the motion compensation unit in the form of symbols (621), which can have, for example, X, Y, and reference picture components. Motion compensation also includes interpolation of sample values ​​fetched from the reference picture memory, motion vector prediction mechanisms, etc., when sub-sample accurate motion vectors are used.

[0035] The output samples of the aggregator (655) can be subjected to various loop filtering techniques in a loop filter unit (656). Video compression techniques can include in-loop filtering techniques controlled by parameters contained in the coded video bitstream and provided to the loop filter unit (656) as symbols (621) from the parser (620), but can also react to previously reconstructed, loop-filtered sample values ​​as well as meta-information obtained during the decoding of coded pictures or previous (in decoding order) portions of the coded video sequence.

[0036] The output of the loop filter unit (656) can be a sample stream that can be output to a rendering device (512) or stored in a reference picture memory (656) for use in future inter-picture prediction.

[0037] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. Once a coded picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (620)), the current reference picture (656) can become part of the reference picture buffer (657), and a new current picture memory can be reallocated before starting reconstruction of the next coded picture.

[0038] The video decoder (510) may perform decoding operations according to a given video compression technique, such as those documented in standards such as ITU-T Rec. H.265. An encoded video sequence may conform to the syntax specified by the video compression technique or standard being used in the sense that it follows the syntax of the video compression technique or standard as specified in the video compression technique document or standard, particularly the profile document therein. Compliance also requires that the complexity of the encoded video sequence be within a range defined by the level of the video compression technique or standard. In some cases, the level constrains the maximum picture size, maximum frame rate, maximum reconstructed sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may in some cases be further constrained by the Hypothetical Reference Decoder (HRD) specification and HRD buffer management metadata signaled in the encoded video sequence.

[0039] In one embodiment, the receiver (610) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0040] FIG. 7 may be a functional block diagram of a video encoder (503) according to one embodiment of the present disclosure.

[0041] The encoder (503) may receive video samples from a video source (501) that is not part of the encoder and that may capture video to be encoded by the encoder (503).

[0042] The video source (501) may provide a source video sequence to be encoded by the encoder (503) in the form of a digital video sample stream, which may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 Y CrCB, RGB, etc.), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media presentation system, the video source (501) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (503) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that, when viewed sequentially, impart motion. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples, depending on the sampling structure, color space, etc., in use. Those skilled in the art will readily understand the relationship between pixels and samples. The following discussion will focus on samples.

[0043] According to one embodiment, the encoder (503) may encode and compress pictures of a source video sequence into an encoded video sequence (743) in real time or under any other time constraint required by the application. Enforcing an appropriate encoding rate is one of the functions of the controller (750). The controller controls and is operatively coupled to other functional units, as described below. For clarity, coupling is not depicted. Parameters set by the controller may include rate control-related parameters (picture skip, quantization, lambda value for rate-distortion optimization techniques, ...), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc. Those skilled in the art can readily identify other functions of the controller (750) as they may pertain to a video encoder (503) optimized for a particular system design.

[0044] Some video encoders operate in what those skilled in the art readily recognize as a "coding loop." As an oversimplified explanation, the encoding loop can consist of an encoding portion of the encoder (730) (hereafter "source coder") (responsible for creating symbols based on the input picture to be encoded and reference pictures), and a (local) decoder (733) embedded in the encoder (503) that reconstructs the symbols to create sample data that a (remote) decoder will also create (since any compression between the symbols and the encoded video bitstream is lossless in the video compression techniques contemplated in the disclosed subject matter). That reconstructed sample stream is input to a reference picture memory (734). Because decoding of the symbol stream produces bit-perfect results regardless of the decoder's location (local or remote), the contents of the reference picture buffer are also bit-perfect between the local and remote encoders. In other words, the predictive portion of the encoder "sees" the exact same sample values ​​as the decoder would "see" when using prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, for example due to channel errors) is well known to those skilled in the art.

[0045] The operation of the "local" decoder (733) can be the same as the operation of the "remote" decoder (510), which has already been described in detail above in connection with Figure 6. Briefly referring also to Figure 6, the entropy decoder portion of the decoder (510), including the channel (612), receiver (610), buffer (615), and parser (620), need not be fully implemented in the local decoder (733), because symbols are available and the encoding / decoding of symbols for the encoded video sequence by the entropy coder (745) and parser (620) can be lossless.

[0046] An observation that can be made at this point is that any decoder technology other than parsing / entropy decoding that is present in the decoder must necessarily be present in the corresponding encoder in substantially identical functional form. A description of the encoder technology can be omitted, as it is the inverse of the decoder technology that has been comprehensively described. Only in certain areas is a more detailed description necessary, and is provided below.

[0047] As part of its operation, the source coder (730) may perform motion-compensated predictive coding, which predictively codes an input frame with reference to one or more previously coded frames from a video sequence designated as “reference frames.” In this manner, the coding engine (732) codes differences between pixel blocks of the input frame and pixel blocks of reference frames that may be selected as predictive references for the input frame.

[0048] The local video decoder (733) may decode the coded video data of frames that may be designated as reference frames based on symbols created by the source coder (730). The operation of the coding engine (732) may advantageously be a lossy process. When the coded video data is decoded by a video decoder (not shown in FIG. 6 ), the reconstructed video sequence may typically be a replica of the source video sequence, with some error. The local video decoder (733) may create a replica of the decoding process that may be performed by the video decoder on the reference frames and store the reconstructed reference frames in a reference picture cache (734). In this way, the encoder (503) can locally store copies (free of transmission errors) of reconstructed reference frames that have content in common with reconstructed reference frames obtained by the far-end video decoder.

[0049] The predictor (735) may perform the prediction search for the coding engine (732). That is, for a new frame to be encoded, the predictor (735) may search the reference picture memory (734) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., that serve as suitable prediction references for the new picture. The predictor (735) can operate sample block by sample block, pixel block by pixel block, to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (735), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (734).

[0050] The controller (750) may manage the encoding operations of the video coder (730), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0051] The output of all the aforementioned functional units may be entropy coded in an entropy coder (745) that converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc.

[0052] The transmitter (740) may buffer the coded video sequence produced by the entropy coder (745) and prepare it for transmission over a communication channel (760), which may be a hardware / software link to a storage device that stores the coded video data. The transmitter (740) may also merge the coded video data from the video coder (730) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0053] The controller (750) may manage the operation of the encoder (503). During encoding, the controller (750) may assign each coded picture a particular coding picture type, which may affect the coding technique that may be applied to the respective picture. For example, pictures are often assigned as one of the following frame types:

[0054] An Intra Picture (I-Picture) is one that can be coded and decoded without using other frames in the sequence as a source of prediction. Some video codecs allow for different types of Intra Pictures, such as Independent Decoder Refresh Pictures. Those skilled in the art are aware of these variations of I-Pictures and their respective uses and characteristics.

[0055] A predictive picture (P picture) may be one that can be coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0056] A bi-directionally predictive picture (B-picture) may be encoded and decoded using intra- or inter-prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, a multiple predictive picture may use more than two reference pictures and associated metadata to reconstruct a single block.

[0057] A source picture is typically spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coded block by block. Blocks are predictively coded with reference to other (already coded) blocks, as determined by the coding assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded, or they may be predictively coded with reference to previously coded blocks of the same picture (spatial or intra-prediction). Pixel blocks of a P-picture may be predictively coded by spatial prediction or temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be predictively coded by spatial prediction or temporal prediction with reference to one or two previously coded reference pictures.

[0058] The video coder (503) may perform coding operations in accordance with a predetermined video coding technique or standard, such as ITU-T Rec. H.265. In doing so, the video coder (503) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancies in the input video sequence. Thus, the coded video data may conform to the syntax specified by the video coding technique or standard being used.

[0059] In one embodiment, the transmitter (740) may transmit additional data along with the coded video. The video coder (730) may include such data as part of the coded video sequence. The additional data may include temporal, spatial, or SNR enhancement layers, redundant pictures or slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, or other forms of redundant data.

[0060] In an embodiment, the line index of the closest reference line may be 0. The closest reference line may be called the zero reference line, and other lines may be called non-zero reference lines. The maximum reference line number that may be reported is denoted as N.

[0061] In an embodiment, for non-zero reference lines, intra mode indexes may be signaled from a mode list that is a subset of the zero reference line MPM list, i.e., a secondary MPM list. Thus, modes not included in the zero reference line MPM list are not used for non-zero reference lines, and reference line indexes may be signaled before intra prediction modes.

[0062] In one embodiment, planar and DC modes in the zero reference line MPM list are excluded from the non-zero reference line mode list, and all other modes in the zero reference line MPM list are used for the non-zero reference lines.

[0063] In another embodiment, the mode list generation process for non-zero reference lines is the same as the zero reference line MPM list generation process, except that planar modes and DC modes are excluded from the mode list for non-zero reference lines.

[0064] In one embodiment, the size of the mode list for non-zero reference lines is four, while the size of the MPM list for zero reference lines is six.

[0065] In another embodiment, the size of the mode list for non-zero reference lines is three, and the size of the MPM list for zero reference lines is six.

[0066] In another embodiment, reporting of mode indices for non-zero reference lines is the same as reporting of MPM indexes for zero reference lines, except that five bins are used to report MPM indexes for zero reference lines, whereas up to three bins are used to report mode indices for non-zero reference lines.

[0067] In another embodiment, reporting of mode indices for non-zero reference lines is the same as reporting of MPM indexes for zero reference lines, except that five bins are used to report MPM indexes for zero reference lines, whereas up to two bins are used to report mode indices for non-zero reference lines.

[0068] In another embodiment, planar modes, DC modes, and modes with MPM index equal to 5 in the MPM list of the zero reference line are excluded for non-zero reference lines.

[0069] 8 is a flowchart of an example process 800 for signaling an intra-prediction mode used to encode a current block in a coded video bitstream. In some implementations, one or more process blocks of FIG. 8 may be performed by the decoder 510. In some implementations, one or more process blocks of FIG. 8 may be performed by another device, such as the encoder 503, or a device group separate from or including the decoder 510.

[0070] 8, the process 800 may include generating a first most probable mode (MPM) list corresponding to a zero reference line of the current block (block 810). The first MPM list may include a first plurality of intra-prediction modes.

[0071] 8, process 800 may include generating a second MPM list corresponding to one or more non-zero reference lines of the current block (block 820). The second MPM list may include a second plurality of intra-prediction modes, which may be a subset of the first plurality of intra-prediction modes.

[0072] As further shown in FIG. 8, process 800 may include notifying a reference line index indicating a reference line to be used to encode the current block from among the zero reference line and one or more non-zero reference lines (block 830).

[0073] 8, process 800 may include determining whether the reference line index indicates that the reference line is a zero reference line (block 840). Based on the reference line index indicating that the reference line is a zero reference line, process 800 may then signal an intra-mode index indicating an intra-prediction mode from within a first MPM list (block 850). Based on the reference line index indicating that the reference line is one of one or more non-zero reference lines, process 800 may then signal an intra-mode index indicating an intra-prediction mode from within a second MPM list (block 850).

[0074] In one embodiment, the zero reference line may include the closest reference line among multiple reference lines of the current block.

[0075] In one embodiment, planar intra prediction modes may be excluded from the second MPM list.

[0076] In one embodiment, the DC intra prediction mode may be excluded from the second MPM list.

[0077] In one embodiment, the planar and DC intra prediction modes may be excluded from the second MPM list.

[0078] In one embodiment, the second MPM list may be smaller than the first MPM list.

[0079] In one embodiment, the number of the first plurality of intra-prediction modes may be six, and the number of the second plurality of intra-prediction modes may be four.

[0080] In one embodiment, the intra-mode index is signaled using a first number of bins based on the reference line index indicating that the reference line is a zero reference line, and the intra-mode index is signaled using a second number of bins that is less than the first number of bins based on the reference line index indicating that the reference line is one of one or more non-zero reference lines.

[0081] In one embodiment, the first number of bins may be five and the second number of bins may be three.

[0082] In one embodiment, the reference line index may be signaled before the intra-mode index.

[0083] 8 illustrates example blocks of process 800, in some implementations, process 800 may include additional blocks, fewer blocks, different blocks, or blocks in a different arrangement than that shown in FIG 8. Additionally, or alternatively, two or more of the blocks of process 800 may be performed in parallel.

[0084] Additionally, the proposed methods may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transitory computer-readable medium to perform one or more of the proposed methods.

[0085] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 9 illustrates a computer system 1200 suitable for implementing certain embodiments of the disclosed subject matter.

[0086] Computer software can be coded using any suitable machine code or computer language that can be subjected to mechanisms such as assembly, compilation, linking, etc. to create code containing instructions that can be executed by a computer central processing unit (CPU), graphics processing unit (GPU), etc. directly, or through interpretation or microcode execution.

[0087] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0088] 9 for computer system 1200 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of computer system 1200.

[0089] Computer system 1200 may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, such as, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), olfactory input (not shown), etc. Human interface devices may also be used to capture certain media that do not necessarily involve direct conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and moving images (e.g., two-dimensional moving images, three-dimensional moving images including stereoscopic moving images).

[0090] The input human interface devices may include one or more of a keyboard 901, a mouse 902, a trackpad 903, a touchscreen 910, a data glove 1204, a joystick 905, a microphone 906, a scanner 907, and a camera 908 (only one of each is shown).

[0091] The computer system 1200 may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen 910, data gloves 1204, or joystick 905, although some haptic feedback devices do not function as input devices), audio output devices (e.g., speakers 909, headphones (not shown)), visual output devices (e.g., cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light emitting diode (OLED) screens, each with or without touchscreen input capability and each with or without haptic feedback capability, some of which may output two-dimensional visual output or three- or more-dimensional output by means of stereographic output or the like, virtual reality glasses (not shown), screens 910 including holographic displays and smoke tanks (not shown)), and printers (not shown).

[0092] The computer system 1200 may also include human-accessible storage devices and their associated media, such as a CD / DVD ROM / RW 920 with optical media 921 such as CDs / DVDs, thumb drives 922, removable hard drives or solid state drives 923, legacy magnetic media such as tape or floppy disks (not shown), specialized ROM / ASIC / PLD-based devices (not shown) such as security dongles, etc.

[0093] Those skilled in the art will also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not encompass transmission media, carrier waves, or other transitory signals.

[0094] The computer system 1200 may also include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, or optical. Furthermore, the networks may be local, wide-area, metropolitan, vehicular, industrial, real-time, delay-tolerant networks, etc. Examples of networks include local area networks such as Ethernet and wireless LAN; cellular networks including Global System for Mobile Communications (GSM), third generation (3G), fourth generation (4G), fifth generation (5G), Long-Term Evolution (LTE), etc.; TV wired or wireless wide-area digital networks including cable, satellite, and terrestrial broadcast television; vehicular and industrial networks including CANbus; etc. Particular networks typically require an external network interface adapter that connects to a particular general-purpose data port or peripheral bus (949) (e.g., a universal serial bus (USB) port on computer system 1200, etc.), or are otherwise typically integrated within the core of computer system 1200 by attaching to a system bus described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using any of these networks, computer system 1200 can communicate with other entities. Such communications may be unidirectional receive only (e.g., broadcast television), unidirectional transmit only (e.g., a CANbus to a particular CANbus device), or bidirectional, e.g., to other computer systems using local-area or wide-area digital networks. As noted above, each of these networks and network interfaces may use particular protocols and protocol stacks.

[0095] The aforementioned human interface devices, human accessible storage, and network interfaces may be attached to the core 940 of the computer system 1200 .

[0096] The core 940 may include one or more central processing units (CPUs) 941, graphics processing units (GPUs) 942, specialized programmable processing units in the form of field programmable gate areas (FPGAs) 943, task-specific hardware accelerators 944, etc. These devices, along with read-only memory (ROM) 945, random access memory (RAM) 946, and internal mass storage devices 947 such as internal, non-user-accessible hard drives, solid-state drives (SSDs), etc., are connected via a system bus 1248. In some computer systems, the system bus 1248 may be accessible in the form of one or more physical plugs, allowing expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus 1248 or via a peripheral bus 949. Peripheral bus architectures include Peripheral Component Interconnect (PCI), USB, etc.

[0097] The CPU 941, GPU 942, FPGA 943, and accelerator 944 may combine to execute specific instructions that may constitute the aforementioned computer code. That computer code may be stored in ROM 945 or RAM 946. Temporary data may also be stored in RAM 946, while persistent data may be stored, for example, in internal mass storage device 947. Fast storage and retrieval from any memory device is enabled through the use of cache memory, which may be closely associated with one or more of the CPU 941, GPU 942, mass storage device 947, ROM 945, RAM 946, etc.

[0098] The computer-readable medium may bear computer code for performing various computer-implemented operations, and the medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0099] By way of example and not limitation, computer system 1200 having the architecture, and in particular core 940, may provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be specific storage of core 940 of a non-transitory nature, such as core-internal mass storage 947 or ROM 945, as well as media associated with user-accessible mass storage devices as introduced above. Software implementing various embodiments of the present disclosure may be stored in such devices and executed by core 940. Computer-readable media may include one or more memory devices or chips according to particular needs. The software may cause core 940, and in particular the processors therein (including a CPU, GPU, FPGA, etc.), to perform specific processes or portions of specific processes described herein, including defining data structures stored in RAM 946 and modifying such data structures, according to software-defined processes. Additionally or alternatively, a computer system may provide functionality as a result of logic implemented in hardware or otherwise embedded in circuitry (e.g., accelerator 944) that can operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software may include logic, where appropriate, and vice versa. References to computer-readable media may include, where appropriate, circuitry (such as an integrated circuit (IC)) that stores software to be executed, circuitry in which logic to be executed is embedded, or both. The present disclosure includes any appropriate combination of hardware and software.

[0100] While this disclosure has described several exemplary embodiments, there are alterations, alterations, permutations, and various equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art can devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope. [Explanation of symbols]

[0101] 300 Communication Systems 410 terminal 420 terminal 430 terminals 440 terminals 450 Network 501 Video source, camera 502 Uncompressed Video Sample Stream 503 Video encoder, video source, 504 Video Bitstream 505 Streaming Server 506 Streaming Client 507 Copy, Video Bitstream 508 Streaming Client 509 Copy, Video Bitstream 510 Video Decoder 511 Video Sample Stream 512 displays, rendering devices 513 Capture Subsystem 610 Receiver 612 channels 615 Buffer memory, buffer 620 Parser 621 Symbol 651 Scaler / Descaler Unit 652 Intra prediction unit, intra picture prediction unit 653 Motion Compensation Prediction Unit 655 Aggregator 656 Loop filter, Loop filter unit, Reference picture memory, Reference picture 657 Reference Picture Memory, Reference Picture Buffer 730 Source Coder, Video Coder 732 encoding engine 733 Local Decoder, Local Video Decoder 734 Reference Picture Memory, Reference Picture Cache 735 Predictor 740 Transmitter 743 coded video sequence 745 Entropy Coder 750 Controller 760 communication channels 800 processes 901 Keyboard 902 Mouse 903 Trackpad 905 Joystick 906 Mike 907 Scanner 908 Camera 909 Speaker 910 Touchscreen 921 Optical media 922 thumb drive 923 Solid State Drive 940 cores 941 Central Processing Unit (CPU) 942 Graphics Processing Unit (GPU) 943 Field Programmable Gate Area (FPGA) 944 Hardware Accelerator 945 Read-Only Memory (ROM) 946 Random Access Memory (RAM) 947 Mass storage device 949 Peripheral Bus 1200 Computer System 1204 Data Gloves 1248 system bus

Claims

[Claim 1] 1. A method executed by at least one processor for signaling an intra-prediction mode used to code a current block in a coded video bitstream, comprising: the at least one processor: generating a first most probable mode list (MPM list) corresponding to a zero reference line of the current block, the first MPM list including a first plurality of intra-prediction modes; generating a second MPM list corresponding to one or more non-zero reference lines of the current block, the second MPM list including a second plurality of intra prediction modes, the second plurality of intra prediction modes including a subset of the first plurality of intra prediction modes; signaling a reference line index indicating a reference line to be used for encoding the current block among the zero reference line and the one or more non-zero reference lines; signaling an intra mode index indicating the intra prediction mode; The intra mode index indicates the intra prediction mode in a first MPM list based on the reference line index indicating that the reference line is the zero reference line, or A method in which the intra mode index indicates the intra prediction mode in the second MPM list based on the reference line index indicating that the reference line is one of the one or more non-zero reference lines.

Images