Video decoding method, apparatus, and computer program

Abstract

To provide a video decoding method performed by at least one processor to control multi-line intra prediction using non-zero reference lines.SOLUTION: A method includes the steps of determining whether the intra prediction mode of a first neighboring block of the current block is angular mode, determining whether the intra prediction mode of a second neighboring block of the current block is angular mode, and generating a most probable mode list including six candidate modes for intra prediction of the current block. The six candidate modes are all angular modes.SELECTED DRAWING: None

Description

Technical Field

[0001] This application claims the benefit of U.S. Provisional Application No. 62 / 731,776, filed on Sep. 14, 2018, and U.S. Application No. 16 / 234,993, filed on Dec. 28, 2018, the disclosures of which are incorporated herein by reference in their entireties.

[0002] This disclosure relates to a set of advanced video coding techniques. More particularly, it relates to an MPM list generation scheme for multi-line intra prediction.

Background Art

[0003] FIG. 5 shows the intra prediction modes used in High Efficiency Video Coding (HEVC). There are a total of 35 intra prediction modes in HEVC. Mode 10 is the horizontal mode (501), mode 26 is the vertical mode (502), and modes 2, 18, and 34 are the diagonal modes (503). The intra prediction modes are signaled by three most probable modes (MPMs) and the remaining 32 modes.

Summary of the Invention

Problems to be Solved by the Invention

[0004] To encode the intra mode, a most probable mode (MPM) list of size 3 is constructed based on the intra modes of adjacent blocks. This MPM list is referred to as the MPM list or the primary MPM list. If the intra mode is not from the MPM list, a flag is signaled to indicate whether the intra mode belongs to the selected mode.

[0005] The process of generating the MPM list is shown below. Here, leftIntraDir indicates the mode in the left block, and aboveIntraDir indicates the mode in the upper block. If the left block and the upper block are not currently available, set leftIntraDir or aboveIntraDir to the index DC_IDX. Also, the variables "offset" and "mod" are constant values, set to 29 and 32 respectively. ·If(leftIntraDir==aboveIntraDir&&leftIntraDir>DC_IDX) ○MPM[0]=leftIntraDir; ○MPM[1]=((leftIntraDir+offset)%mod)+2; ○MPM[2]=((leftIntraDir-1)%mod)+2; ·Else if(leftIntraDir==aboveIntraDir) ○MPM[0]=PLANAR_IDX; ○MPM[1]=DC_IDX; ○MPM[2]=VER_IDX; ·Else if(leftIntraDir!=aboveIntraDir) ○MPM[0]=leftIntraDir; ○MPM[1]=aboveIntraDir; ○If(leftIntraDir>0&&aboveIntraDir>0) ■MPM[2]=PLANAR_IDX; ○Else ■MPM[2]=(leftIntraDir+aboveIntraDir)<2?VER_IDX:DC_IDX;

Means for Solving the Problem

[0006] According to at least one embodiment, a video decoding method executed by at least one processor for controlling multi-line intra prediction using a non-zero reference line, the method comprising determining whether an intra prediction mode of a first adjacent block of a current block is an angular mode. The method may further comprise determining whether an intra prediction mode of a second adjacent block of the current block is an angular mode. The method may further comprise generating an MPM list, the MPM list including six candidate modes for intra prediction of the current block, all of the six candidate modes being angular modes. When it is determined that the intra prediction mode of the first adjacent block is an angular mode, the MPM list may be generated to include the intra prediction mode of the first adjacent block, and when it is determined that the intra prediction mode of the second adjacent block is an angular mode, the MPM list may be generated to include the intra prediction mode of the second adjacent block.

[0007] According to at least one embodiment, there is provided an apparatus for decoding a video sequence by controlling multi-line intra prediction using a non-zero reference line. The apparatus may include at least one memory arranged to store computer program code, and at least one processor arranged to access the at least one memory and operate based on the computer program code. The computer program code may include first determination code arranged to cause the at least one processor to determine whether an intra prediction mode of a first adjacent block of a current block is an angular mode. The computer program code may further include second determination code arranged to cause the at least one processor to determine whether an intra prediction mode of a second adjacent block of the current block is an angular mode. The computer program code may further include generation code arranged to cause the at least one processor to generate an MPM list, where the MPM list includes six candidate modes for intra prediction of the current block, and all of the six candidate modes are angular modes. The generation code is further arranged to cause the at least one processor to generate the MPM list such that when it is determined that the intra prediction mode of the first adjacent block is an angular mode, the intra prediction mode of the first adjacent block is included in the MPM list, and when it is determined that the intra prediction mode of the second adjacent block is an angular mode, the intra prediction mode of the second adjacent block is included in the MPM list.

[0008] According to at least one embodiment, a non-transitory computer-readable storage medium storing instructions that may cause at least one processor to determine whether an intra prediction mode of a first adjacent block of a current block is an angular mode. The instructions may further cause the at least one processor to determine whether an intra prediction mode of a second adjacent block of the current block is an angular mode. The instructions may further cause the at least one processor to generate a MPM list, the MPM list including six candidate modes for intra prediction of the current block, all of the six candidate modes being angular modes. The instructions may cause the at least one processor to generate the MPM list such that when it is determined that the intra prediction mode of the first adjacent block is an angular mode, the intra prediction mode of the first adjacent block is included in the MPM list, and when it is determined that the intra prediction mode of the second adjacent block is an angular mode, the intra prediction mode of the second adjacent block is included in the MPM list.

Brief Description of the Drawings

[0009] Other features, characteristics, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. In the drawings,

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

[0010] FIG. 1 shows a simplified block diagram of a communication system (100) according to an embodiment of the present disclosure. The system (100) may include at least two terminals (110-120) connected to each other via a network (150). In the case of unidirectional data transmission, the first terminal (110) can encode video data at a local location and transmit it to another terminal (120) via the network (150). The second terminal (120) can receive the encoded video data of another terminal from the network (150), decode the encoded data, and display the restored video data. Unidirectional data transmission is common in media service applications and the like.

[0011] FIG. 1 shows, for example, a second pair of terminals (130, 140) provided to support the bi-directional transmission of encoded video that may occur during a video conference. In the case of bi-directional data transmission, each terminal (130, 140) can encode video data captured at a local location and transmit it to another terminal via the network (150). Each terminal (130, 140) can further receive the encoded video data transmitted by the other terminal, decode the encoded data, and display the restored video data on a local display device.

[0012] In FIG. 1, the terminals (110 to 140) may be, for example, a server, a personal computer, a smartphone, and / or any other type of terminal. For example, the terminals (110 to 140) may be a laptop computer, a tablet, a media player, and / or dedicated video conferencing equipment. The network (150) represents any number of networks for transmitting encoded video data among the terminals (110 to 140), including, for example, wired and / or wireless communication networks. The communication network (150) can exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of the examination of this application, the architecture and topology of the network (150) are not important for the operation of the present disclosure, unless otherwise described herein below.

[0013] As an application example of the disclosed subject matter, FIG. 2 shows the arrangement of video encoders and decoders in a streaming environment. The disclosed subject matter can be utilized with other applications that support video, including, for example, video conferencing, digital TV, storage of compressed video on digital media such as CDs, DVDs, memory sticks, etc.

[0014] As shown in FIG. 2, the streaming system (200) can include a capture subsystem (213), which includes a video source (201) and an encoder (203). The streaming system (200) may further include at least one streaming server (205) and / or at least one streaming client (206).

[0015] The video source (201) can create, for example, an uncompressed video sample stream (202). The sample stream (202) is drawn as a thick line to emphasize that it has a large amount of data compared to the encoded video bitstream, and the sample stream (202) can be processed by an encoder (203) connected to a camera (201). The encoder (203) can include hardware, software, or a combination thereof to implement or carry out each aspect of the disclosed subject matter described in more detail below. The encoder (203) may generate an encoded video bitstream (204). The encoded video bitstream (204) is drawn as a thin line to emphasize that it has a small amount of data compared to the uncompressed video sample stream (202) and may be stored in a streaming server (205) for future use. One or more streaming clients (206) can access the streaming server (205) to retrieve a video bitstream (209) as a copy of the encoded video bitstream (204).

[0016] The streaming client (206) can include a video decoder (210) and a display (212). The video decoder (210) can decode, for example, a video bitstream (209) as an incoming copy of the encoded video bitstream (204) and create an outgoing video sample stream (211) that can be rendered on the display (212) or another rendering device (not shown). In some streaming systems, the video bitstreams (204, 209) can be encoded according to a particular video encoding / compression standard. Examples of these standards include, but are not limited to, the ITU-T H.265 recommendation. A video encoding standard, informally called Versatile Video Coding (VVC), is under development. Embodiments of the present disclosure can be used in the context of VVC.

[0017] FIG. 3 shows an exemplary functional block diagram of a video decoder (210) connected to a display (212) according to an embodiment of the present disclosure.

[0018] The video decoder (210) may include a channel (312), a receiver (310), a buffer memory (315), an entropy decoder / parser (320), a scaler / inverse transform unit (351), an intra prediction unit (352), a motion compensation prediction unit (353), an aggregator (355), a loop filter unit (356), a reference picture memory (357), and a current picture memory (358). In at least one embodiment, the video decoder (210) may include an integrated circuit, a series of integrated circuits, and / or other electronic circuits. The video decoder (210) may further be implemented, in part or in whole, by software executed on one or more CPUs with associated memory.

[0019] In this and other embodiments, a receiver (310) can receive one or more encoded video sequences to be decoded by a decoder (210), receiving one encoded video sequence at a time, and the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. It can receive an encoded video sequence from a channel (312), which is a hardware / software link to a storage device for storing the encoded video data. The receiver (310) can receive the encoded video data and other data, e.g., encoded audio data and / or auxiliary data streams that can be transferred to respective usage entities (not shown). The receiver (310) can separate the encoded video sequences from other data. To handle network jitter, a buffer memory (315) can be coupled between the receiver (310) and an entropy decoder / parser (320) (hereinafter referred to as "parser"). When the receiver (310) receives data from a storage / transfer device having sufficient bandwidth and controllability, or an isochronous real-time network, the buffer (315) may not be used or may be small. To utilize a packet network such as the Internet as much as possible, a buffer (315) may be required, and the buffer (315) may be relatively large and have an adaptive size.

[0020] The video decoder (210) may include a parser (320) for reconstructing symbols (321) based on an entropy - encoded video sequence. The categories of these symbols include, for example, information for managing the operation of the video decoder (210), and information for controlling a rendering device such as a display (212) that may be coupled to the decoder as shown in FIG. 2. The control information for the rendering device may be in the form of Supplementary Enhancement Information (SEI message) or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (320) analyzes / entropy - decodes the received encoded video sequence. The encoding of the encoded video sequence can comply with a video encoding technology or standard and can follow principles well - known to those skilled in the art, including variable - length coding, Huffman coding, arithmetic coding with or without context - dependence, etc. The parser (320) extracts a subgroup parameter set for at least one of the pixel subgroups in the video decoder based on at least one parameter corresponding to a group. The subgroups may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), etc. The parser (320) can further extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence. Unit, TU), a Prediction Unit (PU), etc.

[0021] The parser (320) can create symbols (321) by performing an entropy - decoding / analysis operation on the video sequence received from the buffer (315).

[0022] The reconstruction of symbol (321) can involve multiple different units depending on the type of the encoded video picture or a part thereof (e.g., inter-picture and intra-picture, inter-block and intra-block), and other factors. The units involved and the form of involvement can be controlled by subgroup control information parsed by parser (320) from the encoded video sequence. For the sake of brevity, the flow of such subgroup control information between parser (320) and the following multiple units is not illustrated.

[0023] In addition to the function blocks mentioned, decoder (210) can conceptually be subdivided into several functional units described below. In an actual implementation operating under commercial constraints, multiple of these units can interact closely with each other and can be at least partially integrated with each other. However, for the purpose of explaining the disclosed subject matter, it is appropriate to conceptually subdivide into the following functional units.

[0024] One unit may be a scaler / inverse transform unit (351). The scaler / inverse transform unit (351) can receive quantization transform coefficients and control information from parser (320) as symbol (321), including the transform method, block size, quantization factor, quantization scaling matrix, etc. used. The scaler / inverse transform unit (351) can output a block including sample values that can be input to aggregator (355).

[0025] In some cases, the output samples of the scaler / inverse transform unit (351) may belong to intra-coded blocks, i.e., blocks that do not utilize prediction information from previously reconstructed pictures, but can utilize prediction information from previously reconstructed portions of the current picture. Such prediction information can be provided by the intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) utilizes information from the current (partially reconstructed) picture in the current picture memory (358) where the surroundings have already been reconstructed to generate a block of the same size and shape as the block being reconstructed. In some cases, for each sample, the aggregator 355 adds the prediction information generated by the intra prediction unit 352 to the output sample information provided by the scaler / inverse transform unit 351.

[0026] In other cases, the output samples of the scaler / inverse transform unit (351) may belong to blocks that have already been inter-coded and possibly motion-compensated. In such cases, the motion-compensation prediction unit (353) accesses the reference picture memory (357) to obtain the samples used for prediction. Based on the symbols 321 belonging to the block, after motion-compensating the obtained samples, these samples can be added by the aggregator 355 to the output of the scaler / inverse transform unit 351 (in this case, called residual samples or a residual signal) to generate output sample information. The address in the reference picture memory (357) from which the motion-compensation prediction unit (353) obtains the prediction samples can be controlled by the motion vector. The motion vector may be available to the motion-compensation prediction unit (353) in the form of symbols (321), and the symbols (321) may have, for example, x, Y, and reference picture components. Motion compensation may further include interpolation of the sample values obtained from the reference picture memory (357), a motion vector prediction mechanism, etc., when an accurate sub-sample motion vector is used.

[0027] The output sample of the aggregator (355) may undergo various loop filtering techniques in the loop filter unit (356). The video compression technology includes in-loop filter technology, and the in-loop filter technology is included in the encoded video bitstream and is controlled by parameters applied to the loop filter unit (356) as symbols (321) from the parser (320). However, the video compression technology may respond to meta information obtained during the period of decoding the previous part (in the decoding order) of the encoded picture or the encoded video sequence, or may respond to previously constructed loop filter processed sample values.

[0028] The output of the loop filter unit (356) may be a sample stream, and the sample stream can be output to a rendering device such as a display (212) and can be stored in the reference picture memory (356) for use in future inter-picture prediction.

[0029] When some encoded pictures are completely reconstructed, they can be used as reference pictures for future prediction. When an encoded picture is completely reconstructed and the encoded picture is recognized as a reference picture (e.g., by the parser (320)), the current reference picture stored in the current picture memory (358) can become part of the reference picture memory (357), and a new current picture memory can be reallocated before starting the reconstruction of subsequent encoded pictures.

[0030] The video decoder (210) can perform a decoding operation according to a predetermined video compression technique that can be recorded in, for example, the ITU-T H.265 standard. In the sense that the encoded video sequence conforms to the syntax of the video compression technique or standard, the encoded video sequence can conform to the syntax specified by the video compression technique or standard being used, such as being specified in a video compression technique document or standard, more specifically a profile document thereof. Also, in order to conform to some video compression techniques or standards, the complexity of the encoded video sequence is within a range limited by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstructed sample rate (measured, for example, in mega samples per second), maximum reference picture size, etc. In some cases, the limits set by the level may be further limited by the specifications of the Hypothetical Reference Decoder (HRD) and the metadata of HRD buffer management signaled in the encoded video sequence.

[0031] In an embodiment, the receiver (310) can receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The additional data can be utilized by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or SNR extension layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0032] FIG. 4 shows an exemplary functional block diagram of a video encoder (203) associated with a video source (201) according to an embodiment of the present disclosure.

[0033] The video coder (203) may include, for example, a coder as a source coder (430), an encoding engine (432), a (local) decoder (433), a reference picture memory (434), a predictor (435), a transmitter (440), an entropy coder (445), a controller (450), and a channel (460).

[0034] The coder (203) may receive video samples from a video source (201) (which is not part of the coder), and the video source may capture a video image to be encoded by the coder (203).

[0035] The video source (201) can provide a source video sequence to be encoded by the coder (203) in the form of a digital video sample stream, and the digital video sample stream may have any suitable bit depth (e.g., x bits, 10 bits, 12 bits, ···), any color space (e.g., BT.601 Y CrCB, RGB, ···), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (201) may be a storage device for storing previously prepared videos. In a video conferencing system, the video source (203) may be a photographing device for capturing local image information as a video sequence. The video data is provided as a plurality of individual pictures that give movement when viewed in order. Each picture itself may be organized as a spatial pixel array, and each pixel can include one or more samples depending on the sampling structure, color space, etc. used. The relationship between pixels and samples is easily understandable to those skilled in the art. Hereinafter, the description will focus on samples.

[0036] According to an embodiment, the encoder (203) can encode pictures of a source video sequence and compress them into an encoded video sequence (443) under real-time or any other time constraints required by the application. Executing at an appropriate encoding speed is one function of the controller (450). The controller (450) may further control other functional units described below and be functionally coupled to these units. For simplicity, the couplings are not illustrated. The parameters set by the controller (450) may include rate control related parameters (such as picture skip, quantizer, λ value of rate distortion optimization techniques), picture size, picture group (GOP) layout, maximum motion vector search range, etc. Other functions of the controller (450) can be easily recognized by those skilled in the art, and these functions may belong to a video encoder (203) optimized for a specific system design.

[0037] Some video encoders operate in a manner that those skilled in the art would readily recognize as an "encoding loop." As a very simplified explanation, the encoding loop may include an encoding portion of the source encoder (430) (responsible for constructing symbols based on the input picture and reference pictures to be encoded), and a (local) decoder (433) embedded in the encoder (203). In certain video compression techniques, when the compression between the symbols and the encoded video bitstream is reversible, the decoder (433) reconstructs the symbols to create the sample data that the (remote) decoder would also attempt to create. The reconstructed sample stream may be input into the reference picture memory (434). Decoding of the symbol stream results in accurate bits regardless of the decoder location (local or remote), so the contents of the reference picture memory are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" from the prediction portion of the encoder are exactly the same as the sample values "seen" when the decoder attempts to use prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs, for example, due to channel errors when synchronization cannot be maintained) is well-known to those skilled in the art.

[0038] The operation of the "local" decoder (433) may be basically the same as the operation of the "remote" decoder (210) described in detail above in relation to FIG. 3. However, since the symbols are available and the entropy encoder (445) and parser (320) can encode / decode the symbols losslessly into the encoded video sequence, in the local decoder (433), the entropy decoding portion of the decoder (210) including the channel (312), receiver (310), buffer (315), and parser (320) need not be fully implemented.

[0039] In this case, it is observed that any decoder technology other than parsing / entropy decoding existing in the decoder will necessarily exist in the corresponding encoder in basically the same functional form. Therefore, the disclosed subject matter focuses on the operation of the decoder. Since the encoder technology is the reverse of the fully described decoder technology, the description of the encoder technology can be simplified. A more detailed description is necessary only in a specific area and is provided below.

[0040] The source encoder (430) can perform motion-compensated predictive encoding as part of its operation, referring to one or more encoded frames designated as "reference frames" from the video sequence and performing predictive encoding on the input frame. In this way, the encoding engine (432) encodes the difference between the pixel block of the input frame and the pixel block of the reference frame that can be selected as the prediction reference for the input frame.

[0041] The local video decoder (433) can decode the encoded video data of the frame designated as the reference frame based on the symbols created by the source encoder (430). The operation of the encoding engine (432) may advantageously be an irreversible process. When the encoded video data is decoded by a video decoder (not shown in FIG. 4), the reconstructed video sequence may generally be a replica of the source video sequence with some errors. The local video decoder (433) can copy the decoding process that the video decoder can perform on the reference frame and store the reconstructed reference frame in the reference picture memory (434). In this way, the encoder (203) locally stores copies of the reconstructed reference frames, and these copies have the same content as the reconstructed reference frames obtained by the remote video decoder (in the absence of transmission errors).

[0042] The predictor (435) can perform a prediction search for the encoding engine (432). That is, for a new frame to be encoded, the predictor (435) can search in the reference picture memory (434) for sample data that can be used as an appropriate prediction reference for the new picture (as a candidate reference pixel block), or for specific metadata such as reference picture motion vectors, block shapes, etc. The predictor (435) can find an appropriate prediction reference by operating on a per-pixel block basis based on the sample block. In some cases, the input picture may have a prediction reference extracted from a plurality of reference pictures stored in the reference picture memory (434) as determined based on the search results obtained by the predictor (435).

[0043] The controller (450) can manage the encoding operation of the video encoder (430), including setting parameters and subgroup parameters for encoding video data, for example.

[0044] In the entropy encoder (445), entropy encoding may be performed on the outputs of all the above functional units. The entropy encoder performs lossless compression on the symbols generated by the various functional units according to techniques known to those skilled in the art, such as Huffman coding, variable-length coding, arithmetic coding, etc., to convert these symbols into an encoded video sequence.

[0045] The transmitter (440) can buffer the encoded video sequence created by the entropy encoder (445) to prepare it for transmission via the communication channel (460), which may be a hardware / software link to a storage device for storing the encoded video data. The transmitter (440) can merge the encoded video data from the video encoder (430) with other data to be transmitted, such as encoded audio data and / or an auxiliary data stream (source not shown).

[0046] The controller (450) can manage the operation of the coder (203). During encoding, the controller (450) assigns a specific encoded picture type to each encoded picture, which may affect the encoding technique that can be applied to each picture. For example, a picture is typically assigned as an intra picture (I picture), a predictive picture (P picture), or a bi-directional predictive picture (B picture).

[0047] An intra picture (I picture) is a picture that is encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow different types of intra pictures that include independent decoder refresh (IDR) pictures. Those skilled in the art are aware of these variations of I pictures and their corresponding uses and characteristics.

[0048] A predictive picture (P picture) may be a picture that is encoded and decoded using intra prediction or inter prediction that predicts the sample values of each block using at most one motion vector and a reference index.

[0049] A bi-directional predictive picture (B picture) may be a picture that is encoded and decoded using intra prediction or inter prediction that predicts the sample values of each block using at most two motion vectors and a reference index. Similarly, a multi-predictive picture can perform the reconstruction of a single block using more than two reference pictures and associated metadata.

[0050] The source picture can generally be subdivided into a plurality of spatially sample blocks (for example, blocks having 4×4, 8×8, 4×8, or 16×16 samples respectively) and encoded block by block. With reference to other (already encoded) blocks, predictive encoding is performed on these blocks, and the other blocks are determined by the encoding assignment applied to the corresponding pictures of the blocks. For example, non-predictive encoding may be performed on the blocks of the I picture, or predictive encoding (spatial prediction or intra prediction) may be performed on the blocks of the I picture with reference to the already encoded blocks of the same picture. The pixel blocks of the P picture perform non-predictive encoding via spatial prediction or temporal prediction with reference to one already encoded reference picture. The blocks of the B picture perform non-predictive encoding via spatial prediction or temporal prediction with reference to one or two already encoded reference pictures.

[0051] The video encoder (203) can execute an encoding operation based on, for example, a predetermined video encoding technique or standard of the ITU-T H.265 recommendation. In the operation of the video encoder (203), the video encoder (203) can execute various compression operations including a predictive encoding operation that utilizes temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video encoding technique or standard used.

[0052] In an embodiment, the transmitter (440) can transmit additional data together with the encoded video. The video encoder (430) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, supplementary enhancement information (SEI) messages, video user utility information (VUI) parameter set fragments, and the like.

[0053] FIG. 6 is a diagram of the intra prediction mode in VVC draft 2.

[0054] In VVC Draft 2, as shown in FIG. 6, there are a total of 87 intra prediction modes. Mode 18 (601) is the horizontal mode, mode 50 (602) is the vertical mode, and modes 2 (603), 34 (604), and 66 (605) are the diagonal modes. Modes -1 to -10 and modes 67 to 76 are called Wide-Angle Intra Prediction (WAIP) modes (606, 707).

[0055] In VVC Draft 2, the size of the MPM list is still 3, and the generation process of the MPM list is the same as that of HEVC. However, since there are 67 signaling modes in VVC Draft 2, there is a difference that "offset" is changed to 61 and "mod" is changed to 64.

[0056] The next section of VVC Draft 2 describes the luminance intra mode encoding process in which IntraPredModeY[xPb][yPb] is obtained. 1. The adjacent positions (xNbA, yNbA) and (xNbB, yNbB) are set equal to (xPb - 1, yPb) and (xPb, yPb - 1), respectively. 2. When X is replaced with A or B, the variable candIntraPredModeX is derived by the following steps. - Call the block availability derivation process specified in Section 6.4.X[Ed.(BB):Neighbouring blocks availability checking process tbd], where the position (xCurr, yCurr) is set equal to (xPb, yPb), and the adjacent position (xNbY, yNbY) is set equal to (xNbX, yNbX) as input, and the output is assigned to availableX. - Obtain the candidate intra prediction mode candIntraPredModeX according to the following steps. ​​- If one or more of the following conditions are true, set candIntraPredModeX to be equal to INTRA_DC. - The variable availableX is equal to FALSE. - CuPredMode[xNbX][yNbX] is not equal to MODE_INTRA. - X is equal to B, and yPb-1 is smaller than ((yPb >> CtbLog2SizeY) << CtbLog2SizeY) - Otherwise, set candIntraPredModeX to be equal to IntraPredModeY[xNbX][yNbX] - Otherwise, set candIntraPredModeX to be equal to IntraPredModeY[xNbX][yNbX] - Otherwise, set candIntraPredModeX to be equal to IntraPredModeY[xNbX][yNbX] 3. Obtain candModeList[x] according to the following steps, where x = 0...2. - If candIntraPredModeB is equal to candIntraPredModeA, the following processing is applied. - If candIntraPredModeA is less than 2 (i.e., equal to INTRA_PLANAR or INTRA_DC), obtain candModeList[x] according to the following steps, where x = 0..2. candModeList[0]=INTRA_PLANAR (8 - 1) candModeList[1]=INTRA_DC (8 - 2) candModeList[2]=INTRA_ANGULAR50(8 - 3) - Otherwise, obtain candModeList[x] according to the following steps, where x = 0..2. candModeList[0]=candIntraPredModeA (8 - 4) candModeList[1]=2+((candIntraPredModeA + 61)% 64)(8 - 5) candModeList[2]=2+((candIntraPredModeA - 1)% (8 - 6) - Otherwise (when candIntraPredModeB is not equal to candIntraPredModeA), the following process is applied. - Obtain candModeList[0] and candModeList[1] according to the following steps. candModeList[0] = candIntraPredModeA (8 - 7) candModeList[1] = candIntraPredModeB (8 - 8) - If neither candModeList[0] nor candModeList[1] is equal to INTRA_PLANAR, set candModeList[2] to be equal to INTRA_PLANAR. - Otherwise, if neither candModeList[0] nor candModeList[1] is equal to INTRA_DC, set candModeList[2] to be equal to INTRA_DC. - Otherwise, set candModeList[2] to be equal to INTRA_ANGULAR50. 4. Apply the following process to obtain IntraPredModeY[xPb][yPb]. - When intra_luma_mpm_flag[xPb][yPb] is equal to 1, set IntraPredModeY[xPb][yPb] to be equal to candModeList[intra_luma_mpm_idx[xPb][yPb]] - Otherwise, use the following steps to obtain IntraPredModeY[xPb][yPb]. 1. Correct the array candModeList[x] according to the following steps, where x = 0..2. i. If candModeList[0] is greater than candModeList[1] swap these two values as follows. ​(candModeList[0], candModeList[1]) = Swap(candModeList[0], candModeList[1]) (8 - 9) ii. If candModeList[0] is greater than candModeList[2] then swap these two values as follows. (candModeList[0], candModeList[2]) = Swap(candModeList[0], candModeList[2]) (8 - 10) iii. candModeList[1] If it is greater than candModeList[2], then swap these two values as follows. (candModeList[1], candModeList[2]) = Swap(candModeList[1], candModeList[2]) (8 - 11) 2. Obtain IntraPredModeY[xPb][yPb] according to the following procedure. i. Set IntraPredModeY[xPb][yPb] to be equal to intra_luma_mpm_remainder[xPb][yPb]. ii. If i is equal to 0 to 2, 0 and 2 are included, and IntraPredModeY[xPb][yPb] is greater than or equal to candModeList[i], then increase the value of IntraPredModeY[xPb][yPb] by 1.

[0057] In the above, there is a variable IntraPredModeY[x][y], where x = xPb..xPb + cbWidth - 1, and y = yPb..yPb + cbHeight - 1, and it is arranged to be equal to IntraPredModeY[xPb][yPb]

[0058] ​In the development of VVC Draft 2, an MPM list of size 6 is proposed. The MPM list includes the Planar mode and the DC mode. Two adjacent modes, the left mode and the top mode, are used to generate the other four MPMs.

[0059] Multi-line intra prediction is proposed for performing intra prediction using more reference lines. Among them, which reference lines are used to generate the intra predictor is determined and signaled by the coder. The reference line index is signaled before the intra prediction mode. Also, when a non-zero reference line index is signaled, the Planar mode / DC mode is excluded from the intra prediction mode. Figure 7 shows an example of four reference lines (710), and each reference line (710) is composed of six segments (i.e., segments A to F) together with the top-left reference sample. Also, in segments A and F, the samples closest to segments B and E are embedded respectively.

[0060] In multi-line intra prediction, when the signaled reference line index is non-zero, the Planar mode and the DC mode are excluded from the generation of the MPM list and mode coding. Also, it is proposed to generate an MPM list of size 6 by two adjacent modes. Therefore, for the method of generating six angular MPMs for non-zero lines when only two adjacent modes can be accessed, it is an open question.

[0061] The proposed methods may be used separately or in combination in any order.

[0062] In the following description, the line index of the closest reference line is 0 (zero reference line). The maximum reference line number to be signaled is shown as N.

[0063] Similarly, as shown in FIG. 8, the upper (top) block (701) and the left block (702) are defined as follows.

[0064] As shown in FIG. 8, the pixel at the upper left position within the current block (703) is denoted as (x, y). A block in which the y - coordinates of all included samples are y or more and the x - coordinates of all included samples are less than x is called the left block. A block in which the y - coordinates of all included samples are less than y is called the upper block. FIG. 8 shows an example of the left blocks (L1, Lx, and Ln) and the upper blocks (A1, A2, Ax, An) of the current block (703).

[0065] The two adjacent modes mentioned below can be from the upper side (701) of the current block (703) or from the left side (702) of the current block (703). The following shows some examples of the two adjacent modes.

[0066] In one example, both of the two adjacent modes are from the left side (702).

[0067] In another example, both of the two adjacent modes are from the upper side (701).

[0068] In another example, one of the adjacent modes is from the left side (702) and the other adjacent mode is from the upper side (701).

[0069] In another example, when the width of the current block (703) is greater than the height of the current block (703), both of the two adjacent modes are from the upper side (701); or when the height of the current block (703) is greater than the width, both of the two adjacent modes are from the left side (702); or when the width of the current block (703) is equal to the height, one adjacent mode is from the upper side (701) and the other adjacent mode is from the left side (702).

[0070] Based on the block width, block height, and the ratio of the block width to the block height, by selecting and using the reference sample side (as explained in the above example), two (or more) adjacent blocks can be obtained.

[0071] In one example, when the width / height of the block is greater than a predetermined threshold, two (or more) adjacent blocks are selected only from the upper side (701). Examples of the threshold value include, but are not limited to, 2, 4, 8, 16, 32, and 64.

[0072] In another example, when the height / width of the block is greater than a predetermined threshold, two (or more) adjacent blocks are selected only from the left side (702). Examples of the threshold value include, but are not limited to, 2, 4, 8, 16, 32, and 64.

[0073] In the following description, when the adjacent block mode is not available, the mode is set to the Planar mode or the DC mode.

[0074] In the following description, when the range of the signaled mode number is in the range from 0 to M (including 0 and M), M may be any positive integer such as 34 or 66. The adjacent mode of a given mode X is limited such that when X is greater than 2 and less than M - 1, the adjacent modes of X are X - 1 and X + 1. When X is equal to 2, the adjacent modes of X are 3 and M (or M - 1). When X is equal to M - 1, the adjacent modes of X are X - 1 and X + 1 (or 2). When X is equal to M, the adjacent modes of X are M - 1 and 2 (or 3).

[0075] In the following description, when the mode is not the Planar mode or the DC mode, or when the mode generates prediction samples according to a predetermined prediction direction, such as the intra prediction modes 2 to 66 limited in VVC draft 2, the mode is called the angular mode. The two variables Offset and mod are in the following two groups. 1) Offset = mod - 3, mod = M - 2; 2) Offset = mod - 3, mod = M - 1;

[0076] If the signaled reference line index is non - zero, the following method can generate six angular MPMs through two adjacent modes. The following method or example may be used alone or in combination in any order.

[0077] In one embodiment, when at least one of the two adjacent modes is an angular mode, the following algorithm is used to generate six angular MPMs. The two adjacent modes are denoted as Mode_A and Mode_B. The variable ang_mode[] is used to record the angular modes of the adjacent modes. The variable ang_count is used to indicate the number of angular modes, and mpm_index is used to indicate the index of the MPM list. Initially, ang_count and mpm_index are set to 0. IncludedMode[] is used to indicate whether each mode is included in the MPM list, and all elements in the array IncludedMode[] are initially set to False. · If Mode_A is an angular mode, MPM [mpm_index]=Mode_A, ang_count += 1, mpm_index += 1; · If Mode_B is an angular mode, MPM [mpm_index]=Mode_B, ang_count += 1, mpm_index += 1; · When (diff = 0; diff <= 2 && mpm_index < 6; diff++) { · When (idx = 0; idx < ang_count; idx++) { ○ MPM[mpm_index]=((ang_mode[idx]+offset - diff)%mod)+2; ○ When (includedMode[MPM[mpm_index]]==false)) {, ■ includedMode[MPM[mpm_index++]] = true; ○ If mpm_index == 6, break; ○ MPM[mpm_index] = ((ang_mode[idx] - 1 + diff) % mod) + 2; ○ If (includedMode[MPM[mpm_index]] == false), { ■ includedMode[MPM[mpm_index++]] = true;}

[0078] In one embodiment, when only one of the adjacent modes is an angular mode shown as ang_neighbor, ang_neighbor and its two adjacent modes (shown as mode_L and mode_R) are added to the MPM list, and one adjacent mode of mode_L and one adjacent mode of mode_R are added to the MPM list. Finally, by adding the vertical / horizontal mode, six angular MPMs are generated. These six angular modes may be added to the MPM list in any order. In one example, six angular MPMs are generated as follows. · MPM[0] = ang_mode · MPM[1] = ((ang_mode + offset) % mod) + 2; · MPM[2] = ((ang_mode - 1) % mod) + 2; · MPM[3] = ((ang_mode - 1 + offset) % mod) + 2; · MPM[4] = ((ang_mode) % mod + 2; · MPM[5] is the vertical or horizontal mode. In one example, if the vertical mode is not included in MPM[0] to MPM[4], MPM[5] is set to the vertical mode. Otherwise, MPM[5] is set to the horizontal mode.

[0079] In other embodiments, when only one of the adjacent modes is the angular mode shown as ang_neighbor, ang_neighbor and its two adjacent modes (shown as mode_L and mode_R) are added to the MPM list, and one adjacent mode of mode_L (shown as mode_L_L) and one adjacent mode of mode_R (shown as mode_R_R) are added to the MPM list. Finally, one adjacent mode of mode_L_L or mode_R_R is added to the MPM list. These six angular modes may be added to the MPM list in any order. In one example, six angular MPMs are generated as follows. ·MPM[0]=ang_mode ·MPM[1]=((ang_mode+offset%mod)+2; ·MPM[2]=((ang_mode-1)%mod)+2; ·MPM[3]=((ang_mode-1+offset)%mod)+2; ·MPM[4]=((ang_mode)%mod+2; ·MPM[5]=((ang_mode+1)%mod)+2; In another example, six angular MPMs are generated as follows. ·MPM[0]=ang_mode ·MPM[1]=((ang_mode+offset)%mod)+2; ·MPM[2]=((ang_mode-1)%mod)+2; ·MPM[3]=((ang_mode-1+offset)%mod)+2; ·MPM[4]=((ang_mode)%mod)+2; ·MPM[5]=((ang_mode-2+offset)%mod)+2;

[0080] In other embodiments, when only one of the adjacent modes is an angular mode shown as ang_neighbor, six angular MPMs are obtained as follows, and these six angular modes are added to the MPM list in any order. The following is an example. ·MPM[0]=ang_mode ·MPM[1]=((ang_mode+offset)%mod)+2; ·MPM[2]=((ang_mode-1)%mod)+2; ·MPM[3]=((ang_mode-2+offset)%mod)+2; ·MPM[4]=((ang_mode+1)%mod)+2; ·MPM[5] is the vertical or horizontal mode. In one example, if the vertical mode is not included in MPM[0] to MPM[4], MPM[5] is set to the vertical mode. Otherwise, MPM[5] is set to the horizontal mode.

[0081] In other embodiments, when two adjacent modes among the adjacent modes are angular modes and they are adjacent modes, the MPM list is generated as follows. The two adjacent modes are shown as Mode_A and Mode_B and are added to the MPM list. The variables ang_max and ang_min are used to record the maximum mode and the minimum mode between Mode_A and Mode_B. ·When Mode_A is greater than Mode_B, ang_max is set to Mode_A and ang_min is set to Mode_B. ·When ang_min is equal to 2 and ang_max is equal to M-1 or M, swap the values of ang_min and ang_max. ·Obtain the other four angular MPMs as follows. These four modes may be added in any order. The following is an example. ·MPM[2]=((ang_min+offset)%mod)+2; ·MPM[3]=((ang_max-1)%mod)+2; ·MPM[4]=((ang_min-1+offset)%mod)+2; ·MPM[5] = ((ang_max) % mod) + 2;

[0082] In other embodiments, two adjacent modes are represented by Mode_A and Mode_B. Both of these two adjacent modes are angular modes, and when abs(Mode_A - Mode_B) > 2 && abs(Mode_A - Mode_B) <= Thres, the following algorithm is used to generate six angular MPMs. Thres is a positive integer, Thres is greater than 2, for example, Thres = 61 or 62 or 63. · First, add Mode_A and Mode_B to the MPM list · Then, add the two adjacent modes of Mode_A and the two adjacent modes of Mode_B to the MPM list as well. · Also, these six angular MPMs may be added to the MPM list in any order. · The following shows an example of generating six MPMs. ○MPM[0] = Mode_A ○MPM[1] = Mode_B ○MPM[2] = ((Mode_A + offset) % mod) + 2; ○MPM[3] = ((Mode_A - 1) % mod) + 2; ○MPM[4] = ((Mode_B + offset) % mod) + 2; ○MPM[5] = ((Mode_B - 1) % mod) + 2;

[0083] In other embodiments, two adjacent modes are represented by Mode_A and Mode_B. Both of these two adjacent modes are angular modes, and when abs(Mode_A - Mode_B) == 2 || abs(Mode_A - Mode_B) > Thres, where Thres is a positive integer and Thres is greater than 2, for example, Thres = 61 or 62 or 63, the following algorithm is used to obtain six angular MPMs. · Use the variables ang_max and ang_min to record the maximum mode and the minimum mode between Mode_A and Mode_B. ·When Mode_A is greater than Mode_B, ang_max is set to Mode_A and ang_min is set to Mode_B. ·Then, three adjacent modes of Mode_A and Mode_B are added to the MPM list. ·Finally, the vertical or horizontal mode is added to the MPM list, and these six MPMs may be added to the MPM list in any order. ·The following shows an example. ·MPM[0] = ang_min ·MPM[1] = ang_max ·MPM[2] = ((ang_min - 1) % mod) + 2; ·MPM[3] = ((ang_min + offset) % mod) + 2; ·MPM[4] = ((ang_max - 1) % mod) + 2; ·MPM[5] is the vertical or horizontal mode. In an example, if the vertical mode is not included in MPM[0] to MPM[4], MPM[5] is set to the vertical mode. Otherwise, MPM[5] is set to the horizontal mode.

[0084] In other embodiments, if both of the two adjacent modes are Planar mode or DC mode, six default modes are used to fill the MPM list. These six default modes may be added to the MPM list in any order. ·In one embodiment, the six default modes are {50, 18, 2, 34, 66, 26}. ·In other embodiments, the six default modes are {50, 18, 2, 34, 26, 42}. ·In other embodiments, when the width of the current block is greater than the height, the six default modes are {50, 18, 34, 66, 42, 58}. ·In other embodiments, when the height of the current block is greater than the width, the six default modes are {50, 18, 34, 2, 10, 26}. · In other embodiments, when the width of the current block is equal to the height, the six default modes are {50, 18, 2, 34, 26, 42}.

[0085] In other embodiments, first, add all adjacent modes that are in the angle mode to the MPM list. Then, for each of these adjacent modes that are in the angle mode, if the angle modes ((ang_mode - 1) % mod) + 2 and ((ang_mode + offset) % mod) + 2 are not included in the MPM list, where ang_mode is indicated, add them to the MPM list. · In one embodiment, if the MPM list is not yet fully filled, add some default modes. The default mode list is any one of the alternatives described by the above item number h. · In other embodiments, if the MPM list is not yet fully filled, for each mode that is already in the MPM list (indicated as mpm_mode), if the angle modes ((mpm_mode - 1) % mod) + 2 and ((mpm_mode + offset) % mod) + 2 are not included in the MPM list, add them to the MPM list.

[0086] In at least one embodiment, the above techniques can be implemented by an integrated circuit, a series of integrated circuits, and / or other electronic circuits. In at least one embodiment, the techniques can be partially or fully realized by software executed on one or more CPUs having associated memory.

[0087] Use computer-readable instructions to implement the above techniques as computer software and physically store it on one or more computer-readable media. For example, FIG. 9 shows a computer system (800) suitable for implementing some embodiments of the present disclosure.

[0088] Computer software can be coded using any suitable machine code or computer language, and for any suitable machine code or computer language, by applying mechanisms such as assembly, compilation, linking, etc., code can be created that contains instructions that can be directly executed by a computer central processing unit (CPU), a graphics processing unit (GPU), etc., or executed by interpretation, microcode, etc.

[0089] Instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0090] The components for the computer system (800) shown in FIG. 9 are essentially illustrative and are not intended to limit the scope or functionality of the use of computer software for implementing the embodiments of the present disclosure. The arrangement of the components should not be construed as having dependencies or requirements related to any of the components shown in the non-limiting embodiments of the computer system (800), or combinations thereof.

[0091] The computer system (800) may include several human-machine interface input devices. Such human-machine interface input devices can respond to inputs by one or more human users, such as, for example, tactile inputs (e.g., keystrokes, slides, data glove movements), audio inputs (e.g., voice, clapping), visual inputs (e.g., gestures), olfactory inputs (not shown), etc. The human-machine interface device can also be used to capture certain media that is not necessarily directly related to conscious human input, such as, for example, audio (e.g., voice, music, ambient sound), images (e.g., scanned images, photographic images obtained from a still image capture device), video (e.g., 2D video, 3D video including stereoscopic video).

[0092] The input human-machine interface device may have one or more of a keyboard (801), a mouse (802), a touch pad (803), a touch panel (810), a data glove, a joystick (805), a microphone (806), a scanner (807), and a photographing device (808) (only one of each is shown).

[0093] The computer system (800) may further include several human-machine interface output devices. Such human-machine interface output devices can stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such human-machine interface output devices may be tactile output devices (for example, there is a tactile feedback device provided by a touch panel (810), a data glove, or a joystick (805), and there is also a tactile feedback device that does not function as an input device). For example, such a device may be an audio output device (for example, a speaker (809), headphones (not shown)), a visual output device (for example, a screen (810), a CRT screen, an LCD screen, a plasma screen, an OLED screen, each of which may or may not have touch panel input capability and tactile feedback capability, and some of the screens output 2D vision or output an output of three or more dimensions by means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and a printer (not shown).

[0094] The computer system (800) may further include a memory device and its associated media accessible by a human, for example, an optical medium such as a CD / DVDROM / RW (820) having a medium (821) such as a CD / DVD, a thumb drive (822), a removable hard drive or solid state drive (823), conventional magnetic media such as magnetic tapes and floppy disks (not shown), devices such as dongles (not shown) based on dedicated ROM / ASIC / PLD, etc.

[0095] One of ordinary skill in the art can understand that the term "computer-readable medium" as used in connection with the subject matter disclosed in this application does not include a transmission medium, a carrier wave, or other transient signals.

[0096] The computer system (800) may further include an interface to one or more communication networks. The network may be, for example, a wireless, wired, or optical network. The network may be a local area network, wide area network, metropolitan network, vehicle industrial network, real-time network, delay-tolerant network, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks (including GSM, 3G, 4G, 5G, LTE, etc.), wired or wireless wide-area digital networks for television (including cable television, satellite television, and terrestrial television), vehicle and industrial networks (including CANBus), etc. Some networks generally require an external network interface adapter connected to a certain general-purpose data port or peripheral bus (849) (e.g., the USB port of the computer system (800)), and other interfaces are generally integrated into the core of the computer system (800) by being connected to a system bus as described later (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). The computer system (800) can communicate with other entities using any of these networks. Such communication can be one-way reception only (e.g., broadcast television), one-way transmission only (e.g., CAN bus to a certain CANbus device), or two-way, e.g., communication to other computer systems using a local area or wide-area digital network. Specific protocols and protocol stacks can be utilized for each of the above networks and network interfaces.

[0097] The human-machine interface device, the storage device accessible by humans, and the network interface may be connected to the core (840) of the computer system (800).

[0098] The core (840) may include one or more central processing units (CPUs) (841), a graphics processing unit (GPU) (842), a dedicated programmable processing unit in the form of a field-programmable gate array (FPGA) (843), a hardware accelerator (844) for a certain task, etc. These devices may be connected via a system bus (848) together with a read-only memory (ROM) (845), a random access memory (846), an internal mass storage device such as an internal hard disk drive or SSD that is not accessible to the user (847). In some computer systems, access to the system bus (848) can be realized in the form of one or more physical plugs to achieve expansion by additional CPUs, GPUs, etc. Peripheral devices may be connected directly or via a peripheral bus (849) to the core's system bus (848). The architecture of the peripheral bus includes PCI, USB, etc.

[0099] The CPU (841), GPU (842), FPGA (843) and accelerator (844) can execute some instructions, and by combining these instructions, the above computer code can be constituted. The computer code is stored in the ROM (845) or the RAM (846). Temporary data may be stored in the RAM (846), and permanent data may be stored, for example, in the internal mass storage device (847). By using cache memory, high-speed storage and retrieval to any storage device are made possible, and the cache memory can be closely associated with one or more CPUs (841), GPUs (842), mass storage devices (847), ROM (845), RAM (846), etc.

[0100] The computer-readable medium can have computer code for performing various operations implemented by a computer. The medium and the computer code may be a medium and computer code specially designed and constructed for the purposes of the present disclosure, or may be of a type well-known and usable to those skilled in the computer software art.

[0101] By way of example and not limitation, a computer system having an architecture (800), particularly a core (840), can provide functions by a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software incorporated in one or more tangible computer-readable media. Such computer-readable media may be media related to a mass storage device accessible by a user as introduced above, and may also be a specific storage device of the core (840) having a non-transitory nature, such as an internal mass storage device (847) or ROM (845) of the core. The software for implementing each embodiment of the present disclosure is stored in such a device and executed by the core (840). Depending on specific needs, the computer-readable media may include one or more storage devices or chips. The software includes defining a data configuration stored in a RAM (846) for a core (840), particularly a processor (including a CPU, GPU, FPGA, etc.) therein, and modifying such a data configuration based on a process defined by the software, and causing the core (840) to execute a specific process or a specific part of a specific process described herein. Alternatively or in addition, the computer system provides functions by logic incorporated in a circuit (e.g., an accelerator (844)) in a hardwired or other form, and the logic can execute a specific process or a specific part of a specific process described herein by operating instead of or together with the software. Where appropriate, the mentioned software may include logic, and conversely, the mentioned logic may include software. Where appropriate, the mentioned computer-readable media may include a circuit (e.g., an integrated circuit (IC)) in which the executed software is stored, a circuit embodying the executed logic, or both. The present disclosure includes any suitable combination of hardware and software.

[0102] Although some non-limiting examples have been described above, modifications, substitutions, and various alternative equivalents within the scope of the present disclosure exist. Therefore, it should be understood that although not explicitly shown or described herein, those skilled in the art can conceive of many systems and methods that embody the principles of the present disclosure and thus fall within its spirit and scope.

[0103] (Appendix 1) A video decoding method executed by at least one processor for controlling multi-line intra prediction using non-zero reference lines, comprising: determining whether an intra prediction mode of a first adjacent block of a current block is an angular mode; determining whether an intra prediction mode of a second adjacent block of the current block is an angular mode; generating a most probable mode (MPM) list including six candidate modes for intra prediction of the current block; wherein all of the six candidate modes are angular modes; and when it is determined that the intra prediction mode of the first adjacent block is an angular mode, the intra prediction mode of the first adjacent block is included in the MPM list, and when it is determined that the intra prediction mode of the second adjacent block is an angular mode, the intra prediction mode of the second adjacent block is included in the MPM list. (Appendix 2) The step of generating the MPM list includes performing the following operations when it is determined that the intra prediction mode of the first adjacent block is an angular mode and it is determined that the intra prediction mode of the second adjacent block is not an angular mode: setting a first candidate mode of the MPM list to the intra prediction mode of the first adjacent block; setting a second candidate mode of the MPM list to a first adjacent angular mode of the intra prediction mode of the first adjacent block; Set the third candidate mode of the MPM list to a second adjacent angle mode different from the first adjacent angle mode of the intra prediction mode of the first adjacent block. Set the fourth candidate mode of the MPM list to an adjacent angle mode of the first adjacent angle mode other than the intra prediction mode of the first adjacent block. Set the fifth candidate mode of the MPM list to an adjacent angle mode of the second adjacent angle mode other than the intra prediction mode of the first adjacent block. The video decoding method according to Supplementary Note 1, wherein the sixth candidate mode of the MPM list is set to an adjacent angle mode of the fourth candidate mode other than the first adjacent angle mode, or an adjacent angle mode of the fifth candidate mode other than the second adjacent angle mode. (Supplementary Note 3) In the index of the MPM list, the order from the lowest to the highest of the most likely candidate modes is the first candidate mode to the sixth candidate mode, according to the video decoding method described in Supplementary Note 2. (Supplementary Note 4) Before generating the MPM list, the step includes performing the following operations when it is determined that the intra prediction mode of the first adjacent block is an angle mode and the intra prediction mode of the second adjacent block is an adjacent angle mode with respect to the intra prediction mode of the first adjacent block. Set the first candidate mode of the MPM list to the intra prediction mode of the first adjacent block. Set the second candidate mode of the MPM list to the intra prediction mode of the second adjacent block. Determine the magnitude relationship of the angle modes between the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block. Here, the smaller angle mode is the first mode, and the larger angle mode is the second mode. In the first case, the first mode is equal to the minimum angle mode signaled in the codec standard, the second mode is equal to the maximum angle mode signaled in the codec standard, the codec standard is used by the at least one processor to control multi-line intra prediction using the non-zero reference line, and in the first case, the following operations are performed: Set the third candidate mode of the MPM list to the first adjacent angle mode of the second mode, which has a value between the first mode and the second mode. Set the fourth candidate mode of the MPM list to the first adjacent angle mode of the first mode, which has a value between the first mode and the second mode. Set the fifth candidate mode of the MPM list to an adjacent angle mode of the first adjacent angle mode of the second mode other than the second mode. Set the sixth candidate mode of the MPM list to an adjacent angle mode of the first adjacent angle mode of the first mode other than the first mode. In a second case different from the first case, the following operations are performed: Set the third candidate mode of the MPM list to the second adjacent angle mode of the first mode, which has a value not between the first mode and the second mode. Set the fourth candidate mode of the MPM list to the second adjacent angle mode of the second mode, which has a value not between the first mode and the second mode. Set the fifth candidate mode of the MPM list to an adjacent angle mode of the second adjacent angle mode of the first mode other than the first mode. The video decoding method according to Appendix 1, wherein the sixth candidate mode of the MPM list is set to an adjacent angle mode of the second adjacent angle mode of the second mode other than the second mode. (Appendix 5) The video decoding method according to Appendix 4, wherein in the index of the MPM list, the order from the lowest to the highest of the most likely candidate modes is from the first candidate mode to the sixth candidate mode. (Appendix 6) The step of generating the MPM list includes the step of determining that the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block are angular modes, and that the absolute value of the difference between the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block is greater than 2 and less than or equal to a predetermined threshold, where the predetermined threshold is a positive integer greater than 2, and performing the following operations: Setting the first candidate mode of the MPM list to the intra prediction mode of the first adjacent block; Setting the second candidate mode of the MPM list to the intra prediction mode of the second adjacent block; Setting the third candidate mode of the MPM list to the first adjacent angle mode of the intra prediction mode of the first adjacent block; Setting the fourth candidate mode of the MPM list to the second adjacent angle mode different from the first adjacent angle mode of the intra prediction mode of the first adjacent block; Setting the fifth candidate mode of the MPM list to the first adjacent angle mode of the intra prediction mode of the second adjacent block; Setting the sixth candidate mode of the MPM list to the second adjacent angle mode of the intra prediction mode of the second adjacent block, where the second adjacent angle mode of the intra prediction mode of the second adjacent block is different from the first adjacent angle mode of the intra prediction mode of the second adjacent block, as described in Supplementary Note 1 of the video decoding method. (Supplementary Note 7) In the index of the MPM list, the order from the lowest to the highest of the most probable candidate modes is from the first candidate mode to the sixth candidate mode, as described in Supplementary Note 6 of the video decoding method. (Supplementary Note 8) Furthermore, determining whether the intra prediction mode of the first adjacent block of the current block is a planar mode or a direct current (DC) mode; determining whether the intra prediction mode of the second adjacent block of the current block is a planar mode or a DC mode, including: The step of generating the MPM list includes performing at least one of the following operations when it is determined that the intra prediction mode of the first adjacent block is the planar mode or the DC mode and it is determined that the intra prediction mode of the second adjacent block is the planar mode or the DC mode: Setting each of the six candidate modes to a value among {50, 18, 2, 34, 66, 26}; Setting each of the six candidate modes to a value among {50, 18, 2, 34, 26, 42}; The six candidate modes are the video decoding method described in Appendix 1 corresponding to the modes of Versatile Video Coding (VVC). (Appendix 9) In the index of the MPM list, the order from the lowest to the highest of the most probable candidate modes is the order of {50, 18, 2, 34, 66, 26} or the order of {50, 18, 2, 34, 26, 42}, which is the video decoding method described in Appendix 8. (Appendix 10) The step of generating the MPM list includes performing the following operations when it is determined that the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block are angular modes: Adding the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block to the MPM list; Adding each adjacent angular mode between the first adjacent block and the second adjacent block that has not been previously added to the MPM list to the MPM list; After adding each adjacent angular mode between the first adjacent block and the second adjacent block to the MPM list, if the MPM list is not completely filled, until the MPM list is completely filled, adding adjacent angular modes of the modes included in the MPM list that have not been previously added to the MPM list to the MPM list, which is the video decoding method described in Appendix 1. (Appendix 11) An apparatus for decoding a video sequence by controlling multi-line intra prediction using a non-zero reference line, At least one memory arranged to store computer program code, and at least one processor arranged to access the at least one memory and operate based on the computer program code, wherein the computer program code comprises: first determination code arranged to cause the at least one processor to determine whether an intra prediction mode of a first adjacent block of a current block is an angular mode; second determination code arranged to cause the at least one processor to determine whether an intra prediction mode of a second adjacent block of the current block is an angular mode; generation code arranged to cause the at least one processor to generate a most probable mode (MPM) list, the MPM list including six candidate modes for intra prediction of the current block, wherein all of the six candidate modes are angular modes, and the generation code is arranged to cause the at least one processor to generate the MPM list such that when it is determined that the intra prediction mode of the first adjacent block is an angular mode, the MPM list includes the intra prediction mode of the first adjacent block, and when it is determined that the intra prediction mode of the second adjacent block is an angular mode, the MPM list includes the intra prediction mode of the second adjacent block. (Appendix 12) When the generation code determines that the intra prediction mode of the first adjacent block is an angular mode and the intra prediction mode of the second adjacent block is not an angular mode, the generation code is arranged to cause the at least one processor to perform the following operations: set a first candidate mode of the MPM list to the intra prediction mode of the first adjacent block; set a second candidate mode of the MPM list to a first adjacent angular mode of the intra prediction mode of the first adjacent block; Set the third candidate mode of the MPM list to a second adjacent angle mode different from the first adjacent angle mode of the intra prediction mode of the first adjacent block. Set the fourth candidate mode of the MPM list to an adjacent angle mode of the first adjacent angle mode other than the intra prediction mode of the first adjacent block. Set the fifth candidate mode of the MPM list to an adjacent angle mode of the second adjacent angle mode other than the intra prediction mode of the first adjacent block. The apparatus according to Supplementary Note 11, wherein the sixth candidate mode of the MPM list is set to an adjacent angle mode of the fourth candidate mode other than the first adjacent angle mode, or an adjacent angle mode of the fifth candidate mode other than the second adjacent angle mode. (Supplementary Note 13) The apparatus according to Supplementary Note 12, wherein the generation code causes the at least one processor to generate the MPM list such that, at the index of the MPM list, the order from the lowest to the highest of the most likely candidate modes is from the first candidate mode to the sixth candidate mode. (Supplementary Note 14) When the generation code determines that the intra prediction mode of the first adjacent block is an angle mode and the intra prediction mode of the second adjacent block is an adjacent angle mode with respect to the intra prediction mode of the first adjacent block, it is arranged to cause the at least one processor to execute the following operations: Set the first candidate mode of the MPM list to the intra prediction mode of the first adjacent block. Set the second candidate mode of the MPM list to the intra prediction mode of the second adjacent block. Determine the magnitude relationship of the angle modes between the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block, where the smaller angle mode is the first mode and the larger angle mode is the second mode. In the first case, the first mode is equal to the minimum angle mode signaled in the codec standard, and the second mode is equal to the maximum angle mode signaled in the codec standard. The codec standard is used by the at least one processor to control multi-line intra prediction using the non-zero reference line. In the first case, the following operations are performed: Set the third candidate mode of the MPM list to the first adjacent angle mode of the second mode, which has a value between the first mode and the second mode. Set the fourth candidate mode of the MPM list to the first adjacent angle mode of the first mode, which has a value between the first mode and the second mode. Set the fifth candidate mode of the MPM list to an adjacent angle mode of the first adjacent angle mode of the second mode, other than the second mode. Set the sixth candidate mode of the MPM list to an adjacent angle mode of the first adjacent angle mode of the first mode, other than the first mode. In a second case different from the first case, the following operations are performed: Set the third candidate mode of the MPM list to the second adjacent angle mode of the first mode, which has a value not between the first mode and the second mode. Set the fourth candidate mode of the MPM list to the second adjacent angle mode of the second mode, which has a value not between the first mode and the second mode. Set the fifth candidate mode of the MPM list to an adjacent angle mode of the second adjacent angle mode of the first mode, other than the first mode. The apparatus according to Supplementary Note 11, wherein the sixth candidate mode of the MPM list is set to an adjacent angle mode of the second adjacent angle mode of the second mode, other than the second mode. (Supplementary Note 15) The generation code is arranged to cause the at least one processor to generate the MPM list such that, at the index of the MPM list, the order from the lowest to the highest of the most probable candidate modes is the first candidate mode to the sixth candidate mode, according to the apparatus described in appended note 14. (Appended note 16) The generation code is arranged to cause the at least one processor to perform the following operations when it is determined that the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block are angular modes, and the absolute value of the difference between the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block is greater than 2 and less than or equal to a predetermined threshold, where the predetermined threshold is a positive integer greater than 2: Set the first candidate mode of the MPM list to the intra prediction mode of the first adjacent block. Set the second candidate mode of the MPM list to the intra prediction mode of the second adjacent block. Set the third candidate mode of the MPM list to the first adjacent angular mode of the intra prediction mode of the first adjacent block. Set the fourth candidate mode of the MPM list to the second adjacent angular mode different from the first adjacent angular mode of the intra prediction mode of the first adjacent block. Set the fifth candidate mode of the MPM list to the first adjacent angular mode of the intra prediction mode of the second adjacent block. Set the sixth candidate mode of the MPM list to the second adjacent angular mode of the intra prediction mode of the second adjacent block, where the second adjacent angular mode of the intra prediction mode of the second adjacent block is different from the first adjacent angular mode of the intra prediction mode of the second adjacent block, according to the apparatus described in appended note 11. (Appended note 17) The generation code is arranged to cause the at least one processor to generate the MPM list such that, at the index of the MPM list, the order from the lowest to the highest of the most probable candidate modes is the first candidate mode to the sixth candidate mode, according to the apparatus described in appended note 16. (Appendix 18) The computer program code further includes a third determination code arranged to cause the at least one processor to determine whether the intra prediction mode of the first adjacent block of the current block is a planar mode or a DC mode; and a fourth determination code arranged to cause the at least one processor to determine whether the intra prediction mode of the second adjacent block of the current block is a planar mode or a DC mode, and includes When the generation code determines that the intra prediction mode of the first adjacent block is a planar mode or a DC mode, and determines that the intra prediction mode of the second adjacent block is a planar mode or a DC mode, it is arranged to cause the at least one processor to execute at least one of the following operations set the six candidate modes to values in {50, 18, 2, 34, 66, 26} respectively; set the six candidate modes to values in {50, 18, 2, 34, 26, 42} respectively; The six candidate modes are the device described in Appendix 11 corresponding to the modes of multi-functional video coding VVC. (Appendix 19) When the generation code determines that the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block are angular modes, it is arranged to cause the at least one processor to execute the following operations add the intra prediction mode of the first adjacent block and the intra prediction mode of the second adjacent block to the MPM list; add each adjacent angular mode between the first adjacent block and the second adjacent block that has not been previously added to the MPM list to the MPM list; 12. The apparatus of claim 11, further comprising: if, after adding each adjacent angle mode of the first adjacent block and the second adjacent block to the MPM list, the MPM list is not completely filled, the apparatus adds adjacent angle modes of modes included in the MPM list that were not previously added to the MPM list to the MPM list until the MPM list is completely filled. (Appendix 20) A computer program causing a computer to carry out the method according to any one of claims 1 to 11. [Prior art documents] [Non-patent literature]

[0104] [Non-Patent Document 1] CHANG, Yao-Jen et al.,Improved intra prediction method based on arbitrary reference tier coding schemes,2016 Picture Coding Symposium,IEEE,April 24, 2017,pp.1-5,<URL:https: / / ieeexplore.ieee.org / document / <7906339> ,<DOI: 10.1109 / PCS.2016.<7906339> [Non-Patent Document 2] BROSS, Benjamin et al.,Versatile Video Coding (Draft 2),Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 11th Meeting: Ljubljana, SI, 10-18 July 2018, [JVET-K1001-v4],JVET-K1001 (version 4),ITU-T, August 17, 2018, <URL:http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 11_Ljubljana / wg11 / JVET-K1001-v4.zip>: JVET-K1001-v4.docx: pp.43-46

Claims

1. A video decoding method executed by at least one processor for controlling multi-line intra prediction using a non-zero reference line, comprising: determining what the intra prediction mode of the first adjacent block of the current block is; determining what the intra prediction mode of the second adjacent block of the current block is; generating a most probable mode (MPM) list including a plurality of candidate modes for intra prediction of the current block, wherein when it is determined that the intra prediction mode of the first adjacent block is an angular mode and the intra prediction mode of the second adjacent block is not an angular mode, the step of generating the MPM list is: setting the first candidate mode of the MPM list to the intra prediction mode of the first adjacent block; setting the second candidate mode of the MPM list to the first adjacent angular mode of the intra prediction mode of the first adjacent block; setting the third candidate mode of the MPM list to the second adjacent angular mode of the intra prediction mode of the first adjacent block, which is different from the first adjacent angular mode; setting the fourth candidate mode of the MPM list to the adjacent angular mode of the first adjacent angular mode, which is not the intra prediction mode of the first adjacent block; including setting the fifth candidate mode of the MPM list to the adjacent angular mode of the second adjacent angular mode, which is not the intra prediction mode of the first adjacent block; wherein when it is determined that the intra prediction mode of the first adjacent block of the current block is a planar mode or a DC mode and the intra prediction mode of the second adjacent block of the current block is a planar mode or a DC mode, the step of generating the MPM list is: constructing the MPM list such that the candidate modes of the MPM list include V, H, V−k, and V + k, or constructing the MPM list such that the candidate modes of the MPM list include V, H, H−k, and H + k, where V is a vertical mode, H is a horizontal mode, and k is a predetermined natural number.

2. The MPM list generated when it is determined that the intra prediction mode of the first adjacent block is an angular mode and the intra prediction mode of the second adjacent block is not an angular mode is: MPM[0] = ang_mode; MPM[1] = ((ang_mode + offset) % mod) + 2; MPM[2] = ((ang_mode - 1) % mod) + 2; MPM[3] = ((ang_mode - 1 + offset) % mod) + 2; MPM[4] = ((ang_mode) % mod) + 2; The method according to claim 1, comprising the above, wherein ang_mode indicates the intra prediction mode of the first adjacent block.

3. The method according to claim 1 or 2, wherein in the index of the MPM list, the MPM list is generated such that the priority order of the most likely candidate modes is from the first candidate mode to the last candidate mode.

4. An apparatus for encoding a video sequence by controlling multi-line intra prediction using a non-zero reference line, comprising: At least one memory arranged to store computer program code; At least one processor configured to access the at least one memory and execute the method according to any one of claims 1 to 3 based on the computer program code.

5. A computer program for causing the at least one processor to execute the method according to any one of claims 1 to 3.

Images