Reference sample padding and filtering for intra prediction in video compression

Abstract

A method and apparatus for controlling intra prediction for decoding a video sequence is provided. [Solution] The method includes setting a table including a plurality of intra prediction modes respectively corresponding to a plurality of intra prediction angles, wherein the plurality of intra prediction angles include diagonal intra prediction angles respectively corresponding to diagonal directions of all shapes of a coding unit, and selecting one of the plurality of intra prediction modes from the set table to decode the video sequence.

Description

[Technical Field]

[0001] Reference to Related Applications This application claims priority to U.S. Provisional Patent Application No. 62 / 698,526, filed with the U.S. Patent and Trademark Office on July 16, 2018. This provisional application is incorporated herein by reference in its entirety.

[0002] Methods and apparatus according to embodiments relate to video processing, and more particularly to reference sample padding and filtering for intra prediction in video compression. [Background technology]

[0003] The intra prediction modes used in High Efficiency Video Coding (HEVC) are shown in Figure 1. In HEVC, mode 10 (101) is the horizontal mode, mode 26 (102) is the vertical mode, and mode 2 (103), mode 18 (104), and mode 34 (105) are diagonal modes. Mode 2 (103) and mode 34 (105) indicate the same prediction direction.

[0004] Furthermore, HEVC calls the wide-angle intra prediction mode a prediction mode that covers a wider angle than the range of prediction directions covered by conventional intra prediction modes. This wide angle is only applicable to non-square blocks such as the following:

[0005] Angle greater than 45 degrees in the upper right direction (intra prediction mode 34 in HEVC) when the block width exceeds the block height.

[0006] Angle greater than 45 degrees in the bottom-left direction (intra prediction mode 2 in HEVC) when the block height exceeds the block width.

[0007] Furthermore, in HEVC, the intra sample substitution process of the intra mode is described in Section 8.4.4.2 reproduced below, which includes a reference sample substitution process, a neighboring process filtering process, and an intra prediction process. 8.4.4.2.2 Reference Sample Replacement Process for Intra-Sample Prediction The inputs to this process are: Reference samples p[x][y] for x=-1, y=-1...nTbS*2-1 and x=0...nTbS*2-1, y=-1 for intra-sample prediction; Transform block size nTbS, Variable cIdx that indicates the color component of the current block is. The output of this process is the modified reference samples p[x][y] for x=-1, y=-1...nTbS*2-1 and x=0...nTbS*2-1, y=-1 for intra-sample prediction. The variable bitDepth is calculated as follows: If cIdx is equal to 0, set bitDepth to BitDepth Y Set it equal to Otherwise, set bitDepth to BitDepth C Set it equal to The values ​​of the samples p[x][y] for x=-1, y=-1...nTbS*2-1 and x=0...nTbS*2-1, y=-1 are modified as follows: If all samples p[x][y] in x=-1,y=-1...nTbS*2-1 and x=0...nTbS*2-1,y=-1 are evaluated as "unusable for intra prediction", then all samples p[x Set the value of ][y] to 1<<(bitDepth-1). Otherwise (if at least one but not all samples p[x][y] are evaluated as "unusable for intra prediction"), perform the following ordered steps: 1. If p[-1][nTbS*2-1] is evaluated as "unusable for intra prediction," sequentially search from x=-1,y=nTbS*2-1 to x=-1,y=-1, then sequentially search from x=0,y=-1 to x=nTbS*2-1,y=-1. If a sample p[x][y] evaluated as "usable for intra prediction" is found, terminate the search and assign the value of p[x][y] to p[-1][nTbS*2-1]. 2. Sequentially search from x=-1, y=nTbS*2-2 to x=-1, y=-1, and if p[x][y] is evaluated as "unusable for intra prediction", substitute the value of p[x][y+1] for the value of p[x][y]. 3. For x=0...nTbS*2-1, y=-1, if p[x][y] is evaluated as "unusable for intra prediction", assign the value of p[x-1][y] to the value of p[x][y]. All samples p[x][y] in x=-1, y=-1...nTbS*2-1 and x=0...nTbS*2-1, y=-1 are evaluated as "available for intra prediction". 8.4.4.2.3 Adjacent sample filtering process The inputs to this process are: Adjacent samples p[x][y] of x=-1,y=-1...nTbS*2-1 and x=0...nTbS*2-1,y=-1 Variable nTbS indicating the transformation block size is. The output of this process is the filtered samples pF[x][y] for x=-1, y=-1...nTbS*2-1 and x=0...nTbS*2-1, y=-1. The variable filterFlag is calculated as follows: Set filterFlag equal to 0 if one or more of the following conditions are true: predModeIntra is equal to INTRA_DC. nTbS is equal to 4. Otherwise, the following applies: Set the variable minDistVerHor equal to Min(Abs(predModeIntra-26), Abs(predModeIntra-10)). The specification of the variable intraHorVerDistThres[nTbS] is shown in Table 8-3. The variable filterFlag is calculated as follows: If minDistVerHor exceeds intraHorVerDistThres[nTbS], set filterFlag to a value equal to 1. Otherwise, set filterFlag equal to 0.

[0008] [Table 1]

[0009] If filterFlag is equal to 1, then apply the following: The variable biIntFlag is calculated as follows: Set biIntFlag equal to 1 if all of the following conditions are true: strong_intra_smoothing_enabled_flag equals 1. nTbS is equal to 32. Abs(p[-1][-1]+p[nTbS*2-1][-1]-2*p[nTbS-1][-1])<(1<<(BitDepth Y -5)) Abs(p[-1][-1]+p[-1][nTbS*2-1]-2*p[-1][nTbS-1])<(1<<(BitDepth Y -5)) Otherwise, set biIntFlag equal to 0. The filtering is performed as follows: If biIntFlag is equal to 1, the filtered sample values ​​pF[x][y] for x=-1, y=-1...63 and x=0...63, y=-1 are calculated as follows: pF[-1][-1]=p[-1][-1](8-30) pF[-1][y]=((63-y)*p[-1][-1]+(y+1)*p[-1]

[63] +32)>>6 Set y=0...62 (8-31) pF[-1]

[63] =p[-1]

[63] (8-32) pF[x][-1]=((63-x)*p[-1][-1]+(x+1)*p

[63] [-1]+32)>>6 Let x=0...62 (8-33) pF

[63] [-1]=p

[63] [-1](8-34) Otherwise (when biIntFlag is equal to 0), the filtered sample values ​​pF[x][y] for x=-1, y=-1...nTbS*2-1 and x=0...nTbS*2-1, y=-1 are calculated as follows: pF[-1][-1]=(p[-1][0]+2*p[-1][-1]+p[0][-1]+2)>>2(8-35) pF[-1][y]=(p[-1][y+1]+2*p[-1][y]+p[-1][y-1]+2)>>2 y=0...nTbS*2-2 (8-36) pF[-1][nTbS*2-1]=p[-1][nTbS*2-1](8-37) pF[x][-1]=(p[x-1][-1]+2*p[x][-1]+p[x+1][-1]+2)>>2 Let x=0...nTbS*2-2 (8-38) pF[nTbS*2-1][-1]=p[nTbS*2-1][-1](8-39) 8.4.4.2.6 Intra prediction mode specifications in the INTRA_ANGULAR2 and INTRA_ANGULAR34 ranges The inputs to this process are: Intra prediction mode predModeIntra Adjacent samples p[x][y] of x=-1,y=-1...nTbS*2-1 and x=0...nTbS*2-1,y=-1 Variable nTbS indicating the transformation block size Variable cIdx that indicates the color component of the current block is. The output of this process is the predicted samples predSamples[x][y] for x,y=0...nTbS-1. Figure 8-2 shows all 33 intra angles, and Table 8-4 shows the specification of the mapping table between predModeIntra and the angle parameter intraPredAngle.

[0010] [Table 2]

[0011] Additionally, Table 8-5 shows the specification of the mapping table between predModeIntra and the inverse angle parameter invAngle.

[0012] [Table 3]

[0013] The values ​​of the predicted samples predSamples[x][y] for x, y=0...nTbS-1 are calculated as follows: If predModeIntra is 18 or greater, apply the following steps in order: 1. Define the reference sample array ref[x] as follows: The following applies: ref[x]=p[-1+x][-1]where x=0...nTbS(8-47) If intraPredAngle is less than 0, extend the primary reference sample sequence as follows: If (nTbS*intraPredAngle)>>5 is less than -1, ref[x]=p[-1][-1+((x*invAngle+128)>>8)] where x=-1...(nTbS*intraPredAngle)>>5(8-48) Otherwise, ref[x]=p[-1+x][-1]where x=nTbS+1...2*nTbS(8-49) 2. The predicted sample values ​​predSamples[x][y] for x, y=0...nTbS-1 are calculated as follows: a. The index variable iIdx and the magnification factor iFact are calculated as follows: iIdx=((y+1)*intraPredAngle)>>5(8-50) iFact=((y+1)*intraPredAngle)&31(8-51) b. Depending on the value of iFact, the following applies: If iFact is not equal to 0, the predicted sample values ​​predSamples[x][y] are calculated as follows: demand. predSamples[x][y]= ((32-iFact)*ref[x+iIdx+1]+iFact*ref[x+iIdx+2]+16)>>5(8-52) Otherwise, the predicted sample values ​​predSamples[x][y] are calculated as follows: predSamples[x][y]=ref[x+iIdx+1](8-53) If c.predModeIntra is equal to 26 (vertical), and cIdx is equal to 0, and nTbS is less than 32, apply the following filtering where x=0, y=0...nTbS-1: predSamples[x][y]=Clip1 Y (p[x][-1]+((p[-1][y]-p[-1][-1])>>1))(8-54) Otherwise (if predModeIntra is less than 18), apply the following steps in order: 1. Define the reference sample array ref[x] as follows: The following applies: ref[x]=p[-1][-1+x]where x=0...nTbS(8-55) If intraPredAngle is less than 0, extend the primary reference sample sequence as follows: If (nTbS*intraPredAngle)>>5 is less than -1, ref[x]=p[-1+((x*invAngle+128)>>8)][-1] where x=-1...(nTbS*intraPredAngle)>>5(8-56) Otherwise, ref[x]=p[-1][-1+x]where x=nTbS+1...2*nTbS(8-57) 2. The values ​​of the predicted samples predSamples[x][y] for x, y=0...nTbS-1 are calculated as follows: a. The index variable iIdx and the magnification factor iFact are calculated as follows: iIdx=((x+1)*intraPredAngle)>>5(8-58) iFact=((x+1)*intraPredAngle)&31(8-59) b. Depending on the value of iFact, the following applies: If iFact is not equal to 0, the predicted sample values ​​predSamples[x][y] are calculated as follows: predSamples[x][y]= ((32-iFact)*ref[y+iIdx+1]+iFact*ref[y+iIdx+2]+16)>>5(8-60) Otherwise, the predicted sample values ​​predSamples[x][y] are calculated as follows: predSamples[x][y]=ref[y+iIdx+1](8-61) If c.predModeIntra is equal to 10 (horizontal) and cIdx is equal to 0 and nTbS is less than 32, apply the following filtering where x=0...nTbS-1, y=0: predSamples[x][y]=Clip1 Y (p[-1][y]+((p[x][-1]-p[-1][-1])>>1))(8-62) Summary of the Invention [Means for solving the problem]

[0014] According to an embodiment, a method for controlling intra prediction for decoding a video sequence, performed by at least one processor, includes configuring a table including a plurality of intra prediction modes corresponding to a plurality of intra prediction angles, the plurality of intra prediction angles including diagonal intra prediction angles corresponding to diagonal directions of all shapes of a coding unit, and further including selecting one of the plurality of intra prediction modes from the configured table for decoding the video sequence.

[0015] According to an embodiment, an apparatus for controlling intra prediction for decoding a video sequence includes at least one memory configured to store computer program code and at least one processor configured to access the at least one memory and operate according to the computer program code. The computer program code includes setting code configured to cause the at least one processor to set a table including a plurality of intra prediction modes corresponding to a plurality of intra prediction angles, respectively. The plurality of intra prediction angles include diagonal intra prediction angles corresponding to diagonal directions of all shapes of a coding unit, respectively. The computer program code further includes selection code configured to cause the at least one processor to select one of the plurality of intra prediction modes from the set table for decoding the video sequence.

[0016] According to an embodiment, a non-transitory computer-readable storage medium stores instructions that cause a processor to configure a table including a plurality of intra-prediction modes corresponding to a plurality of intra-prediction angles, the plurality of intra-prediction angles including diagonal intra-prediction angles corresponding to diagonal directions of all shapes of a coding unit, and further cause the processor to select one of the plurality of intra-prediction modes from the configured table for decoding a video sequence. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram of HEVC intra-prediction modes. [Figure 2] FIG. 1 is a simplified block diagram of a communication system according to one embodiment. [Figure 3] FIG. 2 is a diagram of a video encoder and decoder configuration in a streaming environment according to one embodiment. [Figure 4] FIG. 2 is a functional block diagram of a video decoder according to an embodiment. [Figure 5] FIG. 2 is a functional block diagram of a video encoder according to an embodiment. [Figure 6A] FIG. 1 is a diagram of intra-prediction modes of the Versatile Video Coding (VVC) Test Model (VTM 1). [Figure 6B] FIG. 10 is a diagram illustrating changing the intra prediction mode for a non-square block to a wide-angle intra prediction mode. [Figure 7A] FIG. 10 is a diagram of predicted angles within and beyond the diagonal, according to one embodiment. [Figure 7B] FIG. 10 illustrates adjusting prediction angles beyond the diagonal, according to one embodiment. [Figure 8] 1 is a flowchart illustrating a method for controlling intra prediction for decoding a video sequence, according to one embodiment. [Figure 9] 1 is a simplified block diagram of an apparatus for controlling intra-prediction for decoding a video sequence according to one embodiment; [Figure 10] FIG. 1 illustrates a computer system suitable for implementing embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0018] FIG. 2 is a simplified block diagram of a communication system (200) according to one embodiment. The communication system (200) may include at least two terminals (210-220) interconnected via a network (250). For one-way data transmission, a first terminal (210) may locally encode video data for transmission to another terminal (220) via the network (250). The second terminal (220) may receive the other terminal's encoded video data from the network (250), decode the encoded data, and display the reconstructed video data. One-way data transmission is common in media handling applications.

[0019] 2 shows another set of terminals (230, 240) provided to support two-way transmission of encoded video, such as might occur during a video conference. For the two-way transmission of data, each terminal (230, 240) may encode video data captured locally for transmission to other terminals over a network (250). Again, each terminal (230, 240) may receive encoded video data transmitted by other terminals, decode the encoded data, and display the recovered video data on a local display device.

[0020] In FIG. 2 , the terminals (210-240) may be depicted as servers, personal computers, and smartphones, although the principles of the embodiments are not so limited. The embodiments find application with laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. The network (250) represents any number of networks (e.g., including wired and / or wireless communication networks) that carry encoded video data between the terminals (210-240). The communication network (250) may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of the network (250) may not be important to the operation of the embodiments, unless otherwise described herein.

[0021] 3 is a diagram of a video encoder and decoder configuration in a streaming environment, according to one embodiment. For example, the disclosed subject matter is equally applicable to other uses in which video can be used, including video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0022] A streaming system may include a video source (301), e.g., an imaging subsystem (313), which may include a digital camera, that generates an uncompressed video sample stream (302). This sample stream (302) (shown as a thick line to emphasize its large amount of data compared to an encoded video bitstream) may be processed by an encoder (303) connected to the camera (301). The encoder (303) may include hardware, software, or a combination thereof that enables or implements aspects of the disclosed subject matter, as described in more detail below. An encoded video bitstream (304) (shown as a thin line to emphasize its small amount of data compared to the sample stream) may be stored on a streaming server (305) for later use. One or more streaming clients (306, 308) may access the streaming server (305) to obtain copies (307, 309) of the encoded video bitstream (304). The client (306) may include a video decoder (310) that decodes a copy of an input encoded video bitstream (307) to generate an output video sample stream (311) that can be rendered on a display (312) or other rendering device (not shown). In some streaming systems, the video bitstreams (304, 307, 309) may be encoded according to a predetermined video encoding / compression standard. Examples of such standards include ITU-T Recommendation H265. A video coding standard informally known as VVC is under development. The disclosed subject matter may be used in connection with VVC.

[0023] FIG. 4 is a functional block diagram of a video decoder (310) according to one embodiment.

[0024] The receiver (410) may receive one or more codec video sequences to be decoded by the decoder (310). In some embodiments, the receiver receives one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel (412), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (410) may also receive coded video data with other data, such as coded audio data and / or ancillary data streams, and may forward these data streams to corresponding points of use (not shown). The receiver (410) may separate the coded video sequences from the other data. To address network jitter, a buffer memory (415) may be connected between the receiver (410) and the entropy decoder / parser (420) (hereinafter "parser"). If the receiver 410 is receiving data from a store-and-forward device or an isosynchronous network with sufficient bandwidth and control, the buffer 415 may not be required or may be small. For use with best-effort packet networks such as the Internet, a buffer 415 may be required, and the buffer 415 may be relatively large, with the advantage that it can be optimally sized.

[0025] The video decoder (310) may include a parser (420) that reconstructs symbols (421) from the entropy-encoded video sequence. The symbol categories include information used to manage the operation of the decoder (310) and may also include information for controlling a rendering device, such as a display (312), that is not an integral part of the decoder but is connectable to the decoder, as shown in FIG. 4. The control information for one or more rendering devices may take the form of a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (420) may parse and entropy decode the received coded video sequence. The coding of the coded video sequence may conform to a video coding technique or standard and may follow principles well known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding, and the like, with or without context-dependent coding. The parser (420) may extract a set of subgroup parameters for at least one subgroup of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. A subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The entropy decoder / parser may also extract information such as transform coefficients, quantizer parameter (QP) values, motion vectors, etc. from the coded video sequence.

[0026] The parser (420) may generate symbols (421) by performing entropy decoding / parsing operations on the video sequence received from the buffer (415). The parser (420) may receive the encoded data and selectively decode particular symbols (421). Additionally, the parser (420) may determine whether a particular symbol (421) is to be provided to a motion compensation prediction unit (453), a scaler / inverse transform unit (451), an intra prediction unit (452), or a loop filter unit (454).

[0027] The reconstruction of the symbols (421) can involve several different parts depending on the type of coded video picture or part thereof (e.g., inter-picture and intra-picture, inter-block and intra-block, etc.) and other factors. Which parts are involved and how can be controlled by subgroup control information parsed from the coded video sequence by the parser (420). The flow of such subgroup control information between the parser (420) and the parts described below is not shown for clarity.

[0028] Beyond the functional blocks already described, the decoder (310) may be conceptually subdivided into several functional components, as described below. While in a practical implementation operating within commercial constraints, many of these components may interact closely with one another and may be at least partially integrated with one another, the conceptual subdivision into functional components described below is consistent with the purpose of describing the disclosed subject matter.

[0029] The first part is the scalar / inverse transform unit (451), which receives quantized transform coefficients and control information, including the transform coefficients, block size, quantization factor, quantization scaling matrix, etc., as symbols (421) from the parser (420). The scalar / inverse transform unit (451) can output blocks comprising sample values, which can be input to the aggregator (455).

[0030] In some cases, the output samples of the scaler / inverse transform (451) may involve intra-coded blocks, which are blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by the intra-picture predictor (452). In some cases, the intra-picture predictor (452) uses surrounding, already reconstructed information from the current (partially reconstructed) picture (456) to generate blocks of the same size and shape as the block being reconstructed. In some cases, the aggregator (455) adds, on a sample-by-sample basis, the prediction information generated by the intra predictor (452) to the output sample information provided by the scaler / inverse transform (451).

[0031] In other cases, the output samples of the scaler / inverse transform unit (451) may relate to inter-coded blocks, and possibly motion-compensated blocks. In this case, the motion compensation prediction unit (453) may access a reference picture memory (457) to obtain samples used for prediction. After motion-compensating the obtained samples according to the symbols (421) associated with the blocks, the aggregator (455) may append these samples to the output of the scaler / inverse transform unit (referred to as residual samples or residual signals in this case) to generate output sample information. The addresses in the reference picture memory from which the motion compensation unit obtains the prediction samples may be controlled by a motion vector and available to the motion compensation unit in the form of a symbol (421) that may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​obtained from the reference picture memory when using sub-sample accurate motion vectors, motion vector prediction mechanisms, and the like.

[0032] The output samples of the aggregator (455) may be subjected to various loop filtering techniques in the loop filter unit (454). The video compression techniques may include in-loop filtering techniques controlled by parameters contained in the coded video bitstream and made available to the loop filter unit (454) as symbols (421) from the parser (420). However, the in-loop filtering techniques may also be influenced by meta-information obtained when decoding previous portions (earlier in decoding order) of the coded picture or coded video sequence, and by previously reconstructed loop-filtered sample values.

[0033] The output of the loop filter unit (454) can be a sample stream that can be output to a drawing device (312) and stored in a reference picture memory (456) for use in subsequent intra-picture prediction.

[0034] Once a given coded picture has been fully reconstructed, it can be used as a reference picture for subsequent prediction. Once a coded picture has been fully reconstructed and identified as a reference picture (e.g., by the parser (420)), the current reference picture (456) can become part of the reference picture buffer (457), allowing the current picture memory to be reallocated before reconstruction of a subsequent coded picture begins.

[0035] The video decoder (310) may perform decoding operations according to a specified video compression technique, which may be described in a standard document such as ITU-T Recommendation H265. The coded video sequence may conform to the syntax defined by the video compression technique or standard used, in the sense of adhering to the syntax of the video compression technique or standard as defined in the video compression technique document or standard, particularly for profiles, as defined in that document. Compliance may also require that the complexity of the coded video sequence be within limits imposed by the level of the video compression technique or standard. In some cases, the level may impose limitations on the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. In some cases, the limitations imposed by the level may be further restricted by a Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0036] In one embodiment, the receiver (410) may receive additional (redundant) data along with the coded video. The additional data may be included as part of one or more coded video sequences. The additional data may be used by the video decoder (310) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0037] FIG. 5 is a functional block diagram of a video encoder (303) according to an embodiment.

[0038] The encoder (303) may receive video samples from a video source (301) (not part of the encoder) that may capture video images that are encoded by the encoder (303).

[0039] The video source (301) may provide a raw video sequence to be encoded by the encoder (303) in the form of a digital video sample stream, which may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media handling system, the video source (301) may be a storage device that stores pre-prepared video. In a video conferencing system, the video source (301) may be a camera that captures image information of a particular location as a video sequence. Video data may also be provided as multiple individual pictures that, when viewed in sequence, create motion. The pictures themselves may be organized as a spatial array of pixels, where each pixel may comprise one or more samples depending on the sampling structure, color space, etc., used. Those skilled in the art will readily understand the relationship between pixels and samples. The following discussion focuses on samples.

[0040] According to one embodiment, the encoder (303) may encode and compress pictures of a source video sequence into a coded video sequence (543) in real time, or may encode and compress pictures subject to any other time constraints required by the application. Achieving the appropriate coding rate is one function of the controller (550). The controller controls and is operatively connected to other functional components described below. For clarity, connections are not shown. Parameters set by the controller may include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, etc.), picture size, group of pictures (GOP) placement, maximum motion vector search range, etc. One skilled in the art can readily identify other functions of the controller (550) relevant to the video encoder (303) optimized for a given system design.

[0041] Some video encoders operate within what those skilled in the art will readily understand as a "coding loop." In a highly simplified manner, the encoding loop may consist of an encoding portion, an encoder (530) (hereinafter "source coder") (responsible for generating symbols based on the input picture being coded and generating one or more reference pictures), and a (local) decoder (533) embedded in the encoder (303) that reconstructs the symbols to generate sample data that a (separate) decoder would also have generated (so as to avoid any loss in compression between the symbols and the coded video bitstream in video compression techniques contemplated by the disclosed subject matter). This reconstructed sample stream is input to a reference picture memory (534). Decoding the symbol stream results in a bit-invariant result regardless of the decoder's location (local or elsewhere), and so the contents of the reference picture buffer are also bit-invariant between the local and the elsewhere encoder. In other words, the predictive portion of the encoder "sees" the exact same sample values ​​for reference picture samples that the decoder "sees" when using prediction during decoding. This basic principle of reference picture synchronicity (and also the resulting drift when synchronicity cannot be maintained due to, for example, channel errors) is well known to those skilled in the art.

[0042] The operation of the "local" decoder (533) can be the same as the operation of the "alternate" decoder (310), which has already been described in detail above in conjunction with Figure 4. However, and with brief reference to Figure 4, the entropy decoding portion of the decoder (310), including the channel (412), receiver (410), buffer (415), and parser (420), need not be entirely implemented in the local decoder (533), provided that symbols are available and the entropy coder (545) and parser (420) can perform lossless encoding / decoding of symbols for the coded video sequence.

[0043] A remark that can be made at this point is that any decoder technique other than parsing / entropy decoding that is present in the decoder must necessarily be present in the corresponding encoder in approximately the same functional form. A description of the encoder technique can be omitted if it is the opposite of the decoder technique that is comprehensively described. Only in certain areas is a more detailed description necessary, which is provided below.

[0044] As part of its operation, the source coder (530) may perform motion-compensated predictive coding, predictively coding an input frame with reference to one or more previously coded frames from the video sequence designated as "reference frames." In this manner, the coding engine (532) codes the differences between pixel blocks of the input frame and pixel blocks of one or more reference frames that can be selected as one or more predictive references for the input frame.

[0045] The local video decoder (533) can decode the encoded video data of frames that can be designated as reference frames based on the symbols generated by the source coder (530). An advantage of the encoding engine (532) is that its operation can be a lossy process. If the encoded video data can be decoded by a video decoder (not shown in FIG. 4), the reconstructed video sequence can generally be a reproduction of the source video sequence, including some errors. The local video decoder (533) reproduces the decoding process that can be performed by the video decoder on the reference frames and can store the reconstructed reference frames in a reference picture cache (534). In this way, the encoder (303) can locally store copies of reconstructed reference frames that have common content with reconstructed reference frames obtained by a remote video decoder (without transmission errors).

[0046] The predictor (535) may perform a predictive search for the coding engine (532). That is, for a new frame to be coded, the predictor (535) may search the reference picture memory (534) to find sample data (as candidate reference pixel blocks) or predefined metadata, such as reference picture motion vectors, block shapes, etc., that can be used as suitable prediction references for the new picture. The predictor (535) may operate in a pixel block-to-sample block manner to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (535), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (534).

[0047] The controller (550) may manage the encoding operations of the video coder (530), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0048] The output of all the above functional elements may be subjected to entropy coding in an entropy coder (545), which converts the symbols produced by the various functional elements into a coded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art, for example, Huffman coding, variable length coding, arithmetic coding, etc.

[0049] The sender (540) may buffer the coded video sequence produced by the entropy coder (545) and prepare the coded video sequence for transmission over a communication channel (560), which may be a hardware or software link to a storage device capable of storing the coded video data. The sender (540) may merge the coded video data from the video coder (530) with other data to be transmitted, such as coded audio data and / or ancillary data streams (sources not shown).

[0050] A controller (550) may manage the operation of the encoder (303). During encoding, the controller (550) may assign each coded picture a predetermined coded picture type, which may affect the coding technique that may be applied to the respective picture. For example, pictures may often be assigned as one of the following frame types:

[0051] An intra-picture (I-picture) may be one that can be coded and decoded without using any other frame in a sequence as a source of prediction. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh Pictures. Those skilled in the art will recognize such variations of I-pictures and their corresponding uses and features.

[0052] A predicted picture (P-picture) may be one that can be coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0053] Bidirectionally predicted pictures (B-pictures) may be those that can be coded and decoded using intra- or inter-prediction, which uses at most two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multi-predicted pictures can use more than two reference pictures and associated metadata for the reconstruction of a block.

[0054] In general, multiple source pictures may be spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and the blocks may be coded one by one. Blocks may be coded predictively with reference to other (already coded) blocks as determined by the coding assignment applied to the picture to which the block corresponds. For example, blocks of an I-picture may be coded without prediction, or with prediction with reference to already coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P-picture may be coded without spatial or temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be coded without spatial or temporal prediction with reference to one or two previously coded reference pictures.

[0055] The video coder (303) may perform encoding operations according to a prescribed video coding technique or standard, such as ITU-T Recommendation H265. In doing so, the video coder (303) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancies in the input video sequence. Thus, the encoded video data may conform to a syntax defined by the video coding technique or standard being used.

[0056] In one embodiment, the transmitter (540) may transmit additional data along with the coded video. The video coder (530) may include such data as part of the coded video sequence. The additional data may comprise other forms of redundant data, such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0057] FIG. 6A is a diagram of intra prediction modes of VTM 1.

[0058] Referring to Figure 6A, an intra-prediction scheme based on 65 directional modes has been proposed to capture any edge direction shown in natural video, and this intra-prediction scheme based on 65 directional modes has been studied for the development of VVC. In VTM 1, mode 2 and mode 66 indicate the same prediction direction.

[0059] For non-square blocks, some traditional angular intra-prediction modes may be changed to wide-angle modes. The changed modes may be signaled using the original method and remapped to wide-angle mode indices after parsing. The total number of intra-prediction modes may remain unchanged (i.e., 35 as in VTM 1), and the intra-mode coding step may remain unchanged.

[0060] For the 35 intra prediction modes, the modified intra prediction modes may be those shown in Table 1 below, where W denotes block width and H denotes block height.

[0061] [Table 4]

[0062] For the 67 intra prediction modes, the modified intra prediction modes may be those shown in Table 2 below, where W denotes block width and H denotes block height.

[0063] [Table 5]

[0064] FIG. 6B is a diagram showing how the intra prediction mode for a non-square block is changed to a wide-angle intra prediction mode.

[0065] Referring to Figure 6B, Mode 2 (601) and Mode 3 (602) are changed to wide angle Mode 35 (603) and Mode 36 (604), whose direction is opposite that of Mode 3 (602), and whose direction is opposite that of Mode 4 (605).

[0066] However, in VTM 1, when wide-angle intra-prediction directions are applied, the neighboring reference samples in the upper row or left column are not sufficient for non-square blocks.

[0067] The embodiments herein may be used separately or combined in any order. In this description, if the width is greater than or equal to the height, the top row is referred to as the long side and the left column is referred to as the short side. Otherwise, the top row is referred to as the short side and the left column is referred to as the long side. The block width is denoted by nWidth and the block height is denoted by nHeight.

[0068] When padding an array or buffer of N reference samples, this means filling the reference sample values ​​with adjacent reconstructed samples located at the relevant positions of the reference samples, or copying the reference sample values ​​from already filled reference samples, or determining the reference sample values ​​from already filled reference samples using a prescribed function (e.g., linear extrapolation).

[0069] Embodiments may include padding to different numbers of reference samples depending on coding information including, but not limited to, block size, block width, block height, block width to height ratio, block area size, and intra-prediction direction.

[0070] In one embodiment, we propose padding to 2*max(nWidth, nHeight)+1 samples for both the above and left neighbor reference samples.

[0071] Another embodiment proposes padding to 2*max(nWidth,nHeight)+1 samples for the long side and nWidth+nHeight+1 samples for the short side.

[0072] Another embodiment proposes padding to 2*nWidth+3 samples for the top reference sample and 2*nHeight+3 samples for the left reference sample.

[0073] In another embodiment, if width / height<=4 or height / width<=4, we propose to pad to 2*nWidth+3 samples for the top reference sample and 2*nHeight+3 samples for the left reference sample. Otherwise, pad to 2*max(nWidth,nHeight)+1 samples for the long side and 2*min(nWidth,nHeight)+1+M samples for the short side, where M can be 10, 11, 12, 13 or 14.

[0074] In another embodiment, we propose padding to 2*nWidth+M samples for the top reference sample and 2*nHeight+N samples for the left reference sample, where the values ​​of M and N may depend on coding information, including but not limited to block size, block width, block height, block width-to-height ratio, block area size, and intra-prediction direction.

[0075] An embodiment may include removing some conventional angles on the short sides and adding an equal number of wider angles on the long sides. After removing the angles on the short sides, all remaining angles are within the diagonal direction of each block shape, where the diagonal direction is indicated by the line connecting the upper right corner and the lower left corner.

[0076] FIG. 7A is a diagram of predicted angles within and beyond the diagonal, according to one embodiment.

[0077] Referring to FIG. 7A, the prediction angle (710) (solid arrow) located within the filled triangular region (720) in the upper left corner is an intra-prediction angle within the diagonal direction, and the prediction angle (730) (dotted angle) located outside the filled triangular region (720) in the upper left corner is an intra-prediction angle beyond the diagonal direction.

[0078] In one embodiment, for the case of 35 intra modes, some deletion modes are shown in Table 3, where W denotes block width and H denotes block height.

[0079] [Table 6]

[0080] In one embodiment, for 67 intra modes, some deletion modes are shown in Table 4, where W denotes block width and H denotes block height.

[0081] [Table 7]

[0082] An embodiment may include removing some conventional angles on the short side and adding an equal number of wide angles on the long side. After removing the angles on the short side, all remaining angles are within the diagonal angles of each block shape except for N predicted angles. Example values ​​of N include, but are not limited to, 1, 2, 3, and 4. The N predicted angles are then further adjusted to be within the diagonal angles.

[0083] FIG. 7B illustrates adjusting prediction angles beyond the diagonal, according to one embodiment.

[0084] Referring to FIG. 7B, the predicted angle (730) (dotted arrow) beyond the diagonal is adjusted to a predicted angle (740) (thick arrow) within the diagonal (within the upper left solid triangular region (720)).

[0085] When adjusting one or more predicted angles, the angles already within the diagonal direction are also further adjusted to even out the distribution of all available predicted angles, e.g., the predicted angles become evenly distributed.

[0086] Instead of restricting the prediction angles to be within the diagonal direction, as described above, embodiments may include restricting the prediction angles to utilize only reference samples within a specified range (or group or number) of reference samples for the available prediction angles.

[0087] In one embodiment, a restriction is made that only the topmost m*nWidth+1+offsetX (including the top-left corner) reference samples can be used for intra prediction, and intra-prediction angles that may use reference samples other than the topmost n*nWidth+1+offsetX samples are not allowed. Example values ​​of offsetX may include, but are not limited to, 1, 2, 3, 4,..., 14,... and 48. Example values ​​of m and n may include, but are not limited to, 1, 2, 3, and 4.

[0088] In one embodiment, a restriction is made that only the topmost m*nHeight+1+offsetY (including the top-left corner) reference samples can be used for intra prediction, and intra-prediction angles that may use reference samples other than the topmost n*nHeight+1+offsetY samples are not allowed. Example values ​​of offsetY may include, but are not limited to, 1, 2, 3, 4,..., 14,... and 48. Example values ​​of m and n may include, but are not limited to, 1, 2, 3, and 4.

[0089] In one embodiment, a restriction is made that only m*max(nHeight, nWidth)+1+offset (including the top-left corner) reference samples can be used for intra prediction, and intra prediction angles that may use reference samples other than n*max(nHeight, nWidth)+1+offset samples are not allowed. Example values ​​of offset may include, but are not limited to, 1, 2, 3, 4,..., 14,... and 48. Example values ​​of m and n may include, but are not limited to, 1, 2, 3, and 4.

[0090] In one embodiment, a restriction is made that only m*min(nHeight, nWidth)+1+offset (including the top-left corner) reference samples can be used for intra prediction, and intra prediction angles that may use reference samples other than n*min(nHeight, nWidth)+1+offset samples are not allowed. Example values ​​of offset may include, but are not limited to, 1, 2, 3, 4,..., 14,... and 48. Example values ​​of m and n may include, but are not limited to, 1, 2, 3, and 4.

[0091] Embodiments may include removing three, five, six, seven or eight modes on the short side when the width / height ratio is 2 (or 1 / 2), 4 (or 1 / 4), 8 (or 1 / 8), 16 (or 1 / 16) or 32 (1 / 32) for the 35 intra prediction modes, and removing six, ten, twelve, fourteen or sixteen modes on the short side when the width / height ratio is 2 (or 1 / 2), 4 (or 1 / 4), 8 (or 1 / 8), 16 (or 1 / 16) or 32 (1 / 32) for the 67 intra prediction modes.

[0092] In one embodiment, for the remaining modes of the short side, if the angle of this mode exceeds the diagonal direction of this block, this angle will be mapped to the diagonal direction of this block.For example, in the case of 35 intra-prediction modes, when width / height=2 (or 1 / 2), and the angle of mode 5 (or mode 31) is 17 / 32, and it exceeds the diagonal direction of this block, the angle of mode 5 (or mode 31) will be mapped to 16 / 32; when width / height=4 (or 1 / 4), and the angle of mode 7 (or mode 29) is 9 / 32, and it exceeds the diagonal direction of this block, the angle of mode 7 (or mode 29) will be mapped to 8 / 32; when width / height=8 (or 1 / 8), and the angle of mode 8 (or mode 28) is 5 / 32, and it exceeds the diagonal direction of this block, the angle of mode 8 (or mode 28) will be mapped to 4 / 32.

[0093] In another example, in the case of 67 intra prediction modes, when width / height = 2 (or 1 / 2), and the angle of mode 8 (or mode 60) is 17 / 32 and crosses the diagonal direction of the block, the angle of mode 8 (or mode 60) is mapped to 16 / 32; when width / height = 4 (or 1 / 4), and the angle of mode 12 (or mode 56) is 9 / 32 and crosses the diagonal direction of the block, the angle of mode 12 (or mode 56) is mapped to 8 / 32; and when width / height = 8 (or 1 / 8), and the angle of mode 14 (or mode 54) is 5 / 32 and crosses the diagonal direction of the block, the angle of mode 14 (or mode 54) is mapped to 4 / 32.

[0094] Embodiments may include modifying or setting the angles of intra prediction modes such that the angles include the diagonal directions of all block shapes.

[0095] In one embodiment, it is proposed that the angle table contain angles α such that tan(α) is equal to {1 / 32, 2 / 32, 4 / 32, 8 / 32, 16 / 32, 32 / 32}. In another embodiment, it is proposed that the angle table contain angles α such that tan(α) is equal to {1 / 64, 2 / 64, 4 / 64, 8 / 64, 16 / 64, 32 / 64, 64 / 64} for a predicted angle accuracy increase from 1 / 32 samples to 1 / 64 samples. In another embodiment, it is proposed that the angle table contain angles α such that tan(α) is equal to {1 / 128, 2 / 128, 4 / 128, 8 / 128, 16 / 128, 32 / 128, 64 / 128, 128 / 128} for a predicted angle accuracy increase from 1 / 32 samples to 1 / 128 samples.

[0096] In another embodiment, for 35 intra prediction modes, the angle tables may be as represented by Tables 5 and 6 below.

[0097] [Table 8]

[0098] [Table 9]

[0099] In another embodiment, for 67 intra prediction modes, the angle table may be as represented by Table 7 below.

[0100] [Table 10]

[0101] In the above Tables 5 to 7, intraPredAngle indicates the intra prediction angle, predModeIntra indicates the intra prediction mode, and invAngle indicates the inverse angle.

[0102] Figure 8 is a flow chart illustrating a method (800) for controlling intra prediction for decoding a video sequence, according to one embodiment. In some implementations, one or more of the process blocks of Figure 8 may be performed by the decoder (310). In some implementations, one or more of the process blocks of Figure 8 may be performed by another device or group of devices that are separate from or include the decoder (310), such as the encoder (303).

[0103] Referring to FIG. 8, in a first block (810), the method (800) includes setting a table including a plurality of intra prediction modes each corresponding to a plurality of intra prediction angles, the plurality of intra prediction angles including diagonal intra prediction angles each corresponding to a diagonal direction of all shapes of the coding unit.

[0104] Each of the diagonal intra-prediction angles can be denoted by α, where tan(α) is equal to {1 / 32, 2 / 32, 4 / 32, 8 / 32, 16 / 32, 32 / 32}.

[0105] The number of intra prediction modes included in the set table may be 67, and the set table may include Table 7 above.

[0106] Each of the diagonal intra-prediction angles can be denoted by α, where tan(α) is equal to {1 / 64, 2 / 64, 4 / 64, 8 / 64, 16 / 64, 32 / 64, 64 / 64}.

[0107] Each of the diagonal intra-prediction angles can be denoted by α, where tan(α) is equal to {1 / 128, 2 / 128, 4 / 128, 8 / 128, 16 / 128, 32 / 128, 64 / 128, 128 / 128}.

[0108] The number of intra prediction modes included in the set table may be 35, and the set table may include Table 5 above.

[0109] In a second block (820), the method (800) includes selecting one of a plurality of intra-prediction modes from the configured table for decoding the video sequence.

[0110] The method (800) may further include determining a ratio of width to height of the coding unit and removing a first number of intra prediction modes from the configured table based on the determined ratio, where the first number corresponds to the determined ratio and the shorter of the width and height. Selecting one of the intra prediction modes may include selecting one of the intra prediction modes from the table from which the first number of intra prediction modes have been removed for decoding the video sequence.

[0111] Removing the first number of intra prediction modes may include removing the first number of intra prediction modes from the set table based on a determined ratio of 4, 8, 16, or 32 based on a second number of intra prediction modes included in the table, which is 35, where the first number is 5, 6, 7, or 8, respectively, and removing the first number of intra prediction modes from the set table based on a determined ratio of 1 / 4, 1 / 8, 1 / 16, or 1 / 32, where the first number is 5, 6, 7, or 8, respectively.

[0112] Removing the first number of intra prediction modes may include removing the first number of intra prediction modes from the set table based on a determined ratio of 4, 8, 16, or 32 based on a second number of intra prediction modes included in the table, which is 67, where the first number is 10, 12, 14, or 16, respectively, and removing the first number of intra prediction modes from the set table based on a determined ratio of 1 / 4, 1 / 8, 1 / 16, or 1 / 32, where the first number is 10, 12, 14, or 16, respectively.

[0113] The method may further include, for each of a plurality of intra prediction modes included in the table from which the first number of intra prediction modes have been removed, mapping an intra prediction angle associated with that intra prediction mode from among the plurality of intra prediction angles to a diagonal direction based on an intra prediction angle associated with that intra prediction mode from among the plurality of intra prediction angles that is greater than a diagonal direction of the coding unit.

[0114] Although Figure 8 illustrates example blocks of method 800, in some implementations, method 800 may include additional blocks, fewer blocks, different blocks, or blocks arranged differently than those shown in Figure 8. Additionally or alternatively, two or more of the blocks of method 800 may be performed in parallel.

[0115] Additionally, the proposed methods may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transitory computer-readable medium to perform one or more of the proposed methods.

[0116] FIG. 9 is a simplified block diagram of an apparatus (900) for controlling intra prediction for decoding a video sequence according to one embodiment.

[0117] Referring to FIG. 9, the device (900) includes a setting code (910) and a selection code (920).

[0118] The setting code (910) is configured to set a table including a plurality of intra prediction modes respectively corresponding to a plurality of intra prediction angles, and the plurality of intra prediction angles include diagonal intra prediction angles respectively corresponding to the diagonal directions of all shapes of the coding unit.

[0119] Each of the diagonal intra-prediction angles can be denoted by α, where tan(α) is equal to {1 / 32, 2 / 32, 4 / 32, 8 / 32, 16 / 32, 32 / 32}.

[0120] The number of intra prediction modes included in the set table may be 67, and the set table may include Table 7 above.

[0121] Each of the diagonal intra-prediction angles can be denoted by α, where tan(α) is equal to {1 / 64, 2 / 64, 4 / 64, 8 / 64, 16 / 64, 32 / 64, 64 / 64}.

[0122] Each of the diagonal intra-prediction angles can be denoted by α, where tan(α) is equal to {1 / 128, 2 / 128, 4 / 128, 8 / 128, 16 / 128, 32 / 128, 64 / 128, 128 / 128}.

[0123] The number of intra prediction modes included in the set table may be 35, and the set table may include Table 5 above.

[0124] The selection code (920) is configured to select one of a plurality of intra-prediction modes from the established table for decoding the video sequence.

[0125] The apparatus (900) may further include a determination code (930) configured to determine a ratio of width to height of the coding unit, and a deletion code (940) configured to delete a first number of intra-prediction modes from the configured table based on the determined ratio, where the first number corresponds to the determined ratio and the shorter of the width and height. The selection code (920) may be further configured to cause the at least one processor to select one of the intra-prediction modes from the table from which the first number of intra-prediction modes have been deleted, for decoding the video sequence.

[0126] The deletion code (930) may be further configured to delete a first number of intra prediction modes from the configured table based on a determined ratio of 4, 8, 16, or 32 based on a second number of intra prediction modes included in the table, which is 35, where the first number is 5, 6, 7, or 8, respectively, and to delete a first number of intra prediction modes from the configured table based on a determined ratio of 1 / 4, 1 / 8, 1 / 16, or 1 / 32, where the first number is 5, 6, 7, or 8, respectively.

[0127] The deletion code (930) may be further configured to: delete a first number of intra prediction modes from the configured table based on a determined ratio of 4, 8, 16, or 32 based on a second number of intra prediction modes included in the table, which is 67, where the first number is 10, 12, 14, or 16, respectively; and delete a first number of intra prediction modes from the configured table based on a determined ratio of 1 / 4, 1 / 8, 1 / 16, or 1 / 32, where the first number is 10, 12, 14, or 16, respectively.

[0128] The apparatus (900) may further include a mapping code (950) configured to, for each of a plurality of intra prediction modes included in the table from which the first number of intra prediction modes have been deleted, map the intra prediction angle associated with that intra prediction mode from among the plurality of intra prediction angles to a diagonal direction based on the intra prediction angle associated with that intra prediction mode from among the plurality of intra prediction angles that is greater than a diagonal direction of the coding unit.

[0129] The techniques described above can be implemented using computer readable instructions and as computer software physically stored on one or more computer readable media.

[0130] FIG. 10 is a diagram of a computer system (1000) suitable for implementing embodiments.

[0131] Computer software may be coded using any suitable machine code or computer language that may be assembled, compiled, linked, or similar mechanisms to produce code comprising instructions that may be executed directly, interpreted, executed through microcode execution, or the like, by a computer central processing unit (CPU), graphics processing unit (GPU), or the like.

[0132] The instructions may be executed on or components of various types of computers including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0133] 10 for computer system 1000 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments. Neither should the arrangement of components be interpreted as having any dependency or requirement regarding any one or combination of components shown in the exemplary embodiment of computer system 1000.

[0134] The computer system (1000) may include several human interface input devices. Such human interface input devices may respond to input by one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). The human interface devices may also be used to capture media that does not necessarily involve direct conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video, including stereoscopic video).

[0135] The input human interface devices may include one or more (only one of each is shown) of a keyboard (1001), a mouse (1002), a trackpad (1003), a touch screen (1010), a data glove (1004), a joystick (1005), a microphone (1006), a scanner (1007), and a camera (1008).

[0136] The computer system (1000) may also include several human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses through, for example, tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., touch screens (1010), haptic feedback via data gloves (1004) or joysticks (1005), although some haptic feedback devices do not function as input devices), audio output devices (such as speakers (1009), headphones (not shown)), visual output devices (such as screens (1010), including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, and organic light emitting diode (OLED) screens), each with or without touch screen input capability, each with or without haptic feedback capability, some of which may be capable of outputting two-dimensional visual output or output in more than three dimensions through means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, smoke tanks (not shown), and printers (not shown).

[0137] The computer system (1000) may also include human-accessible storage devices and associated media, such as optical media including CD / DVD ROM / RW (1020), including media such as CD / DVD (1021), thumb-drives (1022), removable hard drives or solid-state drives (1023), legacy magnetic media (not shown) such as tape and floppy disks, and dedicated ROM / ASIC / PLD-based devices (not shown) such as security dongles.

[0138] Those skilled in the art will also appreciate that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not encompass transmission media, carrier waves, or other transitory signals.

[0139] The computer system 1000 may also include interfaces to one or more communications networks. For example, the networks may be wireless, wired, or optical. Furthermore, the networks may be local, wide-area, urban, vehicular, industrial, real-time, delay-tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, global systems for mobile communications (GSM), cellular networks including third-generation (3G), fourth-generation (4G), fifth-generation (5G), Long-Term Evolution (LTE), etc., TV wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial networks including CANBus, etc. Generally, some networks require an external network interface adapter that attaches to some general-purpose data port or peripheral bus (1049) (e.g., a Universal Serial Bus (USB) port on the computer system 1000), while others, as described below, are generally integrated into the core of the computer system 1000 by attaching to the system bus (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system 1000 can communicate with others. Such communication can be one-way, receive-only (e.g., television broadcasts), one-way transmit-only (e.g., a device transmitting from a CAN bus to a specific CAN bus), or bidirectional (e.g., a device transmitting from a CAN bus to another computer system using a local or wide-area digital network). Several protocols and protocol stacks can be used for each of the networks and network interfaces described above.

[0140] The above-mentioned human interface devices, human-accessible storage devices and network interfaces may be attached to the core (1040) of the computer system (1000).

[0141] The core (1040) may include one or more central processing units (CPUs) (1041), graphics processing units (GPUs) (1042), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (1043), task-specific hardware accelerators (1044), etc. These devices may be connected through a system bus (1048) along with read-only memory (ROM) (1045), random access memory (RAM) (1046), and internal mass storage devices (1047) such as internal hard drives or solid-state drives (SSDs) that are not user accessible. In some computer systems, the system bus (1048) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus (1048) or through a peripheral bus (1049). Peripheral bus architectures include peripheral component interconnect (PCI), USB, and the like.

[0142] The CPU (1041), GPU (1042), FPGA (1043), and accelerator (1044) can execute predetermined instructions, which may combine to form the aforementioned computer code. This computer code may be stored in ROM (1045) or RAM (1046). Additionally, changing data may be stored in RAM (1046), while unchanging data may be stored, for example, in an internal mass storage device (1047). Cache memory, which may be closely associated with one or more of the CPU (1041), GPU (1042), mass storage device (1047), ROM (1045), RAM (1046), etc., may be used to enable fast storage and fast readout of any memory device.

[0143] Computer code for performing various computer-implemented operations may be provided on computer-readable media, and the media and computer code may be those specially designed and constructed for the embodiments, or they may be of the kind well known and available to those skilled in the computer software arts.

[0144] By way of example and not limitation, the architecture (1000), and in particular a computer system having a core (1040), may perform its functions as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be media associated with the user-accessible mass storage devices introduced above, as well as any storage of the core (1040) that is non-transitory in nature, such as the in-core mass storage device (1047) or ROM (1045). Software implementing various embodiments may be stored on such devices and executed by the core (1040). The computer-readable media may include one or more memory devices or chips, depending on specific requirements. Software may be used to cause the core (1040), and in particular the processor (including a CPU, GPU, FPGA, etc.) in the core (1040), to perform certain processes or portions of certain processes described herein, including defining data structures stored in RAM (1046) and modifying such data structures according to processes defined by the software. Additionally or alternatively, a computer system may perform functions as a result of logic hardwired or otherwise incorporated into circuitry (e.g., accelerator (1044)), which may operate in place of or cooperate with software to perform certain processes or portions of certain processes described herein. References to software may encompass references to logic, and vice versa (where applicable). References to computer-readable media may encompass references to circuitry (e.g., integrated circuits (ICs)) that stores software for execution, circuitry embodying logic for execution, or both (where applicable). Embodiments include any appropriate combination of hardware and software.

[0145] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are within its spirit and scope. [Explanation of symbols]

[0146] 101 Mode 10 102 Mode 26 103 Mode 2 104 Mode 18 105 Mode 34 200 Communication Systems 210 First Terminal 220 Second Terminal 250 Communication Network 301 Video Source 302 Video Sample Stream 303 Video Encoder 304 Encoded Video Bitstream 305 Streaming Server 306 Client 307 Encoded Video Bitstream 310 Video Decoder 311 Output video sample stream 312 Drawing Devices 313 Imaging Subsystem 410 Receiver 412 channels 415 Buffer Memory 420 Parser 421 Symbol 451 Scaler / Inverse Converter 452 Intra-picture prediction unit 452 Intra Prediction Unit 453 Motion Compensation Prediction Unit 454 Loop filter section 455 Aggregator 456 Current Picture 457 Reference Picture Buffer 530 Source Coder 532 encoding engine 533 decoder 534 Reference Picture Memory 535 Predictor 540 Transmitter 543 coded video sequence 545 Entropy Coder 550 Controller 560 Communication Channels 601 Mode 2 602 Mode 3 603 Mode 35 604 Mode 36 605 Mode 4 710 Prediction Angle 720 triangle area 730 Prediction Angle 740 Prediction Angle 800 ways 810 Block 820 Block 900 equipment 910 setting code 920 selection code 930 Removal Code 950 Mapping Code 1000 Computer Systems 1001 keyboard 1002 Mouse 1003 Trackpad 1004 Data Gloves 1005 Joystick 1006 Mike 1007 Scanner 1008 Camera 1009 Speaker 1010 Touch Screen 1010 screen 1020 CD / DVD ROM / RW 1021 CD / DVD 1022 thumb-drive 1023 Solid State Drive 1040 cores 1041 CPU 1042 GPU 1043 FPGA 1044 Accelerator 1045 ROM 1046 RAM 1047 Internal mass storage 1048 System Bus 1049 Peripheral Bus

Claims

[Claim 1] 1. A method for controlling intra prediction for decoding a video sequence, the method being performed by at least one processor, the method comprising: setting a table including a plurality of intra prediction angles and a plurality of intra prediction modes corresponding to the plurality of intra prediction angles, the plurality of intra prediction angles including diagonal intra prediction angles corresponding to diagonal directions of a plurality of different rectangular coding units having different width to height ratios; selecting one of the plurality of intra-prediction modes from the configured table for decoding the video sequence; A method for providing

Images