Method, apparatus, and computer program for entropy coding for intra-predictive mode

Abstract

The present invention relates to a method, apparatus, and storage medium for entropy coding for intra-predictive modes in encoding and / or decoding. [Solution] The method includes the steps of receiving an encoded video bitstream relating to a block, and deriving at least one context relating to the block's set index and mode index based on information about the block and mode information of at least one adjacent block of the block. The set index indicates a plurality of intra-mode sets, and the mode index indicates an intra-predicted mode. The method also includes the steps of extracting the set index and mode index from the encoded video bitstream according to at least one context, and determining the intra-predicted mode of the block based on the set index and mode index.

Description

Technical Field

[0001] This application is based on U.S. Provisional Application No. 63 / 219,230, filed Jul. 7, 2021, and U.S. Application No. 17 / 573,845, filed Jan. 12, 2022, and claims the benefit of priority thereto, and both are hereby incorporated by reference in their entireties.

[0002] The present disclosure relates to video encoding and / or decoding techniques, and more particularly, to an improved design and signaling of entropy coding for intra prediction modes.

Background Art

[0003] The background description presented herein is for the purpose of generally presenting the context of the disclosure. The work of the inventors named herein, in the context of the background section described, and aspects of the description that may not otherwise qualify as prior art at the time of filing of this application, are not, expressly or impliedly, admitted as prior art to the present disclosure.

[0004] Video encoding and decoding can be performed using interpicture prediction with motion compensation. Uncompressed digital video consists of a series of pictures, each picture having spatial dimensions of, for example, 1920 x 1080 luminance samples and associated fully sampled or subsampled chrominance samples. The series of pictures can have a fixed or variable picture rate (or frame rate), for example, 60 pictures per second, i.e., a picture rate of 60 frames per second. Uncompressed video has specific bitrate requirements for streaming or data processing. For example, video with 8 bits per pixel per color channel, a pixel resolution of 1920 x 1080, a frame rate of 60 frames / second, and 4:2:0 chroma subsampling requires a bandwidth of nearly 1.5 Gbit / s. One hour of such video requires more than 600 GByte of storage space.

[0005] One objective of video encoding and decoding may be to reduce the redundancy of an uncompressed input video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage space requirements by orders of magnitude or more. Both lossless and lossy compression, as well as combinations thereof, can be used. Lossless compression refers to a technique in which the decoding process can reconstruct an exact copy of the original signal from the compressed original signal. Lossy compression refers to an encoding / decoding process in which the original video information is not fully preserved during coding and is not fully recoverable during decoding. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough to make the reconstructed signal useful for its intended purpose, despite some loss of information. In the case of video, lossy compression is widely used in many applications. The acceptable amount of distortion depends on the application. For example, users of certain consumer video streaming applications may tolerate higher distortion than users of movie or television broadcast applications. The compression ratio achievable by a particular coding algorithm can be selected or adjusted to reflect various strain tolerances, and higher acceptable strains generally allow for coding algorithms that produce higher losses and higher compression ratios.

[0006] Video encoders and decoders can utilize techniques from several broad categories and steps, including, for example, motion compensation, Fourier transform, quantization, and entropy coding.

[0007] Video codec technology may include a technique known as intra-coding. In intra-coding, sample values ​​are represented without referencing samples or other data from a previously reconstructed reference picture. In some video codecs, a picture is spatially subdivided into multiple blocks of samples. If all block samples are coded in intra-mode, the picture can be called an intra-picture. Intra-pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and thus can be used as the first picture in the video bitstream and video session being coded, or as a still image. The intra-predicted block samples can then be subjected to a conversion to the frequency domain, and the resulting conversion coefficients can be quantized before entropy coding. Intra-prediction represents a technique that minimizes the sample values ​​in the pre-conversion domain. In some cases, the smaller the converted DC value and the smaller the AC coefficient, the fewer bits are required to represent the block after entropy coding with a given quantization step size.

[0008] Traditional intra-coding, such as that known from MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques include methods that attempt to encode / decode blocks based on surrounding sample data and / or metadata obtained during encoding and / or decoding of blocks of data that are spatially adjacent to and preceding the data being encoded or decoded. Such techniques will hereafter be referred to as “intra-prediction” techniques. In at least some cases, intra-prediction uses only reference data from the current picture being reconstructed, and not from other reference pictures.

[0009] Numerous different forms of intra-prediction can exist. In a given video coding technique, if two or more such techniques are available, the techniques used can be referred to as intra-prediction modes. One or more intra-prediction modes may be provided in a particular codec. In a particular case, a mode may have submodes and / or be associated with various parameters, and the mode / submode information and intra-coding parameters of a block of video may be coded individually or collectively included in a mode codeword. The choice of which codeword to use for a given combination of mode, submode, and / or parameters can affect the coding efficiency gain through intra-prediction, and the entropy coding technique used to convert the codeword into a bitstream may also be affected.

[0010] Certain modes of intra-prediction were introduced in H.264, improved in H.265, and further refined with newer coding techniques such as Joint Exploration Models (JEM), Versatile Video Coding (VVC), and Benchmark Sets (BMS). Generally, intra-prediction can form predictor blocks using available neighboring sample values. For example, values ​​from a specific set of available neighboring samples along a particular direction and / or line may be copied into the predictor block. The reference for the direction to use may be coded into the bitstream or predicted itself.

[0011] Referring to Figure 1A, the lower right shows a subset of nine predictor directions defined by H.265's 33 possible predictor directions (corresponding to 33 of the 35 intra-modes defined in H.265). The point where the arrows converge (101) represents the sample being predicted. The arrows indicate that adjacent samples from that direction are used to predict the sample at position 101. For example, arrow (102) indicates that sample (101) is predicted from one or more adjacent samples to the upper right at a 45-degree angle from the horizontal. Similarly, arrow (103) indicates that sample (101) is predicted from one or more adjacent samples to the lower left at a 22.5-degree angle from the horizontal.

[0012] Referring again to Figure 1A, a 4x4 sample square block (104) (shown by a thick dashed line) is depicted in the upper left. The square block (104) contains 16 samples, each labeled with "S" and its position in the Y dimension (e.g., row index) and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample (from the top) in the Y dimension and the first sample (from the left) in the X dimension. Similarly, sample S44 is the fourth sample in both the Y and X dimensions within block (104). Since this block is 4x4 sample size, S44 is in the lower right. Furthermore, an example of a reference sample following a similar numbering scheme is shown. The reference sample is labeled with R and its Y position (e.g., row index) and X position (column index) relative to block (104). In both H.264 and H.265, a predicted sample adjacent to the block being reconstructed is used.

[0013] Intra-picture prediction in block 104 may begin by appropriately copying reference sample values ​​from adjacent samples according to the signaled prediction direction. For example, suppose the encoded video bitstream contains signaling for block 104 that points to the prediction direction of arrow (102), i.e., that a sample is predicted from one or more prediction samples or samples 45 degrees to the upper right from the horizontal direction. In such a case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05, and sample S44 is predicted from reference sample R08.

[0014] In certain cases, particularly when the direction cannot be divided equally into 45-degree intervals, the values ​​of multiple reference samples may be combined, for example, by interpolation, in order to calculate a reference sample.

[0015] As video coding technology continues to advance, the number of possible directions is increasing. In H.264 (2003), for example, nine different directions are available for intra-prediction. This increased to 33 in H.265 (2013), and JEM / VVC / BMS can support up to 65 directions as of this disclosure. Empirical studies are being conducted to help identify the most favorable intra-prediction directions, and these most favorable directions can be encoded into a small number of bits, accepting a specific bit penalty for each direction, using certain techniques in entropy coding. Furthermore, these directions themselves may be predicted from the adjacent directions used in the intra-prediction of the decoded adjacent blocks.

[0016] Figure 1B shows a schematic diagram (180) illustrating 65 intra-prediction directions according to JEM, illustrating the increasing number of prediction directions in the various coding techniques developed.

[0017] The method of mapping bits representing intra-prediction directions to intra-prediction directions in an encoded video bitstream can vary depending on the video coding technique, ranging from simple direct mapping of prediction directions to intra-prediction modes to complex adaptive schemes involving codewords, most probable modes, and similar techniques. However, in any case, video content may have certain directions that are statistically less likely to occur than other particular directions with respect to intra-prediction. Since the goal of video compression is to reduce redundancy, in a well-designed video coding technique, these less likely directions may be represented by more bits than the more likely directions.

[0018] Interpicture prediction, or interpretation, can be based on motion compensation. In motion compensation, sample data from a previously reconstructed picture or part thereof (reference picture) can be used to predict a newly reconstructed picture or part of a picture (e.g., a block) after being spatially shifted in the direction indicated by a motion vector (MV). In some cases, the reference picture can be the same as the picture currently being reconstructed. The MV can have two dimensions, X and Y, or three dimensions, with a third dimension (similar to the time dimension) indicating the reference picture to be used.

[0019] Some video compression techniques allow the current motion vector (MV) applicable to a particular region of sample data to be predicted from other MVs, for example, from other MVs related to sample data in other regions that are spatially adjacent to the region being reconstructed and precede the current MV in decoding order. Doing so can significantly reduce the overall amount of data required to code the MV by relying on removing redundancy in correlated MVs, thereby improving compression efficiency. MV prediction can work effectively because, for example, when coding an input video signal originating from a camera (known as natural video), there is a statistical likelihood that regions larger than the region to which a single MV is applicable will move in a similar direction within the video sequence and therefore, in some cases, can be predicted using similar motion vectors derived from the MVs of adjacent regions. This results in the actual MV for a given region being similar to or identical to the MV predicted from the surrounding MVs. Such an MV can then be represented with fewer bits than would be used if the MV were coded directly after entropy coding rather than being predicted from (one or more) adjacent MVs. In some cases, MV prediction can be an example of lossless compression of the signal (i.e., MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself may be lossy, for example, due to rounding errors when calculating the predictor from several surrounding MVs.

[0020] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Recommendation H.265, “High Efficiency Video Coding,” December 2016). Of the many MV prediction mechanisms specified in H.265, the technique referred to as “spatial merging” will be explained below.

[0021] Specifically, referring to Figure 2, the current block (201) has samples found by the encoder during the motion search process that are predictable from a spatially shifted, preceding block of the same size. Instead of directly coding the MV, the MV can be derived from metadata associated with one or more reference pictures, such as from the immediately preceding reference picture (in decoding order), using the MV associated with one of five surrounding samples denoted as A0, A1, and B0, B1, B2 (202 through 206, respectively). In H.265, the MV prediction can use predictors from the same reference pictures used by the adjacent block. [Overview of the project]

[0022] This disclosure describes various embodiments of methods, apparatus, and computer-readable storage media for encoding and / or decoding video.

[0023] According to one aspect, one embodiment of the present disclosure provides a method for entropy coding for an intra-predictive mode in video decoding. The method includes the step of an apparatus receiving an encoded video bitstream relating to a block. The apparatus includes a memory for storing instructions and a processor communicating with the memory. The method also includes the step of the apparatus deriving at least one context relating to a set index and a mode index of a block based on information of the block and mode information of at least one adjacent block of the block, wherein the set index indicates an intra-mode set from a plurality of intra-mode sets and the mode index indicates an intra-predictive mode from the intra-mode set; the step of the apparatus extracting the set index and the mode index from the encoded video bitstream according to the at least one context; and the step of the apparatus determining the intra-predictive mode of the block based on the set index and the mode index.

[0024] According to another aspect, an embodiment of the present disclosure provides a method for entropy coding for an intra prediction mode in video coding. The method includes the step of receiving, by a device, a block to be encoded in an encoded video bitstream. The device includes a memory for storing instructions and a processor communicating with the memory. The method also includes the step of deriving, by the device, at least one context regarding a set index and a mode index of the block based on at least one of information of the block and mode information of at least one adjacent block of the block, wherein the set index indicates an intra mode set from a plurality of intra mode sets, and the mode index indicates an intra prediction mode from the intra mode set; and the step of encoding, by the device, the set index and the mode index into the encoded video bitstream according to the at least one context.

[0025] According to another aspect, an embodiment of the present disclosure provides an apparatus for video encoding and / or decoding. The apparatus includes a memory for storing instructions and a processor communicating with the memory. When the processor executes the instructions, the processor is configured to cause the apparatus to execute the above-described method for video decoding and / or encoding.

[0026] In another aspect, an embodiment of the present disclosure provides a non-transitory computer-readable medium storing instructions, which, when executed by a computer for video decoding and / or encoding, cause the computer to execute the above-described method for video decoding and / or encoding.

[0027] The above and other aspects and their implementations will be described in more detail in the drawings, the specification, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Further features, properties, and various advantages of the matters related to the disclosure will become even more apparent from the following detailed description and the accompanying drawings. [Figure 1A] A schematic diagram of an exemplary subset of the intra prediction direction mode is shown. [Figure 1B] An exemplary intra prediction direction diagram is shown. [Figure 2] A schematic diagram of spatial merge candidates for motion vector prediction in an example of the current block and its surroundings is shown. [Figure 3] A schematic diagram of a simplified block diagram of a communication system (300) according to an exemplary embodiment is shown. [Figure 4] A schematic diagram of a simplified block diagram of a communication system (400) according to an exemplary embodiment is shown. [Figure 5] A schematic diagram of a simplified block diagram of a video decoder according to an exemplary embodiment is shown. [Figure 6] A schematic diagram of a simplified block diagram of a video encoder according to an exemplary embodiment is shown. [Figure 7] A block diagram of a video encoder according to another exemplary embodiment is shown. [Figure 8] A block diagram of a video decoder according to another exemplary embodiment is shown. [Figure 9] The directional intra prediction mode according to an exemplary embodiment of the present disclosure is shown. [Figure 10] The non-directional intra prediction mode according to an exemplary embodiment of the present disclosure is shown. [Figure 11] The recursive intra prediction mode according to an exemplary embodiment of the present disclosure is shown. [Figure 12] A flowchart of a method according to an exemplary embodiment of the present disclosure is shown. [Figure 13] A flowchart of another method according to an exemplary embodiment of the present disclosure is shown. [Figure 14] A schematic diagram of a computer system according to an exemplary embodiment of the present disclosure is shown. [Modes for carrying out the invention]

[0029] Next, the present invention will be described in detail below with reference to the accompanying drawings, which constitute a part of the present invention and illustrate specific examples of embodiments. However, it should be noted that the present invention can be embodied in a variety of different forms, and therefore, the subject matter or claims are intended to be interpreted as not being limited to any of the embodiments described below. It should also be noted that the present invention can be embodied as a method, apparatus, component, or system. Accordingly, embodiments of the present invention may take the form of, for example, hardware, software, firmware, or any combination thereof.

[0030] Throughout the specification and claims, terms may have meanings implied or suggested in context beyond their expressly stated meanings. The phrases “in one embodiment” or “in some embodiments” used herein do not necessarily refer to the same embodiment, and the phrases “in another embodiment” or “in other embodiments” used herein do not necessarily refer to different embodiments. Similarly, the phrases “in one implementation” or “in some implementations” used herein do not necessarily refer to the same implementation, and the phrases “in another implementation” or “in other implementations” used herein do not necessarily refer to different implementations. The intention is, for example, that the claims include, in whole or in part, exemplary combinations of embodiments / implementations.

[0031] Generally, terms can be understood, at least partially, from their usage in context. For example, terms such as “and,” “or,” or “and / or” as used here can have a variety of meanings, at least partially depending on the context in which such terms are used. Typically, when “or” is used to relate a list such as A, B, or C, it is intended to mean A, B, and C, used here in an inclusive sense, and A, B, or C, used here in an exclusive sense. Similarly, the terms “one or more” or “at least one” as used here can be used, at least partially depending on the context, to describe some mechanism, structure, or characteristic in a singular sense, or to describe a combination of mechanisms, structures, or characteristics in a plural sense. Likewise, terms such as “a,” “an,” or “the” can be understood, at least partially depending on the context, to convey a singular usage, or to convey a plural usage. Furthermore, the terms “based on” or “determined by” are not necessarily intended to convey an exclusive set of factors, but rather can be understood to allow for the existence of further factors that are not necessarily explicitly described, also depending at least partially on the context.

[0032] Figure 3 illustrates a simplified block diagram of a communication system (300) according to one embodiment of the present disclosure. The communication system (300) includes a plurality of terminal devices that can communicate with each other, for example, via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). In the example of Figure 3, the first pair of terminal devices (310) and (320) can perform one-way transmission of data. For example, terminal device (310) may encode video data (for example, a stream of video pictures captured by terminal device (310)) for transmission to other terminal devices (320) via the network (350). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. Terminal device (320) may receive encoded video data from the network (350), decode the encoded video data to restore the video pictures, and display the video pictures according to the restored video data. One-way data transmission can be implemented in media service provision applications and similar systems.

[0033] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) for bidirectional transmission of encoded video data, which may be performed, for example, in a video conferencing application. In the bidirectional transmission of data, in one example, each terminal device of terminal devices (330) and (340) may encode video data (for example, a stream of video pictures captured by that terminal device) for transmission to the other terminal device of terminal devices (330) and (340) via the network (350). Each terminal device of terminal devices (330) and (340) may also receive encoded video data transmitted by the other terminal device of terminal devices (330) and (340), decode the encoded video data to restore the video pictures, and display the video pictures on an accessible display device according to the restored video data.

[0034] In the example in Figure 3, terminal devices (310), (320), (330), and (340) may be implemented as servers, personal computers, and smartphones, but the applicability of the underlying principles of this disclosure may not be so limited. Embodiments of this disclosure may be implemented as desktop computers, laptop computers, tablet computers, media players, wearable computers, dedicated video conferencing equipment, and / or similar devices. Network (350) represents any number or type of network that transmits encoded video data between terminal devices (310), (320), (330), and (340), including, for example, wired communication networks and / or wireless communication networks. Communication network (350) may exchange data over circuit-switched channels, packet-switched channels, and / or other corresponding channels. Typical networks include far-field communication networks, local area networks, wide area networks, and / or the Internet. For the purposes of this description, the architecture and topology of the network (350) may be considered irrelevant to the operation of this disclosure unless explicitly described herein.

[0035] Figure 4 illustrates the arrangement of a video encoder and video decoder in a video streaming environment as an example of an application relating to the disclosed matters. The disclosed matters may be equally applicable to other video applications, including, for example, video conferencing, digital TV broadcasting, games, virtual reality, and the storage of compressed video on digital media including CDs, DVDs, memory sticks, and similar devices.

[0036] The video streaming system may include a video capture subsystem (413) which may include a video source (401), such as a digital camera, that produces a stream (402) of uncompressed video pictures or images. In one example, the stream of video pictures (402) includes samples recorded by the digital camera of the video source (401). The stream of video pictures (402) is drawn as a thick line to emphasize that it has a higher data volume compared to encoded video data (404) (or encoded video bitstream) and may be processed by an electronic device (420) which includes a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the matters relating to the disclosures described in more detail below. The encoded video data (404) (or encoded video bitstream (404)) is depicted as a thin line to emphasize that it has a lower data volume compared to the uncompressed video picture stream (402), and can be stored in a streaming server (405) for later use or provided directly to a downstream video device (not shown). One or more streaming client subsystems, such as the client subsystems (406) and (408) in Figure 4, can access the streaming server (405) to retrieve copies (407) and (409) of the encoded video data (404). The client subsystem (406) may include a video decoder (410), for example, in an electronic device (430). The video decoder (410) can decode the incoming copy of the encoded video data (407) to produce an outgoing video picture stream (411), which is uncompressed and can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown). The video decoder (410) may be configured to perform some or all of the various functions described in this disclosure.In some streaming systems, encoded video data (404), (407), and (409) (e.g., video bitstream) can be encoded according to specific video encoding / compression standards. Examples of these standards include ITU-T Recommendation H.265. In one example, a video encoding standard under development is informally known as Versatile Video Coding (VVC). The disclosures may be used in the context of VVC and other video encoding standards.

[0037] The electronic devices (420) and (430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[0038] Figure 5 shows a block diagram of a video decoder (510) according to one of the embodiments of the present disclosure described below. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuit). The video decoder (510) can be used in place of the video decoder (410) in the example of Figure 4.

[0039] The receiver (531) may receive one or more encoded video sequences to be decoded by the video decoder (510). In the same embodiment or another embodiment, one encoded video sequence may be decoded at a time, and the decoding of each encoded video sequence is independent of other encoded video sequences. Each video sequence may relate to multiple video frames or images. The encoded video sequence may be received from a channel (501) which may be a storage device for storing encoded video data or a hardware / software link to a streaming source for transmitting encoded video data. The receiver (531) may receive the encoded video data together with other data, such as encoded audio data and / or auxiliary data streams, which may be transferred to their respective processing circuits (not shown). The receiver (531) may isolate the encoded video sequence from other data. To counteract network jitter, a buffer memory (515) may be placed between the receiver (531) and the entropy decoder / parser 520 (hereinafter, “Parser (520)”). In certain applications, the buffer memory (515) may be implemented as part of the video decoder (510). In other applications, it may be separate and external to the video decoder (510) (not shown). In yet other applications, for example, to counter network jitter, a buffer memory (not shown) may exist outside the video decoder (510), and an additional buffer memory (515) may exist inside the video decoder (510) to handle playback timing, for example. When the receiver (531) is receiving data from a storage / transfer device with sufficient bandwidth and controllability or from an isosynchronous network, the buffer memory (515) may not be required or may be small. For example, when used on a best-effort packet network such as the Internet, a sufficiently sized buffer memory (515) may be required, and its size may be relatively large.Such buffer memory may be implemented with an adaptable size and may also be implemented at least partially in an operating system or similar element (not shown) outside the video decoder (510).

[0040] The video decoder (510) may include a parser (520) for reconstructing symbols (521) from the encoded video sequence. The categories of these symbols may include information used to manage the operation of the video decoder (510) and may also include information for controlling rendering devices, such as a display (512) (e.g., a display screen), which may or may not be an integrated part of the electronic device (530) but can be coupled to the electronic device (530), as shown in Figure 5. The control information for (one or more) rendering devices may take the form of Supplementary Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not shown). The parser (520) may syntactically parse / entropically decode the encoded video sequence it receives. The entropy coding of the encoded video sequence may be by video coding technique or standard and may follow a variety of principles, including variable-length coding, Huffman coding, context-dependent or non-context-dependent arithmetic coding, etc. The parser(520) can extract from the encoded video sequence a set of subgroup parameters relating to at least one of the pixel subgroups in the video decoder, based on at least one parameter corresponding to a subgroup. Subgroups may include groups of pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), and predictive units (PU). The parser(520) can also extract information from the encoded video sequence information such as transform coefficients (e.g., Fourier transform coefficients), quantization parameter values, and motion vectors.

[0041] The parser (520) may perform entropy decoding / parsing on the video sequence received from buffer memory (515) to generate symbols (521).

[0042] The reconstruction of the symbol (521) may involve multiple different processing units or functional units, depending on the type of encoded video picture or part thereof and other factors (e.g., interpicture and intrapicture, interblock and intrablock, etc.). The units involved and how they are involved may be controlled by subgroup control information parsed from the encoded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the following multiple processing units or functional units is not illustrated for simplicity.

[0043] Beyond the functional blocks described above, the video decoder (510) can be conceptually subdivided into numerous functional units, as described below. In a practical implementation operating under commercial constraints, many of these functional units may closely interact with each other and be at least partially integrated. However, for the purpose of clearly illustrating the various functions of the disclosed matters, the following disclosure adopts a conceptual subdivision into functional units.

[0044] The first unit may include a scaler / inverse unit (551). The scaler / inverse unit (551) may receive quantized transformation coefficients as (one or more) symbols (521) from the parser (520), along with control information including information indicating which type of inverse transformation should be used, such as block size, quantization coefficients / parameters, quantization scaling matrix, and similar information. The scaler / inverse unit (551) may output a block containing sample values ​​that can be input to the aggregator (555).

[0045] In some cases, the output samples of the scaler / inverse transform (551) may relate to intracoded blocks, i.e., blocks that do not use prediction information from previously reconstructed pictures but can use prediction information from previously reconstructed portions of the current picture. Such prediction information can be provided by an intrapicture prediction unit (552). In some cases, the intrapicture prediction unit (552) may generate a block of the same size and shape as the block being reconstructed, using information from surrounding blocks that have already been reconstructed and stored in the current picture buffer (558). The current picture buffer (558) buffers, for example, partially reconstructed current pictures and / or fully reconstructed current pictures. In some implementations, the aggregator (555) may, for each sample, add the prediction information generated by the intra prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551).

[0046] In other cases, the output samples of the scaler / inverse unit (551) may relate to an intercoded, potentially motion-compensated block. In such cases, the motion-compensated prediction unit (553) can access the reference picture memory (557) to fetch samples to be used for interpicture prediction. After the fetched samples are motion-compensated according to the symbols (521) related to the block, these samples can be appended by the aggregator (555) to the output of the scaler / inverse unit (551) (the output of unit (551) may be referred to as residual samples or residual signals) to generate output sample information. From there, the addresses in the reference picture memory (557) from which the motion-compensated prediction unit (553) fetches prediction samples can be controlled by a motion vector and are available to the motion-compensated prediction unit (553) in the form of symbols (521) which may have, for example, X, Y components (shift) and a reference picture component (time). Motion compensation may also include interpolation of sample values ​​fetched from reference picture memory (557) when the precise motion vectors of subsamples are used, and may be associated with motion vector prediction mechanisms, etc.

[0047] The output samples from the aggregator (555) can be subjected to various loop filtering techniques in the loop filter unit (556). The video compression technique may include in-loop filtering, which is controlled by parameters included in the encoded video sequence (also called the encoded video bitstream) and made available to the loop filter unit (556) as symbols (521) from the parser (520), and can also respond to metadata obtained during the decoding of preceding portions of the encoded picture or encoded video sequence (in decoding order), as well as to sample values ​​that have been previously reconstructed and loop-filtered. As will be described in more detail later, several types of loop filters may be included as part of the loop filter unit (556) in various orders.

[0048] The output of the loop filter unit (556) can be a sample stream that can be output to the rendering device (512), and this can also be stored in reference picture memory (557) for use in future interpicture prediction.

[0049] A particular encoded picture, once fully reconstructed, can be used as a reference picture for predicting future interpictures. For example, once the encoded picture corresponding to the current picture is fully reconstructed and that encoded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) may become part of the reference picture memory (557), and a new current picture buffer may be reallocated before the reconstruction of the next encoded picture begins.

[0050] The video decoder (510) may perform decoding according to a predetermined video compression technique adopted in a standard such as ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax defined by the video compression technique or standard used, in the sense that it faithfully adheres to both the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, a profile may select specific tools from all the tools available in the video compression technique or standard, such that only those tools are available for use under that profile. In addition, the complexity of the encoded video sequence may be kept within the range defined by the level of the video compression technique or standard in order to comply with the standard. Where applicable, the level may restrict the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limitations set by the level may be further restricted, where applicable, through the Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled in the encoded video sequence.

[0051] In some embodiments, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of one or more encoded video sequences. The additional data may be used by the video decoder (510) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may take the form of, for example, a temporal, spatial, or signal-to-noise ratio (SNR) enhancement layer, redundant slices, redundant pictures, or forward error correction codes.

[0052] Figure 6 shows a block diagram of a video encoder (603) according to one embodiment of the present disclosure. The video encoder (603) may be included in an electronic device (620). The electronic device (620) may further include a transmitter (640) (e.g., a transmitting circuit). The video encoder (603) can be used in place of the video encoder (403) in the example of Figure 4.

[0053] The video encoder (603) may receive video samples from a video source (601) (not part of the electronic device (620) in the example of Figure 6) which can capture (one or more) video images to be encoded by the encoder (603). In another example, the video source (601) may be implemented as part of the electronic device (620).

[0054] The video source (601) may provide a source video sequence encoded by a video encoder (603) in the form of a digital video sample stream, which can have any preferred bit depth (e.g., 8 bits, 10 bits, 12 bits, ...), any color space (e.g., BT.601 YCrCB, RGB, XYZ, ...), and any preferred sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service delivery system, the video source (601) may be a storage device capable of storing pre-prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures or images that convey motion when viewed sequentially. These pictures themselves may be organized as a spatial array of pixels, and each pixel may have one or more samples depending on the sampling structure, color space, and similar elements used. Those skilled in the art will immediately understand the relationship between pixels and samples. The following description focuses on samples.

[0055] In some embodiments, the video encoder (603) can encode and compress pictures from a source video sequence into an encoded video sequence (643) in real time or under other time constraints required by the application. Enforcing an appropriate encoding rate constitutes one function of the controller (650). In some embodiments, the controller (650) can be functionally coupled to and control other functional units, such as those described later. The coupling is not illustrated for simplicity. Parameters set by the controller (650) may include rate control-related parameters (picture skip, quantizer, lambda value of rate distortion optimization technique, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, and similar. The controller (650) can be configured to have other preferred functions related to the video encoder (603) that are optimized for a particular system design.

[0056] In some embodiments, the video encoder (603) may be configured to operate in a coding loop. In an oversimplified explanation, in one example, the coding loop may include a source coder (630) (for example, responsible for creating symbols, such as a symbol stream, based on the input picture to be encoded and one or more reference pictures) and a (local) decoder (633) embedded in the video encoder (603). The embedded decoder (633) processes the encoded video stream by the source coder (630) without entropy coding (where the compression between the symbols and the encoded video bitstream in entropy coding is reversible in the video compression techniques considered in the disclosure), the decoder (633) reconstructs the symbols to generate sample data in the same way that a (remote) decoder would. The reconstructed sample stream (sample data) is input to a reference picture memory (634). Symbol stream decoding yields bit-accurate results independent of decoder location (local or remote), so the contents in reference picture memory (634) are also bit-accurate between the local and remote encoders. In other words, the predictive portion of the encoder "sees" the exact same sample values ​​as the reference picture samples that the decoder "sees" when using predictions during decoding. This fundamental principle of reference picture synchronization (and the resulting drift when synchronization cannot be maintained, for example, due to channel errors) is used to improve coding quality.

[0057] The operation of the “local” decoder (633) can be the same as that of a “remote” decoder, such as a video decoder (510), which has already been described in detail above in relation to Figure 5. However, also briefly referring to Figure 5, since symbols are available and the encoding / decoding of symbols to an encoded video sequence by the entropy coder (645) and parser (520) can be reversible, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and parser (520), does not need to be fully implemented in the local decoder (633) within the encoder.

[0058] At this point, it can be realized that decoder techniques, excluding parsing / entropy decoding which exist only within the decoder, must necessarily exist in substantially the same functional form within the corresponding encoder. For this reason, the disclosure may occasionally focus on decoder operation, which is applied to the decoding portion of the encoder. The description of encoder techniques can therefore be omitted, as it is the inverse of the thoroughly described decoder techniques. A more detailed description of the encoder is provided below only in certain areas or embodiments.

[0059] During operation, in some implementations, the source coder (630) may perform motion-compensated predictive coding, which predictively encodes the input picture against one or more previously encoded pictures from a video sequence designated as “reference pictures”. Thus, the coding engine (632) codes the difference (or residual) in the color channels between the pixel blocks of the input picture and the pixel blocks of one or more reference pictures that may be selected as prediction criteria for the input picture.

[0060] The local video decoder (633) can decode the encoded video data of a picture that may be designated as a reference picture based on symbols created by the source coder (630). The operation of the coding engine (632) can, advantageously, be a lossy process. When the encoded video data can be decoded by a video decoder (not shown in Figure 6), the reconstructed video sequence may typically be a replica of the source video sequence with some error. The local video decoder (633) can replicate the decoding process that may be performed by the video decoder on the reference picture and cause the reconstructed reference picture to be stored in the reference picture cache (634). Thus, the video encoder (603) can locally store a copy of the reconstructed reference picture that has content common to the reconstructed reference picture that will be obtained by the far-end (remote) video decoder.

[0061] The predictor (635) may perform a predictive search for the coding engine (632). That is, with respect to a new picture to be coded, the predictor (636) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors or block shapes that can serve as appropriate predictive criteria for the new picture. The predictor (635) may operate pixel block by pixel to find appropriate predictive references. The input picture may have predictive criteria drawn from multiple reference pictures stored in the reference picture memory (634), as determined by the search results obtained by the predictor (635).

[0062] The controller (650) may manage the coding process of the source coder (630), including, for example, setting parameters and subgroup parameters used to encode video data.

[0063] The outputs of all the aforementioned functional units can be subjected to entropy coding in the entropy coder (645). The entropy coder (645) converts the symbols generated by the various functional units into encoded video sequences by lossless compression of symbols according to techniques such as Huffman coding, variable-length coding, and arithmetic coding.

[0064] The transmitter (640) may buffer (one or more) encoded video sequences generated by the entropy coder (645) and prepare them for transmission over the communication channel (660). The communication channel (660) may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (640) may merge the encoded video data from the video coder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0065] The controller (650) may manage the operation of the video encoder (603). In coding, the controller (650) may assign each encoded picture a specific encoded picture type that may influence the coding techniques that may be applied to that picture. For example, a picture may often be assigned one of the following picture types:

[0066] An intra-picture (I-picture) may be one that can be encoded and decoded without using any other picture in the sequence as a source for prediction. Some video codecs allow several different types of intra-pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art know these variations of I-pictures, as well as their respective uses and characteristics.

[0067] A prediction picture (P-picture) may be encoded and decoded using intra-prediction or inter-prediction, with at most one motion vector and a reference index to predict the sample values ​​of each block.

[0068] A bidirectional predictive picture (B-picture) may be one that can be encoded and decoded using intra-prediction or inter-prediction, using up to two motion vectors and a reference index to predict the sample values ​​of each block. Similarly, a multiple predictive picture may use three or more reference pictures and associated metadata for the reconstruction of a single block.

[0069] A source picture can generally be subdivided spatially into multiple sample coding blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each), and each block can be coded. Blocks can be coded predictively by referencing other (already coded) blocks determined by the coding assignment applied to each picture within those blocks. For example, blocks of an I-picture can be coded unpredictably, or they can be coded predictively by referencing already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks of a P-picture can be coded unpredictably, or via spatial or temporal prediction by referencing one previously coded reference picture. Blocks of a B-picture can be coded unpredictably, or via spatial or temporal prediction by referencing one or two previously coded reference pictures. The source picture or an intermediate picture may be subdivided into other types of blocks for other purposes. The subdivision of coding blocks and other types of blocks may or may not follow the same method, as will be described in more detail later.

[0070] The video encoder (603) may perform coding operations in accordance with a predetermined video coding technique or standard, such as ITU-T Recommendation H.265. In its operation, the video encoder (603) may perform various compression operations, including predictive coding operations that take advantage of temporal and spatial redundancy in the input video sequence. The encoded video data may therefore conform to the syntax defined by the video coding technique or standard being used.

[0071] In some embodiments, the transmitter (640) may transmit additional data along with the encoded video. The source coder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, and the like.

[0072] Video can be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) utilizes spatial correlations within a given picture, while inter-picture prediction utilizes temporal or other correlations between pictures. For example, a particular picture being encoded / decoded, called the current picture, may be divided into multiple blocks. Blocks within the current picture may be coded by a vector called a motion vector, when they are analogous to reference blocks in a previously coded and still-buffering reference picture within the video. The motion vector points to a reference block in the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0073] In some embodiments, a dual prediction technique can be used for interpicture prediction. Such a dual prediction technique uses two reference pictures, such as a first reference picture and a second reference picture, both of which precede the current picture in decoding order (but may be past or future in display order, respectively). A block in the current picture can be coded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block can be predicted in conjunction with the combination of the first and second reference blocks.

[0074] Furthermore, merge mode techniques may be used to improve coding efficiency in interpicture prediction.

[0075] According to some embodiments of this disclosure, predictions such as interpicture prediction and intrapicture prediction are performed in units of blocks. For example, a picture in a sequence of video pictures may be divided into multiple coding tree units (CTUs) for compression, and these CTUs in the picture may have the same size, for example, 64x64 pixels, 32x32 pixels, or 16x16 pixels. Generally, a CTU may include three parallel coding tree blocks (CTBs), which are one luma CTB and two chroma CTBs. Each CTU may be recursively quadtree-divided into one or more coding units (CUs). For example, a 64x64 pixel CTU may be divided into one 64x64 pixel CU or four 32x32 pixel CUs. Each of one or more of the 32x32 blocks may be further divided into four 16x16 pixel CUs. In some embodiments, each CU may be analyzed during coding to determine its prediction type from among various prediction types, such as inter-prediction type or intra-prediction type. A CU may be divided into one or more prediction units (PUs) depending on its temporal and / or spatial predictability. Generally, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, the prediction operation during coding (encoding / decoding) is performed in units of prediction blocks. The division of a CU into PUs (or PBs for different color channels) can be carried out in various spatial patterns. For example, a luma or chroma PB may include a matrix of sample values ​​(e.g., luma values), such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 samples, and similar.

[0076] Figure 7 shows a diagram of a video encoder (703) according to another embodiment of the present disclosure. The video encoder (703) is configured to receive sample values ​​of a processing block (e.g., a prediction block) in the current video picture in a sequence of video pictures and to encode the processing block into an encoded picture which is part of an encoded video sequence. This example of a video encoder (703) may be used instead of the video encoder (403) in the example of Figure 4.

[0077] For example, the video encoder (703) receives a matrix of sample values ​​relating to a processing block, such as an 8x8 sample of a prediction block. The video encoder (703) then determines, for example using rate-distortion optimization (RDO), whether the processing block is best coded using intra-mode, inter-mode, or bi-prediction mode. If it is determined that the processing block is coded in intra-mode, the video encoder (703) can encode the processing block into an encoded picture using the intra-prediction technique. If it is determined that the processing block is coded in inter-mode or bi-prediction mode, the video encoder (703) can encode the processing block into an encoded picture using the inter-prediction technique or the bi-prediction technique, respectively. In some embodiments, merge mode may be used as a submode of interpicture prediction in which motion vectors are derived from one or more motion vector predictors without benefiting from the encoded motion vector components outside the predictors. In some other embodiments, there may be motion vector components applicable to the target block. Therefore, the video encoder (703) may include components not explicitly shown in Figure 7, such as a mode determination module for determining the prediction mode of a processing block.

[0078] In the example shown in Figure 7, the video encoder (703) includes an interencoder (730), an intraencoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725), all coupled together as shown in the configuration example in Figure 7.

[0079] The interencoder (730) is configured to receive a sample of the current block (e.g., a processing block), compare the block with one or more reference blocks in the reference picture (e.g., blocks in preceding and succeeding pictures in display order), generate interprediction information (e.g., a description of redundant information according to the intercoding technique, motion vectors, merge mode information), and then use some preferred technique to calculate an interprediction result (e.g., a predicted block) based on the interprediction information. In some examples, the reference picture is a reference picture decoded based on encoded video information using a decoding unit (633) embedded in the example encoder (620) of Figure 6 (shown in more detail later as a residual decoder 72 in Figure 7).

[0080] The intra encoder (722) is configured to receive a sample of the current block (e.g., a processing block), compare the block to an already encoded block in the same picture, generate a converted quantization coefficient, and in some cases also generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). Based on the intra prediction information and a reference block in the same picture, the intra prediction result (e.g., a prediction block) may be calculated.

[0081] The general controller (721) may be configured to determine general control data and control other components of the video encoder (703) based on the general control data. In one example, the general controller (721) determines the prediction mode of a block and provides control signals to the switch (726) based on that prediction mode. For example, when the prediction mode is intra-mode, the general controller (721) controls the switch (726) to select the intra-mode result for use by the residual calculator (723) and controls the entropy encoder (725) to select the intra-prediction information and include it in the bitstream. When the prediction mode of a block is inter-mode, the general controller (721) controls the switch (726) to select the inter-prediction result for use by the residual calculator (723) and controls the entropy encoder (725) to select the inter-prediction information and include it in the bitstream.

[0082] A residual calculator (723) may be configured to calculate the difference (residual data) between a received block and the prediction result of a block selected from an intra-encoder (722) or an inter-encoder (730). A residual encoder (724) may be configured to encode the residual data to generate conversion coefficients. For example, the residual encoder (724) may be configured to convert the residual data from the spatial domain to the frequency domain to generate conversion coefficients. The conversion coefficients are then subjected to a quantization process to obtain quantized conversion coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform to generate decoded residual data. The decoded residual data can be suitably used by the intra-encoder (722) and the inter-encoder (730). For example, an interencoder (730) can generate a decoded block based on decoded residual data and interprediction information, and an intraencoder (722) can generate a decoded block based on decoded residual data and intraprediction information. The decoded block is suitably processed to generate a decoded picture, which can be buffered in a memory circuit (not shown) and used as a reference picture.

[0083] The entropy encoder (725) may be configured to format the bitstream to include the encoded blocks and to perform entropy coding. The entropy encoder (725) may be configured to include various types of information in the bitstream. For example, the entropy encoder (725) may be configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other preferred information in the bitstream. When coding blocks in either inter-mode or bi-prediction mode merge submode, residual information may not be present.

[0084] Figure 8 shows an example of a video decoder (810) according to another embodiment of the present disclosure. The video decoder (810) is configured to receive an encoded picture which is part of an encoded video sequence, and to decode the encoded picture to produce a reconstructed picture. In one example, the video decoder (810) may be used instead of the video decoder (410) in the example of Figure 4.

[0085] In the example shown in Figure 8, the video decoder (810) includes an entropy decoder (871), an interdecoder (880), a residual decoder (873), a reconfiguration module (874), and an intradecoder (872), all coupled together as shown in the configuration example in Figure 8.

[0086] The entropy decoder (871) may be configured to reconstruct specific symbols from the encoded picture that represent the syntax elements constituting the encoded picture. Such symbols may include, for example, the mode in which the block is coded (e.g., intra-mode, inter-mode, bi-prediction mode, merge sub-mode, or other sub-mode), prediction information (e.g., intra-prediction information or inter-prediction information) that can identify specific samples or metadata used for prediction by the intra-decoder (872) or inter-decoder (880), residual information in the form of quantized transformation coefficients, and similar things. In one example, when the prediction mode is inter-mode or bi-prediction mode, inter-prediction information is provided to the inter-decoder (880), and when the prediction type is intra-prediction type, intra-prediction information is provided to the intra-decoder (872). The residual information can be dequantized and provided to the residual decoder (873).

[0087] The interdecoder (880) may be configured to receive interprediction information and generate interprediction results based on the interprediction information.

[0088] The intra decoder (872) may be configured to receive intra prediction information and generate prediction results based on the intra prediction information.

[0089] The residual decoder (873) may be configured to perform inverse quantization to extract the dequantized transformation coefficients, and then process the dequantized transformation coefficients to convert the residual from the frequency domain to the spatial domain. The residual decoder (873) may also utilize certain control information (including quantizer parameters (QP)) that may be provided by the entropy decoder (871) (the data path is not shown as this may only be low-data-volume control information).

[0090] The reconstruction module (874) may be configured to combine the residual output by the residual decoder (873) and the prediction result (optionally output by the inter or intra prediction module) in the spatial domain to form a reconstruction block that forms part of the reconstruction picture as part of the reconstruction image. In addition, other suitable processes, such as deblocking and similar processes, may be performed to improve visual quality.

[0091] The video encoders (403), (603), and (703), and the video decoders (410), (510), and (810) may be implemented using any preferred technology. In some embodiments, the video encoders (403), (603), and (703), and the video decoders (410), (510), and (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603), and (703), and the video decoders (410), (510), and (810) may be implemented using one or more processors that execute software instructions.

[0092] Returning to the intra-prediction process, a prediction block is generated by predicting a sample within a block (e.g., a luma or chroma prediction block, or a coding block if it is not further divided into prediction blocks) using adjacent samples, the next adjacent sample, or samples from one or more other lines, or a combination thereof. The residual between the actual block to be coded and the prediction block can then be processed by transformation and subsequent quantization. Various intra-prediction modes can be made available, and parameters related to intra-mode selection and other parameters can be signaled in the bitstream. These various intra-prediction modes may relate, for example, to one or more line positions for predicting a sample, the direction in which prediction samples are selected from one or more prediction lines along that position, and other special intra-prediction modes.

[0093] For example, a set of intra-prediction modes (also interchangeably referred to as “intra-modes”) may include a predetermined number of directional intra-prediction modes. As described above in relation to the implementation example in Figure 1, these intra-prediction modes correspond to a predetermined number of directions, and along those directions, out-of-block samples may be selected as predictions for predicted samples within a particular block. In another specific implementation example, eight main directional modes corresponding to angles from 45 to 207 degrees with respect to the horizontal axis may be supported and predefined.

[0094] In some other implementations of intra-prediction, to further leverage the greater spatial redundancy of directional textures, the directional intra-modes can be extended to a finer-grained set of angles. For example, the 8-angle implementation above can be configured to provide 8 nominal angles called V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED, as shown in Figure 9, with a predetermined number (e.g., 7) finer angles added to each nominal angle. Such extensions can make a larger total number of directional angles (e.g., 56 in this example) available for intra-prediction, corresponding to the same number of predefined directional intra-modes. Predicted angles can be represented by the nominal intra-angle plus an angle delta. In the particular example above, there are 7 finer angular directions for each nominal angle, and the angle delta can be a step size of 3 degrees multiplied by -3 to 3.

[0095] The directional intra-prediction described above is sometimes called unidirectional intra-prediction and is different from bidirectional intra-prediction (also called intra-bidirectional prediction), which will be explained later in this disclosure.

[0096] In some implementations, a predetermined number of non-directed intra-prediction modes may be predefined and available in place of, or in addition to, the directional intra-modes above. For example, five non-directed intra-modes called smoothing intra-prediction modes may be defined. These non-directed intra-prediction modes may specifically be called DC, PAETH, SMOOTH, SMOOTH_V, and SMOOTH_H intra-modes. Figure 10 shows the prediction of samples in a particular block under examples of these non-directed modes. As an example, Figure 10 shows that a 4x4 block 1002 is predicted by samples from the upper adjacent line and / or the left adjacent line. A particular sample 1010 in block 1002 may correspond to sample 1004 directly above sample 1010 in the upper adjacent line of block 1002, sample 1006 to the upper left of sample 1010 as the intersection of the upper and left adjacent lines, and sample 1008 directly to the left of sample 1010 in the left adjacent line of block 1002. In the DC intra-prediction mode example, the average of the left and top adjacent samples 1008 and 1004 can be used as the predictor for sample 1010. In the PAETH intra-prediction mode example, the top, left, and top-left reference samples 1004, 1008, and 1006 are fetched, and the value closest to (top + left - top-left) among these three reference samples can be set as the predictor for sample 1010. In the SMOOTH_V intra-prediction mode example, sample 1010 can be predicted by vertical quadratic interpolation of the top-left adjacent sample 1006 and the left adjacent sample 1008. In the SMOOTH_H intra-prediction mode example, sample 1010 can be predicted by horizontal quadratic interpolation of the top-left adjacent sample 1006 and the top adjacent sample 1004. In the SMOOTH intra-prediction mode example, sample 1010 can be predicted by the average of the vertical quadratic interpolation and the horizontal quadratic interpolation. The above non-directed intra-mode implementation is merely presented as a non-limiting example. Other adjacent lines, other non-directed sample selections, and methods of combining predicted samples to predict specific samples within a prediction block are also considered.

[0097] The selection of a specific intra-predictive mode by the encoder from the directional or non-directional modes above, at various coding levels (picture, slice, block, unit, etc.), can be signaled within the bitstream. In some implementations, eight exemplary nominal directional modes may first be signaled along with five non-angle smoothing modes (a total of 13 options). Then, if the signaled mode is one of the eight nominal angle intra-modes, an index is further signaled to indicate the selected angle delta for the corresponding signaled nominal angle. In some other implementations, all intra-predictive modes may be indexed together for signaling (e.g., 56 directional modes plus 5 non-directional modes to generate 61 intra-predictive modes).

[0098] In some implementations, the 56 or other number of directional intra-prediction modes in this example may be implemented with a unified directional predictor that projects each sample in a block onto the location of a reference subsample and interpolates the reference sample with a two-tap bilinear filter.

[0099] In some implementations, an additional filter mode called FILTER INTRA mode may be designed to capture the attenuation of spatial correlation with references on edges. In these modes, for some patches within a block, predicted samples within the block may be used as intra-predictive reference samples in addition to samples outside the block. These modes may be predefined and available, for example, for intra-predictions for at least Luma blocks (or Luma blocks only). A predetermined number (e.g., 5) of filter intra modes can be predesigned, each represented by a set of n-tap filters (e.g., 7-tap filters) that reflect the correlation between a sample in a 4x2 patch and its adjacent n neighboring samples. In other words, the weighting coefficients of the n-tap filters may be position-dependent. Taking 8x8 blocks, 4x2 patches, and 7-tap filtering as an example, an 8x8 block 1102 can be divided into eight 4x2 patches, as shown in Figure 11. These patches are shown as B0, B1, B2, B3, B4, B5, B6, and B7 in Figure 11. For each patch, the seven adjacent patches (shown as R0-R7 in Figure 11) can be used to predict the samples currently within the patch. In the case of patch B0, it is possible that all adjacent patches have already been reconstructed. However, in other patches, some of the adjacent patches may be currently within the block and therefore not reconstructed, in which case the predicted values ​​of the immediately adjacent patches are used as references. For example, in patch B7 shown in Figure 11, not all adjacent patches have been reconstructed, and therefore, adjacent patches that are part of B4, B5, and / or B6 are used instead.

[0100] In some implementations of intra-prediction, one color component may be predicted using one or more other color components. Color components can be any of the components in the YCrCb, RGB, XYZ color space, etc. For example, prediction of chroma components (e.g., chroma blocks) from luma components (e.g., luma reference samples), known as chroma-from-luma (i.e., CfL), can be implemented. In some implementations, cross-color prediction is only allowed from luma to chroma. For example, chroma samples within a chroma block may be modeled as a linear function of matching reconstructed luma samples. CfL prediction is, CfL(α) = α × L AC +DC (1) It can be implemented as follows.

[0101] Here, L AC α represents the AC contribution of the luma component, α represents the parameters of the linear model, and DC represents the DC contribution of the chroma component. For example, the AC component is obtained for each sample in a block, while the DC component is obtained for the entire block. Specifically, the reconstructed luma samples may be subsampled to the chroma resolution, and then the average luma value (DC of luma) may be subtracted from each luma value to form the AC contribution of luma. The AC contribution of luma is then used in the linear mode of equation (1) to predict the AC value of the chroma component. Instead of requiring the decoder to calculate scaling parameters to approximate or predict the chroma AC component from the luma AC contribution, one example CfL implementation may determine the parameter α based on the original chroma samples and signal them in the bitstream. This reduces the complexity of the decoder and produces more accurate predictions. With respect to the DC contribution of the chroma component, it may be calculated using an intra-DC mode within the chroma component in some implementation examples.

[0102] In some embodiments, for a particular coding block, coding unit, prediction block, or prediction unit being intracoded, its intra-mode needs to be signaled by one or more syntax elements in the bitstream. As described above, the number of possible intra-prediction modes can be enormous, and 62 intra-prediction modes may be available, including 56 directional intra-prediction modes, 5 non-directional modes, and 1 chroma-from-lumen mode (e.g., for chroma components only). To signal these intra-prediction modes, a first syntax may be signaled to indicate which nominal angular mode or non-directional mode is equal to the nominal mode of the current block. Then, if the mode of the current block is a directional mode, a second syntax may be signaled to indicate which delta angle is equal to that of the current block. In some situations during video encoding or decoding, a strong correlation may exist between the intra-prediction modes of the current block and its adjacent blocks.

[0103] In various embodiments, this correlation can be leveraged to design a more efficient syntax for intra-mode coding. In some implementations, the intra-predictive modes available for the current block may be divided into multiple intra-predictive mode sets according to the intra-predictive modes of its adjacent blocks. To obtain the intra-predictive modes for the current block, a mode set index indicating the mode set index of the current block's intra-predictive modes can be signaled first, and then a mode index indicating the index of the intra-predictive modes within the mode set can be signaled.

[0104] Herein, in various embodiments of the present disclosure, “XYZ is signaled” can mean that XYZ is encoded into an encoded bitstream in an encoding process, and / or, after the encoded bitstream has been transmitted from one device to another, “XYZ is signaled” can mean that XYZ is decoded / extracted from the encoded bitstream in a decoding process.

[0105] For example, in some of the implementations described above, the number of available intra-prediction modes can include 62 distinct modes, such as 56 directional intra-prediction modes (e.g., 8 nominal directions with 7 fine angles for each nominal direction), 5 non-directional modes, and 1 chroma-from-luma mode (e.g., for chroma components only). When an intra-mode is selected in a particular coding block, coding unit, prediction block, or prediction coding process, the signaling corresponding to the selected intra-mode must be included in the bitstream. All 62 modes must be distinguishable in some way by signaling with (one or more) syntax. For example, these 62 modes could be signaled using a single syntax for 62 indices, each corresponding to one mode. In some other implementations, one syntax may be signaled to indicate which nominal angular mode or non-directional mode is used as the nominal mode in the current block, and another syntax may be additionally signaled to indicate which delta angle is selected for the current block when the nominal mode of the current block is a directional mode.

[0106] Various syntax related to intra-coding typically occupies a large portion of the bitstream, and intra-mode selection must be signaled frequently, for example, at various coding levels. Therefore, reducing the number of bits used for intra-mode signaling is crucial for improving video coding efficiency. In practice, the use of various intra-prediction modes may follow certain statistical patterns, and signaling efficiency can be improved by designing intra-mode indexing and signaling syntax using such usage patterns. Also, generally, some correlation may exist between blocks in intra-mode selection. Such correlations can be obtained statistically and offline and can be taken into consideration in the design of (one or more) syntax for signaling intra-mode selection. The goal is to generally reduce the number of bits used to signal syntax elements within the encoded bitstream. For example, some general statistics may indicate that there may be a strong correlation between the current block and the optimal intra-prediction mode of its adjacent blocks. Such correlations can be leveraged when designing syntax for intra-mode coding.

[0107] In various embodiments, the intra-predictive modes or mode options available for the currently coded block may be divided into multiple intra-predictive mode sets. Each set may be assigned a mode set index. Each set may contain a certain number of intra-mode predictive modes. The manner in which the available intra-predictive modes are divided and ordered, and in which the intra-predictive modes are ordered within each mode set, may be determined based on the correlation between intra-predictive modes between blocks, and at least in part according to the intra-predictive modes used by its neighboring blocks. The intra-predictive modes used by neighboring blocks may be referred to as “reference intra-predictive modes” or “reference modes”. Intra-predictive modes may be determined and selected for a particular unit. The selection of intra-predictive modes may be signaled. First, a mode set index may be signaled to indicate the mode set index of the intra-predictive mode set containing the selected intra-predictive modes. Next, a mode index (or mode position index within the set) may be signaled to indicate the index of the selected intra-predictive modes within that mode set.

[0108] The general implementation of intra-prediction mode partitioning and ordering shown above, as well as the specific examples below, utilize statistical effects and adjacent correlations to dynamically index these modes, thereby optimizing the syntax design for signaling their selection within the encoded video bitstream and improving coding efficiency. For example, these implementations can help reduce the number of syntax elements for signaling and facilitate more efficient context generation for entropy coding.

[0109] The various embodiments and / or implementations described herein may be used separately or in any order. Furthermore, some, all, or partial or complete combinations of these embodiments and / or implementations may be embodied as part of an encoder and / or decoder, and may be implemented in hardware and / or software. For example, they may be hardcoded into a dedicated processing circuit (e.g., one or more integrated circuits). In another example, they may be implemented by one or more processors executing a program stored on a non-temporary computer-readable medium.

[0110] Several challenges / problems may exist in relation to intra-mode coding. For example, because there is a strong correlation between a sample in a block and its adjacent blocks, when dividing intra-prediction modes into multiple different mode sets, the probability of selecting an intra-prediction mode from a different mode set may be correlated with the intra-prediction modes of adjacent blocks. This strong correlation and associated probability distribution are not efficiently utilized, leading to inefficiencies and poor performance in some intra-mode coding designs.

[0111] This disclosure describes various embodiments for intra-predictive mode coding in video encoding and / or decoding, addressing at least one of the challenges / problems described above, achieving efficient intra-mode coding, and improving the performance of intra-mode coding.

[0112] In various embodiments, one or more contexts can be designed / derived for entropy coding of set indices and / or mode indices in a bitstream to indicate an intra-predictive mode for the current block, in order to leverage the correlation and associated probability distribution between a sample in the current block and its adjacent blocks. These one or more contexts can be used in the coding process to encode the set indices and / or mode indices into the encoded bitstream, and / or in the decoding process to decode the set indices and / or mode indices from the encoded bitstream.

[0113] The context for coding the set index and / or mode index of the current coding block may be selected from a set of contexts. The selection of context may be indicated by an index. The selection from the set of contexts may depend on various factors. For example, the selection of the coding context for the set index and / or mode index of the current block may depend on the information of the current block and the mode information of at least one adjacent block among the adjacent blocks of the current block. For example, the at least one adjacent block may include the adjacent block above (top) and / or to the left of the current block. In another example, the at least one adjacent block may also include the adjacent block to the upper left of the current block.

[0114] In various embodiments, referring to Method 1200 for an intra-predictive mode in video decoding shown in Figure 12, Method 1200 may include some or all of the following steps: Step 1210, receiving an encoded video bitstream relating to a block by an apparatus having a memory for storing instructions and a processor communicating with the memory; Step 1220, the apparatus derives at least one context relating to a set index and a mode index of a block based on information of the block and at least one of mode information of at least one adjacent block of the block, where the set index indicates an intra-mode set from a plurality of intra-mode sets and the mode index indicates an intra-predictive mode from an intra-mode set; Step 1230, the apparatus extracts the set index and mode index from the encoded video bitstream according to the at least one context; and / or Step 1240, the apparatus determines the intra-predictive mode of the block based on the set index and mode index. In various embodiments, an intramode set can refer to a set of intramodes having N intramodes, where N can be a positive integer, and an intramode set can have a set index that points to a set of intramodes in a group of intramode sets, and / or an intramode set can have a mode index that points to a specific intramode among the N intramodes in the intramode set. In some implementations, step 1220 may include the device deriving at least one context relating to the set index and mode index of the block based on at least one of the information of the block or the mode information of at least one adjacent block of the block, where the set index points to an intramode set from a plurality of intramode sets, and the mode index points to an intrapredictive mode from the intramode set.

[0115] In various embodiments, referring to Method 1300 for an intra-predictive mode in video encoding shown in Figure 13, Method 1300 may include some or all of the following steps: Step 1310, receiving a block to be encoded into an encoded video bitstream by an apparatus having a memory for storing instructions and a processor communicating with the memory; Step 1320, the apparatus derives at least one context relating to a set index and a mode index of a block based on at least one of the information of the block and the mode information of at least one adjacent block of the block, where the set index indicates an intra-mode set from a plurality of intra-mode sets, and the mode index indicates an intra-predictive mode from an intra-mode set; and / or Step 1330, the apparatus encodes the set index and mode index into an encoded video bitstream according to the at least one context. In some implementations, step 1320 may include the device deriving at least one context relating to the set index and mode index of the block based on at least one of the information of the block or the mode information of at least one adjacent block of the block, wherein the set index indicates an intra-mode set from a plurality of intra-mode sets, and the mode index indicates an intra-predictive mode from an intra-mode set.

[0116] In various embodiments of this disclosure, the size of a block (e.g., coding blocks, prediction blocks, or transformation blocks, but not limited to those described below) may refer to the width or height of the block. The width or height of the block may be an integer in pixels. In various embodiments of this disclosure, the size of a block may refer to the area size of the block. The area size of the block may be an integer calculated by multiplying the width of the block in pixels by the height of the block in pixels. In some various embodiments of this disclosure, the size of a block may refer to the maximum width or height of the block, the minimum width or height of the block, or the aspect ratio of the block. The aspect ratio of the block may be calculated by dividing the width of the block by the height, or by dividing the height of the block by the width.

[0117] In some implementations, the intra-mode mode type may include at least one of the following: directional mode, non-directional mode, smoothing mode (e.g., smooth, smooth_v, smooth_h), DC mode, PAETH mode, and / or a mode that generates a prediction sample according to a given prediction direction. In some other implementations, in loose classification, the directional mode may be broad and include any mode other than the smoothing mode (smooth, smooth_v, smooth_h), DC mode, or PAETH mode, and any mode that generates a prediction sample according to a given prediction direction. In some other implementations, the non-directional mode may include the smoothing mode (e.g., smooth, smooth_v, smooth_h), DC mode, PAETH mode, and lumens for chroma mode. In some other implementations, in loose classification, the non-directional mode may be broad and include any mode other than the directional mode.

[0118] In some implementations, the intra-prediction modes available for the current block may be divided / split into multiple intra-prediction mode sets. To obtain the intra-prediction modes for the current block, a mode set index indicating the mode set index of the current block's intra-prediction modes may first be signaled, and then a mode index indicating the index of the intra-prediction mode within the mode set may be signaled.

[0119] Herein, in various embodiments of the present disclosure, the “first” modeset refers not only to “one” modeset but also to the “first” modeset having the smallest modeset index, the “second” modeset refers not only to “another” modeset but also to the “second” modeset having the second smallest modeset index, and so on. For example, the number of intra-predictive modesets can be denoted by M, and the modeset index can be, for example, in the range of 1 to M, or 0 to M-1. If the modeset index is in the range of 1 to M, the “first” modeset is the “first” modeset having a modeset index of 1, the “second” modeset is the “second” modeset having a modeset index of 2, and so on. If the modeset index is in the range of 0 to M-1, the “first” modeset is the “first” modeset having a modeset index of 0, the “second” modeset is the “second” modeset having a modeset index of 1, and so on.

[0120] Herein, in various embodiments of the present disclosure, “XYZ is signaled” can mean that XYZ is encoded into an encoded bitstream in an encoding process, and / or, after the encoded bitstream has been transmitted from one device to another, “XYZ is signaled” can mean that XYZ is decoded / extracted from the encoded bitstream in a decoding process.

[0121] Here, in various embodiments of the present disclosure, “block” may refer to a prediction block, a coding block, a transformation block, or a coding unit (CU).

[0122] Referring to step 1210, the device may be the electronic device (530) in Figure 5 or the video decoder (810) in Figure 8. In some implementations, the device may be the decoder (633) within the encoder (620) in Figure 6. In other implementations, the device may be part of the electronic device (530) in Figure 5, part of the video decoder (810) in Figure 8, or part of the decoder (633) within the encoder (620) in Figure 6. The encoded video bitstream may be the encoded video sequence in Figure 8, or the intermediate encoded data in Figure 6 or Figure 7. The block may refer to a coding block or encoded block.

[0123] Referring to step 1220, the device can derive at least one context relating to the set index and mode index of the block based on at least one of the block information and the mode information of at least one adjacent block of the block, wherein the set index indicates an intra-mode set from a plurality of intra-mode sets, and the mode index indicates an intra-predictive mode from an intra-mode set. In some implementations, the block information has the size of the block, and / or the mode information of at least one adjacent block has at least one of the number of directional modes of the at least one adjacent block, and / or the mode set index of the at least one adjacent block. In one example, the at least one adjacent block may include the adjacent block above (top) and / or to the left of the current block. In another example, the at least one adjacent block may also include the adjacent block to the upper left of the current block.

[0124] In some implementations, the context for signaling intramodeset indices and mode indices within intramodesets is referred to as a cumulative density function (CDF), which depends on the mode information of the current block and / or its adjacent blocks, and this mode information includes, but is not limited to, the block size of the current block, the number of directional modes of at least one of the adjacent blocks, and / or the modeset index of at least one of the adjacent blocks.

[0125] In some other implementations, the device may further construct a list of intra-modes for the block and / or divide the intra-modes in the list into multiple intra-mode sets relating to the block. For example, an intra-mode list for the current block is constructed based on a set of predefined rules. After constructing the intra-mode list, the intra-predictive modes are divided into multiple intra-predictive mode sets according to their corresponding indices in the intra-mode list. To signal the intra-predictive modes for the current block, a mode set index indicating the mode set index of the current block's intra-predictive modes may be signaled first, and then a mode index indicating the index of the intra-predictive mode within the mode set may be signaled. The predefined rules may differ for different block sizes.

[0126] In various embodiments, step 1220 may include deriving context for set indices and mode indices based on the number of adjacent blocks coded in directional intra-predictive mode. For example, the context for signaling intra-mode set indices and / or mode indices within intra-mode sets depends on the number of adjacent blocks coded using directional intra-predictive mode.

[0127] In some embodiments, deriving contexts for set indices and mode indices based on the number of at least one adjacent blocks coded in directional intra-predictive mode may include deriving a first context as the context for set indices and mode indices in response to the number of at least one adjacent blocks coded in directional intra-predictive mode being zero, deriving a second context as the context for set indices and mode indices in response to the number of at least one adjacent blocks coded in directional intra-predictive mode being one, and / or deriving a third context as the context for set indices and mode indices in response to the number of at least one adjacent blocks coded in directional intra-predictive mode being greater than one.

[0128] In some implementations, the selection of context for the current block can be made from a total of three, four, or five different contexts. In a specific implementation example, if neither the upper nor left adjacent block is coded in directional intra-predictive mode, the first context (e.g., with a context index value of 0) is selected as the context for the current block's set index and mode index. If only one of the upper or left adjacent blocks is coded in directional intra-predictive mode, the second context (e.g., with a context index value of 1) is selected as the context for the current block's set index and mode index. In all other cases, the third context (e.g., with a context index value of 2) is selected as the context for the current block's set index and mode index. Such implementations are based on statistical observations of a strong correlation between the number of adjacent blocks coded in directional intra-predictive mode and the intra-predictive mode indicated by the current block's set index and mode index.

[0129] In some embodiments, deriving contexts for set indices and mode indices based on the number of at least one adjacent blocks coded in directional intra-predictive mode means deriving a first context as context for set indices and mode indices in response to the number of at least one adjacent blocks coded in directional intra-predictive mode being zero, deriving a second context as context for set indices and mode indices in response to the number of at least one adjacent blocks coded in directional intra-predictive mode being one, and the at least one coded in directional intra-predictive mode This may include deriving a third context as a context for set indices and mode indices in response to the number of adjacent blocks being 2 and at least one adjacent block being coded in the same directional intra-prediction mode, and / or deriving a fourth context as a context for set indices and mode indices in response to the number of at least one adjacent block being coded in a directional intra-prediction mode being 2 and at least one adjacent block being coded in different directional intra-prediction modes, or in response to the number of at least one adjacent block being coded in a directional intra-prediction mode being greater than 2.

[0130] In some implementations, if there are no adjacent blocks coded in directional intra-predictive mode, a first context (e.g., with a context index value of 0) is selected as the context for the current block's set index and mode index. If there is exactly one adjacent block coded in directional intra-predictive mode, a second context (e.g., with a context index value of 1) is selected as the context for the current block's set index and mode index. If there are two adjacent blocks coded in directional intra-predictive mode, and these two directional intra-predictive modes are identical, a third context (e.g., with a context index value of 2) is selected as the context for the current block's set index and mode index. In all other cases, a fourth context (e.g., with a context index value of 3) is selected as the context for the current block's set index and mode index.

[0131] In some embodiments, deriving contexts for set indices and mode indices based on the number of at least one adjacent blocks coded in directional intra-predictive mode may include deriving a first context as the context for set indices and mode indices in response to zero number of at least one adjacent blocks coded in directional intra-predictive mode; deriving a second context as the context for set indices and mode indices in response to one number of at least one adjacent blocks coded in directional intra-predictive mode; deriving a third context as the context for set indices and mode indices in response to two number of at least one adjacent blocks coded in directional intra-predictive mode; and / or deriving a fourth context as the context for set indices and mode indices in response to more than two number of at least one adjacent blocks coded in directional intra-predictive mode.

[0132] In some implementations, if there are no adjacent blocks coded in directional intra-predictive mode, a first context (e.g., with a context index value of 0) is selected as the context for the current block's set index and mode index. If there is only one adjacent block coded in directional intra-predictive mode, a second context (e.g., with a context index value of 1) is selected as the context for the current block's set index and mode index. If there are two adjacent blocks coded in directional intra-predictive mode, a third context (e.g., with a context index value of 2) is selected as the context for the current block's set index and mode index. If there are more than two adjacent blocks coded in directional intra-predictive mode, a fourth context (e.g., with a context index value of 3) is selected as the context for the current block's set index and mode index.

[0133] In some implementations, the directional intra-predictive mode includes at least one of either the interconnected mode or the intra-block copy mode. In some other implementations, the directional intra-predictive mode includes an intra-inter mode that has a directional intra mode component. For example, if adjacent blocks are coded in either the interconnected mode or the intra-block copy mode, it is considered a non-directional mode. In another example, if adjacent blocks are in the intra-inter mode, the intra-predictive mode used in the intra-inter mode is used to derive a context or determine the selection of a context from a set of contexts.

[0134] In various embodiments, step 1220 may include deriving context for set indices and mode indices based on the size of the block. For example, the context for signaling intramode set indices and / or mode indices within an intramode set depends on the block size of the current block. The block size may refer to the block width of the current block, the block height of the current block, the block area size (or width × height of the current block), the sum of the width and height of the current block, the maximum value between the width and height of the current block, or the minimum value between the width and height of the current block.

[0135] In some embodiments, deriving contexts for set indices and mode indices based on the size of the block involves deriving a first context as the context for set indices and mode indices in response to the block size being less than or equal to a first threshold, and / or deriving a second context as the context for set indices and mode indices in response to the block size being greater than the first threshold. For example, the first threshold is 8 × 8.

[0136] In some implementations, if the current block's block size is 8x8 or less, a first context (for example, one with a context index value of 0) is selected as the context for the current block's set index and mode index. Otherwise, a second context (for example, one with a context index value of 1) is selected as the context for the current block's set index and mode index.

[0137] In some embodiments, deriving contexts for set indices and mode indices based on the size of the block involves deriving a first context as the context for set indices and mode indices in response to the block size being less than a first threshold; deriving a second context as the context for set indices and mode indices in response to the block size being greater than or equal to the first threshold and less than a second threshold; deriving a third context as the context for set indices and mode indices in response to the block size being greater than or equal to the second threshold and less than a third threshold; and / or deriving a fourth context as the context for set indices and mode indices in response to the block size being greater than or equal to the third threshold. For example, the first threshold is an area of ​​64 (or 8×8), the second threshold is an area of ​​256 (or 16×16), and / or the third threshold is an area of ​​1024 (or 32×32).

[0138] In some implementations, if the current block's block area size is less than a first threshold (T1), a first context (e.g., one with a context index value of 0) is selected as the context for the current block's set index and mode index. If the current block's block area size is greater than or equal to T1 but less than a second threshold (T2), a second context (e.g., one with a context index value of 1) is selected as the context for the current block's set index and mode index. If the current block's block area size is greater than or equal to T2 but less than a third threshold (T3), a third context (e.g., one with a context index value of 2) is selected as the context for the current block's set index and mode index. In all other cases, a fourth context (e.g., one with a context index value of 3) is selected as the context for the current block's set index and mode index. For example, T1 might be set to 64, T2 to 256, and / or T3 to 1024. In another example, T1 may be set to 16, T2 to 64, and T3 to 256.

[0139] In various embodiments, the method 1200 may further optionally include the apparatus determining a set of acceptable intra-predictive modes for the block based on at least one of the mode information of the block's at least one adjacent block, or the size of the block.

[0140] In some implementations, the number of intra-predictive modes allowed for a single block may depend on the intra-predictive modes of its neighboring blocks and / or the block size of the current block. For example, depending on the intra-predictive modes of its neighboring blocks and / or the block size of the current block, one set of modes may be selected from all available sets, and only the intra-predictive modes within the selected set may be used to code the current block.

[0141] In various embodiments, determining a set of acceptable intra-prediction modes for the block includes determining a set of acceptable intra-prediction modes having a non-directional prediction mode and a nominal directional mode in response to the number of non-directional prediction modes for at least one adjacent block being two or more; determining a set of acceptable intra-prediction modes having a non-directional prediction mode and a nominal directional mode in response to the number of non-directional prediction modes for at least one adjacent block being two or more, and in response to the size of the block being greater than or equal to a first threshold; and / or determining a set of acceptable intra-prediction modes having all intra-prediction modes in response to the number of directional prediction modes for at least one adjacent block being one or more. For example, the first threshold is 8×8, 16×16, or 32×32.

[0142] In some implementations, if at least two of the intra-predictive modes of an adjacent block are non-directional modes, only those non-directional modes and the nominal directional mode are allowed for the current block.

[0143] In some other implementations, if at least two of the intra-predictive modes of the adjacent blocks are non-directional modes and / or the block size of the current block is greater than or equal to a threshold (TH1), then only those non-directional modes and nominal directional modes are allowed for the current block. For example, TH1 is set to 32x32. In another example, TH1 may be set to 8x8, 16x16, or 64x64.

[0144] In some other implementations, if at least one (or two) of the intra-prediction modes of the adjacent block is a directional intra-prediction mode, then all intra-prediction modes currently available for the block are allowed.

[0145] In various embodiments, Method 1200 may further optionally include the apparatus determining two or more acceptable intra-predictive modes for the block based on at least one of the mode information of at least one adjacent block of the block, or the size of the block. In some implementations, two or more mode sets may be selected from all available mode sets depending on the intra-predictive modes of its adjacent blocks and / or the block size of the current block, and only the intra-predictive modes within the selected two or more mode sets may be used to code the current block.

[0146] In some implementations, the adjacent blocks of the current block may include one or more of the following: the top (above or above) block of the current block, the block to the left of the current block, or the top-left (upper left or upper left) block of the current block.

[0147] Referring to step 1240, the device may determine the intra-predictive modes of the block based on the set index and the mode index. In some implementations, the device may construct a list of intra-modes of the block according to a predetermined rule based on the size of the block, and / or the device may divide the intra-modes in the list into multiple intra-mode sets for the block. The predetermined rule may be one of a set of predetermined rules, which may be selected based on the size of the block. In some other implementations, an intra-mode list is constructed for the current block based on the predetermined rule. After constructing the intra-mode list, the intra-predictive modes are divided into multiple intra-predictive mode sets according to their corresponding indices in the intra-mode list. To signal the intra-predictive modes of the current block, first a mode set index indicating the mode set index of the current block's intra-predictive modes may be signaled, and then a mode index indicating the index of the intra-predictive modes within the mode set may be signaled. The predetermined rules may differ depending on the block size, which can be determined by the number of samples within the block.

[0148] The embodiments of this disclosure may be used separately or in combination in any order. Furthermore, each of these methods (or embodiments), encoder, and decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-temporary computer-readable medium. The embodiments of this disclosure may be applied to a luma block or a chroma block, in which case these embodiments may be applied separately to two or more color components or to two or more color components together.

[0149] In this disclosure, the steps or operations in the various embodiments may be combined in any number or in any order as desired. In this disclosure, two or more steps or operations in the various embodiments may be performed in parallel.

[0150] The technologies described above can be implemented as computer software using computer-readable instructions, physically stored on one or more computer-readable media. For example, Figure 14 shows a computer system (2600) suitable for implementing a particular embodiment of the disclosed subject matter.

[0151] Computer software can be coded using any suitable machine code or computer language that can be assembled, compiled, linked, or subjected to similar mechanisms to produce code having instructions that can be executed directly or via interpretation, microcode execution, and similar means by one or more computer central processing units (CPUs), graphics processing units (GPUs), and similar devices.

[0152] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, game consoles, Internet of Things devices, and similar devices.

[0153] The components shown in Figure 14 with respect to the computer system (2600) are essentially illustrative and are not intended to imply any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Nor should the configuration of the components be construed as having any dependency or requirement on any one or combination of the components shown in this exemplary embodiment of the computer system (2600).

[0154] The computer system (2600) may include certain human interface input devices. Such human interface input devices may respond to input from one or more human users, for example, via tactile input (e.g., keystrokes, swipes, moving a data glove), audio input (e.g., voice, applause), visual input (e.g., gestures), or olfactory input (not shown). The human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., conversations, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still camera), or video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0155] The input human interface device may include one or more of the following: keyboard (2601), mouse (2602), trackpad (2603), touchscreen (2610), data glove (not shown), joystick (2605), microphone (2606), scanner (2607), and camera (2608) (only one of each is shown).

[0156] The computer system (2600) may also include certain human interface output devices. Such human interface output devices may stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (2610), data glove (not shown), or joystick (2605), although there may also be tactile feedback devices that do not function as input devices), audio output devices (e.g., speakers (2609), headphones (not shown), etc.), visual output devices (e.g., screens (2610) including CRT screens, LCD screens, plasma screens, and OLED screens (each having or not having touchscreen input functionality; each having or not having tactile feedback functionality; some of these may be able to output two-dimensional visual output, or output of four or more dimensions through means such as stereoscopic output, etc.), virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), etc.), and printers (not shown).

[0157] The computer system (2600) may also include, for example, optical media including CD / DVD ROM / RW (2620) having CD / DVD or similar media (2621), thumb drives (2622), removable hard drives or / or solid-state drives (2623), legacy magnetic media such as tapes and floppy disks (registered trademarks, not shown), specialized ROM / ASIC / PLD-based devices such as security dongles (not shown), and similar devices, and related media.

[0158] Those skilled in the art will also understand that the term “computer-readable medium” as used in connection with the matters disclosed herein does not include transmission media, carrier waves, or other transient signals.

[0159] The computer system (2600) may also include an interface (2654) to one or more communication networks (2655). The networks may be, for example, wireless, wired, or optical. The networks may further be local, wide-area, metropolitan, vehicle and industrial, real-time, or latency-tolerant. Examples of networks include local area networks such as Ethernet®, cellular networks including wireless LANs, GSM, 3G, 4G, 5G, LTE and similar technologies, wired or wireless wide-area digital television networks including cable TV, satellite TV, and terrestrial broadcast TV, and vehicle and industrial networks including CANbus. Certain networks generally require an external network interface adapter attached to a specific general-purpose data port or peripheral bus (2649) (e.g., a USB port on the computer system (2600)), while others are generally integrated into the core of the computer system (2600) by attachment to a system bus as described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (2600) can communicate with other entities. Such communication may be unidirectional reception only (e.g., broadcast television), unidirectional transmission only (e.g., CANbus to a specific CANbus device), or bidirectional to other computer systems, for example, using local or wide-area digital networks. Specific protocols and protocol stacks may be used on each of the networks and network interfaces as described above.

[0160] The aforementioned human interface device, human-accessible storage device, and network interface can be mounted on the core (2640) of the computer system (2600).

[0161] The core (2640) may include one or more central processing units (CPUs) (2641), graphics processing units (GPUs) (2642), specialized programmable processing units in the form of field-programmable gate arrays (FPGAs) (2643), hardware accelerators for specific tasks (2644), graphics adapters (2650), and the like. These devices may be connected via a system bus (2648) along with read-only memory (ROM) (2645), random access memory (2646), internal mass storage such as internal non-user-accessible hard drives, SSDs (2647), and similar devices (2647). In some computer systems, the system bus (2648) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, and similar devices. Peripheral devices may be attached either directly to the core's system bus (2648) or via a peripheral bus (2649). In one example, a screen (2610) can be connected to a graphics adapter (2650). Peripheral bus architectures include PCI, USB, and similar technologies.

[0162] The CPU (2641), GPU (2642), FPGA (2643), and accelerator (2644) can execute certain instructions that, in combination, constitute the aforementioned computer code. This computer code may be stored in ROM (2645) or RAM (2646). Transient data may also be stored in RAM (2646), while permanent data may be stored, for example, in internal mass storage (2647). Fast storage and retrieval to any of the memory devices may be made possible by the use of cache memory that may be associated with one or more CPUs (2641), GPUs (2642), mass storage (2647), ROMs (2645), RAM (2646), and similar devices.

[0163] A computer-readable medium may have computer code thereon for performing various computer implementation processes. The medium and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may be of a type that is well known and available to those skilled in the computer software technology.

[0164] As a non-limiting example, a computer system having architecture (2600), in particular core (2640), can provide functionality as a result of (one or more) processors (including CPUs, GPUs, FPGAs, accelerators, and similar) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be specific storage of core (2640) that is non-transient in nature, such as internal mass storage (2647) or ROM (2645) within the core, and media related to user-accessible mass storage as described above. Software implementing various embodiments of this disclosure can be stored in such devices and executed by core (2640). The computer-readable media may include one or more memory devices or chips, depending on the specific needs. The software may cause the core (2640) and, in particular, the processor within it (including CPUs, GPUs, FPGAs, and similar devices) to execute the specific processes or specific parts of the specific processes described herein, including defining data structures to be stored in RAM (2646) and modifying such data structures according to processes defined by the software. In addition, or alternatively, a computer system may provide functionality as a result of logic wired or otherwise embodied in circuits (e.g., accelerators (2644)) that can operate in place of or with the software to execute the specific processes or specific parts of the specific processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may include circuits containing software for execution (e.g., integrated circuits (ICs), etc.), circuits embodying logic for execution, or both, where appropriate. This disclosure includes preferred combinations of hardware and software.

[0165] While the specific invention has been described with reference to exemplary embodiments, this description is not intended to be limiting. Various modifications and further embodiments of these exemplary embodiments of the invention will become apparent to those skilled in the art from this description. As will be readily apparent to those skilled in the art, these modifications and various other modifications can be made with respect to the exemplary embodiments illustrated and described herein without departing from the spirit and scope of the invention. Accordingly, the appended claims are intended to cover such modifications and alternative embodiments. Certain proportions may be exaggerated in the illustrations, and other proportions may be minimized. Accordingly, this disclosure and the drawings should be considered illustrative and not limiting.

Claims

[Claim 1] A method for entropy coding for an intra-predictive mode in video decoding, the method being performed by a device having a memory for storing instructions and a processor for communicating with the memory, The steps include receiving an encoded video bitstream for a block, The step of deriving at least one context relating to the set index and mode index of the block based on at least one of the information of the block and the mode information of at least one adjacent block of the block, The aforementioned set index indicates an intra-mode set from a plurality of intra-mode sets. The mode index is derived by the step of indicating an intra-predicted mode from the intra-mode set, The steps include extracting the set index and the mode index from the encoded video bitstream according to at least one context, A step of determining the intra prediction mode of the block based on the set index and the mode index, A method of having.

Images