Improved intra mode coding

Abstract

The present disclosure provides various schemes for intra mode encoding and decoding of video. For example, a method for performing intra prediction on a current block of an encoded video stream is disclosed. The method may comprise: receiving an encoded video stream, the encoded video stream comprising a set index and a location index, the set index and the location index for intra prediction of a current block; determining an intra prediction mode corresponding to at least one adjacent block of the current block in the video stream as a reference mode; determining, based on the reference mode, a manner in which P predefined intra prediction modes are divided into M intra prediction mode sets, each intra prediction mode set comprising Ni position-indexed intra prediction modes, P and M being integers greater than 1, Ni being the number of prediction modes in the i-th intra prediction mode set, M being the number of prediction modes in the i-th intra prediction mode set, and M being the number of prediction modes in the i-th intra prediction mode set; i is an integer representing an intra prediction mode set index in the M intra prediction mode sets; identifying a target intra-frame prediction mode in P predefined intra-frame prediction modes based on the received set index and position index and the division mode; and decoding the current block of the encoded video stream based at least on a target intra prediction mode identified according to the set index and the location index of the current block.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims priority to U.S. non-provisional patent application No. 17 / 573,306 filed on January 11, 2022, which claims priority to U.S. provisional application No. 63 / 209,806, entitled “IMPROVED INTRA MODE CODING,” filed on June 11, 2021. The entire contents of both applications are incorporated herein by reference. Technical Field

[0003] The present disclosure describes a set of advanced video coding techniques. More specifically, the disclosed techniques relate to various schemes for intra-mode coding. Background Art

[0004] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. To the extent described in this background section, the work of the presently named inventors and aspects of the description that may not be prior art at the time of filing this application are neither explicitly nor implicitly admitted to be prior art to the present disclosure.

[0005] Video encoding and decoding can be performed using inter-picture prediction with motion compensation. An uncompressed digital video may include a series of pictures, each picture having spatial dimensions of, for example, 1920×1080 luminance samples and associated fully sampled or subsampled chrominance samples. The series of pictures may have a fixed or variable picture rate (or frame rate), for example, 60 pictures per second or 60 frames per second. Uncompressed video has specific bit rate requirements for streaming or data processing. For example, a video with a pixel resolution of 1920x1080, a frame rate of 60 frames per second, chrominance subsampling of 4:2:0, and 8 bits per pixel per color channel requires a bandwidth of nearly 1.5 Gbit / s. One hour of such video requires more than 600 GBytes of storage space.

[0006] One purpose of video encoding and decoding is to reduce redundancy in an uncompressed input video signal by compression. In some cases, compression helps reduce the above bandwidth and / or storage space requirements by two orders of magnitude or more. Lossless compression and lossy compression and their combinations can be used. Lossless compression refers to such a technology, in which an exact copy of the original signal can be reconstructed from the compressed original signal via a decoding process. Lossy compression refers to an encoding / decoding process in which the original video information is not fully retained during encoding and cannot be fully restored during decoding. When lossy compression is used, the reconstructed signal may be different from the original signal, but the distortion between the original signal and the reconstructed signal is small enough that the reconstructed signal is useful for the intended application, despite some information loss. In the case of video, lossy compression is widely used in many applications. The amount of distortion allowed depends on the application. For example, users of certain consumer video streaming applications may tolerate higher distortion than users of movie or television broadcast applications. The compression ratio achievable by a specific encoding algorithm can be selected or adjusted to reflect various distortion tolerances: higher tolerable distortion generally allows encoding algorithms that produce higher losses and higher compression ratios.

[0007] Video encoders and decoders may utilize several broad categories and steps of techniques, including, for example, motion compensation, Fourier transforms, quantization, and entropy coding.

[0008] Video codec techniques may include a technique known as intra-coding. In intra-coding, sample values ​​are represented without reference to samples or other data from a previously reconstructed reference picture. In some video codecs, pictures are spatially subdivided into blocks of samples. When all sample blocks are encoded in intra-mode, the picture may be referred to as an intra-picture. Intra-pictures and their derivatives (e.g., independent decoder refresh pictures) can be used to reset the decoder state and can therefore be used as the first picture in a coded video bitstream and video session, or as a still image. After intra-prediction, the samples of the block can then be transformed to the frequency domain, and the transform coefficients so generated can be quantized before entropy coding. Intra-prediction represents a technique for minimizing sample values ​​in the pre-transform domain. In some cases, the smaller the DC value after transformation, the smaller the AC coefficient, and the fewer bits required to represent the entropy-coded block at a given quantization step size.

[0009] For example, conventional intra-frame coding known from MPEG-2 generation coding techniques does not use intra-frame prediction. However, some newer video compression techniques include techniques that attempt to encode / decode blocks based on, for example, surrounding sample data and / or metadata, which are obtained during encoding and / or decoding of spatially adjacent blocks and precede the data block being intra-coded or decoded in decoding order. Such techniques are hereinafter referred to as "intra-frame prediction" techniques. Note that, at least in some cases, intra-frame prediction uses only reference data from the current picture being reconstructed, and does not use reference data from other reference pictures.

[0010] There can be many different forms of intra-frame prediction. When more than one such technique is available in a given video coding technique, the technique used may be referred to as an intra-frame prediction mode. One or more intra-frame prediction modes may be provided in a particular codec. In some cases, a mode may have sub-modes and / or may be associated with various parameters, and the mode / sub-mode information and intra-frame coding parameters of a video block may be encoded individually or collectively, included in a mode codeword. For a given mode, sub-mode, and / or parameter combination, which codeword is used will have an impact on the coding efficiency obtained by intra-frame prediction, and the entropy coding technique used to convert the codeword into a bitstream will also have an impact.

[0011] H.264 introduced specific intra prediction modes, which were improved in H.265 and further improved in newer coding techniques, such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Sets (BMS). Typically, for intra prediction, the values ​​of neighboring samples that have become available can be used to form a prediction block. For example, the available values ​​of a particular set of neighboring samples along a particular direction and / or line can be copied into the predictor block. A reference to the direction in use can be encoded in the bitstream, or it can be predicted itself.

[0012] refer to Figure 1A , a subset of 9 prediction directions specified in the 33 possible intra-frame prediction directions of H.265 (corresponding to the 33 angular modes of the 35 intra-frame modes specified in H.265) is depicted in the lower right. The point (101) where the arrows converge represents the sample being predicted. The arrows represent the direction in which the neighboring samples are used to predict the sample at 101. For example, arrow (102) indicates that sample (101) is predicted from one or more neighboring samples to the upper right at an angle of 45° to the horizontal direction. Similarly, arrow (103) indicates that sample (101) is predicted from one or more neighboring samples to the lower left of sample (101) at an angle of 22.5° to the horizontal direction.

[0013] Still reference Figure 1A, a square block (104) of 4×4 samples is depicted in the upper left (indicated by the thick dashed line). The square block (104) includes 16 samples, each sample labeled "S", its position in the Y dimension (e.g., row index) and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. Since the size of the block is 4×4 samples, S44 is located in the lower right corner. Reference samples are also shown following a similar numbering scheme. The reference samples are labeled (104) with R, their Y position (e.g., row index) and X position (column index) relative to the block. In H.264 and H.265, prediction samples adjacent to the block being reconstructed are used.

[0014] The intra picture prediction of block 104 may start by copying reference sample values ​​from neighboring samples indicated by the signaled prediction direction. For example, assume that the encoded video bitstream contains signaling indicating the prediction direction of arrow (102) for this block 104 - i.e., predicting one or more prediction samples from the upper right sample at an angle of 45° to the horizontal. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Sample S44 is then predicted from reference sample R08.

[0015] In some cases, the values ​​of multiple reference samples may be combined, for example, by interpolation, in order to calculate the reference sample; in particular, when the direction is not divisible by 45°.

[0016] As video coding techniques have continued to develop, the number of possible directions has increased. For example, in H.264 (2003), nine different directions were available for intra prediction. This was increased to 33 in H.265 (2013), and at the time of this disclosure, JEM / VVC / BMS can support up to 65 directions. Experimental studies have been conducted to help identify the most appropriate intra prediction directions, and certain techniques in entropy coding can be used to encode those most appropriate directions with a small number of bits, accepting some bit loss in the direction. In addition, the direction itself can sometimes be predicted based on the neighboring directions used in the intra prediction of the already decoded neighboring blocks.

[0017] Figure 1B A schematic diagram (180) is shown depicting 65 intra prediction directions according to JEM to illustrate the increase in the number of prediction directions in various coding techniques developed over time.

[0018] The manner in which bits representing the intra prediction direction are mapped to the prediction direction in the coded video bitstream can vary depending on the video coding technique; and can range from, for example, a simple direct mapping of prediction directions to intra prediction modes to codewords, to complex adaptive schemes involving the most probable mode, and similar techniques. However, in all cases, some directions of intra prediction are statistically less likely to occur in the video content than some other directions. Since the goal of video compression is to reduce redundancy, in a well-designed video coding technique, those less likely directions will be represented by more bits than more likely directions.

[0019] Inter-picture prediction or inter-prediction can be based on motion compensation. In motion compensation, sample data from a previously reconstructed picture or a portion thereof (reference picture) can be used to predict a newly reconstructed picture or picture portion (e.g., block) after being spatially shifted in the direction indicated by a motion vector (hereinafter MV). In some cases, the reference picture can be the same as the picture currently being reconstructed. The MV can have two dimensions, X and Y, or three dimensions, the third dimension being an indication of the reference picture in use (similar to the time dimension).

[0020] In some video compression techniques, the current MV applicable to a region of sample data may be predicted from other MVs, for example, from other MVs related to other regions of sample data that are spatially adjacent to the region being reconstructed and precede the current MV in decoding order. Doing so can significantly reduce the total amount of data required to encode the MV by removing redundancy in the related MVs, thereby improving compression efficiency. MV prediction can work effectively, for example, because when encoding an input video signal derived from a camera (called natural video), there is a statistical probability that regions larger than the region to which a single MV can be applied move in similar directions in the video sequence, so in some cases, similar motion vectors derived from MVs of neighboring regions can be used for prediction. This results in the MV found for a given region being similar or identical to the MV predicted from the surrounding MVs. After entropy coding, such an MV can again be represented with fewer bits than if the MV was directly encoded rather than predicted from neighboring MVs. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., MV) derived from the original signal (i.e., sample stream). In other cases, the MV prediction itself may be lossy, for example, due to round-off errors when computing the predicted value from several surrounding MVs.

[0021] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016). Among the many MV prediction mechanisms specified by H.265, a technique referred to as "spatial merging" is described below.

[0022] Specifically, refer to Figure 2 , the current block (201) includes samples that the encoder found during motion search to be predictable from a previous block of the same size that has been spatially shifted. Instead of encoding the MV directly, the MV can be derived from metadata associated with one or more reference pictures, for example, from the most recent (in decoding order) reference picture, using the MV associated with any of the five surrounding samples (denoted A0, A1 and B0, B1, B2 (202 to 206, respectively). In H.265, MV prediction can use a predictor from the same reference picture that a neighboring block is using. Summary of the invention

[0023] Aspects of the present disclosure provide methods and apparatus for video encoding and decoding, including various schemes for intra-mode encoding.

[0024] In some example implementations, a method for performing intra-frame prediction on a current block of an encoded video stream is disclosed. The method may include: receiving an encoded video stream, the encoded video stream including a set index and a position index, the set index and the position index being used for intra-frame prediction of the current block; determining an intra-frame prediction mode corresponding to at least one neighboring block of the current block in the video stream as a reference mode; determining a way to divide P predefined intra-frame prediction modes into M intra-frame prediction mode sets based on the reference mode, each intra-frame prediction mode set including N i The intra prediction mode indexed by the position, P and M are integers greater than 1, N i is the number of prediction modes in the i-th intra-frame prediction mode set, i is an integer representing the intra-frame prediction mode set index in the M intra-frame prediction mode sets; based on the received set index and position index and the division method, identifying the target intra-frame prediction mode in P predefined intra-frame prediction modes; and decoding the current block of the encoded video stream based on at least the target intra-frame prediction mode identified according to the set index and position index of the current block.

[0025] In the above implementation, N max is a power of 2, N max YesN i In some implementations, when M is greater than 2, N i is selected from a first quantity and a second quantity. In some implementations, N iis the same value for all M intra-prediction mode sets. In some implementations, one of the M intra-prediction mode sets includes the second number of intra-prediction modes, and all other intra-prediction mode sets in the M intra-prediction mode sets include the first number of intra-prediction modes. In some implementations, M=4, and N max =16. In some implementations, P=61, the first number is 16, and the second number is 13.

[0026] In some of the above example implementations, the P predefined intra prediction modes may include a subset of non-directional intra prediction modes; and the relative order of the subset of non-directional intra prediction modes in the set of M intra prediction modes is fixed and is independent of the reference mode.

[0027] In some of the above example implementations, the subsets of non-directional intra prediction modes are all in the intra prediction mode set located at the first position among the M intra prediction mode sets, and are irrelevant to the reference mode.

[0028] In some example implementations above, the subset of non-directional intra prediction modes includes a DC prediction mode, which is always placed first in a set of intra prediction modes at the first position, regardless of a reference mode.

[0029] In some of the example implementations above, a recursive-filtering-based intra prediction mode is represented as a sub-mode of the DC prediction mode. In addition, a subset of non-directional intra prediction modes includes a SMOOTH prediction mode, which is always positioned as the second in the set of intra prediction modes located in the first position, regardless of the reference mode. In some implementations, a subset of non-directional intra prediction modes includes a SMOOTH_V prediction mode, which is always positioned as the third in the set of intra prediction modes located in the first position, regardless of the reference mode.

[0030] In some of the example implementations above, the subset of non-directional intra prediction modes includes the SMOOTH_H prediction mode, which is always positioned as the fourth in the set of intra prediction modes at the first position, regardless of the reference mode.

[0031] In some of the example implementations above, the subset of non-directional intra prediction modes includes the PAETH prediction mode, which is always positioned as the fifth one in the set of intra prediction modes at the first position, regardless of the reference mode.

[0032] In some of the above example implementations, the P predefined intra-frame prediction modes may include an intra-frame prediction mode based on recursive filtering; the intra-frame prediction mode based on recursive filtering is placed in sequence as an intra-frame prediction mode set located at the first position in the M intra-frame prediction mode sets, regardless of the reference mode; and the intra-frame prediction mode based on recursive filtering is represented as being independent of any DC intra-frame prediction mode.

[0033] In some of the above example implementations, when the target intra prediction mode identified for the current block is a recursive filtering-based intra prediction mode, the method may further include determining to select one of a group of recursive filtering-based intra prediction modes based on a set index and a position index, respectively.

[0034] In some of the above example implementations, the intra prediction mode based on recursive filtering is always placed sixth in the prediction mode set located in the first position frame, regardless of the reference mode. In some of the above example implementations, the P predefined intra prediction modes include at least one directional intra prediction mode, at least one non-directional intra prediction mode, and at least one luma-to-chroma intra prediction mode.

[0035] Aspects of the present disclosure also provide a device including a processing circuit configured to perform any of the above-mentioned method implementations for video decoding and / or encoding. Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions, which, when executed by a computer to perform video decoding and / or encoding, cause the computer to perform any of the above-mentioned method implementations for video decoding and / or encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Further features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0037] Figure 1A A schematic diagram showing an example set of examples of intra-prediction directional modes.

[0038] Figure 1B A diagram showing exemplary intra prediction directions;

[0039] Figure 2 A schematic diagram showing a current block and its surrounding spatial merging candidates for motion vector prediction in an example;

[0040] Figure 3 A schematic diagram showing a simplified block diagram of a communication system (300) according to an example embodiment;

[0041] Figure 4 A schematic diagram showing a simplified block diagram of a communication system (400) according to an example embodiment;

[0042] Figure 5 A schematic diagram showing a simplified block diagram of a video decoder according to an example embodiment;

[0043] Figure 6 A schematic diagram showing a simplified block diagram of a video encoder according to an example embodiment;

[0044] Figure 7 shows a block diagram of a video encoder according to another example embodiment;

[0045] Figure 8 shows a block diagram of a video decoder according to another example embodiment;

[0046] Fig. 9 A directional intra prediction mode according to an example embodiment of the present disclosure is shown;

[0047] Fig.10 A non-directional intra prediction mode according to an example embodiment of the present disclosure is shown;

[0048] Fig.11 A recursive intra prediction mode according to an example embodiment of the present disclosure is shown;

[0049] Fig.12 Demonstrating transform block partitioning and scanning of intra prediction blocks according to an example embodiment of the present disclosure;

[0050] Fig.13 Demonstrating transform block partitioning and scanning of an inter-prediction block according to an example embodiment of the present disclosure;

[0051] Fig.14 shows a flow chart according to an example embodiment of the present disclosure;

[0052] Fig.15 A schematic diagram showing a computer system according to an example embodiment of the present disclosure; DETAILED DESCRIPTION

[0053] Figure 3 A simplified block diagram of a communication system (300) according to an embodiment of the present disclosure is shown. The communication system (300) includes a plurality of terminal devices capable of communicating with each other via, for example, a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). Figure 3In the example of , a first pair of terminal devices (310) and (320) can perform unidirectional transmission of data. For example, the terminal device (310) can encode video data (e.g., a video picture stream captured by the terminal device (310)) for transmission to another terminal device (320) via the network (350). The encoded video data can be transmitted in the form of one or more encoded video bit streams. The terminal device (320) can receive the encoded video data from the network (350), decode the encoded video data to restore the video picture, and display the video picture based on the restored video data. The unidirectional data transmission can be implemented in a media service application, etc.

[0054] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of encoded video data that can be implemented, for example, during a video conferencing application. For the bidirectional transmission of data, in one example, each of the terminal devices (330) and (340) can encode video data (e.g., a video picture stream captured by the terminal device) for transmission to the other of the terminal devices (330) and (340) via the network (350). Each of the terminal devices (330) and (340) can also receive the encoded video data transmitted by the other of the terminal devices (330) and (340), and can decode the encoded video data to restore the video picture, and can display the video picture on an accessible display device based on the restored video data.

[0055] exist Figure 3 In the example of , terminal devices (310), (320), (330) and (340) can be implemented as servers, personal computers and smart phones, but the application of the basic principles of the present disclosure may not be limited to this. Embodiments of the present disclosure can be implemented in desktop computers, laptop computers, tablet computers, media players, wearable computers, dedicated video conferencing devices, etc. Network (350) represents any number or type of networks that transmit encoded video data between terminal devices (310), (320), (330) and (340), including, for example, line (wired) and / or wireless communication networks. The communication network (350) can exchange data in circuit switching, packet switching and / or other types of channels. Representative networks include telecommunication networks, local area networks, wide area networks and / or the Internet. For the purpose of the current discussion, the architecture and topology of the network (350) may be unimportant to the operation of the present disclosure unless explicitly explained in this document.

[0056] As examples of applications of the disclosed subject matter, Figure 4The video encoder and video decoder are shown placed in a video streaming environment. The disclosed subject matter is equally applicable to other video applications, including, for example, video conferencing, digital television, broadcasting, gaming, virtual reality, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0057] The video streaming system may include: a video capture subsystem (413), which may include a video source (401), such as a digital camera; creating an uncompressed video picture or image stream (402). In one example, the video picture stream (402) includes samples recorded by the digital camera of the video source 401. The video picture stream (402) is depicted as a thick line to emphasize the high amount of data when compared to the encoded video data (404) (or encoded video bitstream), and can be processed by an electronic device (420) including a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to implement or implement various aspects of the disclosed subject matter as described in more detail below. The encoded video data (404) (or encoded video bitstream (404)) is depicted as a thin line to emphasize the lower amount of data when compared to the uncompressed video picture stream (402), and can be stored on a streaming server (405) for future use, or directly stored to a downstream video device (not shown). One or more streaming client subsystems (e.g., Figure 4 The client subsystems (406) and (408) in the video server (405) can access the streaming server (405) to retrieve copies (407) and (409) of the encoded video data (404). The client subsystem (406) may include, for example, a video decoder (410) in an electronic device (430). The video decoder (410) decodes the input copy (407) of the encoded video data and creates an output video picture stream (411) that is uncompressed and can be presented on a display (412) (e.g., a display screen) or other presentation device (not shown). The video decoder 410 can be configured to perform some or all of the various functions described in the present disclosure. In some streaming systems, the encoded video data (404), (407) and (409) (e.g., a video bitstream) can be encoded according to certain video encoding / compression standards. Examples of these standards include ITU-T Recommendation H.265. In one example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed subject matter can be used in the context of VVC as well as other video coding standards.

[0058] Note that the electronic devices (420) and (430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[0059] Figure 5 A block diagram of a video decoder (510) according to any embodiment of the present disclosure below is shown. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuit). The video decoder (510) may be used to replace Figure 4 A video decoder (410) is shown in an example.

[0060] The receiver (531) may receive one or more encoded video sequences to be decoded by the video decoder (510). In the same or another embodiment, the encoded video sequences may be decoded one at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. Each video sequence may be associated with multiple video frames or images. The encoded video sequence may be received from a channel (501), which may be a hardware / software link to a storage device storing the encoded video data or a stream source transmitting the encoded video data. The receiver (531) may receive the encoded video data and other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their corresponding processing circuits (not shown). The receiver (531) may separate the encoded video sequence from the other data. To combat network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) may be implemented as part of the video decoder (510). In other applications, there may be a buffer memory (not shown) outside the video decoder (510) and separate from the video decoder. In other applications, there may be a buffer memory (not shown) outside the video decoder (510), for example, for the purpose of combating network jitter, and there may be another additional buffer memory (515) inside the video decoder (510), for example, for handling playback timing. When the receiver (531) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous network, the buffer memory (515) may not be required, or may be small. For use on a best-effort packet network such as the Internet, a buffer memory (515) of sufficient size may be required, which may be relatively large. Such a buffer memory may be implemented with an adaptive size and may be implemented at least in part in an operating system or similar element (not shown) external to the video decoder (510).

[0061] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from the encoded video sequence. The categories of these symbols include information for managing the operation of the video decoder (510) and potentially controlling a presentation device such as a display (512) (e.g., a display screen), which may or may not be an integral part of the electronic device (530), but may be coupled to the electronic device (530), such as Figure 5 As shown. The control information for the rendering device may be in the form of a supplemental enhancement information (SEI message) or a video usability information (VUI) parameter set fragment (not shown). The parser (520) may parse / entropy decode the encoded video sequence received by the parser (520). The entropy coding of the encoded video sequence may be based on a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a set of subgroup parameters of at least one pixel subgroup in the video decoder from the encoded video sequence based on at least one parameter corresponding to the subgroup. The subgroup may include a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (Cu), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (520) may also extract information from the encoded video sequence, such as transform coefficients (e.g., Fourier transform coefficients), quantizer parameter values, motion vectors, etc.

[0062] The parser (520) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to create symbols (521).

[0063] Depending on the type of coded video picture or part thereof (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors, the reconstruction of the symbol (521) may involve a number of different processing or functional units. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the coded video sequence by the parser (520). For simplicity, such subgroup control information flow between the parser (520) and the following multiple processing or functional units is not described.

[0064] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into a number of functional units as described below. In a practical implementation operating under commercial constraints, many of these functional units interact closely with each other and may be at least partially integrated with each other. However, in order to clearly describe the various functions of the disclosed subject matter, the conceptual subdivision into the following functional units is adopted in the following disclosure.

[0065] The first unit may include a scalar / inverse transform unit (551). The scalar / inverse transform unit (551) may receive quantized transform coefficients and control information, including information on what type of inverse transform to use, block size, quantization factors / parameters, quantization scaling matrices, etc., as symbols (521) from the parser (520). The scalar / inverse transform unit (551) may output blocks including sample values, which may be input into the aggregator (555).

[0066] In some cases, the output samples of the scalar / inverse transform unit (551) may belong to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed image, but may use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates a block of the same size and shape as the block being reconstructed using surrounding block information that has already been reconstructed and stored in a current picture buffer (558). The current picture buffer (558) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. In some implementations, the aggregator (555) may add the prediction information that has been generated by the intra-prediction unit (552) to the output sample information provided by the scalar / inverse transform unit (551) on a per-sample basis.

[0067] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to an inter-frame coded and possibly motion compensated block. In this case, the motion compensated prediction unit (553) may access the reference picture memory (557) to obtain samples for prediction. After the extracted samples are motion compensated according to the symbols (521) associated with the block, these samples may be added to the output of the scaler / inverse transform unit (551) by the aggregator (555) (the output of unit 551 may be referred to as residual samples or residual signal) to generate output sample information. The addresses within the reference picture memory (557) from which the motion compensated prediction unit (553) obtains the predicted samples may be controlled by motion vectors, and the motion compensated prediction unit (553) may obtain these addresses in the form of symbols (521), which may have, for example, X, Y components (offset) and reference picture components (time). When sub-sampled accurate motion vectors are used, motion compensation may also include interpolation of sample values ​​obtained from the reference picture memory (557) and is also associated with motion vector prediction mechanisms, etc.

[0068] The output samples of the aggregator (555) may be subjected to various loop filtering techniques in a loop filter unit (556). The video compression techniques may include loop filtering techniques that are controlled by parameters contained in the coded video sequence (also referred to as the coded video bitstream) and made available to the loop filter unit (556) as symbols (521) from the parser (520), but may also be responsive to meta information obtained during decoding of a previous (in decoding order) portion of the coded picture or coded video sequence and to previously reconstructed and loop filtered sample values. Several types of loop filters may be included as part of the loop filter unit 556 in various orders, as will be described in further detail below.

[0069] The output of the loop filter unit (556) may be a sample stream that may be output to a rendering device (512) and stored in a reference picture memory (557) for use in future inter-picture prediction.

[0070] Once fully reconstructed, certain coded pictures can be used as reference pictures for future inter-picture prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed, and the coded picture has been identified as a reference picture (e.g., by a parser (520)), the current picture buffer (558) can become part of the reference picture memory (557), and a new current picture buffer can be reallocated before starting reconstruction of the next coded picture.

[0071] The video decoder (510) may perform decoding operations according to a predetermined video compression technique employed in a standard such as ITU-T Rec. H.265. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used in the sense that the encoded video sequence conforms to the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, the profile may select certain tools from all tools available in the video compression technique or standard as the only tools available under the profile. To conform to the standard, the complexity of the encoded video sequence may be within a range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (e.g., measured in megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the level may be further limited by the metadata of the hypothetical reference decoder (HRD) specification and HRD buffer management signaled in the encoded video sequence.

[0072] In some example embodiments, the receiver (531) may receive additional (redundant) data with the encoded video. The additional data may be included as part of the encoded video sequence. The video decoder (510) may use the additional data to correctly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0073] Figure 6 A block diagram of a video encoder (603) according to an example embodiment of the present disclosure is shown. The video encoder (603) may be included in an electronic device (620). The electronic device (620) may also include a transmitter (640) (e.g., a transmitting circuit). The video encoder (603) may be used to replace Figure 4 The video encoder (403) in the example.

[0074] The video encoder (603) can be used to obtain the video source (601) (which is not Figure 6 In another example, the video source (601) may be implemented as a part of the electronic device (620) to receive the video samples. The video source may capture the video pictures to be encoded by the video encoder (603). In another example, the video source (601) may be implemented as a part of the electronic device (620).

[0075] The video source (601) may provide a source video sequence to be encoded by the video encoder (603) in the form of a digital video sample stream, which may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (601) may be a storage device capable of storing previously prepared videos. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. Video data may be provided as multiple individual pictures or images that impart motion when viewed sequentially. The pictures themselves may be organized as a spatial array of pixels, where each pixel may include one or more samples, depending on the sampling structure, color space, etc. in use. The relationship between pixels and samples may be readily understood by those skilled in the art. The following description focuses on samples.

[0076] According to some example embodiments, the video encoder (603) may encode and compress pictures of a source video sequence into an encoded video sequence (643) in real time or under any other time constraints required by the application. Implementing an appropriate encoding speed constitutes a function of the controller (650). In some embodiments, the controller (650) may be functionally coupled to and control other functional units as described below. For simplicity, the coupling is not described. The parameters set by the controller (650) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technique, ...), picture size, picture group (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured to have other suitable functions related to the video encoder (603) optimized for a specific system design.

[0077] In some example embodiments, the video encoder (603) is configured to operate in an encoding loop. As an oversimplified description, in one example, the encoding loop may include a source encoder (630) (e.g., responsible for creating symbols, e.g., a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols in a manner similar to what the (remote) decoder would also create to create sample data, even though the embedded decoder 633 processes the video stream encoded by the source encoder 630 without entropy coding (because any compression between the symbols and the encoded video bitstream in entropy coding may be lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since the decoding of the symbol stream results in a bit-accurate result independent of the decoder location (local or remote), the contents in the reference picture memory (634) are also bit-accurate between the local encoder and the remote encoder. In other words, when prediction is used during decoding, the predicted part of the encoder "sees" as reference picture samples exactly the same sample values ​​as the sample values ​​"seen" by the decoder. This basic principle of reference picture synchronization (and the resulting drift if synchronization cannot be maintained, e.g., due to channel errors) is used to improve encoding quality.

[0078] The operation of the "local" decoder (633) may be identical to the operation of a "remote" decoder such as the video decoder (510), described above in conjunction with Figure 5 However, a brief reference to Figure 5, since symbols are available and the symbol encoding / decoding of the encoded video sequence by the entropy encoder (645) and the parser (520) can be lossless, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and the parser (520), may not be fully implemented in the local decoder (633) in the encoder.

[0079] It can be observed at this point that, in addition to the parsing / entropy decoding that may exist in the decoder, any decoder technology does not necessarily need to exist in the corresponding encoder in a substantially identical functional form. For this reason, the disclosed subject matter can sometimes focus on the decoder operation, which is similar to the decoding portion of the encoder. Therefore, the description of the encoder technology can be simplified because these technologies are the inverse of the fully described decoder technology. Only in certain areas or aspects, a more detailed description of the encoder is provided below.

[0080] During operation, in some example implementations, the source encoder (630) may perform motion compensated predictive coding, which predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures". In this manner, the encoding engine (632) encodes the differences (or residuals) in the color channels between pixel blocks of the input picture and pixel blocks of a reference picture that may be selected as a prediction reference for the input picture. The term "residual" and its adjective form "residual" may be used interchangeably.

[0081] The local video decoder (633) can decode the encoded video data of the picture that can be designated as the reference picture based on the symbols created by the source encoder (630). The operation of the encoding engine (632) can advantageously be a lossy process. When the encoded video data can be decoded at the video decoder ( Figure 6 When decoded at a local (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding process that may be performed by the video decoder on the reference picture, and may cause the reconstructed reference picture to be stored in the reference picture cache (634). In this way, the video encoder (603) may locally store copies of the reconstructed reference pictures that have the same content (absent transmission errors) as the reconstructed reference pictures that will be obtained by the far-end (remote) video decoder.

[0082] The predictor (635) may perform a prediction search on the encoding engine (632). That is, for a new picture to be encoded, the predictor (635) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may be used as appropriate prediction references for the new picture. The predictor (635) may operate on a sample block-pixel block basis to find an appropriate prediction reference. In some cases, as determined by the search results obtained by the predictor (635), the input picture may have prediction references extracted from multiple reference pictures stored in the reference picture memory (634).

[0083] The controller (650) may manage encoding operations of the source encoder (630), including, for example, the setting of parameters and sub-group parameters for encoding video data.

[0084] The outputs of all the aforementioned functional units may undergo entropy coding in an entropy encoder (645). The entropy encoder (645) converts the symbols generated by the various functional units into a coded video sequence by lossless compression of the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0085] The transmitter (640) can buffer the encoded video sequence created by the entropy encoder (645) in preparation for transmission via a communication channel (660), which can be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or ancillary data streams (source not shown).

[0086] The controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can assign a specific coded picture type to each coded picture, which can affect the coding techniques that can be applied to the corresponding picture. For example, a picture can generally be assigned one of the following picture types:

[0087] An intra picture (I picture) may be a picture that is encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, independent decoder refresh ("IDR") pictures. Those skilled in the art are aware of those variants of I pictures and their corresponding applications and features.

[0088] A prediction picture (P picture) may be a picture that is encoded and decoded using intra prediction or inter prediction by predicting sample values ​​of each block using at most one motion vector and a reference index.

[0089] Bidirectional prediction pictures (B pictures) can be pictures that use up to two motion vectors and reference indices to predict the sample values ​​of each block, and are encoded and decoded using intra-frame prediction or inter-frame prediction. Similarly, multi-prediction pictures can use more than two reference pictures and related metadata to reconstruct a single block.

[0090] The source picture may typically be spatially subdivided into a plurality of blocks of samples (e.g., each block of samples being a 4×4, 8×8, 4×8, or 16×16 block of samples) and encoded on a block-by-block basis. Blocks may be predictively encoded with reference to other (already encoded) blocks determined by the coding allocation applied to the corresponding picture of the block. For example, blocks of an I picture may be non-predictively encoded, or may be predictively encoded (spatial prediction or intra-frame prediction) with reference to already encoded blocks of the same picture. Blocks of pixels of a P picture may be predictively encoded via spatial prediction or via temporal prediction with reference to a previously encoded reference picture. Blocks of a B picture may be predictively encoded via spatial prediction or via temporal prediction with reference to one or two previously encoded reference pictures. For other purposes, source pictures or intermediately processed pictures may be subdivided into other types of blocks. The division of coding blocks and other types of blocks may or may not follow the same approach, as described in further detail below.

[0091] The video encoder (603) may perform encoding operations according to a predetermined video encoding technique or standard (e.g., ITU-T Rec. H.265). In its operation, the video encoder (603) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard being used.

[0092] In some example embodiments, the transmitter (640) may transmit additional data along with the encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data (e.g., redundant pictures and slices), SEI messages, VUI parameter set fragments, etc.

[0093] Video can be captured as multiple source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits temporal or other correlations between pictures. In one example, a particular picture in encoding / decoding, referred to as the current picture, is divided into blocks. When a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture can be encoded by a vector called a motion vector. In the case of using multiple reference pictures, the motion vector points to a reference block in a reference picture and can have a third dimension that identifies the reference picture.

[0094] In some example embodiments, a bidirectional prediction technique may be used in inter-picture prediction. According to the bidirectional prediction technique, two reference pictures are used, for example, a first reference picture and a second reference picture, both of which precede the current picture in the video in decoding order (but may be in the past or future in display order, respectively). A block in the current picture may be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block may be jointly predicted by a combination of the first reference block and the second reference block.

[0095] In addition, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0096] According to some example embodiments of the present disclosure, prediction is performed in units of blocks, for example, inter-picture prediction and intra-picture prediction. For example, a picture in a video picture sequence is divided into coding tree units (CTUs) for compression, and the CTUs in the picture have the same size, for example, 64×64 pixels, 32×32 pixels, or 16×16 pixels. Typically, a CTU may include three parallel coding tree blocks (CTBs): one luminance CTB and two chrominance CTBs. Each CTU may be recursively quadtree-divided into one or more coding units (CUs). For example, a 64×64 pixel CTU may be divided into a 64×64 pixel CU or four 32×32 pixel CUs. Each of one or more of the 32×32 blocks may be further divided into four CUs of 16×16 pixels. In some example embodiments, each CU may be analyzed during the encoder to determine the prediction type of the CU among various prediction types, for example, an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) according to temporal and / or spatial predictability. Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In an embodiment, the prediction operation in encoding (encoding / decoding) is performed in units of prediction blocks. The CU can be divided into PUs (or different color channel PBs) in various spatial modes. For example, a luma or chroma PB may contain a matrix of sample values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 samples, etc.

[0097] Figure 7 A diagram of a video encoder (703) according to another example embodiment of the present disclosure is shown. The video encoder (703) is configured to receive a processed block (e.g., a prediction block) of sample values ​​within a current video picture in a sequence of video pictures and encode the processed block into an encoded image as part of an encoded video sequence. The example video encoder (703) may be used in place of Figure 4 The video encoder (403) in the example.

[0098] For example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a prediction block of 8×8 samples, etc. The video encoder (703) then determines whether to best encode the processing block using intra mode, inter mode, or a bidirectional prediction mode such as rate-distortion optimization (RDO). When it is determined that the processing block is to be encoded in intra mode, the video encoder (703) may encode the processing block into a coded picture using intra prediction techniques; and when it is determined that the processing block is to be encoded in inter mode or bidirectional prediction mode, the video encoder (703) may encode the processing block into a coded picture using inter prediction or bidirectional prediction techniques, respectively. In some example embodiments, merge mode may be used as a sub-mode of inter-picture prediction, in which motion vectors are derived from one or more motion vector predictors without benefiting from encoded motion vector components outside the predictors. In some example embodiments, there may be a motion vector component applicable to object blocks. Therefore, the video encoder (703) may include in Figure 7 Components not explicitly shown in the figure, for example, a mode decision module for determining a prediction mode for a processing block.

[0099] exist Figure 7 In the example of FIG. 7 , the video encoder ( 703 ) includes Figure 7 The example arrangement shown in FIG. 1 is an inter-frame encoder (730), an intra-frame encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725) coupled together.

[0100] The inter-frame encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture in display order), generate inter-frame prediction information (e.g., description of redundant information according to an inter-frame coding technique, motion vectors, merge mode information), and calculate an inter-frame prediction result (e.g., a prediction block) based on the inter-frame prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture decoded based on the encoded video information, using a decoded reference picture embedded in Figure 6 The decoding unit 633 (shown as Figure 7 residual decoder 728, as described in further detail below).

[0101] The intra encoder (722) is configured to receive samples of a current block (e.g., a processing block), compare the block with blocks that have been encoded in the same picture, generate quantization coefficients after transformation, and in some cases, generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). The intra encoder (722) can calculate an intra prediction result (e.g., a prediction block) based on the intra prediction information and a reference block in the same picture.

[0102] The general controller (721) can be configured to determine general control data and control other components of the video encoder (703) based on the general control data. In one example, the general controller (721) determines the prediction mode of the block and provides a control signal to the switch (726) based on the prediction mode. For example, when the prediction mode is intra-frame mode, the general controller (721) controls the switch (726) to select the intra-frame mode result for use by the residual calculator (723), and controls the entropy encoder (725) to select the intra-frame prediction information and include the intra-frame prediction information in the bitstream; when the prediction mode of the block is inter-frame mode, the general controller (721) controls the switch (726) to select the inter-frame prediction result for use by the residual calculator (723), and controls the entropy encoder (725) to select the inter-frame prediction information and include the inter-frame prediction information in the bitstream.

[0103] The residual calculator (723) may be configured to calculate the difference (residual data) between the received block and the prediction result of the block selected from the intra encoder (722) or the inter encoder (730). The residual encoder (724) may be configured to encode the residual data to generate a transform coefficient. In one example, the residual encoder (724) may be configured to convert the residual data from the spatial domain to the frequency domain to generate a transform coefficient. The transform coefficient is then quantized to obtain a quantized transform coefficient. In various example embodiments, the video encoder (703) further includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data may be appropriately used by the intra encoder (722) and the inter encoder (730). For example, the inter encoder (730) may generate a decoded block based on the decoded residual data and the inter prediction information, and the intra encoder (722) may generate a decoded block based on the decoded residual data and the intra prediction information. The decoded blocks are appropriately processed to generate a decoded picture, and the decoded picture may be buffered in a memory circuit (not shown) and used as a reference picture.

[0104] The entropy encoder (725) may be configured to format the bitstream to include the coded blocks and perform entropy coding. The entropy encoder (725) may be configured to include various information in the bitstream. For example, the entropy encoder (725) may be configured to include general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. When the block is encoded in the merge sub-mode of the inter-frame mode or the bidirectional prediction mode, there may be no residual information.

[0105] Figure 8 A diagram of an example video decoder (810) according to another embodiment of the present disclosure is shown. The video decoder (810) is configured to receive an encoded picture as part of an encoded video sequence and decode the encoded picture to generate a reconstructed picture. In one example, the video decoder (810) can be used instead of Figure 4 A video decoder (410) is shown in an example.

[0106] exist Figure 8 In the example of FIG. 8 , the video decoder ( 810 ) includes Figure 8 An entropy decoder (871), an inter-frame decoder (880), a residual decoder (873), a reconstruction module (874), and an intra-frame decoder (872) coupled together are shown in an example arrangement.

[0107] The entropy decoder (871) can be configured to reconstruct certain symbols representing syntax elements that make up the encoded image from the encoded image. Such symbols can include, for example, a mode for encoding a block (e.g., intra mode, inter mode, bidirectional prediction mode, merge submode, or another submode), prediction information (e.g., intra prediction information or inter prediction information) that can identify a specific sample or metadata used for prediction by an intra decoder (872) or an inter decoder (880), residual information such as quantized transform coefficients, etc. In one example, when the prediction mode is an inter or bidirectional prediction mode, the inter prediction information is provided to the inter decoder (880); and when the prediction type is an intra prediction type, the intra prediction information is provided to the intra decoder (872). The residual information can be subjected to inverse quantization and provided to the residual decoder (873).

[0108] The inter-frame decoder (880) may be configured to receive the inter-frame prediction information and generate an inter-frame prediction result based on the inter-frame prediction information.

[0109] The intra decoder (872) may be configured to receive intra prediction information and generate a prediction result based on the intra prediction information.

[0110] The residual decoder (873) may be configured to perform inverse quantization to extract dequantized transform coefficients and process the dequantized transform coefficients to convert the residual from the frequency domain to the spatial domain. The residual decoder (873) may also utilize certain control information (to include quantizer parameters (QP)) that may be provided by the entropy decoder (871) (data path not shown as this may only be low data volume control information).

[0111] The reconstruction module (874) may be configured to combine the residual output by the residual decoder (873) and the prediction result (output by the inter-frame or intra-frame prediction module, as appropriate) in the spatial domain to form a reconstructed block, which forms part of the reconstructed picture as part of the reconstructed video. Note that other suitable operations such as deblocking operations may also be performed to improve visual quality.

[0112] Note that the video encoders (403), (603) and (703) and the video decoders (410), (510) and (810) may be implemented using any suitable technology. In some example embodiments, the video encoders (403), (603) and (703) and the video decoders (410), (510) and (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603) and (603) and the video decoders (410), (510) and (810) may be implemented using one or more processors that execute software instructions.

[0113] Returning to the intra prediction process, in this process, samples in a block (e.g., a luma or chroma prediction block, or a coding block if not further divided into prediction blocks) are predicted by samples of an adjacent, next adjacent, or other row or rows, or a combination thereof, to generate a prediction block. The residual between the actual block being encoded and the prediction block can then be processed via a transform and subsequent quantization. Various intra prediction modes can be made available, and parameters related to intra mode selection and other parameters can be signaled in the bitstream. For example, various intra prediction modes can involve one or more row positions for prediction samples, the direction along which prediction samples are selected from one or more prediction rows, and other special intra prediction modes.

[0114] For example, a set of intra-prediction modes (interchangeably referred to as "intra-modes") may include a predetermined number of directional intra-prediction modes. As described above with respect to the example implementation of FIG. 1, these intra-prediction modes may correspond to a predetermined number of directions along which out-of-block samples are selected as predictions for samples being predicted in a particular block. In another specific example implementation, eight (8) primary directional modes corresponding to angles of 45° to 207° from the horizontal axis may be supported and predefined.

[0115] In some other implementations of intra prediction, in order to further exploit more kinds of spatial redundancy in directional texture, the directional intra mode can be further extended to a set of angles with finer granularity. For example, the 8-angle implementation above can be configured to provide eight nominal angles, called V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED, as shown in FIG. Fig. 9 As shown, and for each nominal angle, a predetermined number (e.g., 7) of finer angles can be added. With this extension, a greater total number (e.g., 56 in this example) of directional angles can be used for intra prediction, corresponding to the same number of predefined directional intra modes. The predicted angle can be represented by the nominal intra angle plus the angle increment. For the above specific example with 7 finer angular directions for each nominal angle, the angle increment can be a step size of -3 to 3 times 3°.

[0116] In some implementations, as an alternative or supplement to the above-mentioned directional intra-frame mode, a predetermined number of non-directional intra-frame prediction modes may also be predefined and made available. For example, five non-directional intra-frame modes referred to as smooth intra-frame prediction modes may be specified. These non-directional intra-frame mode prediction modes may be specifically referred to as DC, PAETH, SMOOTH, SMOOTH_V, and SMOOTH_H intra-frame modes. Fig.10 The prediction of samples for a particular block under these example non-directional modes is shown in . As an example, Fig.10A 4×4 block 1002 is shown predicted by samples from a top neighboring row and / or a left neighboring row. A particular sample 1010 in the block 1002 may correspond directly to a top sample 1004 of the sample 1010 in the top neighboring row of the block 1002, an upper left sample 1006 of the sample 1010 as the intersection of the top and left neighboring rows, and an immediate left sample 1008 of the sample 1010 in the left neighboring row of the block 1002. For the example DC intra prediction mode, an average of the left and upper neighboring samples 1008 and 1004 may be used as a prediction value for the sample 1010. For the example PAETH intra prediction mode, the top, left, and upper left reference samples 1004, 1008, and 1006 may be obtained, and then the value closest to (top+left-upper left) of the three reference samples may be set as the prediction value for the sample 1010. For the example SMOOTH_V intra prediction mode, sample 1010 may be predicted by quadratic interpolation in the vertical direction of the upper left neighboring sample 1006 and the left neighboring sample 1008. For the example SMOOTH_H intra prediction mode, sample 1010 may be predicted by quadratic interpolation in the horizontal direction of the upper left neighboring sample 1006 and the top neighboring sample 1004. For the example SMOOTH intra prediction mode, sample 1010 may be predicted by the average of the quadratic interpolation in the vertical and horizontal directions. The above non-directional intra mode implementations are described only as non-limiting examples. Other adjacent rows, other non-directional sample selections, and ways to combine prediction samples to predict specific samples in a prediction block are also contemplated.

[0117] The encoder selects a specific intra-frame prediction mode from the above-mentioned directional or non-directional mode at various coding levels (pictures, slices, blocks, units, etc.) and can be signaled in the bitstream. In some example implementations, 8 exemplary nominal directional modes and 5 non-angle smooth modes (a total of 13 options) can be signaled first. Then, if the signaled mode is one of the 8 nominal angle intra-frame modes, the signaling index is further signaled to indicate the selected angle increment relative to the nominal angle of the corresponding signaling. In some other example implementations, all intra-frame prediction modes can be indexed together (e.g., 56 directional modes plus 5 non-directional modes to generate 61 intra-frame prediction modes) for signaling.

[0118] In some example implementations, the example 56 or other number of directional intra prediction modes may be implemented with a unified directional predictor that projects each sample of the block to a reference subsample location and interpolates the reference samples via a 2-tap bilinear filter.

[0119] In some implementations, in order to capture the attenuated spatial correlation with the reference on the edges, additional filter modes called filter intra modes may be designed. For these modes, in addition to the samples outside the block, the predicted samples within the block can also be used as intra prediction reference samples for some tiles within the block. For example, these modes can be predefined and can be used for intra prediction of at least luminance blocks (or only luminance blocks). A predetermined number (e.g., five) of filter intra modes can be pre-designed, each mode being represented by a set of n-tap filters (e.g., 7-tap filters), reflecting the correlation between, for example, a sample in a 4×2 tile and its n neighboring samples. In other words, the weighting factors of the n-tap filter can be position-dependent. Taking 8×8 blocks, 4×2 tiles, and 7-tap filtering as an example, as Fig.11 As shown, the 8×8 block 1102 can be divided into eight 4×2 tiles. Fig.11 In , these patches are represented by B0, B1, B1, B3, B4, B5, B6, and B7. For each patch, its 7 neighbors (in Fig.11 In the example, R0~R7 are used to predict the samples in the current patch. For patch B0, all neighbors may have been reconstructed. However, for other patches, some neighbors are in the current block and thus may not have been reconstructed. The predicted values ​​of the direct neighbors are used as references. For example, Fig.11 All neighbors of patch B7 are shown to be unreconstructed, so the predicted samples of the neighbors are used instead.

[0120] In some implementations of intra prediction, one color component may be predicted using one or more other color components. The color component may be any of the components in YCrCb, RGB, XYZ color space, etc. For example, it may be possible to predict a chrominance component (e.g., a chrominance block) from a luma component (e.g., a luma reference sample), referred to as predicting chrominance from luma, or CfL. In some example implementations, only cross-color prediction from luma to chrominance may be allowed. For example, the chrominance samples in a chrominance block may be modeled as a linear function of coincident reconstructed luma samples. CfL prediction may be implemented as follows:

[0121] CfL(α)=α×L AC +DC (1)

[0122] Among them, L ACRepresents the AC contribution of the luma component, α represents the parameters of the linear model, and DC represents the DC contribution of the chroma component. For example, an AC component is obtained for each sample of a block, and a DC component is obtained for the entire block. Specifically, the reconstructed luma samples can be subsampled to the chroma resolution, and then the average luma value (the DC of luma) can be subtracted from each luma value to form the AC contribution in luma. The AC contribution of luma is then used in the linear model of equation (1) to predict the AC value of the chroma component. In order to approximate or predict the chroma AC component from the luma AC contribution, rather than requiring the decoder to calculate the scaling parameters, an example CfL implementation can determine the parameter α based on the original chroma samples and signal it in the bitstream. This reduces the complexity of the decoder and produces more accurate predictions. As for the DC contribution of the chroma component, in some example implementations, it can be calculated using the intra-frame DC model within the chroma component.

[0123] The residual of the intra-prediction block or inter-prediction block can then be transformed, and the transform coefficients are subsequently quantized. For the purpose of performing the transform, the intra-frame and inter-frame coding blocks can be further divided into multiple transform blocks (sometimes used interchangeably as "transform units" even though the term "unit" is generally used to refer to a set of three color channels, for example, a "coding unit" will include a luminance coding block and a chrominance coding block) before the transform. In some implementations, the maximum partition depth of the coding block (or prediction block) can be specified (the term "coding block" can be used interchangeably with "coding block"). For example, this division may not exceed 2 levels. The division of the prediction block into transform blocks can be handled differently between intra-frame prediction blocks and inter-frame prediction blocks. However, in some implementations, this division can be similar between intra-frame prediction blocks and inter-frame prediction blocks.

[0124] In some example implementations, for intra-coded blocks, transform partitioning may be performed in such a way that all transform blocks have the same size, and the transform blocks are encoded in a raster scan order. Fig.12 An example of such a transform block partitioning of an intra-coded block is shown in FIG. Fig.12 The coding block 1202 is shown to be divided into 16 transform blocks of equal block size as shown at 1206 via the mid-level quadtree split 1204. The example raster scan order for encoding is represented by Fig.12 The ordered arrows in are shown.

[0125] In some example implementations, for inter-coded blocks, the transform unit partitioning may be performed in a recursive manner, with the partitioning depth reaching a predefined number of levels (e.g., 2 levels). Fig.13 As shown, for any subpartition and any level, the partitioning can be stopped or continued recursively. Specifically, Fig.13An example is shown where a block 1302 is divided into four quadtree sub-blocks 1304, and one of the sub-blocks is further divided into four second-level transform blocks, while the division of the other sub-blocks stops after the first level, resulting in a total of 7 transform blocks of two different sizes. An example raster scan order for encoding is given by Fig.13 The ordered arrows in FIG. Fig.13 An example implementation of quadtree partitioning of up to two levels of square transform blocks is shown, but in some generation implementations, transform partitioning can support 1:1 (square), 1:2 / 2:1, and 1:4 / 4:1 transform block shapes and sizes, ranging from 4×4 to 64×64. In some example implementations, if the coding block is less than or equal to 64×64, the transform block partitioning can be applied only to the luma component (in other words, the chroma transform block will be the same as the coding block under this condition). Otherwise, if the coding block width or height is greater than 64, the luma and chroma coding blocks can be implicitly partitioned into multiples of min(W,64)×min(H,64) and min(W,32)×min(H,32) transform blocks, respectively.

[0126] Each of the above transform blocks can then be subjected to a primary transform. The primary transform essentially moves the residual in the transform block from the spatial domain to the frequency domain. In some implementations of the actual primary transform, in order to support the example extended coding block partitions above, multiple transform sizes (ranging from 4 points to 64 points for each of the two dimensions) and transform shapes (square; rectangular with width / height ratios of 2:1 / 1:2 and 4:1 / 1:4) can be allowed.

[0127] Turning specifically to intra-mode coding, and for a particular coding block, coding unit, prediction block, or prediction unit that is intra-coded, its intra-mode needs to be signaled by one or more syntax elements in the bitstream. As described above, the number of possible intra-prediction modes may be large. For example, in some implementations described above, the number of available intra-prediction modes may include 62 different modes, including, for example, 56 directional intra-prediction modes (e.g., 8 nominal directions, with 7 fine angles in each nominal direction), 5 non-directional modes, and one chrominance-derived luminance mode (for chrominance components only). Once an intra-mode is selected during the encoding process of a particular coding block, coding unit, prediction block, or prediction, the signaling corresponding to the selected intra-mode needs to be included in the bitstream. The signaling syntax must be able to distinguish all 62 modes in some way. For example, these 62 modes can be signaled using a single syntax of 62 indexes, each corresponding to a mode. In some other example implementations, one syntax may be signaled to indicate which nominal angle or non-directional mode is used as the nominal mode in the current block, and then, if the nominal mode of the current block is the directional mode, another syntax may be additionally signaled to indicate which incremental angle is selected for the current block.

[0128] Since the various syntaxes related to intra-frame coding usually occupy a large part of the bitstream, and the intra-frame mode selection must be signaled very frequently, for example, at various coding levels, reducing the number of bits used for intra-frame mode signaling becomes crucial to improving video coding efficiency. In practice, the use of various intra-frame prediction modes can follow certain statistical patterns, and this usage pattern can be used to design the index of intra-frame mode and signaling syntax, so that signaling efficiency can be enhanced. In addition, on average, there may be some correlation between intra-frame mode selections from block to block. This correlation can be obtained offline on a statistical basis and considered in the syntax design of signaling for selecting intra-frame modes. The goal is to reduce the number of bits of signaling syntax elements in the coded bitstream on average. For example, some general statistics may indicate that there may be a strong correlation between the best intra-frame prediction mode of the current block and its neighboring blocks. This correlation can be exploited when designing the syntax for intra-frame mode coding.

[0129] The following various example implementations can be used alone or in any order. In addition, each of these implementations can be implemented as part of an encoder and / or decoder and can be implemented in hardware or software. For example, it can be hard-coded in a dedicated processing circuit (e.g., one or more integrated circuits). In another example, it can be implemented by executing a program stored in a non-transitory computer-readable medium by one or more processors.

[0130] In some example implementations, the available intra-frame prediction modes or mode options of the current block being encoded may be divided into multiple intra-frame prediction mode sets. Each set may be assigned a mode set index. Each set may contain multiple intra-frame mode prediction modes. Based on the correlation between the intra-frame prediction modes between blocks, the manner in which the available intra-frame prediction modes are divided and sorted and the intra-frame prediction modes in each mode set may be sorted may be determined at least in part according to the intra-frame prediction modes used by its neighboring blocks. The intra-frame prediction mode used by the neighboring blocks may be referred to as a "reference intra-frame prediction mode" or a "reference mode". The intra-frame prediction mode of a specific unit may be determined and selected. The selection of the intra-frame prediction mode may be signaled. First, a mode set index may be signaled to indicate the mode set index of the intra-frame prediction mode set containing the selected intra-frame prediction mode. Second, a mode index (or a mode position index within a set) may be signaled to indicate the index of the selected intra-frame prediction mode within the mode set.

[0131] The general implementation of intra-frame prediction mode partitioning and sorting above and the specific examples below use statistical effects and neighboring correlations to dynamically index these modes so that the design of the syntax for signaling its selection in the encoded video bitstream can be optimized to improve coding efficiency. For example, these implementations can help reduce the number of syntax signals and help more efficient context generation for entropy coding.

[0132] In some example embodiments, the set of predefined intra-prediction modes available to the encoder may include P modes. For example, P may be 62 (e.g., 56 directional modes, 5 non-directional modes, and 1 luma-chroma mode), or any other predefined number. The number of intra-prediction mode sets may be indicated by M, and the number of intra-prediction modes for each mode set may be indicated by N. i To indicate, where i is used to indicate the mode set index, ranging from 1 to M, or from 0 to M-1. i The maximum value in is denoted as N max .

[0133] In some example implementations, N max can be restricted to numbers equal to powers of 2. For example, N max It can be limited to 8 or 16 or 32. This design can help to efficiently use the signaling bits in the signaling syntax.

[0134] In some example implementations, when M is greater than 2, N of all M pattern sets i can be restricted to having only a maximum of two different values. In this way, the syntax of the signalling pattern index within each pattern set can be simplified and streamlined.

[0135] In some example implementations, if the number of available intra prediction modes for the current block is equal to N max multiple of , the number of patterns in each pattern set is equal and is set to N max In other words, if P can be max divisible by N, then each of the M pattern sets will contain the same N max For example, if P = 64 and M = 4, each mode set may contain 16 (N max =16, or 2 to the power of 4) intra prediction mode. In this way, N i It is the same number in all M mode sets, which further optimizes the syntax design of signaling mode indexes within the M mode sets for the selected intra prediction mode.

[0136] In some example implementations, if the number of available intra prediction modes for the current block (P) is not equal to N maxThe number of intra modes in a set can be less than N max , but the number of intra modes in all other mode sets is equal to N max In other words, among all M pattern sets, N i Has two possible values. One of the values ​​can be N max Another value can be less than N max Except for one pattern set with a smaller number of patterns, all the M pattern sets have N max Intra-frame prediction modes. For example, P can be 62, M can be 4, and N max It can be 16, 3 of the 4 mode groups can each contain 16 modes, and another mode set can contain 14 modes. For another example, the total number of available intra prediction modes for the luma component can be 61, M is set to 4, N0 is 13, and N1, N2, and N3 are all equal to 16. In addition, such an implementation can help simplify the syntax design of the signaling mode index in the M mode sets for the selected intra prediction mode.

[0137] In some example implementations, the (relative) positions of all non-directional modes are predetermined and fixed, independent of the intra-prediction modes of neighboring blocks. In other words, the placement and arrangement of non-directional modes into a mode set and into a mode set among all available modes may be independent of the intra-prediction modes (or reference modes) of neighboring blocks, while the arrangement of other intra-prediction modes into a mode set or into a mode set may depend on the reference mode. In this way, the dynamic determination of the mode set may be simplified, taking advantage of the statistics that non-directional modes may not be so correlated between neighboring blocks.

[0138] In some example implementations, all non-directional modes among the P available modes may be placed in the first intra-mode set (with the lowest mode set index), regardless of the intra-prediction modes of neighboring blocks.

[0139] In some example implementations, the DC prediction mode (non-directional mode) may always be placed at the first relative position (lowest position index within the mode set) of the first intra mode set (the mode set with the lowest index), regardless of the intra prediction modes of neighboring blocks.

[0140] In some example implementations, one or more recursive filtering based intra prediction modes may be signaled as sub-modes of the DC prediction mode. Thus, only when the mode set index and the position index indicate that the DC mode is selected (e.g., both the mode index and the position index are the lowest, e.g., both are equal to 0 in the previous example implementations), the recursive filtering based intra prediction mode may be additionally signaled.

[0141] In some example implementations, the SMOOTH prediction mode described above may always be placed at the second position (second lowest position index) of the first intra mode set (the mode set with the lowest mode set index), regardless of the intra prediction modes of neighboring blocks.

[0142] In some example implementations, the SMOOTH_V prediction mode described above may always be placed at the third position (third lowest position index) of the first intra-mode set (the mode set with the lowest mode set index), regardless of the intra-prediction modes of neighboring blocks.

[0143] In some example implementations, the SMOOTH_H prediction mode described above may always be placed at the fourth position (fourth lowest position index) of the first intra mode set (the mode set with the lowest mode set index), regardless of the intra prediction modes of neighboring blocks.

[0144] In some example implementations, the PAETH prediction mode described above may always be placed at the fifth position (fifth lowest position index) of the first intra mode set (the mode set with the lowest mode set index), regardless of the intra prediction modes of neighboring blocks.

[0145] In some other or further example implementations, the recursive filtering based intra prediction mode can be signaled independently from the DC intra prediction mode and can be one of the P modes instead of a sub-mode, and can be placed in the first intra mode set (the mode set with the lowest mode set index), regardless of the intra prediction modes of neighboring blocks.

[0146] In some further example implementations, there may be a set of recursive filtering based intra prediction modes and one of them is selected, and additional signaling may be used to indicate which recursive filtering based intra prediction mode is selected in the set of recursive filtering based intra prediction modes for the current block.

[0147] In some example implementations, when independent of DC intra-prediction mode signaling, the recursive filtering based intra-prediction mode may always be placed at the sixth position of the first intra-mode set (the mode set with the lowest mode set index), regardless of the intra-prediction modes of neighboring blocks. In this case, the DC intra-prediction mode may be placed at the lowest index position in the lowest index mode set.

[0148] Fig.14A flowchart 1400 of an example method following the basic principles of the implementation for intra-frame mode coding described above is shown. The example method flow starts at 1401. At S1410, an encoded video stream is received, the encoded video stream includes a set index and a position index, the set index and the position index are used for intra-frame prediction of a current block. At S1420, an intra-frame prediction mode corresponding to at least one neighboring block of the current block in the video stream is determined as a reference mode. At S1430, a manner of dividing P predefined intra-frame prediction modes into M intra-frame prediction mode sets is determined based on the reference mode, each intra-frame prediction mode set includes N i The intra prediction mode indexed by the position, P and M are integers greater than 1, N i is the number of prediction modes in the i-th intra-frame prediction mode set, and i is an integer representing the intra-frame prediction mode set index in the M intra-frame prediction mode sets. In S1440, based on the received set index and position index and the division method, a target intra-frame prediction mode is identified in the P predefined intra-frame prediction modes. In S1450, a current block of the encoded video stream is decoded based on at least the target intra-frame prediction mode identified according to the set index and position index of the current block. The example method flow ends at S1499. The above method flow also applies encoding.

[0149] The embodiments of the present disclosure may be used alone or in combination in any order. In addition, each of the method (or embodiment), encoder, and decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium. The embodiments of the present invention may be applied to luminance blocks or chrominance blocks.

[0150] The above techniques may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. Fig.15 A computer system (1500) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0151] Computer software may be encoded using any suitable machine code or computer language, which may be subjected to assembly, compilation, linking or similar mechanisms to create code comprising instructions that may be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or through interpretation, microcode execution, etc.

[0152] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, etc.

[0153] Fig.15The components of the computer system (1500) shown in the example are exemplary in nature and are not intended to impose any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. The configuration of components should also not be interpreted as having any dependency or requirement on any one component or combination of components shown in the exemplary embodiment of the computer system (1500).

[0154] The computer system (1500) may include certain human interface input devices. Such human interface input devices may respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), olfactory input (not shown). The human interface devices may also be used to capture certain media that are not necessarily directly related to a person's conscious input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0155] Input human interface devices may include one or more of the following (only one of each is depicted): keyboard (1501), mouse (1502), trackpad (1503), touch screen (1510), data gloves (not shown), joystick (1505), microphone (1506), scanner (1507), camera (1508).

[0156] The computer system (1500) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen (1510), a data glove (not shown), or a joystick (1505), but there may also be tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (1509), headphones (not shown)), visual output devices (e.g., screens (1510), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which are capable of outputting two-dimensional visual output or more than three-dimensional output through means such as stereo output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)) and printers (not shown).

[0157] The computer system (1500) may also include human-accessible storage devices and their associated media, such as optical media including a CD / DVD ROM / RW (1520) with a CD / DVD or similar media (1520), a thumb drive (1522), a removable hard drive or solid state drive (1523), traditional magnetic media such as tapes and floppy disks (not shown), dedicated ROM / ASIC / PLD devices such as a security dongle (not shown), and the like.

[0158] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0159] The computer system (1500) may also include an interface (1554) to one or more communication networks (1555). The network may be, for example, wireless, wired, optical. The network may also be local, wide, metropolitan, vehicular and industrial, real-time, delay-tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable, satellite and terrestrial broadcast television, vehicle and industrial networks including CANBus, etc. Some networks typically require an external network interface adapter connected to some common data port or peripheral bus (1549) (e.g., a USB port of the computer system (1500)); others are typically integrated into the core of the computer system (1500) by connecting to a system bus as described below (e.g., an Ethernet interface in a PC computer system or a cellular network interface in a smartphone computer system). Using any of these networks, the computer system (1500) can communicate with other entities. This communication can be one-way, receive-only (e.g., broadcast television), one-way, send-only (e.g., CANbus to certain CANbus devices), or two-way, for example, to other computer systems using local or wide area digital networks. As described above, certain protocols and protocol stacks can be used on each of these networks and network interfaces.

[0160] The aforementioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core (1540) of the computer system (1500).

[0161] The core (1540) may include one or more central processing units (CPUs) (1541), graphics processing units (GPUs) (1542), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (1543), hardware accelerators (1544) for specific tasks, graphics adapters (1550), etc. These devices, along with read-only memory (ROM) (1545), random access memory (1546), internal mass storage (1547) such as internal non-user accessible hard drives, SSDs, etc., may be connected via a system bus (1548). In some computer systems, the system bus (1548) may be accessible in the form of one or more physical plugs to allow expansion of additional CPUs, GPUs, etc. Peripheral devices may be connected to the core's system bus (1548) directly or via a peripheral bus (1549). In one example, a screen (1510) may be connected to a graphics adapter (1550). The architecture of the peripheral bus includes PCI, USB, etc.

[0162] The CPU (1541), GPU (1542), FPGA (1543) and accelerator (1544) can execute certain instructions, which can be combined to form the above-mentioned computer code. The computer code can be stored in ROM (1545) or RAM (1546). Transitional data can be stored in RAM (1546), while permanent data can be stored in, for example, internal mass storage (1547). Fast storage and retrieval of any storage device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1541), GPUs (1542), mass storage (1547), ROM (1545), RAM (1546), etc.

[0163] The computer readable medium may have computer codes for performing various computer-implemented operations. The media and computer codes may be specially designed and constructed for the purposes of the present disclosure, or may be of a type well known and available to those skilled in the art of computer software.

[0164] As a non-limiting example, a computer system having architecture (1500), in particular a core (1540), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software contained in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage as described above and certain memories of a core (1540) having non-transitory properties, for example, a core internal mass storage (1547) or a ROM (1545). Software implementing various embodiments of the present disclosure can be stored in such a device and executed by the core (1540). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the core (1540) and in particular the processor therein (including a CPU, GPU, FPGA, etc.) to perform a specific process or a specific part of a specific process described herein, including defining a data structure stored in RAM (1546) and modifying such a data structure according to a software-defined process. In addition or in the alternative, the computer system may provide functionality as a result of logic hardwired or otherwise contained in circuits (e.g., accelerators (1544)) that may operate in place of or in conjunction with software to perform specific processes or specific portions of specific processes described herein. Where appropriate, references to software may include logic and vice versa. Where appropriate, references to computer-readable media may include circuits (e.g., integrated circuits (ICs)) storing software for execution, circuits containing logic for execution, or both. The present disclosure encompasses any suitable combination of hardware and software.

[0165] Although the present disclosure has described several exemplary embodiments, there are changes, permutations and various alternative equivalents that fall within the scope of the present disclosure. In the above implementations and embodiments, any operation of the process can be combined or arranged in any quantity or order as needed. In addition, two or more operations of the above process can be performed in parallel. Therefore, it should be understood that those skilled in the art will be able to design many systems and methods, although not explicitly shown or described herein, these systems and methods embody the principles of the present disclosure, and are therefore within the spirit and scope of the present disclosure.

[0166] Appendix A: Abbreviations

[0167] JEM: Joint Exploration Model

[0168] VVC: Versatile Video Coding

[0169] BMS: Benchmark Set

[0170] MV: Motion Vector

[0171] HEVC: High Efficiency Video Coding

[0172] SEI: Supplemental Enhancement Information

[0173] VUI: Video Availability Information

[0174] GOP: Group of Pictures

[0175] TU: Transform Unit

[0176] PU: prediction unit

[0177] CTU: Coding Tree Unit

[0178] CTB: Coding Tree Block

[0179] PB: prediction block

[0180] HRD: Hypothesized Reference Decoder

[0181] SNR: Signal to Noise Ratio

[0182] CPU: Central Processing Unit

[0183] GPU: Graphics Processing Unit

[0184] CRT: cathode ray tube

[0185] LCD: Liquid Crystal Display

[0186] OLED: Organic Light Emitting Diode

[0187] CD: compact disc

[0188] DVD: Digital Video Disc

[0189] ROM: Read Only Memory

[0190] RAM: Random Access Memory

[0191] ASIC: Application-Specific Integrated Circuit

[0192] PLD: Programmable Logic Device

[0193] LAN: Local Area Network

[0194] GSM: Global System for Mobile Communications

[0195] LTE: Long Term Evolution

[0196] CANBus: Controller Area Network Bus

[0197] USB: Universal Serial Bus

[0198] PCI: Peripheral Component Interconnect

[0199] FPGA: Field Programmable Gate Array

[0200] SSD: Solid State Drive

[0201] IC: Integrated Circuit

[0202] HDR: High Dynamic Range

[0203] SDR: Standard Dynamic Range

[0204] JVET: Joint Video Exploration Team

[0205] MPM: Most Probable Mode

[0206] WAIP: Wide Angle Intra Prediction

[0207] CU: Coding Unit

[0208] PU: prediction unit

[0209] TU: Transform Unit

[0210] CTU: Coding Tree Unit

[0211] PDPC: Position Dependent Prediction Combination

[0212] ISP: Intra-frame sub-partitioning

[0213] SPS: Sequence parameter settings

[0214] PPS: Picture Parameter Set

[0215] APS: Adaptive Parameter Set

[0216] VPS: Video Parameter Set

[0217] DPS: Decoding Parameter Set

[0218] ALF: Adaptive Loop Filter

[0219] SAO: Sample Adaptive Offset CC-ALF: Cross-Component Adaptive Loop Filter CDEF: Constrained Directional Enhancement Filter

[0220] CCSO: Cross Component Sample Offset

[0221] LSO: Local Sample Offset

[0222] LR: Loop Restoration Filter

[0223] AV1: AOMedia Video 1

[0224] AV2: AOMedia Video 2

Claims

1. A method for performing intra-frame prediction on a current block of a coded video stream, characterized in that include: receiving a coded video stream, the coded video stream comprising a set index and a position index, the set index and the position index being used for intra-frame prediction of the current block; Determining an intra prediction mode corresponding to at least one neighboring block of the current block in the video stream as a reference mode; Determine a way to divide P predefined intra prediction modes into M intra prediction mode sets based on the reference mode, each intra prediction mode set includes N i The intra prediction mode indexed by the position, P and M are integers greater than 1, N i is the number of prediction modes in the i-th intra prediction mode set, and i is an integer representing an intra prediction mode set index in the M intra prediction mode sets; Based on the received set index and the position index and the manner of partitioning, identifying a target intra prediction mode among the P predefined intra prediction modes; as well as decoding the current block of the encoded video stream based at least on the target intra prediction mode identified according to the set index and the position index of the current block; wherein the P predefined intra prediction modes include a subset of non-directional intra prediction modes; and a relative order of the subset of non-directional intra prediction modes in the set of M intra prediction modes is fixed and has nothing to do with the reference mode; or The P predefined intra-frame prediction modes include an intra-frame prediction mode based on recursive filtering, and the intra-frame prediction mode based on recursive filtering is sequentially placed as an intra-frame prediction mode set located at the first position in the M intra-frame prediction mode sets, regardless of the reference mode; and the intra-frame prediction mode based on recursive filtering is represented as being independent of any DC intra-frame prediction mode.

2. The method according to claim 1, characterized in that: N max is a power of 2, N max YesN i The maximum value in .

3. The method according to claim 2, characterized in that When M is greater than 2, N i is selected from a first quantity and a second quantity.

4. The method according to claim 3, characterized in that N i It is the same value for all M intra prediction mode sets.

5. The method according to claim 3, characterized in that: One of the M intra-prediction mode sets includes the second number of intra-prediction modes, and all other intra-prediction mode sets of the M intra-prediction mode sets include the first number of intra-prediction modes.

6. The method according to claim 3, characterized in that M = 4, and N max =16.

7. The method according to claim 6, characterized in that P=61, the first number is 16, and the second number is 13.

8. The method according to any one of claims 1 to 7, characterized in that The subset of the non-directional intra prediction modes are all in the intra prediction mode set located at the first position among the M intra prediction mode sets, and are irrelevant to the reference mode.

9. The method according to claim 8, characterized in that The subset of non-directional intra prediction modes includes a DC prediction mode, which is always placed first in a set of intra prediction modes at the first position regardless of the reference mode.

10. The method according to claim 9, characterized in that The intra prediction mode based on recursive filtering is represented as a sub-mode of the DC prediction mode.

11. The method according to claim 8, characterized in that The subset of non-directional intra prediction modes comprises a SMOOTH prediction mode, which is always positioned as the second one of the set of intra prediction modes at the first position, regardless of the reference mode.

12. The method according to claim 8, characterized in that The subset of non-directional intra prediction modes includes a SMOOTH_V prediction mode, which is always positioned as the third one in the set of intra prediction modes at the first position, regardless of the reference mode.

13. The method according to claim 8, characterized in that The subset of non-directional intra prediction modes includes a SMOOTH_H prediction mode, which is always positioned as the fourth in the set of intra prediction modes at the first position, regardless of the reference mode.

14. The method according to claim 8, characterized in that The subset of non-directional intra prediction modes comprises a PAETH prediction mode, which is always positioned as the fifth one in the set of intra prediction modes at the first position, regardless of the reference mode.

15. The method according to claim 1, characterized in that When the target intra prediction mode identified for the current block is the recursive filtering based intra prediction mode, the method further includes determining to select one of a group of recursive filtering based intra prediction modes based on the set index and the position index, respectively.

16. The method according to claim 1, characterized in that The recursive filtering based intra prediction mode is always placed sixth in the intra prediction mode set located at the first position, regardless of the reference mode.

17. The method according to claim 1, characterized in that The P predefined intra prediction modes include at least one directional intra prediction mode, at least one non-directional intra prediction mode, and at least one luma-to-chroma intra prediction mode.

18. A device for performing intra-frame prediction on a current block of a coded video stream, characterized in that comprising a processing circuit, the processing circuit being configured to: Receiving the encoded video stream, the encoded video stream comprising a set index and a position index, the set index and the position index being used for intra-frame prediction of the current block; Determining an intra prediction mode corresponding to at least one neighboring block of the current block in the video stream as a reference mode; Determine a way to divide P predefined intra prediction modes into M intra prediction mode sets based on the reference mode, each intra prediction mode set includes N i The intra prediction mode indexed by the position, P and M are integers greater than 1, N i is the number of prediction modes in the i-th intra prediction mode set, and i is an integer representing an intra prediction mode set index in the M intra prediction mode sets; Based on the received set index and the position index and the manner of partitioning, identifying a target intra prediction mode among the P predefined intra prediction modes; as well as decoding the current block of the encoded video stream based at least on the target intra prediction mode identified according to the set index and the position index of the current block; wherein the P predefined intra prediction modes include a subset of non-directional intra prediction modes; and a relative order of the subset of non-directional intra prediction modes in the set of M intra prediction modes is fixed and has nothing to do with the reference mode; or The P predefined intra-frame prediction modes include an intra-frame prediction mode based on recursive filtering, and the intra-frame prediction mode based on recursive filtering is sequentially placed as an intra-frame prediction mode set located at the first position in the M intra-frame prediction mode sets, regardless of the reference mode; and the intra-frame prediction mode based on recursive filtering is represented as being independent of any DC intra-frame prediction mode.

Images