Improved intra mode coding

Abstract

The present disclosure provides various schemes for intra mode encoding and decoding of video. For example, a method for performing intra prediction on a current block of an encoded video stream is disclosed. The method can include receiving the encoded video stream, the encoded video stream including a set index and a position index for intra prediction of the current block; determining an intra prediction mode corresponding to at least one neighboring block of the current block in the video stream as a reference mode; determining a manner of partitioning P predefined intra prediction modes into M sets of intra prediction modes based on the reference mode, each set of intra prediction modes including N i intra prediction modes of the P predefined intra prediction modes, P and M being integers greater than 1, N i being a number of prediction modes in an i-th set of intra prediction modes, i being an integer representing a set index of the M sets of intra prediction modes; identifying a target intra prediction mode among the P predefined intra prediction modes based on the received set index and position index and the determined manner of partitioning; and decoding the current block of the encoded video stream based at least on the target intra prediction mode identified from the set index and the position index of the current block.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to U.S. non-provisional patent application No. 17 / 573,306, filed January 11, 2022, which claims priority to U.S. provisional application No. 63 / 209,806, filed June 11, 2021, entitled "Improved Intra-Frame Mode Coding". The entire contents of both applications are incorporated herein by reference. Technical Field

[0003] This disclosure describes a set of advanced video coding techniques. More specifically, the disclosed techniques relate to various schemes for intra-frame mode coding. Background Technology

[0004] The background description provided herein is intended to provide a general overview of the context of this disclosure. To the extent described in this background section, the work of the currently named inventors and aspects of the description that may not conform to the prior art at the time of filing this application are neither explicitly nor implicitly considered to be prior art of this disclosure.

[0005] Video encoding and decoding can be performed using inter-frame picture prediction with motion compensation. Uncompressed digital video can comprise a series of pictures, each with a spatial dimension of, for example, 1920×1080 luminance samples and associated fully sampled or subsampled chrominance samples. This series of pictures can have a fixed or variable picture rate (or frame rate), for example, 60 pictures per second or 60 frames per second. Uncompressed video has specific bitrate requirements for streaming or data processing. For example, a video with a pixel resolution of 1920x1080, a frame rate of 60 frames per second, chrominance subsampling of 4:2:0, and 8 bits per pixel per color channel requires a bandwidth of approximately 1.5 Gbit / s. One hour of such video would require more than 600 GBytes of storage space.

[0006] One objective of video encoding and decoding is to reduce redundancy in the uncompressed input video signal through compression. In some cases, compression helps reduce the aforementioned bandwidth and / or storage requirements by two orders of magnitude or more. Lossless compression and lossy compression, as well as combinations thereof, can be employed. Lossless compression refers to a technique in which an exact copy of the original signal can be reconstructed from the compressed original signal via a decoding process. Lossy compression refers to an encoding / decoding process in which the original video information is not fully preserved during encoding and cannot be fully recovered during decoding. When using lossy compression, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal is useful for the intended application, despite some information loss. In the case of video, lossy compression is widely used in many applications. The amount of distortion tolerated depends on the application. For example, users of some consumer video streaming applications may tolerate higher distortion than users of film or television broadcasting applications. The compression ratio achievable by a particular encoding algorithm can be selected or tuned to reflect various distortion tolerances: higher tolerable distortion generally allows encoding algorithms that produce higher losses and higher compression ratios.

[0007] Video encoders and decoders can utilize several classes and steps of techniques, including, for example, motion compensation, Fourier transform, quantization, and entropy coding.

[0008] Video codec techniques can include techniques called intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from a previously reconstructed reference picture. In some video codecs, a picture is spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, the picture can be called an intra-frame picture. Intra-frame pictures and their derived pictures (e.g., separate decoder refresh pictures) can be used to reset the decoder state and thus can be used as the first picture in the encoded video bitstream and video session, or as a still image. After intra-frame prediction, the samples of the block can then be transformed to the frequency domain, and the transform coefficients thus generated can be quantized before entropy coding. Intra-frame prediction represents a technique for minimizing sample values ​​in the pre-transform domain. In some cases, the smaller the transformed DC value, the smaller the AC coefficients, and the fewer bits are needed to represent the entropy-coded block at a given quantization step size.

[0009] For example, traditional intra-frame coding, as known from MPEG-2 generation coding techniques, does not use intra-frame prediction. However, some newer video compression techniques include attempts to encode / decode blocks based on, for example, surrounding sample data and / or metadata, which are obtained during the encoding and / or decoding of spatially adjacent blocks and precede the data blocks being intra-frame encoded or decoded in the decoding order. This technique is hereby referred to as "intra-frame prediction." Note that, at least in some cases, intra-frame prediction uses only reference data from the current frame in the reconstruction, and not reference data from other reference frames.

[0010] There can be many different forms of intra prediction. When more than one such technique is available in a given video coding technique, the technique used can be called an intra prediction mode. One or more intra prediction modes can be provided in a particular codec. In some cases, a mode may have submodes and / or may be associated with various parameters, and the mode / submode information and intra coding parameters of the video block may be encoded individually or collectively, included in the mode codeword. For a given combination of mode, submode, and / or parameters, which codeword is used will affect the coding efficiency obtained through intra prediction, and the entropy coding technique used to convert the codeword into a bitstream will also have an impact.

[0011] H.264 introduced specific intra-prediction modes, which were improved in H.265 and further refined in newer coding techniques such as Joint Exploration Model (JEM), Universal Video Coding (VVC), and Baseline Set (BMS). Typically, for intra-prediction, neighboring sample values ​​that have become available can be used to form prediction blocks. For example, available values ​​from a specific set of neighboring samples along a particular direction and / or line can be copied into the predictor block. The reference to the direction in use can be encoded in the bitstream or predicted itself.

[0012] refer to Figure 1A The lower right corner depicts a subset of nine prediction directions from the 33 possible intra-frame prediction directions of H.265 (corresponding to the 33 angular modes of the 35 intra-frame modes specified in H.265). The point (101) where the arrows converge represents the sample being predicted. The arrows indicate the directions from which neighboring samples are used to predict the sample at 101. For example, arrow (102) indicates that sample (101) is predicted from one or more neighboring samples at an angle of 45° to the upper right of the horizontal direction. Similarly, arrow (103) indicates that sample (101) is predicted from one or more neighboring samples at an angle of 22.5° to the lower left of sample (101) at an angle of 22.5° to the horizontal direction.

[0013] Still referencing Figure 1AA square block (104) of 4×4 samples is depicted in the upper left (represented by a thick dashed line). The square block (104) comprises 16 samples, each labeled "S" with its position in the Y dimension (e.g., row index) and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (counting from the top) and the first sample in the X dimension (counting from the left). Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. Since the block size is 4×4 samples, S44 is located in the lower right corner. Reference samples following a similar numbering scheme are also shown. Reference samples are labeled (104) with R, their Y position relative to the block (e.g., row index), and X position (column index). In H.264 and H.265, predicted samples adjacent to the block in the reconstruction are used.

[0014] Intra-frame picture prediction for block 104 can begin by copying reference sample values ​​from neighboring samples indicated by the prediction direction of the signaling. For example, suppose the encoded video bitstream contains signaling that indicates the prediction direction of arrow (102) for this block 104—that is, predicting one or more prediction samples from samples at a 45° angle to the upper right of the horizontal direction. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Then sample S44 is predicted from reference sample R08.

[0015] In some cases, the values ​​of multiple reference samples can be combined, for example, by interpolation, to calculate the reference sample; especially when the direction is not divisible by 45°.

[0016] With the continuous development of video coding technology, the number of possible directions has increased. For example, in H.264 (2003), nine different directions were available for intra-frame prediction. This increased to 33 in H.265 (2013), and at the time of this disclosure, JEM / VVC / BMS can support up to 65 directions. Experimental studies have been conducted to help identify the most suitable intra-frame prediction directions, and certain techniques in entropy coding can be used to encode those most suitable directions with a small number of bits, accepting a bit loss for the direction. Furthermore, sometimes the direction itself can be predicted based on the adjacent directions used in the intra-frame prediction of already decoded adjacent blocks.

[0017] Figure 1B A schematic diagram (180) is shown, depicting 65 intra-frame prediction directions according to JEM, to illustrate the increasing number of prediction directions in various coding techniques over time.

[0018] The way bits representing intra-prediction directions are mapped to prediction directions in the encoded video bitstream can vary depending on the video coding technique; and the range can be from, for example, a simple direct mapping from prediction direction to intra-prediction mode to codewords, to complex adaptive schemes involving the most probable mode, and similar techniques. However, in all cases, some intra-prediction directions are statistically less likely to appear in the video content compared to some other directions. Since the goal of video compression is to reduce redundancy, in well-designed video coding techniques, those less likely directions will be represented by more bits than the more likely directions.

[0019] Inter-frame image prediction, or inter-frame prediction, can be based on motion compensation. In motion compensation, sample data from a previously reconstructed image or a portion thereof (the reference image) is spatially shifted in a direction indicated by a motion vector (hereafter referred to as MV) and can then be used to predict a newly reconstructed image or image portion (e.g., a patch). In some cases, the reference image can be the same as the image currently being reconstructed. The MV can have two dimensions, X and Y, or three dimensions, with the third dimension being an indication of the reference image in use (similar to a temporal dimension).

[0020] In some video compression techniques, the current motion vector (MV) applicable to a region of sample data can be predicted from other MVs, for example, from those other MVs related to other regions of the sample data that are spatially adjacent to the region being reconstructed and precede the current MV in the decoding order. This significantly reduces the total amount of data required to encode the MV by removing redundancy from related MVs, thus improving compression efficiency. MV prediction can work effectively, for example, because when encoding an input video signal derived from a camera (called natural video), there is a statistical probability that regions larger than the region to which a single MV can be applied move in similar directions within the video sequence. Therefore, in some cases, similar motion vectors derived from MVs in neighboring regions can be used for prediction. This results in an MV found for a given region being similar to or identical to an MV predicted from surrounding MVs. After entropy coding, this MV can then be represented with fewer bits than if the MV were directly encoded instead of predicted from neighboring MVs. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself may be lossy, for example, when calculating predictions from several surrounding MVs due to rounding errors.

[0021] Various video prediction mechanisms are described in H.265 / HEVC (ITU-T TRec.H.265, “High Efficiency Video Coding”, December 2016). Among the many video prediction mechanisms specified in H.265, one technique, hereinafter referred to as “spatial merging,” is described below.

[0022] Specifically, refer to Figure 2 The current block (201) includes samples discovered by the encoder during motion search so that predictions can be made from previous blocks of the same size that have been spatially shifted. Instead of directly encoding the MV, it can be derived from metadata associated with one or more reference images, for example, from the most recent (in decoding order) reference image, using the MV associated with any of the five surrounding samples (denoted as A0, A1 and B0, B1, B2 (202 to 206 respectively)). In H.265, MV prediction can use a predictor from the same reference image being used by adjacent blocks. Summary of the Invention

[0023] This disclosure provides methods and apparatus for video encoding and decoding, including various schemes for intra-frame mode coding.

[0024] In some example implementations, a method for performing intra-prediction on a current block of an encoded video stream is disclosed. The method may include: receiving an encoded video stream, the encoded video stream including a set index and a position index, the set index and position index being used for intra-prediction of the current block; determining an intra-prediction mode corresponding to at least one adjacent block of the current block in the video stream as a reference mode; and determining, based on the reference mode, a method for dividing P predefined intra-prediction modes into M intra-prediction mode sets, each intra-prediction mode set including N... i Intra-prediction modes at position indices, where P and M are integers greater than 1, and N... i It is the number of prediction modes in the i-th intra-prediction mode set, where i is an integer representing the set index of the intra-prediction mode in the M intra-prediction mode sets; based on the received set index and position index and the partitioning method, identify the target intra-prediction mode among the P predefined intra-prediction modes; and decode the current block of the encoded video stream based at least on the target intra-prediction mode identified according to the set index and position index of the current block.

[0025] In the above implementation, N max It is a power of 2, N max It is N i The maximum value in M. In some implementations, when M is greater than 2, N is the maximum value in M. i Selected from the first and second quantities. In some implementations, N iThis is the same value for all M intra-prediction mode sets. In some implementations, one of the M intra-prediction mode sets includes a second number of intra-prediction modes, and all other intra-prediction mode sets in the M set include a first number of intra-prediction modes. In some implementations, M = 4, and N max =16. In some implementations, P=61, the first quantity is 16, and the second quantity is 13.

[0026] In some of the example implementations above, the P predefined intra-prediction modes may include a subset of non-directional intra-prediction modes; and the relative order of the subsets of non-directional intra-prediction modes in the M intra-prediction mode set is fixed and independent of the reference mode.

[0027] In some of the example implementations above, the subset of non-directional intra-prediction modes is located in the first intra-prediction mode set among the M intra-prediction mode sets, and is independent of the reference mode.

[0028] In some of the example implementations above, a subset of non-directional intra-prediction modes includes DC prediction modes, which are always placed first in the set of intra-prediction modes, regardless of the reference mode.

[0029] In some of the example implementations above, recursive-filtering-based intra-prediction modes are represented as sub-modes of the DC prediction mode. Furthermore, a subset of non-directional intra-prediction modes includes the SMOOTH prediction mode, which is always positioned as the second in the first set of intra-prediction modes, independent of the reference mode. In some implementations, a subset of non-directional intra-prediction modes includes the SMOOTH_V prediction mode, which is always positioned as the third in the first set of intra-prediction modes, independent of the reference mode.

[0030] In some of the example implementations above, a subset of non-directional intra-prediction modes includes the SMOOTH_H prediction mode, which is always positioned as the fourth in the set of intra-prediction modes in the first position, regardless of the reference mode.

[0031] In some of the example implementations above, the subset of non-directional intra-prediction modes includes the PAETH prediction mode, which is always positioned as the fifth in the set of intra-prediction modes in the first position, regardless of the reference mode.

[0032] In some of the example implementations above, the P predefined intra-prediction modes may include recursive filtering-based intra-prediction modes; the recursive filtering-based intra-prediction modes are arranged sequentially as the first intra-prediction mode set in the M intra-prediction mode set, independent of the reference mode; and the recursive filtering-based intra-prediction modes are represented as independent of any DC intra-prediction mode.

[0033] In some of the example implementations above, when the target intra-prediction mode identified for the current block is an intra-prediction mode based on recursive filtering, the method may further include determining one of a set of intra-prediction modes based on recursive filtering, respectively, based on a set index and a position index.

[0034] In some of the example implementations above, the intra-prediction mode based on recursive filtering is always placed as the sixth in the prediction mode set within the first position frame, regardless of the reference mode. In some of the example implementations above, the P predefined intra-prediction modes include at least one directional intra-prediction mode, at least one non-directional intra-prediction mode, and at least one luma-to-chroma intra-prediction mode.

[0035] This disclosure also provides an apparatus including processing circuitry configured to perform any of the above-described methods for video decoding and / or encoding. This disclosure further provides a non-transitory computer-readable medium storing instructions that, when executed by a computer for video decoding and / or encoding, cause the computer to perform any of the above-described methods for video decoding and / or encoding. Attached Figure Description

[0036] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, wherein:

[0037] Figure 1A A schematic diagram of an example set of intra-frame prediction orientation modes is shown.

[0038] Figure 1B An illustration of an exemplary intra-frame prediction direction is shown;

[0039] Figure 2 A schematic diagram of the current block and its surrounding spatial merging candidates for motion vector prediction is shown in one example.

[0040] Figure 3 A simplified block diagram of a communication system (300) according to an example embodiment is shown;

[0041] Figure 4 A simplified block diagram of a communication system (400) according to an example embodiment is shown;

[0042] Figure 5 A simplified block diagram of a video decoder according to an example embodiment is shown in the schematic diagram;

[0043] Figure 6 A simplified block diagram of a video encoder according to an example embodiment is shown in the diagram.

[0044] Figure 7 A block diagram of a video encoder according to another example embodiment is shown;

[0045] Figure 8 A block diagram of a video decoder according to another example embodiment is shown;

[0046] Figure 9 A directional intra-frame prediction mode according to an example embodiment of the present disclosure is shown;

[0047] Figure 10 A non-directional intra-frame prediction mode according to an example embodiment of the present disclosure is shown;

[0048] Figure 11 A recursive intra-frame prediction mode according to an example embodiment of the present disclosure is shown;

[0049] Figure 12 This demonstrates transform block partitioning and scanning of intra-prediction blocks according to an exemplary embodiment of the present disclosure;

[0050] Figure 13 This demonstrates the transform block partitioning and scanning of inter-frame prediction blocks according to an exemplary embodiment of the present disclosure;

[0051] Figure 14 A flowchart is shown according to an example embodiment of the present disclosure;

[0052] Figure 15 A schematic diagram of a computer system according to an example embodiment of the present disclosure is shown; Detailed Implementation

[0053] Figure 3 A simplified block diagram of a communication system (300) according to an embodiment of the present disclosure is shown. The communication system (300) includes a plurality of terminal devices capable of communicating with each other via, for example, a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). Figure 3In the example, the first pair of terminal devices (310) and (320) can perform unidirectional data transmission. For example, terminal device (310) can encode video data (e.g., a video image stream captured by terminal device (310)) for transmission via network (350) to another terminal device (320). The encoded video data can be transmitted in the form of one or more encoded video bitstreams. Terminal device (320) can receive the encoded video data from network (350), decode the encoded video data to recover the video images, and display the video images based on the recovered video data. Unidirectional data transmission can be implemented in media service applications, etc.

[0054] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of encoded video data, which can be implemented, for example, during a video conferencing application. For bidirectional data transmission, in one example, each of the terminal devices (330) and (340) can encode video data (e.g., a stream of video images captured by the terminal device) for transmission via a network (350) to the other terminal device (330) and (340). Each of the terminal devices (330) and (340) can also receive encoded video data transmitted by the other terminal device (330) and (340), and can decode the encoded video data to recover the video images, and can display the video images on an accessible display device based on the recovered video data.

[0055] exist Figure 3 In the examples, the terminal devices (310), (320), (330), and (340) can be implemented as servers, personal computers, and smartphones, but the application of the underlying principles of this disclosure is not limited to these. Embodiments of this disclosure can be implemented in desktop computers, laptop computers, tablet computers, media players, wearable computers, dedicated video conferencing equipment, etc. Network (350) refers to any number or type of network that transmits encoded video data between the terminal devices (310), (320), (330), and (340), including, for example, wired (wired) and / or wireless communication networks. The communication network (350) can exchange data in circuit-switched, packet-switched, and / or other types of channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, the architecture and topology of the network (350) may be of little importance to the operation of this disclosure unless explicitly explained herein.

[0056] As an example of the application of the disclosed topic, Figure 4The diagram illustrates the placement of a video encoder and video decoder within a video streaming environment. The disclosed concepts can also be applied to other video applications, including, for example, video conferencing, digital television, broadcasting, gaming, virtual reality, and storing compressed video on digital media such as CDs, DVDs, and Memory Sticks.

[0057] The video streaming system may include: a video capture subsystem (413) that may include a video source (401), such as a digital camera; and the ability to create uncompressed video pictures or image streams (402). In one example, the video picture stream (402) includes samples recorded by the digital camera of the video source 401. The video picture stream (402) is depicted as a thick line emphasizing the high data volume when compared to encoded video data (404) (or encoded video bitstream), and may be processed by an electronic device (420) including a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to implement or enforce aspects of the disclosed subject matter as described in more detail below. The encoded video data (404) (or encoded video bitstream (404)) is depicted as a thin line to emphasize the lower data volume when compared to the uncompressed video picture stream (402), and may be stored on a streaming server (405) for future use or directly stored to a downstream video device (not shown). One or more streaming client subsystems (e.g., Figure 4 The client subsystems (406) and (408) can access the streaming server (405) to retrieve copies (407) and (409) of the encoded video data (404). The client subsystem (406) may include, for example, a video decoder (410) in an electronic device (430). The video decoder (410) decodes the input copy (407) of the encoded video data and creates an uncompressed output video picture stream (411) that can be presented on a display (412) (e.g., a screen) or other presentation device (not shown). The video decoder 410 may be configured to perform some or all of the various functions described in this disclosure. In some streaming systems, the encoded video data (404), (407), and (409) (e.g., a video bitstream) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In one example, the video coding standard under development is informally referred to as Universal Video Coding (VVC). The disclosed subject matter can be used in the context of VVC and other video coding standards.

[0058] Note that electronic devices (420) and (430) may include other components (not shown). For example, electronic device (420) may include a video decoder (not shown), and electronic device (430) may also include a video encoder (not shown).

[0059] Figure 5 A block diagram of a video decoder (510) according to any embodiment of the present disclosure is shown. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., receiving circuitry). The video decoder (510) may be used in place of... Figure 4 The video decoder (410) in the example.

[0060] The receiver (531) can receive one or more encoded video sequences to be decoded by the video decoder (510). In the same or another embodiment, one encoded video sequence can be decoded at a time, wherein the decoding of each encoded video sequence is independent of other encoded video sequences. Each video sequence can be associated with multiple video frames or images. Encoded video sequences can be received from a channel (501), which can be a hardware / software link to a storage device storing encoded video data or a streaming source for transmitting encoded video data. The receiver (531) can receive encoded video data and other data, such as encoded audio data and / or auxiliary data streams, which can be forwarded to their respective processing circuitry (not shown). The receiver (531) can separate the encoded video sequences from other data. To combat network jitter, a buffer memory (515) can be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) can be implemented as part of the video decoder (510). In other applications, buffer memory (not shown) may be present outside of and separate from the video decoder (510). In other applications, a buffer memory (not shown) may be present outside the video decoder (510), for example, to combat network jitter, and another additional buffer memory (515) may be present inside the video decoder (510), for example, to handle playback timing. The buffer memory (515) may be unnecessary or small when the receiver (531) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous network. For use on best-effort packet networks such as the Internet, a sufficiently large buffer memory (515) may be required, and its size may be relatively large. Such a buffer memory may be implemented with an adaptive size and may be implemented, at least partially, in an operating system or similar component (not shown) outside the video decoder (510).

[0061] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from the encoded video sequence. These symbols include information for managing the operation of the video decoder (510) and information that potentially controls a presentation device such as a display (512) (e.g., a screen), which may or may not be part of the electronic device (530), but may be coupled to the electronic device (530), such as... Figure 5 As shown. The control information used to present the device may be in the form of supplemental enhancement information (SEI message) or video availability information (VUI) parameter set fragments (not shown). The parser (520) may perform parsing / entropy decoding on the encoded video sequence received by the parser (520). The entropy coding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a set of subgroup parameters of at least one pixel subgroup from the encoded video sequence based on at least one parameter corresponding to the subgroup. The subgroup may include picture group (GOP), picture, tile, slice, macroblock, coding unit (Cu), block, transform unit (TU), prediction unit (PU), etc. The parser (520) may also extract information from the encoded video sequence, such as transform coefficients (e.g., Fourier transform coefficients), quantizer parameter values, motion vectors, etc.

[0062] The parser (520) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) in order to create symbols (521).

[0063] Depending on the type of encoded video picture or its components (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors, the reconstruction of the symbol (521) can involve multiple different processing or functional units. The involved units and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (520). For simplicity, the flow of such subgroup control information between the parser (520) and the multiple processing or functional units below is not described.

[0064] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into multiple functional units as described below. In practical implementations operating under commercial constraints, many of these functional units interact closely with each other and can be at least partially integrated with one another. However, for the sake of clarity in describing the various functions of the disclosed subject matter, the following disclosure adopts a conceptual subdivision into the following functional units.

[0065] The first unit may include a scaler / inverse transform unit (551). The scaler / inverse transform unit (551) may receive quantized transform coefficients and control information, including information on the type of inverse transform used, block size, quantization factor / parameter, quantization scaling matrix, etc., as symbols (521) from the parser (520). The scaler / inverse transform unit (551) may output blocks containing sample values, which may be input into the aggregator (555).

[0066] In some cases, the output samples of the scaler / inverse transform unit (551) may belong to intra-coded blocks, i.e., blocks that do not use prediction information from previously reconstructed images, but can use prediction information from previously reconstructed portions of the current image. This prediction information can be provided by the intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) uses the surrounding block information already reconstructed and stored in the current picture buffer (558) to generate blocks with the same size and shape as the blocks in the reconstruction. The current picture buffer (558) buffers, for example, partially reconstructed current images and / or fully reconstructed current images. In some implementations, the aggregator (555) may add the prediction information already generated by the intra-picture prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551) based on each sample.

[0067] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to inter-frame coded blocks and may be motion-compensated. In this case, the motion compensation prediction unit (553) can access the reference image memory (557) to obtain samples for prediction. After motion compensation of the extracted samples according to the symbols (521) associated with the block, these samples can be added by the aggregator (555) to the output of the scaler / inverse transform unit (551) (the output of unit 551 may be referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference image memory (557) from which the motion compensation prediction unit (553) obtains the predicted samples can be controlled by motion vectors, and the motion compensation prediction unit (553) can obtain these addresses in the form of symbols (521), which may have, for example, X, Y components (offset) and reference image components (time). When using subsampled precise motion vectors, motion compensation may also include interpolation of sampled values ​​obtained from the reference image memory (557) and is also associated with motion vector prediction mechanisms, etc.

[0068] The output samples of the aggregator (555) can undergo various loop filtering techniques in the loop filter unit (556). Video compression techniques may include loop filtering techniques controlled by parameters contained in the encoded video sequence (also known as the encoded video bitstream) and available to the loop filter unit (556) as symbols (521) from the parser (520), but may also be in response to metadata obtained during decoding of a previous (in the order of decoding) portion of the encoded image or encoded video sequence, as well as to previously reconstructed and loop-filtered sample values. Several types of loop filters may be included in various orders as part of the loop filter unit 556, which will be described in further detail below.

[0069] The output of the loop filter unit (556) can be a sample stream, which can be output to the presentation device (512) and stored in the reference image memory (557) for future inter-frame image prediction.

[0070] Once fully reconstructed, certain encoded images can be used as reference images for future inter-frame image prediction. For example, once the encoded image corresponding to the current image has been fully reconstructed and that encoded image has been identified as a reference image (e.g., by the parser (520)), the current image buffer (558) can become part of the reference image memory (557), and a new current image buffer can be reallocated before the reconstruction of the next encoded image begins.

[0071] The video decoder (510) can perform decoding operations according to a predetermined video compression technique adopted in standards such as ITU-T Rec.H.265. The encoded video sequence can conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence conforms to the syntax of the video compression technique or standard and the brief documented in the video compression technique or standard. Specifically, the brief may select certain tools from all available tools in the video compression technique or standard as the only tools available under that brief. The complexity of the encoded video sequence for conformance can be within the range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (e.g., measured in megasamples per second), maximum reference picture size, etc. In some cases, the limitations set by the level can be further restricted by the assumed reference decoder (HRD) specification and metadata managed by the HRD buffer of signaling in the encoded video sequence.

[0072] In some example embodiments, the receiver (531) may receive additional (redundant) data with encoded video. This additional data may be included as part of the encoded video sequence. The video decoder (510) may use the additional data to correctly decode the data and / or more accurately reconstruct the original video data. The additional data may be, for example, in the form of temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0073] Figure 6 A block diagram of a video encoder (603) according to an example embodiment of the present disclosure is shown. The video encoder (603) may be included in an electronic device (620). The electronic device (620) may also include a transmitter (640) (e.g., a transmitting circuit). The video encoder (603) may be used in place of... Figure 4 The video encoder (403) in the example.

[0074] The video encoder (603) can obtain data from the video source (601) (which is not...). Figure 6 In one example, a portion of an electronic device (620) receives a video sample, the video source of which can capture video images to be encoded by a video encoder (603). In another example, a video source (601) can be implemented as part of an electronic device (620).

[0075] The video source (601) may be provided as a digital video sample stream of a sequence of source videos to be encoded by a video encoder (603). This digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601YCrCb, RGB, ...), and any suitable sampling structure (e.g., YCrCb4:2:0, YCrCb4:4:4). In a media service system, the video source (601) may be a storage device capable of storing previously prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. Video data may be provided as a plurality of individual pictures or images, which, when viewed sequentially, are given motion. The pictures themselves may be organized as a spatial array of pixels, wherein each pixel may include one or more samples, depending on the sampling structure, color space, etc., used. Those skilled in the art will readily understand the relationship between pixels and samples. The following description focuses on samples.

[0076] According to some example embodiments, the video encoder (603) can encode and compress images of a source video sequence into an encoded video sequence (643) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate constitutes a function of the controller (650). In some embodiments, the controller (650) may be functionally coupled to and control other functional units as described below. For simplicity, coupling is not described. Parameters set by the controller (650) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured to have other suitable functions related to the video encoder (603) optimized for a particular system design.

[0077] In some example embodiments, the video encoder (603) is configured to operate within an encoding loop. As an oversimplification, in one example, the encoding loop may include a source encoder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on the input picture to be encoded and a reference picture) and a (local) decoder (633) embedded within the video encoder (603). The decoder (633) reconstructs the symbols in a manner similar to that created by the (remote) decoder to create sample data, even though the embedded decoder 633 processes the video stream encoded by the source encoder 630 without entropy coding (because any compression between the symbols and the encoded video bitstream in entropy coding is likely lossless in video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since decoding of the symbol stream results in bit-accurate results independent of the decoder location (local or remote), the contents of the reference picture memory (634) are also bit-accurate between the local and remote encoders. In other words, when prediction is used during decoding, the encoder's prediction portion, acting as a reference picture sample, "sees" the exact same sample values ​​as the decoder. This fundamental principle of reference picture synchronization (and the resulting drift, if synchronization cannot be maintained, for example, due to channel errors) is used to improve coding quality.

[0078] The operation of the "local" decoder (633) can be the same as that of a "remote" decoder, such as a video decoder (510), as described above. Figure 5 It was described in detail. However, a brief reference is provided. Figure 5Since symbols are available and the encoding / decoding of symbols for the encoded video sequence by the entropy encoder (645) and the parser (520) can be lossless, the entropy decoding part of the video decoder (510), including the buffer memory (515) and the parser (520), can be implemented in the local decoder (633) in the encoder without being fully integrated.

[0079] At this point, it can be observed that, apart from the parsing / entropy decoding that may exist in the decoder, any decoder technique does not necessarily need to exist in the corresponding encoder in essentially the same functional form. Therefore, the disclosed subject matter can sometimes focus on decoder operations that are similar to the decoding part of the encoder. Thus, the description of encoder techniques can be simplified, as these techniques are the inverse of a fully described decoder technique. Only in certain areas or aspects is a more detailed description of the encoder provided below.

[0080] During operation, in some example implementations, the source encoder (630) may perform motion-compensated predictive coding, which predictively encodes the input image by referencing one or more previously encoded images from the video sequence designated as "reference images." In this way, the encoding engine (632) encodes the differences (or residuals) in the color channel between pixel blocks of the input image and pixel blocks of the reference image, which can be selected as a predictive reference to the input image. The term "residual" and its adjective form "residual" are used interchangeably.

[0081] The local video decoder (633) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (630). The operation of the encoding engine (632) can advantageously be a lossy process. When the encoded video data can be decoded by the video decoder (633... Figure 6 When decoded at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding process performed by the video decoder on the reference image, and can store the reconstructed reference image in a reference image cache (634). In this way, the video encoder (603) can locally store copies of the reconstructed reference images that have the same content as the reconstructed reference images that will be obtained by the remote video decoder (without transmission errors).

[0082] The predictor (635) can perform a prediction search on the encoding engine (632). That is, for a new image to be encoded, the predictor (635) can search the reference image memory (634) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., which can be used as appropriate prediction references for the new image. The predictor (635) can operate on a sample block-by-pixel basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (635), the input image may have prediction references extracted from multiple reference images stored in the reference image memory (634).

[0083] The controller (650) can manage the encoding operations of the source encoder (630), including, for example, the settings of parameters and subgroup parameters for encoding video data.

[0084] The outputs of all the aforementioned functional units can undergo entropy encoding (645) in the entropy encoder. The entropy encoder (645) converts the symbols generated by the various functional units into an encoded video sequence by lossless compression of the symbols according to techniques such as Huffman coding, variable length coding, and arithmetic coding.

[0085] The transmitter (640) can buffer the encoded video sequence created by the entropy encoder (645) in preparation for transmission via a communication channel (660), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0086] The controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can assign a specific encoded picture type to each encoded picture, which can affect the encoding techniques that can be applied to the corresponding picture. For example, a picture can typically be designated as one of the following picture types:

[0087] An intra-frame picture (I-picture) can be a picture that is encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (“IDR”) pictures. Those skilled in the art will recognize those variations of I-pictures and their corresponding applications and characteristics.

[0088] A predicted image (P-image) can be an image that uses at most one motion vector and a reference index to predict the sample value of each block, and is encoded and decoded using intra-frame prediction or inter-frame prediction.

[0089] A bidirectional prediction image (B-image) can be an image that uses up to two motion vectors and a reference index to predict sample values ​​for each block, and is encoded and decoded using intra-frame prediction or inter-frame prediction. Similarly, a multi-prediction image can reconstruct a single block using two or more reference images and associated metadata.

[0090] Source images can typically be spatially subdivided into multiple sample blocks (e.g., each sample block is 4×4, 8×8, 4×8, or 16×16 sample blocks) and encoded on a block-by-block basis. Blocks can be predictedly encoded by referencing other (already encoded) blocks determined by the encoding allocation applied to the corresponding images of the block. For example, blocks of image I can be unpredictably encoded, or they can be predicted (spatial prediction or intra-frame prediction) by referencing already encoded blocks of the same image. Pixel blocks of image P can be predictedly encoded by reference to a previously encoded reference image, either spatially or temporally. Blocks of image B can be predictedly encoded by reference to one or two previously encoded reference images, either spatially or temporally. For other purposes, source images or intermediate images can be subdivided into other types of blocks. The partitioning of encoded blocks and other types of blocks can follow or not follow the same pattern, as described in further detail below.

[0091] The video encoder (603) can perform encoding operations according to a predetermined video coding technique or standard (e.g., ITU-T Rec.H.265). In its operation, the video encoder (603) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard being used.

[0092] In some example embodiments, the transmitter (640) may transmit additional data along with the encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data (e.g., redundant images and slices), SEI messages, VUI parameter set fragments, etc.

[0093] Video can be captured as multiple source images (video pictures) in a time series. Intra-frame picture prediction (often abbreviated as intra-prediction) utilizes spatial correlations within a given picture, while inter-frame picture prediction utilizes temporal or other correlations between pictures. In one example, a specific picture in the encoding / decoding process, referred to as the current picture, is divided into blocks. When a block in the current picture resembles a reference block in a previously encoded and still buffered reference picture in the video, the block in the current picture can be encoded using a vector called a motion vector. In the case of using multiple reference pictures, the motion vector points to the reference block in the reference picture and can have a third dimension that identifies the reference picture.

[0094] In some example embodiments, bidirectional prediction techniques can be used in inter-frame image prediction. According to bidirectional prediction, two reference images are used, for example, a first reference image and a second reference image, both of which precede the current image in the video in the decoding order (but can be displayed in the past or future, respectively). A block in the current image can be encoded by a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. The block can be jointly predicted by combining the first and second reference blocks.

[0095] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0096] According to some example embodiments of this disclosure, prediction is performed on a block-by-block basis, such as inter-frame picture prediction and intra-frame picture prediction. For example, pictures in a video picture sequence are divided into coding tree units (CTUs) for compression, with CTUs in the pictures having the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Typically, a CTU may include three parallel coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU may be recursively quadtree-divided into one or more coding units (CUs). For example, a 64×64 pixel CTU may be divided into one 64×64 pixel CU or four 32×32 pixel CUs. Each of one or more of the 32×32 blocks is further divided into four 16×16 pixel CUs. In some example embodiments, each CU may be analyzed during encoder operation to determine the prediction type of the CU among various prediction types, such as inter-frame prediction type or intra-frame prediction type. Based on temporal and / or spatial predictability, CUs are divided into one or more prediction units (PUs). Typically, each PU includes one luma prediction block (PB) and two chroma PBs. In embodiments, prediction operations in encoding (encoding / decoding) are performed on a block-by-block basis. A CU can be divided into PUs (or different color channel PBs) in various spatial patterns. For example, a luma or chroma PB may contain a matrix of sample values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 samples, etc.

[0097] Figure 7 A diagram of a video encoder (703) according to another example embodiment of the present disclosure is shown. The video encoder (703) is configured to receive a processing block (e.g., a prediction block) of sample values ​​within a current video image in a video image sequence, and to encode the processing block into an encoded image as part of an encoded video sequence. The example video encoder (703) can be used instead of Figure 4 The video encoder (403) in the example.

[0098] For example, the video encoder (703) receives a sample value matrix of a processing block, such as an 8×8 sample prediction block. The video encoder (703) then determines the optimal encoding method for the processing block: intra-frame mode, inter-frame mode, or bidirectional prediction mode, such as rate-distortion optimization (RDO). When it is determined that the processing block should be encoded in intra-frame mode, the video encoder (703) can use intra-frame prediction techniques to encode the processing block into an encoded picture; and when it is determined that the processing block should be encoded in inter-frame mode or bidirectional prediction mode, the video encoder (703) can use inter-frame prediction or bidirectional prediction techniques to encode the processing block into an encoded picture, respectively. In some example embodiments, a merge mode can be used as a sub-mode for inter-frame picture prediction, wherein motion vectors are derived from one or more motion vector predictors without benefiting from encoded motion vector components outside the predictors. In some example embodiments, there may be motion vector components applicable to object blocks. Therefore, the video encoder (703) can include... Figure 7 Components not explicitly shown, such as the mode determination module for determining the prediction mode of the processing block.

[0099] exist Figure 7 In the example, the video encoder (703) includes, for example, Figure 7 The example arrangement shows an inter-frame encoder (730), an intra-frame encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725) coupled together.

[0100] An inter-frame encoder (730) is configured to receive samples of the current block (e.g., the processing block), compare that block with one or more reference blocks in a reference image (e.g., blocks in previous and subsequent images in display order), generate inter-frame prediction information (e.g., descriptions of redundancy information based on inter-frame coding techniques, motion vectors, merging mode information), and compute inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some examples, the reference image is a decoded reference image based on the decoded coded video information, using embedded... Figure 6 Decoding unit 633 in example encoder 620 (shown as) Figure 7 The residual decoder 728, as described in further detail below.

[0101] The intra encoder (722) is configured to receive samples of the current block (e.g., the processing block), compare the block with blocks already encoded in the same image, generate quantization coefficients after transformation, and in some cases, also generate intra prediction information (e.g., intra prediction direction information based on one or more intra coding techniques). The intra encoder (722) can compute intra prediction results (e.g., prediction blocks) based on intra prediction information and reference blocks in the same image.

[0102] The general controller (721) can be configured to determine general control data and control other components of the video encoder (703) based on the general control data. In one example, the general controller (721) determines the prediction mode of a block and provides control signals to the switch (726) based on the prediction mode. For example, when the prediction mode is intra-frame mode, the general controller (721) controls the switch (726) to select intra-frame mode results for use by the residual calculator (723) and controls the entropy encoder (725) to select intra-frame prediction information and include the intra-frame prediction information in the bitstream; when the prediction mode of the block is inter-frame mode, the general controller (721) controls the switch (726) to select inter-frame prediction results for use by the residual calculator (723) and controls the entropy encoder (725) to select inter-frame prediction information and include the inter-frame prediction information in the bitstream.

[0103] A residual calculator (723) can be configured to calculate the difference (residual data) between a received block and the prediction result of that block selected from the intra encoder (722) or inter encoder (730). A residual encoder (724) can be configured to encode the residual data to generate transform coefficients. In one example, the residual encoder (724) can be configured to transform the residual data from the spatial domain to the frequency domain to generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various example embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be appropriately used by the intra encoder (722) and the inter encoder (730). For example, the inter encoder (730) can generate decoded blocks based on the decoded residual data and inter-frame prediction information, and the intra encoder (722) can generate decoded blocks based on the decoded residual data and intra-frame prediction information. The decoded blocks are processed appropriately to generate a decoded image, which can be cached in a memory circuit (not shown) and used as a reference image.

[0104] The entropy encoder (725) can be configured to format the bitstream to include coded blocks and perform entropy encoding. The entropy encoder (725) is configured to include various information in the bitstream. For example, the entropy encoder (725) can be configured to include general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. Residual information may be omitted when encoding blocks in a merged sub-mode of inter-frame mode or bidirectional prediction mode.

[0105] Figure 8 A diagram illustrating an example video decoder (810) according to another embodiment of the present disclosure is shown. The video decoder (810) is configured to receive an encoded picture as part of an encoded video sequence and decode the encoded picture to generate a reconstructed picture. In one example, the video decoder (810) may be used instead of Figure 4 The video decoder (410) in the example.

[0106] exist Figure 8 In the example, the video decoder (810) includes, for example, Figure 8 The example arrangement shows the entropy decoder (871), inter-frame decoder (880), residual decoder (873), reconstruction module (874), and intra-frame decoder (872) coupled together.

[0107] The entropy decoder (871) can be configured to reconstruct certain symbols representing the syntax elements constituting the encoded image from the encoded image. Such symbols may include, for example, the mode encoding the block (e.g., intra-mode, inter-mode, bidirectional prediction mode, merged sub-mode, or another sub-mode), prediction information (e.g., intra-prediction information or inter-prediction information) that can identify specific samples or metadata used for prediction by the intra-decoder (872) or inter-decoder (880), such as residual information in the form of quantization transform coefficients, etc. In one example, when the prediction mode is inter-mode or bidirectional prediction mode, inter-prediction information is provided to the inter-decoder (880); and when the prediction type is intra-prediction type, intra-prediction information is provided to the intra-decoder (872). The residual information may undergo inverse quantization and be provided to the residual decoder (873).

[0108] The inter-frame decoder (880) can be configured to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0109] The intra-frame decoder (872) can be configured to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0110] The residual decoder (873) can be configured to perform inverse quantization to extract the dequantized transform coefficients and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also utilize some control information (including quantizer parameters (QP)) that can be provided by the entropy decoder (871) (the data path is not shown because this may only be low data volume control information).

[0111] The reconstruction module (874) can be configured to combine the residual output by the residual decoder (873) and the prediction results (output by the inter-frame or intra-frame prediction module, as appropriate) in the spatial domain to form a reconstruction block that forms part of the reconstructed image as part of the reconstructed video. Note that other suitable operations, such as deblocking, can also be performed to improve visual quality.

[0112] Note that the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) can be implemented using any suitable technology. In some example embodiments, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) can be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603), and (603) and the video decoders (410), (510), and (810) can be implemented using one or more processors that execute software instructions.

[0113] Returning to the intra-prediction process, samples in a block (e.g., a luma or chroma prediction block, or a coded block, if not further divided into prediction blocks) are predicted from adjacent, next-adjacent, or other rows of samples, or combinations thereof, to generate a prediction block. The residual between the actual block being encoded and the prediction block can then be processed via transform and subsequent quantization. Various intra-prediction modes can be made available, and parameters related to intra-prediction mode selection and other parameters can be signaled in the bitstream. For example, various intra-prediction modes can involve one or more row positions for predicting samples, the direction along which the predicting samples are selected from one or more prediction rows, and other special intra-prediction modes.

[0114] For example, a set of intra-prediction modes (which may be interchangeably referred to as "intra-modes") may include a predetermined number of directional intra-prediction modes. As described above with respect to the example implementation of Figure 1, these intra-prediction modes may correspond to a predetermined number of directions along which out-of-block samples are selected as predictions for samples being predicted in a particular block. In another particular example implementation, eight (8) master directional modes corresponding to angles from 45° to 207° with respect to the horizontal axis may be supported and predefined.

[0115] In some other implementations of intra-frame prediction, to further utilize the greater variety of spatial redundancy in oriented textures, oriented intra-frame modes can be extended to have a more fine-grained set of angles. For example, the 8-angle implementation above can be configured to provide eight nominal angles, named V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED, as shown below. Figure 9 As shown, and for each nominal angle, a predetermined number (e.g., 7) of finer angles can be added. With this expansion, corresponding to the same number of predefined directional intra-frame patterns, a larger total number of directional angles (e.g., 56 in this example) can be used for intra-frame prediction. The predicted angle can be represented by the nominal intra-frame angle plus an angle increment. For the specific example above with 7 finer angular directions per nominal angle, the angle increment can be a step size from -3 to 3 multiplied by 3°.

[0116] In some implementations, as an alternative to or supplement to the aforementioned directional intra-prediction modes, a predetermined number of non-directional intra-prediction modes can be predefined and made available. For example, five non-directional intra-prediction modes, referred to as smooth intra-prediction modes, can be specified. These non-directional intra-prediction modes can be specifically named DC, PAETH, SMOOTH, SMOOTH_V, and SMOOTH_H intra-prediction modes. Figure 10 The image shows predictions for samples of a specific block under these example non-directional modes. As an example, Figure 10A 4×4 block 1002 is shown, predicted by samples from the top adjacent row and / or the left adjacent row. A specific sample 1010 in block 1002 can directly correspond to the top sample 1004 of sample 1010 in the top adjacent row of block 1002, the top-left sample 1006 of sample 1010 which is the intersection of the top and left adjacent rows, and the direct left sample 1008 of sample 1010 in the left adjacent row of block 1002. For the example DC intra-prediction mode, the average of the left and top adjacent samples 1008 and 1004 can be used as the predicted value of sample 1010. For the example PAETH intra-prediction mode, the top, left, and top-left reference samples 1004, 1008, and 1006 can be obtained, and the value closest to (top + left - top-left) among these three reference samples can be set as the predicted value of sample 1010. For the example SMOOTH_V intra-prediction mode, sample 1010 can be predicted by quadratic interpolation in the vertical direction between the top-left neighbor sample 1006 and the left neighbor sample 1008. For the example SMOOTH_H intra-prediction mode, sample 1010 can be predicted by quadratic interpolation in the horizontal direction between the top-left neighbor sample 1006 and the top neighbor sample 1004. For the example SMOOTH intra-prediction mode, sample 1010 can be predicted by averaging the quadratic interpolations in the vertical and horizontal directions. The above non-directional intra-prediction mode implementations are only illustrative examples and are not restrictive. Other adjacent rows, other non-directional sample selections, and methods of combining predicted samples to predict specific samples in the prediction block are also considered.

[0117] The encoder can signal specific intra-prediction modes selected from the aforementioned directional or non-directional modes at various coding levels (picture, slice, block, unit, etc.) within the bitstream. In some example implementations, the eight exemplary nominal directional modes and five non-angular smoothing modes (a total of 13 options) can be signaled first. Then, if the signaled mode is one of the eight nominal angular intra-prediction modes, further signaling indexing is used to indicate the selected angular increment relative to the nominal angle of the corresponding signal. In some other example implementations, all intra-prediction modes can be indexed together (e.g., 56 directional modes plus 5 non-directional modes, resulting in 61 intra-prediction modes) for signaling.

[0118] In some example implementations, example 56 or other numbers of directional intra-frame prediction modes can be implemented using a unified direction predictor that projects each sample of the block to a reference subsample location and interpolates the reference sample through a 2-tap bilinear filter.

[0119] In some implementations, additional filter patterns, referred to as filter intra-patterns, can be designed to capture the attenuation spatial correlation with references on the edges. For these patterns, in addition to out-of-block samples, intra-block predicted samples can also be used as intra-prediction reference samples for some patches within the block. For example, these patterns can be predefined and used for intra-prediction of at least luma blocks (or luma blocks only). A predetermined number (e.g., five) of filter intra-patterns can be pre-designed, each pattern represented by a set of n-tap filters (e.g., 7-tap filters) reflecting the correlation between samples in, for example, a 4×2 patch and its n neighboring samples. In other words, the weighting factors of the n-tap filters can be position-dependent. For example, with an 8×8 block, 4×2 patch, and 7-tap filtering, as shown... Figure 11 As shown, an 8×8 block of 1102 can be divided into eight 4×2 patches. In Figure 11 In this context, these patches are represented by B0, B1, B1, B3, B4, B5, B6, and B7. For each patch, its 7 neighbors (in...) Figure 11 The values ​​(denoted by R0 to R7) can be used to predict samples in the current patch. For patch B0, all neighbors may have already been reconstructed. However, for other patches, some neighbors are in the current block and may not have been reconstructed; in this case, the predicted values ​​of the direct neighbors are used as a reference. For example, ... Figure 11 None of the neighbors of patch B7 shown have been reconstructed, so the predicted samples of the neighbors are used instead.

[0120] In some implementations of intra-frame prediction, one or more other color components can be used to predict a single color component. The color component can be any component from the YCrCb, RGB, XYZ color space, etc. For example, it is possible to predict a chromaticity component (e.g., a chromaticity patch) from a luminance component (e.g., a luminance reference sample), a process known as luminance-to-chromaticity prediction, or CfL. In some example implementations, only cross-color prediction from luminance to chromaticity may be allowed. For example, chromaticity samples in a chromaticity patch can be modeled as a linear function of overlapping reconstructed luminance samples. CfL prediction can be implemented as follows:

[0121] CfL(α)=α×L AC +DC (1)

[0122] Among them, L ACLet α represent the AC contribution of the luminance component, α represent the parameters of the linear model, and DC represent the DC contribution of the chrominance component. For example, the AC component is obtained for each sample of the block, while the DC component is obtained for the entire block. Specifically, the reconstructed luminance samples can be subsampled to chrominance resolution, and then the average luminance value (DC of luminance) can be subtracted from each luminance value to form the AC contribution in luminance. The AC contribution of luminance is then used in the linear mode of equation (1) to predict the AC value of the chrominance component. In order to approximate or predict the chrominance AC component from the luminance AC contribution, rather than requiring the decoder to calculate the scaling parameters, the example CfL implementation can determine the parameter α based on the original chrominance samples and signal it in the bitstream. This reduces the complexity of the decoder and produces more accurate predictions. As for the DC contribution of the chrominance component, in some example implementations, it can be calculated using the intra-frame DC mode within the chrominance component.

[0123] The residuals of the intra-frame or inter-frame prediction blocks can then be transformed, followed by quantization of the transform coefficients. For the purpose of performing the transform, the intra-frame and inter-frame coded blocks can be further divided into multiple transform blocks (sometimes used interchangeably as "transform units," even though the term "unit" is generally used to refer to a set of three-color channels; for example, a "coding unit" would include a luma-coded block and a chroma-coded block). In some implementations, the maximum partition depth of the coded block (or prediction block) can be specified (the term "coded block" can be used interchangeably with "coded block"). For example, this partitioning may not exceed two levels. The partitioning of prediction blocks into transform blocks can be handled differently between intra-frame and inter-frame prediction blocks. However, in some implementations, this partitioning can be similar between intra-frame and inter-frame prediction blocks.

[0124] In some example implementations, intra-frame coded blocks can be divided into transform blocks of the same size and encoded in raster scan order. Figure 12 The image shows an example of such transform block partitioning for intra-frame coded blocks. Specifically, Figure 12 The diagram illustrates how encoded block 1202 is divided into 16 transform blocks of equal size via intermediate-level quadtree splitting 1204, as shown in 1206. An example raster scan order for encoding is... Figure 12 The ordered arrows are shown in the diagram.

[0125] In some example implementations, for inter-frame coded blocks, transform unit partitioning can be performed recursively, with a partitioning depth reaching a predefined number of levels (e.g., 2 levels). Figure 13 As shown, for any subpartition and any level, the partitioning can recursively stop or continue. Specifically, Figure 13An example is shown where block 1302 is divided into four quadtree sub-blocks 1304, and one of these sub-blocks is further divided into four second-level transform blocks, while the division of the other sub-blocks stops after the first level, resulting in a total of seven transform blocks of two different sizes. The example raster scan order used for encoding is... Figure 13 The ordered arrows in the diagram further illustrate this. Although Figure 13 An example implementation of quadtree partitioning for square transform blocks up to two levels is shown. However, in some generative implementations, transform partitioning can support transform block shapes and sizes of 1:1 (square), 1:2 / 2:1, and 1:4 / 4:1, ranging from 4×4 to 64×64. In some example implementations, if the coded block is less than or equal to 64×64, the transform block partitioning can be applied only to the luma component (in other words, the chroma transform block will be the same as the coded block under this condition). Otherwise, if the coded block width or height is greater than 64, the luma and chroma coded blocks can be implicitly partitioned into multiples of min(W,64)×min(H,64) and min(W,32)×min(H,32) transform blocks, respectively.

[0126] Each of the above transform blocks can then undergo a primary transform. The primary transform essentially moves the residuals in the transform block from the spatial domain to the frequency domain. In some implementations of the actual primary transform, to support the extended coded block partitioning example above, multiple transform sizes (ranging from 4 to 64 points for each of the two dimensions) and transform shapes (squares; rectangles with width / height ratios of 2:1 / 1:2 and 4:1 / 1:4) can be allowed.

[0127] Specifically, intra-mode coding is adopted, and for a particular coding block, coding unit, prediction block, or prediction unit within intra-frame coding, its intra-mode needs to be signaled by one or more syntax elements in the bitstream. As mentioned above, the number of possible intra-frame prediction modes can be large. For example, in some implementations described above, the number of available intra-frame prediction modes may include 62 different modes, including, for example, 56 directional intra-frame prediction modes (e.g., 8 nominal directions with 7 fine angles in each nominal direction), 5 non-directional modes, and one chroma-derived luma mode (for chroma components only). Once an intra-frame mode is selected during the coding process of a particular coding block, coding unit, prediction block, or prediction, the signaling corresponding to the selected intra-frame mode needs to be included in the bitstream. The signaling syntax must be able to distinguish all 62 modes in some way. For example, these 62 modes can be signaled using a single syntax with 62 indices, each index corresponding to a mode. In some other example implementations, one syntax can be signaled to indicate which nominal angle or non-directional mode is used as the nominal mode in the current block, and then, if the nominal mode of the current block is a directional mode, another syntax can be signaled to indicate which incremental angle is selected for the current block.

[0128] Since the various syntaxes associated with intra-frame coding typically occupy a large portion of the bitstream, and intra-frame mode selection must be signaled very frequently, for example, at various coding levels, reducing the number of bits used for intra-frame mode signaling becomes crucial for improving video coding efficiency. In practice, the use of various intra-frame prediction modes can follow certain statistical patterns, and these patterns can be used to design indexes for intra-frame mode and signaling syntaxes, thereby enhancing signaling efficiency. Furthermore, on average, there may be some correlation between intra-frame mode selections between blocks. This correlation can be obtained offline on a statistical basis and considered in the design of the signaling syntax used for selecting intra-frame modes. The goal is to reduce the average number of bits for signaling syntax elements in the coded bitstream. For example, some general statistics may indicate a strong correlation between the best intra-frame prediction modes of the current block and its neighboring blocks. This correlation can be utilized when designing the syntax for intra-frame mode coding.

[0129] The various example implementations below can be used individually or in any order. Furthermore, each of these implementations can be implemented as part of an encoder and / or decoder, and can be implemented in hardware or software. For example, it can be hard-coded in dedicated processing circuitry (e.g., one or more integrated circuits). In another example, it can be implemented by executing a program stored on a non-transitory computer-readable medium via one or more processors.

[0130] In some example implementations, the available intra-prediction modes or mode options for the currently encoded block can be divided into multiple sets of intra-prediction modes. Each set can be assigned a mode set index. Each set can contain multiple intra-prediction modes. Based on the correlation between intra-prediction modes between blocks, the partitioning and sorting of available intra-prediction modes, as well as the sorting of intra-prediction modes within each mode set, can be determined at least in part based on the intra-prediction modes used by neighboring blocks. The intra-prediction modes used by neighboring blocks can be referred to as "reference intra-prediction modes" or "reference modes." The intra-prediction mode for a specific unit can be determined and selected. The selection of the intra-prediction mode can be signaled. First, the mode set index can be signaled to indicate the mode set index of the set of intra-prediction modes containing the selected intra-prediction mode. Second, the mode index (or mode position index within the set) can be signaled to indicate the index of the selected intra-prediction mode within the mode set.

[0131] The general implementation of intra-prediction mode partitioning and sorting described above, along with the specific examples below, utilizes statistical effects and adjacency correlations to dynamically index these modes. This allows for optimization of the syntax used to select signaling patterns within the encoded video bitstream, improving coding efficiency. For example, these implementations can help reduce the number of signaling syntaxes and facilitate more efficient context generation for entropy coding.

[0132] In some example embodiments, the set of predefined intra-prediction modes available to the encoder may include P modes. For example, P may be 62 (e.g., 56 directional modes, 5 non-directional modes, and 1 luma mode for chroma mode), or any other predefined number. The number of intra-prediction modes in the set may be indicated by M, and the number of intra-prediction modes in each mode set may be indicated by N. i The value N is used to indicate the pattern set index, which ranges from, for example, 1 to M, or from 0 to M-1. i The maximum value in is represented by N. max .

[0133] In some example implementations, N max It can be restricted to numbers that are powers of 2. For example, N max It can be limited to 8, 16, or 32. This design helps to efficiently use signaling bits in the signaling syntax.

[0134] In some example implementations, when M is greater than 2, N of all M pattern sets i It can be restricted to having at most two distinct values. In this way, the syntax of the signaling pattern index within each pattern set can be simplified and streamlined.

[0135] In some example implementations, if the number of available intra-prediction modes for the current block is equal to N max If the number of patterns is a multiple of N, then the number of patterns in each pattern set is equal and is set to N. max In other words, if P can be obtained from N max If divisible, then each of the M pattern sets will contain the same N. max 16 intra-frame prediction modes. For example, if P = 64 and M = 4, then each mode set can contain 16 (N) intra-frame prediction modes. max =16, or 2 to the power of 4) intra-frame prediction mode. In this way, N i The number is the same across all M mode sets, which further optimizes the syntax design of the signaling mode index within the M mode sets for the selected intra-frame prediction mode.

[0136] In some example implementations, if the number of available intra-prediction modes for the current block (P) is not equal to N maxIf the number of intra-frame modes in a set is a multiple of N, then the number of intra-frame modes in a set can be less than N. max However, the number of intra-frame modes in all other mode sets is equal to N. max In other words, among all M pattern sets, N i It has two possible values. One of these values ​​can be N. max Another value can be less than N. max Of the M pattern sets, all except one with a smaller number of patterns have N patterns. max Intra-frame prediction modes. For example, P can be 62, M can be 4, N max The number of modes can be 16; three of the four mode groups can each contain 16 modes, and the other mode group can contain 14 modes. For example, the total number of available intra-prediction modes for the luma component can be 61, with M set to 4, N0 to 13, and N1, N2, and N3 all equal to 16. Furthermore, this implementation simplifies the syntax design for signaling mode indexing within the M mode groups for the selected intra-prediction mode.

[0137] In some example implementations, the (relative) positions of all non-directional modes are predetermined and fixed, independent of the intra-prediction modes of adjacent blocks. In other words, placing and arranging non-directional modes within the mode set and within the mode set of all available modes can be independent of the intra-prediction modes (or reference modes) of adjacent blocks, while arranging other intra-prediction modes within or within the mode set can depend on the reference mode. In this way, the dynamic determination of the mode set can be simplified, taking advantage of the statistics that non-directional modes may not be so correlated between adjacent blocks.

[0138] In some example implementations, all non-directional modes among the P available modes can be placed in the first intra-mode set (with the lowest mode set index), regardless of the intra-predictive modes of adjacent blocks.

[0139] In some example implementations, the DC prediction mode (non-directional mode) can always be placed in the first relative position (lowest position index within the mode set) of the first intra-mode set (the mode set with the lowest index), regardless of the intra-prediction modes of adjacent blocks.

[0140] In some example implementations, one or more recursive filtering-based intra-prediction modes can be signaled as sub-modes of the DC prediction mode. Therefore, a recursive filtering-based intra-prediction mode can only be signaled separately if the mode set index and the position index indicate that the DC mode has been selected (e.g., both the mode index and the position index are the lowest, e.g., both equal to 0 in the previous example implementation).

[0141] In some example implementations, the SMOOTH prediction mode, as described above, can always be placed in the second position (second lowest position index) of the first intra-mode set (the mode set with the lowest mode set index), regardless of the intra-prediction modes of adjacent blocks.

[0142] In some example implementations, the SMOOTH_V prediction mode, as described above, can always be placed in the third position (third lowest position index) of the first intra-mode set (the mode set with the lowest mode set index), regardless of the intra-prediction modes of adjacent blocks.

[0143] In some example implementations, the SMOOTH_H prediction mode, as described above, can always be placed in the fourth position (fourth lowest position index) of the first intra-mode set (the mode set with the lowest mode set index), regardless of the intra-prediction modes of adjacent blocks.

[0144] In some example implementations, the PAETH prediction mode, as described above, can always be placed in the fifth position (the fifth lowest position index) of the first intra-mode set (the mode set with the lowest mode set index), regardless of the intra-prediction modes of adjacent blocks.

[0145] In some other or further example implementations, the intra-prediction mode based on recursive filtering can be signaled independently of the DC intra-prediction mode, and can be one of the P modes rather than a sub-mode, and can be placed in the first intra-mode set (the set of modes with the lowest mode set index), independent of the intra-prediction modes of adjacent blocks.

[0146] In some further example implementations, there may be a set of recursive filtering-based intra-prediction modes, and one of them may be selected, and additional signaling may be used to indicate which recursive filtering-based intra-prediction mode to select in the set of recursive filtering-based intra-prediction modes for the current block.

[0147] In some example implementations, when independent of DC intra-prediction mode signaling, the intra-prediction mode based on recursive filtering can always be placed at the sixth position of the first intra-prediction mode set (the mode set with the lowest mode set index), regardless of the intra-prediction modes of adjacent blocks. In this case, the DC intra-prediction mode can be placed at the lowest index position in the lowest index mode set.

[0148] Figure 14A flowchart 1400 illustrates an example method following the basic principles of the above-described implementation for intra-frame mode coding. The example method flow begins at 1401. In S1410, an encoded video stream is received, the encoded video stream including a set index and a position index, which are used for intra-frame prediction of the current block. In S1420, an intra-frame prediction mode corresponding to at least one adjacent block in the video stream is determined as a reference mode. In S1430, based on the reference mode, a method for dividing P predefined intra-frame prediction modes into M intra-frame prediction mode sets is determined, each intra-frame prediction mode set including N... i Intra-prediction modes at position indices, where P and M are integers greater than 1, and N... i This represents the number of prediction modes in the i-th intra-prediction mode set, where i is an integer representing the set index of the M intra-prediction mode sets. In S1440, based on the received set index and position index, and the partitioning method, the target intra-prediction mode is identified among the P predefined intra-prediction modes. In S1450, the current block of the encoded video stream is decoded based at least on the target intra-prediction mode identified according to the set index and position index of the current block. The example method flow ends in S1499. The above method flow also applies encoding.

[0149] The embodiments of this disclosure can be used individually or in any combination in any order. Furthermore, each of the method (or embodiment), encoder, and decoder can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium. The embodiments of this invention can be applied to luma blocks or chroma blocks.

[0150] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 15 A computer system (1500) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0151] Computer software can be encoded using any suitable machine code or computer language, and can be assembled, compiled, linked or similarly to create code containing instructions that can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed through interpretation, microcode execution, etc.

[0152] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, and Internet of Things (IoT) devices.

[0153] Figure 15The components of the computer system (1500) shown are exemplary in nature and are not intended to impose any limitation on the scope or functionality of computer software used to implement embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement on any component or combination of components shown in the exemplary embodiments of the computer system (1500).

[0154] The computer system (1500) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users via, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., speech, clapping), visual input (e.g., gestures), and olfactory input (not shown). The human-machine interface device may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0155] The input human-machine interface device may include one or more of the following (only one of each is depicted): keyboard (1501), mouse (1502), trackpad (1503), touch screen (1510), data glove (not shown), joystick (1505), microphone (1506), scanner (1507), and camera (1508).

[0156] The computer system (1500) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (1510), data gloves (not shown), or joystick (1505), but may also include tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (1509), headphones (not shown)), visual output devices (e.g., screens (1510), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input capability, each with or without tactile feedback capability—some of which are capable of outputting two-dimensional or more than three-dimensional visual output in a manner such as stereoscopic output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)), and printers (not shown).

[0157] The computer system (1500) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1520) having CD / DVD or similar media (1520), thumb drives (1522), removable hard disk drives or solid-state drives (1523), conventional magnetic media such as magnetic tapes and floppy disks (not shown), dedicated ROM / ASIC / PLD devices such as security dongles (not shown), etc.

[0158] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0159] The computer system (1500) may also include an interface (1554) to one or more communication networks (1555). The network may be, for example, wireless, wired, or optical. The network may also be local area, wide area, metropolitan, vehicular, and industrial, real-time, latency-tolerant, etc. Examples of networks include local area networks such as Ethernet and wireless LANs; cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; cable or wireless wide area digital television networks including cable television, satellite television, and terrestrial broadcast television; and vehicular and industrial networks including CANBus, etc. Some networks typically require external network interface adapters to connect to certain general-purpose data ports or peripheral buses (1549) (e.g., a USB port of the computer system (1500); others are typically integrated into the core of the computer system (1500) via connection to system buses as described below (e.g., an Ethernet interface in a PC computer system or a cellular network interface in a smartphone computer system). Using any of these networks, the computer system (1500) can communicate with other entities. This communication can be unidirectional, receiving-only (e.g., broadcast television), unidirectional, transmitting-only (e.g., to a CANbus device), or bidirectional, such as to other computer systems using local or wide area digital networks. As mentioned above, certain protocols and protocol stacks can be used on each of these networks and network interfaces.

[0160] The aforementioned human-machine interface devices, human-accessible storage devices, and network interfaces can be attached to the core (1540) of the computer system (1500).

[0161] The core (1540) may include one or more central processing units (CPU) (1541), graphics processing units (GPUs) (1542), dedicated programmable processing units in the form of field-programmable gate areas (FPGAs) (1543), task-specific hardware accelerators (1544), graphics adapters (1550), etc. These devices, along with read-only memory (ROM) (1545), random access memory (1546), and internal mass storage such as internal non-user-accessible hard drives (SDs) (1547), may be connected via a system bus (1548). In some computer systems, the system bus (1548) may be accessed as one or more physical connectors to allow for the expansion of additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (1548) or via a peripheral bus (1549). In one example, a screen (1510) may be connected to a graphics adapter (1550). Peripheral bus architectures include PCI, USB, etc.

[0162] The CPU (1541), GPU (1542), FPGA (1543), and accelerator (1544) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (1545) or RAM (1546). Transient data can be stored in RAM (1546), while permanent data can be stored, for example, in internal mass storage (1547). Fast storage and retrieval of any storage device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1541), GPUs (1542), mass storage (1547), ROM (1545), RAM (1546), etc.

[0163] Computer-readable media may contain computer code for performing operations of various computer implementations. The media and computer code may be specifically designed and constructed for the purposes of this disclosure, or may be of a type known and available to those skilled in the art of computer software.

[0164] As a non-limiting example, a computer system having an architecture (1500), particularly a core (1540), can provide functionality as a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be a medium associated with a user-accessible mass storage as described above, as well as some memory of the non-transitory core (1540), such as internal mass storage (1547) or ROM (1545). Software implementing various embodiments of this disclosure can be stored in such a device and executed by the core (1540). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the core (1540), and particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (1546) and modifying such data structures according to software-defined processes. In addition, or alternatively, the computer system may provide functionality as a result of hard-wired or otherwise incorporated logic (e.g., an accelerator (1544)) that may replace or operate with software to perform the specific process or a specific portion of the specific process described herein. References to software may include logic, and vice versa, where appropriate. References to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry containing logic for execution, or both, where appropriate. This disclosure includes any suitable combination of hardware and software.

[0165] While several exemplary embodiments have been described in this disclosure, variations, substitutions, and various alternative equivalents fall within the scope of this disclosure. In the above implementations and embodiments, any operation of the process can be combined or arranged in any number or order as needed. Furthermore, two or more operations of the above process can be performed in parallel. Therefore, it should be understood that those skilled in the art will be able to design numerous systems and methods, although not explicitly shown or described herein, that embody the principles of this disclosure and are therefore within the spirit and scope of this disclosure.

[0166] Appendix A: Abbreviations

[0167] JEM: Collaborative Exploration Mode

[0168] VVC: Multi-functional Video Coding

[0169] BMS: Benchmark Set

[0170] MV: Motion Vector

[0171] HEVC: High-Efficiency Video Coding

[0172] SEI: Supplemental Enhancement Information

[0173] VUI: Video Availability Information

[0174] GOP: Image Group

[0175] TU: Transformation Unit

[0176] PU: Prediction Unit

[0177] CTU: Coding Tree Unit

[0178] CTB: Coded Tree Block

[0179] PB: Prediction Block

[0180] HRD: Assuming a reference decoder

[0181] SNR: Signal-to-noise ratio

[0182] CPU: Central Processing Unit

[0183] GPU: Graphics Processing Unit

[0184] CRT: Cathode Ray Tube

[0185] LCD: Liquid Crystal Display

[0186] OLED: Organic Light Emitting Diode

[0187] CD: CD-ROM

[0188] DVD: Digital Video Disc

[0189] ROM: Read-Only Memory

[0190] RAM: Random Access Memory

[0191] ASIC: Application-Specific Integrated Circuit

[0192] PLD: Programmable Logic Device

[0193] LAN: Local Area Network

[0194] GSM: Global System for Mobile Communications

[0195] LTE: Long Term Evolution

[0196] CANBus: Controller Area Network Bus

[0197] USB: Universal Serial Bus

[0198] PCI: Peripheral Component Interconnect

[0199] FPGA: Field Programmable Gate Domain

[0200] SSD: Solid State Drive

[0201] IC: Integrated Circuit

[0202] HDR: High Dynamic Range

[0203] SDR: Standard Dynamic Range

[0204] JVET: Joint Video Exploration Team

[0205] MPM: Most Likely Pattern

[0206] WAIP: Wide-angle Intra-frame Prediction

[0207] CU: Encoding Unit

[0208] PU: Prediction Unit

[0209] TU: Transformation Unit

[0210] CTU: Coding Tree Unit

[0211] PDPC: Location-Related Prediction Combination

[0212] ISP: Intra-Frame Sub-Partition

[0213] SPS: Sequence Parameter Settings

[0214] PPS: Image Parameter Set

[0215] APS: Adaptive Parameter Set

[0216] VPS: Video Parameter Set

[0217] DPS: Decoding Parameter Set

[0218] ALF: Adaptive Loop Filter

[0219] SAO: Sample Adaptive Offset; CC-ALF: Cross Component Adaptive Loop Filter; CDEF: Constraint Direction Enhancement Filter

[0220] CCSO: Cross Component Sample Offset

[0221] LSO: Local Sample Offset

[0222] LR: Loop Recovery Filter

[0223] AV1: AOMedia Video 1

[0224] AV2: AOMedia Video 2

Claims

1. A method for performing intra-frame prediction on the current block of an coded video stream, characterized in that, include: Receive an encoded video stream, the encoded video stream including a set index and a position index, the set index and the position index being used for intra-frame prediction of the current block; The intra-prediction mode corresponding to at least one neighboring block of the current block in the video stream is determined as the reference mode; Based on the reference mode, a method is determined to divide P predefined intra-prediction modes into M intra-prediction mode sets, each intra-prediction mode set including N... i Intra-prediction modes at position indices, where P and M are integers greater than 1, and N... i It is the number of prediction modes in the i-th intra-prediction mode set, where i is an integer representing the index of the intra-prediction mode set in the M intra-prediction mode sets; Based on the received set index and the position index, and the partitioning method, the target intra-prediction mode is identified among the P predefined intra-prediction modes. as well as The current block of the encoded video stream is decoded at least based on the target intra-frame prediction pattern identified according to the set index and the position index of the current block; Wherein, the P predefined intra-prediction modes include a subset of non-directional intra-prediction modes; and the relative order of the subsets of non-directional intra-prediction modes in the M intra-prediction mode sets is fixed and independent of the reference mode; or The P predefined intra-prediction modes include recursive filtering-based intra-prediction modes, which are arranged sequentially as the first intra-prediction mode set in the M intra-prediction mode set, independent of the reference mode; and the recursive filtering-based intra-prediction modes are represented as independent of any DC intra-prediction mode.

2. The method according to claim 1, characterized in that, N max It is a power of 2, N max It is N i The maximum value in.

3. The method according to claim 2, characterized in that, When M is greater than 2, N i Selected from the first quantity and the second quantity.

4. The method according to claim 3, characterized in that, N i The value is the same for all M intra-frame prediction mode sets.

5. The method according to claim 3, characterized in that, One of the M intra-prediction mode sets includes the second number of intra-prediction modes, and all other intra-prediction mode sets in the M intra-prediction mode sets include the first number of intra-prediction modes.

6. The method according to claim 3, characterized in that, M=4, and N max =16.

7. The method according to claim 6, characterized in that, P = 61, the first quantity is 16, and the second quantity is 13.

8. The method according to any one of claims 1-7, characterized in that, The subset of the non-directional intra-prediction modes are all within the intra-prediction mode set located at the first position in the M intra-prediction mode sets, and are independent of the reference mode.

9. The method according to claim 8, characterized in that, The subset of non-directional intra-prediction modes includes DC prediction modes, which are always placed as the first in the set of intra-prediction modes located at the first position, regardless of the reference mode.

10. The method according to claim 9, characterized in that, The intra-frame prediction mode based on recursive filtering is represented as a sub-mode of the DC prediction mode.

11. The method according to claim 8, characterized in that, The subset of non-directional intra-prediction modes includes the SMOOTH prediction mode, which is always positioned as the second in the set of intra-prediction modes at the first position, regardless of the reference mode.

12. The method according to claim 8, characterized in that, The subset of non-directional intra-prediction modes includes the SMOOTH_V prediction mode, which is always positioned as the third in the set of intra-prediction modes at the first position, regardless of the reference mode.

13. The method according to claim 8, characterized in that, The subset of non-directional intra-prediction modes includes the SMOOTH_H prediction mode, which is always positioned as the fourth in the set of intra-prediction modes at the first position, regardless of the reference mode.

14. The method according to claim 8, characterized in that, The subset of non-directional intra-prediction modes includes the PAETH prediction mode, which is always positioned as the fifth in the set of intra-prediction modes at the first position, regardless of the reference mode.

15. The method according to claim 1, characterized in that, When the target intra-prediction mode identified for the current block is the recursive filtering-based intra-prediction mode, the method further includes determining to select one of a set of recursive filtering-based intra-prediction modes based on the set index and the position index, respectively.

16. The method according to claim 1, characterized in that, The intra-prediction mode based on recursive filtering is always placed as the sixth in the set of intra-prediction modes located at the first position, regardless of the reference mode.

17. The method according to claim 1, characterized in that, The P predefined intra-prediction modes include at least one directional intra-prediction mode, at least one non-directional intra-prediction mode, and at least one luminance-to-chrominance intra-prediction mode.

18. An apparatus for performing intra-frame prediction on a current block of an coded video stream, characterized in that, Includes processing circuitry, the processing circuitry being configured to: The encoded video stream is received, the encoded video stream including a set index and a position index, the set index and the position index being used for intra-frame prediction of the current block; The intra-prediction mode corresponding to at least one neighboring block of the current block in the video stream is determined as the reference mode; Based on the reference mode, a method is determined to divide P predefined intra-prediction modes into M intra-prediction mode sets, each intra-prediction mode set including N... i Intra-prediction modes at position indices, where P and M are integers greater than 1, and N... i It is the number of prediction modes in the i-th intra-prediction mode set, where i is an integer representing the index of the intra-prediction mode set in the M intra-prediction mode sets; Based on the received set index and the position index, and the partitioning method, the target intra-prediction mode is identified among the P predefined intra-prediction modes. as well as The current block of the encoded video stream is decoded at least based on the target intra-frame prediction pattern identified according to the set index and the position index of the current block; Wherein, the P predefined intra-prediction modes include a subset of non-directional intra-prediction modes; and the relative order of the subsets of non-directional intra-prediction modes in the M intra-prediction mode sets is fixed and independent of the reference mode; or The P predefined intra-prediction modes include recursive filtering-based intra-prediction modes, which are arranged sequentially as the first intra-prediction mode set in the M intra-prediction mode set, independent of the reference mode; and the recursive filtering-based intra-prediction modes are represented as independent of any DC intra-prediction mode.

Images