Method and apparatus for video encoding and video decoding

Abstract

A method and apparatus for video decoding and video encoding. The decoding method comprises the following steps: receiving a video code stream containing a plurality of coded blocks, wherein the plurality of coded blocks comprise a current block; deriving a first context for entropy encoding a set index based on encoded information, the encoded information comprising information of the current block and / or mode information of at least one neighboring block of the current block; the set index indicates an intra mode set from a plurality of intra mode sets, and the intra mode set comprises a plurality of intra prediction modes; deriving a second context for entropy encoding a mode index of the current block based on the encoded information, the mode index indicating an intra prediction mode from the set of intra modes; performing entropy decoding on the set index according to the first context; performing entropy decoding on the mode index according to the second context; and determining the intra prediction mode of the current block based on the set index and the mode index.

Description

[0001] Related Applications

[0002] This application is a divisional of the Chinese patent application with application number 202280006506.6 filed on January 27, 2022. The entire contents of the parent application are incorporated herein. Technical Field

[0003] The present disclosure relates to video encoding and / or decoding techniques, and in particular to methods and related devices for video decoding. Background Art

[0004]

[0013] Certain aspects of this background description provided herein are not necessarily prior art to the present disclosure.

[0005] Video encoding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a series of pictures, each of which has a spatial dimension of, for example, 1920×1080 luminance samples and associated full or sub-sampled chrominance samples. The series of pictures can have a fixed or variable picture rate (alternatively referred to as a frame rate) of, for example, 60 pictures per second or 60 frames per second. Uncompressed video has specific bit rate requirements for streaming or data processing. For example, a video with a pixel resolution of 1920×1080, a frame rate of 60 frames per second, and a chrominance subsampling of 4:2:0 at 8 bits per pixel per color channel requires a bandwidth of nearly 1.5 Gbit / s. One hour of such video requires more than 600 gigabytes of storage space.

[0006] One purpose of video encoding and decoding can be to reduce the redundancy of uncompressed input video signals by compression. Compression can help reduce the bandwidth and / or storage space requirements mentioned above, in some cases reducing by two orders of magnitude or more. Both lossless compression and lossy compression and their combinations can be used. Lossless compression refers to a technique that can reconstruct an exact copy of the original signal from the compressed original signal via a decoding process. Lossy compression refers to an encoding / decoding process in which the original video information is not fully retained during encoding and is not fully restored during decoding. When lossy compression is used, the reconstructed signal may be different from the original signal, but the distortion between the original signal and the reconstructed signal is small enough to present a reconstructed signal useful for the intended application, although there is some information loss. In the case of video, lossy compression is widely used in many applications. The amount of tolerable distortion depends on the application. For example, users of certain consumer video streaming applications may tolerate higher distortion than users of movie or television broadcast applications. The compression ratio that can be achieved by a specific encoding algorithm can be selected or adjusted to reflect various distortion tolerances: higher tolerable distortion generally allows encoding algorithms that produce higher losses and higher compression ratios.

[0007] Video encoders and decoders may utilize techniques from several broad categories and steps, including, for example, motion compensation, Fourier transforms, quantization, and entropy coding.

[0008] Video codec techniques may include techniques known as intra-coding. In intra-coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, pictures are spatially subdivided into blocks of samples. When all sample blocks are encoded in intra-mode, the picture may be referred to as an intra-picture. Intra-pictures and their derivatives, such as independent decoder refresh pictures, may be used to reset decoder states, and may therefore be used as the first picture in an encoded video stream and video session or as a still image. Samples of the block after intra-prediction may then be subjected to a frequency domain transform, and the transform coefficients thus generated may be quantized before entropy coding. Intra-prediction represents a technique for minimizing sample values ​​in the pre-transform domain. In some cases, the smaller the DC value after the transform and the smaller the AC coefficient, the fewer bits are required to represent the block after entropy coding at a given quantization step size.

[0009] Conventional intra-frame coding, such as that known from, for example, MPEG (Moving Picture Experts Group, MPEG)-2 generation coding techniques, does not use intra-frame prediction. However, some newer video compression techniques include techniques that attempt to encode / decode blocks based on metadata and / or surrounding sample data obtained during encoding and / or decoding of, for example, spatially adjacent and preceding in decoding order the data block being intra-coded or decoded. Such techniques are hereinafter referred to as "intra-frame prediction" techniques. Note that, in at least some cases, intra-frame prediction uses only reference data from the current picture being reconstructed, and does not use reference data from other reference pictures.

[0010] There may be many different forms of intra-frame prediction. When more than one such technique may be used in a given video coding technique, the techniques used may be referred to as intra-frame prediction modes. One or more intra-frame prediction modes may be provided in a particular codec. In some cases, a mode may have sub-modes and / or may be associated with various parameters, and the mode / sub-mode information and intra-frame coding parameters for a video block may be encoded separately or included together in a mode codeword. Which codeword is used for a given mode, sub-mode, and / or parameter combination may have an impact on the coding efficiency gain through intra-frame prediction, and therefore may have an impact on the entropy coding technique used to convert the codeword into a bitstream.

[0011] Some modes of intra prediction were introduced by H.264, refined in H.265, and further refined in newer coding technologies such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Set (BMS). Typically, for intra prediction, the predictor block can be formed using neighboring sample values ​​that have become available. For example, the available values ​​of a specific set of neighboring samples along a certain direction and / or line can be copied to the predictor block. A reference to the direction used can be encoded in the bitstream or it can itself be predicted.

[0012] 1A, a subset of nine predictor directions specified in the 33 possible intra-frame predictor directions of H.265 (corresponding to the 33 angular modes of the 35 intra-frame modes specified in H.265) is depicted at the bottom right. The point (101) where the arrows converge represents the sample being predicted. The arrows represent the direction in which the sample at 101 is predicted using neighboring samples. For example, arrow (102) indicates that sample (101) is predicted based on one or more neighboring samples at an angle of 45 degrees to the horizontal direction to the upper right. Similarly, arrow (103) indicates that sample (101) is predicted based on one or more neighboring samples at an angle of 22.5 degrees to the horizontal direction to the lower left of sample (101).

[0013] Still referring to FIG. 1A , a square block (104) of 4×4 samples is depicted at the upper left (indicated by a bold dashed line). The square block (104) includes 16 samples, each of which is labeled with "S", its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 is the fourth sample in both the Y and X dimensions in the block (104). Since the size of the block is 4×4 samples, S44 is at the lower right. Further shown are example reference samples that follow a similar numbering scheme. The reference samples are labeled with R, their Y position (e.g., row index), and X position (column index) relative to the block (104). In both H.264 and H.265, prediction samples that are adjacently adjacent to the block being reconstructed are used.

[0014] The intra picture prediction of block 104 may start by copying reference sample values ​​from neighboring samples according to a signaled prediction direction. For example, assume that the encoded video code stream includes signaling indicating the prediction direction of arrow (102) for this block 104 - i.e., the samples are predicted according to one or more prediction samples at a 45 degree angle to the horizontal direction to the upper right. In this case, samples S41, S32, S23, and S14 are predicted according to the same reference sample R05. Then, sample S44 is predicted according to reference sample R08.

[0015] In some cases, the values ​​of multiple reference samples may be combined, such as by interpolation, in order to calculate the reference sample; in particular, when the direction is not divisible by 45 degrees.

[0016] As video coding technology continues to develop, the number of possible directions is also increasing. For example, in H.264 (2003), nine different directions can be used for intra-frame prediction. In H.265 (2013), this increased to 33, and at the time of this disclosure, JEM / VVC / BMS can support up to 65 directions. Experimental studies have been conducted to help identify the most appropriate intra-frame prediction directions, and certain techniques in entropy coding can be used to encode those most appropriate directions with a small number of bits, thereby accepting a specific bit penalty for the direction. In addition, the direction itself can sometimes be predicted based on the neighboring directions used in the intra-frame prediction of the decoded neighboring blocks.

[0017] FIG. 1B shows a schematic diagram ( 180 ) depicting 65 intra prediction directions according to JEM to illustrate that the number of prediction directions increases in various encoding techniques as they evolve over time.

[0018] The manner in which the bits representing the intra prediction direction are mapped to the prediction direction in the encoded video bitstream may vary from one video coding technique to another; and may range, for example, from a simple direct mapping of the prediction direction to the intra prediction mode, to codewords, to complex adaptive schemes involving the most probable mode, and similar techniques. However, in all cases, there may be certain directions for leading prediction that are statistically less likely to appear in the video content than certain other directions. Since the goal of video compression is to reduce redundancy, in a well-designed video coding technique, those less probable directions will likely be represented by a larger number of bits than more probable directions.

[0019] Inter-picture prediction or inter-prediction may be based on motion compensation. In motion compensation, sample data from a previously reconstructed picture or a portion thereof (reference picture) may be used to predict a newly reconstructed picture or picture portion (e.g., block) after being spatially shifted in the direction indicated by a motion vector (hereinafter MV). In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV may have two dimensions, X and Y, or three dimensions, where the third dimension is an indication of the reference picture in use (similar to the time dimension).

[0020] In some video compression techniques, the current MV applicable to a region of sample data can be predicted based on other MVs, for example, based on other MVs related to other regions of sample data that are spatially adjacent to the region being reconstructed and that precede the current MV in the decoding order. This can significantly reduce the total amount of data required to encode the MV by eliminating redundancy in the relevant MVs, thereby improving compression efficiency. For example, since there is a statistical possibility that a region larger than the region applicable to a single MV moves in a similar direction in a video sequence when encoding an input video signal derived from a camera (referred to as natural video), and thus similar motion vectors derived from MVs of neighboring regions can be used for prediction in some cases, MV prediction can work effectively. This causes the actual MV for a given region to be similar or identical to the MV predicted from the surrounding MVs. Such an MV can also be represented with fewer bits after entropy coding than would be used if the MV was directly encoded rather than predicted from neighboring MVs. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., an MV) derived from an original signal (i.e., a sample stream). In other cases, the MV prediction itself may be lossy, for example because of round-off errors when computing the predictor from several surrounding MVs.

[0021] Various MV prediction mechanisms are described in H.265 / HEVC (High Efficiency Video Coding, HEVC) (ITU (International Telecommunication Union, ITU)-T H.265 Recommendation, "High Efficiency Video Coding", December 2016). Among the various MV prediction mechanisms specified by H.265, the following is a technique referred to as "spatial merging".

[0022] 2, the current block (201) includes samples that have been found by the encoder during the motion search process to be predictable from a previous block of the same size that has been spatially shifted. Instead of encoding the MV directly, the MV may be used as represented by A0 , A 1 and B 0 , B 1 , B 2 The MV associated with any of the five surrounding samples (202 to 206, respectively) is derived from metadata associated with one or more reference pictures, for example, from the latest (in decoding order) reference picture. In H.265, MV prediction can use a predictor from the same reference picture used by a neighboring block.

[0023] In some cases of video encoding and / or decoding, there may be a strong correlation between the intra prediction modes of the current block and its neighboring blocks. Therefore, when the intra prediction modes are divided into different mode sets, the probability of selecting an intra prediction mode from different mode sets may be correlated with the intra prediction modes of neighboring blocks. This strong correlation and the associated probability distribution are not effectively utilized, resulting in inefficiency and poor performance of some intra mode coding designs. Summary of the invention

[0024] The present disclosure describes various embodiments of methods, apparatus, and computer-readable storage media for video encoding and / or decoding.

[0025] According to one aspect, an embodiment of the present disclosure provides a method for video decoding. The method includes receiving an encoded video code stream of a block. The method also includes deriving at least one context of a set index and a mode index for the block based on information of the block and / or mode information of at least one neighboring block of the block, wherein: the set index indicates an intra-frame mode set from a plurality of intra-frame mode sets, and the mode index indicates an intra-frame prediction mode from the intra-frame mode set; extracting the set index and the mode index from the encoded video code stream according to at least one context; and determining the intra-frame prediction mode of the block based on the set index and the mode index.

[0026] According to another aspect, an embodiment of the present disclosure provides a method for entropy encoding of intra-frame prediction modes in video coding. The method includes receiving a block to be encoded into an encoded video bitstream. The method also includes deriving at least one context of a set index and a mode index for the block based on information of the block and / or mode information of at least one neighboring block of the block, wherein: the set index indicates an intra-frame mode set from a plurality of intra-frame mode sets, and the mode index indicates an intra-frame prediction mode from the intra-frame mode set; and encoding the set index and the mode index into the encoded video bitstream according to at least one context.

[0027] According to another aspect, an embodiment of the present disclosure provides a device for video decoding. The device includes: a receiving module configured to receive an encoded video code stream of a block; an acquisition module configured to derive at least one context of a set index and a mode index for the block based on information of the block and / or mode information of at least one neighboring block of the block, wherein the set index indicates an intra-frame mode set from multiple intra-frame mode sets, and the mode index indicates an intra-frame prediction mode from the intra-frame mode set; an extraction module configured to extract a set index and a mode index from the encoded video code stream according to at least one context; and a determination module configured to determine the intra-frame prediction mode of the block based on the set index and the mode index. According to another aspect, an embodiment of the present disclosure provides a device for video encoding and / or decoding. The device includes: a memory storing instructions; and a processor communicating with the memory. When the processor executes the instructions, the processor is configured to cause the device to perform the above method for video decoding and / or encoding.

[0028] In another aspect, embodiments of the present disclosure provide a non-transitory computer-readable medium storing instructions which, when executed by a computer for video decoding and / or encoding, cause the computer to perform the above method for video decoding and / or encoding.

[0029] According to the method for video decoding provided by the implementation of the present disclosure, an encoded video code stream of a block is received, and at least one context of the set index and the mode index of the block is obtained based on the information of the block and / or the mode information of at least one neighboring block of the block, and the set index and the mode index are extracted from the encoded video code stream according to the at least one context, and then the intra-frame prediction mode of the block is determined based on the set index and the mode index. Thus, the selection of the encoding context of the set index and / or the mode index of the current block depends on the information of the current block and the mode information of at least one neighboring block of the current block. In this way, the correlation between the mode information of the current block and the neighboring blocks is well utilized, thereby improving the efficiency and performance of intra-frame mode encoding and decoding.

[0030] The above aspects and other aspects and their implementations are described in more detail in the drawings, the description and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings.

[0032] FIG. 1A is a schematic diagram showing an exemplary subset of intra prediction direction modes.

[0033] FIG. 1B shows a diagram of exemplary intra prediction directions.

[0034] FIG. 2 is a schematic diagram showing spatial merging candidates of a current block and its surroundings for motion vector prediction in one example.

[0035] Figure 3 A schematic diagram showing a simplified block diagram of a communication system (300) according to some example embodiments.

[0036] Figure 4 A schematic diagram showing a simplified block diagram of a communication system (400) according to some example embodiments.

[0037] Figure 5 A schematic diagram showing a simplified block diagram of a video decoder according to some example embodiments.

[0038] Figure 6 A schematic diagram showing a simplified block diagram of a video encoder according to some example embodiments.

[0039] Figure 7 A block diagram of a video encoder according to further example embodiments is shown.

[0040] Figure 8 A block diagram of a video decoder according to further example embodiments is shown.

[0041] Fig. 9 Directional intra prediction modes according to some example embodiments of the present disclosure are shown.

[0042] Fig.10 A non-directional intra prediction mode according to some example embodiments of the present disclosure is shown.

[0043] Fig.11 A recursive intra prediction mode according to some example embodiments of the present disclosure is shown.

[0044] Fig.12 A flow chart illustrating a method according to some example embodiments of the present disclosure is shown.

[0045] Fig.13 A flow chart illustrating another method according to some example embodiments of the present disclosure is shown.

[0046] Fig.14 A schematic diagram of a computer system according to some example implementations of the present disclosure is shown. DETAILED DESCRIPTION

[0047] The present invention will now be described in detail hereinafter with reference to the accompanying drawings, which form a part of the present invention and illustrate specific examples of implementation by way of illustration. However, it is noted that the present invention may be implemented in a variety of different forms, and therefore, the subject matter covered or claimed is intended to be construed as not being limited to any of the implementations to be set forth below. It is also noted that the present invention may be implemented as a method, apparatus, component, or system. Thus, embodiments of the present invention may, for example, take the form of hardware, software, firmware, or any combination thereof.

[0048] Throughout the specification and claims, terms may have slightly different meanings that are suggested or implied in the context beyond the explicitly stated meaning. The phrases "in one embodiment" or "in some embodiments" as used herein do not necessarily refer to the same embodiment, and the phrases "in another embodiment" or "in other embodiments" as used herein do not necessarily refer to different embodiments. Similarly, the phrases "in one implementation" or "in some implementations" as used herein do not necessarily refer to the same implementation, and the phrases "in another implementation" or "in other implementations" as used herein do not necessarily refer to different implementations. For example, it is intended that the claimed subject matter includes a combination of the whole or part of the exemplary embodiments / implementations.

[0049] Typically, terms can be understood at least in part from usage in context. For example, terms such as "and", "or" or "and / or" as used herein can include a variety of meanings depending at least in part on the context in which such terms are used. Typically, "or" (if used in an associative list, e.g., A, B, or C) is intended to mean: A, B, and C, used herein in an inclusive sense; and A, B, or C, used herein in an exclusive sense. In addition, depending at least in part on the context, the terms "one or more" or "at least one" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or can be used to describe a combination of features, structures, or characteristics in a plural sense. Similarly, terms such as "one", "an", or "the" can also be understood to convey singular usage or to convey plural usage, depending at least in part on the context. In addition, the term "based on" or "determined by..." can be understood to not necessarily be intended to convey an exclusive set of factors, and can alternatively allow the presence of additional factors that are not necessarily explicitly described, which also depends at least in part on the context.

[0050] Figure 3A simplified block diagram of a communication system (300) according to some embodiments of the present disclosure is shown. The communication system (300) includes a plurality of terminal devices that can communicate with each other via, for example, a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). Figure 3 In the example of , a first pair of terminal devices (310) and (320) can perform unidirectional data transmission. For example, the terminal device (310) can encode video data (e.g., a video picture stream captured by the terminal device (310)) for transmission to another terminal device (320) via a network (350). The encoded video data can be transmitted in the form of one or more encoded video code streams. The terminal device (320) can receive the encoded video data from the network (350), decode the encoded video data to restore the video picture, and display the video picture based on the restored video data. The unidirectional data transmission can be implemented in a media service application, etc.

[0051] In other examples, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of encoded video data, which bidirectional transmission can be implemented, for example, during a video conferencing application. For the bidirectional transmission of data, in an example, each of the terminal devices (330) and (340) can encode video data (e.g., a video picture stream captured by the terminal device) for transmission to the other of the terminal devices (330) and (340) via a network (350). Each of the terminal devices (330) and (340) can also receive the encoded video data transmitted by the other of the terminal devices (330) and (340), and can decode the encoded video data to restore the video picture, and can display the video picture at an accessible display device based on the restored video data.

[0052] exist Figure 3In the example of , terminal devices (310), (320), (330) and (340) can be implemented as servers, personal computers and smart phones, but the applicability of the basic principles of the present disclosure may not be limited to this. Implementations of the present disclosure may be implemented in desktop computers, laptop computers, tablet computers, media players, wearable computers, dedicated video conferencing equipment, etc. Network (350) represents any number or type of network that transmits encoded video data among terminal devices (310), (320), (330) and (340), including, for example, wired (wired) and / or wireless communication networks. The communication network (350) can exchange data in circuit switching channels, packet switching channels and / or other types of channels. Representative networks include telecommunication networks, local area networks, wide area networks and / or the Internet. For the purposes of this discussion, unless explicitly stated herein, the architecture and topology of the network (350) may be unimportant to the operation of the present disclosure.

[0053] As examples of applications of the disclosed subject matter, Figure 4 The placement of the video encoder and the video decoder in a video streaming environment is shown. The disclosed subject matter can be equally applicable to other video applications, including, for example, video conferencing, digital TV broadcasting, gaming, virtual reality, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0054] The video streaming system may include a video capture subsystem (413), which may include a video source (401), such as a digital camera, for creating an uncompressed video picture or image stream (402). In an example, the video picture stream (402) includes samples recorded by the digital camera of the video source 401. The video picture stream (402) is depicted as a thick line to emphasize the high amount of data when compared to the encoded video data (404) (or encoded video bitstream), and the video picture stream (402) may be processed by an electronic device (420) coupled to the video source (401) including a video encoder (403). The video encoder (403) may include hardware, software, or a combination thereof to implement or implement aspects of the disclosed subject matter as described in more detail below. The encoded video data (404) (or encoded video stream (404)) is depicted as a thin line to emphasize the lower data volume when compared to the uncompressed video picture stream (402), and the encoded video data (404) (or encoded video stream (404)) can be stored on the streaming server (405) for future use or directly to a downstream video device (not shown). One or more streaming client subsystems, such as Figure 4The client subsystems (406) and (408) in the video transmission system can access the streaming server (405) to retrieve copies (407) and (409) of the encoded video data (404). The client subsystem (406) can include, for example, a video decoder (410) in the electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and creates an outgoing video picture stream (411) that is uncompressed and can be presented on a display (412) (e.g., a display screen) or another rendering device (not depicted). The video decoder 410 can be configured to perform some or all of the various functions described in the present disclosure. In some streaming systems, the encoded video data (404), (407) and (409) (e.g., video bitstreams) can be encoded according to certain video coding standards / video compression standards. Examples of these standards include ITU-T Recommendation H.265. In the example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed subject matter can be used in the context of VVC and other video coding standards.

[0055] Note that the electronic devices (420) and (430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[0056] Figure 5 A block diagram of a video decoder (510) according to any of the following embodiments of the present disclosure is shown. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., receiving circuitry). The video decoder (510) may be used to replace Figure 4 A video decoder (410) in an example of FIG.

[0057] The receiver (531) can receive one or more encoded video sequences to be decoded by the video decoder (510). In the same embodiment or another embodiment, one encoded video sequence can be decoded at a time, wherein the decoding of each encoded video sequence is independent of other encoded video sequences. Each video sequence can be associated with multiple video frames or images. The encoded video sequence can be received from a channel (501), which can be a hardware / software link to a storage device storing the encoded video data or a streaming source transmitting the encoded video data. The receiver (531) can receive the encoded video data and other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to its corresponding processing circuit system (not depicted). The receiver (531) can separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (515) can be set between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) can be implemented as part of the video decoder (510). In other applications, the buffer memory (515) may be external to the video decoder (510) and separate from the video decoder (510) (not depicted). In still other applications, there may be a buffer memory (not depicted) external to the video decoder (510) for purposes such as preventing network jitter, and there may be another additional buffer memory (515) internal to the video decoder (510), for example to handle playback timing. When the receiver (531) receives data from a store / forward device with sufficient bandwidth and controllability or from an isosynchronous network, the buffer memory (515) may not be required, or the buffer memory (515) may be small. For use over an optimal packet network such as the Internet, a buffer memory (515) of sufficient size may be required, and its size may be relatively large. Such a buffer memory may be implemented with an adaptive size and may be implemented at least in part in an operating system or similar element (not depicted) external to the video decoder (510).

[0058] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from the encoded video sequence. The categories of these symbols include: information for managing the operation of the video decoder (510); and potential information for controlling a presentation device such as a display (512) (e.g., a display screen), which may or may not be a component of the electronic device (530) but may be coupled to the electronic device (530), such as Figure 5As shown. The control information for (one or more) rendering devices may be in the form of a Supplemental Enhancement Information (SEI message) or a Video Usability Information (VUI) parameter set fragment (not depicted). The parser (520) may parse / entropy decode the encoded video sequence received by the parser (520). The entropy encoding of the encoded video sequence may be performed according to a video coding technique or a video coding standard, and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a subgroup parameter set for at least one subgroup of the subgroups of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the subgroup. The subgroup may include a Group of Picture (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), etc. The parser (520) may also extract information such as transform coefficients (eg, Fourier transform coefficients), quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0059] The parser (520) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515), thereby creating symbols (521).

[0060] Depending on the type of the coded video picture or part of the coded video picture (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors, the reconstruction of the symbol (521) may involve multiple different processing or functional units. The units involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (520). For simplicity, this subgroup control information flow between the parser (520) and the following multiple processing or functional units is not described.

[0061] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into a number of functional units as described below. In a practical implementation operating under commercial constraints, many of these functional units interact closely with each other and can be at least partially integrated with each other. However, for the purpose of clearly describing the various functions of the disclosed subject matter, the conceptual subdivision into the following functional units is adopted in the following disclosure.

[0062] The first unit may include a sealer / inverse transform unit (551). The sealer / inverse transform unit (551) may receive quantized transform coefficients as (one or more) symbols (521) and control information from the parser (520), including information indicating which type of inverse transform to use, block size, quantization factors / parameters, quantization scaling matrix, etc. The sealer / inverse transform unit (551) may output blocks including sample values ​​that may be input into an aggregator (555).

[0063] In some cases, the output samples of the scaler / inverse transform (551) may belong to an intra-coded block; that is, a block that does not use predictive information from a previously reconstructed picture but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) may use surrounding block information that has already been reconstructed and stored in a current picture buffer (558) to generate a block of the same size and shape as the block being reconstructed. For example, the current picture buffer (558) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some implementations, the aggregator (555) may add the prediction information that has been generated by the intra-picture prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551) on a per-sample basis.

[0064] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to a block that has been inter-coded and possibly motion compensated. In such a case, the motion compensated prediction unit (553) may access the reference picture memory (557) to obtain samples for inter-picture prediction. After motion compensation of the obtained samples according to the symbols (521) belonging to the block, these samples may be added to the output of the scaler / inverse transform unit (551) (the output of unit 551 may be referred to as residual samples or residual signal) by an aggregator (555) to generate output sample information. The address within the reference picture memory (557) from which the motion compensated prediction unit (553) obtains the predicted samples may be controlled by a motion vector, which is provided to the motion compensated prediction unit (553) in the form of a symbol (521), which may have, for example, an X, Y component (shift) and a reference picture component (time). Motion compensation may also include interpolation of sample values ​​retrieved from a reference picture memory (557) when using sub-sample accurate motion vectors, and may also be associated with motion vector prediction mechanisms, etc.

[0065] The output samples of the aggregator (555) may be subjected to various loop filtering techniques in a loop filter unit (556). The video compression techniques may include in-loop filter techniques controlled by parameters included in the coded video sequence (also referred to as the coded video bitstream) that are available to the loop filter unit (556) as symbols (521) from the parser (520), but the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of the coded picture or coded video sequence, as well as to previously reconstructed and loop filtered sample values. Several types of loop filters may be included as part of the loop filter unit 556 in various orders, as will be described in more detail below.

[0066] The output of the loop filter unit (556) may be a sample stream that may be output to a rendering device (512) and stored in a reference picture memory (557) for use in future inter-picture prediction.

[0067] Once fully reconstructed, certain coded pictures may be used as reference pictures for future inter-picture prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) may become part of the reference picture memory (557) and a new current picture buffer may be reallocated before starting to reconstruct a subsequent coded picture.

[0068] The video decoder (510) may perform decoding operations according to a predetermined video compression technique adopted in a standard such as ITU-T H.265 Recommendation. The encoded video sequence may conform to the syntax specified by the video compression technology or standard used in the sense that the encoded video sequence follows both the syntax of the video compression technology or standard and the profile recorded in the video compression technology or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technology or standard as tools that can only be used under the profile. In order to conform to the standard, the complexity of the encoded video sequence may be within a range defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further defined by the Hypothetical Reference Decoder (HRD) specification and the metadata of the HRD buffer management signaled in the encoded video sequence.

[0069] In some example embodiments, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence (one or more). The additional data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0070] Figure 6 A block diagram of a video encoder (603) according to some example embodiments of the present disclosure is shown. The video encoder (603) may be included in an electronic device (620). The electronic device (620) may also include a transmitter (640) (e.g., a transmission circuit system). The video encoder (603) may be used to replace Figure 4 A video encoder (403) in an example.

[0071] The video encoder (603) can be used to obtain the video source (601) (the video source (601) is not Figure 6 In some examples, the video source (601) can receive video samples as part of the electronic device (620), and the video source (601) can capture (one or more) video images to be encoded by the video encoder (603). In other examples, the video source (601) can be implemented as part of the electronic device (620).

[0072] The video source (601) can provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (603), and the digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601YCrCb, RGB, XYZ, ...) and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb4:4:4). In a media service system, the video source (601) can be a storage device capable of storing previously prepared videos. In a video conferencing system, the video source (601) can be a camera that captures local image information as a video sequence. The video data can be provided as multiple separate pictures or images that are given motion when viewed in sequence. The picture itself can be organized as a spatial pixel array, where each pixel can include one or more samples, depending on the sampling structure, color space, etc. used. The relationship between pixels and samples can be easily understood by a person skilled in the art. The following description focuses on samples.

[0073] According to some example embodiments, the video encoder (603) may encode and compress pictures of a source video sequence into an encoded video sequence (643) in real time or under any other time constraints required by the application. Implementing a suitable encoding speed constitutes a function of the controller (650). In some embodiments, the controller (650) may be functionally coupled to and control other functional units as described below. For simplicity, the coupling is not depicted. The parameters set by the controller (650) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization techniques, etc.), picture size, picture group (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured to have other suitable functions that belong to the video encoder (603) optimized for certain system designs.

[0074] In some example embodiments, the video encoder (603) may be configured to operate in an encoding loop. As an oversimplified description, in an example, the encoding loop may include a source encoder (630) (e.g., responsible for creating symbols such as a symbol stream based on an input picture to be encoded and (one or more) reference pictures) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how a (remote) decoder would create sample data, even if the embedded decoder 633 processes the encoded video stream of the source encoder 630 without entropy coding (because in the video compression techniques considered in the disclosed subject matter, any compression between the encoded video code stream and the symbols in entropy coding can be lossless). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since the decoding of the symbol stream produces a bit-accurate result that is independent of the decoder location (local or remote), the contents of the reference picture memory (634) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction part of the encoder are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, for example due to channel errors) is used to improve encoding quality.

[0075] The operation of the "local" decoder (633) can be combined with the above Figure 5 The operation of the "remote" decoder described in detail is identical to that of the video decoder (510). However, reference is also briefly made to Figure 5, when symbols are available and the entropy encoder (645) and parser (520) may not be able to fully implement the entropy decoding portion of the video decoder (510) including the buffer memory (515) and the parser (520) in the encoder's local decoder (633) to encode / decode the symbols into an encoded video sequence that can be lossless.

[0076] At this point, it can be observed that any decoder technology other than parsing / entropy decoding, which may only exist in a decoder, may also necessarily need to exist in a corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter may sometimes focus on decoder operations associated with the decoding portion of the encoder. Since encoder technology is mutually inverse to the decoder technology that has been fully described, the description of encoder technology can be simplified. Only in certain areas or aspects, a more detailed description of the encoder is provided below.

[0077] During operation, in some example implementations, the source encoder (630) may perform motion compensated predictive encoding that predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as “reference pictures.” In this manner, the encoding engine (632) encodes the differences (or residuals) in color channels between pixel blocks of an input picture and pixel blocks of reference picture(s) that may be selected as prediction reference(s) for the input picture.

[0078] The local video decoder (633) can decode the encoded video data of the picture that can be designated as the reference picture based on the symbols created by the source encoder (630). The operation of the encoding engine (632) can advantageously be lossy processing. When the encoded video data can be decoded at the video decoder ( Figure 6 When the video encoder (603) is decoded at a local video decoder (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding process that may be performed by the video decoder on the reference picture, and may cause the reconstructed reference picture to be stored in the reference picture cache (634). In this way, the video encoder (603) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the far-end (remote) video decoder.

[0079] The predictor (635) may perform a prediction search for the encoding engine (632). That is, for a new picture to be encoded, the predictor (635) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may be used as appropriate prediction references for the new picture. The predictor (635) may operate on a pixel block by pixel block basis to find an appropriate prediction reference. In some cases, as determined by the search results obtained by the predictor (635), the input picture may have prediction references taken from multiple reference pictures stored in the reference picture memory (634).

[0080] The controller (650) can manage encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters for encoding video data.

[0081] The outputs of all the above-mentioned functional units may be subjected to entropy coding in the entropy encoder (645). The entropy encoder (645) converts the symbols generated by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0082] The transmitter (640) can buffer the encoded video sequence(s) created by the entropy encoder (645) in preparation for transmission via a communication channel (660), which can be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0083] The controller (650) may manage the operation of the video encoder (603). During encoding, the controller (650) may assign a certain coded picture type to each coded picture, which may affect the coding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned one of the following picture types:

[0084] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those of ordinary skill in the art are aware of those variations of I pictures and their corresponding applications and features.

[0085] A predictive picture (P picture) may be a picture that may be encoded and decoded using intra prediction or inter prediction, which predicts sample values ​​of each block using at most one motion vector and a reference index.

[0086] Bidirectional predictive pictures (B pictures) can be pictures that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses up to two motion vectors and reference indices to predict sample values ​​for each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0087] The source picture may typically be spatially subdivided into a plurality of coding blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples, respectively), and encoded block by block. These blocks may be predictively encoded with reference to other (already encoded) blocks, which are determined by the coding allocations applied to the corresponding pictures of the blocks. For example, blocks of an I picture may be non-predictively encoded, or blocks of an I picture may be predictively encoded (spatial prediction or intra-frame prediction) with reference to already encoded blocks of the same picture. Pixel blocks of a P picture may be predictively encoded via spatial prediction or via temporal prediction with reference to a previously encoded reference picture. Blocks of a B picture may be predictively encoded via spatial prediction or via temporal prediction with reference to one or two previously encoded reference pictures. Source pictures or intermediately processed pictures may be subdivided into other types of blocks for other purposes. The division of coding blocks and other types of blocks may or may not follow the same approach, as described in further detail below.

[0088] The video encoder (603) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In its operation, the video encoder (603) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0089] In some example embodiments, the transmitter (640) may transmit additional data along with the encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set segments, etc.

[0090] Video can be captured as multiple source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often shortened to intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits temporal or other correlations between pictures. For example, a particular picture in encoding / decoding, which is referred to as the current picture, can be divided into blocks. The blocks in the current picture can be encoded by a vector called a motion vector when they are similar to reference blocks in a reference picture that was previously encoded in the video and is still buffered. The motion vector points to a reference block in a reference picture, and in the case of using multiple reference pictures, the motion vector may have a third dimension that identifies the reference picture.

[0091] In some example embodiments, bidirectional prediction techniques may be used for inter-picture prediction. According to such bidirectional prediction techniques, two reference pictures are used, such as a first reference picture and a second reference picture that are both subsequent to the current picture in the video in decoding order (but may be in the past or future, respectively, in display order). A block in the current picture may be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block may be jointly predicted by a combination of the first reference block and the second reference block.

[0092] In addition, merge mode techniques can be used for inter-picture prediction to improve coding efficiency.

[0093] According to some example embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, a picture in a video picture sequence is divided into coding tree units (Coding Tree Unit, CTU) for compression, and the CTUs in the picture may have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU may include three parallel coding tree blocks (Coding Tree Block, CTB): a luminance CTB and two chrominance CTBs. Each CTU may be recursively partitioned into one or more coding units (CUs) with a quadtree. For example, a 64×64 pixel CTU may be partitioned into a 64×64 pixel CU, or four 32×32 pixel CUs. Each of one or more of the 32×32 blocks may be further partitioned into four 16×16 pixel CUs. In some example embodiments, each CU may be analyzed during encoding to determine a prediction type for the CU among various prediction types, such as an inter prediction type or an intra prediction type. Depending on temporal and / or spatial predictability, a CU may be partitioned into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In some embodiments, prediction operations in decoding (encoding / decoding) are performed in units of prediction blocks. Partitioning a CU into PUs (or PBs of different color channels) may be performed in various spatial modes. For example, a luma or chroma PB may include a matrix of values ​​(e.g., luma values) for samples such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 samples, and the like.

[0094] Figure 7 A diagram of a video encoder (703) according to some further example embodiments of the present disclosure is shown. The video encoder (703) is configured to receive a processed block (e.g., a prediction block) of sample values ​​within a current video picture in a sequence of video pictures, and to encode the processed block into an encoded picture that is part of an encoded video sequence. The example video encoder (703) may be used instead of Figure 4 The video encoder (403) in the example.

[0095] For example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a prediction block of 8×8 samples. The video encoder (703) then uses, for example, Rate-Distortion Optimization (RDO) to determine whether to best encode the processing block using intra mode, inter mode, or bidirectional prediction mode. In the case where it is determined that the processing block is encoded in intra mode, the video encoder (703) can encode the processing block into an encoded picture using intra prediction techniques; and in the case where it is determined that the processing block is encoded in inter mode or bidirectional prediction mode, the video encoder (703) can encode the processing block into an encoded picture using inter prediction or bidirectional prediction techniques, respectively. In some example embodiments, merge mode can be used as a sub-mode of inter-picture prediction, wherein a motion vector is derived from an encoded motion vector component external to one or more motion vector predictors without the aid of the predictor. In some other example embodiments, there may be a motion vector component applicable to the subject block. Therefore, the video encoder (703) may include Figure 7 Components not explicitly shown, such as a mode decision module for determining a prediction mode for a processing block.

[0096] exist Figure 7 In the example of FIG. 7 , the video encoder ( 703 ) includes Figure 7 An inter-frame encoder (730), an intra-frame encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721) and an entropy encoder (725) coupled together are shown in the example arrangement.

[0097] The inter-frame encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the block to one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture in display order), generate inter-frame prediction information (e.g., redundant information description according to an inter-frame coding technique, motion vectors, merge mode information), and calculate an inter-frame prediction result (e.g., a predicted block) based on the inter-frame prediction information using any suitable technique. In some examples, the reference picture is encoded using an embedded image. Figure 6 The decoding unit 633 (shown as Figure 7 The residual decoder 728, as described in further detail below, decodes the decoded reference picture based on the encoded video information.

[0098] The intra encoder (722) is configured to receive samples of a current block (e.g., a processing block), compare the block with an already encoded block in the same picture, and generate quantization coefficients after transformation, and in some cases also generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). The intra encoder (722) can calculate an intra prediction result (e.g., a predicted block) based on the intra prediction information and a reference block in the same picture.

[0099] The general controller (721) can be configured to determine general control data and control other components of the video encoder (703) based on the general control data. In an example, the general controller (721) determines a prediction mode of a block and provides a control signal to a switch (726) based on the prediction mode. For example, when the prediction mode is an intra-frame mode, the general controller (721) controls the switch (726) to select the intra-frame mode result for use by the residual calculator (723), and controls the entropy encoder (725) to select the intra-frame prediction information and include the intra-frame prediction information in the bitstream; and when the prediction mode of the block is an inter-frame mode, the general controller (721) controls the switch (726) to select the inter-frame prediction result for use by the residual calculator (723), and controls the entropy encoder (725) to select the inter-frame prediction information and include the inter-frame prediction information in the bitstream.

[0100] The residual calculator (723) may be configured to calculate the difference (residual data) between the received block and the prediction result of the block selected from the intra encoder (722) or the inter encoder (730). The residual encoder (724) may be configured to encode the residual data to generate a transform coefficient. For example, the residual encoder (724) may be configured to convert the residual data from the spatial domain to the frequency domain to generate a transform coefficient. Then, the transform coefficient is subjected to a quantization process to obtain a quantized transform coefficient. In various example embodiments, the video encoder (703) further includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data may be appropriately used by the intra encoder (722) and the inter encoder (730). For example, the inter encoder (730) may generate a decoded block based on the decoded residual data and the inter prediction information, and the intra encoder (722) may generate a decoded block based on the decoded residual data and the intra prediction information. The decoded blocks are appropriately processed to generate decoded pictures, and these decoded pictures may be buffered in a memory circuit (not shown) and used as reference pictures.

[0101] The entropy encoder (725) may be configured to format the code stream to include the encoded blocks and perform entropy encoding. The entropy encoder (725) is configured to include various information in the code stream. For example, the entropy encoder (725) may be configured to include general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the code stream. When the block is encoded in the merge sub-mode of the inter-frame mode or the bi-prediction mode, there may be no residual information.

[0102] Figure 8 A diagram of an example video decoder (810) according to another embodiment of the present disclosure is shown. The video decoder (810) is configured to receive an encoded picture as part of an encoded video sequence and decode the encoded picture to generate a reconstructed picture. In an example, the video decoder (810) may be used instead of Figure 4 A video decoder (410) in an example of FIG.

[0103] exist Figure 8 In the example of FIG. 8 , the video decoder ( 810 ) includes Figure 8 An entropy decoder (871), an inter-frame decoder (880), a residual decoder (873), a reconstruction module (874) and an intra-frame decoder (872) coupled together are shown in the example arrangement of.

[0104] The entropy decoder (871) can be configured to reconstruct certain symbols representing syntax elements constituting the encoded picture from the encoded picture. Such symbols may include, for example, a mode for encoding a block (e.g., intra mode, inter mode, bi-prediction mode, merge sub-mode, or another sub-mode), prediction information (e.g., intra prediction information or inter prediction information) that can identify certain samples or metadata used by the intra decoder (872) or the inter decoder (880) for prediction, residual information in the form of, for example, quantized transform coefficients, etc. In an example, when the prediction mode is inter mode or bi-prediction mode, the inter prediction information is provided to the inter decoder (880); and when the prediction type is an intra prediction type, the intra prediction information is provided to the intra decoder (872). The residual information may be subjected to inverse quantization and provided to the residual decoder (873).

[0105] The inter-frame decoder (880) may be configured to receive the inter-frame prediction information and generate an inter-frame prediction result based on the inter-frame prediction information.

[0106] The intra decoder (872) may be configured to receive intra prediction information and generate a prediction result based on the intra prediction information.

[0107] The residual decoder (873) can be configured to perform inverse quantization to extract dequantized transform coefficients, and process the dequantized transform coefficients to convert the residual from the frequency domain to the spatial domain. The residual decoder (873) can also utilize certain control information (to include quantizer parameters (Quantizer Parameter, QP)), which can be provided by the entropy decoder (871) (since this may only be low-data-volume control information, the data path is not depicted).

[0108] The reconstruction module (874) can be configured to combine the residual output by the residual decoder (873) with the prediction result (output by the inter-frame prediction module or the intra-frame prediction module, as the case may be) in the spatial domain to form a reconstructed block, which forms part of the reconstructed picture as part of the reconstructed video. Note that other suitable operations such as deblocking operations can also be performed to improve visual quality.

[0109] Note that the video encoders (403), (603) and (703) and the video decoders (410), (510) and (810) may be implemented using any suitable technology. In some example embodiments, the video encoders (403), (603) and (703) and the video decoders (410), (510) and (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603) and (703) and the video decoders (410), (510) and (810) may be implemented using one or more processors executing software instructions.

[0110] Returning to the intra prediction process, in the intra prediction process, samples in a block (e.g., a luminance or chrominance prediction block, or a coding block without further partitioning into prediction blocks) are predicted by samples of adjacent, next adjacent, or other rows or combinations thereof to generate a prediction block. The residual between the actual block being encoded and the prediction block can then be processed via a transform followed by quantization. Various intra prediction modes can be made available, and parameters related to intra mode selection and other parameters can be signaled in the codestream. For example, various intra prediction modes can belong to: one or more row positions for prediction samples; directions along which prediction samples are selected from one or more prediction rows; and other special intra prediction modes.

[0111] For example, a set of intra prediction modes (interchangeably referred to as "intra modes") may include a predefined number of directional intra prediction modes. As described above with respect to the example implementation of FIG. 1 , these intra prediction modes may correspond to a predefined number of directions along which out-of-block samples are selected as predictions for samples being predicted in a particular block. In another specific example implementation, eight (8) primary directional modes corresponding to predefined angles of 45 to 207 degrees from the horizontal axis may be supported and predefined.

[0112] In some other implementations of intra prediction, in order to further exploit more types of spatial redundancy in directional texture, the directional intra mode can be further extended to a set of angles with finer granularity. Fig. 9 As shown in , the above 8-angle implementation can be configured to provide eight nominal angles referred to as V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED, and for each nominal angle, a predefined number (e.g., 7) of finer angles can be added. With such an extension, a greater total number (e.g., 56 in this example) of directional angles can be used for intra-frame prediction, corresponding to the same number of predefined directional intra-frame modes. The predicted angle can be represented by the nominal intra-frame angle plus the angle increment. For the above specific example with 7 finer angle directions for each nominal angle, the angle increment can be a step size of -3 to 3 times 3 degrees.

[0113] The above directional intra prediction may also be referred to as unidirectional intra prediction, which is different from bidirectional intra prediction (also referred to as intra bi-prediction) described in a later part of this disclosure.

[0114] In some implementations, a predefined number of non-directional intra prediction modes may be predefined and made available as an alternative or in addition to the above directional intra modes. For example, five non-directional intra modes referred to as smooth intra prediction modes may be specified. These non-directional intra mode prediction modes may be specifically referred to as DC, PAETH, SMOOTH, SMOOTH_V, and SMOOTH_H intra modes. Fig.10 The prediction of samples for a particular block under these example non-directional modes is shown in FIG. As an example, Fig.10A 4×4 block 1002 is shown being predicted by samples from a top neighboring line and / or a left neighboring line. A particular sample 1010 in the block 1002 may correspond to a sample 1004 directly above the sample 1010 in the top neighboring row of the block 1002, a sample 1006 above and to the left of the sample 1010 as the intersection of the top and left neighboring rows, and a sample 1008 directly to the left of the sample 1010 in the left neighboring row of the block 1002. For the example DC intra prediction mode, an average of the left neighboring sample 1008 and the above neighboring sample 1004 may be used as a predictor for the sample 1010. For the example PAETH intra prediction mode, a top reference sample 1004, a left reference sample 1008, and an upper left reference sample 1006 may be obtained, and whichever value of the three reference samples is closest to (top+left−upperleft) may then be set as the predictor for the sample 1010. For the example SMOOTH_V intra prediction mode, sample 1010 may be predicted by quadratic interpolation in the vertical direction of the upper left neighboring sample 1006 and the left neighboring sample 1008. For the example SMOOTH_H intra prediction mode, sample 1010 may be predicted by quadratic interpolation in the horizontal direction of the upper left neighboring sample 1006 and the top neighboring sample 1004. For the example SMOOTH intra prediction mode, sample 1010 may be predicted by the average of the quadratic interpolation in the vertical and horizontal directions. The above non-directional intra mode implementations are shown only as non-limiting examples. Other non-directional selections of samples and other neighboring lines, as well as ways of combining prediction samples for predicting specific samples in a prediction block, may also be envisioned.

[0115] The encoder's selection of a particular intra-prediction mode from the above directional or non-directional modes at various coding levels (picture, slice, block, unit, etc.) can be signaled in the code stream. In some example implementations, an exemplary 8 nominal directional modes and 5 non-angular smoothing modes (a total of 13 options) can be first signaled. Then, if the signaled mode is one of the 8 nominal angular intra-modes, an index is further signaled to indicate the selected angle increment to the corresponding signaled nominal angle. In some other example implementations, all intra-prediction modes can be indexed together (e.g., 56 directional modes plus 5 non-directional modes to produce 61 intra-prediction modes) for signaling.

[0116] In some example implementations, the example 56 or other number of directional intra prediction modes may be implemented using a unified directional predictor that projects each sample of a block to a reference subsample location and interpolates the reference samples via a 2-tap bilinear filter.

[0117] In some implementations, in order to capture the attenuated spatial correlation with the reference on the edge, additional filter modes called FILTER INTRA modes can be designed. For these modes, the prediction samples within the block other than the samples outside the block can be used as intra-frame prediction reference samples for some patches within the block. For example, these modes can be predefined and can be used for intra-frame prediction of at least luminance blocks (or only luminance blocks). A predefined number (for example, five) of filter intra-frame modes can be pre-designed, each filter intra-frame mode is represented by a set of n-tap filters (for example, 7-tap filters), and the n-tap filters reflect the correlation between the samples in, for example, a 4×2 block and its n adjacent neighbors. In other words, the weighting factors of the n-tap filters can be position-dependent. Taking 8×8 blocks, 4×2 blocks and 7-tap filtering as an example, as Fig.11 As shown in FIG. 1 , the 8×8 block 1102 can be divided into 8 4×2 blocks. These blocks are Fig.11 In the example, the blocks are indicated by B0, B1, B2, B3, B4, B5, B6, and B7. For each block, its 7 neighbors (in Fig.11 The predicted values ​​of the direct neighbors are used as references. For example, Fig.11 All neighbors of block B7 indicated in are not reconstructed, so prediction samples of neighbors, such as part of B4, B5 and / or B6, are used instead.

[0118] In some implementations of intra prediction, one color component may be predicted using one or more other color components. The color component may be any of the components in YCrCb, RGB, XYZ color space, etc. For example, prediction of a chroma component (e.g., a chroma block) from a luma component (e.g., a luma reference sample) may be implemented, referred to as predicting chroma from luma or CfL. In some example implementations, many allow only cross-color prediction from luma to chroma. For example, the chroma samples in a chroma block may be modeled as a linear function of the corresponding reconstructed luma samples. CfL prediction may be implemented as follows:

[0119] CfL(α)=α×L AC +DC (1)

[0120] Among them, L ACrepresents the AC contribution of the luma component, α represents the parameters of the linear model, and DC represents the DC contribution of the chroma component. For example, an AC component is obtained for each sample of a block, and a DC component is obtained for the entire block. Specifically, the reconstructed luma samples can be subsampled to the chroma resolution, and then the average luma value (the DC of luma) can be subtracted from each luma value to form the AC contribution in luma. The AC contribution of luma is then used in the linear mode of equation (1) to predict the AC values ​​of the chroma components. In order to approximate or predict the chroma AC components based on the luma AC contribution, rather than requiring the decoder to calculate the scaling parameters, the example CfL implementation can determine the parameter α based on the original chroma samples and signal them in the bitstream. This reduces decoder complexity and produces more accurate predictions. As for the DC contribution of the chroma component, in some example implementations, it can be calculated using an intra-frame DC mode within the chroma component.

[0121] In some embodiments, for a particular coding block, coding unit, prediction block, or prediction unit encoded within a frame, its intra mode needs to be signaled by one or more syntax elements in the code stream. As described above, the number of possible intra prediction modes may be huge, and there may be 62 intra prediction modes available: 56 directional intra prediction modes, 5 non-directional modes, and one chrominance mode predicted from luminance (e.g., only for chrominance components). In order to signal these intra prediction modes, a first syntax may be signaled to indicate which nominal angle or non-directional mode is equal to the nominal mode of the current block. Then, if the mode of the current block is a directional mode, a second syntax may be signaled to indicate which incremental angle is equal to the incremental angle of the current block. In some cases of video encoding and / or decoding, there may be a strong correlation between the intra prediction modes of the current block and its neighboring blocks.

[0122] In some embodiments, this correlation can be exploited to design a more efficient syntax for intra-mode coding. In some implementations, the available intra-prediction modes of the current block can be partitioned into multiple intra-prediction mode sets according to the intra-prediction modes of neighboring blocks of the current block. To obtain the intra-prediction mode of the current block, first, a mode set index can be signaled to indicate the mode set index of the intra-prediction mode for the current block; and second, a mode index can be signaled to indicate the index of the intra-prediction mode within the mode set.

[0123] Here, in some embodiments of the present disclosure, “XYZ is signaled” may refer to XYZ being encoded into the encoded code stream during the encoding process; and / or, after the encoded code stream is transmitted from one device to another device, “XYZ is signaled” may refer to XYZ being decoded / extracted from the encoded code stream during the decoding process.

[0124] For example, in some of the implementations described above, the plurality of available intra prediction modes may include 62 different modes, including, for example, 56 directional intra prediction modes (e.g., 8 nominal directions, each with 7 fine angles), 5 non-directional modes, and one chrominance mode predicted from luminance (for chrominance components only). Once an intra mode is selected during the encoding process of a particular coding block, coding unit, prediction block, or prediction, it is necessary to include signaling corresponding to the selected intra mode in the bitstream. (One or more) signaling syntaxes must be able to distinguish all these 62 modes in some way. For example, a single syntax of 62 indexes may be used to signal these 62 modes, each index corresponding to one mode. In some other example implementations, one syntax may be signaled to indicate which nominal angle or non-directional mode is used as the nominal mode in the current block, and then, if the nominal mode of the current block is a directional mode, another syntax may be additionally signaled to indicate which incremental angle is selected for the current block.

[0125] Since the various syntaxes belonging to intra-frame coding usually occupy a large part of the code stream, and the intra-frame mode selection must be frequently performed in signaling, for example at various coding levels, the reduction of the number of bits used for intra-frame mode signaling becomes crucial in improving video coding efficiency. In practice, the use of various intra-frame prediction modes can follow certain statistical patterns, and such usage patterns can be used to design the indexes of intra-frame modes and signaling syntax, so that signaling efficiency can be enhanced. In addition, there may be some correlations between block-to-block intra-frame mode selections. Such correlations can be obtained on a statistical basis and considered in the design of the syntax for signaling of intra-frame mode selection. The goal is to reduce the number of bits of signaling syntax elements in the encoded code stream on average. For example, some general statistics show that there may be a strong correlation between the best intra-frame prediction mode of the current block and its neighboring blocks. Such correlations can be exploited when designing syntax for intra-frame mode coding.

[0126] In some embodiments, the available intra prediction modes or mode options of the current block being encoded can be divided into multiple intra prediction mode sets. Each set can be assigned a mode set index. Each set can contain multiple intra mode prediction modes. The manner in which the available intra prediction modes are divided and sorted and the intra prediction modes are sorted in each of the mode sets can be determined based on the correlation between the intra prediction modes between blocks, at least in part according to the intra prediction modes used by its neighboring blocks. The intra prediction mode used by the neighboring blocks can be referred to as a "reference intra prediction mode" or "reference mode". The intra prediction mode of a specific unit can be determined and selected. The selection of the intra prediction mode can be notified by signal. First, the mode set index can be notified by signal to indicate the mode set index of the intra prediction mode set containing the selected intra prediction mode. Secondly, the mode index (alternatively referred to as the mode position index within the set) can be notified by signal to indicate the index of the intra prediction mode selected within the mode set.

[0127] The general implementation of intra-frame prediction mode segmentation and sorting above and the specific examples below take advantage of statistical effects and neighboring correlations to dynamically index these modes, so that the design of the syntax for signaling its selection in the encoded video bitstream can be optimized to improve coding efficiency. For example, these implementations can help reduce the number of syntaxes for signaling and help more efficient context generation for entropy coding.

[0128] Various embodiments and / or implementations described in this disclosure may be used alone or in any order. In addition, a part, all or any part or entire combination of these embodiments and / or implementations may be implemented as a part of an encoder and / or decoder, and may be implemented in hardware and / or software. For example, they may be hard-coded in a dedicated processing circuit system (e.g., one or more integrated circuits). In other examples, they may be implemented by executing a program stored in a non-transient computer-readable medium by one or more processors.

[0129] There may be some issues / problems associated with intra-mode coding. For example, due to the strong correlation between samples in the current block and its neighboring blocks, when the intra-prediction modes are partitioned into different mode sets, the probability of selecting an intra-prediction mode from different mode sets may be correlated with the intra-prediction modes of neighboring blocks. This strong correlation and the associated probability distribution are not effectively exploited, resulting in inefficiency and poor performance of some intra-mode coding designs.

[0130] The present disclosure describes some embodiments of intra-frame prediction mode coding in video encoding and / or decoding, thereby solving at least one of the problems / difficulties discussed above, thereby achieving efficient intra-frame mode coding and improving the performance of intra-frame mode coding.

[0131] In some embodiments, in order to utilize the correlation and associated probability distribution between samples in the current block and its neighboring blocks, one or more contexts may be designed / derived for entropy encoding of a set index and / or a mode index, where the mode index is used to indicate an intra-frame prediction mode of the current block in the bitstream. One or more contexts may be used to encode the set index and / or the mode index into the encoded bitstream during the encoding process; and / or one or more contexts may be used to decode the set index and / or the mode index from the encoded bitstream during the decoding process.

[0132] A context for encoding a set index and / or a mode index of a current coding block may be selected from a set of contexts. The selection of a context may be indicated by an index. The selection from a set of contexts may depend on various factors. For example, the selection of a coding context of a set index and / or a mode index of a current block may depend on information of the current block and mode information of at least one of the neighboring blocks of the current block. For one example, at least one neighboring block may include an upper (top) and / or left neighboring block of the current block. For other examples, at least one neighboring block may also include an upper left neighboring block of the current block.

[0133] In some embodiments, reference Fig.12 , a method 1200 for intra-frame prediction mode encoding in video decoding, the method 1200 may include some or all of the following steps: step 1210, receiving an encoded video code stream of a block; step 1220, deriving at least one context of a set index and a mode index for the block based on information of the block and / or mode information of at least one neighboring block of the block, wherein: the set index indicates an intra-frame mode set from a plurality of intra-frame mode sets, and the mode index indicates an intra-frame prediction mode from the intra-frame mode set; step 1230, extracting the set index and the mode index from the encoded video code stream according to at least one context; and / or step 1240, determining the intra-frame prediction mode of the block based on the set index and the mode index. In some embodiments, the intra-frame mode set may refer to a group of intra-frame modes having N intra-frame modes, where N may be a positive integer; the intra-frame mode set may have a set index indicating a group of intra-frame modes in a group of intra-frame mode sets; and / or the intra-frame mode set may have a mode index indicating a specific intra-frame mode in the N intra-frame modes in the intra-frame mode set.

[0134] In some embodiments, reference Fig.13, a method 1300 for intra-frame prediction mode encoding in video encoding, the method 1300 may include some or all of the following steps: step 1310, receiving a block to be encoded into an encoded video bitstream; step 1320, deriving at least one context of a set index and a mode index for the block based on information of the block and / or mode information of at least one neighboring block of the block, wherein: the set index indicates an intra-frame mode set from a plurality of intra-frame mode sets, and the mode index indicates an intra-frame prediction mode from the intra-frame mode set; and / or step 1330, encoding the set index and the mode index into the encoded video bitstream according to at least one context.

[0135] In some embodiments of the present disclosure, the size of a block (such as but not limited to a coding block, a prediction block, or a transform block) may refer to the width or height of the block. The width or height of the block may be an integer in pixels. In some embodiments of the present disclosure, the size of a block may refer to the area size of the block. The area size of the block may be an integer calculated by multiplying the width of the block in pixels by the height of the block. In some embodiments of the present disclosure, the size of a block may refer to the maximum value of the width or height of the block, the minimum value of the width or height of the block, or the aspect ratio of the block. The aspect ratio of the block may be calculated as the width of the block divided by the height, or may be calculated as the height of the block divided by the width.

[0136] In some implementations, the mode type of the intra mode may include at least one of the following: a directional mode, a non-directional mode, a smoothing mode (e.g., smooth, smooth_v, smooth_h), a DC mode, a PAETH mode, and / or a mode that generates prediction samples according to a given prediction direction. In some other implementations, in a loose classification, the directional mode may broadly include: any mode that is not a smooth (smooth, smooth_v, smooth_h), DC, or PAETH mode; and any mode that generates prediction samples according to a given prediction direction. In some other implementations, the non-directional mode may include a smoothing mode (e.g., smooth, smooth_v, smooth_h), a DC mode, a PAETH mode, and a luma-for-chroma mode. In some other implementations, in a loose classification, the non-directional mode may broadly include any mode that is not a directional mode.

[0137] In some implementations, the available intra prediction modes for the current block may be divided / split into multiple intra prediction mode sets. In order to obtain the intra prediction mode of the current block, first, a mode set index may be signaled to indicate the mode set index of the intra prediction mode for the current block; second, a mode index may be signaled to indicate the index of the intra prediction mode within the mode set.

[0138] Here, in some embodiments of the present disclosure, the "first" mode set refers not only to "one" mode set, but to the "first" mode set with the minimum mode set index, the "second" mode set refers not only to "another" mode set but to the "second" mode set with the second minimum mode set index, and so on. For example, the number of intra-frame prediction mode sets may be indicated by M, and the range of mode set indexes may be, for example, from 1 to M or from 0 to M-1. When the mode set index ranges from 1 to M, the "first" mode set is the "first" mode set with a mode set index of 1, the "second" mode set is the "second" mode set with a mode set index of 2, and so on. When the mode set index ranges from 0 to M-1, the "first" mode set is the "first" mode set with a mode set index of 0, the "second" mode set is the "second" mode set with a mode set index of 1, and so on.

[0139] Here, in some embodiments of the present disclosure, “XYZ is signaled” may refer to XYZ being encoded into the encoded code stream during the encoding process; and / or, after the encoded code stream is transmitted from one device to another device, “XYZ is signaled” may refer to XYZ being decoded / extracted from the encoded code stream during the decoding process.

[0140] Here, in some embodiments of the present disclosure, a "block" may refer to a prediction block, a coding block, a transform block, or a coding unit (CU).

[0141] The device can be Figure 5 An electronic device (530) or Figure 8 In some implementations, the device may be a video decoder (810) in Figure 6 In other implementations, the device may be Figure 5 a portion of an electronic device (530) in Figure 8 A portion of a video decoder (810) in Figure 6 The encoded video stream may be a part of the decoder (633) in the encoder (603) in FIG. Figure 8 The encoded video sequence in Figure 6 or Figure 7 The intermediate encoded data in . A block can refer to a coding block or an encoded block.

[0142] The device can derive at least one context of a set index and a mode index for the block based on the information of the block and / or the mode information of at least one neighboring block of the block, wherein: the set index indicates an intra-mode set from a plurality of intra-mode sets, and the mode index indicates an intra-prediction mode from the intra-mode set. In some implementations, the information of the block includes the size of the block; and / or the mode information of at least one neighboring block includes at least one of the following: the number of directional modes of at least one neighboring block, and / or the mode set index of at least one neighboring block. For one example, at least one neighboring block may include an upper (top) and / or left neighboring block of the current block. For other examples, at least one neighboring block may also include an upper left neighboring block of the current block.

[0143] In some implementations, the context for signaling an intra-mode set index and a mode index within an intra-mode set, which may be referred to as a cumulative density function (CDF), depends on mode information of a current block and / or its neighboring blocks, including but not limited to a block size of the current block, a number of directional modes of at least one of the neighboring blocks, and / or a mode set index of at least one of the neighboring blocks.

[0144] In some other implementations, the device may also construct a list of intra-frame modes for the block, and / or divide the list of intra-frame modes into multiple intra-frame mode sets for the block. For example, the intra-frame mode list is constructed for the current block based on a set of predefined rules. After constructing the intra-frame mode list, the intra-frame prediction mode is divided into multiple intra-frame prediction mode sets according to its corresponding index in the intra-frame mode list. In order to signal the intra-frame prediction mode of the current block, first, a mode set index may be signaled to indicate the mode set index of the intra-frame prediction mode for the current block; and secondly, a mode index may be signaled to indicate the index of the intra-frame prediction mode within the mode set. The predefined rules may be different for different block sizes.

[0145] In some embodiments, step 1220 may include deriving context for the set index and the mode index based on the number of at least one neighboring block encoded using the directional intra prediction mode. For example, the context for signaling the intra mode set index and / or the mode index within the intra mode set depends on the number of neighboring blocks encoded using the directional intra prediction mode.

[0146] In some embodiments, deriving a context for a set index and a mode index based on the number of at least one neighboring block encoded using a directional intra-frame prediction mode may include: deriving a first context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being zero; deriving a second context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being one; and / or deriving a third context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being greater than one.

[0147] In some implementations, the context selection for the current block may be made from a total of three, four, or five different contexts. In a specific example implementation, if none of the neighboring blocks above or to the left are encoded using a directional intra-frame prediction mode, a first context (e.g., where the context index value is 0) is selected as the context for the set index and mode index for the current block. If only one of the neighboring blocks above and to the left is encoded using a directional intra-frame prediction mode, a second context (e.g., where the context index value is 1) is selected as the context for the set index and mode index for the current block. Otherwise, a third context (e.g., where the context index value is 2) is selected as the context for the set index and mode index for the current block. Such an implementation is based on a statistical observation of a strong correlation between the number of neighboring blocks encoded using a directional intra-frame prediction mode and the intra-frame prediction mode indicated by the set index and mode index for the current block.

[0148] In some embodiments, deriving a context for a set index and a mode index based on the number of at least one neighboring block encoded using a directional intra-frame prediction mode may include: deriving a first context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being zero; deriving a second context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being one; deriving a third context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being two and at least one neighboring block being encoded using the same directional intra-frame prediction mode; and / or deriving a fourth context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being two and at least one neighboring block being encoded using a different directional intra-frame prediction mode, or in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being greater than two.

[0149] In some implementations, if there is no neighboring block encoded using a directional intra prediction mode, a first context (e.g., where the context index value is 0) is selected as the context for the set index and mode index for the current block. If there is only one neighboring block encoded using a directional intra prediction mode, a second context (e.g., where the context index value is 1) is selected as the context for the set index and mode index for the current block. If there are two neighboring blocks encoded using a directional intra prediction mode, and the two directional intra prediction modes are the same, a third context (e.g., where the context index value is 2) is selected as the context for the set index and mode index for the current block. Otherwise, a fourth context (e.g., where the context index value is 3) is selected as the context for the set index and mode index for the current block.

[0150] In some embodiments, deriving a context for a set index and a mode index based on the number of at least one neighboring block encoded using a directional intra-frame prediction mode may include: deriving a first context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being zero; deriving a second context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being one; deriving a third context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being two; and / or deriving a fourth context as a context for the set index and the mode index in response to the number of at least one neighboring block encoded using a directional intra-frame prediction mode being greater than two.

[0151] In some implementations, if there is no neighboring block encoded using a directional intra prediction mode, a first context (e.g., where the context index value is 0) is selected as the context for the set index and mode index for the current block. If there is only one neighboring block encoded using a directional intra prediction mode, a second context (e.g., where the context index value is 1) is selected as the context for the set index and mode index for the current block. If there are two neighboring blocks encoded using a directional intra prediction mode, a third context (e.g., where the context index value is 2) is selected as the context for the set index and mode index for the current block. If there are more than two neighboring blocks encoded using a directional intra prediction mode, a fourth context (e.g., where the context index value is 3) is selected as the context for the set index and mode index for the current block.

[0152] In some implementations, the directional intra prediction mode includes at least one of: an inter coding mode or an intra block copy mode. In some other implementations, the directional intra prediction mode includes an intra inter mode that includes a directional intra mode component. For one example, if the neighboring block is encoded using the inter coding mode or the intra block copy mode, the neighboring block is considered to be a non-directional mode. For other examples, if the neighboring block is in an intra inter mode, the intra prediction mode used in the intra inter mode is used to derive the context or to determine the selection of the context from the context set.

[0153] In some embodiments, step 1220 may include deriving a context for the set index and the mode index based on the size of the block. For example, the context for signaling the intra mode set index and / or the mode index within the intra mode set depends on the block size of the current block. The block size may refer to the block width of the current block, the block height of the current block, the block area size (or the width*height of the current block), the sum of the width and height of the current block, the maximum value between the height and width of the current block, or the minimum value between the height and width of the current block.

[0154] In some embodiments, deriving context for set index and mode index based on the size of the block includes: deriving a first context as context for set index and mode index in response to the size of the block being equal to or less than a first threshold; and / or deriving a second context as context for set index and mode index in response to the size of the block being greater than the first threshold. For an example, the first threshold is 8×8.

[0155] In some implementations, if the block size of the current block is equal to or less than 8×8, a first context (e.g., where the context index value is 0) is selected as the context for the set index and the mode index for the current block. Otherwise, a second context (e.g., where the context index value is 1) is selected as the context for the set index and the mode index for the current block.

[0156] In some embodiments, deriving context for set index and mode index based on block size includes: deriving a first context as context for set index and mode index in response to the block size being less than a first threshold; deriving a second context as context for set index and mode index in response to the block size being equal to or greater than the first threshold and the block size being less than a second threshold; deriving a third context as context for set index and mode index in response to the block size being equal to or greater than the second threshold and the block size being less than a third threshold; and / or deriving a fourth context as context for set index and mode index in response to the block size being equal to or greater than the third threshold. For an example, the first threshold is that the area is 64 (or 8×8); the second threshold is that the area is 256 (or 16×16); and / or the third threshold is that the area is 1024 (or 32×32).

[0157] In some implementations, if the block area size of the current block is less than a first threshold (T1), a first context (e.g., where the context index value is 0) is selected as the context for the set index and mode index of the current block. If the area size of the current block is equal to or greater than T1 but less than a second threshold (T2), a second context (e.g., where the context index value is 1) is selected as the context for the set index and mode index of the current block. If the area size of the current block is equal to or greater than T2 but less than a third threshold (T3), a third context (e.g., where the context index value is 2) is selected as the context for the set index and mode index of the current block. Otherwise, a fourth context (e.g., where the context index value is 3) is selected as the context for the set index and mode index of the current block. For example, T1 is set to 64, T2 is set to 256, and / or T3 is set to 1024. For other examples, T1 may be set to 16, T2 may be set to 64, and T3 may be set to 256.

[0158] In some embodiments, the method 1200 may also and optionally include determining a set of allowed intra prediction modes for the block based on at least one of the following: mode information of at least one neighboring block of the block or a size of the block.

[0159] In some implementations, multiple allowed intra-prediction modes for a block may depend on the intra-prediction modes of neighboring blocks of the current block and / or the block size of the current block. For one example, depending on the intra-prediction modes of neighboring blocks of the current block and / or the block size of the current block, a mode set is selected from all available mode sets, and only the intra-prediction modes in the selected mode set can be used to encode the current block.

[0160] In some embodiments, determining a set of allowed intra-frame prediction modes for a block includes: in response to the number of non-directional prediction modes for at least one neighboring block being equal to or greater than two, determining the set of allowed intra-frame prediction modes for the block to include a non-directional prediction mode and a nominal directional mode; in response to the number of non-directional prediction modes for at least one neighboring block being equal to or greater than two and the size of the block being equal to or greater than a first threshold, determining the set of allowed intra-frame prediction modes for the block to include a non-directional prediction mode and a nominal directional mode; and / or in response to the number of directional prediction modes for at least one neighboring block being equal to or greater than one, determining the set of allowed intra-frame prediction modes for the block to include all intra-frame prediction modes. For example, the first threshold is 8×8, 16×16, or 32×32.

[0161] In some implementations, if at least two of the intra prediction modes of neighboring blocks of the current block are non-directional modes, only the non-directional mode and the nominal directional mode are allowed for the current block.

[0162] In some other implementations, if at least two of the intra prediction modes of the neighboring blocks of the current block are non-directional modes and / or the block size of the current block is greater than or equal to a threshold (TH1), then the current block only allows non-directional mode and nominal directional mode. For one example, TH1 is set to 32×32. For other examples, TH1 can be set to 8×8, 16×16, or 64×64.

[0163] In some other implementations, if at least one (or both) of the intra prediction modes of neighboring blocks of the current block are directional intra prediction modes, all available intra prediction modes for the current block are allowed.

[0164] In some embodiments, method 1200 may also and optionally include determining multiple sets of allowed intra-prediction modes for the block based on mode information of at least one neighboring block of the block and / or the size of the block. In some implementations, depending on the intra-prediction modes of neighboring blocks of the current block and / or the block size of the current block, more than one mode set is selected from all available mode sets, and only the intra-prediction modes in the selected more than one mode set can be used to encode the current block.

[0165] In some implementations, the neighboring blocks of the current block may include one or more of the following: a top (upper or above) block of the current block, a left block of the current block, an upper-left (upper-left or above-left) block of the current block.

[0166] Referring to step 1240, the device may determine the intra-frame prediction mode of the block based on the set index and the mode index. In some implementations, the device may construct a list of intra-frame modes for the block according to a predefined rule based on the size of the block, and / or the device may divide the list of intra-frame modes into multiple intra-frame mode sets for the block. The predefined rule may be one of a set of predefined rules and may be selected from the set of predefined rules based on the size of the block. In some other implementations, an intra-frame mode list is constructed for the current block based on the predefined rule. After the intra-frame mode list is constructed, the intra-frame prediction mode is divided into multiple intra-frame prediction mode sets according to its corresponding index in the intra-frame mode list. In order to signal the intra-frame prediction mode of the current block, first, a mode set index may be signaled to indicate a mode set index for the intra-frame prediction mode of the current block; and secondly, a mode index may be signaled to indicate an index of an intra-frame prediction mode within a mode set. The predefined rules may be different for different block sizes, and the block size may be indicated by the number of samples in the block.

[0167] The embodiments of the present disclosure may be used alone or in combination in any order. In addition, each of the method (or embodiment), encoder, and decoder may be implemented by a processing circuit system (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium. The embodiments of the present disclosure may be applied to luminance blocks or chrominance blocks; and in chrominance blocks, the embodiments may be applied to more than one color component individually, or may be applied to more than one color component together.

[0168] In the present disclosure, any steps or operations in some embodiments may be combined in any number or in any order as required. In the present disclosure, two or more steps or operations in some embodiments may be performed in parallel.

[0169] The above techniques may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. Fig.14 A computer system (2600) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0170] Computer software may be encoded using any suitable machine code or computer language, which may be subjected to mechanisms such as assembly, compilation, and linking to create code comprising instructions that may be executed directly by one or more computer central processing units (CPU), graphics processing units (GPU), etc., or through interpretation, microcode execution, etc.

[0171] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0172] Fig.14 The components for the computer system (2600) shown in the example are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Nor should the configuration of components be interpreted as having any dependency or requirement related to any one or combination of components shown in the exemplary embodiment of the computer system (2600).

[0173] The computer system (2600) may include certain human-machine interface input devices. Such human-machine interface input devices may be responsive to input by one or more human users through, for example, tactile input (such as keystrokes, swipes, data glove movements), audio input (such as voice, tapping), visual input (such as gestures), olfactory input (not depicted). The human-machine interface device may also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (such as voice, music, ambient sounds), images (such as scanned images, photographic images obtained from a still image camera), video (such as two-dimensional video, three-dimensional video including stereoscopic video).

[0174] The input human-machine interface device may include one or more of the following (only one of each item is drawn): keyboard (2601), mouse (2602), touch pad (2603), touch screen (2610), data gloves (not shown), joystick (2605), microphone (2606), scanner (2607), camera (2608).

[0175] The computer system (2600) may also include certain human interface output devices. Such human interface output devices may stimulate one or more senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: tactile output devices (for example, tactile feedback through a touch screen (2610), a data glove (not shown), or a joystick (2605), but there may also be tactile feedback devices that do not serve as input devices); audio output devices (such as: speakers (2609), headphones (not depicted)); visual output devices (such as screens (2610), including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light-emitting diode (OLED) screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which may be able to output two-dimensional visual outputs or more than three-dimensional outputs through methods such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and cigarette cans (not depicted)); and printers (not depicted).

[0176] The computer system (2600) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD read-only memory (ROM) / read-write (RW) (2620) including CD / DVD and other media (2621), thumb drives (2622), removable hard drives or solid-state drives (2623), legacy magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / application specific integrated circuit (ASIC) / programmable logic device (PLD) based devices such as security dongles (not depicted), etc.

[0177] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0178] The computer system (2600) may also include an interface (2654) to one or more communication networks (2655). The network may be, for example, wireless, wired, optical. The network may also be local, wide, metropolitan, vehicle-mounted, industrial, real-time, delay-tolerant, etc. Examples of networks include: local area networks such as Ethernet; wireless LANs; cellular networks including GSM (Global System for Mobile Communication), 3G (Third Generation), 4G (Fourth Generation), 5G (Fifth Generation), LTE (Long Term Evolution), etc.; television wired or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television; vehicle-mounted networks and industrial networks including CAN buses, etc. Some networks typically require an external network interface adapter that is attached to some universal data port or peripheral bus (2649) (such as, for example, a USB (Universal Serial Bus, USB) port of the computer system (2600)); other networks are typically integrated into the core of the computer system (2600) by attaching to a system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smart phone computer system). The computer system (2600) can use any of these networks to communicate with other entities. Such communication can be one-way receive-only (e.g., broadcast television), one-way send-only (e.g., a CAN bus to some CAN bus devices), or two-way, for example, using a local area digital network or a wide area digital network to other computer systems. Certain protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0179] The above-mentioned human-machine interface device, human-accessible storage device, and network interface may be attached to the core ( 2640 ) of the computer system ( 2600 ).

[0180] The core (2640) may include one or more central processing units (CPUs) (2641), graphics processing units (GPUs) (2642), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (2643), hardware accelerators for certain tasks (2644), graphics adapters (2650), etc. These devices, along with read-only memory (ROM) (2645), random access memory (2646), internal mass storage devices such as internal non-user accessible hard drives, SSDs (Solid State Drives, SSDs), etc. (2647) may be connected via a system bus (2648). In some computer systems, the system bus (2648) may be accessed in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, etc. Peripheral devices may be attached to the core's system bus (2648) directly or via a peripheral bus (2649). In an example, a screen (2610) may be connected to a graphics adapter (2650). The architecture of the peripheral bus includes PCI (Peripheral Component Interconnect, PCI), USB, etc.

[0181] The CPU (2641), GPU (2642), FPGA (2643) and accelerator (2644) can execute certain instructions, which can be combined to form the computer code mentioned above. The computer code can be stored in ROM (2645) or RAM (2646). Transient data can also be stored in RAM (2646), while permanent data can be stored in, for example, an internal mass storage device (2647). Fast storage and retrieval of any storage device in the storage device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (2641), GPUs (2642), mass storage devices (2647), ROMs (2645), RAMs (2646), etc.

[0182] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be those of a type well known and available to those skilled in the art of computer software.

[0183] As a non-limiting example, a computer system (2600) having an architecture, and in particular a core (2640), can provide functionality due to a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage device as described above and certain storage devices of the core (2640) having non-transitory properties, such as a core internal mass storage device (2647) or a ROM (2645). Software implementing various embodiments of the present disclosure can be stored in such a device and executed by the core (2640). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can enable the core (2640) and in particular the processor therein (including a CPU, GPU, FPGA, etc.) to perform specific processing or specific parts of specific processing described herein, including defining data structures stored in RAM (2646) and modifying such data structures according to processing defined by the software. Additionally or alternatively, a computer system may provide functionality due to logic hardwired or otherwise embodied in circuits (e.g., accelerators (2644)) that may operate in place of or in conjunction with software to perform specific processing or specific portions of specific processing described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuits storing software for execution (such as integrated circuits (ICs)), circuits implementing logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0184] Although specific inventions have been described with reference to illustrative embodiments, this description is not meant to be limiting. Based on this specification, various modifications of the illustrative embodiments of the present invention and additional embodiments will be apparent to those of ordinary skill in the art. Those skilled in the art will readily recognize that these and various other modifications may be made to the exemplary embodiments shown and described herein without departing from the spirit and scope of the present invention. Therefore, it is contemplated that the appended claims will cover any such modifications and alternative embodiments. Certain proportions in the illustrations may be exaggerated, while other proportions may be minimized. Therefore, the present disclosure and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A method for video decoding, characterized in that: The method comprises: Receiving a video bitstream including a plurality of encoded blocks, wherein the plurality of encoded blocks include a current block; Deriving a first context for entropy encoding a set index based on encoded information, wherein: the encoded information includes at least one of the following: information of the current block, or mode information of at least one neighboring block of the current block; the set index indicates an intra-mode set from a plurality of intra-mode sets, and the intra-mode set includes a plurality of intra-prediction modes; deriving a second context for entropy encoding a mode index of the current block based on the encoded information, wherein the mode index indicates an intra prediction mode from the intra mode set; entropy decoding the set index according to the first context; entropy decoding the mode index according to the second context; and The intra prediction mode of the current block is determined based on the set index and the mode index.

2. The method according to claim 1, characterized in that: The information of the current block includes the size of the block; and The pattern information of the at least one neighboring block includes at least one of the following: the number of direction patterns of the at least one neighboring block, or a pattern set index of the at least one neighboring block.

3. The method according to claim 1, characterized in that: The encoded information includes the number of neighboring blocks encoded using the directional intra prediction mode.

4. The method according to claim 3, characterized in that: The directional intra prediction mode includes at least one of the following: an inter coding mode, or an intra block copy mode.

5. The method according to claim 3, characterized in that: The directional intra prediction mode comprises an intra inter mode, and the intra inter mode contains a directional intra mode component.

6. The method according to claim 1, characterized in that: The encoded information includes the size of the current block.

7. The method according to claim 1, characterized in that Also includes: A set of allowed intra-prediction modes for the current block is determined based on the mode information of the at least one neighboring block of the current block and / or the size of the current block.

8. The method according to claim 7, characterized in that Determining a set of allowed intra prediction modes for the current block includes: If the number of non-directional prediction modes for the at least one neighboring block is equal to or greater than two, determining that the set of allowed intra prediction modes for the current block includes the non-directional prediction mode and a nominal directional mode; If the number of non-directional prediction modes for the at least one neighboring block is equal to or greater than two and the size of the current block is equal to or greater than a first threshold, determining that the set of allowed intra prediction modes for the current block includes the non-directional prediction mode and a nominal directional mode; and If the number of directional prediction modes for the at least one neighboring block is equal to or greater than one, determining that the set of allowed intra prediction modes for the current block includes all intra prediction modes.

9. The method according to claim 8, characterized in that The first threshold is 32×32.

10. The method according to claim 1, characterized in that Also includes: Two or more groups of allowed intra prediction modes for the current block are determined based on the mode information of the at least one neighboring block of the current block and / or the size of the current block.

11. A method for video encoding, characterized in that The method comprises: Receiving video data comprising a plurality of blocks, wherein the plurality of blocks includes a current block; Determining an intra prediction mode of the current block; Deriving a first context for entropy encoding a set index based on encoded information, wherein: the encoded information includes at least one of the following: information of the current block, or mode information of at least one neighboring block of the current block; the set index indicates an intra-mode set from a plurality of intra-mode sets, and the intra-mode set includes a plurality of intra-prediction modes; deriving a second context for entropy encoding a mode index of the current block based on the encoded information, wherein the mode index indicates an intra prediction mode from the intra mode set; entropy encoding the set index according to the first context; and The mode index is entropy encoded according to the second context.

12. The method according to claim 11, characterized in that: The information of the current block includes the size of the block; and The pattern information of the at least one neighboring block includes at least one of the following: the number of direction patterns of the at least one neighboring block, or a pattern set index of the at least one neighboring block.

13. The method according to claim 11, characterized in that: The encoded information includes the number of neighboring blocks encoded using the directional intra prediction mode.

14. The method according to claim 11, characterized in that: The encoded information includes the size of the current block.

15. The method according to claim 11, characterized in that Also includes: A set of allowed intra-prediction modes for the current block is determined based on the mode information of the at least one neighboring block of the current block and / or the size of the current block.

16. The method according to claim 11, characterized in that Also includes: Two or more groups of allowed intra prediction modes for the current block are determined based on the mode information of the at least one neighboring block of the current block and / or the size of the current block.

17. A device for video decoding, characterized in that: The device comprises: A receiving module, configured to receive a video stream including a plurality of encoded blocks, wherein the plurality of encoded blocks include a current block; A first acquisition module is configured to obtain a first context for entropy encoding a set index based on encoded information, wherein: the encoded information includes at least one of the following: information of the current block, or mode information of at least one neighboring block of the current block; the set index indicates an intra-mode set from a plurality of intra-mode sets, and the intra-mode set includes a plurality of intra-prediction modes; A second acquisition module, configured to derive a second context for entropy encoding a mode index of the current block based on the encoded information, wherein the mode index indicates an intra prediction mode from the intra mode set; A first decoding module, configured to perform entropy decoding on the set index according to the first context; A second decoding module, configured to perform entropy decoding on the mode index according to the second context; and A determination module is used to determine the intra-frame prediction mode of the current block based on the set index and the mode index.

18. A method for video encoding, characterized in that The device comprises: A receiving module, configured to receive video data comprising a plurality of blocks, wherein the plurality of blocks include a current block; A determination module, used to determine the intra-frame prediction mode of the current block; A first acquisition module is configured to obtain a first context for entropy encoding a set index based on encoded information, wherein: the encoded information includes at least one of the following: information of the current block, or mode information of at least one neighboring block of the current block; the set index indicates an intra-mode set from a plurality of intra-mode sets, and the intra-mode set includes a plurality of intra-prediction modes; A second acquisition module, configured to derive a second context for entropy encoding a mode index of the current block based on the encoded information, wherein the mode index indicates an intra prediction mode from the intra mode set; A first encoding module, configured to perform entropy encoding on the set index according to the first context; and A second encoding module is used to perform entropy encoding on the mode index according to the second context.

19. An electronic device, characterized in that: include: a memory for storing instructions; as well as A processor in communication with the memory, wherein when the processor executes the instructions, the processor is configured to cause the device to perform the method of any one of claims 1 to 16.

20. A non-transitory computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by a processor, the instructions cause the processor to perform the method of any one of claims 1 to 16.

Images