Video encoding and decoding method, computing system and storage medium

Abstract

The invention relates to a video encoding and decoding method, a computing system and a storage medium. Various implementations described herein include methods and systems for video coding. In one aspect, a method includes receiving a video bitstream, the video bitstream including a plurality of blocks. The method further includes identifying two or more parameters associated with an intra prediction mode for a first block of the plurality of blocks by parsing the single syntax from the video bitstream. The method further includes reconstructing the first block using the intra prediction mode and the two or more parameters.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the priority of U.S. Provisional Patent Application No. 63 / 624,268, filed on January 23, 2024, with the invention title "Joint Intra Mode Coding", and U.S. Patent Application No. 18 / 815,666, filed on August 26, 2024, with the invention title "Joint Intra Mode Coding". The entire contents of both are incorporated herein by reference in their entirety. Technical Field

[0003] The disclosed embodiments generally relate to video coding, and more particularly to video encoding and decoding methods, computing systems, and storage media, including but not limited to systems and methods for intra - prediction mode coding. Background Art

[0004] Digital video is supported by various electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc. These electronic devices send and receive digital video data over a communication network or otherwise transmit digital video data, and / or store digital video data on a storage device. Since the bandwidth capacity of the communication network and the memory resources of the storage device are both limited, video coding can be used to compress video data according to one or more video coding standards before transmitting or storing the video data. Video coding can be performed by hardware and / or software on an electronic device / client device or a server providing cloud services.

[0005] Video coding typically uses prediction methods that take advantage of the redundancy inherent in video data (e.g., inter-frame prediction, intra-frame prediction, etc.). Video coding aims to compress video data into a form that uses a lower bitrate while avoiding or minimizing degradation of video quality. A variety of video codec standards have been developed. For example, High-Efficiency Video Coding (HEVC / H.265) is a video compression standard designed as part of the MPEG-H project. The HEVC / H.265 standard was released by ITU-T and ISO / IEC in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4), respectively. Versatile Video Coding (VVC / H.266) is a video compression standard designed to be a successor to HEVC. The VVC / H.266 standard was released by ITU-T and ISO / IEC in 2020 (version 1) and 2022 (version 2), respectively. AOMedia Video 1 (AV1) is an open video coding format designed as an alternative to HEVC. The validation version 1.0.0 and errata 1 of the specification were released on January 8, 2019. Summary of the Invention

[0006] Among other things, the present disclosure describes a set of methods for video (image) compression, particularly related to intra-frame prediction modes and joint signaling techniques for multiple intra-frame prediction parameters (e.g., using a single syntax). For example, a single syntax can be used to identify a selected combination of two intra-frame prediction parameters (e.g., angular prediction mode and reference cue index) stored in a lookup table or list. By providing a lookup table filled with multiple entries that reflect more likely combinations of intra-frame prediction parameters, the signaling overhead for entropy coding can be reduced since a single syntax is used to convey information associated with multiple intra-frame prediction parameters. In addition, the lookup table can also be used for adjacent blocks, and due to additional information collected from adjacent blocks (e.g., combinations stored in the lookup table, and / or indices of entries accessed in the lookup table by adjacent blocks), the accuracy of the prediction for the current block can be improved.

[0007] According to some embodiments, a video decoding method includes: (i) receiving a video bitstream (e.g., an encoded video sequence) that includes a plurality of blocks (e.g., corresponding to a set of pictures); (ii) identifying two or more parameters associated with an intra-frame prediction mode for a first block among the plurality of blocks by parsing a single syntax from the video bitstream; and, (iii) reconstructing the first block using the intra-frame prediction mode and the two or more parameters.

[0008] According to some embodiments, a video encoding method includes: (i) receiving video data (e.g., a source video sequence) that includes a plurality of blocks (e.g., corresponding to a set of pictures); (ii) when two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be jointly signaled, writing a single syntax to a video bitstream to indicate the two or more parameters; (iii) when two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be separately signaled, writing separate syntaxes to the video bitstream to indicate each of the two or more parameters; and (iv) encoding the first block using the intra prediction mode and the two or more parameters.

[0009] According to some embodiments, a method of processing visual media data includes: (i) obtaining a source video sequence that includes a plurality of frames; and (ii) performing a conversion between the source video sequence and a video bitstream of the visual media data according to formatting rules. The bitstream includes a plurality of blocks and syntax elements, the plurality of blocks includes a first block, and the syntax elements indicate two or more parameters associated with an intra prediction mode for the first block. The formatting rules specify that the first block is to be reconstructed using the intra prediction mode and the two or more parameters.

[0010] According to some embodiments, a computing system is provided, such as a streaming system, a server system, a personal computer system, or other electronic device. The computing system includes control circuitry and a memory storing one or more instruction sets. The one or more instruction sets include instructions for performing any of the methods described herein. In some embodiments, the computing system includes an encoder component and a decoder component (e.g., a transcoder).

[0011] According to some embodiments, a non - volatile computer - readable storage medium is provided. The non - volatile computer - readable storage medium stores one or more instruction sets executable by a computing system. The one or more instruction sets include instructions for performing any of the methods described herein.

[0012] Thus, methods, devices, and systems for encoding and decoding video are disclosed. Such methods, devices, and systems may supplement or replace conventional methods, devices, and systems for video encoding / decoding.

[0013] The features and advantages described in the specification are not necessarily all inclusive, and in particular, for those of ordinary skill in the art, some additional features and advantages will be apparent from the accompanying drawings, the specification, and the claims provided according to the present disclosure. Additionally, it should be noted that the language used in the specification is mainly selected for readability and guidance purposes and does not necessarily serve to depict or delimit the subject matter described herein. Description of the Drawings

[0014] For a more detailed understanding of the present disclosure, a more specific description can be made by referring to the features of various embodiments, some of which are shown in the accompanying drawings. However, the accompanying drawings only show the relevant features of the present disclosure and are not necessarily considered restrictive, because those skilled in the art will understand after reading the present disclosure that the specification may also allow other effective features.

[0015] Figure 1 is a block diagram showing an example communication system according to some embodiments.

[0016] Figure 2A is a block diagram showing example elements of an encoder component according to some embodiments.

[0017] Figure 2B is a block diagram showing example elements of a decoder component according to some embodiments.

[0018] Figure 3 is a block diagram showing an example server system according to some embodiments.

[0019] Figure 4A shows an example intra-frame prediction mode according to some embodiments.

[0020] Figure 4B shows an intra-frame prediction scheme based on various reference lines according to some embodiments.

[0021] Figure 5A shows an example video decoding process according to some embodiments.

[0022] Figure 5B shows an example video encoding process according to some embodiments.

[0023] In accordance with common practice, the various features shown in the drawings are not necessarily drawn to scale, and like reference numerals may be used to represent similar features throughout the specification and drawings. Detailed Description

[0024] The present disclosure describes video / image compression techniques that include intra prediction modes and techniques for jointly transmitting multiple intra prediction parameters using a single syntax. For example, a single syntax can be used to identify multiple intra prediction parameters (e.g., such as an angular prediction mode and a reference index). The single syntax can correspond to an index of a look-up table or a list (e.g., entropy coded), where each entry in the table has values for multiple parameters. The advantage of joint signaling is reduced signaling overhead (e.g., using one syntax instead of multiple syntaxes to convey information associated with multiple intra prediction parameters). Additionally, prediction accuracy can be improved by using a look-up table populated with entries reflecting the most likely combinations of intra prediction parameters, which can also obtain information from neighboring blocks. For example, signaling an index in a table representing the most likely intra prediction parameter combinations can improve both signaling overhead and coding accuracy.

[0025] Example systems and devices

[0026] Figure 1 is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to electronic devices 120-m) communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is a streaming system, such as used with video-enabled applications such as video conferencing applications, digital television applications, and media storage and / or distribution applications.

[0027] The source device 102 includes a video source 104 (e.g., a camera assembly or media storage) and an encoder component 106. In some embodiments, the video source 104 is a digital camera (e.g., configured to create an uncompressed video sample stream). The encoder component 106 generates one or more encoded video bitstreams based on the video stream. The data volume of the video stream from the video source 104 is higher compared to the encoded video bitstreams 108 generated by the encoder component 106. Since the data volume of the encoded video bitstreams 108 is lower (less data) compared to the video stream from the video source, the encoded video bitstreams 108 require less bandwidth to transmit and less storage space to store compared to the video stream from the video source 104. In some embodiments, the source device 102 does not include the encoder component 106 (e.g., configured to send uncompressed video to one or more networks 110).

[0028] One or more networks 110 represent any number of networks for transferring information between the source device 102, the server system 112, and / or the electronic devices 120, including, for example, wired communication networks and / or wireless communication networks. One or more networks 110 may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and / or the Internet.

[0029] One or more networks 110 include the server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is or includes a streaming server (e.g., configured to store and / or distribute video content such as an encoded video stream from the source device 102). The server system 112 includes an encoder component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the encoder component 114 includes an encoder component and / or a decoder component. In various embodiments, the encoder component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the encoder component 114 is configured to decode the encoded video bitstream 108 and re-encode the video data using different coding standards and / or methods to generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings based on the encoded video bitstream 108. In some embodiments, the server system 112 acts as a Media-Aware Network Element (MANE). For example, the server system 112 may be configured to trim the encoded video bitstream 108 to create potentially different bitstreams for one or more electronic devices 120. In some embodiments, the MANE is provided separately from the server system 112.

[0030] The electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode the encoded video data 116 to generate an output video stream that can be presented on a display or other type of display device. In some embodiments, one or more of the electronic devices 120 do not include a display component (e.g., are communicatively coupled to an external display device and / or include media storage). In some embodiments, the electronic device 120 is a streaming client. In some embodiments, the electronic device 120 is configured to access the server system 112 to obtain the encoded video data 116.

[0031] The source device and / or the plurality of electronic devices 120 are sometimes referred to as "terminal devices" or "user devices". In some embodiments, the source device 102 and / or one or more of the electronic devices 120 are examples of server systems, personal computers, portable devices (e.g., smart phones, tablets, or laptop computers), wearable devices, video conferencing devices, and / or other types of electronic devices.

[0032] In an example operation of the communication system 100, the source device 102 transmits an encoded video bitstream 108 to the server system 112. For example, the source device 102 may encode a picture stream captured by the source device. The server system 112 receives the encoded video bitstream 108 and may use the encoder component 114 to decode and / or encode the encoded video bitstream 108. For example, the server system 112 may perform encoding on the video data that is more suitable for network transmission and / or storage. The server system 112 may transmit the encoded video data 116 (e.g., one or more encoded video bitstreams) to one or more of the electronic devices 120. Each of the electronic devices 120 may decode the encoded video data 116 and selectively display the video pictures.

[0033] Figure 2A is a block diagram showing example elements of an encoder component 106 according to some embodiments. The encoder component 106 receives video data (e.g., a source video sequence) from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives a video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder component 106). The video source 104 may provide a source video sequence in the form of a digital video sample stream, which may have any suitable bit depth (e.g., 8 bits, 10 bits, or 12 bits), any color space (e.g., BT.601 Y CrCb or RGB), and any suitable sampling structure (e.g., Y CrCb 4:2:0 or Y CrCb 4:4:4). In some embodiments, the video source 104 is a storage device that stores previously captured / prepared video. In some embodiments, the video source 104 is a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures that, when viewed in sequence, are given motion. The pictures themselves may be constructed as spatial pixel arrays, where each pixel may include one or more samples depending on the sampling structure, color space, etc. used. Those of ordinary skill in the art can readily understand the relationship between pixels and samples.

[0034] The encoder component 106 is configured to encode and / or compress pictures of a source video sequence into an encoded video sequence 216 in real time or under other time constraints required by the application. In some embodiments, the encoder component 106 is configured to perform a conversion between the source video sequence and a bitstream of visual media data (e.g., a video bitstream). Implementing an appropriate encoding speed is a function of the controller 204. In some embodiments, the controller 204 controls other functional units as described below and is functionally coupled to other functional units. Parameters set by the controller 204 may include rate control related parameters (e.g., picture skipping, quantizer, and / or lambda value of rate-distortion optimization techniques), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those of ordinary skill in the art can readily identify other functions of the controller 204, which may relate to the encoder component 106 optimized for a specific system design.

[0035] In some embodiments, the encoder component 106 is configured to operate in an encoding loop. In a simplified example, the encoding loop includes a source encoder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and one or more reference pictures) and a (local) decoder 210. The decoder 210 reconstructs the symbols in a manner similar to how a (remote) decoder creates sample data to create sample data (when the compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream (sample data) is input into the reference picture memory 208. Since the decoding of the symbol stream produces a bit-exact result independent of the decoder location (local or remote), the content in the reference picture memory 208 is also bit-exactly corresponding between the local encoder and the remote encoder. In this way, the prediction part of the encoder interprets the reference picture sample values as the same sample values as those interpreted by the decoder when using prediction during decoding.

[0036] The operation of the decoder 210 may be the same as the operation of a remote decoder (such as the decoder component 122), which is described in detail below in conjunction with Figure 2B However, briefly referring to Figure 2B , when the symbols are available and the entropy decoding part of the decoder component 122, including the buffer memory 252 and the parser 254, can encode / decode the symbols into the encoded video sequence losslessly through the entropy encoder 214 and the parser 254, it may not be fully implemented in the local decoder 210.

[0037] Except for parsing / entropy decoding, the decoder techniques described herein may exist in corresponding encoders in substantially the same functional form. For this reason, the disclosed subject matter focuses on decoder operations. Additionally, the description of encoder techniques may be simplified because encoder techniques are inverse to decoder techniques.

[0038] As part of its operation, the source encoder 202 may perform motion-compensated predictive coding, which predictively encodes an input frame by referring to one or more previously encoded frames designated as reference frames in a video sequence. In this way, the encoding engine 212 encodes the difference between a pixel block of the input frame and pixel blocks of one or more reference frames, which may be selected as the predictive reference for the input frame. The controller 204 may manage the encoding operations of the source encoder 202, including, for example, setting parameters and subgroup parameters for encoding video data.

[0039] The decoder 210 decodes the encoded video data of frames that may be designated as reference frames based on the symbols created by the source encoder 202. The operation of the encoding engine 212 may advantageously be a lossy process. When the encoded video data is decoded at a video decoder ( Figure 2A not shown), the reconstructed video sequence may be a copy of the source video sequence with some errors. The decoder 210 replicates the decoding process that may be performed by a remote video decoder on a reference frame and may cause the reconstructed reference frame to be stored in the reference picture memory 208. In this way, the encoder component 106 may locally store a copy of the reconstructed reference frame that has the same content (in the absence of transmission errors) as the reconstructed reference frame that will be obtained by the remote video decoder.

[0040] The predictor 206 may perform a prediction search for the encoding engine 212. That is, for a new frame to be encoded, the predictor 206 may search the reference picture memory 208 for sample data (as candidate reference pixel blocks) or some metadata, such as reference picture motion vectors, block shapes, etc., that may serve as an appropriate prediction reference for the new picture. The predictor 206 may operate on a per-pixel block basis of sample blocks to find a suitable prediction reference. Based on the search results obtained by the predictor 206, it may be determined that the input picture may have a prediction reference taken from multiple reference pictures stored in the reference picture memory 208.

[0041] The outputs of all the foregoing functional units may be entropy encoded in the entropy encoder 214. The entropy encoder 214 losslessly compresses the symbols generated by various functional units according to techniques known to those of ordinary skill in the art (e.g., Huffman coding, variable-length coding, and / or arithmetic coding), thereby converting the symbols into an encoded video sequence.

[0042] In some embodiments, the output of the entropy encoder 214 is coupled to a transmitter. The transmitter may be configured to buffer the encoded video sequence created by the entropy encoder 214, thus preparing for transmission over a communication channel 218, which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter may be configured to merge the encoded video data from the source encoder 202 with other data to be transmitted (e.g., encoded audio data and / or auxiliary data streams (sources not shown)). In some embodiments, the transmitter may transmit additional data when transmitting the encoded video. The source encoder 202 may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures, and redundant slices, supplementary enhancement information (SEI) messages, visual usability information (VUI) parameter set segments, etc.

[0043] The controller 204 may manage the operation of the encoder components 106. During encoding, the controller 204 may assign a certain encoded picture type to each encoded picture, but this may affect the encoding techniques applicable to the corresponding picture. For example, a picture may be assigned as an intra picture (I picture), a predictive picture (P picture), or a bi-predictive picture (B picture). Intra pictures can be encoded and decoded without using any other frames in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including for example independent decoder refresh (IDR) pictures. Those of ordinary skill in the art are aware of the variants of I pictures and their corresponding applications and characteristics, and thus will not be elaborated herein. Predictive pictures can be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and a reference index to predict the sample values of each block. Bi-predictive pictures can be encoded and decoded using intra prediction or inter prediction, which uses at most two motion vectors and reference indices to predict the sample values of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstructing a single block.

[0044] Source pictures can typically be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples each), and encoded block by block. These blocks can be predictively encoded with reference to other (already encoded) blocks, which are determined according to the encoding assignment of the corresponding picture applied to the block. For example, blocks of an I picture can be non-predictively encoded, or the block can be predictively encoded with reference to already encoded blocks of the same picture (spatial prediction or intra-frame prediction). Pixel blocks of a P picture can be non-predictively encoded with reference to a previously encoded reference picture through spatial prediction or through temporal prediction. Blocks of a B picture can be non-predictively encoded with reference to one or two previously encoded reference pictures through spatial prediction or through temporal prediction.

[0045] The video captured can be a plurality of source pictures (video pictures) in a time series. Intra-picture prediction (usually simplified to intra-frame prediction) exploits the spatial correlation in a given picture, while inter-picture prediction exploits the (temporal or other) correlation between pictures. In an example, a particular picture being encoded / decoded is partitioned into blocks, and the particular picture being encoded / decoded is referred to as the current picture. When a block in the current picture is similar to a reference block in a reference picture that has been previously encoded and is still buffered in the video, the block in the current picture can be encoded by a vector called a motion vector. The motion vector points to the reference block in the reference picture, and in the case of using multiple reference pictures, the motion vector can have a third dimension identifying the reference picture.

[0046] The encoder component 106 can perform encoding operations according to a predetermined video coding technique or standard (such as any of the techniques or standards described herein). In its operation, the encoder component 106 can perform various compression operations, including predictive coding operations that utilize temporal redundancy and spatial redundancy in the input video sequence. Thus, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0047] Figure 2B is a block diagram showing example elements of a decoder component 122 according to some embodiments. Figure 2B The decoder component 122 in is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to the loop filter 256 and configured to transmit data to (e.g., transmit via a wired or wireless connection) the display 124.

[0048] In some embodiments, decoder component 122 includes a receiver coupled to channel 218, and the receiver is configured to receive data from channel 218 (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more encoded video sequences to be decoded by decoder component 122. In some embodiments, the decoding of each encoded video sequence is independent of other encoded video sequences. Each encoded video sequence may be received from channel 218, which may be a hardware / software link to a storage device storing the encoded video data. The receiver may receive encoded video data and other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective consuming entities (not depicted). The receiver may separate the encoded video sequences from the other data. In some embodiments, the receiver receives additional (redundant) data with the encoded video. The additional data may be included as part of the encoded video sequence. Decoder component 122 may use the additional data to decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0049] According to some embodiments, decoder component 122 includes buffer memory 252, parser 254 (sometimes also referred to as an entropy decoder), scaler / inverse transform unit 258, intra picture prediction unit 262, motion compensation prediction unit 260, aggregator 268, loop filter unit 256, reference picture memory 266, and current picture memory 264. In some embodiments, decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuits. Decoder component 122 may be implemented at least partially in software.

[0050] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., to prevent network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, in addition to buffer memory 252 inside decoder component 122 (e.g., which is configured to handle playout timing), a separate buffer memory is provided outside decoder component 122 (e.g., to prevent network jitter). When receiving data from a storage / forwarding device with sufficient bandwidth and controllability or from an isochronous network, buffer memory 252 may not be needed, or buffer memory 252 may be made smaller. For use on a best-effort packet network such as the Internet, buffer memory 252 may be needed, which may be relatively large and / or have an adaptive size, and may be implemented at least partially in an operating system or similar element outside decoder component 122.

[0051] The parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. The symbols may include, for example, information for managing the operation of decoder components 122 and / or information for controlling a display device such as display 124. The control information for one or more display devices may be in the form of, for example, Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not depicted). The parser 254 parses (entropy decodes) the encoded video sequence. The encoding of the encoded video sequence may be according to a video coding technology or standard and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser 254 may extract subgroup parameter sets for at least one subgroup of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the subgroup. The subgroup may include Group of Pictures (GOP), picture, tile, slice, macroblock, Coding Unit (CU), block, Transform Unit (TU), Prediction Unit (PU), etc. The parser 254 may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0052] The reconstruction of symbols 270 may involve multiple different units, depending on the type of the encoded video picture or a portion thereof (such as: inter-picture and intra-picture, inter-block and intra-block) and other factors. Which units are involved and the manner of involvement may be controlled by subgroup control information, which is parsed by the parser 254 from the encoded video sequence. For clarity, such subgroup control information flows between the parser 254 and the multiple units below are not depicted.

[0053] The decoder components 122 may be conceptually subdivided into multiple functional units, and in some implementations, these units interact closely with each other and may be at least partially integrated with each other. However, for clarity, the conceptual subdivision into functional units is maintained here.

[0054] The scaler / inverse transform unit 258 receives, from the parser 254, the quantized transform coefficients as one or more symbols 270, and control information such as the transform mode to be used, block size, quantization factor, and / or quantization scaling matrix. The scaler / inverse transform unit 258 may output a block including sample values, and the block of sample values may be input into the aggregator 268. In some cases, the output samples of the scaler / inverse transform unit 258 belong to intra-coded blocks; that is, blocks that do not use prediction information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by the intra-picture prediction unit 262. The intra-picture prediction unit 262 may generate a block having the same size and shape as the block being reconstructed, using the surrounding reconstructed information of the current (partially reconstructed) picture obtained from the current picture memory 264. The aggregator 268 may add, based on each sample, the predictive information generated by the intra-picture prediction unit 262 to the output sample information provided by the scaler / inverse transform unit 258.

[0055] In other cases, the output samples of the scaler / inverse transform unit 258 belong to inter-coded and potentially motion-compensated blocks. In such a case, the motion compensation prediction unit 260 may access the reference picture memory 266 to extract samples for prediction. After motion-compensating the extracted samples according to the symbols 270 belonging to the block, these samples may be added by the aggregator 268 to the output of the scaler / inverse transform unit 258 (referred to as residual samples or a residual signal in this case), thereby generating output sample information. The extraction of the prediction samples by the motion compensation prediction unit 260 from an address within the reference picture memory 266 may be controlled by a motion vector. The motion vector may be provided to the motion compensation prediction unit 260 in the form of symbols 270, which may have, for example, X, Y, and reference picture components. Motion compensation may also include, for example, interpolation of the sample values extracted from the reference picture memory 266 when using sub-sampled accurate motion vectors, a motion vector prediction mechanism, and the like.

[0056] The output samples of the aggregator 268 may be employed by various loop filtering techniques in the loop filter unit 256. Video compression techniques may include in-loop filter techniques that are controlled by parameters included in the encoded video bitstream, and the parameters may be available to the loop filter unit 256 as symbols 270 from the parser 254. However, video compression techniques may also respond to meta-information obtained during the decoding of a previously decoded (in decoding order) portion of an encoded picture or an encoded video sequence, and to previously reconstructed and loop-filtered sample values. The output of the loop filter unit 256 may be a sample stream, which may be output to a display device such as the display 124 and stored in the reference picture memory 266 for future inter-picture prediction.

[0057] Once reconstructed, some of the encoded pictures can be used as reference pictures for future prediction. Once an encoded picture has been reconstructed and the encoded picture has been identified as a reference picture (e.g., by parser 254), the current reference picture can become part of the reference picture memory 266 and a new current picture memory can be reallocated before starting to reconstruct subsequent encoded pictures.

[0058] The decoder component 122 can perform decoding operations according to a predetermined video compression technique, which can be recorded in a standard (such as any of the standards described herein). In the sense that the encoded video sequence follows the syntax of the video compression technique or standard, the encoded video sequence can conform to the syntax specified by the video compression technique or standard used, i.e., conform to the syntax specified in the video compression technique document or standard, particularly in the profile thereof. Additionally, to conform to some video compression techniques or standards, the complexity of the encoded video sequence can be within the range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (e.g., measured in megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the level can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata for HRD buffer management signaled in the encoded video sequence.

[0059] Figure 3 is a block diagram showing a server system 112 according to some embodiments. The server system 112 includes a control circuit 302, one or more network interfaces 304, a memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuit 302 includes one or more processors (e.g., CPU, GPU, and / or DPU). In some embodiments, the control circuit includes one or more field programmable gate arrays, hardware accelerators, and / or one or more integrated circuits (e.g., application specific integrated circuits).

[0060] One or more network interfaces 304 may be configured to connect to one or more communication networks (e.g., wireless networks, wired networks, and / or optical networks). The communication network may be a local area network, wide area network, metropolitan area network, vehicle network, industrial network, real-time network, delay-tolerant network, etc. Examples of communication networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial networks including CANBus, etc. Such communication may be one-way reception only (e.g., broadcast television), one-way transmission only (e.g., CANbus connected to certain CANbus devices), or two-way, e.g., connecting to other computer systems using local or wide area digital networks. Such communication may include communication to one or more cloud computing networks.

[0061] The user interface 306 includes one or more output devices 308 and / or one or more input devices 310. One or more input devices 310 may include one or more of the following: keyboard, mouse, touchpad, touchscreen, data glove, joystick, microphone, scanner, camera, etc. One or more output devices 308 may include one or more of the following: audio output devices (e.g., speakers), visual output devices (e.g., displays or monitors), etc.

[0062] The memory 314 may include high-speed random access memory (such as DRAM, SRAM, DDR RAM, and / or other random access solid-state memory devices) and / or non-volatile memory (such as one or more disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state storage devices). The memory 314 optionally includes one or more storage devices remote from the control circuit 302. The memory 314, or, optionally, one or more non-volatile solid-state storage devices within the memory 314 includes non-volatile computer-readable storage media. In some embodiments, the memory 314 or the non-volatile computer-readable storage media of the memory 314 stores the following programs, modules, instructions, and data structures or subsets or supersets thereof:

[0063] · An operating system 316, including processes for handling various basic system services and performing hardware-related tasks;

[0064] · A network communication module 318 for connecting the server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired connections and / or wireless connections);

[0065] · A codec module 320 for performing various functions related to encoding and / or decoding data, such as video data. In some embodiments, the codec module 320 is an instance of the encoder component 114. The codec module 320 includes, but is not limited to, one or more of the following modules:

[0066] o A decoding module 322 for performing various functions related to decoding encoded data, such as the functions related to the decoder component 122 described above; and

[0067] o An encoding module 340 for performing various functions related to data encoding, such as the functions related to the encoder component 106 described above; and

[0068] · A picture memory 352 for storing pictures and picture data, e.g., for use with the codec module 320. In some embodiments, the picture memory 352 includes one or more of the following: a reference picture memory 208, a buffer memory 252, a current picture memory 264, and a reference picture memory 266.

[0069] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform various functions related to the parser 254 described above), a transform module 326 (e.g., configured to perform various functions related to the scaler / inverse transform unit 258 described above), a prediction module 328 (e.g., configured to perform various functions related to the motion compensation prediction unit 260 and / or the intra picture prediction unit 262 described above), and a filter module 330 (e.g., configured to perform various functions related to the loop filter 256 described above).

[0070] In some embodiments, the encoding module 340 includes a code module 342 (e.g., configured to perform various functions related to the source encoder 202 and / or the encoding engine 212 described above) and a prediction module 344 (e.g., configured to perform various functions related to the predictor 206 described above). In some embodiments, the decoding module 322 and / or the encoding module 340 includes Figure 3 a subset of the modules shown. For example, both the decoding module 322 and the encoding module 340 use a shared prediction module.

[0071] Each of the above-identified modules stored in the memory 314 corresponds to an instruction set for performing the functions described in the present application. The above-identified modules (e.g., instruction sets) need not be implemented as separate software programs, processes, or modules, and thus, in various embodiments, various subsets of these modules may be combined or otherwise rearranged. For example, the codec module 320 optionally does not include separate decoding and encoding modules, but instead uses the same set of modules to perform both sets of functions. In some embodiments, the memory 314 stores a subset of the above-identified modules and data structures. In some embodiments, the memory 314 stores additional modules and data structures not described above.

[0072] Although Figure 3 a server system 112 according to some embodiments is shown, Figure 3 it is presented more as a functional description of the various features that may be present in one or more server systems than as a structural diagram of the embodiments described in the present application. In practice, the items shown separately may be combined and some items may be separated. For example, Figure 3 some of the items shown separately in may be implemented on a single server, and a single item may be implemented by one or more servers. The actual number of servers used to implement the server system 112 and how the features are distributed among these servers will vary depending on the implementation and, optionally, partially depend on the data traffic processed by the server system during peak usage periods as well as during average usage periods.

[0073] Example encoding techniques

[0074] The encoding processes and techniques described below may be performed at the above devices and systems (e.g., the source device 102, the server system 112, and / or the electronic device 120). Hereinafter, the intra prediction mode and the technique for jointly transmitting intra mode coding parameters will be described.

[0075] Figure 4A depicts a subset of the prediction value directions for various directional intra prediction modes. For directional intra prediction, some methods support 8 directional modes corresponding to angles from 45 degrees to 207 degrees. To exploit more kinds of spatial redundancy in the directional texture, the directional intra mode can be extended to a set of angles with finer granularity. For example, the 8 angles can be represented as nominal angles. These 8 nominal angles are named V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED, and in Figure 4AShown in. For each nominal angle, there may be 7 finer angles, resulting in a total of 56 direction angles. The predicted angle can be described by adding the nominal in-frame angle to the angle increment. Thus, there are eight nominal in-frame prediction modes, each associated with a set of angle increment offsets ranging from -3 to +3. Figure 4A Eight nominal modes (solid arrows) are shown, and an example of the set of angle increment offsets (dashed arrows) around the D67_PRED nominal mode is shown. The point 402 where the arrows converge represents the sample to be predicted. The arrows indicate the direction of using adjacent samples to predict the sample at point 402. For example, D45_PRED indicates that the sample is predicted based on one or more adjacent samples in the upper right direction at an angle of 45 degrees with respect to the horizontal direction. Similarly, D203_PRED indicates that the sample is predicted based on one or more adjacent samples in the lower left direction of the sample at an angle of 22.5 degrees with respect to the horizontal direction.

[0076] In some embodiments, the reference index for intra prediction refers to the index of adjacent neighboring samples or non-adjacent neighboring samples used to perform intra prediction. Figure 4B An example of multiple reference lines according to some embodiments is shown. Although Figure 4B all reference lines are shown adjacent to each other, in some embodiments, the allowed reference lines may also not be adjacent to each other. In some embodiments, the cross-component chrominance prediction mode refers to a chrominance intra prediction mode that uses the corresponding luma samples to predict the samples in the current chrominance block.

[0077] In some embodiments, sample prediction in a coding block or a prediction block may be based on one of a set of reference lines. In other words, instead of always using the closest neighboring line (e.g., the upper neighboring line immediately adjacent to the coding block or the left neighboring line immediately adjacent to the coding block), multiple reference lines may be provided as selection options for intra prediction. Such an implementation of intra prediction may be referred to as Multiple Reference Line Selection (MRLS). In some embodiments, the encoder determines and signals which of the multiple reference lines is used to generate the intra prediction value. On the decoder side, after parsing the reference line index, the reconstructed reference samples may be identified by looking up the specified reference line according to the intra prediction mode (such as directional, non - directional, and other intra prediction modes), and then the intra prediction of the current intra prediction block is generated. In some embodiments, the reference line index may be signaled at the coding block level, and only one reference line may be selected from the multiple reference lines for the intra prediction of a coding block. In some examples, multiple (more than one) reference lines may be selected to be used together for intra prediction. For example, the multiple reference lines may be combined, averaged, interpolated, or in any other way with or without weights to generate the prediction. In some exemplary implementations, MRLS may be applied to only some components and not to other components (e.g., applied to the luminance component and may not be applied to the chrominance component(s)).

[0078] In Figure 4BIn [the figure], an example of 4 reference lines MRLS is shown. An intra-coded block 404 can be predicted based on one of the 4 horizontal reference lines 406, 408, 410, and 412 and the 4 vertical reference lines 414, 416, 418, and 420. Among these reference lines, 406 and 414 are directly adjacent reference lines. The reference lines can be indexed according to their respective distances from the coded block. For example, reference lines 406 and 414 can be referred to as zero reference lines, while the other reference lines can be referred to as non-zero reference lines. Specifically, reference lines 408 and 416 can be referred to as the first reference lines; reference lines 410 and 418 can be referred to as the second reference lines; and reference lines 412 and 420 can be referred to as the third reference lines. For example, if the coded block 404 has an angular / directional intra prediction mode D45_PRED, that is, in a direction 45 degrees from the horizontal direction, samples (e.g., sample 438) in the coded block 404 are predicted based on one or more prediction samples in the upper right. If the coded block 404 uses a reference sample from the zero reference line 406, sample 438 will be predicted based on reference sample 422. If the coded block 404 uses a reference sample from the third reference line 412, sample 438 will be predicted based on reference sample 428. Similarly, if the second reference line 410 and the first reference line 408 are used respectively, reference samples 426 and 424 will be used for prediction. For other angular intra prediction modes such as the D135_PRED mode, reference samples 430, 432, 434, or 436 can be used to predict sample 440 in the coded block 404.

[0079] In some embodiments, for one or more components of a coded block, multiple syntaxes related to intra mode coding can be jointly signaled instead of separately signaling the intra prediction mode and the reference line. In some embodiments, by jointly signaling multiple syntaxes using a single indicator, lower bits can be used to signal higher probability combinations (e.g., angular intra prediction mode and reference line index, or other parameters). For example, a single indicator can be used to specify the reference line index and the intra prediction mode. In some embodiments, the use of reference lines enables inheritance of information from adjacent blocks. In some embodiments, a look-up table that contains a combination of intra prediction syntax and is used for one or more adjacent blocks can also be used for the current block, and information related to the index of the entry in the look-up table used can provide additional information (e.g., statistical information or other usage information) for prediction of the current block. Since the number of bits used is reduced when multiple syntaxes are jointly signaled, the overhead for entropy coding is saved.

[0080] In some embodiments, a lookup table (or list, or set, or group) is constructed based on the encoded mode information of neighboring blocks and / or non-neighboring blocks. Each entry in the lookup table includes one or more syntaxes for performing intra prediction, and an index in the lookup table is written into the bitstream. In some embodiments, syntaxes related to the intra prediction mode of neighboring encoded blocks and / or non-neighboring encoded blocks (e.g., intra prediction mode index, reference cue index, matrix-based intra prediction mode index, cross-component intra prediction mode) are inserted into the lookup table.

[0081] In some embodiments, a combination of syntaxes related to the intra prediction mode is derived based on the encoded information of neighboring encoded blocks and / or non-neighboring encoded blocks. For example, the intra prediction mode of a left neighboring block and the reference cue index of an upper neighboring block can be combined to generate an entry in the lookup table. In some embodiments, the intra prediction mode of a neighboring block is an angular intra prediction mode, and the angle is slightly adjusted by adding an offset value (such as +N or -N, where N is a positive integer, such as 1, 2, or 4), while keeping the reference cue index unchanged, and then the modified combination is added to the lookup table. In other words, before adding the offset to determine the angular prediction mode of the current block, the actual angle of the prediction mode of the neighboring block is used as the starting point for selecting the angular prediction mode.

[0082] In some embodiments, each entry in the lookup table includes a corresponding intra prediction mode and reference cue index. In some embodiments, each entry in the lookup table includes the intra prediction modes of the luminance component and the chrominance component. Syntax signaled is used to indicate whether any combination of the intra prediction mode and reference cue index in the lookup table is used to encode the current block. If the syntax signaled indicates the use of the lookup table, the index of the lookup table can be further written into the bitstream to indicate which combination is selected (e.g., corresponding to the respective entry in the lookup table). If the syntax signaled indicates not using the lookup table, the intra prediction mode and / or reference cue index are written into the bitstream separately. In some embodiments, the intra prediction mode only includes the intra prediction mode for the luminance component. In some embodiments, when each entry in the lookup table includes a corresponding intra prediction mode and reference cue index, the intra prediction mode can include the luminance intra prediction mode and the chrominance intra prediction mode. For example, in some embodiments, when the luminance block and the chrominance block share the same block partition and / or transform partition, the intra prediction mode is used for both the luminance and chrominance intra prediction modes. In some embodiments, the lookup table includes an entry specifying the use of the cross-component chrominance prediction mode. When this entry is selected, additional syntax can be further signaled to indicate which syntax is used to encode the block.

[0083] In some embodiments, a plurality of look-up tables are constructed based on the coded mode information of adjacent blocks and / or non-adjacent blocks. For example, the entries in the first look-up table indicate combinations of N1 intra-mode related syntaxes, the entries in the second look-up table indicate combinations of N2 intra-mode related syntaxes, and the entries in the third look-up table indicate combinations of N3 intra-mode related syntaxes, where N1, N2, and N3 are all positive integers. In one example, N1 is 3, N2 is 2, and N3 is 1. In some embodiments, based on the coded information of adjacent blocks, if the most likely mode of an adjacent block is different from that of the current block, a different look-up table is used. For example, if the smooth mode is used to code an adjacent block, the smooth mode may also be used to code the current block. Based on this information, the size of the look-up table can be made smaller due to the information implicitly derived from adjacent blocks.

[0084] In some embodiments, the selection of the look-up table is first written into the bitstream. Further, an index corresponding to an entry in the selected table is signaled, and the corresponding combination of syntax related to the intra prediction mode is used for a coded block.

[0085] In some embodiments, a first flag is written into the bitstream at the block level to indicate whether multiple syntaxes related to intra mode coding are jointly signaled for one or more components. When the value signaled by the first flag indicates that multiple syntaxes related to intra mode coding are jointly signaled, subsequent indicators are used to indicate a specific combination of entries in the look-up table or list. When the value signaled by the first flag indicates that multiple syntaxes related to intra mode coding are not jointly signaled, the syntaxes related to intra mode coding are signaled separately.

[0086] In some embodiments, a first flag may be written into the bitstream at the block level to indicate whether multiple syntaxes related to intra mode coding are jointly signaled for all components. When the value signaled by the first flag indicates that multiple syntaxes related to intra mode coding are not jointly signaled for all components, a second flag is signaled to indicate whether multiple syntaxes related to intra mode coding are jointly signaled for a subset of color components (e.g., two chrominance components Cb, Cr).

[0087] In some embodiments, at the sequence / frame / picture / slice / superblock level, a high level syntax (HLS) is written into the bitstream to indicate whether joint signaling of intra mode related syntaxes is used. In some embodiments, a sequence level indicator is first written into the bitstream to indicate whether joint signaling is allowed for each sequence. If so, at the frame / picture / slice level, another high level syntax is signaled to further indicate the use of the joint signaling method described herein.

[0088] Figure 5A It is a flowchart showing a video decoding method 500 according to some embodiments. The method 500 may be executed at a computing system (e.g., the server system 112, the source device 102, or the electronic device 120), which has a control circuit and a memory storing instructions for execution by the control circuit. In some embodiments, the method 500 is executed by executing the instructions stored in the memory of the computing system (e.g., the memory 314).

[0089] The system receives (502) a video bitstream (e.g., an encoded video sequence), which includes a plurality of blocks. The system identifies (504) two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks by parsing a single syntax from the video bitstream. The system reconstructs (506) the first block using the intra prediction mode and the two or more parameters. In this way, a plurality of syntaxes related to intra mode coding can be jointly used for one or more components of one encoded block by signaling representation.

[0090] In some embodiments, a lookup table (or list, or set, or group) is constructed based on the encoded mode information of adjacent blocks and / or non - adjacent blocks. Each entry in the lookup table includes a plurality of syntaxes for performing intra prediction, and an index in the lookup table can be written into the bitstream. In some embodiments, the syntaxes related to the intra prediction mode of adjacent encoded blocks and / or non - adjacent encoded blocks may be inserted into the lookup table. Examples of the syntaxes related to the intra prediction mode include, but are not limited to: intra prediction mode index, reference cue index, matrix - based intra prediction mode index, cross - component intra prediction mode.

[0091] In some embodiments, a syntax combination related to the intra prediction mode can be derived based on the encoded information of adjacent encoded blocks and / or non - adjacent encoded blocks. In some embodiments, the intra prediction mode of a left - adjacent block and the reference cue index of an upper - adjacent block are combined to generate an entry in the lookup table. In some embodiments, the intra prediction mode of an adjacent block is an angular intra prediction mode, and the angle of the mode is slightly adjusted by adding an offset value (such as +N or - N), but the reference cue index remains unchanged, and then it is added to the lookup table. N is a positive integer, such as 1 or 2 or 4.

[0092] In some embodiments, each entry in the lookup table includes an intra prediction mode and a reference cue index, and a syntax can be written into the bitstream to indicate whether any combination of the intra prediction mode and the reference cue index in the lookup table is used. For example, if so, an index in the lookup table can be further written into the bitstream to indicate which combination is selected. In this example, if not, the intra prediction mode and / or the reference cue index are written into the bitstream separately.

[0093] In some embodiments, the intra prediction mode only includes the intra prediction of the luminance component. In some embodiments, the intra prediction mode herein includes a luminance intra prediction mode and a chrominance intra prediction mode. In some embodiments, this only applies to the case where the luminance block and the chrominance block share the same block partition and / or transform partition. In some embodiments, a cross-component chrominance prediction mode is used for one entry. If this entry is selected, additional syntax can be further signaled to indicate which syntax is used to encode the selected block.

[0094] In some embodiments, each entry in the lookup table includes the intra prediction modes of the luminance component and the chrominance component.

[0095] In some embodiments, multiple lookup tables are constructed based on the encoded mode information of adjacent blocks and / or non-adjacent blocks, and the meanings of each entry in different lookup tables may be different. For example, the entries in the first lookup table correspond to a combination of N1 syntaxes related to the intra mode, the entries in the second lookup table correspond to a combination of N2 syntaxes related to the intra mode, and the entries in the third lookup table correspond to a combination of N3 syntaxes related to the intra mode. N1, N2, and N3 are all positive integers. For example, N1 can be 3, N2 can be 2, and N3 can be 1.

[0096] In some embodiments, the selection of the lookup table is first written into the bitstream, and if a lookup table is selected, an index of the corresponding table is further signaled, and the corresponding combination of the syntax related to the intra prediction mode is used for an encoded block.

[0097] In some embodiments, a first flag can be written into the bitstream at the block level to indicate whether multiple syntaxes related to the intra mode encoding are signaled jointly for one or more components.

[0098] In some embodiments, when the value signaled by the first flag indicates that multiple syntaxes related to intra mode coding are jointly signaled, the above method is applied. Otherwise, when the value signaled by the first flag indicates that multiple syntaxes related to intra mode coding are not jointly signaled, the syntaxes related to intra mode coding are signaled separately. In some embodiments, the first flag can be written into the bitstream at the block level to indicate whether multiple syntaxes related to intra mode coding are jointly signaled for all components. If the value signaled by the first flag indicates that multiple syntaxes related to intra mode coding are not jointly signaled for all components, a second flag can be signaled to indicate whether multiple syntaxes related to intra mode coding are jointly signaled for a subset of color components (e.g., two chrominance components).

[0099] In some embodiments, a high-level syntax can be written into the bitstream at the sequence, frame, picture, slice, or superblock level to indicate whether joint signaling of intra mode related syntaxes is used. In some embodiments, a sequence-level indicator is first written into the bitstream to indicate whether joint syntax signaling is allowed for each sequence. If so, another high-level syntax is signaled at the frame, picture, or slice level to further indicate the use of the joint syntax signaling method described herein.

[0100] Figure 5B FIG. 7 is a flowchart showing a video coding method 550 according to some embodiments. Method 550 can be executed at a computing system (e.g., server system 112, source device 102, or electronic device 120) having a control circuit and a memory storing instructions for execution by the control circuit. In some embodiments, method 550 is executed by executing instructions stored in the memory (e.g., memory 314) of the computing system. In some embodiments, method 550 is executed by the same system as method 500 described above.

[0101] The system receives (552) video data (e.g., a source video sequence) that includes a plurality of blocks (e.g., corresponding to a group of pictures). When two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be jointly signaled, the system writes (554) a single syntax to a video bitstream to indicate the two or more parameters. When two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be separately signaled, the system writes (556) separate syntaxes to the video bitstream to indicate each of the two or more parameters. The system encodes (558) the first block using the intra prediction mode and the two or more parameters. As described above, the encoding process may be mirrored with the decoding process (e.g., the joint signaling technique described herein). For the sake of brevity, these details are not repeated here.

[0102] Although Figure 5A and Figure 5B a number of logical stages are shown in a particular order, stages that are not order-dependent may be reordered and other stages may be combined or split. For one of ordinary skill in the art, some reorderings or other groupings not specifically recited will be apparent, and thus the orderings and groupings presented in this application are not exhaustive. Additionally, it should be recognized that the stages may be implemented by hardware, firmware, software, or any combination thereof.

[0103] Turning now to some example embodiments.

[0104] (A1) In one aspect, some embodiments include a video decoding method (e.g., method 500). In some embodiments, the method is performed at a computing system (e.g., server system 112) that has a memory and control circuitry. In some embodiments, the method is performed at an encoding module (e.g., encoding module 320). In some embodiments, the method is performed at a source coding component (e.g., source encoder 202), an encoding engine (e.g., encoding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes: (i) receiving a video bitstream (e.g., an encoded video sequence) that includes a plurality of blocks; (ii) identifying two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks by parsing a single syntax from the video bitstream; and, (iii) reconstructing the first block using the intra prediction mode and the two or more parameters. For example, multiple syntaxes related to intra mode coding may be jointly signaled for one or more components of an encoded block.

[0105] (A2) In some embodiments of A1, the two or more parameters include a reference cue index and an index of the intra prediction mode.

[0106] (A3)In some embodiments of A2, the intra prediction mode is used for the luminance component of the first block. For example, the intra prediction mode only includes the intra prediction of the luminance component. In some embodiments, the intra prediction mode includes a luminance intra prediction mode and a chrominance intra prediction mode. In some embodiments, each entry in the lookup table includes the intra prediction modes of the luminance component and the chrominance component.

[0107] (A4)In some embodiments of A2, when the luminance component and the chrominance component of the first block share one or more of the same block partitions and the same transform partitions, the intra prediction mode corresponds to the luminance component and the chrominance component; and when the luminance component and the chrominance component do not share the same block partition or the same transform partition, the intra prediction mode only corresponds to the luminance component. For example, the shared intra prediction mode information is applied only when the luminance block and the chrominance block share the same block partition and / or transform partition.

[0108] (A5)In some embodiments of any one of A1 to A4, two or more parameters are identified using a single syntax value and a lookup table. For example, a lookup table (or list, set, or group) is constructed based on the encoded mode information of adjacent blocks and / or non - adjacent blocks, each entry in the lookup table includes multiple syntaxes for performing intra prediction, and the index of the lookup table is written into the bitstream.

[0109] (A6)In some embodiments of A5, the method further includes: filling the lookup table with the intra prediction mode indicators of one or more previously decoded blocks. For example, the syntaxes related to the intra prediction mode of adjacent encoded blocks and / or non - adjacent encoded blocks can be inserted into the lookup table. Examples of the syntaxes related to the intra prediction mode include: intra prediction mode index, reference cue index, matrix - based intra prediction mode index, and cross - component intra prediction mode.

[0110] (A7)In some embodiments of A5 or A6, the method further includes: generating entries for the lookup table based on the encoded information of one or more previously decoded blocks. For example, the syntax combinations related to the intra prediction mode can be derived based on the encoded information of adjacent encoded blocks and / or non - adjacent encoded blocks.

[0111] (A8)In some embodiments of any one of A5 to A7, the intra prediction mode of the left - adjacent block is combined with the reference cue index of the upper - adjacent block to generate an entry for the lookup table. For example, the intra prediction mode of the left - adjacent block and the reference cue index of the upper - adjacent block are combined to generate an entry in the lookup table.

[0112] (A9)In some embodiments of any one of A5 to A8, the angular intra prediction mode with an offset value of the second block is combined with the reference cue index of the third block to generate an entry for the look-up table. For example, the intra prediction mode of adjacent blocks is the angular intra prediction mode, and the angle is slightly adjusted by adding an offset value (such as +N or -N), while keeping the reference cue index unchanged, and then it is added to the look-up table. In this example, N is a positive integer, such as 1 or 2 or 4.

[0113] (A10)In some embodiments of any one of A5 to A9, the entries in the look-up table correspond to cross-component chrominance prediction modes; and when this entry in the look-up table is selected for the first block, a signaling is used to indicate an additional indicator in the video bitstream to indicate which syntax is used to encode the first block. For example, when the cross-component chrominance prediction mode is used for an entry and this entry is selected, a signaling can be further used to indicate an additional syntax to indicate which syntax is used to encode the selected block.

[0114] (A11)In some embodiments of any one of A1 to A10: (i) when an indicator in the video bitstream indicates that two or more parameters are signaled jointly, two or more parameters are identified by parsing a single syntax; and (ii) the method further includes: when an indicator in the video bitstream indicates that two or more parameters are signaled separately, two or more parameters are identified based on multiple syntaxes in the video bitstream. For example, each entry in the look-up table contains an intra prediction mode and a reference cue index, and a single syntax can be written into the bitstream to indicate whether any combination of the intra prediction mode and the reference cue index in the look-up table is used. If so, an index of the look-up table can be further written into the bitstream to indicate which combination is selected. If not, the intra prediction mode and the reference cue index are written into the bitstream separately. As another example, a flag can be written into the bitstream (e.g., at the block level) to indicate whether multiple syntaxes related to intra-mode encoding are signaled jointly for one or more components. For example, when the value signaled by the flag indicates that multiple syntaxes related to intra-mode encoding are signaled jointly, the above method is applied. Otherwise, when the value signaled by the flag indicates that multiple syntaxes related to intra-mode encoding are not signaled jointly, the syntaxes related to intra-mode encoding are signaled separately.

[0115] (A12)In some embodiments of A11, on the high-level syntax of the video bitstream, a signaling indicator is used. For example, on the level of sequence, frame, picture, slice, or superblock, the high-level syntax can be written into the bitstream to indicate whether to use the joint signaling of the syntax related to the intra mode. As another example, a sequence-level indicator is first written into the bitstream to indicate whether joint signaling is allowed for a specific sequence. If so, another high-level syntax can be signaled at the frame, picture, or slice level to further indicate the use of joint signaling for these components.

[0116] (A13)In some embodiments of any one of A1 to A12: (i) the method further includes: determining, based on a first indicator in the video bitstream, whether to jointly use signaling to represent the intra mode parameters for all components; (ii) when the first indicator indicates jointly using signaling to represent the intra mode parameters for all components, identifying two or more parameters by parsing a single syntax; (iii) when the first indicator indicates not jointly using signaling to represent the intra mode parameters for all components, parsing a second indicator that indicates whether to jointly use signaling to represent the intra mode parameters for the current component; and, (iv) when the second indicator indicates jointly using signaling to represent the intra mode parameters for the current component, identifying two or more parameters by parsing a single syntax. For example, a first flag can be written into the bitstream at the block level to indicate whether to jointly use signaling to represent multiple syntaxes related to intra mode coding for all components. If the value signaled by the first flag indicates not jointly using signaling to represent the syntax related to intra mode coding for all components, a second flag can be signaled to indicate whether to jointly use signaling to represent multiple syntaxes related to intra mode coding for a subset of color components (e.g., two chrominance components). In some embodiments, when the first indicator indicates not jointly using signaling to represent the intra mode parameters for all components, the second indicator will not be signaled / parsed. In some embodiments, when the second indicator indicates not jointly using signaling to represent the intra mode parameters for the current component, two or more parameters are identified by parsing multiple syntaxes.

[0117] (A14)In some embodiments of any one of A1 to A13, the method further includes: selecting a first lookup table to be used with the first block from a set of lookup tables, wherein two or more parameters are identified using a single syntax value and the first lookup table. For example, multiple lookup tables may be constructed based on the coded pattern information of adjacent blocks and / or non-adjacent blocks, and the meaning of each entry in different lookup tables may be different. For example, an entry in the first lookup table indicates a combination of N1 syntaxes related to intra modes, an entry in the second lookup table indicates a combination of N2 syntaxes related to intra modes, and an entry in the third lookup table indicates a combination of N3 syntaxes related to intra modes. N1, N2, and N3 are all positive integers. In one example, N1 is 3, N2 is 2, and N3 is 1. In some embodiments, the coded information of one or more previously decoded blocks (e.g., adjacent blocks) is used to populate the set of lookup tables.

[0118] (A15)In some embodiments of A14, the first lookup table is selected based on an indicator in the video bitstream. For example, the selection of the lookup table is first written into the bitstream, and if a lookup table is selected, the index in the corresponding table is further signaled, and the corresponding combination of syntaxes related to the intra prediction mode is used for an encoded block. As an example, if multiple adjacent blocks are encoded using a smooth mode, the first block may also be encoded using the smooth mode, and the lookup table corresponding to the smooth mode is selected. In this way, the size of each lookup table can be reduced.

[0119] (B1)In another aspect, some embodiments include a video encoding method (e.g., method 550). In some embodiments, the method is executed at a computing system (e.g., server system 112) having a memory and control circuitry. In some embodiments, the method is executed at an encoding module (e.g., encoding module 320). The method includes: (i) receiving video data (e.g., a source video sequence) including a plurality of blocks; (ii) writing a single syntax into the video bitstream to indicate two or more parameters when two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be jointly signaled; (iii) writing separate syntaxes into the video bitstream to indicate each of the two or more parameters when the two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be separately signaled; and (iv) encoding the first block using the intra prediction mode and the two or more parameters.

[0120] (B2)In some embodiments of B1, the two or more parameters include a reference cue index and an index of the intra prediction mode.

[0121] (B3) In some embodiments of B1 or B2, one or more instruction sets further include: instructions for writing an indicator to a video bitstream, the indicator indicating whether two or more parameters are jointly signaled.

[0122] (B4) In some embodiments of B1 to B3, one or more instruction sets further include instructions for: (i) writing a first indicator to a video bitstream, the first indicator indicating whether intra mode parameters are jointly signaled for all components in a component set; and, (ii) when the first indicator indicates that intra mode parameters are not jointly signaled for all components in the component set, writing a second indicator to the video bitstream, the second indicator indicating whether intra mode parameters are jointly signaled for a particular component in the component set.

[0123] (C1) In another aspect, some embodiments include a method for processing visual media data. In some embodiments, the method is performed at a computing system (e.g., server system 112) that has a memory and control circuitry. In some embodiments, the method is performed at an encoding module (e.g., encoding module 320). The method includes: (i) obtaining a source video sequence that includes a plurality of frames; and, (ii) performing a conversion between the source video sequence and a video bitstream of visual media data according to formatting rules. The video bitstream includes a plurality of blocks and syntax elements, the plurality of blocks including a first block, and the syntax elements indicating two or more parameters associated with an intra prediction mode of the first block. The formatting rules specify that the first block will be reconstructed using the intra prediction mode and the two or more parameters.

[0124] In another aspect, some embodiments include a computing system (e.g., server system 112) that includes control circuitry (e.g., control circuitry 302) and a memory (e.g., memory 314) coupled to the control circuitry, the memory storing one or more instruction sets configured to be executed by the control circuitry, the one or more instruction sets including instructions for performing any of the methods described herein (e.g., A1 - A15, B1 - B4, and C1 above).

[0125] In yet another aspect, some embodiments include a non - volatile computer - readable storage medium storing one or more instruction sets executed by control circuitry of a computing system, the one or more instruction sets including instructions for performing any of the methods described herein (e.g., A1 - A15, B1 - B4, and C1 above).

[0126] Unless otherwise specified, any syntax element (e.g., indicator) described herein may be high-level syntax (HLS). As used herein, at levels above the block level, signaling is used to represent HLS. For example, HLS may correspond to the sequence level, frame level, slice level, or tile level. As another example, signaling may be used to represent HLS elements in a video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptation parameter set (APS), slice header, picture header, tile header, and / or CTU header.

[0127] It should be understood that although the terms "first", "second", etc. may be used in this application to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. The terms used in this application are only used to describe specific embodiments and are not intended to limit the claims. As used in the description of the embodiments and the appended claims, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used in this application refers to and encompasses any and all possible combinations of one or more of the associated listed items. It should also be understood that when used in this specification, the terms "comprises" and / or "comprising" specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0128] As used herein, depending on the context, the term "if" may be interpreted to mean "when...", "after...", "in response to determining", "in accordance with determining", or "in response to detecting" that the stated prerequisite is true. Similarly, depending on the context, the phrase "if it is determined [that the stated prerequisite is true]" or "if [the stated prerequisite is true]" or "when [the stated prerequisite is true]" may be interpreted to mean "after determining", "in response to determining", "in accordance with determining", "after detecting", or "in response to detecting" that the stated prerequisite is true.

[0129] For purposes of explanation, the foregoing description has been made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of operation and the practical application, thereby enabling others skilled in the art to implement.

Claims

1. A video decoding method, which is executed at a computing system having a memory and one or more processors, characterized in that, The method includes: Receiving a video bitstream, the video bitstream including a plurality of blocks; Identifying two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks by parsing a single syntax from the video bitstream; and Reconstructing the first block using the intra prediction mode and the two or more parameters.

2. The method according to claim 1, wherein The two or more parameters include a reference cue index and an index of the intra prediction mode.

3. The method according to claim 2, wherein The intra prediction mode is for a luminance component of the first block.

4. The method according to claim 2, wherein when one or more of a block partition and a transform partition of the luminance component and a chrominance component of the first block are shared, the intra prediction mode corresponds to the luminance component and the chrominance component; and when the luminance component and the chrominance component do not share the same block partition or the same transform partition, the intra prediction mode corresponds only to the luminance component.

5. The method according to claim 1, wherein Identifying the two or more parameters using a value of the single syntax and a lookup table.

6. The method according to claim 5, characterized in that, The method further includes: populating the lookup table using an intra prediction mode indicator of one or more previously decoded blocks.

7. The method according to claim 5, characterized in that, The method further includes: generating entries of the lookup table based on encoded information of one or more previously decoded blocks.

8. The method according to claim 7, wherein An intra prediction mode of a left adjacent block and a reference cue index of an upper adjacent block are combined to generate the entry of the lookup table.

9. The method according to claim 7, wherein An angular intra prediction mode with an offset value of a second block and a reference cue index of a third block are combined to generate the entry of the lookup table.

10. The method according to claim 5, wherein entries in the lookup table correspond to cross-component chrominance prediction modes; and when an entry in the lookup table is selected for the first block, an additional indicator in the video bitstream is signaled to indicate a syntax used for encoding the first block.

11. The method according to claim 1, wherein When an indicator in the video bitstream indicates that the two or more parameters are signaled jointly, the two or more parameters are identified by parsing the single syntax; and The method further includes: when the indicator in the video bitstream indicates that the two or more parameters are signaled separately, the two or more parameters are identified based on a plurality of syntaxes in the video bitstream.

12. The method according to claim 11, wherein The indicator is signaled in a high-level syntax of the video bitstream.

13. The method according to claim 1, wherein The method further includes: determining whether intra mode parameters for all components are signaled jointly based on a first indicator in the video bitstream; when the first indicator indicates that the intra mode parameters for all components are signaled jointly, the two or more parameters are identified by parsing the single syntax; when the first indicator indicates that the intra mode parameters for all components are not signaled jointly, a second indicator is parsed, the second indicator indicating whether intra mode parameters for the current component are signaled jointly; and When the second indicator indicates that signaling is used jointly to represent the intra mode parameters for the current component, the two or more parameters are identified by parsing the single syntax.

14. The method according to claim 1, wherein The method further includes: selecting a first look-up table to be used with the first block from a set of look-up tables, wherein the two or more parameters are identified using the value of the single syntax and the first look-up table; wherein the first look-up table is selected based on an indicator in the video bitstream.

15. A video encoding method, the method being performed at a computing system having a memory and one or more processors, characterized in that, The method includes: receiving video data, the video data including a plurality of blocks; when two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be signaled jointly, writing a single syntax to the video bitstream to indicate the two or more parameters; when two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be signaled separately, writing separate syntaxes to the video bitstream to indicate each of the two or more parameters; and encoding the first block using the intra prediction mode and the two or more parameters.

16. A computing system, characterized in that, Comprising: a control circuit; a memory; and one or more instruction sets stored in the memory and configured to be executed by the control circuit, the one or more instruction sets including instructions for: receiving a video bitstream, the video bitstream including a plurality of blocks; identifying two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks by parsing a single syntax from the video bitstream; and reconstructing the first block using the intra prediction mode and the two or more parameters.

17. A computing system, characterized in that, Comprising: a control circuit; a memory; and one or more instruction sets stored in the memory and configured to be executed by the control circuit, the one or more instruction sets including instructions for: receiving video data, the video data including a plurality of blocks; when two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be signaled jointly, writing a single syntax to the video bitstream to indicate the two or more parameters; when two or more parameters associated with an intra prediction mode for a first block among the plurality of blocks are to be signaled separately, writing separate syntaxes to the video bitstream to indicate each of the two or more parameters; and encoding the first block using the intra prediction mode and the two or more parameters.

18. The computing system according to claim 16, wherein The one or more instruction sets further include: instructions for writing an indicator to the video bitstream, the indicator indicating whether the two or more parameters are signaled jointly.

19. The computing system according to claim 16, wherein The one or more instruction sets further include instructions for: writing a first indicator to the video bitstream, the first indicator indicating whether intra mode parameters are signaled jointly for all components in a set of components; and When the first indicator indicates that signaling is not used jointly for all components in the component set to represent the intra mode parameter, a second indicator is written into the video bitstream, and the second indicator indicates whether signaling is used jointly for a specific component in the component set to represent the intra mode parameter.

20. A non-volatile computer-readable storage medium, characterized in that, There is stored one or more instruction sets, and the one or more instruction sets are configured to be executed by a computing device having a control circuit and a memory, and the one or more instruction sets include instructions for the following operations: Obtain a source video sequence, where the source video sequence includes a plurality of frames; And According to format rules, perform conversion between the source video sequence and the video bitstream of visual media data, wherein the video bitstream includes: A plurality of blocks, the plurality of blocks including a first block, and Syntax elements, the syntax elements indicating two or more parameters associated with the intra prediction mode of the first block; and wherein the format rules specify that the first block will be reconstructed using the intra prediction mode and the two or more parameters.

Images