Video encoding and decoding method and apparatus, and storage medium

Abstract

A video encoding and decoding method, apparatus, and medium are provided. An example method of video decoding includes receiving a video bitstream including a plurality of frames, the plurality of frames including a current frame; and identifying a primary reference frame for context initialization of the current frame. The method further includes identifying a reference frame of the current frame based on the primary reference frame, and reconstructing the current frame by applying a cross component sample offset (CCSO) mode using the reference frame.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 725,507, filed November 26, 2024, entitled “Parameters Inheritance for Sample Offsets”, and U.S. Patent Application No. 19 / 366,484, filed October 22, 2025, entitled “Parameters Inheritance for Sample Offsets”, which are incorporated herein by reference in their entirety. Technical Field

[0002] The disclosed embodiments generally relate to video encoding and decoding, including but not limited to systems and methods for loop filtering (e.g., cross-component offset filtering) of video data. Background Technology

[0003] Various electronic devices support digital video, such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. Electronic devices send and receive, or otherwise transmit, digital video data via communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited storage resources of storage devices, video encoding is used to compress video data according to one or more video encoding standards before transmission or storage. Video encoding can be performed by hardware and / or software on electronic / client devices, or by servers providing cloud services.

[0004] Video coding typically uses prediction methods that leverage the inherent redundancy in video data (e.g., inter-frame prediction, intra-frame prediction, etc.). Video coding aims to compress video data into a form using a lower bit rate while avoiding or minimizing video quality degradation. Several video codec standards have been developed. For example, High-Efficiency Video Coding (HEVC) (HEVC / H.265) is a video compression standard designed as part of the MPEG-H project. ITU-T and ISO / IEC released the HEVC / H.265 standard in 2013 (Revision 1), 2014 (Revision 2), 2015 (Revision 3), and 2016 (Revision 4), respectively. Versatile Video Coding (VVC) (also known as VVC / H.266) is a video compression standard designed as a successor to HEVC. ITU-T and ISO / IEC released the VVC / H.266 standard in 2020 (Revision 1) and 2022 (Revision 2), respectively. AOMedia Video 1 (AV1) is an open video coding format designed to replace HEVC. On January 8, 2019, a validated version 1.0.0 of the specification with Errata 1 was released. Summary of the Invention

[0005] As mentioned above, encoding (compression) reduces bandwidth and / or storage space requirements. Both lossless and lossy compression can be used, as described in detail later. Lossless compression refers to a technique that can reconstruct an exact copy of the original signal from the compressed original signal via a decoding process. Lossy compression refers to an encoding / decoding process in which the original video information is not fully preserved during encoding and cannot be fully recovered during decoding. When using lossy compression, the reconstructed signal may not be exactly the same as the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal is useful for the intended application. The tolerable level of distortion depends on the application. For example, users of some consumer video streaming applications may tolerate higher distortion than users of film or television broadcasting applications. The compression ratio achievable by a particular encoding algorithm can be selected or adjusted to reflect various distortion tolerances: higher tolerable distortion generally allows for encoding algorithms that produce higher loss and higher compression ratios.

[0006] This disclosure describes methods, systems, and non-transitory computer-readable storage media for applying loop filtering in video (image) compression. In-loop filtering techniques are applied to adjust reconstructed image samples to further reduce reconstruction errors. Cross-component sample offset (CCSO) filtering can be implemented to derive an offset value added to the current sample of the second color component by applying co-located reconstructed samples of the first color component and associated adjacent reconstructed samples, thereby adjusting the reconstructed value of the current sample. When frame-level or block-level cross-component sample offset syntax reuses syntax from a reference frame, a reference frame index can be implicitly derived or explicitly signaled. The signaling of the reference frame index can depend on the master reference frame used for frame context initialization or the derived master reference frame. By selectively signaling the reference frame index based on the context-based master reference frame, signaling overhead can be reduced and coding efficiency can be improved. In some methods, when signaling the reference frame index, signaling can be performed for a reduced set and / or multiple color components to reduce / minimize the signaling overhead of the encoded video bitstream.

[0007] According to some embodiments, a video decoding method includes: (i) receiving a video bitstream comprising a plurality of frames (including the current frame); (ii) identifying a master reference frame for context initialization of the current frame; (iii) identifying a reference frame of the current frame based on the master reference frame; and (iv) reconstructing the current frame using the reference frame.

[0008] According to some embodiments, a video encoding method includes: (i) receiving video data comprising a plurality of frames (including the current frame); (ii) identifying a master reference frame for context initialization of the current frame; (iii) identifying a reference frame of the current frame based on the master reference frame; and (iv) encoding the current frame using the reference frame.

[0009] According to some embodiments, a computing system, such as a streaming system, server system, personal computer system, or other electronic device, is provided. The computing system includes control circuitry and a memory storing one or more sets of instructions. The set of instructions includes instructions for performing any of the methods described herein. In some embodiments, the computing system includes an encoder component and a decoder component (e.g., a transcoder). According to some embodiments, a non-transitory computer-readable storage medium is provided that stores one or more sets of instructions executable by the computing system. The set of instructions includes instructions for performing any of the methods described herein.

[0010] Therefore, apparatus and systems having methods for encoding and decoding video are disclosed. These methods, apparatus, and systems can complement or replace conventional methods, apparatus, and systems for video encoding / decoding.

[0011] Not all features and advantages described in the specification are necessarily included. In particular, some additional features and advantages will be apparent to those skilled in the art in light of the accompanying drawings, specification, and claims provided in this disclosure. Furthermore, it should be noted that the language used in the specification has been chosen primarily for readability and guidance purposes, and not for describing or limiting the subject matter described herein. Attached Figure Description

[0012] To gain a more detailed understanding of this disclosure, a more specific description can be made by referring to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the accompanying drawings merely illustrate relevant features of this disclosure and are therefore not necessarily to be considered limiting; it will be understood by those skilled in the art, upon reading this disclosure, that the specification may contain other valid features.

[0013] Figure 1 This is a block diagram illustrating an exemplary communication system according to some embodiments; Figure 2A This is a block diagram illustrating example elements of an encoder assembly according to some embodiments; Figure 2B This is a block diagram illustrating example elements of a decoder component according to some embodiments; Figure 3 This is a block diagram illustrating an exemplary server system according to some embodiments; Figure 4A The calculation process of the prediction block according to some embodiments is illustrated; Figure 4B The calculation process of residual blocks according to some embodiments is illustrated; Figure 4C The calculation process for reconstructed blocks according to some embodiments is illustrated; Figure 5A An exemplary in-loop filtering stage according to some embodiments is shown; Figure 5B An exemplary in-loop filtering stage according to some embodiments is shown; Figure 5C An exemplary in-loop filtering stage according to some embodiments is shown; Figure 6A An example video decoding process according to some embodiments is shown; Figure 6B An example video encoding process according to some embodiments is shown.

[0014] By convention, the various features shown in the accompanying drawings are not necessarily drawn to scale, and the same reference numerals may be used to denote the same features throughout the specification and the accompanying drawings. Detailed Implementation

[0015] This disclosure describes methods, systems, and non-transitory computer-readable storage media for applying loop filtering in video (image) compression. For example, a master reference frame for context initialization of the current frame can be identified. A reference frame for the current frame can be identified based on the master reference frame. The current frame can be reconstructed by applying a Cross-Component Sample Offset (CCSO) mode using the reference frame. Using the master reference frame to identify the reference frame for the current frame saves signaling overhead because, in some cases, the index of the signal transmission reference frame is not required.

[0016] This disclosure primarily describes techniques for CCSO filtering in video coding, specifically enabling the reuse of syntax from reference frames at the frame or block level. Mechanisms are described for implicitly deriving or explicitly signaling reference frame indices based on the master reference frame used for context initialization, and for selectively signaling indices for a reduced set of reference frames. Additionally, techniques are described for reusing CCSO syntax flags across multiple color channels (such as Y, U, and V), or for allowing separate signaling for the Y channel while U and V share flags. These methods yield significant benefits, including reduced signaling overhead in the video bitstream, improved coding efficiency, and enhanced flexibility in syntax management. By leveraging context initialization and optimized reference frame selection, unnecessary signaling can be reduced / minimized, resulting in more efficient video compression, which is particularly advantageous for bandwidth-constrained or storage-constrained applications.

[0017] Figure 1 This is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to 120-m) which are communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is a streaming system, such as for applications supporting video, such as video conferencing applications, digital television applications, and media storage and / or distribution applications.

[0018] Source device 102 includes a video source 104 (e.g., a camera assembly or media storage) and an encoder assembly 106. In some embodiments, the video source 104 is a digital camera (e.g., configured to create an uncompressed video sample stream). The encoder assembly 106 generates one or more encoded video streams from the video stream. The video stream from the video source 104 has a higher data volume compared to the encoded video stream 108 generated by the encoder assembly 106. Because the encoded video stream 108 has a lower data volume (less data) compared to the video stream from the video source, it requires less bandwidth for transmission and less storage space for storage compared to the video stream from the video source 104. In some embodiments, source device 102 does not include the encoder assembly 106 (e.g., configured to transmit uncompressed video to network 110).

[0019] One or more networks 110 represent any number of networks that transmit information between source device 102, server system 112, and / or electronic device 120, including, for example, wired (connected) and / or wireless communication networks. One or more networks 110 may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet.

[0020] One or more networks 110 include a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is a streaming server or includes a streaming server (e.g., configured to store and / or distribute video content, such as an encoded video stream from source device 102). The server system 112 includes a codec component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the codec component 114 includes an encoder component and / or a decoder component. In various embodiments, the codec component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the codec component 114 is configured to decode the encoded video stream 108 and re-encode the video data using different encoding standards and / or methods to generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings based on the encoded video stream 108. In some embodiments, the server system 112 serves as a Media-Aware Network Element (MANE). For example, server system 112 can be configured to trim encoded video stream 108 to customize potentially different streams for one or more electronic devices 120. In some embodiments, a MANE is provided separately from server system 112.

[0021] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode encoded video data 116 to generate an output video stream that can be displayed on a display or other type of presentation device. In some embodiments, one or more electronic devices 120 do not include a display component (e.g., communicatively coupled to an external display device and / or includes media storage). In some embodiments, electronic device 120 is a streaming client. In some embodiments, electronic device 120 is configured to access server system 112 to obtain encoded video data 116.

[0022] The source device and / or multiple electronic devices 120 are sometimes referred to as “terminal devices” or “user devices”. In some embodiments, the source device 102 and / or one or more electronic devices 120 are instances of server systems, personal computers, portable devices (e.g., smartphones, tablets, or laptops), wearable devices, video conferencing equipment, and / or other types of electronic devices.

[0023] In an example operation of communication system 100, source device 102 sends an encoded video stream 108 to server system 112. For example, source device 102 may encode a stream of images captured by the source device. Server system 112 receives the encoded video stream 108 and may decode and / or encode the encoded video stream 108 using codec component 114. For example, server system 112 may apply encoding to the video data that is more suitable for network transmission and / or storage. Server system 112 may send encoded video data 116 (e.g., one or more encoded video streams) to one or more electronic devices 120. Each electronic device 120 may decode the encoded video data 116 and optionally display video images.

[0024] Figure 2AThis is a block diagram illustrating example elements of an encoder assembly 106 according to some embodiments. The encoder assembly 106 receives video data (a source video sequence) from a video source 104. In some embodiments, the encoder assembly includes a receiver (e.g., transceiver) assembly configured to receive the source video sequence. In some embodiments, the encoder assembly 106 receives the video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder assembly 106). The video source 104 may provide the source video sequence as a digital video sample stream, which may be any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 YCrCB or RGB), and any suitable sampling structure (e.g., YCrCb 4:2:0 or YCrCb 4:4:4). In some embodiments, the video source 104 is a storage device storing previously acquired / prepared video. In some embodiments, the video source 104 is a camera that acquires local image information as a video sequence. The video data may be provided as multiple individual pictures, which are given motion when viewed sequentially. An image itself can be constructed as a spatial pixel array, where each pixel may include one or more samples, depending on the sampling structure, color space, etc. Those skilled in the art can readily understand the relationship between pixels and samples.

[0025] Encoder component 106 is configured to encode and / or compress images of a source video sequence into an encoded video sequence 216 in real time or under any other time constraints required by the application. In some embodiments, encoder component 106 is configured to perform a conversion between the source video sequence and a bitstream of visual media data (e.g., a video bitstream). Implementing an appropriate encoding rate is a function of controller 204. In some embodiments, controller 204 controls and is functionally coupled to other functional units described below. Parameters set by controller 204 may include rate control-related parameters (e.g., image skipping, quantizer and / or rate-distortion optimization technique λ value), image size, group of images (GOP) layout, maximum motion vector search range, etc. Other functions of controller 204 will be readily understood by those skilled in the art, as these functions may involve encoder component 106 optimized for a particular system design.

[0026] In some embodiments, encoder component 106 is configured to operate within an encoding loop. In a simplified example, the encoding loop includes a source encoder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder 210. Decoder 210 reconstructs the symbols to create sample data in a manner similar to how a (remote) decoder creates sample data (when compression between the symbols and the encoded video stream is lossless). The reconstructed sample stream (sample data) is input to reference image memory 208. Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of reference image memory 208 also correspond bit-accurately between the local encoder and the remote encoder. Thus, the reference image samples decoded by the encoder's prediction portion are the same as the sample values ​​decoded by the decoder during prediction. This principle of reference image synchronization (and the drift that occurs, for example, when synchronization cannot be maintained due to channel errors) is known to those skilled in the art.

[0027] The operation of decoder 210 can be combined with, for example, the following: Figure 2B The remote decoder is the same as that of decoder component 122 described in detail. However, a brief additional reference is provided. Figure 2B When symbols are available and can be losslessly encoded / decoded into an encoded video sequence by entropy encoder 214 and parser 254, the entropy decoding portion of decoder component 122, including buffer memory 252 and parser 254, may not be fully implemented in local decoder 210.

[0028] Besides parsing / entropy decoding, the decoder techniques described in this paper can exist in the corresponding encoders with essentially the same functionalities. Therefore, the subject matter focuses on decoder operations. The description of encoder techniques can be simplified because encoder techniques are inverses of decoder techniques.

[0029] As part of its operation, the source encoder 202 can perform motion-compensated predictive coding, referencing one or more previously encoded frames in the video sequence designated as reference frames, to predictively encode the input frame. In this manner, the encoding engine 212 encodes the differences between pixel blocks in the input frame and pixel blocks in the reference frame, which can be selected as the prediction reference for the input frame. The controller 204 can manage the encoding operations of the source encoder 202, including, for example, setting parameters and subgroup parameters for encoding the video data.

[0030] Decoder 210 decodes encoded video data of frames that can be designated as reference frames, based on symbols created by source encoder 202. The operation of encoding engine 212 can advantageously be a lossy process. When encoded video data is processed by video decoder (… Figure 2A During decoding at (not shown), the reconstructed video sequence may be a copy of the source video sequence with some errors. Decoder 210 replicates the decoding process, which can be performed on the reference frame by a remote video decoder, and allows the reconstructed reference frame to be stored in reference image memory 208. In this way, encoder component 106 locally stores a copy of the reconstructed reference frame that shares the same content (no transmission errors) as the reconstructed reference frame to be obtained by the remote video decoder.

[0031] Predictor 206 can perform a prediction search against encoding engine 212. That is, for a new frame to be encoded, predictor 206 can search the reference image memory 208 for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. Predictor 206 can operate pixel-by-pixel based on sample blocks to find suitable prediction references. As determined by the search results obtained by predictor 206, the input image may have prediction references obtained from multiple reference images stored in reference image memory 208.

[0032] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder 214. The entropy encoder 214 converts the symbols generated by the various functional units into an encoded video sequence by lossless compression of the symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, and / or arithmetic coding).

[0033] In some embodiments, the output of entropy encoder 214 is coupled to a transmitter. The transmitter may be configured to buffer the encoded video sequence created by entropy encoder 214 in preparation for transmission via communication channel 218, which may be a hardware / software link to a storage device storing the encoded video data. The transmitter may be configured to combine encoded video data from source encoder 202 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown). In some embodiments, the transmitter may transmit additional data while transmitting the encoded video. Source encoder 202 may include such data as part of the encoded video sequence. Additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and redundant slices, Supplementary Enhancement Information (SEI) messages, fragments of Visual Usability Information (VUI) parameter sets, etc.

[0034] Controller 204 manages the operation of encoder component 106. During encoding, controller 204 can assign a specific type of encoded picture to each encoded picture, but this may affect the encoding technique applied to the corresponding picture. For example, a picture can be designated as an intra-picture (I-picture), a predictive picture (P-picture), or a bidirectional predictive picture (B-picture). Intra-pictures can be encoded and decoded without using any other frames in the sequence as a prediction source. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Variations of I-pictures and their corresponding applications and characteristics are familiar to those skilled in the art and will not be repeated here. Predictive pictures can be encoded and decoded using intra-prediction or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​for each block. Bidirectional predictive pictures can be encoded and decoded using intra-prediction or inter-prediction, which uses at most two motion vectors and reference indices to predict the sample values ​​for each block. Similarly, multiple predictive images can be used with more than two reference images and associated metadata to reconstruct a single block.

[0035] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignments of the corresponding images applied to the blocks. For example, blocks in an I-image can be non-predictively coded, or the block can be predictively coded (spatial or intra-frame prediction) with reference to already coded blocks in the same image. Pixel blocks in a P-image can be non-predictively coded with reference to a previously coded reference image via spatial prediction or temporal prediction. Blocks in a B-image can be non-predictively coded with reference to one or two previously coded reference images via spatial prediction or temporal prediction.

[0036] The acquired video can be presented as multiple source images (video images) in a time-series format. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In an embodiment, a specific image being encoded / decoded is divided into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector may have a third dimension that identifies the reference image.

[0037] Figure 2BThis is a block diagram illustrating example elements of a decoder component 122 according to some embodiments. Figure 2B The decoder component 122 is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to the loop filter 256, which is configured to transmit data to the display 124 (e.g., via a wired or wireless connection).

[0038] In some embodiments, decoder component 122 includes a receiver coupled to channel 218, configured (e.g., via a wired or wireless connection) to receive data from channel 218. The receiver may be configured to receive one or more encoded video sequences to be decoded by decoder component 122. In some embodiments, decoding of each encoded video sequence is independent of other encoded video sequences. Each encoded video sequence may be received from channel 218, which may be a hardware / software link to a storage device storing the encoded video data. The receiver may receive encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective user entities (not shown). The receiver may separate the encoded video sequences from other data. In some embodiments, the receiver receives additional (redundant) data along with the encoded video. This additional data may be part of the encoded video sequence. This additional data may be used by decoder component 122 to decode the data and / or more accurately reconstruct the original video data. Additional data can take the form of time, space, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0039] According to some embodiments, decoder component 122 includes: buffer memory 252, parser 254 (sometimes also called entropy decoder), scaler / inverse transform unit 258, intra-frame image prediction unit 262, motion compensation prediction unit 260, aggregator 268, loop filter unit 256, reference image memory 266, and current image memory 264. In some embodiments, decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuits. Decoder component 122 is at least partially implementable in software.

[0040] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., to prevent network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, a separate buffer memory is provided outside decoder component 122 (e.g., to prevent network jitter) in addition to buffer memory 252 inside decoder component 122 (e.g., configured to handle broadcast timing). Buffer memory 252 may not be necessary or may be made smaller when receiving data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network. Buffer memory 252 may also be required for use on packet networks such as the Internet, and may be relatively large and / or adaptively sized, and may be at least partially implemented in an operating system or similar element outside decoder component 122.

[0041] Parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. Symbols may include, for example, information for managing the operation of decoder component 122, and / or information for controlling a presentation device such as display 124. Control information for the presentation device may be, for example, in the form of Supplemental Enhancement Information (SEI) messages or fragments of Video Usability Information (VUI) parameter sets (not shown). Parser 254 parses (entropy decodes) the encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow principles known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. Parser 254 may extract a subset of parameters from the encoded video sequence for at least one subset of pixels in a subgroup for use in the video decoder, based on at least one parameter corresponding to a group. Subgroups can include Group of Pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), and so on. The parser 254 can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0042] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of symbol 270 may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by parser 254. For brevity, the flow of such subgroup control information between parser 254 and the various units described below is not depicted.

[0043] The decoder component 122 may be conceptually subdivided into multiple functional units, and in some implementations, these units interact closely with each other and may be at least partially integrated with each other. However, for clarity, this document retains the conceptual subdivision of functional units.

[0044] The scaler / inverse transform unit 258 receives from the parser 254 the quantization transform coefficients as symbols 270 and control information (such as the transform mode, block size, quantization factor, and / or quantization scaling matrix, etc.). The scaler / inverse transform unit 258 can output a block including sample values, which can be input into the aggregator 268.

[0045] In some cases, the output samples of the scaler / inverse transform unit 258 belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images, but can use predictive information from previously reconstructed portions of the current image. Such predictive information can be provided by the intra-picture prediction unit 262. The intra-picture prediction unit 262 can use surrounding reconstructed information extracted from the current (partially reconstructed) image in the current image memory 264 to generate blocks of the same size and shape as the blocks being reconstructed. The aggregator 268 can add the predictive information generated by the intra-picture prediction unit 262 to the output sample information provided by the scaler / inverse transform unit 258 based on each sample.

[0046] In other cases, the output samples of the scaler / inverse transform unit 258 belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit 260 can access the reference image memory 266 to extract samples for prediction. After motion compensation is performed on the extracted samples according to the symbols 270 belonging to the block, these samples can be added by the aggregator 268 to the output of the scaler / inverse transform unit 258 (referred to as residual samples or residual signals in this case) to generate output sample information. The address in the reference image memory 266 from which the motion compensation prediction unit 260 extracts the predicted samples can be controlled by motion vectors. Motion vectors can be provided to the motion compensation prediction unit 260 in the form of symbols 270, which can have, for example, X components, Y components, and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory 266 when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0047] The output samples of aggregator 268 can be employed by various loop filtering techniques in loop filter unit 256. Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video bitstream, which are used as symbols 270 from parser 254 in loop filter unit 256. However, video compression techniques may also respond to metadata obtained during decoding of a previous (in decoding order) portion of an encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values. The output of loop filter unit 256 may be a sample stream, which can be output to a presentation device, such as display screen 124, and stored in reference picture memory 266 for subsequent inter-frame picture prediction.

[0048] Once reconstructed, certain encoded images can be used as reference images for future predictions. Once the encoded images have been reconstructed and are identified as reference images (e.g., by parser 254), the current reference image can become part of the reference image memory 266, and a new current image memory can be reallocated before reconstructing subsequent encoded images begins.

[0049] Figure 3This is a block diagram illustrating a server system 112 according to some embodiments. The server system 112 includes control circuitry 302, one or more network interfaces 304, memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuitry 302 includes one or more processors (e.g., CPU, GPU, and / or DPU). In some embodiments, the control circuitry includes one or more field-programmable gate arrays (FPGAs), hardware accelerators, and / or one or more integrated circuits (e.g., application-specific integrated circuits).

[0050] Network interface 304 can be configured to interact with one or more communication networks (e.g., wireless, wired, and / or optical networks). User interface 306 includes one or more output devices 308 and / or one or more input devices 310. Input devices 310 may include one or more of the following: keyboard, mouse, touchpad, touchscreen, microphone, scanner, camera, etc. Output devices 308 may include one or more of the following: audio output devices (e.g., speakers), visual output devices (e.g., displays or viewers), etc.

[0051] Memory 314 may include high-speed random access memory (e.g., DRAM, SRAM, DDR RAM, and / or other random access solid-state memory devices) and / or non-volatile memory (e.g., one or more disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state memory devices). Memory 314 may optionally include one or more storage devices located remotely from control circuitry 302. Memory 314, or alternatively, the non-volatile solid-state memory devices within memory 314, include non-transitory computer-readable storage media. In some embodiments, memory 314 or the non-transitory computer-readable storage media of memory 314 stores programs, modules, instructions, and data structures, or subsets or supersets thereof: • Operating system 316, which includes programs for handling various basic system services and programs for performing hardware-related tasks; • Network communication module 318 is used to connect server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired and / or wireless connections); • Encoding / decoding module 320 is used to perform various functions related to encoding and / or decoding data (e.g., video data). In some embodiments, encoding / decoding module 320 is an instance of codec component 114. Encoding / decoding module 320 includes, but is not limited to, one or more of the following: Decoding module 322 is used to perform various functions related to decoding encoded data, such as those previously described with respect to decoder component 122; and Encoding module 340 is used to perform various functions related to encoding data, such as those previously described with respect to encoder component 106; and • Image memory 352 is used to store images and image data, for example, for use with encoding / decoding module 320. In some embodiments, image memory 352 includes one or more of the following memories: reference image memory 208, buffer memory 252, current image memory 264, and reference image memory 266.

[0052] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform various functions previously described with respect to the parser 254), a transformation module 326 (e.g., configured to perform various functions previously described with respect to the scaler / inverse transformation unit 258), a prediction module 328 (e.g., configured to perform various functions previously described with respect to the motion compensation prediction unit 260 and / or the intra-frame image prediction unit 262), and a filter module 330 (e.g., configured to perform various functions previously described with respect to the loop filter 256).

[0053] In some embodiments, the encoding module 340 includes an encoder module 342 (e.g., configured to perform various functions previously described with respect to source encoder 202 and / or encoding engine 212) and a prediction module 344 (e.g., configured to perform various functions previously described with respect to predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 includes... Figure 3 A subset of the modules shown. For example, both decoding module 322 and encoding module 340 use a shared prediction module.

[0054] Each of the modules identified above and stored in memory 314 corresponds to a set of instructions for performing the functions described herein. The modules identified above (e.g., instruction sets) do not need to be implemented as separate software programs, processes, or modules; therefore, in various embodiments, subsets of these modules can be combined or otherwise rearranged. Although Figure 3 A server system 112 according to some embodiments is shown, but Figure 3 This is intended more as a functional description of various features that may exist in one or more server systems, rather than a structural schematic diagram of the embodiments described herein. In fact, as those skilled in the art will recognize, items shown individually can be combined, and some items can be separated. For example, Figure 3Some items shown individually can be implemented on a single server, and a single item can be implemented by one or more servers. The actual number of servers used to implement server system 112 and how features are distributed among them will vary depending on the implementation method and, optionally, in part, depend on the data traffic handled by the server system during peak usage and during average usage.

[0055] The encoding processes and techniques described below can be performed at the devices and systems described above (e.g., source device 102, server system 112, and / or electronic device 120). According to some embodiments, methods for loop filtering (e.g., cross-component offset filtering) are described.

[0056] Cross-Component Sample Offset (CCSO) is a filtering process that uses the co-located reconstructed sample of a first color component and its neighboring reconstructed samples as input to derive an offset value. This offset value is added to the current sample of the second color component to adjust its reconstructed value. An example of the first color component is the luminance color component, and an example of the second color component is the chrominance color component. The first and second color components can be the same color component, such as luminance.

[0057] In some embodiments, a three-step process is used to derive the offset values: (1) calculating delta values ​​using co-occurrence and adjacent reconstructed samples of the first color component, (2) quantizing the delta values ​​into edge classes using a scalar quantizer with configurable intervals, and (3) determining the offset values ​​based on the edge classes using a dedicated LUT. By introducing these steps, the described system achieves improved coding efficiency and enhanced visual quality. Furthermore, the system architecture supports the parallel operation of CCSO filtering with other in-loop filters, such as deblocking filters and constrained direction enhancement filters, to enhance the overall performance of the video coding pipeline. This approach not only improves the quality of the reconstructed video but also ensures compatibility with existing video coding standards, making the solution adaptable and practical for modern video compression systems.

[0058] Figures 4A to 4C An overview of the quantization and subsequent dequantization process is shown. Figure 4A The calculation process for a prediction block according to some embodiments is illustrated. Figure 4A In the example, intra-frame prediction is performed on the current block 402 to generate prediction block 404. The current block 402 includes a set of samples (e.g., pixel blocks), and prediction block 404 includes a set of predictions corresponding to that set of samples. Figure 4B The calculation process of residual blocks according to some embodiments is illustrated. For example... Figure 4BAs shown, the prediction block 404 is subtracted from the current block 402 to generate a residual block 406 that includes a set of residuals. For example, the corresponding difference between each sample and its corresponding prediction is calculated. Figure 4C The calculation process for reconstructed blocks according to some embodiments is illustrated. For example... Figure 4C As shown, residual block 406 undergoes one or more transforms and quantizations to generate a set of residual coefficients. This set of residual coefficients can be sent from the encoder component to the decoder component. This set of residual coefficients undergoes inverse quantization and inverse transform to generate reconstructed residual block 408. Reconstructed residual block 408 is combined with prediction block 404 (e.g., adding the reconstruction residuals of reconstructed residual block 408 to the predictions of prediction block 404) to generate reconstructed block 410 corresponding to the current block 402.

[0059] It is worth noting that the transformations performed during the decoding of a video stream can be the inverse operations of the transformations performed during the encoding of the video stream, and are sometimes referred to as "inverse transforms". For simplicity, the transformations described in this article, whether performed during encoding or decoding, can be referred to as "transforms".

[0060] In the Constrained Directional Enhancement Filter (CDEF), edge detection is performed on each block (e.g., an 8×8 block) using a predefined directional pattern. The CDEF algorithm analyzes pixel values ​​along eight possible directions and selects the direction that minimizes the sum of the squared differences between pixels along that direction and their mean. Once the primary edge direction is determined, CDEF applies a nonlinear filter (primary filter) and a secondary filter at a 45-degree angle to the primary direction along the detected direction. This directional filtering reduces ringing and other artifacts while preserving sharp edges, thereby improving the visual quality of the reconstructed video.

[0061] Figure 5A An example in-loop filtering stage according to some embodiments is shown. Figure 5AIn the example, the in-loop filtering stage applied to decoded frame 502 includes a deblocking filter 504, a CDEF 506, a CCSO filter 508, and a loop recovery filter 510. In some embodiments, the filtered output frame is used as a reference frame for subsequent frames (e.g., stored in a reference frame buffer 514). In some embodiments, a standard film grain synthesis stage is also applied to generate the corresponding display image 512. Unlike the in-loop filtering stage, the results of the film grain synthesis stage (e.g., an out-of-loop filter) do not affect the prediction of subsequent frames. Loop filtering methods may include any filtering process applied to the reconstructed samples (e.g., after adding residuals to the prediction), including Wiener loop filtering, cross-component filtering via CCSO filter 508 and CDEF 506.

[0062] In some embodiments, the CCSO filtering method may use (e.g., by applying CCSO filter 508) co-located reconstructed samples and neighboring reconstructed samples of a first color component as input to perform filtering on the current reconstructed sample of a second color component. In some embodiments, the CCSO filtering method may use (e.g., by applying CCSO filter 508) co-located reconstructed samples of a first color component and its neighboring reconstructed samples as input to derive an offset value that is added to the current sample of the second color component to adjust its reconstructed value. The first color component may refer to the luma color component, and the second color component may refer to the chromaticity color component. The first color component and the second color component may be the same color component (e.g., the luma component). CCSO filter 508 may generate an offset value that is added to the reconstructed samples of the luma and chromaticity components to reduce reconstruction errors. In some embodiments, CCSO filter 508 operates concurrently with CDEF 506. For example, the reconstructed sample after deblocking 504 may be used as input to both CDEF 506 and CCSO filter 508.

[0063] Deblocking filter 504 can be applied across transform block boundaries to eliminate block artifacts caused by quantization errors. In some embodiments, the filter length is determined based on the minimum transform block size on both sides. In some embodiments, deblocking filter 504 uses a finite impulse response (FIR) filter (e.g., a low-pass filter). Edge detection can be used to disable the deblocking filter in transition regions containing high-variance signals (e.g., to avoid blurring actual edges in the original image). In this way, the deblocking filtering method can be applied to reconstructed samples located near block boundaries. Block boundaries can include transform block boundaries, motion-compensated block boundaries, coded block boundaries, and / or fixed block size boundaries.

[0064] CDEF 506 applies a nonlinear deringing filter along a specific (e.g., tilted) direction. CDEF 506 can operate on the output of deblocking filter 504. CDEF 506 can operate in 8×8 units. In some embodiments, eight preset directions are defined by rotating and flipping templates with preset directions. The decoder can use reconstructed pixels to select a prevalent direction index. A primary filter can be applied along the selected direction, and a secondary filter can be applied along an offset direction (e.g., at a 45° angle to the primary direction). In some embodiments, up to eight sets of filter parameters are transmitted (e.g., in the frame header). The filter parameter sets may include primary filter strength indices and secondary filter strength indices for the luma and chroma components. CDEF can apply filtering to reconstructed samples by identifying the direction of each block and then performing highly controllable adaptive filtering on the filter strength along and across that direction.

[0065] In some embodiments, a loop recovery filter 510 is applied to the reconstructed pixels after any prior in-loop filtering stage (e.g., deblocking filter 504, CDEF 506, and / or CCSO filter 508). The loop recovery filter 510 can be applied to loop recovery units (LRUs), such as 64×64, 128×128, and / or 256×256 pixel blocks. Bypass filtering, Wiener filters (e.g., the Wiener loop filtering method), and / or self-guided filters can be applied independently to each LRU. The Wiener loop filtering method can use the current reconstructed sample and a linearly weighted sum of multiple spatially adjacent reconstructed samples as input to derive a corrected value for the current reconstructed sample as output.

[0066] CCSO aims to achieve improved loop filtering for both the luma and chroma components. In some embodiments, the CCSO filtering process involves three main steps. First, the currently reconstructed luma sample (e.g., the output of a deblocking process) is classified using classifier 516. Two types of classifiers exist: an edge-offset (EO) classifier 516E and a band-offset (BO) classifier 516B. These classifiers can operate jointly or individually based on frame-level indicators of the signal transmission. Second, the category associated with the current luma sample is used as an index to retrieve the offset value from a lookup table (LUT), which is determined at the frame level and whose entries are selected from a finite number of predefined values. This LUT is shared throughout the frame. Finally, the offset value derived using the LUT and the category index is added to the corresponding luma and chroma components. An on / off flag (for non-overlapping 256×256 luma samples) at the signal transmission filter unit level indicates whether CCSO filtering is applied to the associated filter unit.

[0067] Continue to refer to Figure 5A In some embodiments associated with offset classification, the CCSO filtering method includes an offset classifier 516B. Based on the offset classifier 516B, the decoder 122 can determine that the target luminance sample set includes a first luminance sample and one or more adjacent luminance samples. This target luminance sample set is provided to a quantizer to generate one or more quantized values, which are further applied by the offset classifier 516B to classify the first color sample 520. In some embodiments associated with edge offset classification, the CCSO filtering method includes an edge offset classifier 516E. Based on the edge offset classifier 516E, the decoder 122 can determine that the target luminance sample set includes a first luminance sample and one or more adjacent luminance samples. The difference between the adjacent luminance samples and the first luminance sample is provided to a quantizer to generate one or more quantized values, which are further applied by the edge offset classifier 516E to classify the first color sample 520. In some embodiments, the first color sample 520 is classified, for example, by the classifier 516 based on the quantized values ​​to determine a first sample offset 518 of the first color sample 520. The first color sample 520 is adjusted based on a first sample offset 518 to enable reconstruction of the current image frame. In some embodiments, the first color sample 520 includes a first chromaticity sample 524C co-located with a first luminance sample 522L in the current image frame, and the first chromaticity sample 524C is adjusted based on the first sample offset 518. Alternatively, in some embodiments, the first color sample 520 is the first luminance sample 522L, and the first luminance sample 524C is adjusted based on the first sample offset 518.

[0068] As described above, CCSO is an edge-preserving loop filter that uses reconstructed samples to calculate sample offsets for the luminance and / or chrominance components. In some embodiments, only luminance samples at locations defined by the filter shape are used to calculate the offsets for the chrominance or luminance components. In some embodiments, the offset values ​​can be derived through the following three steps: • Step 1: Derive values ​​using co-located reconstructed samples of the first color component and their neighboring reconstructed samples. For example, one or more differences between co-located reconstructed samples of the first color component and their neighboring reconstructed samples. Based on a given filter shape (e.g., such as...). Figure 5B (As shown) to select the location of adjacent reconstructed samples. The filter shape index can be written in the frame header; • Step 2: Quantize the derived values ​​using a scalar quantizer. A scalar quantizer is specified by the quantization range and the quantization level. The quantization range refers to the range of values ​​assigned the same integer value, and the quantization level refers to the integer value assigned to all values ​​within the quantization range; and • Step 3: Using the quantized derived values ​​(or quantization levels) as a classifier, offset values ​​can be derived based on the values ​​of a given classifier. For example, a combination of quantized values ​​is used as an index to a selected lookup table, the output of which is the offset value.

[0069] As previously mentioned, CDEF uses a nonlinear low-pass filter along the edge direction. CDEF includes eight predefined direction patterns, performing direction detection in each 8×8 block. Once the direction is determined, a weighted sum of neighboring samples is calculated to reduce ringing artifacts. CDEF employs two types of nonlinear filters: a primary filter and a secondary filter. The primary filter applies taps along the detection direction, while the secondary filter applies taps along a direction at + / -45° to the primary direction.

[0070] The edge detection method in CDEF applies eight predefined direction patterns. The optimal direction is determined by minimizing the sum of the squared differences between the pixels at a selected location indicated by the direction and the mean along that direction. In some conventional systems, such as CCSO, only one filter shape is allowed for the edge direction, and this filter shape is written in the frame header. This frame-level fixed edge direction lacks flexibility in frame content.

[0071] Cross-component offset filtering methods use reconstructed samples to calculate the sample offsets of the luma and / or chroma components using edge-preserving loop filters. In some systems, only luma samples at locations defined by the filter shape are used to calculate the offsets of the chroma or luma components. For each channel, an offset index is written to the frame header. A 1-bit flag is signaled to indicate whether frame-level CCSO syntax (ccso_syntax_reuse_flag) multiplexing is enabled, and a 1-bit flag is signaled to indicate whether block-level CCSO syntax multiplexing (ccso_block_reuse_flag) is enabled. If frame and / or block reuse is enabled, a 3-bit reference frame index is signaled, named ccso_ref_idx in Table 1 below.

[0072]

[0073] For example, the encoder determines and writes the master reference frame in the frame header, which determines the initialization value of the context for the current frame or tile. For the derived master reference frame, the encoder and decoder derive the master reference frame for the current frame / tile (e.g., "A") based on the frame-level quantization index (qindex, or qp parameter value), frame distance, and / or other parameters. For the optimal master reference frame, the encoder further performs optimal selection of the master reference frame. For each reference, its frame context is used as the initial frame context for A, and entropy coding of A is performed. The resulting coded frame size is recorded. The reference frame with the smallest coded frame size is selected as the optimal master reference frame for A. If the derived master reference frame for A is the same as the optimal master reference frame for A, a "0" is signaled in the bitstream; otherwise, a "1" is signaled in the bitstream. Here, "0" indicates the use of the derived master reference frame, while "1" indicates the use of a reference frame signaled later.

[0074] Figure 6A This is a flowchart illustrating a method 600 for decoding video according to some embodiments. Method 600 can be executed at a computing system (e.g., server system 112, source device 102, or electronic device 120) having control circuitry and memory storing instructions executed by the control circuitry. In some embodiments, method 600 is executed by executing instructions stored in memory (e.g., memory 314) of the computing system.

[0075] The system receives (602) a video stream (e.g., an encoded video sequence) comprising multiple frames (e.g., corresponding to a set of pictures), including the current frame. The system identifies (604) a master reference frame for context initialization of the current frame. The system identifies (606) a reference frame for the current frame based on the master reference frame. The system reconstructs (608) the current frame by applying CCSO mode using the reference frame. In this way, the signal transmission of the reference frame index can depend on either the master reference frame used for frame context initialization or a derived master reference frame.

[0076] In some embodiments, a flag (e.g., "ccso_primary_flag") is used to indicate whether ccso_ref_idx matches a primary reference frame or a derived primary reference frame. For example, a signaled value of 0 is transmitted when ccso_ref_idx is the same as the primary reference frame, and a signaled value of 1 is transmitted when ccso_ref_idx is different from the primary reference frame. In cases where ccso_ref_idx differs from the primary reference frame, ccso_ref_idx can be explicitly signaled to specify the frame index used for CCSO syntax multiplexing.

[0077]

[0078] In some embodiments, ccso_ref_idx is not explicitly signaled, and the master reference frame used for frame context initialization or the derived master reference frame is used as the reference frame for CCSO syntax reuse.

[0079]

[0080] In some embodiments, ccso_ref_idx is implicitly derived based on the frame-level quantization index (qindex, or qp parameter value), frame distance, and other possible parameters.

[0081] In some embodiments, when the syntax of a previously decoded frame is reused across component sample offsets, only a subset of reference frames in the reference frame buffer may be used for signal transmission.

[0082] In some embodiments, only the exported primary reference frame and the exported secondary reference frame can be used for syntax reuse, and a flag can be written into the bitstream to indicate which of the two reference frames is used for CCSO syntax reuse.

[0083]

[0084] In some embodiments, only reference frames at the same or lower time level can be used for syntax reuse. In some embodiments, a high-level syntax is signaled to indicate the switching between the various implementations described above. In some embodiments, CCSO syntax reuse mode is not allowed when error resilience mode is enabled.

[0085] In some embodiments, all channels (e.g., Y, U, and V channels) share the "ccso_syntax_reuse_flag" and / or "ccso_block_resue_flag". In some embodiments, the Y channel has its own flags for "ccso_syntax_reuse_flag" and / or "ccso_block_resue_flag", while the U and V channels share these two flags.

[0086] Figure 6B This is a flowchart illustrating a method 650 for encoding video according to some embodiments. Method 650 can be executed at a computing system (e.g., server system 112, source device 102, or electronic device 120) having control circuitry and memory storing instructions executed by the control circuitry. In some embodiments, method 650 is executed by executing instructions stored in memory (e.g., memory 314) of the computing system. Method 650 is executed by the same system as method 600 described above.

[0087] The system receives (652) video data (e.g., a source video sequence) comprising multiple frames (e.g., corresponding to a set of images), including the current frame. The system identifies (654) a master reference frame for context initialization of the current frame. The system identifies (646) a reference frame for the current frame based on the master reference frame. The system encodes (648) the current frame by applying CCSO mode using the reference frame. As previously mentioned, the encoding process can mirror the decoding process described herein (e.g., loop filtering, such as cross-component offset filtering). For the sake of brevity, these details will not be repeated here.

[0088] although Figure 6A and Figure 6B Multiple logical stages are shown in a specific order, but stages that are not dependent on the order can be reordered, and other stages can be combined or decomposed. Some reorderings or other groupings not specifically mentioned will be obvious to those skilled in the art, therefore the ordering and grouping given herein are not exhaustive. Furthermore, it should be recognized that these stages can be implemented in hardware, firmware, software, or any combination thereof.

[0089] Now let’s turn to some exemplary embodiments.

[0090] (A1) In one aspect, some embodiments include a method for video decoding (e.g., method 600). In some embodiments, the method is performed on a computing system (e.g., server system 112) having memory and control circuitry. In some embodiments, the method is performed at an encoding / decoding module (e.g., encoding / decoding module 320). In some embodiments, the method is performed at a source encoding component (e.g., source encoder 202), an encoding engine (e.g., encoding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes: (i) receiving a video bitstream (e.g., an encoded video sequence) comprising a plurality of frames (e.g., corresponding to a set of pictures), said plurality of frames including a current frame; (ii) identifying a master reference frame for context initialization of the current frame; (iii) identifying a reference frame of the current frame based on said master reference frame; and (iv) reconstructing the current frame by applying a cross-component sample offset (CCSO) mode using the reference frame. In this way, the signal transmission of the reference frame index may depend on the master reference frame used for frame context initialization or a derived master reference frame.

[0091] (A2) In some embodiments of A1, the method further includes applying a CCSO mode to the current frame using one or more CCSO syntaxes from a reference frame. For example, when a frame-level or block-level cross-component sample offset syntax reuses syntax from a reference frame, a signal transmission reference frame index may be implicitly derived or explicitly assigned.

[0092] (A3) In some embodiments of A1 or A2, identifying a reference frame includes parsing an indicator from the video stream that indicates whether the reference frame is a primary reference frame. For example, the flag "ccso_primary_flag" is used to indicate whether ccso_ref_idx matches a primary reference frame or a derived primary reference frame. When ccso_ref_idx is the same as the primary reference frame, the signaling value is 0; when ccso_ref_idx is different from the primary reference frame, the signaling value is 1. In the case where ccso_ref_idx is different from the primary reference frame, ccso_ref_idx can be explicitly signaled to specify the frame index used for CCSO syntax multiplexing.

[0093] (A4) In some embodiments of any of A1 to A3, identifying a reference frame includes: (i) selecting a reference frame indicated by an indicator when the video stream includes an indicator; and (ii) selecting a master reference frame as the reference frame when the video stream does not include an indicator. For example, when ccso_ref_idx is not explicitly signaled, the master reference frame used for frame context initialization or the derived master reference frame can be used as the reference frame for CCSO syntax multiplexing.

[0094] (A5) In some embodiments of any of A1 to A4, the master reference frame is a derived master reference frame. In some embodiments, the master reference frame is a master reference frame transmitted via signaling.

[0095] (A6) In some embodiments of any of A1 through A5, reconstructing the current frame using a reference frame includes applying a CCSO mode to the current frame, the CCSO mode using one or more parameters from the reference frame. In some embodiments, multiple color channels (e.g., Y, U, and V channels) share a “ccso_syntax_reuse_flag” and / or a “ccso_block_resue_flag”. As an example, the Y channel may have its own flags for the “ccso_syntax_reuse_flag” and / or the “ccso_block_resue_flag”, and the U and V channels may share these two flags. Sharing flags among multiple color components reduces the signaling overhead of the video bitstream.

[0096] (A7) In some embodiments of any of A1 to A6, a reference frame is identified based on one or more frame-level parameters of the current frame. For example, ccso_ref_idx may be implicitly derived based on the frame-level quantization index (qindex, or qp parameter value), frame distance, and other possible parameters.

[0097] (A8) In some embodiments of any of A1 to A7, the reference frame is selected from the reference frame buffer.

[0098] (A9) In some embodiments of A8, reference frames are selected only from a subset of reference frames in the reference frame buffer. As an example, when multiplexing syntax from a previously decoded frame across component sample offsets, only a subset of reference frames in the OR signaling reference frame buffer may be used. For example, only reference frames at the same or lower time level as the current frame may be used for syntax multiplexing.

[0099] (A10) In some embodiments of A9, a subset of reference frames in the reference frame buffer consists of a primary reference frame and secondary reference frames. For example, only the derived primary reference frame and the derived secondary reference frame can be used for syntax multiplexing, and a flag can be written to the bitstream to indicate which of the two reference frames is used for CCSO syntax multiplexing. Using only a subset of reference frames allows for fewer signaling bits (e.g., a reduced index range for signal transmission), which reduces the signaling overhead of the bitstream.

[0100] (A11) In some embodiments of any of A1 to A10, the method further includes: parsing an indicator from the video stream that indicates how to identify a reference frame for the current frame. For example, a high-level syntax can be signaled to indicate switching between the above embodiments. In some embodiments, CCSO syntax multiplexing mode is not allowed when error recovery mode is enabled.

[0101] (B1) In another aspect, some embodiments include a video encoding method (e.g., method 650). In some embodiments, the method is performed at a computing system (e.g., server system 112) having memory and control circuitry. In some embodiments, the method is performed at an encoding / decoding module (e.g., encoding / decoding module 320). The method includes: (i) receiving video data (e.g., a source video sequence) comprising a plurality of frames (e.g., corresponding to a set of pictures), the plurality of frames including a current frame; (ii) identifying a master reference frame for context initialization of the current frame; (iii) identifying a reference frame of the current frame based on the master reference frame; and (iv) encoding the current frame by applying a CCSO mode using the reference frame. In some embodiments, the method further includes signaling encoded information of the current frame.

[0102] (B2) In some embodiments of B1, one or more parameters from a reference frame are used to apply the CCSO mode.

[0103] (B3) In some embodiments of B1 or B2, the method further includes: transmitting an indicator indicating whether a reference frame is a master reference frame.

[0104] (B4) In some embodiments of any of B1 to B3, the method further includes: (i) transmitting an indicator of the reference frame in the video stream when the reference frame is not the main reference frame; and (ii) not transmitting the indicator when the reference frame is the main reference frame.

[0105] (B5) In some embodiments of any of B1 to B4, a reference frame is identified based on one or more frame-level parameters of the current frame.

[0106] (B6) In some embodiments of any of B1 to B5, the reference frame is selected from the reference frame buffer.

[0107] On the other hand, some embodiments include a computing system (e.g., server system 112) that includes control circuitry (e.g., control circuitry 302) and a memory (e.g., memory 314) coupled to the control circuitry, the memory storing one or more sets of instructions configured to be executed by the control circuitry, the set of instructions including instructions for performing any of the methods described herein (e.g., A1-A11 and B1-B6 above).

[0108] In another aspect, some embodiments include a non-transitory computer-readable storage medium storing one or more sets of instructions for execution by control circuitry of a computing system, including instructions for performing any of the methods described herein (e.g., A1-A11 and B1-B6 above). In some embodiments, the memory or non-transitory computer-readable storage medium stores a video stream including any features disclosed herein (e.g., grammatical and encoding information).

[0109] Unless otherwise stated, any syntax element described herein may be a High-Level Syntax (HLS). As used herein, HLS is transmitted at a level higher than the block level. For example, HLS may correspond to the sequence level, frame level, slice level, or tile level. As another example, HLS elements may be written in Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), slice header, picture header, tile header, and / or CTU header.

[0110] It should be understood that although the terms “first,” “second,” etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the claims. As used in the description of the embodiments and the appended claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. It should be further understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of that feature, integer, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0111] As used herein, depending on the context, the term "if" can be interpreted as meaning "when" or "in" or "in response to determination" or "according to determination" or "in response to detection" when the condition prerequisite is true. Similarly, depending on the context, the phrases "if it is determined [the condition prerequisite is true]" or "if [the condition prerequisite is true]" or "when [the condition prerequisite is true]" can be interpreted as meaning "in determination" or "in response to determination" or "according to determination" or "in response to detection" when the condition prerequisite is true.

[0112] For purposes of explanation, the foregoing description has been illustrated with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in light of the foregoing teachings. The embodiments were chosen and described in order to best explain the principles of operation and practical application, thereby enabling others skilled in the art to implement them.

Claims

1. A method for video decoding, characterized in that, The method includes: Receive a video stream comprising multiple frames, including the current frame; Identify the main reference frame used for context initialization of the current frame; The reference frame of the current frame is identified based on the main reference frame; and The current frame is reconstructed by applying the Cross-Component Sample Offset (CCSO) mode using the reference frame.

2. The method according to claim 1, characterized in that, The method further includes: applying a CCSO mode to the current frame using one or more CCSO syntaxes from the reference frame.

3. The method according to claim 1, characterized in that, Identifying the reference frame of the current frame includes: parsing an indicator from the video stream, the indicator indicating whether the reference frame is the main reference frame.

4. The method according to claim 1, characterized in that, The reference frame for identifying the current frame includes: When the video stream includes an indicator, the reference frame indicated by the indicator is selected; and When the video stream does not include the indicator, the main reference frame is selected as the reference frame.

5. The method according to claim 1, characterized in that, The master reference frame is the exported master reference frame.

6. The method according to claim 1, characterized in that, Reconstructing the current frame using the reference frame includes applying a CCSO mode to the current frame, the CCSO mode using one or more parameters from the reference frame.

7. The method according to claim 1, characterized in that, The reference frame is identified based on one or more frame-level parameters of the current frame.

8. The method according to claim 1, characterized in that, The reference frame is selected from the reference frame buffer.

9. The method according to claim 8, characterized in that, The reference frame is selected only from a subset of the reference frames in the reference frame buffer, and the subset of the reference frames in the reference frame buffer consists of a primary reference frame and a secondary reference frame.

10. The method according to claim 1, characterized in that, The method further includes: parsing an indicator from the video stream, the indicator indicating how to identify a reference frame for the current frame.

11. A video decoding apparatus, characterized in that, The device includes: The receiving module is configured to receive a video stream comprising multiple frames, including the current frame; The first identification module is configured to identify the main reference frame used for context initialization of the current frame; The second identification module is configured to identify the reference frame of the current frame based on the main reference frame; and The reconstruction module is configured to reconstruct the current frame by applying a cross-component sample offset mode using the reference frame.

12. A video encoding method, characterized in that, The method includes: Receive video data comprising multiple frames, including the current frame; Identify the main reference frame used for context initialization of the current frame; The reference frame of the current frame is identified based on the main reference frame; and The current frame is encoded by applying the Cross-Component Sample Offset (CCSO) mode using the reference frame.

13. The method according to claim 12, characterized in that, The CCSO mode is applied using one or more parameters from the reference frame.

14. The method according to claim 12, characterized in that, The method further includes: transmitting a signal indicating whether the reference frame is the main reference frame.

15. The method according to claim 12, characterized in that, The method further includes: When the reference frame is not the main reference frame, an indicator of the reference frame is signaled in the video stream; and When the reference frame is the master reference frame, the indicator is not signaled.

16. The method according to claim 12, characterized in that, The reference frame is identified based on one or more frame-level parameters of the current frame.

17. The method according to claim 12, characterized in that, The reference frame is selected from the reference frame buffer.

18. A video encoding apparatus, characterized in that, The device includes: The receiving module is configured to receive video data comprising multiple frames, including the current frame. The first identification module is configured to identify the main reference frame used for context initialization of the current frame; The second identification module is configured to identify the reference frame of the current frame based on the main reference frame; and The encoding module is configured to encode the current frame by applying a cross-component sample offset mode using the reference frame.

19. A non-transitory computer-readable storage medium storing a video stream, the video stream being generated by a video encoding method, the video stream comprising: Encoded information for multiple frames of video data, including the current frame; and The video encoding method includes: Identify the main reference frame used for context initialization of the current frame; The reference frame of the current frame is identified based on the main reference frame; and The current frame is encoded by applying the Cross-Component Sample Offset (CCSO) mode using the reference frame.

20. The non-transitory computer-readable storage medium according to claim 19, wherein, The video stream further includes: an indicator that indicates whether the reference frame is the main reference frame; The CCSO mode uses one or more parameters from the reference frame.

Images