Cross-component sample offset (CCSO)
By using a cross-component offset filtering method, the offset values between color components are adjusted using a loop filter, which solves the reconstruction error problem in video encoding and improves video compression efficiency and quality.
Patent Information
- Application Number
- CN202580027800.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-05-13
- Filing Date
- 2025-05-15
- Publication Date
- 2026-11-10
AI Technical Summary
Existing video coding technologies struggle to effectively reduce reconstruction errors during compression, particularly the problem of error propagation across color components remains unresolved.
A cross-component offset filtering method is adopted. The offset value is derived by using the co-located reconstructed samples and neighboring reconstructed samples of the first color component through a loop filter, and the current sample of the second color component is adjusted to reduce the reconstruction error.
It effectively reduces errors in the video reconstruction process, improves video compression efficiency and quality, and reduces storage and transmission requirements.
Smart Images

Figure CN123077089A_ABST
Abstract
Description
Related applications
[0001] This application is a continuation of U.S. Patent Application No. 19 / 207,248, filed May 13, 2025, which claims priority to U.S. Provisional Patent Application No. 63 / 688,283, filed August 28, 2024, entitled “CCSO Improvements,” both of which are incorporated herein by reference in their entirety. Technical Field
[0002] The disclosed implementations generally relate to video encoding and decoding, including but not limited to systems and methods for loop filtering (e.g., cross-component offset filtering) of video data. Background Technology
[0003] Digital video is supported by a variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital camera devices, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, video streaming devices, etc. Electronic devices send and receive or otherwise transmit digital video data across communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video encoding can be used to compress video data according to one or more video encoding standards before transmission or storage. Video encoding can be performed by hardware and / or software on servers or electronic / client devices providing cloud services.
[0004] Video coding typically uses prediction methods that leverage the inherent redundancy in video data (e.g., inter-frame prediction, intra-frame prediction, etc.). Video coding aims to compress video data into a form that uses a lower bitrate while avoiding or minimizing video quality degradation. Several video codec standards have been developed. For example, High-Efficiency Video Coding (HEVC / H.265) is a video compression standard designed as part of the MPEG-H (Moving Picture Experts Group-H) project. The ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) and ISO / IEC (International Organization for Standardization / International Electrotechnical Commission) released the HEVC / H.265 standard in 2013 (Revision 1), 2014 (Revision 2), 2015 (Revision 3), and 2016 (Revision 4). Versatile Video Coding (VVC / H.266) is a video compression standard designed as a successor to HEVC. The ITU-T and ISO / IEC released the VVC / H.266 standard in 2020 (Revision 1) and 2022 (Revision 2). AOMedia Video 1 (Alliance for Open Media Video 1) AV1 is an open video coding format designed as an alternative to HEVC. On January 8, 2019, a validated version 1.0.0 of the specification with errata table 1 was released. Summary of the Invention
[0005] As mentioned above, encoding (compression) reduces bandwidth and / or storage space requirements. Both lossless and lossy compression can be used, as described in detail later. Lossless compression refers to a technique that allows an exact copy of the original signal to be reconstructed from the compressed original signal via decoding. Lossy compression refers to encoding / decoding processes where the original video information is not fully preserved during encoding and cannot be fully recovered during decoding. When using lossy compression, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal is useful for the intended application. The tolerable amount of distortion depends on the application. For example, users of some consumer video streaming applications may tolerate higher distortion than users of film or television broadcasting applications. The compression ratio achievable by a particular encoding algorithm can be selected or adjusted to reflect various distortion tolerances: higher tolerable distortion generally allows encoding algorithms that produce higher losses and higher compression ratios.
[0006] This disclosure describes methods, systems, and non-transitory computer-readable storage media for applying loop filters to video (image) compression. The video codec includes multiple functional modules for one or more of the following: intra / inter-frame prediction, transform coding, quantization, entropy coding, and intra-loop filtering. Intra-loop filtering techniques are applied to adjust reconstructed image samples to further reduce reconstruction errors. A cross-component offset filtering method is implemented to derive an offset value by applying co-located reconstructed samples of a first color component and associated neighboring reconstructed samples. This offset value is added to the current sample of a second color component, thereby adjusting the reconstructed value of the current sample. An example of the first color component is a luminance color component, and an example of the second color component is a chrominance color component. In some implementations, the first and second color components correspond to the same color component, such as a luminance sample.
[0007] In various embodiments of this application, samples of the first color component are processed by a cross-component offset filter in a loop filter to determine an offset value, which is then added to samples of the second color component. The cross-component offset filtering is implemented based on a loop filter that uses reconstructed color samples to determine sample offsets for the luminance and / or chrominance components. For example, the sample offset is determined based on the luminance values of a first luminance sample and one or more neighboring luminance samples. In some embodiments, the sample offset is determined based on an edge offset corresponding to a gradient between the first luminance sample and its associated neighboring luminance samples. Alternatively, in some embodiments, the sample offset is determined based on a band offset corresponding to the luminance values (not gradients) of the first luminance sample and its associated neighboring luminance samples. Additionally, in some embodiments, the filter shape used for cross-component sample offset classification is configured to reduce worst-case memory accesses.
[0008] According to some embodiments, a method for video decoding is provided. The method includes: receiving a video bitstream including a current image frame and a first syntax element for a Cross-Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset of a first color sample of the current image frame is generated based on one or more luminance samples. The method further includes: identifying a plurality of candidate luminance sets within a filter range when the first syntax element indicates that the CCSO mode is enabled. The filter range includes a first luminance sample co-located with a first color sample and a plurality of neighboring luminance samples of the first luminance sample, and each candidate luminance set includes a plurality of corresponding luminance samples whose positions are symmetrical with respect to a first position of the first luminance sample. Each luminance sample within the filter range is used for at least one candidate luminance set in the plurality of candidate luminance sets. The method further includes: selecting a target luminance sample set from the plurality of candidate luminance sets, and applying a loop filter to combine the target luminance sample set and the first luminance sample to generate a first sample offset of the first color sample. The method further includes reconstructing the current image frame by adjusting the first color sample based on the first sample offset.
[0009] According to some embodiments, a video encoding method is provided. The method includes: receiving video data including a current image frame; encoding the current image frame; transmitting the encoded current image frame via a video bitstream; and signaling via the video bitstream a first syntax element for a Cross-Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset for a first color sample of the current image frame is generated based on one or more luminance samples. When the CCSO mode is enabled, a target luminance sample set is selected from a plurality of candidate luminance sets within a filter range, and the target luminance sample set and a first luminance sample co-located with a first color sample are combined according to a loop filter to generate the first sample offset for the first color sample. The filter range includes the first luminance sample and a plurality of neighboring luminance samples of the first luminance sample, and each candidate luminance set includes a plurality of corresponding luminance samples whose positions are symmetrical with respect to a first position of the first luminance sample. Each luminance sample within the filter range is used for at least one candidate luminance set from the plurality of candidate luminance sets.
[0010] According to some implementations, a method for bitstream conversion is provided. The method includes: obtaining a source video sequence including a current image frame; and performing a conversion between the source video sequence and a video bitstream. The video bitstream includes the current image frame and a first syntax element for a Cross-Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset of a first color sample of the current image frame is generated based on one or more luminance samples. When CCSO mode is enabled, a target luminance sample set is selected from a plurality of candidate luminance sets within a filter range, and the target luminance sample set and a first luminance sample co-located with a first color sample are combined according to a loop filter to generate a first sample offset of the first color sample. The filter range includes the first luminance sample and a plurality of neighboring luminance samples of the first luminance sample, and each candidate luminance set includes a plurality of corresponding luminance samples whose positions are symmetrical with respect to a first position of the first luminance sample. Each luminance sample within the filter range is used for at least one candidate luminance set from the plurality of candidate luminance sets.
[0011] According to some embodiments, a computing system is provided, such as a streaming system, a server system, a personal computer system, or other electronic device. The computing system includes a control circuitry system and a memory storing one or more sets of instructions. The one or more sets of instructions include instructions for performing any of the methods described herein. In some embodiments, the computing system includes encoder components and / or decoder components.
[0012] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more sets of instructions executable by a computing system. The one or more sets of instructions include instructions for performing any of the methods described herein.
[0013] Therefore, methods, apparatus, and systems for encoding and decoding video are disclosed. Such methods, apparatus, and systems can supplement or replace conventional methods, apparatus, and systems for video encoding and decoding.
[0014] The features and advantages described in this specification are not necessarily all included, and in particular, some additional features and advantages will be apparent to those skilled in the art in light of the accompanying drawings, specification, and claims provided in this disclosure. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes and is not necessarily intended to depict or limit the subject matter described herein. Attached Figure Description
[0015] To provide a more detailed understanding of this disclosure, a more specific description can be made by referring to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the drawings only show relevant features of this disclosure and are therefore not necessarily intended to be limiting, as those skilled in the art will understand upon reading this disclosure that other valid features may be permissible.
[0016] Figure 1 This is a block diagram illustrating an example communication system according to some implementations.
[0017] Figure 2A This is a block diagram illustrating example elements of an encoder component according to some embodiments, and
[0018] Figure 2B This is a block diagram illustrating example elements of a decoder component according to some embodiments.
[0019] Figure 3 This is a block diagram illustrating an example server system according to some implementation methods.
[0020] Figure 4 This is a flowchart of an example process for applying loop filtering in CCSO (Cross-component SampleOffset, CCSO) mode based on filter shape, according to some implementation methods.
[0021] Figure 5A and Figure 5B Two example filter ranges are shown, which identify multiple candidate brightness sets according to some implementations.
[0022] Figure 6A and Figure 6B Several example candidate brightness sets identified in the filter range according to some implementations are shown.
[0023] Figure 7A and Figure 7B Several example candidate brightness sets identified in another filter range having a rectangular shape, according to some implementations, are shown.
[0024] Figure 8A and Figure 8B Several example candidate brightness sets identified in a filter range having a diamond shape, according to some implementations, are shown.
[0025] Figures 9A to 9D These are four example filter shapes that combine more than two adjacent brightness samples in CCSO mode according to some implementation methods.
[0026] Figure 10This is a flowchart illustrating a method for encoding and decoding video according to some embodiments.
[0027] By convention, the various features shown in the accompanying drawings are not necessarily drawn to scale, and similar reference numerals may be used to indicate similar features throughout the specification and the accompanying drawings. Detailed Implementation
[0028] This disclosure describes methods, systems, and non-transitory computer-readable storage media for applying loop filters to video (image) compression. In-loop filtering techniques are applied to adjust reconstructed image samples to further reduce reconstruction errors. A cross-component offset filtering method is implemented to derive an offset value by applying co-located reconstructed samples of a first color component and associated neighboring reconstructed samples. This offset value is added to the current sample of a second color component, thereby adjusting the reconstructed value of the current sample. In various embodiments of this application, the decoder receives a video bitstream from the encoder, the video bitstream including a current image frame and a first syntax element indicating whether a first sample offset of the first color sample is determined based on the values of one or more luminance samples. The offset value is determined using sample values of the first color component in the cross-component offset filtering, and this offset value is added to the samples of the second color component. For example, luminance samples are applied to generate the offset value of a first luminance sample or a first chrominance sample co-located with the first luminance sample.
[0029] More specifically, in some implementations, when the first syntax element indicates that CCSO mode is enabled, the video decoder identifies multiple candidate luminance sets within a filter range. The filter range includes a first luminance sample co-located with a first color sample and multiple neighboring luminance samples of the first luminance sample, and each candidate luminance set includes multiple corresponding luminance samples whose positions are symmetrical with respect to a first position of the first luminance sample. Each luminance sample within the filter range is used for at least one candidate luminance set among the multiple candidate luminance sets. The video decoder selects a target luminance sample set from the multiple candidate luminance sets and applies a loop filter to combine the target luminance sample sets to generate a first sample offset for the first color sample. The video decoder reconstructs the current image frame by adjusting the first color sample based on the first sample offset.
[0030] Figure 1 This is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to 120-m) communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is a streaming system, for example, for use with video-enabled applications such as video conferencing applications, digital TV applications, and media storage and / or distribution applications.
[0031] Source device 102 includes a video source 104 (e.g., a camera device component or media storage device) and an encoder component 106. In some embodiments, the video source 104 is a digital camera device (e.g., configured to create an uncompressed video sample stream). The encoder component 106 generates one or more encoded video bitstreams from the video stream. The video stream from video source 104 can have a higher data volume compared to the encoded video bitstream 108 generated by encoder component 106. Because the encoded video bitstream 108 has a lower data volume (less data) compared to the video stream from video source 104, it requires less bandwidth to transmit and less storage space to store compared to the video stream from video source 104. In some embodiments, source device 102 does not include encoder component 106 (e.g., configured to transmit uncompressed video to network 110).
[0032] One or more networks 110 represent any number of networks that transmit information between source device 102, server system 112, and / or electronic device 120, including, for example, wired (connected) and / or wireless communication networks. One or more networks 110 may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet.
[0033] One or more networks 110 include server systems 112 (e.g., distributed / cloud computing systems). In some embodiments, server system 112 is a streaming server or includes streaming servers (e.g., configured to store and / or distribute video content, such as encoded video streams from source device 102). Server system 112 includes codec components 114 (e.g., configured to encode and / or decode video data). In some embodiments, codec components 114 include encoder components and / or decoder components. In various embodiments, codec components 114 are instantiated as hardware, software, or a combination of hardware and software. In some embodiments, codec components 114 are configured to decode encoded video bitstream 108 and re-encode video data using different encoding standards and / or methods to generate encoded video data 116. In some embodiments, server system 112 is configured to generate multiple video formats and / or encodings based on encoded video bitstream 108. In some embodiments, server system 112 is used as a Media-Aware Network Element (MANE). For example, server system 112 can be configured to trim encoded video bitstream 108 to tailor potentially different bitstreams for one or more of the electronic devices 120. In some implementations, MANE is provided separately from server system 112.
[0034] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode encoded video data 116 to generate an outgoing video stream that can be rendered on a display or other type of rendering device. In some embodiments, one or more of the electronic devices 120 do not include a display component (e.g., communicatively coupled to an external display device and / or includes a media storage device). In some embodiments, electronic device 120 is a streaming client. In some embodiments, electronic device 120 is configured to access server system 112 to obtain encoded video data 116.
[0035] The source device and / or multiple electronic devices 120 are sometimes referred to as “terminal devices” or “user devices”. In some embodiments, one or more of the electronic devices 120 and / or the source device 102 are instances of server systems, personal computers, portable devices (e.g., smartphones, tablets, or laptops), wearable devices, video conferencing equipment, and / or other types of electronic devices.
[0036] In an example operation of communication system 100, source device 102 transmits an encoded video bitstream 108 to server system 112. For example, source device 102 may encode a stream of images captured by the source device. Server system 112 receives the encoded video bitstream 108 and may decode and / or encode the encoded video bitstream 108 using codec component 114. For example, server system 112 may apply encoding to the video data that is better suited for network transmission and / or storage. Server system 112 may transmit encoded video data 116 (e.g., one or more encoded video bitstreams) to one or more electronic devices 120. Each electronic device 120 may decode the encoded video data 116 and optionally display video images.
[0037] Figure 2A This is a block diagram illustrating example elements of an encoder component 106 according to some embodiments. The encoder component 106 receives video data (e.g., a source video sequence) from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives the video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder component 106). The video source 104 may provide the source video sequence in the form of a digital video sample stream, which may have any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 YCrCb or RGB), and any suitable sampling structure (e.g., YCrCb 4:2:0 or YCrCb 4:4:4). In some embodiments, the video source 104 is a storage device storing previously captured / prepared video. In some embodiments, the video source 104 is a camera device that captures local image information as a video sequence. Video data can be provided as multiple individual images that are given motion when viewed sequentially. Each image can be organized as a spatial array of pixels, where, depending on the sampling structure, color space, etc., each pixel may include one or more samples. The relationship between pixels and samples will be readily understood by those skilled in the art.
[0038] Encoder component 106 is configured to encode and / or compress images of a source video sequence into an encoded video sequence 216 in real time or under other time constraints required by the application. In some embodiments, encoder component 106 is configured to perform a conversion between the source video sequence and a visual media data bitstream (e.g., a video bitstream). Implementing an appropriate encoding rate is a function of controller 204. In some embodiments, controller 204 controls and is functionally coupled to other functional units as described below. Parameters set by controller 204 may include rate control-related parameters (e.g., image skipping, quantizer and / or rate-distortion optimization technique λ value), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. Other functions of controller 204 can be readily identified by those skilled in the art, as these functions may belong to encoder component 106 optimized for a particular system design.
[0039] In some implementations, encoder component 106 is configured to operate within an encoding / decoding loop. In a simplified example, the encoding / decoding loop includes a source encoder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder 210. Decoder 210 reconstructs the symbols in a manner similar to that of the (remote) decoder to create sample data (assuming lossless compression between the symbols and the encoded video bitstream). The reconstructed sample stream (sample data) is input to a reference image memory 208. Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of the reference image memory 208 are also bit-accurate between the local and remote encoders. In this way, the encoder's prediction portion interprets the same sample values as those interpreted by the decoder during decoding when using prediction as reference image samples. The principle of this reference image synchronization (and the drift that occurs, for example, due to channel errors, when synchronization cannot be maintained) is known to those skilled in the art.
[0040] The operation of decoder 210 can be combined with a remote decoder, for example, as shown below. Figure 2B The operation of the decoder component 122 is the same as described in the detailed description. However, a brief reference is provided. Figure 2B Since the symbols are available and the encoding of the symbols into an encoded video sequence by the entropy encoder 214 and the decoding of the symbols by the parser 254 can be lossless, the entropy decoding portion of the decoder component 122, including the buffer memory 252 and the parser 254, does not need to be fully implemented in the local decoder 210.
[0041] Besides parsing / entropy decoding, the decoder techniques described in this paper can exist in the corresponding encoders in essentially the same form. For this reason, the subject matter focuses on decoder operations. The description of encoder techniques can be simplified, as encoder techniques are inverses of decoder techniques.
[0042] As part of the operation of the source encoder 202, the source encoder 202 may perform motion-compensated predictive coding, which predictively encodes the input frame with reference to one or more previously encoded frames from the video sequence designated as "reference frames". In this way, the encoding engine 212 encodes the difference between pixel blocks of the input frame and pixel blocks of the reference frame, which may be selected as the prediction reference for the input frame. The controller 204 may manage the encoding operations of the source encoder 202, including, for example, setting parameters and subgroup parameters for encoding the video data.
[0043] Decoder 210 decodes encoded video data based on symbols created by source encoder 202 that can be designated as reference frames. The operation of encoding engine 212 can advantageously handle lossy processing. When encoded video data is processed by video decoder (… Figure 2A When decoded at (not shown), the reconstructed video sequence can be a copy of the source video sequence with some errors. Decoder 210 can replicate the decoding process performed on the reference frame by a remote video decoder, and the reconstructed reference frame can be stored in reference image memory 208. In this way, encoder component 106 locally stores a copy of the reconstructed reference frame that shares common content (no transmission errors) with the reconstructed reference frame to be obtained by the remote video decoder.
[0044] Predictor 206 can perform a prediction search against encoding engine 212. That is, for a new frame to be encoded, predictor 206 can search the reference image memory 208 for sample data (as candidate reference pixel blocks) or specific metadata such as reference image motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new image. Predictor 206 can operate pixel-by-pixel based on the sample blocks to find appropriate prediction references. As determined by the search results obtained by predictor 206, the input image can have prediction references obtained from multiple reference images stored in reference image memory 208.
[0045] The outputs of all the functional units mentioned above can undergo entropy encoding in the entropy encoder 214. The entropy encoder 214 converts the symbols, such as those generated by the various functional units, into an encoded video sequence by lossless compression according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, and / or arithmetic coding).
[0046] In some implementations, the output of entropy encoder 214 is coupled to a transmitter. The transmitter can be configured to buffer encoded video sequences, such as those created by entropy encoder 214, in preparation for transmission via communication channel 218, which can be a hardware / software link to a storage device storing the encoded video data. The transmitter can be configured to combine encoded video data from source encoder 202 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown). In some implementations, the transmitter can transmit additional data along with the encoded video. Source encoder 202 can include such data as part of the encoded video sequence. Additional data can include temporal / spatial / SNR (Signal-to-Noise Ratio, SNR) enhancement layers, other forms of redundant data such as redundant images and slices, Supplementary Enhancement Information (SEI) messages, fragments of Visual Usability Information (VUI) parameter sets, etc.
[0047] Controller 204 can manage the operation of encoder component 106. During encoding, controller 204 can assign a specific type of encoded picture to each encoded picture, which may affect the encoding technique applied to the corresponding picture. For example, pictures can be assigned as intra-pictures (I-pictures), predictive pictures (P-pictures), or bidirectional predictive pictures (B-pictures). Intra-pictures can be encoded and decoded without using any other frames in the sequence as prediction sources. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are familiar with those variations of I-pictures and their corresponding applications and characteristics, and therefore they will not be repeated here. Predictive pictures can be encoded and decoded using inter-frame prediction or intra-frame prediction that uses at most one motion vector and reference index to predict sample values for each block. Bidirectional predictive pictures can be encoded and decoded using inter-frame prediction or intra-frame prediction that uses at most two motion vectors and reference indexes to predict sample values for each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.
[0048] Source images can typically be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and encoded on a block-by-block basis. These blocks can be predictively encoded with reference to other (already encoded) blocks, determined by the coding assignments of the corresponding images applied to the blocks. For example, blocks of image I can be unpredictably encoded, or blocks of image I can be predictively encoded (spatial prediction or intra-frame prediction) with reference to already encoded blocks of the same image. Pixel blocks of image P can be unpredictably encoded with reference to a previously encoded reference image via spatial prediction or temporal prediction. Blocks of image B can be unpredictably encoded with reference to one or two previously encoded reference images via spatial prediction or temporal prediction.
[0049] Video can be captured as multiple source images (video images) in a time-series manner. Intra-frame image prediction (often simply called intra-prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In the example, a specific image during encoding / decoding—referred to as the current image—is segmented into blocks. Where a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference images.
[0050] Encoder component 106 can perform encoding operations according to any predetermined video coding technique or standard described herein. In operation, encoder component 106 can perform various compression operations, including predictive coding operations utilizing temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.
[0051] Figure 2B This is a block diagram illustrating example elements of a decoder component 122 according to some embodiments. Figure 2B The decoder component 122 is coupled to channel 218 and display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to loop filter 256 and configured to transmit data to display 124 (e.g., via a wired or wireless connection).
[0052] In some embodiments, decoder component 122 includes a receiver coupled to channel 218 and configured to receive data from channel 218 (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more encoded video sequences to be decoded by decoder component 122. In some embodiments, decoding of each encoded video sequence is independent of other encoded video sequences. Each encoded video sequence can be received from channel 218, which may be a hardware / software link to a storage device storing the encoded video data. The receiver may receive encoded video data along with other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective user entities (not depicted). The receiver may separate the encoded video sequences from other data. In some embodiments, the receiver receives additional (redundant) data along with the encoded video. This additional data may be included as part of the encoded video sequence. Decoder component 122 may use the additional data to decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.
[0053] According to some embodiments, the decoder component 122 includes a buffer memory 252, a parser 254 (sometimes also referred to as an entropy decoder), a scaler / inverse transform unit 258, an intra-frame image prediction unit 262, a motion compensation prediction unit 260, an aggregator 268, a loop filter unit 256, a reference image memory 266, and a current image memory 264. In some embodiments, the decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuit systems. The decoder component 122 can be implemented at least partially in software.
[0054] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., to combat network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, a separate buffer memory is provided outside decoder component 122 (e.g., to combat network jitter) in addition to buffer memory 252 inside decoder component 122 (e.g., configured to handle broadcast timing). Buffer memory 252 may not be necessary or may be small when receiving data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network. Buffer memory 252 may be required to make the best use of packet networks such as the Internet. Buffer memory 252 may be relatively large and / or have an adaptive size and may be implemented at least partially in an operating system or similar component outside decoder component 122.
[0055] Parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. Symbols may include, for example, information for managing the operation of decoder component 122, and / or information for controlling rendering devices such as display 124. Control information for the rendering device may be in the form of, for example, Supplemental Enhancement Information (SEI) messages or Video Availability Information (VUI) parameter set fragments (not depicted). Parser 254 parses (entropy decodes) the encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow principles known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. Parser 254 may extract a subgroup parameter set from the encoded video sequence for pixels in a subgroup of at least one subgroup of pixels in the video decoder based on at least one parameter corresponding to a group. Subgroups may include picture groups (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser 254 can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.
[0056] Depending on the type of encoded video picture or a portion thereof (e.g., inter-frame and intra-frame pictures, inter-frame and intra-frame blocks) and other factors, the reconstruction of symbol 270 may involve multiple different units. Which units are involved and how they are involved can be controlled by parser 254 through subgroup control information parsed from the encoded video sequence. For clarity, the flow of such subgroup control information between parser 254 and the following multiple units is not depicted.
[0057] The decoder component 122 can be conceptually subdivided into multiple functional units, and in some implementations, these units interact closely with each other and can be at least partially integrated with each other. However, for clarity, this document retains the conceptual subdivision of functional units.
[0058] The scaler / inverse transform unit 258 receives from the parser 254 the quantization transform coefficients as symbols 270, as well as control information (e.g., which transform to use, block size, quantization factor, and / or quantization scaling matrix). The scaler / inverse transform unit 258 can output blocks including sample values, which can be input to the aggregator 268.
[0059] In some cases, the output samples of the scaler / inverse transform unit 258 belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images, but can use predictive information from previously reconstructed portions of the current image. Such predictive information can be provided by the intra-picture prediction unit 262. The intra-picture prediction unit 262 can use surrounding reconstructed information obtained from the current (partially reconstructed) image from the current image memory 264 to generate blocks of the same size and shape as the blocks in the reconstruction. The aggregator 268 can add the prediction information already generated by the intra-picture prediction unit 262 to the output sample information provided by the scaler / inverse transform unit 258 based on each sample.
[0060] In other cases, the output samples of the scaler / inverse transform unit 258 belong to inter-frame coded blocks and potentially to motion-compensated blocks. In such cases, the motion compensation prediction unit 260 can access the reference image memory 266 to obtain samples for prediction. After motion compensation is performed on the obtained samples according to the symbol 270 belonging to the block, these samples can be added by the aggregator 268 to the output of the scaler / inverse transform unit 258 (referred to in this case as residual samples or residual signals) to generate output sample information. The address in the reference image memory 266 from which the motion compensation prediction unit 260 obtains the predicted samples can be controlled by motion vectors. Motion vectors can be used by the motion compensation prediction unit 260 in the form of symbol 270, which can have, for example, an X component, a Y component, and a reference image component. Motion compensation can also include interpolation of sample values obtained from the reference image memory 266, motion vector prediction mechanisms, etc., when using subsampled precise motion vectors.
[0061] The output samples of aggregator 268 can undergo various loop filtering techniques in loop filter unit 256. Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video bitstream and available to loop filter unit 256 as symbols 270 from parser 254. However, video compression techniques may also respond to metadata obtained during decoding of previous (in decoding order) portions of the encoded picture or encoded video sequence, and to sample values obtained from previous reconstruction and loop filtering. The output of loop filter unit 256 may be a sample stream, which can be output to a rendering device such as display 124 and stored in reference picture memory 266 for use in future inter-frame picture prediction.
[0062] Once reconstructed, certain encoded images can be used as reference images for future predictions. Once an encoded image has been reconstructed and has been identified as a reference image (e.g., by parser 254), the current reference image can become part of the reference image memory 266, and a new current image memory can be reallocated before reconstructing subsequent encoded images begins.
[0063] Decoder component 122 can perform decoding operations according to a predetermined video compression technique that may be recorded in a standard, such as any standard described herein. As specified in a video compression technique document or standard, and particularly in a configuration file therein, an encoded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard. Furthermore, to conform to some video compression techniques or standards, the complexity of the encoded video sequence may be within a range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured, for example, megasamples per second), maximum reference picture size, etc. In some cases, the limitations set by the hierarchy can be further restricted by the Hypothetical Reference Decoder (HRD) specification and metadata managed by the HRD buffer used to represent signals in the encoded video sequence.
[0064] Figure 3This is a block diagram illustrating a server system 112 according to some embodiments. The server system 112 includes a control circuitry system 302, one or more network interfaces 304, a memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuitry system 302 includes one or more processors (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and / or a DPU (Data Processing Unit)). In some embodiments, the control circuitry system includes one or more field-programmable gate arrays (FPGAs), hardware accelerators, and / or one or more integrated circuits (e.g., application-specific integrated circuits).
[0065] Network interface 304 can be configured to interface with one or more communication networks (e.g., wireless networks, wired networks, and / or optical networks). Communication networks can be local, wide area, metropolitan area, vehicular and industrial, real-time, latency-tolerant, etc. Examples of communication networks include: local area networks such as Ethernet and wireless LANs; cellular networks including GSM (Global System for Mobile Communication), 3G (the Third Generation), 4G (the Fourth Generation), 5G (the Fifth Generation), LTE (Long Term Evolution), etc.; cable TV, satellite TV, and terrestrial broadcast TV wired or wireless wide area digital networks; vehicular and industrial networks including CANBus (Controller Area Network Bus), etc. Such communication can be one-way receive-only (e.g., broadcast TV), one-way transmit-only (e.g., to a CANBus device), or bidirectional (e.g., to other computer systems using local or wide area digital networks). Such communication can include communication across one or more cloud computing networks.
[0066] User interface 306 includes one or more output devices 308 and / or one or more input devices 310. Input devices 310 may include one or more of the following: keyboard, mouse, touchpad, touch screen, data glove, joystick, microphone, scanner, camera, etc. Output devices 308 may include one or more of the following: audio output devices (e.g., speakers), visual output devices (e.g., displays or monitors), etc.
[0067] Memory 314 may include high-speed random access memory (e.g., DRAM, SRAM, DDR RAM, and / or other random access solid-state memory devices) and / or non-volatile memory (e.g., one or more disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state memory devices). Memory 314 may optionally include one or more storage devices located remotely from the control circuitry system 302. The non-volatile solid-state memory devices within memory 314, or alternatively memory 314, include non-transitory computer-readable storage media. In some embodiments, memory 314 or the non-transitory computer-readable storage media of memory 314 stores programs, modules, instructions, and data structures, or subsets or supersets thereof:
[0068] • Operating system 316, which includes procedures for handling various basic system services and for performing hardware-related tasks;
[0069] • Network communication module 318, which is used to connect server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired and / or wireless connections);
[0070] • Encoding / decoding module 320, which performs various functions related to encoding and / or decoding data, such as video data. In some embodiments, encoding / decoding module 320 is an example of codec component 114. Encoding / decoding module 320 includes, but is not limited to, one or more of the following:
[0071] ◦ Decoding module 322, which performs various functions related to decoding encoded data, such as those previously described with respect to decoder component 122; and
[0072] ◦ Encoding module 340, which performs various functions related to encoding data, such as those previously described with respect to encoder component 106; and
[0073] • For example, an image memory 352 for storing images and image data, used in conjunction with the encoding / decoding module 320. In some embodiments, the image memory 352 includes one or more of the following: a reference image memory 208, a buffer memory 252, a current image memory 264, and a reference image memory 266.
[0074] In some implementations, the decoding module 322 includes a parsing module 324 (e.g., configured to perform various functions previously described with respect to the parser 254), a transformation module 326 (e.g., configured to perform various functions previously described with respect to the scaler / inverse transformation unit 258), a prediction module 328 (e.g., configured to perform various functions previously described with respect to the motion compensation prediction unit 260 and / or the intra-frame image prediction unit 262), and a filter module 330 (e.g., configured to perform various functions previously described with respect to the loop filter 256).
[0075] In some embodiments, the encoding module 340 includes a code module 342 (e.g., configured to perform various functions previously described with respect to source encoder 202 and / or encoding engine 212) and a prediction module 344 (e.g., configured to perform various functions previously described with respect to predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 include... Figure 3 A subset of the modules shown. For example, a prediction module shared by both decoding module 322 and encoding module 340.
[0076] Each of the modules identified above and stored in memory 314 corresponds to a set of instructions for performing the functions described herein. The modules identified above (e.g., instruction sets) do not need to be implemented as separate software programs, processes, or modules, and therefore various subsets of these modules can be combined or otherwise rearranged in various implementations. For example, the codec module 320 may optionally not include separate decoding and encoding modules, but instead use the same set of modules to perform both sets of functions. In some implementations, memory 314 stores a subset of the modules and data structures identified above. In some implementations, memory 314 stores additional modules and data structures not described above, such as an audio processing module.
[0077] Although Figure 3 A server system 112 according to some embodiments is shown, but Figure 3This is intended more as a functional description of various features that can exist in one or more server systems than as a structural diagram of the implementation described herein. In practice, and as those skilled in the art will recognize, items shown individually can be combined and some items can be separated. For example, Figure 3 Some items shown individually can be implemented on a single server, and a single item can be implemented by one or more servers. The actual number of servers used to implement server system 112 and how features are distributed among said servers will vary depending on the implementation method, and optionally, it depends in part on the amount of data traffic processed by the server system during peak usage periods and during average usage periods.
[0078] Figure 4 This is a flowchart of an example process 400 applying in-loop filtering based on filter shape 440 according to some embodiments. The group of images (GOP) includes a sequence of image frames, which also includes a current image frame. The current image frame includes a color image, i.e., a non-monochrome image frame, having multiple color samples (e.g., chroma sample 402 and luma sample 404) that co-occur with each other. After reconstructing the multiple color samples of the current image frame, in-loop filtering is applied to adjust a subset of the color samples, thereby improving the image quality of the current image frame. In some embodiments, reconstructed samples of a first color component and its neighboring reconstructed samples are combined to derive an offset value for a second color component. The reconstructed samples of the second color component co-occur with the reconstructed samples of the first color component and are adjusted by the offset value. The first color component may optionally be the same as or different from the second color component. Specifically, in some embodiments, neighboring reconstructed samples of the first color component are selected based on filter shape 440 and combined for in-loop filtering to derive an offset value for the second color component (e.g., luma sample 404, chroma sample 402).
[0079] In some implementations, the video bitstream 116 signals the first syntax element 408 for CCSO mode 442 of the current image frame. The first syntax element 408 indicates whether a first sample offset 406 of the first color sample 410 of the current image frame is generated based on one or more luminance samples 404. In some implementations, the first syntax element 408 has a first predefined value indicating that CCSO mode 442 is enabled. In some implementations, the third syntax element 408 is signaled on one of the following: sequence header, picture header, slice header, tile header, tile group header, superblock row header, and superblock header.
[0080] Decoder 122 receives a video bitstream 116 from encoder 106. The video bitstream 116 includes the current image frame and a first syntax element 408 for Cross-Component Sample Offset (CCSO) mode, indicating whether a first sample offset 406 of the first color sample 410 of the current image frame is generated based on one or more luminance samples 404. When the first syntax element 408 indicates that CCSO mode 442 is enabled, decoder 122 identifies multiple candidate luminance sets 404B within a filter range 480. Filter range 480 includes a first luminance sample 404C co-located with the first color sample 410 and multiple neighboring luminance samples 404X of the first luminance sample 404C. Each candidate luminance set 404B includes multiple corresponding luminance samples 404X, the positions of which are symmetrical with respect to a first position of the first luminance sample 404C. Each luminance sample 404 located within filter range 480 is used for at least one candidate luminance set among the multiple candidate luminance sets 404B. A target luminance sample set 404A is selected from multiple candidate luminance sets 404B. For example, the target luminance sample set 404A includes a first luminance sample 404C co-located with the first color sample 410 and multiple corresponding luminance samples 404X of the first luminance sample 404C. A loop filter 424 is applied to combine the target luminance sample set 404A to generate a first sample offset 406 for the first color sample 410. The decoder 122 reconstructs the current image frame by adjusting the first color sample 410 based at least on the first sample offset 406.
[0081] In some implementations associated with band-off classification, CCSO mode 442 corresponds to band-off classifier 412B. Based on band-off classifier 412B, decoder 122 determines that the target luminance sample set 404A includes a first luminance sample 404C and one or more neighboring luminance samples 404X. The target luminance sample set 404A is provided to quantizer 430 and used to generate one or more quantized values 404Q, which are further applied by band-off classifier 412B to classify the first color sample 410. For example, filter shape 440 has a cross shape, and neighboring luminance sample set 404A includes first luminance sample 404C, north luminance sample 404N, south luminance sample 404S, west luminance sample 404W, and east luminance sample 404E. These luminance samples 404 are further quantized into quantized values 404QC, 404QN, 404QS, 404QW, and 404QE, respectively.
[0082] Alternatively, in some implementations associated with edge offset classification, CCSO mode 442 corresponds to edge offset classifier 412E. Based on edge offset classifier 412E, decoder 122 determines that the target luminance sample set 404A includes a first luminance sample 404C and one or more neighboring luminance samples 404X. The difference between the neighboring luminance sample 404X and the first luminance sample 404C is provided to quantizer 430 and used to generate one or more quantized values 404Q, which are further applied by the offset classifier 412B to classify the first color sample 410. For example, filter shape 440 has a cross shape, and four differences 404D corresponding to luminance samples 404N, 404S, 404W, and 404E are further quantized into quantized values 404QN, 404QS, 404QW, and 404QE, respectively.
[0083] The first color sample 410 is classified, for example, by classifier 412 based on a quantized value 404Q to determine a first sample offset 406 for the first color sample 410. In the example, the quantized value is 404Q. Lookup table 414 maps multiple combinations of the quantized value 404Q to different sample offset options SO (e.g., SO1 to SO16). Based on lookup table 414, the quantized value 404Q corresponds to one of the combinations in lookup table 414, and the corresponding sample offset option SO is identified as corresponding to a combination of quantized differences 404Q, and is therefore selected for the first sample offset 406. In other words, in some embodiments, decoder 122 classifies the first color sample 410 by identifying one or more combinations of quantized values 404Q in lookup table 414 that associates multiple quantized combinations with multiple offset value options SO (e.g., SO1 to SO16), and determining in lookup table 414 the first sample offset 406 corresponding to the combination of one or more quantized values 404Q.
[0084] In some implementations, a scalar quantizer 430, comprising multiple quantization intervals 418 (QI) and multiple quantization levels 420 (QL), quantizes the value of the luminance sample 404A into multiple integer values within a quantization range 416, and each of one or more quantized values 404Q includes a corresponding integer within the quantization range 416. For each integer value in the quantization range 416, the quantization interval 418 is defined as a range of values assigned to the corresponding integer value. The quantization level 420 corresponds to a range of integer values assigned a difference associated with the quantization interval 418.
[0085] The first color sample 410 is adjusted based on a first sample offset 406, thereby enabling reconstruction of the current image frame. In some embodiments, the first color sample 410 includes a first chromaticity sample 402C co-located with the first luminance sample 404C in the current image frame, and the first chromaticity sample 402C is adjusted based on the first sample offset 406. Alternatively, in some embodiments, the first color sample 410 is the first luminance sample 404C, and the first luminance sample 404C is adjusted based on the first sample offset 406. In other words, in some embodiments, the first color sample 410 is one of the following: the first luminance sample 404C, the first blue difference chromaticity (Cb) sample 402Cr, and the first blue difference chromaticity (Cb) sample 402Cb. The first luminance sample 404C, the first Cb sample 402Cb, and the first Cr sample 402Cr are co-located with each other.
[0086] In some implementations, the target luminance sample set 404A applied by the loop filter 424 is selected from a plurality of candidate luminance sets 404B. The plurality of neighboring luminance samples 404X include a first subset of neighboring luminance samples (e.g., 404N and 404E) and a second subset of neighboring luminance samples (e.g., 404W and 404S). Each first neighboring luminance sample is uniquely associated with a corresponding second neighboring luminance sample. For each first neighboring luminance sample, the corresponding position of the first neighboring luminance sample and the corresponding position of the corresponding second neighboring luminance sample are symmetrical to each other with respect to the position of the first neighboring luminance sample. In other words, in some implementations, the neighboring luminance samples 404X are selected in pairs from the candidate luminance sets 404B, and the candidate luminance sets 404B include a plurality of corresponding luminance samples whose positions are symmetrical with respect to the first luminance sample 404C. In some cases, only one pair of neighboring luminance samples 404X is selected to be combined with the first luminance sample 404C in the loop filter. In some cases, two or more pairs of neighboring luminance samples 404X are selected and applied by the loop filter 424.
[0087] In some implementations, decoder 122 stores in a row buffer one or more upward luminance samples above the first luminance sample 404C, one or more downward luminance samples below the first luminance sample 404C, one or more left column luminance samples to the left of the first luminance sample 404C, or one or more right column luminance samples to the right of the first luminance sample 404C, and uses the luminance samples stored in the row buffer to provide one or more candidate luminance sets 404B, from which the target luminance sample set 404A applied by loop filter 424 is selected.
[0088] In some embodiments, the filter range 480 has a first range shape (e.g., rectangle, rhombus, oblique rectangle, hexagon, octagon). A predefined set of range shapes is identified. The video bitstream 116 also includes a second syntax element 422, which includes a range shape index that selects a range shape from the predefined set of range shapes as the first range shape, for example at the sequence level, frame level, or filter unit level. Conversely, in some embodiments, the second syntax element 422 is not transmitted in the video bitstream 116, and the first range shape is derived, for example, based on a reference region.
[0089] In some embodiments, the target luminance sample set 404A is arranged according to a first filter shape 440 (e.g., a cross shape). A predefined set of filter shapes is identified. The video bitstream 116 also includes a third syntax element 426, which includes a filter shape index that selects a filter shape from the predefined set of filter shapes as the first filter shape 440, for example at the sequence level, frame level, or filter unit level. Conversely, in some embodiments, the third syntax element 426 is not transmitted in the video bitstream 116, and the first filter shape 440 is derived, for example, based on a reference region.
[0090] In some implementations, the filter range 480 has a first range shape, and the video bitstream 116 includes a second syntax element 422 including a range shape index and a third syntax element 426 including a filter shape index. A predefined set of range shapes is identified, and a first range shape is selected from the predefined set of range shape indexes. A predefined set of filter shapes associated with the filter range 480 having the first range shape is identified. After selecting the first range shape, a first filter shape 440 is selected from the predefined set of filter shapes based on the filter shape index. A target luminance sample set 404A includes a first luminance sample 404C, and the target luminance sample set 404A is arranged according to the first filter shape 440.
[0091] Figure 5A and Figure 5B Two example filter ranges 480 are shown, according to some embodiments, in which multiple candidate luminance sets 404B are identified. Filter ranges 480 include a first luminance sample 404C co-located with a first color sample 410 and multiple neighboring luminance samples 404X of the first luminance sample 404C, and each candidate luminance set 404B includes multiple corresponding luminance samples (e.g., two neighboring luminance samples 404X and the first luminance sample 404C), the positions of which are symmetrical with respect to a first position of the first luminance sample 404C. In some embodiments ( Figure 6A and Figure 6B In this embodiment, each luminance sample 404 located within filter range 480 is used for at least one candidate luminance set from a plurality of candidate luminance sets 404B. Alternatively, in some implementations ( Figure 7A , Figure 7B , Figure 8A and Figure 8B In the filter range 480, each of the subsets (e.g., less than all) of the luminance samples 404 located in the filter range 480 is used for at least one candidate luminance set in the plurality of candidate luminance sets 404B.
[0092] In some embodiments, the filter range 480 has a predefined size based on one or more range constraints, and said one or more range constraints include at least one of the number of horizontally neighboring samples (N) and the number of vertical samples (M). Furthermore, in some embodiments, the filter range 480 is centered on a first luminance sample 404C and is defined by the following (including boundaries): the Mth upper row above the first luminance sample 404C, the Mth lower row below the first luminance sample 404C, the Nth left column to the left of the first luminance sample 404C, and the Nth right column to the right of the first luminance sample 404C. The target luminance sample set 404A is selected from a plurality of candidate luminance sets 404B formed based on luminance samples 404 in the filter range 480.
[0093] Reference Figure 5A In some implementations, the first range of the filter range 480 is rectangular in shape. Integer M equals 2, and integer N equals 3. The filter range 480 comprises 5×7 luminance samples centered on the first luminance sample 404C. Each luminance sample in the plurality of candidate luminance sets 404B has a sample position (x, y) relative to the first luminance sample 404C, where x and y are integers. and .
[0094] Reference Figure 5B In some implementations, the first range of filter range 480 is rhomboid in shape. Integer M equals 3, and integer N equals 3. Filter range 480 includes 25 luminance samples centered on the first luminance sample 404C. Each luminance sample in the plurality of candidate luminance sets 404B has a sample position (x, y) relative to the first luminance sample 404C, where x and y are integers. and ,and .
[0095] In some embodiments, the filter range 480 has a predefined size based on one or more range constraints, and said one or more range constraints include at least one of the number of horizontally adjacent samples (N1 and N2) and the number of vertical samples (M1 and M2). Furthermore, in some embodiments, the filter range 480 is centered on the first luminance sample 404C and is defined by the following (including boundaries): an M1-th upper row above the first luminance sample 404C, an M2-th lower row below the first luminance sample 404C, an N1-th left column to the left of the first luminance sample 404C, and an N2-th right column to the right of the first luminance sample 404C.
[0096] Figure 6A and Figure 6B A plurality of example candidate luminance sets 404B identified in a filter range 480 according to some embodiments is shown. The filter range 480 includes a first luminance sample 404C co-located with a first color sample 410 and a plurality of neighboring luminance samples 404X of the first luminance sample 404C, and each candidate luminance set 404B includes a plurality of corresponding luminance samples (e.g., two neighboring luminance samples 404X and the first luminance sample 404C), the positions of which are symmetrical with respect to a first position of the first luminance sample 404C. In some embodiments, each luminance sample 404 located within the filter range 480 is used for at least one candidate luminance set in the plurality of candidate luminance sets 404B. In other words, the filter range 480 includes a first number (F) of neighboring luminance samples of the first luminance sample 404C, and the plurality of candidate luminance sets 404B includes a second number of candidate luminance sets 404B. The second number (e.g., 7 or 6) is equal to half of the first number (e.g., 14 or 12).
[0097] Reference Figure 6A In some embodiments, the filter range 480 has a rectangular range shape. The filter range 480 has a predefined size based on one or more range constraints, and said one or more range constraints include at least one of the number of horizontal neighbor samples (N) and the number of vertical samples (M). In this example, the number of horizontal neighbor samples (N) is equal to 2, and the number of vertical samples (M) is equal to 1. The filter range 480 includes 15 luminance samples, which include a first luminance sample 404C and 14 neighboring luminance samples 404X. The filter range 480 includes 7 candidate luminance sets 404B. Each candidate luminance set 404B includes a pair of neighboring luminance samples 404X, the positions of which are symmetrical with respect to a first position of the first luminance sample 404C. For example, the 7 candidate luminance sets 404B include luminance sample pairs C0, C1, C2, C3, C4, C5, and C6.
[0098] Reference Figure 6B In some embodiments, the filter range 480 has a diamond-shaped range. The number of horizontal neighbor samples (N) is equal to 2, and the number of vertical samples (M) is equal to 2. The filter range 480 includes 13 luminance samples, which include a first luminance sample 404C and 12 neighboring luminance samples 404X. The filter range 480 includes 5 candidate luminance sets 404B. Each candidate luminance set 404B includes a pair of neighboring luminance samples 404X, the positions of which are symmetrical with respect to a first position of the first luminance sample 404C. For example, the 5 candidate luminance sets 404B include luminance sample pairs C0, C1, C2, C3, C4, and C5.
[0099] Figure 7A and Figure 7B A plurality of example candidate luminance sets 404B identified in another filter range 480 having a rectangular shape according to some embodiments is shown. The filter range 480 includes a first luminance sample 404C co-located with a first color sample 410 and a plurality of neighboring luminance samples 404X of the first luminance sample 404C, and each candidate luminance set 404B includes a plurality of corresponding luminance samples (e.g., two neighboring luminance samples 404X and the first luminance sample 404C), the positions of which are symmetrical with respect to a first position of the first luminance sample 404C. The filter range 480 has a predefined size based on one or more range constraints, and the one or more range constraints include at least one of a horizontal neighboring sample number (N) and a vertical sample number (M). The horizontal neighboring sample number (N) is equal to 3, and the vertical sample number (M) is equal to 2. The filter range 480 includes 35 luminance samples, the 35 luminance samples including the first luminance sample 404C and 34 neighboring luminance samples 404X.
[0100] Reference Figure 7A and Figure 7BIn some implementations, each of a subset (e.g., less than all) of the luminance samples 404 located within the filter range 480 is used for at least one candidate luminance set among a plurality of candidate luminance sets 404B. The remaining subsets of luminance samples are not used for any candidate luminance set among the plurality of candidate luminance sets 404B. In one example, each neighboring luminance sample in the plurality of candidate luminance sets 404B (e.g., candidate luminance pairs C0, C1, ..., C8) has a sample position (x, y) relative to the first luminance sample 404C, where x and y are two integers and their sum is odd. In another example, each neighboring luminance sample in the plurality of candidate luminance sets 404B (e.g., candidate luminance pairs C0, C1, ..., C7) has a sample position (x, y) relative to the first luminance sample 404C, where x and y are two integers and their sum is even.
[0101] Figure 8A and Figure 8B A plurality of example candidate luminance sets 404B identified in a filter range 480 having a diamond shape, according to some embodiments, are shown. The filter range 480 includes a first luminance sample 404C co-located with a first color sample 410 and a plurality of neighboring luminance samples 404X of the first luminance sample 404C, and each candidate luminance set 404B includes a plurality of corresponding luminance samples (e.g., two neighboring luminance samples 404X and the first luminance sample 404C), the positions of which are symmetrical with respect to a first position of the first luminance sample 404C. The filter range 480 has a predefined size based on one or more range constraints. (Refer to...) Figure 8A In some implementations, the number of horizontally adjacent samples (N) and the number of vertical samples (M) are equal to 3. (See also...) Figure 8B In some implementations, the number of horizontal neighbor samples (N) and the number of vertical samples (M) are equal to 2.
[0102] In some implementations, each of a subset (e.g., less than all) of the luminance samples 404 located within the filter range 480 is used for at least one candidate luminance set in a plurality of candidate luminance sets 404B, while subsets of the remaining luminance samples are not used for any candidate luminance set in the plurality of candidate luminance sets 404B. In one example, each neighboring luminance sample in the plurality of candidate luminance sets 404B (e.g., candidate luminance pairs C0, C1, ..., and C7) has a sample position (x, y) relative to a first luminance sample 404C, where x and y are two integers and the sum of the two integers is odd. In another example, each neighboring luminance sample in the plurality of candidate luminance sets 404B (e.g., candidate luminance pairs C0, C1, ..., and C3) has a sample position (x, y) relative to a first luminance sample 404C, where x and y are two integers and the sum of the two integers is even.
[0103] Figures 9A to 9D These are four example filter shapes 900, 920, 940, and 960 according to some implementations, combining more than two neighboring luminance samples 404X in CCSO mode. Decoder 122 receives a video bitstream 116 from encoder 106. Video bitstream 116 includes the current image frame and a first syntax element 408 for Cross Component Sample Offset (CCSO) mode, indicating whether a first sample offset 406 of the first color sample 410 of the current image frame is generated based on one or more luminance samples 404. When the first syntax element 408 indicates that CCSO mode 442 is enabled, decoder 122 identifies multiple candidate luminance sets 404B in filter range 480. Filter range 480 includes a first luminance sample 404C co-located with the first color sample 410 and multiple neighboring luminance samples 404X of the first luminance sample 404C. Each candidate luminance set 404B includes multiple corresponding luminance samples 404X, the positions of which are symmetrical with respect to a first position of the first luminance sample 404C. Each luminance sample 404 located within filter range 480 is used for at least one candidate luminance set from a plurality of candidate luminance sets 404B. A target luminance sample set 404A is selected from the plurality of candidate luminance sets 404B. For example, the target luminance sample set 404A includes a first luminance sample 404C co-located with the first color sample 410 and a plurality of corresponding luminance samples 404X of the first luminance sample 404C. A loop filter 424 is applied to combine the target luminance sample set 404A to generate a first sample offset 406 for the first color sample 410. The decoder 122 reconstructs the current image frame by adjusting the first color sample 410 based at least on the first sample offset 406.
[0104] The target luminance sample set 404A includes a first target luminance sample subset 902 and a second target luminance sample subset 904. When a loop filter is applied to combine the target luminance sample set 404A, including the first luminance sample 404C, to generate a first sample offset 406 of the first color sample 410, the first target luminance sample subset 902 is combined to provide a first target luminance value, and the second target luminance sample subset 904 is combined to provide a second target luminance value. The first target luminance value, the second target luminance value, and the first luminance sample 404C are combined to generate the first sample offset 406. (Refer to...) Figure 9A In some embodiments, the first target brightness sample subset 902 includes a top-left brightness sample, a top brightness sample, and a top-right brightness sample, and the second target brightness sample subset 904 includes a bottom-left brightness sample, a bottom brightness sample, and a bottom-right brightness sample. (See also...) Figure 9B In some embodiments, the first target brightness sample subset 902 includes a top-left brightness sample, a left-side brightness sample, and a bottom-left brightness sample, and the second target brightness sample subset 904 includes a top-right brightness sample, a right-side sample, and a bottom-right brightness sample. (See also...) Figure 9C In some embodiments, the first target brightness sample subset 902 includes a top-left brightness sample, a left-side brightness sample, and a top brightness sample, and the second target brightness sample subset 904 includes a right-side brightness sample, a bottom sample, and a bottom-right brightness sample. (See also...) Figure 9D In some implementations, the first target brightness sample subset 902 includes the upper right brightness sample, the top brightness sample, and the right brightness sample, and the second target brightness sample subset 904 includes the lower left brightness sample, the left brightness sample, and the bottom brightness sample.
[0105] In some implementations, a predefined set of filter fusion schemes is identified. The video bitstream 116 also includes a fourth syntax element 428 ( Figure 4The fourth syntax element 428 includes a filter fusion flag that selects a filter fusion scheme from a predefined set of filter fusion schemes to identify the first target luminance sample subset 902 and the second target luminance sample subset 904 in the target luminance sample set 404A. Furthermore, in some embodiments, the first target luminance sample subset 902, the second target luminance sample subset 904, and the first luminance sample 404C are arranged according to a target filter shape 440. The video bitstream 116 includes a third syntax element 426 that includes a filter shape index that selects a filter shape from a predefined set of filter shapes as the target filter shape 440. The third syntax element 426 is signaled in the video bitstream 116 at the sequence level, frame level, slice level, superblock level, or coded block level. The video bitstream 116 also includes a fourth syntax element 428 that is signaled after the third syntax element 426. Additionally, in some implementations, the third syntax element 426 is signaled at the frame level, while the fourth syntax element 428 is signaled at the coded block level.
[0106] Figure 10 This is a flowchart illustrating an example method 1000 for encoding and decoding video according to some embodiments. Method 1000 can be implemented in a computing system (e.g., Figure 1 The method 1000 is executed at a server system 112, source device 102, or electronic device 120, which has a control circuitry and a memory storing instructions for execution by the control circuitry. In some embodiments, method 1000 is applied in conjunction with one or more video codecs, including but not limited to H.264, H.265 / HEVC, H.266 / VVC, AV1, and AVS / AVS2 / AVS3. The cross-component offset filtering method is implemented based on an edge-preserving loop filter that uses reconstructed samples (e.g., luminance sample 404) to determine a sample offset 406 for luminance sample 404 or chrominance sample 402. Luminance sample 404 may be classified based on band offset (e.g., depending on luminance sample 404) or edge offset (e.g., depending on the gradient or difference between a first luminance sample 404C and a neighboring luminance sample 404X). A luminance sample 404 located at a position defined by the filter shape is used to calculate a sample offset 406 for a color sample (e.g., a first luminance sample 404C or a first chromaticity sample 402C). A novel filter shape is proposed for cross-component sample offset classification based on neighboring samples. A filter refers to a pair / set of symmetric or asymmetric samples used to calculate the classifier and offset. The filter shape refers to the shape of all available "filters" within the filter range constraint.
[0107] In some implementations, the filter shape is defined by a specified range constraint around a co-located reconstructed sample "C" (e.g., the first luminance sample 404C). The upper and lower range constraints are denoted by M, while the left and right range constraints are denoted by N. M and N can be the same or different, and M and N are selected from the set {1, 2, 3, ..., L}, where L is a number smaller than the predefined constraints. The filter range (e.g., ...) Figure 4 The shape of the filter range (480) can be rectangular, rhomboid, or any other symmetrical form. Any combination of filter options within the filter range (e.g., filter range 480) produces the filter shape.
[0108] In the example, the filter range 480 is rectangular, where M=2 and N=3 ( Figure 5A The allowed neighbor samples 404B are subject to specific constraints: for rectangular filters, ,and In another example, the filter range 480 has a diamond shape, where M=3 and N=3 ( Figure 5B For a rhombus shape, the constraint is... , where x and y are integers.
[0109] In some implementations, each filter option within the filter range 480 (e.g., candidate luminance set 404B) includes two neighboring samples 404X symmetrical about the center of the filter shape (e.g., first luminance sample 404C). Any combination of these filter options within the filter range 480 can be an option of a loop filter. If there are F neighboring samples 404X within the filter range 480, the combination can include any number of filter options less than or equal to F / 2.
[0110] In some implementations, for rectangular filter ranges (e.g., Figures 7A to 7B The range 480 is used only as a subset of the range (e.g., less than the entire range) to provide the filter shape for the loop filter. In the example, each candidate brightness set 404B includes a rectangular range shape 480 (…). Figure 7A The neighboring samples 404X with position (x, y) within the range. One of the horizontal index x and the vertical index y is even, while the other index is odd. In another example, each candidate brightness set 404B includes a rectangular range shape 480 ( Figure 7B 404X is the nearest neighbor sample with position (x, y) within a given area. The horizontal index (or coordinate) x and the vertical index y are either both even or both odd.
[0111] In some implementations, for the diamond filter range (e.g., Figures 8A to 8BThe range 480 is used only as a subset of the range (e.g., less than the entire range) to provide the filter shape for the loop filter. In the example, each candidate brightness set 404B includes a rectangular filter range 480 ( Figure 8A The set of neighboring samples 404X with positions (x, y) within the range of 480. One of the horizontal index x and the vertical index y is even, while the other is odd. In another example, each candidate brightness set 404B includes a rectangular filter range 480 (… Figure 8B 404X is a neighboring sample with position (x, y) within a given area. The horizontal index x and the vertical index y are either both even or both odd.
[0112] In some implementations, a set of filter shapes can be predefined, and the index of the filter shape can be signaled (e.g., via third syntax element 426) at the sequence level, frame level, or filter unit level.
[0113] In some implementations, the filter range set is predefined, and the filter shape set within the filter shape is predefined.
[0114] In some implementations, for example, the filter range shape and filter shape index are signaled at the sequence level, frame level, or filter unit level via second syntax element 422 and third syntax element 426, respectively.
[0115] In some implementations, the filter range shape and filter shape can be derived without signal notification / analysis.
[0116] In some implementations, the filter shape index is first signaled, for example, by a second syntax element 422, and after the first shape index is signaled, the filter index within the filter shape is signaled, for example, by a third syntax element 426.
[0117] In some implementations, multiple samples can be involved in a single filter option to compute the classifier and / or offset.
[0118] In some implementations, refer to Figure 9A For the vertical direction, the average value of the first luminance sample subset 902 is determined, and the average value of the second luminance sample subset 904 is also determined. In the loop filter, the average value of the first luminance sample subset 902, the average value of the second luminance sample subset 904, and the first luminance sample 404C are combined to determine the first sample offset 406 of the first color sample 410.
[0119] In some implementations, refer to Figure 9CFor the diagonal direction, the average value of the first luminance sample subset 902 is determined, and the average value of the second luminance sample subset 904 is also determined. In the loop filter, the average value of the first luminance sample subset 902, the average value of the second luminance sample subset 904, and the first luminance sample 404C are combined to determine the first sample offset 406 of the first color sample 410.
[0120] In some implementations, a flag named `filter_fusion_flag` is signaled to indicate whether one or more pairs of samples (e.g., target brightness sample set 404A) are used to compute the classifier and / or offset in cross-component sample offset processing. Furthermore, in some implementations, this flag is signaled after the filter option index is signaled at the sequence level, frame level, slice level, superblock level, or coded block level. In some implementations, the filter index and `filter_fusion_flag` are signaled at different levels. In the example, the filter index is signaled at the frame level, but `filter_fusion_flag` is signaled at the coded block level.
[0121] Although Figure 10 Some logical stages are shown in a specific order, but stages that are not in order can be reordered, and other stages can be combined or split. Some reorderings or other groupings not specifically mentioned will be obvious to those skilled in the art, and therefore the orderings and groupings presented herein are not exhaustive. Furthermore, it should be understood that the stages can be implemented in hardware, firmware, software, or any combination thereof.
[0122] While this disclosure has described several exemplary embodiments, there are variations, substitutions, and various alternative equivalents that fall within the scope of this disclosure. Therefore, it will be appreciated that those skilled in the art will be able to design a variety of systems and methods that, while not expressly shown or described herein, embody the principles of this disclosure and are therefore within its spirit and scope.
[0123] Now let's turn to some example implementations.
[0124] (A1) In some embodiments, a method 1000 for decoding video data includes: receiving (operation 1002) a video bitstream including a current image frame and a first syntax element for a Cross-Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset of a first color sample of the current image frame is generated based on one or more luminance samples; when the first syntax element indicates that the CCSO mode is enabled (operation 1004): identifying (operation 1006) a plurality of candidate luminance sets in a filter range, wherein the filter range includes (operation 1008) a first luminance sample co-located with the first color sample and Multiple neighboring brightness samples of a first brightness sample, and each candidate brightness set includes (operation 1010) multiple corresponding brightness samples whose positions are symmetrical with respect to a first position of the first brightness sample, and wherein each brightness sample within the filter range is used (operation 1012) for at least one candidate brightness set in the multiple candidate brightness sets; (operation 1014) a target brightness sample set is selected from the multiple candidate brightness sets; and (operation 1016) a loop filter is applied to combine the target brightness sample sets to generate a first sample offset of a first color sample; and (operation 1018) the current image frame is reconstructed at least by adjusting the first color sample based on the first sample offset.
[0125] (A2) In some embodiments of A1, the filter range has a predetermined size based on one or more range limits, and the one or more range limits include at least one of the number of horizontal neighboring samples (N) and the number of vertical samples (M).
[0126] (A3) In some embodiments of A1 or A2, the first range of the filter range is rectangular in shape, and each luminance sample in the plurality of candidate luminance sets has a sample position (x, y) relative to the first luminance sample, where x and y are integers. and .
[0127] (A4) In some embodiments of A3, the first range of the filter range is rhomboid in shape, and each luminance sample in the plurality of candidate luminance sets has a sample position (x, y) relative to the first luminance sample, where x and y are integers and .
[0128] (A5) In some embodiments of A1 to A4, the filter range includes neighboring luminance samples of a first luminance sample totaling a first number (F), and the plurality of candidate luminance sets includes a second number of candidate luminance sets, the second number being equal to half of the first number. (A6) In some embodiments of A1 to A5, the filter range has a first range shape, and the method further includes: identifying a predefined range shape set, wherein the video bitstream further includes a second syntax element including a range shape index, the range shape index selecting a range shape from the predefined range shape set as the first range shape at one of the following levels: sequence level, frame level, or filter unit level.
[0129] (A7) In some embodiments of A1 to A6, the target luminance sample set is arranged according to the first filter shape. The method further includes: identifying a predefined filter shape set, wherein the video bitstream further includes a third syntax element, the third syntax element including a filter shape index, which selects a filter shape from the predefined filter shape set as the first filter shape at one of the following levels: sequence level, frame level, or filter unit level.
[0130] (A8) In some embodiments of A1 to A7, the filter range has a first range shape, and the target luminance sample set is arranged according to the first filter shape. The method further includes: deriving the first range shape, wherein the first range shape is not signaled in the video bitstream. (A9) In some embodiments of A1 to A8, the target luminance sample set and the first luminance sample are arranged according to the first filter shape, and the first filter shape is not signaled in the video bitstream. The method further includes: deriving the first filter shape.
[0131] (A10) In some embodiments of A1 to A9, the filter range has a first range shape, and the video bitstream includes: a second syntax element including a range shape index and a third syntax element including a filter shape index, the method further including: identifying a predefined range shape set; selecting a first range shape from the predefined range shape set based on the range shape index; identifying a predefined filter shape set associated with the filter range having the first range shape; and selecting a first filter shape from the predefined filter shape set based on the filter shape index, wherein a target luminance sample set and a first luminance sample are arranged according to the first filter shape.
[0132] (A11) In some implementations of A1 to A10, for each candidate luminance set, the plurality of corresponding luminance samples include more than two corresponding luminance samples.
[0133] (A12) In some embodiments of A1 to A11, the target brightness sample set includes a first target brightness sample subset and a second target brightness sample subset, and the first sample offset of applying a loop filter to combine the target brightness sample set including the first brightness sample to generate the first color sample further includes: combining the first target brightness sample subset to provide a first target brightness value; combining the second target brightness sample subset to provide a second target brightness value; and combining the first target brightness value, the second target brightness value, and the first brightness sample to generate the first sample offset.
[0134] (A13) In some embodiments of A12, the first target brightness sample subset includes the top left brightness sample, the top brightness sample and the top right brightness sample, and the second target brightness sample subset includes the bottom left brightness sample, the bottom brightness sample and the bottom right brightness sample.
[0135] (A14) In some embodiments of A12 or A13, the first target brightness sample subset includes the upper left brightness sample, the left side brightness sample and the lower left brightness sample, and the second target brightness sample subset includes the upper right brightness sample, the right side sample and the lower right brightness sample.
[0136] (A15) In some embodiments of any of A12 to A14, the first target brightness sample subset includes the top left brightness sample, the left side brightness sample and the top brightness sample, and the second target brightness sample subset includes the right side brightness sample, the bottom sample and the bottom right brightness sample.
[0137] (A16) In some embodiments of any of A12 to A15, the first target brightness sample subset includes the upper right brightness sample, the top brightness sample and the right brightness sample, and the second target brightness sample subset includes the lower left brightness sample, the left brightness sample and the bottom brightness sample.
[0138] (A17) In some embodiments of any of A12 to A16, the method further includes: identifying a predefined set of filter fusion schemes, wherein the video bitstream further includes a fourth syntax element including a filter fusion flag, the filter fusion flag selecting a filter fusion scheme from the predefined set of filter fusion schemes to identify a first subset of target luminance samples and a second subset of target luminance samples in the target luminance sample set.
[0139] (A18) In some embodiments of A17, the first target luminance sample subset, the second target luminance sample subset, and the first luminance sample are arranged according to the target filter shape. The method further includes: identifying a third syntax element in the video bitstream, the third syntax element including a filter shape index, the filter shape index selecting a filter shape from a predefined set of filter shapes as the target filter shape, wherein the third syntax element is signaled in the video bitstream at the sequence level, frame level, slice level, superblock level, or coded block level; and identifying a fourth syntax element in the video bitstream, wherein the fourth syntax element is signaled after the third syntax element.
[0140] (A19) In some implementations of A18, the third syntax element is signaled at the frame level, while the fourth syntax element is signaled at the coded block level.
[0141] (A20) In some implementations of any of A12 to A19, each candidate luminance set includes two corresponding luminance samples. (A21) A method for decoding video data includes: receiving a video bitstream including a current image frame and a first syntax element for a Cross-Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset of a first color sample of the current image frame is generated based on one or more luminance samples; when the first syntax element indicates that the CCSO mode is enabled: identifying a plurality of candidate luminance sets in a filter range, wherein the filter range includes a first luminance sample co-located with a first color sample and a plurality of neighboring luminance samples of the first luminance sample, and each candidate luminance set includes a plurality of corresponding luminance samples, the positions of the plurality of corresponding luminance samples being symmetrical with respect to a first position of the first luminance sample; selecting a target luminance sample set from the plurality of candidate luminance sets; applying a loop filter to combine the target luminance sample set to generate a first sample offset of the first color sample; and reconstructing the current image frame at least by adjusting the first color sample based on the first sample offset; wherein each neighboring luminance sample in the plurality of candidate luminance sets has a sample position (x, y) relative to the first luminance sample, where x and y are two integers and the sum of the two integers is an odd number.
[0142] (A22) A method for decoding video data includes: receiving a video bitstream including a current image frame and a first syntax element for a Cross-Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset of a first color sample of the current image frame is generated based on one or more luminance samples; when the first syntax element indicates that the CCSO mode is enabled: identifying a plurality of candidate luminance sets in a filter range, wherein the filter range includes a first luminance sample co-located with a first color sample and a plurality of neighboring luminance samples of the first luminance sample, and each candidate luminance set includes a plurality of corresponding luminance samples, the positions of the plurality of corresponding luminance samples being symmetrical with respect to a first position of the first luminance sample; selecting a target luminance sample set from the plurality of candidate luminance sets; applying a loop filter to combine the target luminance sample set to generate a first sample offset of the first color sample; and reconstructing the current image frame at least by adjusting the first color sample based on the first sample offset; wherein each neighboring luminance sample in the plurality of candidate luminance sets has a sample position (x, y) relative to the first luminance sample, where x and y are two integers and the sum of the two integers is even.
[0143] (A23) In some implementations, a video coding method is implemented. The method includes: receiving video data including a current image frame; encoding the current image frame; transmitting the encoded current image frame via a video bitstream; and signaling via the video bitstream a first syntax element for a Cross-Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset of a first color sample of the current image frame is generated based on one or more luminance samples; wherein, when the CCSO mode is enabled, a target luminance sample set is selected from a plurality of candidate luminance sets within a filter range, and the target luminance sample set and a first luminance sample co-located with a first color sample are combined according to a loop filter to generate a first sample offset of the first color sample; wherein the filter range includes the first luminance sample and a plurality of neighboring luminance samples of the first luminance sample, and each candidate luminance set includes a plurality of corresponding luminance samples whose positions are symmetrical with respect to a first position of the first luminance sample; and wherein each luminance sample within the filter range is used for at least one candidate luminance set in the plurality of candidate luminance sets.
[0144] (A24) In some embodiments, a bitstream conversion method is implemented. The method includes: obtaining a source video sequence including a current image frame; and performing a conversion between the source video sequence and a video bitstream, wherein the video bitstream includes: the current image frame; and a first syntax element for a Cross-Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset of a first color sample of the current image frame is generated based on one or more luminance samples; wherein, when the CCSO mode is enabled, a target luminance sample set is selected from a plurality of candidate luminance sets within a filter range, and the target luminance sample set and a first luminance sample co-located with a first color sample are combined according to a loop filter to generate a first sample offset of the first color sample; wherein the filter range includes the first luminance sample and a plurality of neighboring luminance samples of the first luminance sample, and each candidate luminance set includes a plurality of corresponding luminance samples whose positions are symmetrical with respect to a first position of the first luminance sample; and wherein each luminance sample within the filter range is used for at least one candidate luminance set in the plurality of candidate luminance sets.
[0145] On the other hand, some implementations include a computing system (e.g., server system 112) that includes a control circuitry system (e.g., control circuitry system 302) and a memory (e.g., memory 314) coupled to the control circuitry system, the memory storing one or more sets of instructions configured to be executed by the control circuitry system, the set of one or more sets of instructions including instructions for performing any of the methods described herein (e.g., A1 to A24 above).
[0146] In another aspect, some implementations include a non-transitory computer-readable storage medium storing one or more sets of instructions for execution by a control circuitry of a computing system, the set of instructions including instructions for performing any of the methods described herein (e.g., A1 to A24 above).
[0147] The proposed methods can be used individually or in any combination in any order. Furthermore, each of the method (or implementation), encoder, and decoder can be implemented by a processing circuitry system (e.g., one or more processors or one or more integrated circuits). For example, one or more processors execute a program stored on a non-transitory computer-readable medium. In the following text, the term "block" can be interpreted as a prediction block, encoding / decoding block, or encoding / decoding unit, i.e., a CU.
[0148] It will be understood that while the terms “first,” “second,” etc., may be used in this document to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another.
[0149] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the claims. As used in the description of embodiments and appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or,” as used herein, refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.
[0150] As used herein, the term "if" may be interpreted, depending on the context, as meaning "when the prerequisite is true," "after the prerequisite is true," "in response to determining that the prerequisite is true," "based on the determination that the prerequisite is true," or "in response to detecting that the prerequisite is true." Similarly, the phrases "if it is determined [the prerequisite is true]," "if [the prerequisite is true]," or "when [the prerequisite is true]" may be interpreted, depending on the context, as meaning "after determining that the prerequisite is true," "in response to determining that the prerequisite is true," "based on the determination that the prerequisite is true," "after detecting that the prerequisite is true," or "in response to detecting that the prerequisite is true."
[0151] For illustrative purposes, the foregoing description has been described with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the claims to the precise forms disclosed. In view of the above teachings, many modifications and variations are possible. The embodiments were chosen and described in order to best illustrate the principles of operation and practical application, thereby enabling others skilled in the art to implement them.
Claims
1. A method for decoding video data, the method comprising: Receive a video bitstream, the video bitstream including a current image frame and a first syntax element for a cross-component sample offset (CCSO) mode, the first syntax element indicating whether a first sample offset of a first color sample of the current image frame is generated based on one or more luminance samples; When the first syntax element indicates that the CCSO mode is enabled: Multiple candidate brightness sets are identified within a filter range, wherein the filter range includes a first brightness sample co-located with the first color sample and multiple neighboring brightness samples of the first brightness sample, and each candidate brightness set includes multiple corresponding brightness samples whose positions are symmetrical with respect to a first position of the first brightness sample, and wherein each brightness sample located within the filter range is used for at least one candidate brightness set in the multiple candidate brightness sets. Select a target brightness sample set from the plurality of candidate brightness sets; and A loop filter is applied to combine the target brightness sample set to generate the first sample offset of the first color sample; and The current image frame is reconstructed by adjusting the first color sample based on the first sample offset.
2. The method according to claim 1, wherein, The filter range has a predefined size based on one or more range limits, and the one or more range limits include at least one of the number of horizontal neighboring samples (N) and the number of vertical samples (M).
3. The method according to claim 2, wherein, The first range of the filter range has a rectangular shape, and each brightness sample in the plurality of candidate brightness sets has a sample position (x, y) relative to the first brightness sample, where x and y are integers. and .
4. The method according to claim 2, wherein, The first range of the filter range has a rhombus shape, and each brightness sample in the plurality of candidate brightness sets has a sample position (x, y) relative to the first brightness sample, where x and y are integers. .
5. The method according to claim 1, wherein, The filter range includes neighboring brightness samples of the first brightness sample, which is a first number (F), and the plurality of candidate brightness sets include a second number of candidate brightness sets, which is equal to half of the first number.
6. The method according to claim 1, wherein, The filter range has a first range shape, and the method further includes: Identify a predefined range shape set, wherein the video bitstream further includes a second syntax element, the second syntax element including a range shape index, the range shape index selecting a range shape from the predefined range shape set as the first range shape at one of the following levels: sequence level, frame level, or filter unit level.
7. The method according to claim 1, wherein, The method further includes arranging the target brightness sample set according to the shape of the first filter: Identify a predefined set of filter shapes, wherein the video bitstream further includes a third syntax element, the third syntax element including a filter shape index, the filter shape index selecting one of the filter shapes in the predefined set as the first filter shape at one of the following levels: sequence level, frame level, or filter unit level.
8. The method according to claim 1, wherein, The filter range has a first range shape, and the target brightness sample set is arranged according to the first filter shape. The method further includes: The first range shape is derived based on the reference region of the current coded block, wherein the first range shape is not signaled in the video bitstream.
9. The method according to claim 1, wherein, The method further includes arranging the target luminance sample set and the first luminance sample according to the first filter shape, and not notifying the first filter shape in the video bitstream by a signal; The shape of the first filter is derived based on the reference region of the current coding block.
10. The method according to claim 1, wherein, The filter range has a first range shape, and the video bitstream includes: a second syntax element including a range shape index and a third syntax element including a filter shape index; the method further includes: Identify a predefined range of shape sets; The first range shape is selected from the predefined range shape set based on the range shape index; Identify a predefined set of filter shapes associated with the filter range having the first range shape; and A first filter shape is selected from the predefined filter shape set based on the filter shape index, wherein the target brightness sample set and the first brightness sample are arranged according to the first filter shape.
11. The method according to claim 1, wherein, For each candidate brightness set, the plurality of corresponding brightness samples include more than two corresponding brightness samples.
12. The method according to claim 1, wherein, The target brightness sample set includes a first target brightness sample subset and a second target brightness sample subset, and the first sample offset for applying the loop filter to combine the target brightness sample set including the first brightness sample to generate the first color sample further includes: Combine the first target brightness sample subset to provide a first target brightness value; Combine the second target brightness sample subset to provide a second target brightness value; The first target brightness value, the second target brightness value, and the first brightness sample are combined to generate the first sample offset.
13. The method according to claim 12, wherein, The first target brightness sample subset includes the upper left brightness sample, the top brightness sample, and the upper right brightness sample, and the second target brightness sample subset includes the lower left brightness sample, the bottom brightness sample, and the lower right brightness sample.
14. The method according to claim 12, wherein, The first target brightness sample subset includes the upper left brightness sample, the left side brightness sample, and the lower left brightness sample, and the second target brightness sample subset includes the upper right brightness sample, the right side sample, and the lower right brightness sample.
15. The method according to claim 12, wherein, The first target brightness sample subset includes the upper left brightness sample, the left side brightness sample, and the top brightness sample, and the second target brightness sample subset includes the right side brightness sample, the bottom sample, and the lower right brightness sample.
16. The method according to claim 12, wherein, The first target brightness sample subset includes the upper right brightness sample, the top brightness sample, and the right brightness sample, and the second target brightness sample subset includes the lower left brightness sample, the left brightness sample, and the bottom brightness sample.
17. The method according to claim 12, further comprising: Identify a predefined filter fusion scheme set, wherein the video bitstream further includes a fourth syntax element, the fourth syntax element including a filter fusion flag, the filter fusion flag selecting a filter fusion scheme from the predefined filter fusion scheme set to identify the first target brightness sample subset and the second target brightness sample subset in the target brightness sample set.
18. The method according to claim 17, wherein, The method further includes arranging the first target brightness sample subset, the second target brightness sample subset, and the first brightness sample according to the shape of the target filter. Identifying a third syntax element in the video bitstream, the third syntax element including a filter shape index, the filter shape index selecting a filter shape from a predefined set of filter shapes as the target filter shape, wherein the third syntax element is signaled in the video bitstream at the sequence level, frame level, slice level, superblock level, or coding block level; and The fourth syntax element is identified in the video bitstream, wherein the fourth syntax element is signaled after the third syntax element.
19. A computing system, comprising: Control circuit system; as well as A memory storing one or more programs configured to be executed by the control circuitry system, the one or more programs further comprising instructions for performing the following operations: Receive video data including the current image frame; Encode the current image frame; The encoded current image frame is transmitted via video bitstream; as well as The video bitstream is signaled to a first syntax element for the Cross-Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset of the first color sample of the current image frame is generated based on one or more luminance samples; When the CCSO mode is enabled, a target brightness sample set is selected from multiple candidate brightness sets within the filter range, and the target brightness sample set and the first brightness sample co-located with the first color sample are combined according to the loop filter to generate the first sample offset of the first color sample. The filter range includes the first brightness sample and multiple neighboring brightness samples of the first brightness sample, and each candidate brightness set includes multiple corresponding brightness samples, the positions of which are symmetrical with respect to a first position of the first brightness sample; and Each luminance sample within the filter range is used in at least one candidate luminance set from the plurality of candidate luminance sets.
20. A non-transitory computer-readable storage medium storing one or more programs for execution by a control circuitry of a computing system, the one or more programs comprising instructions for performing the following operations: Obtain the source video sequence including the current image frame; as well as Perform the conversion between the source video sequence and the video bitstream, wherein, The video bitstream includes: The current image frame; and A first syntax element for the Cross Component Sample Offset (CCSO) mode, the first syntax element indicating whether a first sample offset of the first color sample of the current image frame is generated based on one or more luminance samples; When the CCSO mode is enabled, a target brightness sample set is selected from multiple candidate brightness sets within the filter range, and the target brightness sample set and the first brightness sample co-located with the first color sample are combined according to the loop filter to generate the first sample offset of the first color sample. The filter range includes the first brightness sample and multiple neighboring brightness samples of the first brightness sample, and each candidate brightness set includes multiple corresponding brightness samples, the positions of which are symmetrical with respect to a first position of the first brightness sample; and Each luminance sample within the filter range is used in at least one candidate luminance set from the plurality of candidate luminance sets.
