Chroma coding enhancement in cross-component sample adaptive offset with virtual boundary

Abstract

An electronic device that performs a method of decoding a video signal. The method includes receiving, from a video signal, a picture frame including a first component and a second component; determining, from a set of one or more samples of the first component associated with a respective sample of the second component, a classifier for the second component; determining, from the classifier, whether to modify a value of the respective sample of the second component of a current block of the picture frame within a virtual boundary; in response to a determination to modify the value of the respective sample of the second component of the current block according to the classifier, determining, from the classifier, a sample offset for the respective sample of the second component; and modifying the value of the respective sample of the second component based on the determined sample offset.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 082,356, filed September 23, 2020, entitled “Cross-component Sample Adaptive Offset,” and U.S. Provisional Application No. 63 / 086,504, filed October 1, 2020, entitled “Cross-component Sample Adaptive Offset,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application generally relates to video encoding / decoding and compression. More specifically, this application relates to methods and apparatus for improving the efficiency of chroma encoding / decoding. Background Technology

[0004] Various electronic devices (such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, video streaming devices, etc.) support digital video. These electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards. Some well-known video codec standards include Universal Video Codec (VVC), High Efficiency Video Codec (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Codec (AVC, also known as H.264 or MPEG-4 Part 10), jointly developed by ISO / IEC MPEG and ITU-T VECG. AOMedia Video1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to its previous standard VP9. Audio and Video Codecs (AVS) are another series of video compression standards developed by the China Audio and Video Codec Standards Working Group, referring to digital audio and digital video compression standards.

[0005] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove inherent redundancy in video data. For block-based video coding and decoding, video frames are divided into one or more stripes, each strip containing multiple video blocks, also known as coding tree units (CTUs). Each CTU may contain a coding unit (CU) or be recursively divided into smaller CUs until a preset minimum CU size is reached. Each CU (also known as a leaf CU) contains one or more transform units (TUs), and each CU also contains one or more prediction units (PUs). Each CU may be coded in intra-frame, inter-frame, or IBC modes. Video blocks in intra-frame coded (I) strips of a video frame are encoded using spatial prediction relative to reference samples in adjacent blocks within the same video frame. Video blocks in inter-frame coded (P or B) strips of a video frame may use spatial prediction relative to reference samples in adjacent blocks within the same video frame or temporal prediction relative to reference samples in other previous and / or future reference video frames.

[0006] A prediction block for the current video block to be encoded is generated based on spatial or temporal predictions of previously encoded reference blocks (such as adjacent blocks). The process of finding the reference block can be accomplished using a block-matching algorithm. The residual data representing the pixel difference between the current block to be encoded and the prediction block is called the residual block or prediction error. Inter-frame coded blocks are encoded based on the motion vector pointing to the reference block in the reference frame that forms the prediction block, and the residual block. The process of determining this motion vector is often called motion estimation. Intra-frame coded blocks are encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain, such as the frequency domain, to generate residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to generate a one-dimensional vector of transform coefficients, which is then entropy-encoded into the video bitstream for further compression.

[0007] The encoded video bitstream is then stored in a computer-readable storage medium (such as flash memory) for access by another electronic device with digital video capabilities or directly transmitted to that electronic device via wired or wireless means. The electronic device then performs video decompression (the reverse of the video compression process described above) by, for example, parsing the encoded video bitstream to obtain semantic elements from it and reconstructing the digital video data from the encoded video bitstream to its original format based at least in part on the semantic elements obtained from the bitstream, and presents the reconstructed digital video data on the display of the electronic device.

[0008] As digital video quality progresses from high definition to 4K×2K and even 8K×4K, the amount of video data to be encoded / decoded grows exponentially. Maintaining image quality while efficiently encoding / decoding video data remains a continuous challenge. Summary of the Invention

[0009] This application describes implementations relating to video data encoding and decoding, and more specifically, methods and apparatus for improving the encoding and decoding efficiency of chroma encoding and decoding, including improving encoding and decoding efficiency by exploring cross-component relationships between luminance and chroma components.

[0010] According to a first aspect of this application, a method for decoding a video signal includes:

[0011] Receive image frames, including the first and second components, from the video signal;

[0012] In response to the current coding tree unit (CTU) not being located in the bottom row of the CTU of a frame, strip, tile, sub-picture, or patch, a set of one or more samples of the first component is determined based on the positional relationship between the corresponding sample of the second component and the virtual boundary.

[0013] Determine the category index for the second component from a set of one or more samples of the first component associated with the corresponding sample of the second component;

[0014] The sample point offset for the corresponding sample point of the second component is determined according to the category index; and

[0015] The cross-component offset sample value of the second component is obtained based on the determined sample offset.

[0016] In at least one embodiment, the method further includes:

[0017] In response to the current CTU being located within the bottom row of a CTU in a frame, strip, tile, sub-picture, or patch, a set of one or more samples of the first component is determined independently of the virtual boundary, wherein the set of one or more samples includes co-located samples of the first component relative to a corresponding sample of the second component or adjacent samples of co-located samples of the first component relative to a corresponding sample of the second component.

[0018] In at least one embodiment, the step of determining a set of one or more samples of the first component based on the positional relationship between the corresponding samples of the second component and the virtual boundary includes:

[0019] When the corresponding sample point of the second component is above the virtual boundary, a first set of one or more sample points of the first component is determined as a set of one or more samples of the first component; and

[0020] When the corresponding sample point of the second component is on or below the virtual boundary, a second set of one or more sample points of the first component is determined as a set of one or more sample points of the first component, wherein the second set of one or more sample points is different from the first set of one or more sample points.

[0021] In at least one embodiment, the first set of one or more samples includes co-located samples of the first component relative to the corresponding sample of the second component or adjacent samples of the co-located samples of the first component relative to the corresponding sample of the second component.

[0022] In at least one embodiment, the virtual boundary includes a virtual boundary within the coding tree block (CTB) of the second component.

[0023] In at least one embodiment, the virtual boundary is parallel to the block boundary, and the virtual boundary is separated from the block boundary by at least one row or one column of sample points.

[0024] According to a second aspect of this application, a method for decoding a video signal includes: receiving a picture frame comprising a first component and a second component from a video signal; determining a classifier for the second component from a set of one or more samples of the first component associated with a corresponding sample of the second component; determining, based on the classifier, whether to modify the value of the corresponding sample of the second component in a current block of the picture frame within a virtual boundary; in response to the determination that the value of the corresponding sample of the second component in the current block of the picture frame within the virtual boundary is modified according to the classifier, determining a sample offset for the corresponding sample of the second component according to the classifier; and modifying the value of the corresponding sample of the second component based on the determined sample offset.

[0025] According to a third aspect of this application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. When executed by the one or more processing units, these programs cause the electronic device to perform a video data decoding method as described above.

[0026] According to a fourth aspect of this application, a non-transitory computer-readable storage medium stores a plurality of programs executed by an electronic device having one or more processing units. When executed by the one or more processing units, these programs cause the electronic device to perform a method for decoding video data as described above. Attached Figure Description

[0027] The accompanying drawings, included to provide a further understanding of the embodiments and incorporated herein and forming part of the specification, illustrate the described embodiments and, together with the specification, serve to explain the underlying principles. The same reference numerals denote corresponding parts.

[0028] Figure 1 This is a block diagram illustrating an exemplary video encoding and decoding system according to some embodiments of the present disclosure.

[0029] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0030] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0031] Figures 4A to 4E This is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some embodiments of this disclosure.

[0032] Figure 5 This is a block diagram depicting four gradient modes used in Sample Adaptive Offset (SAO) according to some embodiments of the present disclosure.

[0033] Figure 6A This is a block diagram illustrating a system and process for cross-component sample adaptive offset (CCSAO) according to some embodiments of the present disclosure.

[0034] Figure 6B This is a block diagram illustrating a system and process for CCSAO applied in parallel with Enhanced Sample Adaptive Offset (ESAO) in the AVS standard, according to some embodiments of this disclosure.

[0035] Figure 6C This is a block diagram illustrating a system and process for applying CCSAO after SAO according to some embodiments of the present disclosure.

[0036] Figure 6D This is a block diagram illustrating the system and process of CCSAO applied in parallel with a cross-component adaptive loop filter (CCALF) according to some embodiments of the present disclosure.

[0037] Figure 7 This is a block diagram illustrating sample processing using CCSAO according to some embodiments of the present disclosure.

[0038] Figure 8 This is a block diagram illustrating a CCSAO process interleaved with vertical and horizontal deblocking filters (DBFs) according to some embodiments of the present disclosure.

[0039] Figure 9 This is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure.

[0040] Figure 10A This is a block diagram illustrating a classifier using different brightness sample locations for classification according to some embodiments of the present disclosure.

[0041] Figure 10B Examples of different shapes for brightness candidates according to some embodiments of this disclosure are shown.

[0042] Figure 11 This is a block diagram illustrating that, according to some embodiments of the present disclosure, co-located chromaticity samples across components other than luminance samples and adjacent chromaticity samples are also fed into sample processing in CCSAO classification.

[0043] Figure 12 An exemplary classifier is shown according to some embodiments of the present disclosure, which replaces co-located luminance sample values ​​with values ​​obtained by weighting co-located and adjacent luminance samples.

[0044] Figure 13A This is a block diagram illustrating, according to some embodiments of the present disclosure, that CCSAO is not applied to the current chromaticity sample if any of the co-occurring and adjacent luminance samples used for classification are outside the current image.

[0045] Figure 13B This is a block diagram illustrating, according to some embodiments of the present disclosure, the application of CCSAO to the current chromaticity sample if any of the co-occurring and adjacent luminance samples used for classification are outside the current image.

[0046] Figure 14 This is a block diagram illustrating, according to some embodiments of the present disclosure, that if the corresponding selected co-occurring or adjacent luminance samples used for classification are outside the virtual space defined by the virtual boundary (VB), then CCSAO is not applied to the current chroma sample.

[0047] Figure 15 The illustration shows the application of repeating or mirror filling to luminance samples outside virtual boundaries according to some embodiments of the present disclosure.

[0048] Figure 16 An additional luminance row buffer is required if all nine co-located adjacent luminance samples are used for classification, according to some embodiments of this disclosure.

[0049] Figure 17An illustration shows an AVS with 9 luminance candidate CCSAOs across VB that can be increased by 2 additional luminance line buffers according to some embodiments of the present disclosure.

[0050] Figure 18 A diagram illustrating a VVC with nine luminance candidate CCSAOs across a VB, according to some embodiments of the present disclosure, can be added with one additional luminance line buffer.

[0051] Figures 19A to 19C The present disclosure illustrates that, in AVS and VVC, if any luminance candidate of a chroma sample crosses a VB (outside the current chroma sample VB), CCSAO is disabled for that chroma sample.

[0052] Figures 20A to 20C The present disclosure illustrates that, in AVS and VVC, if any luminance candidate of a chroma sample crosses a VB (outside the current chroma sample VB), repeat padding is used to enable CCSAO for the chroma sample.

[0053] Figures 21A to 21C The present disclosure illustrates that, in AVS and VVC, if any luminance candidate of a chroma sample crosses a VB (outside the current chroma sample VB), mirror fill is used to enable CCSAO for the chroma sample.

[0054] Figure 22 The present disclosure illustrates the limitation of using a limited number of brightness candidates for classification in some embodiments of the present disclosure.

[0055] Figure 23 This is a block diagram illustrating the SAO classification method disclosed in this disclosure as a post-prediction filter according to some embodiments of this disclosure.

[0056] Figure 24 This is a block diagram illustrating a post-prediction SAO filter according to some embodiments of the present disclosure, in which each component can be classified using current and neighboring samples.

[0057] Figure 25 This is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation in the presence of virtual boundaries, according to some embodiments of the present disclosure. Detailed Implementation

[0058] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. Numerous non-limiting specific details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0059] The first generation of AVS standards includes the Chinese national standards "Information Technology, Advanced Audio and Video Coding, Part 2: Video" (referred to as AVS1) and "Information Technology, Advanced Audio and Video Coding, Part 16: Wireless Television Video" (referred to as AVS+). Compared to the MPEG-2 standard, it can save approximately 50% of the bit rate while maintaining the same perceptual quality. The second generation of AVS standards includes the Chinese national standard series "Information Technology, High-Efficiency Multimedia Coding" (referred to as AVS2), primarily targeting the transmission of ultra-high-definition television programs. AVS2's coding and decoding efficiency is twice that of AVS+. Meanwhile, the video portion of the AVS2 standard was submitted by the Institute of Electrical and Electronics Engineers (IEEE) as an international standard for applications. The AVS3 standard is a new generation of video coding and decoding standards for UHD video applications, aiming to surpass the coding and decoding efficiency of the latest international standard HEVC, offering approximately 30% bit rate savings relative to HEVC. In March 2019, at the 68th AVS meeting, the AVS3-P2 baseline was completed, offering approximately 30% bit rate savings relative to the HEVC standard. Currently, there is a reference software called the High Performance Model (HPM), which is maintained by the AVS group to demonstrate a reference implementation of the AVS3 standard. Similar to HEVC, the AVS3 standard is built on a block-based hybrid video codec framework.

[0060] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0061] In some implementations, target device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to target device 14. In one example, link 16 may include a communication medium enabling source device 12 to transmit encoded video data directly to target device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and transmitted to target device 14. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include a router, switch, base station, or any other means that may facilitate communication from source device 12 to target device 14.

[0062] In some other implementations, encoded video data can be sent from output interface 22 to storage device 32. The target device 14 can then access the encoded video data in storage device 32 via input interface 28. Storage device 32 can include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, storage device 32 can correspond to a file server or another intermediate storage device that can hold the encoded video data generated by source device 12. Target device 14 can access the stored video data from storage device 32 via streaming or downloading. The file server can be any type of computer capable of storing and sending encoded video data to target device 14. Exemplary file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. The target device 14 can access the encoded video data via any standard data connection suitable for accessing encoded video data stored on the file server, including wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modems, etc.), or a combination of both. The transmission of encoded video data from storage device 32 can be streaming, downloading, or a combination of both.

[0063] like Figure 1As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera in a security monitoring system, source device 12 and target device 14 may form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding and decoding and can be applied to wireless and / or wired applications.

[0064] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or transmitter.

[0065] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0066] In some embodiments, the target device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with the target device 14. The display device 34 displays decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0067] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, Advanced Video Codec (AVC), AVS) or extensions of such standards. It should be understood that this application is not limited to any particular video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0068] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0069] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding on video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence.

[0070] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries, thereby removing block artifacts from the reconstructed video. In addition to the block filter, another loop filter 63 can be used to filter the output of the adder 62. Furthermore, loop filters 63, such as Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF), can be applied to the reconstructed CU before it is placed in the reference picture repository and used for encoding and decoding subsequent video blocks. The video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the fixed or programmable hardware units described.

[0071] Video data memory 40 can store video data to be encoded by components of video encoder 20. For example, video data in video data memory 40 can be obtained from video source 18. DPB 64 is a buffer that stores reference video data for use by video encoder 20 (e.g., in intra-frame or inter-frame predictive coding modes) when encoding video data. Video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, video data memory 40 can be on-chip along with other components of video encoder 20, or off-chip relative to those components.

[0072] like Figure 2As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into stripes, tiles, or other larger coding units (CUs) according to a predefined splitting structure (e.g., a quadtree (QT) structure) associated with the video data. The video frame may be divided into a collection of video blocks (or tiles). The prediction processing unit 41 may select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 may provide the resulting intra-frame or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0073] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0074] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a prediction unit (PU) of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of a PU of a video block within the current video frame or picture relative to a prediction block within a reference frame (or other coding unit) associated with the current block (or other coding unit) being encoded within the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0075] Regarding pixel differences, the predicted block can be a block of a reference frame considered to closely match the video block to be encoded, and the pixel differences can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Therefore, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional pixel positions and output a motion vector with fractional-pixel accuracy.

[0076] The motion estimation unit 42 calculates the motion vector of the PU for a video block in an inter-frame predictive coding frame by comparing the position of the PU with the position of the predicted block of a reference frame selected from either a first reference frame list (list 0) or a second reference frame list (list 1), where each reference frame list identifies one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0077] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vectors determined by motion estimation unit 42. Upon receiving the motion vector of the PU for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in a reference frame list within a reference frame list, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block of pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the currently encoded video block. The pixel differences forming the residual video block may include a luminance difference component or a chrominance difference component, or both. Motion compensation unit 44 may also generate syntax elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Syntax elements may include, for example, syntax elements defining motion vectors for identifying prediction blocks, any flags indicating prediction modes, or any other syntax information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0078] In some implementations, the intra-BC unit 48 can generate vectors and acquire prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44; however, these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit 48 can determine the intra-prediction mode to be used for encoding the current block. In some examples, the intra-BC unit 48 can, for example, use various intra-prediction modes to encode the current block during individual encoding passes and test their performance through rate-distortion analysis. Next, the intra-BC unit 48 can select a suitable intra-prediction mode from the various tested intra-prediction modes to use and generate an intra-mode indicator accordingly. For example, the intra-BC unit 48 can use rate-distortion analysis to calculate rate-distortion values ​​for the various tested intra-prediction modes and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes as the suitable intra-prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between the coded block and the original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra-frame BC unit 48 can calculate the ratio based on the distortion and rate for various coded blocks to determine which intra-frame prediction mode exhibits the optimal rate-distortion value for the block.

[0079] In other examples, the intra-frame BC unit 48 may use, in whole or in part, the motion estimation unit 42 and the motion compensation unit 44 to perform such functions for intra-frame BC prediction according to the embodiments described herein. In any case, for intra-frame block copying, in terms of pixel differences, the predicted block may be a block considered to closely match the block to be encoded, the pixel differences may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics, and identifying the predicted block may include calculating values ​​for sub-integer pixel positions.

[0080] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0081] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 as described above, or the intra-block copy prediction performed by the intra-BC unit 48, the intra-prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-prediction processing unit 46 can determine an intra-prediction mode for encoding the current block. To this end, the intra-prediction processing unit 46 can use various intra-prediction modes to encode the current block, for example, during individual encoding passes, and the intra-prediction processing unit 46 (or, in some examples, the mode selection unit) can select a suitable intra-prediction mode from the tested intra-prediction modes for use. The intra-prediction processing unit 46 can provide information indicating the intra-prediction mode selected for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-prediction mode into the bitstream.

[0082] After prediction processing unit 41 determines a prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0083] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0084] After quantization, entropy coding unit 56 entropy-encodes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length codec (CAVLC), context-adaptive binary arithmetic codec (CABAC), syntax-based context-adaptive binary arithmetic codec (SBAC), probabilistic interval partitioned entropy (PIPE) coding, or another entropy coding method or technique. This encoded bitstream can then be transmitted to video decoder 30, or archived in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy-encode motion vectors and other syntax elements used for the current video frame being encoded.

[0085] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0086] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0087] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction processing unit 84, and an intra-frame prediction (BC) unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0088] In some examples, units of the video decoder 30 may be assigned tasks to perform embodiments of this application. Furthermore, in some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may perform embodiments of this application individually or in combination with other units of the video decoder 30 (e.g., motion compensation unit 82, intra-frame prediction processing unit 84, and entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the functionality of the intra-frame BC unit 85 may be performed by other components of the prediction processing unit 81 (e.g., motion compensation unit 82).

[0089] Video data memory 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of video decoder 30. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. Decoded picture buffer (DPB) 92 of video decoder 30 stores reference video data for use by video decoder 30 (e.g., in intra-frame or inter-frame predictive coding modes) when decoding video data. Video data memory 79 and DPB 92 can be formed of any memory device from a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, video data memory 79 and DPB 92 are shown in... Figure 3 The video data memory 79 and DPB 92 are depicted as two distinct components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip along with other components of the video decoder 30, or off-chip relative to those components.

[0090] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors and other syntax elements to the prediction processing unit 81.

[0091] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-predictive processing unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-predictive mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0092] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0093] In some examples, when a video block is encoded according to the intra-BC mode described herein, the intra-BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within a reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0094] Motion compensation unit 82 and / or intra-frame prediction (BC) unit 85 determine prediction information for video blocks in the current video frame by parsing motion vectors and other syntax elements, and then use this prediction information to generate prediction blocks for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frames in the reference frame list for the frame, motion vectors for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0095] Similarly, the intra-BC unit 85 can use some of the received syntax elements, such as flags, to determine which video blocks in the frame are predicted using the intra-BC mode, which video blocks in the frame are in the reconstruction region and should be stored in the DPB 92, the block vector for each intra-BC predicted video block in the frame, the intra-BC prediction state for each intra-BC predicted video block in the frame, and other information for decoding video blocks in the current video frame.

[0096] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0097] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0098] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter 91 may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. Loop filters 91, such as deblocking filters, sample adaptive offset (SAO), and adaptive loop filters (ALF), may be applied to the reconstructed CU before it is placed in the reference picture repository. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., Figure 1 On the display device 34).

[0099] In a typical video coding process, a video sequence usually consists of an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of chrominance samples (Cb). SCr is a two-dimensional array of chrominance samples (Cr). In other instances, a frame may be monochrome and therefore consist of only a two-dimensional array of luma samples.

[0100] Similar to HEVC, the AVS3 standard is built on a block-based hybrid video codec framework. The input video signal is processed block by block (called a coding unit (CU)). Unlike HEVC, which partitions blocks solely based on quadtrees, in AVS3, a coding tree unit (CTU) is divided into multiple CUs to accommodate different local features based on quadtrees / binary trees / extended quadtrees. Furthermore, the concept of multiple partitioned unit types in HEVC is removed; that is, there is no separation between CUs, prediction units (PUs), and transform units (TUs) in AVS3. Instead, each CU is always used as the basic unit for prediction and transform without further partitioning. In AVS3's tree partitioning structure, a CTU is first partitioned based on a quadtree structure. Then, the leaf nodes of each quadtree can be further partitioned based on binary trees and extended quadtree structures.

[0101] like Figure 4A As shown, the video encoder 20 (or more specifically, the segmentation unit 45) generates a coded representation of a frame by first segmenting the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs ordered consecutively from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set such that all CTUs in the video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not limited to a specific size. Figure 4B As shown, each CTU may include a coding tree block (CTB) for luma samples, two corresponding coding tree blocks for chroma samples, and syntax elements for encoding the samples of the coding tree blocks. The syntax elements describe the properties of different types of units in the coded pixel block and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome image or an image with three separate color planes, the CTU may include a single coding tree block and syntax elements for encoding the samples of that coding tree block. The coding tree block can be an N×N sample block.

[0102] To achieve better performance, the video encoder 20 can recursively perform tree splitting on the coding tree blocks of the CTU, such as binary tree splitting, ternary tree splitting, quadtree splitting, or combinations thereof, and divide the CTU into smaller coding units (CUs). Figure 4C As described, the 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Of these four smaller CUs, CU 410 and CU 420 are each divided into four CUs with a block size of 16×16. The two 16×16 CUs, 430 and CU 440, are further divided into four CUs with a block size of 8×8. Figure 4D Depicting as shown Figure 4C The final result of the CTU 400 partitioning process described in the figure is a quadtree data structure, where each leaf node of the quadtree corresponds to a CU of a corresponding size ranging from 32×32 to 8×8. Similar to... Figure 4B The CTU depicted in the image can include, for example, a CU that can comprise a coded block (CB) of luma samples and two corresponding coded blocks of chroma samples of the same size frame, as well as syntax elements for encoding the samples of the coded blocks. In monochrome images or images with three separate color planes, a CU can comprise a single coded block and a syntax structure for encoding the samples of the coded block. It should be noted that... Figure 4C and Figure 4DThe quadtree partitioning depicted is for illustrative purposes only, and a CTU can be split into multiple CUs based on quadtree / ternary / binary partitioning to adapt to varying local characteristics. In multi-type tree structures, a CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to binary and ternary tree structures. Figure 4E As shown, AVS3 has five partition / partition types: quad partition, horizontal binary partition, vertical binary partition, horizontal extended quadtree partition, and vertical extended quadtree partition.

[0103] In some implementations, the video encoder 20 may further segment the coded blocks of the CU into one or more (M×N) prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of samples to which the same prediction (inter-frame or intra-frame) is applied. A prediction unit (PU) of the CU may include a prediction block for luma samples, two corresponding prediction blocks for chroma samples, and syntax elements for predicting the prediction blocks. In a monochrome image or an image with three separate color planes, a PU may include a single prediction block and syntax structures for predicting the prediction blocks. The video encoder 20 may generate a predicted luma block, a predicted Cb block, and a predicted Cr block for each PU of the CU.

[0104] Video encoder 20 can use intra-frame prediction or inter-frame prediction to generate prediction blocks for the PU. If video encoder 20 uses intra-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0105] After the video encoder 20 generates a predicted luminance block, a predicted Cb block, and a predicted Cr block for one or more PUs of the CU, the video encoder 20 can generate a luminance residual block for the CU by subtracting the predicted luminance block of the CU from the original luminance coding block of the CU, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU, respectively, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0106] In addition, such as Figure 4C As shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. A transform unit (TU) of the CU can include a transform block for luminance samples, two corresponding transform blocks for chrominance samples, and syntax elements for transforming the samples in the transform block. Therefore, each TU of the CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU can be a sub-block of the CU's luminance residual block. A Cb transform block can be a sub-block of the CU's Cb residual block. A Cr transform block can be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU can include a single transform block and syntax structures for transforming the samples in that transform block.

[0107] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block for the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0108] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 can quantize the coefficient blocks. Quantization typically refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 can entropy-encode the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 can perform context-adaptive binary arithmetic encoding / decoding (CABAC) on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream comprising a bit sequence that forms a representation of coded frames and associated data; the bitstream is stored in storage device 32 or transmitted to target device 14.

[0109] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements. The video decoder 30 can reconstruct frames of video data, at least in part, based on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally the inverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coded blocks of the current CU by adding the samples of the prediction block of the PU for the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the coded blocks for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0110] SAO is a process of modifying decoded samples by conditionally adding an offset value to each sample after applying a deblocking filter, based on values ​​in a lookup table sent by the encoder. SAO filtering is performed on a region-by-region basis, based on the filter type selected by the CTB via the syntax element `sao-type-idx`. A value of 0 for `sao-type-idx` indicates that the SAO filter is not applied to the CTB, while values ​​of 1 and 2 indicate the use of a band offset and an edge offset filter type, respectively. In the band offset mode specified by `sao-type-idx` equal to 1, the selected offset value depends directly on the sample amplitude. In this mode, the entire sample amplitude range is uniformly divided into 32 segments called bands, and the sample values ​​belonging to four of these bands (which are consecutive within the 32 bands) are modified by adding the transmitted value, represented as a band offset (which can be positive or negative). The main reason for using four consecutive bands is that in smooth regions where band artifacts may occur, the sample amplitude in the CTB tends to concentrate in only a few bands. Furthermore, the design choice using four offsets is consistent with the operation of the edge offset mode, which also uses four offset values. In the edge offset mode specified by sao-type-idx equal to 2, the syntax element sao-eo-class with values ​​from 0 to 3 indicates whether the horizontal direction, the vertical direction, or one of the two diagonal gradient directions is used for edge offset classification in CTB.

[0111] Figure 5This is a block diagram depicting four gradient modes used in SAO according to some embodiments of the present disclosure. Four gradient modes 502, 504, 506, and 508 are used for the corresponding sao-eo-class in the edge offset mode. A sample labeled "p" indicates the center sample to be considered. Two samples labeled "n0" and "n1" specify two adjacent samples along (a) the horizontal (sao-eo-class = 0) gradient mode, (b) the vertical (sao-eo-class = 1) gradient mode, (c) the 135° diagonal (sao-eo-class = 2) gradient mode, and (d) the 45° (sao-eo-class = 3) gradient mode. The sample value p at a certain location is compared with the sample value p at a location such as... Figure 5 The values ​​n0 and n1 of two samples at adjacent positions are compared to classify each sample in the CTB into one of the five EdgeIdx categories. Since this classification is based on the decoded sample value, no additional signaling is required for EdgeIdx classification. Depending on the EdgeIdx category at the sample location, for categories 1 through 4, the offset value from the sent lookup table is added to the sample value. The offset value is always positive for categories 1 and 2, and always negative for categories 3 and 4. Therefore, the filter typically has a smoothing effect in edge offset mode.

[0112] Table 1 below shows the EdgeIdx categories of sample points in SAO edge classification.

[0113]

[0114] Table 1: EdgeIdx Classification of Sample Points in SAO Edge Classification

[0115] For SAO Type 1 and SAO Type 2, a total of four amplitude offset values ​​are sent to the decoder for each CTB. For Type 1, the symbol is also encoded. The offset values ​​and associated syntax elements (such as sao-type-idx and sao-eo-class) are determined by the encoder (typically using a standard optimized for rate-distortion performance). A merging flag can be used to indicate that the SAO parameters are inherited from the left CTB or the upper CTB to make the signaling valid. In summary, SAO is a non-linear filtering operation that allows for additional corrections to the reconstructed signal, and it enhances the signal representation of smooth regions and surrounding edges.

[0116] In some embodiments, this document discloses methods and systems for improving encoding / decoding efficiency or reducing the complexity of Sample Adaptive Offset (SAO) by introducing cross-component information. SAO is used in the HEVC, VVC, AVS2, and AVS3 standards. Although existing SAO designs in the HEVC, VVC, AVS2, and AVS3 standards are used as the basic SAO method in the following description, the cross-component methods described herein can also be applied to other loop filter designs or other encoding / decoding tools with similar design principles, as will be apparent to those skilled in the art of video encoding / decoding. For example, in the AVS3 standard, SAO is replaced by encoding / decoding tools called Enhanced Sample Adaptive Offset (ESAO). However, the CCSAO disclosed herein can also be applied in parallel with ESAO. In another example, CCSAO can be applied in parallel with the Constrained Directional Enhancement Filter (CDEF) in the AV1 standard.

[0117] In existing SAO designs in HEVC, VVC, AVS2, and AVS3 standards, the luma (Y), chroma (Cb), and chroma (Cr) sample offset values ​​are determined independently. That is, for example, the current chroma sample offset is determined only by the current and adjacent chroma sample values, without considering co-located or adjacent luma samples. However, luma samples retain more original image detail than chroma samples, and these luma samples can benefit the determination of the current chroma sample offset. Furthermore, since chroma samples typically lose high-frequency details after color conversion from RGB to YCbCr or after quantization and deblocking filters, introducing luma samples with high-frequency details preserved for chroma offset decisions can benefit chroma sample reconstruction. Therefore, further gains can be expected by exploring cross-component correlations, for example, through the use of cross-component sample adaptive offset (CCSAO) methods and systems.

[0118] Figure 6A This is a block diagram illustrating a system and process for CCSAO according to some embodiments of the present disclosure. Luminance samples following the luminance deblocking filter (DBF Y) are used to determine additional offsets for chrominance Cb and Cr following SAO Cb and SAO Cr. For example, the current chrominance sample 602 is first classified using co-located luminance sample 604 and adjacent (white) luminance sample 606, and the corresponding CCSAO offset value for the corresponding category is added to the current chrominance sample value.

[0119] In some embodiments, CCSAO can also be used in parallel with other codec tools, such as ESAO in the AVS standard or CDEF in the AV1 standard. Figure 6B This is a block diagram illustrating a system and process for CCSAO applied in parallel with ESAO in the AVS standard, according to some embodiments of this disclosure.

[0120] Figure 6C This is a block diagram illustrating a system and process for applying CCSAO after SAO according to some embodiments of the present disclosure. In some embodiments, Figure 6C The location of the CCSAO (i.e., the location of the cross-component adaptive loop filter (CCALF) in the VVC standard) is shown to be after the SAO. In some embodiments, the SAO Y / Cb / Cr can be replaced by, for example, the ESAO in the AVS3 standard.

[0121] Figure 6D This is a block diagram illustrating a system and process for CCSAO applied in parallel with CCALF according to some embodiments of this disclosure. In some embodiments, Figure 6D This demonstrates that CCSAO can be applied in parallel with CCALF. In some embodiments, in Figure 6D In this configuration, the positions of CCALF and CCSAO can be switched. In some embodiments, in Figures 6A to 6D In this disclosure, or throughout the entire disclosure, the SAO Y / Cb / Cr block may be replaced by ESAO Y / Cb / Cr (in AVS3) or by CDEF (in AVS1).

[0122] In some embodiments, the current chroma sample classification again uses the SAO type (EO or BO), classification, and category of the corresponding luminance sample. The corresponding CCSAO offset can be transmitted by signal or derived from the decoder itself. For example, let h_Y be the corresponding luminance SAO offset, and h_Cb and h_Cr be the CCSAO Cb and Cr offsets, respectively. h_Cb (or h_Cr) = w * h_Y, where w can be selected from a defined table. For example, ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4, etc., where |w| only includes values ​​that are powers of 2.

[0123] In some embodiments, the comparison scores [-8, 8] of the co-located luminance sample (Y0) and the eight adjacent luminance samples are used, which generates a total of 17 categories.

[0124]

[0125] In some embodiments, the classification methods described above can be combined. For example, comparison scores can be combined with SAO BO (32-band classification) to increase diversity, resulting in a total of 17*32 categories. In some embodiments, Cb and Cr can use the same class to reduce complexity or save bits.

[0126] Figure 7 This is a block diagram illustrating sample processing using CCSAO according to some embodiments of the present disclosure. Specifically, Figure 7The diagram illustrates that inputs to the CCSAO can incorporate both vertical and horizontal DBF inputs to simplify category determination or increase flexibility. For example, let Y0_DBF_V, Y0_DPF_H, and Y0 be the co-position luminance samples at the inputs of DBF_V, DBF_H, and SAO, respectively. Let Yi_DBF_V, Yi_DBF_H, and Yi be the eight adjacent luminance samples at the inputs of DBF_V, DBF_H, and SAO, respectively, where i = 1 to 8.

[0127] Max Y0=max(Y0_DBF_V,Y0_DBF_H,Y0_DBF)

[0128] Max Yi=max(Yi_DBF_V,Yi_DBF_H,Yi_DBF)

[0129] The maximum Y0 and maximum Yi are then fed into the CCSAO classification.

[0130] Figure 8 This is a block diagram illustrating the CCSAO process interleaved with vertical and horizontal DBFs according to some embodiments of the present disclosure. In some embodiments, the CCSAO blocks in Figures 6, 7, and 8 can be selective. For example, Y0_DBF_V and Yi_DBF_V are used for the first CCSAO_V, which applies the same sampling processing as in Figure 6, while using the input of the DBF_V luminance sample as the CCSAO input.

[0131] In some embodiments, the implemented CCSAO syntax is shown in Table 2 below.

[0132]

[0133] Table 2: CCSAO Syntax Examples

[0134] In some embodiments, when signaling CCSAO Cb and Cr offset values, if an additional chroma offset is signaled, other chroma component offsets can be derived using a plus or minus sign or weighting to save bit overhead. For example, let h_Cb and h_Cr be the offsets of CCSAO Cb and Cr, respectively. When w is explicitly signaled, where w = +-|w| (with a finite number of |w| candidates), h_Cr can be derived from h_Cb without explicitly signaling h_Cr itself.

[0135] h_Cr=w*h_Cb

[0136] Figure 9 This is a flowchart illustrating an exemplary process 900 for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure.

[0137] The video decoder 30 receives a video signal (910) comprising a first component and a second component. In some embodiments, the first component is the luminance component of the video signal, and the second component is the chrominance component of the video signal.

[0138] The video decoder 30 also receives multiple offsets (920) associated with the second component.

[0139] The video decoder 30 then uses the characteristic measurement of the first component to obtain the classification category associated with the second component (930). For example, in Figure 6, the current chroma sample 602 is first classified using the co-located luminance sample 604 and the adjacent (white) luminance sample 606, and the corresponding CCSAO offset value is added to the current chroma sample.

[0140] The video decoder 30 also selects a first offset (940) from multiple offsets for the second component based on the classification category.

[0141] The video decoder 30 additionally modifies the second component (950) based on the selected first offset.

[0142] In some embodiments, obtaining a classification category (930) associated with a second component using characteristic measurements of the first component includes: obtaining a corresponding classification category for each corresponding sample of the second component using corresponding samples of the first component, wherein the corresponding sample of the first component is a corresponding sample point of the first component that is co-located with the corresponding sample point of the second component. For example, the current chromaticity sample classification again uses the SAO type (EO or BO), classification, and category of the co-located luminance sample.

[0143] In some embodiments, obtaining a classification category (930) associated with a second component using characteristic measurements of the first component includes: obtaining a corresponding classification category for each sample of the second component using corresponding samples of the first component, wherein the corresponding samples of the first component are reconstructed before or after deblocking. In some embodiments, the first component is deblocked at a deblocking filter (DBF). In some embodiments, the first component is deblocked at a luminance deblocking filter (DBFY). For example, instead of FIG6 or 7, the CCSAO input may also be before DBFY.

[0144] In some embodiments, characteristic measurements are derived by dividing the range of sample values ​​of the first component into several frequency bands and selecting frequency bands based on the intensity values ​​of the sample points in the first component. In some embodiments, characteristic measurements are derived from band offset (BO).

[0145] In some embodiments, characteristic measurements are derived based on the direction and intensity of edge information of samples in the first component. In some embodiments, characteristic measurements are derived from edge offset (EO).

[0146] In some embodiments, modifying the second component (950) includes adding the selected first offset directly to the second component. For example, adding the corresponding CCSAO offset value to the current chromaticity component sample.

[0147] In some embodiments, modifying the second component (950) includes mapping the selected first offset to the second offset and adding the mapped second offset to the second component. For example, when signaling CCSAO Cb and Cr offset values, if an additional chroma offset is signaled, other chroma component offsets can be derived by using a plus or minus sign or weighting to save bit overhead.

[0148] In some embodiments, receiving a video signal (910) includes a receive syntax element that indicates whether a method for decoding video signals using CCSAO is enabled for a video signal in a sequence parameter set (SPS). In some embodiments, cc_sao_enabled_flag indicates whether CCSAO is enabled at the sequence level.

[0149] In some embodiments, receiving a video signal (910) includes receiving a syntax element that indicates whether a method for decoding the video signal using CCSAO is enabled for a second component at the slice level. In some embodiments, slice_cc_sao_cb_flag or slice_cc_sao_cr_flag indicates whether CCSAO is enabled in the corresponding slice for cb or cr.

[0150] In some embodiments, receiving multiple offsets (920) associated with the second component includes receiving different offsets for different coding tree units (CTUs). In some embodiments, for a CTU, cc_sao_offset_sign_flag indicates the sign for the offset, and cc_sao_offset_abs indicates the CCSAO Cb and Cr offset values ​​for the current CTU.

[0151] In some embodiments, receiving multiple offsets (920) associated with the second component includes a receive syntax element indicating whether the received offset of the CTU is the same as the received offset of one of the adjacent CTUs of that CTU, wherein the adjacent CTU is either the left-side adjacent CTU or the top-side adjacent CTU. For example, cc_sao_merge_up_flag indicates whether to merge CCSAO offsets from the left-side CTU or the top CTU.

[0152] In some embodiments, the video signal further includes a third component, and the method of decoding the video signal using CCSAO further includes: receiving a plurality of second offsets associated with the third component; using characteristic measurements of the first component to obtain a second classification category associated with the third component; selecting a third offset for the third component from the plurality of second offsets according to the second classification category; and modifying the third component based on the selected third offset.

[0153] Figure 11 This is a block diagram illustrating that, according to some embodiments of the present disclosure, co-located (1102) chromaticity samples across components other than luminance samples and adjacent (white) chromaticity samples are also fed into sample processing in CCSAO classification. Figure 6A , Figure 6B and Figure 11 The input for CCSAO classification is shown. Figure 11 In the current chromaticity sample point, the chromaticity sample point across components is 1104, the chromaticity sample point in the same position is 1102, and the luminance sample point in the same position is 1106.

[0154] In some embodiments, the classifier example (C0) uses the same luminance sample value (Y0) for classification. Let band_num be the number of equally divided frequency bands of the luminance dynamic range, and bit_depth be the sequence bit depth, then the class index of the current chrominance sample is:

[0155] Class(C0)=(Y0*band_num)>>bit_depth

[0156] In some embodiments, the classification takes into account rounding, for example:

[0157] Class(C0)=((Y0*band_num)+(1<<bit_depth))> >bit_depth

[0158] Table 3 below lists some examples of band_num and bit_depth. Table 3 shows three classification examples where the number of bands is different for each classification example.

[0159]

[0160] Table 3: Exemplary band_num and bit_depth for each category index

[0161] In some embodiments, the classifier uses different brightness sample locations for C0 classification. Figure 10A This is a block diagram illustrating a classifier that uses different brightness sample point positions for C0 classification according to some embodiments of the present disclosure, for example, using adjacent Y7 instead of Y0 for C0 classification.

[0162] In some embodiments, different classifiers can be switched at the Sequence Parameter Set (SPS), Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Picture Header (PH), Strip Header (SH), Coding Tree Unit (CTU), and Coding Unit (CU) levels. For example, in Figure 10, Y0 is used for POC0, but Y7 is used for POC1, as shown in Table 4 below.

[0163] POC Classifier C0 band_num General Category 0 Use C0 at position Y0 8 8 1 Use C0 at position Y7 8 8

[0164] Table 4: Different classifiers applied to different images

[0165] In some embodiments, Figure 10B Examples of different shapes for brightness candidates according to some embodiments of this disclosure are shown. For example, constraints may be applied to these shapes. In some cases, the total number of brightness candidates must be a power of 2, such as... Figure 10B As shown in (b), (c), and (d). In some cases, the number of luminance candidates must be horizontally and vertically symmetrical with respect to the (central) chromaticity sample point, such as... Figure 10B As shown in (a), (c), (d) and (e).

[0166] In some embodiments, the C0 position and C0 band_num can be...

[0167] SPS / APS / PPS / PH / SH / CTU / CU level combinations and switching. Different combinations can use different classifiers, as shown in Table 5 below.

[0168] POC Classifier C0 band_num General Category 0 Use C0 at position Y0 16 16 1 Use C0 at position Y7 8 8

[0169] Table 5: Different combinations of classifiers and bandwidth numbers applied to different images

[0170] In some embodiments, the co-located luminance sample value (Y0) is replaced by a value (Yp) obtained by weighting the co-located luminance sample and the adjacent luminance sample. Figure 12 An exemplary classifier according to some embodiments of the present disclosure is shown, which replaces co-located luminance sample values ​​with values ​​obtained by weighting co-located and adjacent luminance samples. The co-located luminance sample value (Y0) can be replaced by a phase correction value (Yp) obtained by weighting adjacent luminance samples. Different Yp values ​​can be different classifiers.

[0171] In some embodiments, different Yp values ​​are applied to different chroma formats. For example, in Figure 12 In (a), Yp is used for 420 chroma format, and in (b), Yp is used for 422 chroma format, and Y0 is used for 444 chroma format.

[0172] In some embodiments, another classifier (C1) is the comparison score [-8, 8] of the co-located luminance sample (Y0) and the 8 adjacent luminance samples, which produces a total of 17 categories as shown below.

[0173] Initial Class(C1)=0,Loop over neighboring 8luma samples(Yi,i=1to 8)

[0174] if Y0>Yi Class+=1

[0175] else if Y0 <Yi Class-=1

[0176] In some embodiments, the variant (C1') only calculates the comparison score [0, 8], which results in 8 categories. (C1, C1') is a classifier group, and the PH / SH level flag can be signaled to switch between C1 and C1'.

[0177] Initial Class(C1')=0,Loop over neighboring 8luma samples(Yi,i=1to8)

[0178] if Y0>Yi Class+=1

[0179] In some embodiments, different classifiers are combined to produce a general classifier. For example, different classifiers are applied to different images (different POC values), as shown in Table 6 below.

[0180] POC Classifier C0 band_num General Category 0 Combining C0 and C1 16 16×17 1 Combinations C0 and C1' 16 16×9 2 Combining C0 and C1 7 7×17

[0181] Table 6: Different general classifiers applied to different images

[0182] In some embodiments, the classifier example (C2) uses the difference (Yn) between co-located luminance samples and adjacent luminance samples. Figure 12 (c) shows an example of Yn, which has a dynamic range of [-1024, 1023] when the bit depth is 10. Let C2band_num be the number of equal bands that divide the dynamic range of Yn.

[0183] Class(C2) = (Yn + (1 <<bit_depth)*band_num)> >(bit_depth+1).

[0184] In some embodiments, C0 and C2 are combined to produce a general classifier. For example, different classifiers are applied to different images (different POCs), as shown in Table 7 below.

[0185] POC Classifier C0 band_num C2 band_num General Category 0 Combining C0 and C2 16 16 16×17 1 Combining C0 and C2 8 7 8×7

[0186] Table 7: Different general classifiers applied to different images

[0187] In some embodiments, all of the above classifiers (C0, C1, C1', C2) are combined. For example, different classifiers are applied to different images (different POCs), as shown in Table 8 below.

[0188] POC Classifier C0 band_num C2 band_num General Category 0 Combinations C0, C1, and C2 4 4 4×17×4 1 Combinations C0, C1' and C2 6 4 6×9×4

[0189] Table 8: Different general classifiers applied to different images

[0190] In some embodiments, multiple classifiers are used within the same POC. The current frame is segmented into several regions, each using the same classifier. For example, three different classifiers are used in POC0, and which classifier (0, 1, or 2) is used in signal transmission at the CTU level is shown in Table 9 below.

[0191] POC Classifier C0 band_num area 0 Use C0 at position Y0 16 0 0 Use C0 at position Y0 8 1 0 Use C0 at position Y1 8 2

[0192] Table 9: Different general classifiers applied to different regions of the same image

[0193] In some embodiments, the maximum number of multiple classifiers (which may also be referred to as alternative offset sets) can be fixed at the SPS / APS / PPS / PH / SH / CTU / CU level or transmitted via signaling. In one example, the fixed (predefined) maximum number of multiple classifiers is 4. In this case, 4 different classifiers are used in POC0, and which classifier (0, 1, or 2) is used is transmitted via signaling at the CTU level. Truncated-unary (TU) codes can be used to indicate the classifier used for each chroma CTB. For example, as shown in Table 10 below, when the TU code is 0: CCSAO is not applied; when the TU code is 10: set 0 is applied; when the TU code is 110: set 1 is applied; when the TU code is 1110: set 2 is applied; when the TU code is 1111: set 3 is applied. Fixed-length code, Golomb-rice code, and exponential-golomb code can also be used to indicate the classifier (offset set index) for CTB. Three different classifiers are used in POC1.

[0194] POC Classifier C0 band_num area TU code 0 Use C0 at position Y3 6 0 10 0 Use C0 at position Y3 7 1 110 0 Use C0 at position Y1 3 2 1110 0 Use C0 at position Y6 6 3 1111 1 Use C0 at position Y0 16 0 10 1 Use C0 at position Y0 8 1 110 1 Use C0 at position Y1 8 2 1110

[0195] Table 10: Truncated-unary (TU) codes are used to indicate the classifier used for each chroma CTB.

[0196] For the 1280×720 sequence POC0, examples of Cb and Cr CTB offset set indices are given (if the CTU size is 128×128, the number of CTUs in the frame is 10×6). POC0 Cb uses 4 offset sets, while Cr uses 1 offset set. As shown in Table 11 below, when the offset set index is 0: CCSAO is not applied; when the offset set index is 1: set 0 is applied; when the offset set index is 2: set 1 is applied; when the offset set index is 3: set 2 is applied; when the offset set index is 4: set 3 is applied. The type in Table 11 refers to the position of the selected co-located luminance sample (Yi). Different offset sets can have different types, band_nums, and corresponding offsets.

[0197]

[0198] Table 11: Examples of Cb and Cr CTB offset set indices for a 1280×720 sequence POC0 (if the CTU size is 128×128, the number of CTUs in the frame is 10×6).

[0199] In some embodiments, the maximum band_num can be fixed at the SPS / APS / PPS / PH / SH / CTU / CU level or transmitted via signaling. For example, max band_num is fixed at 16 in the decoder, and for each frame, 4 bits are transmitted via signaling to indicate the C0 band_num in the frame. Table 12 below lists some other examples of maximum band_num.

[0200]

[0201]

[0202] Table 12: Examples of maximum band_num and band_num bits

[0203] In some embodiments, restrictions can be applied to the C0 classification, for example, limiting band_num to values ​​that are powers of 2. Instead of explicitly signaling band_num, the signal syntax band_num_shift is used. The decoder can use shift operations to avoid multiplication.

[0204] Class(C0)=(Y0>>band_num_shift)>>bit_depth

[0205] Another example of this is considering rounding to reduce error.

[0206] Class(C0)=((Y0+(1<<(band_num_shift-1)))>>band_num_shift)>>bit_depth

[0207] For example, if band_num_max is 16, then the possible band_num_shift candidates corresponding to band_num = 1, 2, 4, 8, 16 are 0, 1, 2, 3, and 4, as shown in Table 13.

[0208] POC Classifier C0 band_num_shift C0 band_num General Category 0 Use C0 at position Y0 4 16 16 1 Use C0 at position Y7 3 8 8

[0209]

[0210]

[0211] Table 13: Band_num and corresponding Band_num_shift candidates

[0212] In some embodiments, the classifiers applied to Cb and Cr are different. Offsets for Cb and Cr in all categories can be transmitted individually via signaling. For example, different offsets transmitted via signaling are applied to different chromaticity components, as shown in Table 14 below.

[0213] POC Quantity Classifier C0 General Category Send via signal 0 Cb C0 16 16 16 0 Cr C0 5 5 5

[0214] Table 14: Cb and Cr offsets for all categories transmitted individually via signal

[0215] In some embodiments, the maximum offset value is fixed in the Sequence Parameter Set (SPS) / Adaptive Parameter Set (APS) / Picture Parameter Set (PPS) / Picture Header (PH) / Strip Header (SH) or transmitted via a signal. For example, the maximum offset is between [-15, 15].

[0216] In some embodiments, offset signaling can use differential pulse-code modulation (DPCM). For example, the offset {3, 3, 2, 1, -1} can be represented by the signal {3, 0, -1, -1, -2}.

[0217] In some embodiments, the offset can be stored in the APS or a storage buffer for reuse in the next image / strip. An index can be signaled to indicate which stored offsets from previous frames are used for the current image.

[0218] In some embodiments, the classifiers for Cb and Cr are the same. For example, Cb and Cr offsets for all classes can be jointly transmitted via a signal, as shown in Table 15 below.

[0219]

[0220]

[0221] Table 15: Jointly transmitted Cb and Cr offsets for all categories via signal transmission

[0222] In some embodiments, the classifiers for Cb and Cr can be the same. For example, Cb and Cr offsets for all classes can be sent jointly via a signal (with sign flag differences), as shown in Table 16 below. According to Table 16, when the Cb offset is (3, 3, 2, -1), the derived Cr offset is (-3, -3, -2, 1).

[0223]

[0224] Table 16: Jointly transmitted Cb and Cr offsets for all categories (with sign differences) via signal transmission

[0225] In some embodiments, a symbol flag can be sent for each category via a signal. For example, as shown in Table 17 below. According to Table 17, when the Cb offset is (3, 3, 2, -1), the derived Cr offset is (-3, 3, 2, 1) according to the corresponding symbol flag.

[0226]

[0227] Table 17: Jointly transmitted Cb and Cr offsets for all categories via signaling (with symbolic flags for each category via signaling)

[0228] In some embodiments, the classifiers for Cb and Cr can be the same. For example, Cb and Cr offsets for all classes can be jointly transmitted via a signal (with weighted differences), as shown in Table 18 below. The weights (w) can be selected from a defined table, for example, ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4, etc., where |w| only includes values ​​that are powers of 2. According to Table 18, when the Cb offset is (3, 3, 2, -1), the derived Cr offset is (-6, -6, -4, 2) according to the corresponding sign flag.

[0229]

[0230] Table 18: Jointly transmitted Cb and Cr offsets for all categories (with weighted differences) via signal transmission

[0231] In some embodiments, weights can be sent for each category via signals. For example, as shown in Table 19 below. According to Table 19, when the Cb offset is (3, 3, 2, -1), the derived Cr offset is (-6, 12, 0, -1) according to the corresponding symbol flag.

[0232]

[0233] Table 19: Jointly transmitted Cb and Cr offsets for all categories (with transmitted weights for each category)

[0234] In some embodiments, if multiple classifiers are used in the same POC, different sets of offsets are sent individually or jointly via signaling.

[0235] In some embodiments, previously decoded offsets can be stored for use in future frames. An index can be signaled to indicate which set of previously decoded offsets was used in the current frame, reducing offset signaling overhead. For example, a POC0 offset can be reused by a POC2 with a signaling offset setting (idx = 0), as shown in Table 20 below.

[0236]

[0237]

[0238] Table 20: Signal transmission index to indicate which previously decoded offset set is used for the current frame

[0239] In some embodiments, the offset set indices for reuse of Cb and Cr can be different, as shown in Table 21 below, for example.

[0240]

[0241] Table 21: A signal transmission index to indicate which previously decoded offset set is used for the current frame; this index can be different for the Cb and Cr components.

[0242] In some embodiments, offset signaling can use additional syntax including the start point and length to reduce signaling overhead. For example, when band_num = 256, only the offset of band_idx = 37 to 44 is sent via signaling. In the examples in Table 22 below, the syntax for the start point and length is 8-bit fixed-length encoded, which should match the band_num bits.

[0243]

[0244] Table 22: Offset signaling uses additional syntax including start point and length.

[0245] In some embodiments, if the sequence bit depth is greater than 10 (or a specific bit depth), the offset can be quantized before being transmitted through the signal. On the decoder side, as shown in Table 23 below, the decoded offset is dequantized before the decoded offset is applied. For example, for a 12-bit sequence, the decoded offset is left-shifted (dequantized) by 2.

[0246] Offset sent via signal Dequantized, applied offset 0 0 1 4 2 8 3 12 ... 14 56 15 60

[0247] Table 23: Dequantization of decoded offsets before applying decoded offsets

[0248] In some embodiments, the offset can be calculated as CcSaoOffsetVal = (1 - 2 * ...

[0249] ccsao_offset_sign_flag)*(ccsao_offset_abs<<(BitDepth-Min(10,BitDepth))).

[0250] In some embodiments, the sample processing is described below. Let R(x, y) be the input chromaticity sample values ​​before CCSAO, and R'(x, y) be the output chromaticity sample values ​​after CCSAO:

[0251] offset=ccsao_offset[class_index of R(x,y)]

[0252] R'(x,y)=Clip3(0,(1< <bit_depth)–1,R(x,y)+offset)

[0253] Based on the above equation, each chroma sample value R(x, y) is classified using a classifier indicated by the current image. The corresponding offset of the derived class index is added to each chroma sample value R(x, y). The clip function clip3 is applied to (R(x, y) + offset) to make the output chroma sample value R'(x, y) within the bit depth dynamic range, for example, from 0 to (1 < 0). <bit_depth)-1。

[0254] In some embodiments, boundary processing is described below. If any of the co-located luminance sample and the adjacent luminance sample used for classification are outside the current image, CCSAO is not applied to the current chroma sample. Figure 13AThis is a block diagram illustrating, according to some embodiments of the present disclosure, that if any of the co-occurring and adjacent luminance samples used for classification are outside the current image, then CCSAO is not applied to the current chroma sample. For example, in Figure 13A In (a), if a classifier is used, CCSAO is not applied to the chromaticity components in the leftmost column of the current image. For example, if C1' is used, then... Figure 13A As shown in (b), CCSAO is not applied to the chroma components of the left column and top row of the current screen.

[0255] Figure 13B This is a block diagram illustrating the application of CCSAO to the current chromaticity sample if any of the co-occurring and adjacent luminance samples used for classification are outside the current image, according to some embodiments of this disclosure. In some embodiments, if any of the co-occurring and adjacent luminance samples used for classification are outside the current image, one variation is as follows: Figure 13B (a) shows the repeated use of missing samples, or as shown in Figure 13B (b) shows the mirror filling of missing sample points to create sample points for classification, and CCSAO can be applied to the current chromaticity sample points.

[0256] Figure 14 This is a block diagram illustrating, according to some embodiments of the present disclosure, that if the corresponding selected co-position or adjacent luminance samples used for classification are outside the virtual space defined by the virtual boundary, then CCSAO is not applied to the current chroma sample. In some embodiments, the virtual boundary (VB) is a virtual line separating spaces within an image frame. In some embodiments, if the virtual boundary (VB) is applied in the current frame, then CCSAO is not applied to chroma samples whose corresponding luminance positions have already been selected outside the virtual space defined by the virtual boundary. Figure 14 An example of a virtual boundary for a C0 classifier with nine luminance location candidates is shown. For each CTU, CCSAO is not applied to chromaticity samples whose corresponding selected luminance locations lie outside the virtual space enclosed by the virtual boundary. For example, in Figure 14 In (a), when the selected Y7 luminance sample location is located on the other side of the horizontal virtual boundary 1406, CCSAO is not applied to the chrominance sample 1402, which is located 4 pixel rows from the bottom of the frame. For example, in Figure 14 In (b), when the selected Y5 luminance sample point is located on the other side of the vertical virtual boundary 1408, CCSAO is not applied to the chrominance sample point 1404, which is located y pixels away from the right side of the frame.

[0257] Figure 15 The present disclosure illustrates how repeating or mirror filling can be applied to luminance samples outside virtual boundaries according to some embodiments of the present disclosure. Figure 15 (a) shows an example of repeated filling. If the original Y7 is selected as the classifier located at the bottom of VB 1502, the Y4 luminance sample value is used for classification (copied to the Y7 position) instead of the original Y7 luminance sample value. Figure 15 (B) illustrates an example of mirror fill. If Y7 is selected as the classifier located on the bottom side of VB 1504, the Y1 luminance sample value, which is symmetrical to the Y7 value relative to the Y0 luminance sample value, is used for classification instead of the original Y7 luminance sample value. The fill method provides more chroma samples for applying CCSAO, thus allowing for greater codec gain.

[0258] In some embodiments, restrictions may be applied to reduce the row buffer required by CCSAO and simplify boundary processing condition checks. Figure 16 The following describes an embodiment of the present disclosure where, if all nine co-located adjacent luminance samples are used for classification, an additional 1 luminance line buffer is required (i.e., the full line luminance sample of line -5 above the current VB 1602). Figure 10B (a) shows an example of classification using only 6 brightness candidates, which reduces the line buffer and does not require Figure 13A and Figure 13B Any additional boundary checks.

[0259] In some embodiments, using luminance samples for CCSAO classification may increase the luminance line buffer and thus increase the hardware implementation cost of the decoder. Figure 17An illustration of an AVS with nine luma candidate CCSAOs across VB 1702, according to some embodiments of this disclosure, shows an AVS that can add two additional luma line buffers. For luma and chroma samples above the virtual boundary (VB) 1702, DBF / SAO / ALF are processed in the current CTU line. For luma and chroma samples below VB 1702, DBF / SAO / ALF are processed in the next CTU line. In the AVS decoder hardware design, the pre-DBF samples of luma lines -4 to -1, the pre-SAO samples of line -5, the pre-DBF samples of chroma lines -3 to -1, and the pre-SAO samples of line -4 are stored as line buffers for DBF / SAO / ALF processing in the next CTU line. When processing the next CTU line, luma and chroma samples not in the line buffers are unavailable. However, for example, at position -3(b) of the chroma line, the chroma sample is processed in the next CTU line, but CCSAO requires pre-SAO luma sample lines -7, -6, and -5 for classification. Pre-SAO luma sample lines -7 and -6 are not in the line buffer, so they are unavailable. Adding pre-SAO luma sample lines -7 and -6 to the line buffer increases the decoder hardware implementation cost. In some examples, the luma VB (line -4) and chroma VB (line -3) can be different (misaligned).

[0260] Similar to Figure 17 , Figure 18 The illustration shows a VVC where nine luma candidate CCSAOs across VB 1802 can be augmented with one additional luma line buffer according to some embodiments of this disclosure. VB can be different in different standards. In the VVC, the luma VB is line-4 and the chroma VB is line-2, thus nine candidate CCSAOs can be augmented with one additional luma line buffer.

[0261] In some embodiments, in a first solution, CCSAO is disabled for chroma samples if any luminance candidate crosses a VB (outside the current chroma sample VB). Figures 19A to 19C The following describes an embodiment of the present disclosure in which, in AVS and VVC, CCSAO is disabled for any chroma sample if the luminance candidate of any chroma sample crosses VB 1902 (outside the current chroma sample VB). Figure 14 Some examples of this implementation are also shown.

[0262] In some embodiments, in the second solution, for “cross VB” luminance candidates, repeating fill is applied to CCSAO from the luminance row closest to VB and on the other side of VB (e.g., luminance row - 4). Figures 20A to 20CThe present disclosure illustrates that, in AVS and VVC, if any luminance candidate of a chroma sample crosses VB 2002 (outside the current chroma sample VB), repeat padding is used to enable CCSAO for the chroma sample. Figure 14 (a) also shows some examples of this implementation.

[0263] In some embodiments, in the third solution, for "cross VB" luminance candidates, mirror filling is used on CCSAO from luminance VB below. Figures 21A to 21C The present disclosure illustrates that, in AVS and VVC, if any luminance candidate of a chroma sample crosses VB 2102 (outside the current chroma sample VB), mirror fill is used to enable CCSAO for the chroma sample. Figure 14 (b) and Figure 13B (b) shows some examples of this implementation.

[0264] The padding method provides more chroma sample points for the application of CCSAO, thereby enabling greater encoding and decoding gains.

[0265] In some embodiments, at the bottom image (or strip, tile, brick) boundary CTU line, samples below VB are processed in the current CTU line, therefore the special processing described above (solutions 1, 2, 3) will not be applied to the bottom image (or strip, tile, brick) boundary CTU line. For example, a 1920×1080 frame is divided into 128x128 CTUs. A frame contains 15×9 CTUs (rounded up). The bottom CTU line is the 15th CTU line. The decoding process is performed line by line, and for each CTU line, it is performed CTU-by-CTU. Deblocking needs to be applied along the horizontal CTU boundary between the current CTU line and the next CTU line. Since within a CTU, in the bottom 4 / 2 luma / chroma line, DBF samples (VVC case) are processed in the next CTU line and are not available for CCSAO in the current CTU line, CTB VB is applied for each CTU line. However, in the bottom CTU line of the image frame, since there is no next CTU line remaining, the bottom 4 / 2 luma / chroma line DBF samples are available in the current CTU line and are processed by DBF in the current CTU line.

[0266] In some embodiments, restrictions can be applied to reduce the row buffer required by CCSAO and simplify boundary processing condition checks, such as Figure 16 As shown. Figure 22 The present disclosure illustrates the limitation of using a limited number of brightness candidates for classification in some embodiments of the present disclosure. Figure 22 (a) shows the limitation of using only 6 brightness candidates for classification. Figure 22(b) shows the limitation of using only 4 brightness candidates for classification.

[0267] In some embodiments, the implemented CCSAO syntax is shown in Table 24 below. In AVS3, the term patch is similar to stripe, and the patch header is similar to the stripe header. FLC represents fixed-length code. TU represents truncated unary code. EGk represents k-order exponential Golomb code, where k can be fixed.

[0268]

[0269]

[0270] Table 24: Exemplary CCSAO Syntax

[0271] If a higher-level flag is off, lower-level flags can be inferred from the off state of the flags and do not need to be signaled. For example, if ph_cc_sao_cb_flag is false in this image, then ph_cc_sao_cb_band_num_minus1, ph_cc_so_cb_cb_luma_type, cc_sao_cb_offset_sign_flag, cc_sao_cb_offset_abs, ctb_cc_sao_cb_flag, cc_sau_cb_merge_left_flag, and cc_sao_ab_merge_up_flag do not exist and are inferred to be false.

[0272] In some embodiments, SPS ccsao_enabled_flag is conditional on the SPS SAO enable flag, as shown in Table 25 below.

[0273]

[0274] Table 25: SPS ccsao_enabled_flag is conditional on the SPS SAO enable flag.

[0275] In some embodiments, ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_contral_flag indicate whether Cb / Cr CTB on / off control granularity is enabled. If ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_contral_flag are enabled, ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag can be further signaled. Otherwise, whether CCSAO is applied to the current image depends on ph_cc_sao_cb_flag and ph_cc_sao_cr_flag, without further signaling ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag at the CTB level.

[0276] In some embodiments, for ph_cc_sao_cb_type and ph_cc_sao_cr_type, a flag can be further sent via signal to distinguish whether the centered luminance position (Y0 position in Figure 10) is used for chroma sample classification, thereby reducing bit overhead. Similarly, if cc_sao_cb_type and cc_sao_cr_type are sent via signal at the CTB level, a flag can be further sent via signal using the same mechanism. For example, if the number of C0 luminance position candidates is 9, cc_sao_cb_type0_flag is further sent via signal to distinguish whether the centered luminance position is used, as shown in Table 26 below. If the centered luminance position is not used, cc_sao_cb_type_idc is used to indicate which of the remaining 8 adjacent luminance positions is used.

[0277]

[0278] Table 26: Distinguishing whether the center-aligned brightness position is used by sending the signal cc_sao_cb_type0_flag.

[0279] In some embodiments, extensions to intra-frame and inter-frame post-prediction SAO filters are further described below. In some embodiments, the SAO classification method disclosed herein can be used as a post-prediction filter, and the prediction can be intra-frame, inter-frame, or other prediction tools such as intra-frame block copying. Figure 23 This is a block diagram illustrating the SAO classification method disclosed in this disclosure as a post-prediction filter according to some embodiments of this disclosure.

[0280] In some embodiments, a corresponding classifier is selected for each Y, U, and V component. For each component's predicted sample point, it is first classified, and a corresponding offset is added. For example, each component can be classified using the current and neighboring samples. Y is classified using the current Y and neighboring Y samples, and U / V is classified using the current U / V samples, as shown in Table 27 below. Figure 24 This is a block diagram illustrating how each component of a post-prediction SAO filter can be classified using current and neighboring samples according to some embodiments of the present disclosure.

[0281]

[0282]

[0283] Table 27: Selecting the appropriate classifier for each Y, U, and V component

[0284] In some embodiments, the refined prediction samples (Ypred',Upred',Vpred') are updated by adding the corresponding class offset and then used for intra-frame, inter-frame, or other predictions.

[0285] Ypred'=clip3(0,(1< <bit_depth)-1,Ypred+h_Y[i])

[0286] Upred' = clip3(0, (1< <bit_depth)-1,Upred+h_U[i])

[0287] Vpred'=clip3(0,(1< <bit_depth)-1,Vpred+h_V[i])

[0288] In some embodiments, for the chromaticity U and V components, in addition to the current chromaticity component, the cross component (Y) can be used for further offset classification. For example, an additional cross component offset (h'_U, h'_V) can be added to the current component offset (h_U, h_V), as shown in Table 28 below.

[0289]

[0290] Table 28: For chromaticity U and V components, in addition to the current chromaticity component, the cross component (Y) can be used for further offset classification.

[0291] In some embodiments, the refined prediction samples (Upred”, Vpred”) are updated by adding the corresponding class offset and then used for intra-frame, inter-frame or other predictions.

[0292] Upred” = clip3(0, (1< <bit_depth)-1,Upred’+h’_U[i])

[0293] Vpred” = clip3(0, (1 < <bit_depth)-1,Vpred’+h’_V[i])

[0294] In some embodiments, intra-frame and inter-frame prediction can use different SAO filter offsets.

[0295] Figure 25 This is a flowchart illustrating an exemplary process 2500 for decoding a video signal using cross-component correlation in the presence of a VB, according to some embodiments of the present disclosure.

[0296] Video decoder 30 (e.g.) Figure 3 (As shown) Receives image frames (2510) from the video signal, including the first component and the second component.

[0297] The video decoder 30 determines a classifier (2520) for the second component from a set of one or more samples of the first component associated with the corresponding sample of the second component.

[0298] The video decoder 30 determines, based on the classifier, whether to modify the value of the corresponding sample point of the second component of the current block of the picture frame within the virtual boundary (2530).

[0299] In response to the determination of the value of the corresponding sample point of the second component of the current block of the picture frame within the virtual boundary according to the classifier, the video decoder 30 determines the sample point offset (2540) for the corresponding sample point of the second component according to the classifier.

[0300] The video decoder 30 modifies the value (2550) of the corresponding sample of the second component based on the determined sample offset.

[0301] In some embodiments, the virtual boundary includes a first virtual boundary within a first block of a first component and a second virtual boundary within a second block of a second component.

[0302] In some embodiments, the virtual boundary includes a first virtual boundary within a first coding tree block (CTB) of a first component and a second virtual boundary within a second CTB of a second component, and the first virtual boundary is not aligned with the second virtual boundary. In some examples, the first CTB and the second CTB are within a CTU.

[0303] In some embodiments, the virtual boundary is parallel to the block boundary, and the virtual boundary is spaced apart from the block boundary by at least one row or one column of sample points.

[0304] In some embodiments, determining whether to modify the value of a corresponding sample point of the second component of the current block of an image frame within a virtual boundary (2530) based on the classifier includes: determining not to modify the value of a corresponding sample point of the second component of the current block of the image frame within the virtual boundary, based on a determination that a set of one or more samples of the first component associated with the corresponding sample point of the second component is segmented by the first virtual boundary. For example, disabling CCSAO for "cross-VB" chroma samples.

[0305] In some embodiments, determining whether to modify the value of a corresponding sample point of the second component of the current block of an image frame within a virtual boundary, based on the classifier (2530), includes: determining whether to modify the value of a corresponding sample point of the second component of the current block of the image frame within the virtual boundary, based on a determination that a set of one or more samples of the first component associated with the corresponding sample point of the second component is located on a different side of the virtual boundary relative to the corresponding sample point of the second component. For example, disabling CCSAO for "cross-VB" chroma samples.

[0306] In some embodiments, determining whether to modify the value of a corresponding sample point of the second component of the current block of an image frame within a virtual boundary, based on a classifier (2530), includes: copying a second subset of the remaining subset of the set of one or more samples of the first component to replace the first subset, based on a determination that a first subset of the set of one or more samples of the first component associated with the corresponding sample point of the second component is located on a different side of the virtual boundary relative to the corresponding sample point of the second component, and that a remaining subset of the set of one or more samples of the first component associated with the corresponding sample point of the second component is located on the same side of the virtual boundary relative to the corresponding sample point of the second component; and determining to modify the value of the corresponding sample point of the second component of the current block of the image frame within the virtual boundary. For example, when the VB spans luminance samples, repeat padding and / or mirror padding are used from the luminance row.

[0307] In some embodiments, a second subset of the remaining subset of the set of one or more samples from the first component comes from the row nearest to the first subset. For example, repeating fill is used when VB spans luminance samples.

[0308] In some embodiments, a second subset of the remaining subset of the set of one or more samples from the first component is located at a position symmetrical to the first subset of the corresponding sample of the second component at the first virtual boundary. For example, mirror fill is used when the VB spans luminance samples.

[0309] In some embodiments, determining whether to modify the value of a corresponding sample point of the second component of the current block of an image frame within a virtual boundary, based on a classifier (2530), includes: based on a determination that a set of one or more samples of the first component associated with the corresponding sample point of the second component is located on a different side of the virtual boundary relative to the corresponding sample point of the second component, copying a second set of samples of the first component that are on the same side of the virtual boundary relative to the corresponding sample point of the second component to replace said set of one or more samples of the first component; and determining whether to modify the value of the corresponding sample point of the second component of the current block of the image frame within the virtual boundary. For example, when VB is below the luminance sample point, repeat padding and / or mirror padding are used from the luminance row.

[0310] In some embodiments, a second set of samples on the same side of the virtual boundary relative to the corresponding sample of the second component comes from the row on the same side of the virtual boundary relative to the corresponding sample of the second component that is closest to the set of one or more samples of the first component. For example, when VB is below the luminance sample, repeating padding is used from the luminance row.

[0311] In some embodiments, a second set of samples on the same side of a virtual boundary relative to the corresponding samples of the second component is located at a position symmetrical to the first virtual boundary or a set of one or more samples of the first component relative to the corresponding samples of the second component. For example, when VB is below the luminance sample, mirror fill is used from the luminance row.

[0312] In some embodiments, determining whether to modify the value of the corresponding sample point of the second component of the current block of the image frame within the virtual boundary, based on the classifier, includes: determining whether to modify the value of the corresponding sample point of the second component of the image frame within the virtual boundary based on the determination that a set of one or more samples of the first component associated with the corresponding sample point of the second component is segmented by the virtual boundary, a subset of one or more samples of the first component is located on a different side of the virtual boundary relative to the corresponding sample point of the second component, and a subset of one or more samples of the first component is located in a boundary coding tree unit (CTU) row at the bottom of the image frame processing. For example, in the bottom image (or strip, tile, brick) boundary CTU row, samples below VB are processed in the current CTU row, so special processing (solutions 1, 2, 3) is not applied to the bottom image (or strip, tile, brick) boundary CTU row.

[0313] Other embodiments also include various subsets of the above embodiments that are combined or otherwise rearranged in various other embodiments.

[0314] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, such functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium or a communication medium, where a computer-readable storage medium corresponds to a tangible medium such as a data storage medium, and a communication medium includes any medium that facilitates the transfer of a computer program from one place to another (e.g., according to a communication protocol). In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the embodiments described herein. A computer program product may include a computer-readable medium.

[0315] The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claims. The singular forms “a” and “the” used in the description of the embodiments and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will be further understood that, when used in this specification, the term “comprising” specifies the presence of the stated features, elements, and / or components, but does not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0316] It should also be understood that while the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the embodiments, a first electrode may be referred to as a second electrode, and similarly, a second electrode may be referred to as a first electrode. Both the first electrode and the second electrode are electrodes, but they are not the same electrode.

[0317] Throughout this specification, references to "an example," "example," "exemplary example," etc., in singular or plural form indicate that one or more specific features, structures, or characteristics described in connection with the example are included in at least one example of this application. Therefore, phrases such as "in an example," "in the example," "in the exemplary example," etc., appearing in singular or plural form throughout this specification do not necessarily refer to the same example. Furthermore, specific features, structures, or characteristics in one or more examples may include combinations in any suitable manner.

[0318] The description in this application is presented for illustrative and descriptive purposes and is not intended to be exhaustive or limiting of the invention in the disclosed form. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings. The embodiments were chosen and described in order to best explain the principles of the invention, its practical application, and to enable others skilled in the art to understand the various implementations of the invention and to best utilize the basic principles and various implementations with various modifications, as suited to the particular intended use. Therefore, it should be understood that the scope of the claims is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method for decoding a video signal, the method comprising: Receive image frames, including the first and second components, from the video signal; In response to the current coding tree unit (CTU) not being located in the bottom row of the CTU of a frame, strip, tile, sub-picture, or patch, a set of one or more samples of the first component is determined based on the positional relationship between the corresponding samples of the second component and the virtual boundary. Determine the category index for the second component from a set of one or more samples of the first component associated with the corresponding sample of the second component; The sample offset for the corresponding sample point for the second component is selected from multiple sample offsets according to the category index; as well as The cross-component offset sample value of the second component is obtained based on the selected sample offset.

2. The method according to claim 1, further comprising: In response to the current CTU being located within the bottom row of a CTU in a frame, strip, tile, sub-picture, or patch, a set of one or more samples of the first component is determined independently of the virtual boundary, wherein the set of one or more samples includes co-located samples of the first component relative to a corresponding sample of the second component or adjacent samples of co-located samples of the first component relative to a corresponding sample of the second component.

3. The method according to claim 1, wherein, The step of determining a set of one or more samples of the first component based on the positional relationship between the corresponding samples of the second component and the virtual boundary includes: When the corresponding sample point of the second component is above the virtual boundary, a first set of one or more sample points of the first component is determined as a set of one or more samples of the first component; and When the corresponding sample point of the second component is on or below the virtual boundary, a second set of one or more sample points of the first component is determined as a set of one or more sample points of the first component, wherein the second set of one or more sample points is different from the first set of one or more sample points.

4. The method according to claim 3, wherein, The first set of one or more samples includes co-located samples of the first component relative to the corresponding sample of the second component, or adjacent samples of co-located samples of the first component relative to the corresponding sample of the second component.

5. The method according to claim 3, wherein, The virtual boundary includes the virtual boundary within the coding tree block (CTB) of the second component.

6. The method according to claim 3, wherein, The virtual boundary is parallel to the block boundary, and the virtual boundary is separated from the block boundary by at least one row or one column of sample points.

7. The method according to claim 1, wherein, The chroma format of the video signal is 4:2:0 or 4:2:

2.

8. The method according to claim 1, wherein, The color format of the video signal is 4:4:

4.

9. An electronic device comprising: A non-transitory computer-readable storage medium that stores a bit stream; as well as One or more processing units are configured to perform the method according to any one of claims 1 to 8 to decode the bit stream.

10. The electronic device according to claim 9, wherein, The chroma format of the video signal is 4:2:0 or 4:2:

2.

11. The electronic device according to claim 9, wherein, The color format of the video signal is 4:4:

4.

12. A computer-readable storage medium storing a computer program / instructions and a bit stream thereon, wherein the computer program / instructions, when executed by a processor, implement the method of any one of claims 1 to 8 to decode the bit stream.

13. The computer-readable storage medium according to claim 12, wherein, The chroma format of the video signal is 4:2:0 or 4:2:

2.

14. The computer-readable storage medium according to claim 12, wherein, The color format of the video signal is 4:4:4.

Images