Filter strength or length design for asymmetric deblocking at virtual boundaries

Abstract

An apparatus comprising at least one processor; and at least one memory storing instructions that, when executed with the at least one processor, cause the apparatus to: determine a virtual boundary that separates a picture, or a portion of a picture, into a first region and a second region; and determining to perform filtering of at least one pixel of the first region and not to perform filtering of pixels of the second region; and filtering at least one pixel of the first region using the encoding information of the second region.

Description

Technical Field

[0001] Examples and non - limiting embodiments generally relate to video coding, and more particularly to virtual boundaries. Background Art

[0002] Versatile Video Coding (VVC) has the concept of virtual boundaries. From the perspective of coding dependencies, a picture can be divided into different intervals by virtual boundaries. Summary of the Invention

[0003] The following summary of the invention is only for example purposes. The summary of the invention is not intended to limit the scope of the claims.

[0004] According to one aspect, an example apparatus is provided, including: at least one processor; and at least one memory storing instructions that, when executed using the at least one processor, cause the apparatus to: determine a virtual boundary that separates a picture, or a portion of a picture, into a first region and a second region; and determine to perform filtering on at least one pixel of the first region and not perform filtering on pixels of the second region; and filter at least one pixel of the first region using coding information of the second region.

[0005] According to another aspect, an example method is provided, including: determining a virtual boundary that separates a picture, or a portion of a picture, into a first region and a second region; and determining to perform filtering on at least one pixel of the first region and not perform filtering on pixels of the second region; and filtering at least one pixel of the first region using coding information of the second region.

[0006] According to another aspect, an example embodiment provides a non - transitory computer - readable medium including program instructions that, when executed using an apparatus, cause the apparatus to perform at least the following: determine a virtual boundary that separates a picture, or a portion of a picture, into a first region and a second region; and determine to perform filtering on at least one pixel of the first region and not perform filtering on pixels of the second region; and filter at least one pixel of the first region using coding information of the second region.

[0007] According to another aspect, an example apparatus is provided, including: means for determining a virtual boundary that separates a picture, or a portion of a picture, into a first region and a second region; and means for determining to perform filtering on at least one pixel of the first region and not perform filtering on pixels of the second region; and means for filtering at least one pixel of the first region using coding information of the second region. Brief Description of the Drawings

[0008] The foregoing aspects and other features are explained in the following description in conjunction with the accompanying drawings, wherein:

[0009] The foregoing aspects and other features are explained in the following description in conjunction with the accompanying drawings, wherein:

[0010] Figure 1 An electronic device incorporating an embodiment employing the examples described herein is schematically illustrated.

[0011] Figure 2 A user device suitable for incorporating an embodiment employing the examples described herein is schematically illustrated.

[0012] Figure 3 An electronic device incorporating an embodiment employing the examples described herein and connected using wireless and wired network connections is further schematically illustrated.

[0013] Figure 4 A block diagram of an encoder used for data compression in a general sense is schematically illustrated.

[0014] Figure 5 It is illustrated that the coding information of the non-refresh area is not allowed to be used in the refresh area.

[0015] Figure 6 It is illustrated that the coding information of the refresh area is allowed to be used in the non-refresh area.

[0016] Figure 7 Two transform blocks horizontally adjacent to a vertical boundary at a TransformEdge are illustrated.

[0017] Figure 8 A transform block having sub-blocks, a TransformEdge, sub-block edges, a size, and relationships is illustrated.

[0018] Figure 9 It is a diagram illustrating a filter strength (or length) decision for a luminance component, which can be used for two transform blocks located on opposite sides of a virtual boundary.

[0019] Figure 10 It is a diagram illustrating an example of a filter strength (or length) decision for a luminance component, which shows a modification with respect to the diagram shown in Figure 9 The filter strength (or length) decision can be used for one transform block located on one side of a virtual boundary.

[0020] Figure 11 It is a diagram illustrating a filter strength (or length) decision for a chrominance component, which can be used for two transform blocks located on opposite sides of a virtual boundary.

[0021] Figure 12is a diagram illustrating an example of a filter strength (or length) decision for a chrominance component, which shows a modification with respect to the diagram shown in Figure 11 The filter strength (or length) decision can be used for a transform block located on one side of a virtual boundary.

[0022] Figure 13 is a diagram illustrating an example method. DETAILED DESCRIPTION

[0023] Examples of in-loop filters at virtual boundaries are described herein. The models described herein can be used to perform any task, such as data compression, data decompression, video compression, video decompression, image or video classification, object classification, object detection, object tracking, speech recognition, language translation, music transcription, etc.

[0024] Suitable devices and possible mechanisms for implementing aspects of an asymmetric in-loop filter at a virtual boundary are described in detail below. In this regard, reference is first made to Figure 1 and Figure 2 where Figure 1 shows an example block diagram of a device 50. The device can be an Internet of Things (IoT) device configured to perform various functions, such as, for example, collecting information through one or more sensors, receiving or sending information, analyzing information collected or received by the device, and so on. The device can include a neural network weight update encoding system, which can incorporate a codec. Figure 2 shows the layout of a device according to an example embodiment. The elements of Figure 1 and Figure 2 will be explained next.

[0025] The electronic device 50 can be, for example, a mobile terminal or user equipment of a wireless communication system, a sensor device, a tag, or other low-power device. Alternatively, the electronic device can be a non-mobile computer or part of a computer. However, it should be understood that the example embodiments described herein can be implemented within any electronic device or apparatus capable of processing data.

[0026] The device 50 can include a housing 30 for enclosing and protecting the device. The device 50 can also include a display 32 in the form of a liquid crystal display. In other embodiments of the examples described herein, the display can be any suitable display technology suitable for displaying images or videos. The device 50 can also include a keyboard 34 (or touch area 34). In other embodiments of the examples described herein, any suitable data or user interface mechanism can be employed. For example, the user interface can be implemented as a virtual keyboard or data input system as part of a touch-sensitive display.

[0027] The device may include a microphone 36 or any suitable audio input, which may be a digital or analog signal input. The device 50 may also include an audio output device, which in the exemplary embodiments described herein may be any one of the following: headphones 38, a speaker, or an analog audio or digital audio output connection. The device 50 may also include a battery (or in other embodiments described herein, the device may be powered by any suitable mobile energy device, such as a solar cell, a fuel cell, or a wind-up generator). The device may also include a camera 42 capable of recording or capturing images and / or video. The device 50 may also include an infrared port for short-range line-of-sight communication with other devices. In other embodiments, the device 50 may also include any suitable short-range communication solution, such as, for example, a Bluetooth wireless connection or a USB / FireWire wired connection.

[0028] The device 50 may include a controller 56, a processor, or processor circuitry for controlling the device 50. The controller 56 may be connected to a memory 58, which in the exemplary embodiments described herein may store both data in the form of images and audio data and / or may also store instructions for implementation on the controller 56. The controller 56 may also be connected to a codec circuitry 54, which is suitable for performing encoding / compression of neural network weight updates, and / or decoding of audio data and / or video data, or assisting with encoding and / or decoding performed by the controller.

[0029] The device 50 may also include a card reader 48 and a smart card 46, such as a UICC and a UICC reader, for providing user information and suitable for providing authentication information for authenticating and authorizing the user on the network.

[0030] The device 50 may include a radio interface circuitry 52, which is connected to the controller and suitable for generating wireless communication signals, such as for communicating with a cellular communication network, a wireless communication system, or a wireless local area network. The device 50 may also include an antenna 44, which is connected to the radio interface circuitry 52 for transmitting radio frequency signals generated at the radio interface circuitry 52 to other device(s) (such as network nodes) and / or for receiving radio frequency signals from other device(s).

[0031] The device 50 may include a camera capable of recording or detecting individual frames, which will then be passed to the codec 54 or controller for processing. The device may receive video image data or machine learning data from another device for processing before transmission and / or storage. The device 50 may also receive images wirelessly or via a wired connection for encoding / decoding. The structural elements of the device 50 described above represent examples of components for performing corresponding functions.

[0032] about Figure 3 , an example of a system in which an embodiment of the examples described herein may be utilized is shown. System 10 includes a plurality of communication devices that may communicate via one or more networks. System 10 may include any combination of wired or wireless networks, including, but not limited to, wireless cellular telephone networks (such as GSM, UMTS, CDMA, LTE, 4G, 5G networks, etc.), wireless local area networks (WLANs) (such as defined by any IEEE 802.x standards), Bluetooth personal area networks, Ethernet local area networks, token ring local area networks, wide area networks, and the Internet.

[0033] System 10 may include both wired and wireless communication devices and / or apparatus 50 suitable for implementing exemplary embodiments described herein.

[0034] For example, Figure 3 The system shown in FIG. 1 shows a representation of a mobile phone network 11 and the Internet 28, which is accessible to a mobile phone user using a communication link 2 (wired or wireless). Figure 3 The various devices shown in are accessible. Connectivity with the Internet 28 may include, but is not limited to, long-range wireless connections, short-range wireless connections, and various wired connections, including, but not limited to, telephone lines, cable lines, power lines, and similar communication paths.

[0035] The example communication devices shown in system 10 may include, but are not limited to, electronic devices or apparatuses 50, a combination of a personal digital assistant (PDA) and a mobile phone 14, a PDA 16, an integrated messaging device (IMD) 18, a desktop computer 20, a notebook computer 22. The apparatus 50 may be stationary or may be mobile when carried by a moving individual. The apparatus 50 may also be located in a transportation mode, including, but not limited to, a car, truck, taxi, bus, train, boat, airplane, bicycle, motorcycle, or any similar suitable transportation mode, or a head mounted display (HMD) 17.

[0036] The embodiments can also be implemented in a set-top box, i.e., a digital TV (television) receiver, which may or may not have a display or wireless capabilities; the embodiments can also be implemented in a tablet computer or a (notebook) personal computer (PC) that has hardware and / or software for processing neural network data; the embodiments can also be implemented in various operating systems, as well as in a chipset, a processor, a DSP, and / or an embedded system that provides hardware / software-based encoding.

[0037] Some or other devices can send and receive calls and messages and communicate with service providers via a wireless connection 25 to a base station 24. The base station 24 can be connected to a network server 26 to allow communication between the mobile phone network 11 and the Internet 28. The system can include additional communication devices and various types of communication devices.

[0038] The communication devices can communicate using various transmission technologies, including but not limited to, Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Transmission Control Protocol - Internet Protocol (TCP-IP), Short Message Service (SMS), Multimedia Messaging Service (MMS), email, Instant Messaging Service (IMS), Bluetooth, IEEE 802.11, 3GPP Narrowband IoT, and any similar wireless communication technology. The communication devices involved in various embodiments for implementing the examples described herein can communicate using various media, including but not limited to, radio, infrared, laser, cable connection, and any suitable connection.

[0039] In telecommunication and data networks, a channel can refer to a physical channel or a logical channel. A physical channel can refer to a physical transmission medium (such as a wire), while a logical channel can refer to a logical connection on a multiplexed medium capable of carrying several logical channels. Channels can be used to convey information signals (e.g., bitstreams) from one or more senders (or transmitters) to one or more receivers.

[0040] Embodiments can also be implemented in so-called IoT devices. The Internet of Things (IoT) can be defined, for example, as the interconnection of uniquely identifiable embedded computing devices within the existing Internet infrastructure. The convergence of various technologies has enabled and can enable many areas of embedded systems, such as wireless sensor networks, control systems, home / building automation, etc., to be included in the Internet of Things (IoT). To utilize the Internet, IoT devices are provided with an IP address as a unique identifier. IoT devices can be provided with a radio transmitter, such as a WLAN or Bluetooth transmitter or an RFID tag. Alternatively, IoT devices can access an IP-based network via a wired network, such as an Ethernet-based network or a power line connection (PLC).

[0041] One application where an asymmetric loop filter at a virtual boundary and model-level update skipping in compressive incremental learning are important is the use case of neural network-based codecs, such as neural network-based video codecs. The video codec can use one or more neural networks. In the first case, the video codec can be a conventional video codec, such as a versatile video codec (VVC / H.266) that has been modified to include one or more neural networks. Examples of these neural networks are:

[0042] 1. A neural network filter used as one of the loop filters in the loop filter of VVC

[0043] 2. A neural network filter that replaces one or more of the loop filters in the (multiple) loop filters of VVC

[0044] 3. A neural network filter used as a post-processing filter

[0045] 4. A neural network used to perform intra prediction

[0046] 5. A neural network used to perform inter prediction.

[0047] In the second case, often referred to as an end-to-end learning video codec, the video codec can include a neural network that transforms input data into a more compressible representation. The new representation can be quantized, losslessly compressed, then losslessly decompressed, dequantized, and then another neural network can transform its input into reconstructed or decoded data.

[0048] In both of the above cases, one or more neural networks may exist on the decoder side, and an example of a neural network filter is considered. The encoder may fine-tune the neural network filter by using ground-truth data (uncompressed data) available on the encoder side. Fine-tuning may be performed to improve the neural network filter when it is applied to current input data (such as one or more video frames). Fine-tuning may include running one or more optimization iterations on some or all of the learnable weights of the neural network filter. The optimization iterations may include: calculating the gradients of a loss function with respect to some or all of the learnable weights of the neural network filter (e.g., by using the backpropagation algorithm), and then updating some or all of the learnable weights by using an optimizer (such as a stochastic gradient descent optimizer). The loss function may include one or more loss terms. An example loss term may be the mean squared error (MSE). Other distortion metrics may be used as loss terms. The loss function may be calculated by: providing one or more data to the input of the neural network filter, obtaining one or more corresponding outputs from the neural network filter, and calculating the loss terms by using one or more outputs from the neural network filter and one or more ground-truth data. The difference between the weights of the fine-tuned neural network and the weights of the neural network before fine-tuning is referred to as the weight update. This weight update needs to be encoded, provided to the decoder side together with the encoded video data, and used on the decoder side to update the neural network filter. Then, the updated neural network filter is used as part of the video decoding process or part of the video post-processing process. It is desirable to encode the weight update such that it requires a small number of bits. Thus, the examples described herein also consider this use case of a neural network-based codec as a potential application for the compression of weight updates.

[0049] In a further description of the neural network-based codec use case, the MPEG-2 transport stream (TS) specified in ISO / IEC 13818-1 or equivalently in ITU-T Recommendation H.222.0 is a format for carrying audio, video, and other media as well as program metadata or other metadata in a multiplexed stream. Packet identifiers (PIDs) are used to identify elementary streams (also known as packetized elementary streams) within the TS. Thus, the logical channels within the MPEG-2 TS can be considered to correspond to specific PID values.

[0050] Available media file format standards include the ISO base media file format (ISO / IEC 14496-12, which may be abbreviated as ISOBMFF) and the file format for NAL unit structured video (ISO / IEC 14496-15, which is derived from ISOBMFF).

[0051] A video codec consists of an encoder and a decoder. The encoder transforms the input video into a compressed representation suitable for storage / transmission, and the decoder can decompress the compressed video representation back into a visual form. The video encoder and / or the video decoder can also be separated from each other, i.e., it is not necessary to form a codec. Generally, the encoder discards some information in the original video sequence in order to represent the video in a more compact form (i.e., at a lower bit rate).

[0052] Typical hybrid video encoders (e.g., many encoder implementations of ITU-T H.263 and H.264) encode video information in two stages. First, the pixel values in a certain picture region (or "block") are predicted, for example, by means of motion compensation (searching in one of the previously encoded video frames and indicating the region closely corresponding to the block to be encoded) or spatial means (using the pixel values around the block to be encoded in a specified manner). Second, the prediction error (i.e., the difference between the predicted pixel block and the original pixel block) is encoded. This is usually done by: using a specified transform (e.g., the discrete cosine transform (DCT) or its variant) to transform the differences in the pixel values, quantizing the coefficients, and entropy encoding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the accuracy of the pixel representation (picture quality) and the size of the resulting encoded video representation (file size or transmission bit rate).

[0053] In temporal prediction, the source of prediction is the previously decoded picture (also called the reference picture). In intra block copy (IBC; also called intra block copy prediction and current picture reference), prediction is similarly applied to temporal prediction, but the reference picture is the current picture, and only previously decoded samples can be referred to during the prediction process. Inter-layer prediction or inter-view prediction can be similarly applied to temporal prediction, but the reference pictures are the decoded pictures from another scalable layer or from another view, respectively. In some cases, inter-frame prediction may only refer to temporal prediction, while in other cases, inter-frame prediction can generally refer to temporal prediction and any one of intra block copy prediction, inter-layer prediction, and inter-view prediction, as long as they are executed in the same or a similar process as temporal prediction. Inter-frame prediction or temporal prediction is sometimes called motion compensation or motion compensation prediction.

[0054] Inter-frame prediction (which can also be called temporal prediction, motion compensation, or motion compensation prediction) reduces temporal redundancy. In inter-frame prediction, the source of prediction is the previously decoded picture. Intra-frame prediction makes use of the fact that neighboring pixels within the same picture may be correlated. Intra-frame prediction can be performed in the spatial domain or the transform domain, i.e., the sample values or the transform coefficients can be predicted. Intra-frame prediction is usually utilized in intra-frame coding, where no inter-frame prediction is applied.

[0055] One result of the encoding process is a set of encoded parameters, such as motion vectors and quantized transform coefficients. If many of the parameters are first predicted based on spatially adjacent or temporally adjacent parameters, the many parameters can be entropy encoded more efficiently. For example, a motion vector can be predicted based on a spatially neighboring motion vector, and only the difference relative to the motion vector predictor can be encoded. Prediction of encoded parameters and intra prediction can be collectively referred to as intra-picture prediction.

[0056] Figure 4 A block diagram showing the overall structure of a video encoder. Figure 4 An encoder for two layers is presented, but it should be understood that the presented encoder can be similarly extended to encode more than two layers. Figure 4 A video encoder is illustrated, which includes a first encoder portion 500 for a base layer and a second encoder portion 502 for an enhancement layer. Each of the first encoder portion 500 and the second encoder portion 502 may include similar elements for encoding an incoming picture. The encoder portion 500, the encoder portion 502 may include a pixel predictor 302, a pixel predictor 402, a prediction error encoder 303, a prediction error encoder 403, and a prediction error decoder 304, a prediction error decoder 404. Figure 4 Embodiments of the pixel predictor 302, the pixel predictor 402 are also shown as including: an inter-frame predictor (P inter ) 306, an inter-frame predictor (P inter ) 406, an intra-frame predictor (P intra ) 308, an intra-frame predictor (P intra ) 408, a mode selector 310, a mode selector 410, a filter (F) 316, a filter (F) 416, and a reference frame memory (RFM) 318, a reference frame memory (RFM) 418. The pixel predictor 302 of the first encoder portion 500 receives the base layer image (I of the video stream to be encoded at both the inter-frame predictor 306 (which determines the difference between the image and the motion compensation reference frame 318) and the intra-frame predictor 308 (which determines the prediction for an image block based only on the processed portion of the current frame or picture) 0,n) 300. The outputs of both the inter - frame predictor and the intra - frame predictor are passed to the mode selector 310. The intra - frame predictor 308 can have more than one intra - frame prediction mode. Thus, each mode can perform intra - frame prediction and provide the predicted signal to the mode selector 310. The mode selector 310 also receives a copy of the base - layer picture 300. Correspondingly, the pixel predictor 402 of the second encoder section 502 receives the enhancement - layer image (I of the video stream that will be encoded at both the inter - frame predictor 406 (which determines the difference between the image and the motion - compensated reference frame 418) and the intra - frame predictor 408 (which determines the prediction for an image block based only on the processed part of the current frame or picture). 1,n ) 400. The outputs of both the inter - frame predictor and the intra - frame predictor are passed to the mode selector 410. The intra - frame predictor 408 can have more than one intra - frame prediction mode. Thus, each mode can perform intra - frame prediction and provide the predicted signal to the mode selector 410. The mode selector 410 also receives a copy of the enhancement - layer picture 400.

[0057] Depending on which encoding mode is selected to encode the current block, the output of the inter - frame predictor 306, the output of the inter - frame predictor 406, or the output of one of the intra - frame predictor modes in the optional intra - frame predictor mode, or the output of the surface encoder in the mode selector is passed to the output of the mode selector 310, the mode selector 410. The output of the mode selector is passed to the first summing device 321, the first summing device 421. The first summing device can subtract the output of the pixel predictor 302, the pixel predictor 402 from the base - layer picture 300 / enhancement - layer picture 400 to generate the first prediction error signal 320, the first prediction error signal 420 (D n ), which is input to the prediction error encoder 303, the prediction error encoder 403.

[0058] The pixel predictors 302, 402 also receive the combined prediction representation (P’ n ) of the image blocks 312, 412 from the preliminary reconstructors 339, 439 and the outputs 338, 438 (D’ n ) of the prediction error decoders 304, 404. The preliminary reconstructed images 314, 414 (I’ n ) can be passed to the intra - frame predictors 308, 408 and the filters 316, 416. The filters 316, 416 that receive the preliminary representation can filter the preliminary representation and output the final reconstructed images 340, 440 (R’ n ), the final reconstructed images 340, 440 (R’ n) can be stored in reference frame memory 318 and reference frame memory 418. Reference frame memory 318 can be connected to inter - frame predictor 306 to be used as a reference image, against which future base - layer picture 300 is compared in the inter - frame prediction operation. According to some embodiments, in the case where the base layer is selected and indicated as the source for inter - layer sample prediction and / or inter - layer motion information prediction for the enhancement layer, reference frame memory 318 can also be connected to inter - frame predictor 406 to be used as a reference image, against which future enhancement - layer picture 400 is compared in the inter - frame prediction operation. In addition, reference frame memory 418 can be connected to inter - frame predictor 406 to be used as a reference image, against which future enhancement - layer picture 400 is compared in the inter - frame prediction operation.

[0059] According to some embodiments, subject to the base layer being selected and indicated as the source for predicting the filtering parameters of the enhancement layer, the filtering parameters of filter 316 from the first encoder portion 500 can be provided to the second encoder portion 502.

[0060] Prediction error encoders 303 and 403 include: transform units 342 and 442 (T) and quantizers 344 and 444 (Q). Transform units 342 and 442 transform the first prediction error signals 320 and 420 into the transform domain. The transform is, for example, a DCT transform. Quantizers 344 and 444 quantize the transform - domain signals (e.g., DCT coefficients) to form quantized coefficients.

[0061] Prediction error decoders 304 and 404 receive the outputs from prediction error encoders 303 and 403 and perform processes opposite to those of prediction error encoders 303 and 403 to generate decoded prediction error signals 338 and 438. When the decoded prediction error signals 338 and 438 are combined with the predicted representations of image blocks 312 and 412 at second summing devices 339 and 439, preliminary reconstructed images 314 and 414 are produced. Prediction error decoders 304 and 404 can be considered to include: de - quantizers 346 and 446 (Q -1 ), which de - quantize the quantized coefficient values (e.g., DCT coefficients) to reconstruct the transform signals; and inverse transform units 348 and 448 (T -1) which performs an inverse transform on the reconstructed transform signal, where the outputs of the inverse transform unit 348 and the inverse transform unit 448 include (a plurality of) reconstructed blocks. The prediction error decoder may further include a block filter which may filter the (a plurality of) reconstructed blocks according to further decoded information and filter parameters.

[0062] The entropy encoders 330, 430 (E) receive the outputs of the prediction error encoders 303, 403 and may perform suitable entropy coding / variable length coding on the signals to provide error detection and correction capabilities. The outputs of the entropy encoders 330, 430 may be inserted into the bitstream, for example, via the multiplexer 508 (M).

[0063] The concept of virtual boundaries is introduced in VVC. From the perspective of coding dependencies, pictures can be divided into different intervals by virtual boundaries. For example, for 360°: virtual boundaries are used to define the boundaries of different faces of 360° pictures in the CMP format, and for GDR (refer to the US Provisional Application No. 63 / 296,590 titled "New Gradual Decoding Refresh for ECM" filed in January 2022, which is incorporated herein by reference in its entirety), where virtual boundaries separate the refresh area and the non-refresh area of GDR / restored pictures. In VVC, virtual boundaries are specified in the SPS and / or picture headers.

[0064] There are three loop filters in VVC. The three loop filters are deblocking, SAO, and ALF. ECM utilizes new features to enhance the loop filters, including bilateral (JVET-F0034, JVET-V0094), BIF for chrominance (JVET-X0067), CCSAO (JVET-V0153, JVET-Y0106), CCALF (JVET-X0045), an alternative band classifier for ALF (JVET-X0070), and a CCSAO EDGE classifier (JVET-Y0106).

[0065] Loop filtering of the current pixel typically requires using the coding information of its neighbors. Therefore, filtering on one side of the virtual boundary may involve using the coding information on the other side of the virtual boundary.

[0066] For some applications, it is not desirable or may not be allowed for loop filtering to cross virtual boundaries. For example, in GDR, GDR / restored pictures can be divided into a refresh area and a non-refresh area by virtual boundaries. Refer to Figure 5, to avoid leakage, the refresh region 510 cannot use any information of the non-refresh region 530 because it cannot be guaranteed that the non-refresh region 530 is correctly decoded at the decoder. The misdecoded coded information may contaminate the refresh region 510, which may lead to leakage or mismatch of the encoder and decoder at the recovery point picture and consecutive pictures. Therefore, for GDR / recovery pictures, loop filtering cannot cross the virtual boundary 520 from the refresh region 510 to the non-refresh region 530, as indicated by the arrow 540.

[0067] On the other hand, sometimes it is perfectly acceptable to let loop filtering cross the virtual boundary. For example, as Figure 6 shown, in the same example of GDR, the non-refresh region 630 can use the information of the refresh region 610. Therefore, for GDR / recovery images, loop filtering can cross the virtual boundary 620 from the non-refresh region 630 to the refresh region 610, as indicated by the arrow 640.

[0068] In the current designs of VVC and ECM, loop filtering cannot cross the virtual boundary.

[0069] U.S. Provisional Application No. 63 / 362,243, "In-Loop Filtering at Virtual Boundaries", filed in March 2022, which is hereby incorporated by reference in its entirety, proposed several possible options for loop filtering at virtual boundaries. Among the several possible options is asymmetric loop filtering at virtual boundaries. Using this asymmetric option, loop filtering cannot cross the virtual boundary from one side of the virtual boundary to the other side, but can cross the virtual boundary from the other side to this side.

[0070] Specifically, loop filtering for side A of the virtual boundary cannot use the information of side B of the virtual boundary, but loop filtering for side B of the virtual boundary can use the information of this side A. If loop filtering for a pixel in side A of the virtual boundary requires using any information (such as pixels, coding modes, QP, etc.) of the other side B, then loop filtering is either not performed for this pixel or is still performed for this pixel, but the information of the other side is filled.

[0071] When performing asymmetric loop filtering at the virtual boundary, loop filtering for side A cannot use the information of side B, but loop filtering for side B is allowed to use the information of this side A.

[0072] If loop filtering for a pixel in side A requires using the coded information of side B, then loop filtering for this pixel in side A may not be performed properly.

[0073] Generally, loop filtering for pixels in the other side B can be normally performed because loop filtering for such pixels is allowed to use the coding information of both side A and the other side B. However, the other side B can choose not to use the coding information of side A. In this case, if loop filtering for pixels in the other side B requires the use of the coding information of side A, then loop filtering for such pixels in the other side B may not be normally performed.

[0074] Since the coding information of side A is available for the other side B, an offset based on the loop filtering of side A can be added to the output of the loop filtering of the other side B.

[0075] A virtual boundary is a line that is used to separate a picture, or a part of a picture, into two regions: a first region and a second region.

[0076] The virtual boundary can be vertical or horizontal. In VVC and ECM, the virtual boundary syntax is included in the SPS and / or picture header. In one embodiment, such as when performing an asymmetric operation at the virtual boundary, the first region is not allowed to use any information of the second region, but the second region can use the information of the first region.

[0077] In one exemplary embodiment, in the GDR / Recovery Picture, the first region is a clean (refreshed) region, and the second region is a dirty (unrefreshed) region. The clean (refreshed) region cannot use any information of the dirty (unrefreshed) region, but the dirty (unrefreshed) region can use the information of the clean (refreshed) region.

[0078] Loop filtering for a pixel can involve the use of the coding information of its neighbors.

[0079] If loop filtering for a pixel in the first region requires the use of the coding information of the second region (e.g., pixel, coding mode, reference picture, MV, QP, etc.), then loop filtering for such pixel may not be normally performed. The actual loop filtering for a pixel can take one of two possible options: Option 1, where loop filtering for such pixel in the first region is not performed; or Option 2, where loop filtering for such pixel in the first region is still performed, but using the coding information of the second region derived from the first region or set to a predetermined value when needed.

[0080] One embodiment related to Option 2 is that if loop filtering for a pixel in the first region requires the use of a pixel in the second region, then the pixel in the second region is filled according to the pixel in the first region.

[0081] Another embodiment related to Option 2 is that if the loop filtering of a pixel in the first region requires the use of a pixel in the second region, then that pixel in the second region is replaced by a pixel extrapolated from outside the first region.

[0082] Assume that the normal loop filtering of a pixel is the ideal loop filtering of that pixel with all necessary information used, and the actual loop filtering of a pixel is the practical loop filtering of that pixel with or without all necessary information used.

[0083] The actual loop filtering of a pixel in Option 1 or Option 2 generates an output that may be different from the normal loop filtering of that pixel, which can use the coding information of both the first region and the second region.

[0084] The loop filtering for the pixels in the second region can generally be performed normally because the loop filtering for the pixels in the second region is allowed to use the coding information of both the first region and the second region.

[0085] U.S. Provisional Application No. 63 / 388,385, filed on July 12, 2022, entitled "Asymmetric In-Loop Filters At Virtual Boundaries", which is hereby incorporated by reference in its entirety, describes that:

[0086] When the coding information of the second region is to be used to perform the filtering of a pixel in the first region, determine to perform the filtering of that pixel in the first region using the coding information of the second region derived from the first region or using the coding information of the second region set to a certain value, or

[0087] When the coding information of the second region is to be used to perform the filtering of a pixel in the first region, determine not to perform the filtering of that pixel in the first region.

[0088] The features described herein can be used to improve the design of the asymmetric deblocking filter at virtual boundaries. Specifically, the features described herein can be used to provide a longer filter length and more pixels to be filtered.

[0089] In VVC (and also in AVC and HEVC), the deblocking process can be performed at the TU (transform unit) and / or at the sub-block boundary. The purpose of deblocking is to smooth the block boundary and remove the blocky artifacts at the block boundary.

[0090] The deblock filter design includes two main processes. The two main processes are the filter strength (or length) decision and the actual filtering process. The following is a brief overview of the filter length decision in deblock design.

[0091] Two TUs can be adjacent horizontally or vertically, and deblocking can be performed on their vertical or horizontal boundaries. Figure 7 Examples are shown where two TUs of P and Q are adjacent horizontally, and thus, their vertical boundaries can be deblock filtered. As pointed out above, in alternative examples, adjacent blocks can be adjacent vertically and thus have horizontal boundaries.

[0092] The maximum filter length for the luminance component of TU P or TU Q can be determined based on the size (width or height) of the luminance component of TU P or TU Q. Examples are shown in Table 1 below.

[0093] P or Q maxFilterLengthP / Q 32 ≤ size 7 8 ≤ size < 32 3 Others 1

[0094] Table 1. Maximum filter length for the luminance component of P or Q

[0095] The maximum filter length for the chrominance components of TUs P and Q can be determined based on the size (width or height) of the chrominance components of TUs P and Q. Examples are shown in Table 2 below.

[0096] P and Q maxFilterLengthP / Q 8 ≤ size 3 Others 1

[0097] Table 2. Maximum filter length for the chrominance components of P and Q

[0098] A TU can cover a group of sub-blocks, and deblocking can be performed on the sub-block boundaries within the TU. Figure 8 Examples are shown where two TUs of P and Q are adjacent horizontally and Q contains sub-blocks. Those sub-block boundaries can also be deblock filtered.

[0099] For Figure 8 the example shown in, within TU Q, sub-block p and sub-block q are shown, which are adjacent horizontally and their vertical boundaries can be deblock filtered. The maximum filter length for the sub-block boundaries can be determined based on the distance between the sub-block boundaries and the associated TU boundaries. Examples are shown in Table 3 below.

[0100] Distance from the TU boundary maxFilterLengthp / q 0 min(5, maxFilterLengthP / Q) 4 1 8 2 >8 3

[0101] Table 3. Maximum filter length for the boundaries of sub-blocks p and q

[0102] For simplicity, in the following text, the term "block P" is used to mean both TU P and sub-block p, and the term "block Q" is used to mean both TU Q and sub-block q. Then, the maximum filter lengths of block P and block Q (maxFilterLengthP and maxFilterLengthQ) from a table (such as, for example, Example Table 3) can be used to determine which pixels on each side of P and Q will be filtered.

[0103] Figure 9 FIG. illustrates a simplified flow chart for filter strength (or length) decision for the luminance components of block P and block Q, where pi (i = 0, 1,..., 6) are the pixels on the P side, qi (i = 0, 1,..., 6) are the pixels on the Q side, and the index i associated with the pixel indicates the distance of the pixel from the block boundary. The smaller the pixel index i, the closer the pixel is to the boundary of block P and block Q.

[0104] As seen in the flow chart, there are several places 902, 904 that check whether both maxFilterLengthP and maxFilterLengthQ are greater than certain values, such as, for example, greater than value 2 and greater than value 1. If so, more pixels on each side of P and Q will be filtered. Otherwise, as Figure 9 illustrated, fewer pixels (or even no pixels) on each side of P and Q will be filtered.

[0105] For asymmetric deblocking at the virtual boundary, the actual filtering process may not be performed on the first side of the virtual boundary, but only on the second side of the virtual boundary. Note that the first side can still be filtered, but with padding, such as described in U.S. Provisional Patent Application No. 63 / 362,243.

[0106] In the following, it is assumed that block P and block Q are adjacent, and their block boundary is aligned with the virtual boundary, and the actual filtering process is only performed on the Q side and not on the P side. For example, in the GDR case, block P can be in the refresh region and block Q is in the non-refresh region. Thus, it can be determined that filtering is performed on the Q side and not on the P side. The actual deblocking filtering may only be performed on the pixels on the Q side of the virtual boundary and not on the pixels on the P side of the virtual boundary. The encoder and decoder may need to agree on which side is filtered and which side is not filtered, such as in the GDR case. If the P side is not filtered and the Q side is filtered, some variables on the P side may still need to be calculated, and the variable can be used for the Q side. Since the P side is not filtered, as further understood below regarding Figure 10 the filter length decision for the P side can be removed.

[0107] From Figure 9In the flowchart, it should be observed that the check on maxFilterLengthP may affect the number of pixels to be filtered on the Q side.

[0108] For an exemplary embodiment and method for asymmetric deblocking at a virtual boundary, Figure 9 the method shown in can be modified such that the maximum filter length on one side of the virtual boundary will not result in fewer pixels to be filtered on the other side of the virtual boundary. If the check on the maximum filter length on one side of the virtual boundary will result in fewer pixels to be filtered on the other side of the virtual boundary, then this check can be removed from the filter strength (or length) decision process on the other side. Figure 10 An example of a proposed flowchart illustrating this modified method is shown. In Figure 10 , the exemplary embodiment shows a flowchart for the filter strength (or length) decision for the luminance component on the Q side of the virtual boundary. By comparing Figure 10 with Figure 9 , it can be seen that the check on maxFilterLengthP has been removed in several places where this check may result in fewer pixels to be filtered on the Q side. Compared with 902, 1002 removes maxFilterLengthP>2. Compared with 904, 1004 reduces the step to "if(maxFilterLengthQ>1&&dq<(β+(β>>1))>>3)dEp=1". Other parts of the method are modified to remove P-side related steps, as indicated by the cross-through in 1006, 1008, 1010, 1012, and 1014. With the flowchart method proposed by this example, more pixels tend to be filtered on the Q side. The influence of the P side on the Q side can be avoided. The P side may not be filtered. The maximum filter length of P can be compared with a specific number (e.g., 3, 2, 1 as shown in Figure 9 ), but cannot be compared with the maximum filter length of Q. The decision to compare the maximum filter length of P with 3, 2, and / or 1 has an impact on the filter length of Q. If this impact may potentially result in fewer pixels to be filtered on the Q side, then the associated comparison of the maximum filter length of P with 3, 2, and 1 for the filter strength decision of Q will be removed. For example, in Figure 10 , we removed such comparisons in 1002 and 1004 because these two comparisons may result in fewer pixels to be filtered in Q. At the start of the flowchart ( Figure 10 ), there is also another comparison of the maximum filter length of P with 3. However, this comparison does not result in fewer pixels to be filtered in Q. So, we still keep this comparison in the flowchart.

[0109] Since the maximum filter length of P is not compared with the maximum filter length of Q, the claims may need to be modified accordingly.

[0110] Figure 11 A simplified flowchart of the filter strength (or length) decision for the chrominance components of blocks P and Q is shown. As can be seen in this example, there are several places in the flowchart where maxFilterLengthP may affect the filter strength (or length) decision on the Q side.

[0111] With Figure 11 Similarly, Figure 12 A proposed modified flowchart of the filter strength (or length) decision for the chrominance components on the Q side of the virtual boundary is shown, where the check for maxFilterLengthP has been removed from the filter strength (or length) decision on the Q side. 1102 has been changed to 1202, where maxFilterLengthP == 1 has been removed. 1104 has been changed to 1204, where "maxFilterLengthP == maxFilterLengthQ == 1" has been changed to remove maxFilterLengthP == 1 and only have maxFilterLengthQ == 1. 1106 has been changed to 1206, where "maxFilterLengthP == 3 and maxFilterLengthQ == 3" has been changed to only "maxFilterLengthQ == 3". 1108 has been changed to 1208, where "maxFilterLengthP == 1" has been removed. Additionally, the last conditional check for maxFilterLengthQ has been changed from

[0112] maxFliterLengthQ == 3

[0113] Changed to

[0114] maxFliterLengthQ == 1

[0115] The width and / or height of the chrominance block may be as small as 2 pixels, and the actual filtering on the Q side may require using up to 3 pixels on the P side. In other words, block P can be as small as 2x2 pixels, and the filtering of the horizontal pixels in block Q may require up to three horizontal pixels from block P. However, since block P has only two horizontal pixels, this causes problems. To avoid the situation where the actual filtering on the Q side requires three pixels from block P which has only 2 pixels (width or height), it is further proposed to change the maximum filter length on the Q side from

[0116] maxFilterLengthQ = (sizeP >= 8) && (sizeQ >= 8)? 3 : 1 is changed to

[0117] maxFilterLengthQ = (sizeP >= 4) && (sizeQ >= 8)? 3 : 1

[0118] If block P and block Q are adjacent, their block boundaries are aligned with the virtual boundary, and the actual filtering process may be performed only on the P side; and not on the Q side. Note that Figure 10 and Figure 12 are merely examples and should not be considered limiting. The letters "P" and "Q" in the flowcharts described above as well as Figure 10 and Figure 12 should be swapped. Thus, for Figure 10 , instead of removing the "P"-related elements from the steps, equivalent "Q"-related elements may be removed. Similarly, for Figure 12 , instead of deleting the "P"-related elements from the steps, equivalent "Q"-related elements may be removed.

[0119] Asymmetric filter signaling

[0120] In some example embodiments, information about asymmetric loop filtering (e.g., deblocking) at the virtual boundary may be signaled in the sequence parameter set (SPS), picture parameter set (PPS), picture header, and / or slice header. This information may include, for example:

[0121] · (Multiple) flags indicating whether asymmetric filtering is applied at the virtual boundary,

[0122] · If asymmetric filtering is applied at the virtual boundary, (multiple) syntax indicating on which side the actual filtering process is performed.

[0123] It should be understood from the above description that the example method can be extended to deblocking at the virtual boundary. That is, the maximum filter length on one side of the virtual boundary should not result in fewer pixels to be filtered on the other side of the virtual boundary. If the check of the maximum filter length on one side of the virtual boundary would result in fewer pixels to be filtered on the other side of the virtual boundary, then that check may be removed from the filter strength (or length) decision on the other side.

[0124] The above method can be extended to deblocking at slice / tile / sub - picture boundaries. That is, the maximum filter length on one side of the slice / tile / sub - picture boundary should not result in fewer pixels to be filtered on the other side of the slice / tile boundary. If the check of the maximum filter length on one side of the slice / tile / sub - picture boundary will result in fewer pixels to be filtered on the other side of the slice / tile / sub - picture boundary, then the check can be removed from the filter strength (or length) decision on the other side.

[0125] The above feature can be extended to deblocking at block boundaries. That is, the maximum filter length on one side of the block boundary should not result in fewer pixels to be filtered on the other side of the block boundary. If the check of the maximum filter length on one side of the block boundary will result in fewer pixels to be filtered on the other side of the block boundary, then the check should be removed from the filter strength (or length) decision on the other side.

[0126] An example apparatus can be provided, including: at least one processor; and at least one non - transitory memory storing instructions which, when executed using the at least one processor, cause the apparatus to: determine a virtual boundary that separates a picture, or a part of a picture, into a first region and a second region; determine to perform filtering on at least one pixel of the first region and not perform filtering on pixels of the second region; and filter at least one pixel of the first region using coding information of the second region.

[0127] The apparatus may be configured such that when the instruction is executed using at least one processor, the apparatus: determines to perform filtering of at least one pixel of a first region and not to perform filtering of pixels of a second region based on a comparison of a first maximum filter length value for the first region relative to a second maximum filter length value for the second region. The apparatus may be configured such that when the instruction is executed using at least one processor, the apparatus: determines not to perform filtering of pixels of the second region when the comparison indicates that fewer pixels in the second region will be filtered compared to at least two pixels in the first region. The apparatus may be configured such that filtering includes: deblocking at a virtual boundary. The apparatus may be configured such that when the instruction is executed using at least one processor, the apparatus: performs an inspection when the maximum filter length on a first side of the virtual boundary would result in fewer pixels being filtered on a second side of the virtual boundary. The apparatus may be configured such that when the instruction is executed using at least one processor, the apparatus: removes at least one process from the filter length decision for the second side. The apparatus may be configured such that: at least one process includes an inspection of the maximum filter length on the second side of the virtual boundary in the filter length decision. If the inspection process results in fewer pixels being filtered in a first region of the virtual boundary, at least one process may include an inspection process for a certain value or number of the maximum filter length calculated on the second side of the virtual boundary. The apparatus may be configured such that when the instruction is executed using at least one processor, the apparatus: removes at least one process from the filter length decision of the deblocking process on the first side. The apparatus may be configured such that: if the inspection process results in fewer pixels being filtered in a first region of the virtual boundary, at least one process includes an inspection process for a specific value or number of the maximum filter length calculated on the second side of the virtual boundary. The apparatus may be configured such that: the virtual boundary includes a slice boundary or a tile boundary or a sub-picture boundary. The apparatus may be configured such that the virtual boundary includes a block boundary.

[0128] In one example embodiment, when the instruction is executed using at least one processor, the apparatus: determines to perform filtering of at least one pixel of a first region and not to perform filtering of pixels of a second region based on at least one predetermined rule set or at least one syntax set in the SPS, PPS, picture header, or slice header, the at least one predetermined rule set or the at least one syntax set indicating: where the virtual boundary is located in the picture or the portion of the picture, which side of the virtual boundary will be filtered, or which side of the virtual boundary will not be filtered.

[0129] Also refer to Figure 13, an example method 1300 can be provided, including: determining a virtual boundary that separates an image, or a portion of the image, into a first region and a second region, as illustrated by block 1302; determining to perform filtering on at least one pixel of the first region and not perform filtering on the pixels of the second region, as illustrated by block 1304; and filtering at least one pixel of the first region using the coding information of the second region, as illustrated by block 1306.

[0130] An example embodiment can be provided with a non-transitory computer-readable medium that includes program instructions that, when executed using a device, cause the device to perform at least the following: determining a virtual boundary that separates an image, or a portion of the image, into a first region and a second region; determining to perform filtering on at least one pixel of the first region and not perform filtering on the pixels of the second region; and filtering at least one pixel of the first region using the coding information of the second region.

[0131] An example embodiment can be provided with a device that includes: a component for determining a virtual boundary that separates an image, or a portion of the image, into a first region and a second region; a component for determining to perform filtering on at least one pixel of the first region and not perform filtering on the pixels of the second region; and a component for filtering at least one pixel of the first region using the coding information of the second region.

[0132] The term "non-transitory" as used herein is a limitation on the medium itself (i.e., tangible, rather than a signal), rather than a limitation on data storage persistence (e.g., RAM vs. ROM).

[0133] It should be understood that the above description is illustrative only. Various alternatives and modifications can be designed by those skilled in the art. For example, the features detailed in the respective dependent claims can be combined with each other in any suitable combination. In addition, features from the different embodiments above can be selectively combined into new embodiments. Therefore, this description is intended to cover all such alternatives, modifications, and variations that fall within the scope of the appended claims.

[0134] The following acronyms and abbreviations that can be found in the specification and / or the drawings are defined as follows. The acronyms and abbreviations can be appended to each other and / or appended with other characters (e.g., hyphens (-)).

[0135] 3GPP Third Generation Partnership Project

[0136] 4G Fourth Generation Wideband Cellular Network Technology

[0137] 5G Fifth Generation Cellular Network Technology

[0138] IEEE standard series for LANs and MANs of 802.x

[0139] ABC alternative band classifier

[0140] ALF adaptive loop filter

[0141] APS adaptive parameter set

[0142] ASIC application specific integrated circuit

[0143] BD bit depth

[0144] BIF bilateral filter

[0145] BIF-chroma bilateral filter for chrominance

[0146] BIF-luma bilateral filter for luma

[0147] BO band offset

[0148] Cb blue chrominance component

[0149] CCALF or CC-ALF cross-component ALF

[0150] CCSAO cross-component SAO

[0151] CDMA code division multiple access

[0152] CMP cube map projection

[0153] CPE customer premises equipment

[0154] Cr red chrominance component

[0155] CTB coding tree block

[0156] CTU coding tree unit

[0157] CU coding unit

[0158] DBF deblocking filter

[0159] DCT discrete cosine transform

[0160] DSP digital signal processor

[0161] ECM enhanced compression model

[0162] EO edge offset

[0163] FDMA frequency division multiple access

[0164] FPGA field programmable gate array

[0165] GDR Progressive Decoding Refresh

[0166] GSM Global System for Mobile Communications

[0167] H.222.0 MPEG-2 System, a standard for the generic coding of moving pictures and associated audio information

[0168] H.26x A series of video coding standards in the ITU-T domain

[0169] HMD Head-Mounted Display

[0170] IBC Intra Block Copy

[0171] id or ID Identifier

[0172] IEC International Electrotechnical Commission

[0173] IEEE Institute of Electrical and Electronics Engineers

[0174] I / F Interface

[0175] IMD Integrated Messaging Device

[0176] IMS Instant Messaging Service

[0177] I / O Input / Output

[0178] IoT Internet of Things

[0179] IP Internet Protocol

[0180] ISO International Organization for Standardization

[0181] ISOBMFF ISO Base Media File Format

[0182] ITU International Telecommunication Union

[0183] ITU-T ITU Telecommunication Standardization Sector

[0184] JTC Joint Technical Committee

[0185] JVET Joint Video Exploration Team

[0186] LEE Laptop Embedded Equipment

[0187] LME Laptop Mounted Equipment

[0188] LTE Long-Term Evolution

[0189] ML Machine Learning

[0190] MMS Multimedia Messaging Service

[0191] Moving Picture Experts Group

[0192] MPEG-2 is H.222 / H.262 defined by ITU

[0193] Mean Squared Error

[0194] Multi-View

[0195] Network Abstraction Layer

[0196] Neural Network

[0197] Network

[0198] Personal Computer

[0199] Personal Digital Assistant

[0200] Packet Identifier

[0201] Power Line Communication

[0202] Quantization Parameter or Quarter Pixel

[0203] Random Access Memory

[0204] Radio Frequency Identification

[0205] Reference Frame Memory

[0206] Read Only Memory

[0207] Receiver

[0208] Sample Adaptive Offset

[0209] Short Message Service

[0210] Sequence Parameter Set

[0211] Transmission Control Protocol - Internet Protocol

[0212] Time Division Multiple Access

[0213] Transport Stream

[0214] Transformation Unit

[0215] Television

[0216] Transmitter

[0217] Blue Projection of Chrominance Component

[0218] Universal Integrated Circuit Card

[0219] UMTS Universal Mobile Telecommunications System

[0220] USB Universal Serial Bus

[0221] Red projection of the V chrominance component

[0222] V2X Vehicle-to-Everything

[0223] VoIP Voice over Internet Protocol

[0224] VVC Versatile Video Coding

[0225] WLAN Wireless Local Area Network

[0226] Y Luminance component

Claims

1. An apparatus, comprising: at least one processor; and at least one memory storing instructions that, when executed using the at least one processor, cause the apparatus to: determine a virtual boundary that separates an image, or a portion of the image, into a first region and a second region; and determine to perform filtering on at least one pixel of the first region and not perform filtering on pixels of the second region; and filter the at least one pixel of the first region using coding information of the second region.

2. The apparatus according to claim 1, wherein the instructions, when executed using the at least one processor, cause the apparatus to: determine to perform the filtering on the at least one pixel of the first region and not perform filtering on pixels of the second region based on at least one predetermined rule set or at least one syntax set among a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, or a slice header, the at least one predetermined rule set or the at least one syntax set indicating where the virtual boundary is located in the image or the portion of the image, which side of the virtual boundary is to be filtered, or which side of the virtual boundary is not to be filtered.

3. The apparatus according to claim 1, wherein the filtering comprises: A deblocking process at the virtual boundary.

4. The apparatus according to claim 3, wherein the instructions, when executed using the at least one processor, cause the apparatus to: remove at least one process from a filter length decision of the deblocking process on a first side of the virtual boundary.

5. The apparatus according to claim 4, wherein if the inspection process results in fewer pixels to be filtered in the first region of the virtual boundary, the at least one process includes: A process of checking a maximum filter length calculated on a second side of the virtual boundary for a certain value or number.

6. The apparatus according to claim 1, wherein the virtual boundary comprises: A slice boundary, or a tile boundary, or a sub-picture boundary.

7. The apparatus according to claim 1, wherein the virtual boundary includes a block boundary.

8. A method, comprising: determining a virtual boundary that separates an image, or a portion of the image, into a first region and a second region; and determining to perform filtering on at least one pixel of the first region and not perform filtering on pixels of the second region; and filtering the at least one pixel of the first region using coding information of the second region.

9. A non-transitory computer-readable medium including program instructions that, when executed using an apparatus, cause the apparatus to perform at least the following: Determine a virtual boundary that separates the picture, or a part of the picture, into a first region and a second region; and determine to perform filtering on at least one pixel of the first region and not perform filtering on pixels of the second region; and filter the at least one pixel of the first region using coding information of the second region.

10. An apparatus, comprising: means for determining a virtual boundary that separates an image, or a portion of the image, into a first region and a second region; and means for determining to perform filtering on at least one pixel of the first region and not perform filtering on pixels of the second region; and means for filtering the at least one pixel of the first region using coding information of the second region.

Images