In-loop filter at virtual boundaries

Abstract

An apparatus may be configured to: select at least one first pixel from a first region of a picture, wherein the first region is separated from a different second region by a virtual boundary; determining at least one second pixel for filtering of the at least one first pixel, wherein the at least one second pixel is located in the second region; determining whether the at least one first pixel is to be filtered; and based on the determination that the at least one first pixel is to be filtered, causing the at least one first pixel to be filtered based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of encoded information associated with at least one second pixel populated based on the encoded information of the first region.

Description

Technical Field

[0001] The exemplary and non-limiting embodiments relate generally to inputs to filters, and more particularly to in-loop filters with respect to virtual boundaries. Background Art

[0002] As is known, in general video coding, in-loop filtering is not performed across virtual boundaries. Summary of the invention

[0003] The following summary is for illustration only. It is not intended to limit the scope of the claims.

[0004] According to one aspect, a device includes: at least one processor; and at least one memory including computer program code; the at least one memory and the computer program code are configured to, together with the at least one processor, enable the device to at least: select at least one first pixel from a first region of a picture, wherein the first region is separated from a different second region by a virtual boundary; determine at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel is located in the second region; determine whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filter the at least one first pixel based on at least one of the following: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is filled based on the encoding information of the first region.

[0005] According to one aspect, a method includes: selecting at least one first pixel from a first region of a picture, wherein the first region is separated from a different second region by a virtual boundary; determining at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel is located in the second region; determining whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filtering the at least one first pixel based on at least one of the following: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is padded based on the encoding information of the first region.

[0006] According to one aspect, a device includes components for performing the following: selecting at least one first pixel from a first region of a picture, wherein the first region is separated from a different second region by a virtual boundary; determining at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel is located in the second region; determining whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filtering the at least one first pixel based on at least one of the following: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is padded based on the encoding information of the first region.

[0007] According to one aspect, a non-transitory computer-readable medium includes program instructions stored thereon, which, when executed by at least one processor, cause the at least one processor to: select at least one first pixel from a first region of a picture, wherein the first region is separated from a different second region by a virtual boundary; determine at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel is located in the second region; determine whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filter the at least one first pixel based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is padded based on the encoding information of the first region. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above aspects and other features are explained in the following description in conjunction with the accompanying drawings, in which:

[0009] Figure 1 is a block diagram of one possible non-limiting example system in which example embodiments may be practiced;

[0010] Figure 2 is a block diagram of one possible non-limiting exemplary system in which example embodiments may be practiced;

[0011] Figure 3 is a diagram illustrating features as described herein;

[0012] Figure 4 is a diagram illustrating features as described herein;

[0013] Figure 5 is a diagram illustrating features as described herein;

[0014] Figure 6 is a diagram illustrating features as described herein;

[0015] Figure 7 is a diagram illustrating features as described herein;

[0016] Figure 8 is a diagram illustrating features as described herein;

[0017] Fig. 9 is a diagram illustrating features as described herein;

[0018] Fig.10 is a diagram illustrating features as described herein;

[0019] Fig.11 is a diagram illustrating features as described herein;

[0020] Fig.12 is a diagram illustrating features as described herein;

[0021] Fig.13 is a diagram illustrating features as described herein;

[0022] Fig.14 is a diagram illustrating features as described herein;

[0023] Fig.15 is a diagram illustrating features as described herein;

[0024] Fig.16 is a diagram illustrating features as described herein;

[0025] Fig.17 is a diagram illustrating features as described herein;

[0026] Fig.18 is a diagram illustrating features as described herein;

[0027] Fig.19 is a diagram illustrating features as described herein; and

[0028] Fig. 20 is a flow chart illustrating the steps as described herein. DETAILED DESCRIPTION

[0029] The following abbreviations that may appear in the specification and / or drawings are defined as follows:

[0030] 4G: Fourth Generation

[0031] 5G: Fifth Generation

[0032] 5GC: 5G core network

[0033] ANN: Artificial Neural Network

[0034] ALF: Adaptive Loop Filter

[0035] BIF: Bilateral Filter

[0036] CCAF: Cross-Component ALF

[0037] CCSAO: Cross-component SAO

[0038] CDMA: Code Division Multiple Access

[0039] CPU: Central Processing Unit

[0040] CTU: Coding Tree Unit

[0041] CU: Coding Unit

[0042] DSP: Digital Signal Processor

[0043] ECM: Enhanced Compression Model

[0044] eNB (or eNodeB): Evolved Node B (e.g., LTE base station)

[0045] E-UTRA: Evolved Universal Terrestrial Radio Access, also known as LTE radio access technology

[0046] GDR: Gradual Decode Refresh

[0047] gNB (or gNodeB): A base station for 5G / NR, i.e., a node that provides NR user plane and control plane protocol termination towards the UE and is connected to the 5GC via the NG interface

[0048] FDMA: Frequency Division Multiple Access

[0049] GPU: Graphics Processing Unit

[0050] GSM: Global System for Mobile Communications

[0051] HMD: Head-mounted display

[0052] IEEE: Institute of Electrical and Electronics Engineers

[0053] IMD: Integrated Messaging Device

[0054] IMS: Instant Messaging Service

[0055] IoT: Internet of Things

[0056] JVET: Joint Video Experts Team

[0057] LTE: Long Term Evolution

[0058] MMS: Multimedia Messaging Service

[0059] MPEG-I: Moving Picture Experts Group Immersive Codec Family

[0060] MV: Motion Vector

[0061] ng or NG: New Generation

[0062] ng-eNB or NG-eNB: Next Generation eNB

[0063] NR: New Radio

[0064] PC: Personal Computer

[0065] PDA: Personal Digital Assistant

[0066] QP: Quarter Pixel

[0067] SAO: Sample Adaptive Offset

[0068] SMS: Short Message Service

[0069] SPS: Sequence Parameter Set

[0070] TCP-IP: Transmission Control Protocol Internet Protocol

[0071] TDMA: Time Division Multiple Access

[0072] UICC: Universal Integrated Circuit Card

[0073] UMTS: Universal Mobile Telecommunications System

[0074] VVC: Versatile Video Coding

[0075] The following describes suitable devices and possible mechanisms for practicing example embodiments of the present disclosure. Figure 1 , Figure 1An example block diagram of an apparatus 50 is shown. The apparatus may be configured to perform various functions, such as, for example, collecting information through one or more sensors, encoding and / or decoding information, receiving and / or sending information, analyzing information collected or received by the apparatus, etc. A device configured to encode a video scene may (optionally) include: (one or more) microphones for capturing a scene, and / or one or more sensors, such as cameras, for capturing information about the physical environment in which the scene is captured. Alternatively, a device configured to encode a video scene may be configured to receive information about the environment in which the scene is captured and / or a simulated environment. A device configured to decode and / or render a video scene may be configured to receive a Moving Picture Experts Group Immersive Codec Series (MPEG-I) bitstream including an encoded video scene. A device configured to decode and / or render a video scene may include one or more speakers / audio transducers and / or displays, and / or may be configured to send a decoded scene or signal to a device including one or more speakers / audio transducers and / or displays. Devices configured to decode and / or render video scenes may include: a user device, a head mounted display, or another device capable of rendering an AR, VR, and / or MR experience to a user.

[0076] The electronic device 50 may be, for example, a mobile terminal or user equipment of a wireless communication system. Alternatively, the electronic device may be a non-mobile computer or part of a computer. It should be understood that embodiments of the present invention may be implemented in any electronic device or device that can process data. The electronic device 50 may include a device that can access a network and / or cloud through a wired or wireless connection. The electronic device 50 may include one or more processors 56, one or more memories 58, and one or more transceivers 52 interconnected by one or more buses. The one or more processors 56 may include a central processing unit (CPU), and / or a graphics processing unit (GPU). Each transceiver in the one or more transceivers 52 includes a receiver and a transmitter. The one or more buses may be address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, optical fiber, or other optical communication equipment. One or more transceivers may be connected to one or more antennas 44. One or more memories 58 may include computer program code. One or more memories 58 and computer program code may be configured to, together with one or more processors 56, enable the electronic device 50 to perform one or more operations described herein.

[0077] The electronic device 50 can be connected to a node of a network. The network node may include one or more processors, one or more memories, and one or more transceivers interconnected by one or more buses. Each transceiver in the one or more transceivers includes a receiver and a transmitter. The one or more buses may be address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, optical fiber, or other optical communication equipment, etc. The one or more transceivers may be connected to one or more antennas. The one or more memories may include computer program code. The one or more memories and computer program code may be configured to, together with the one or more processors, cause the network node to perform one or more operations in the operations described herein.

[0078] The electronic device 50 may include a microphone 36 or any suitable audio input, which may be a digital or analog signal input. The electronic device 50 may also include an audio output device 38, which in embodiments of the present invention may be any one of headphones, speakers, or analog audio, or digital audio output connections. The electronic device 50 may also include a battery (or in other embodiments of the present invention, the device may be powered by any suitable mobile energy device, such as a solar cell, a fuel cell, or a clockwork generator). The electronic device 50 may also include a camera 42, or other sensors capable of recording or capturing images, and / or videos. Additionally or alternatively, the electronic device 50 may also include a depth sensor. The electronic device 50 may also include a display 32. The electronic device 50 may also include an infrared port for short-range line-of-sight communication with other devices. In other embodiments, the device 50 may also include any suitable short-range communication solution, such as, for example, BLUETOOTH. TM Wireless connection, or USB / Firewire wired connection.

[0079] It should be understood that the electronic device 50 configured to perform the example embodiments of the present disclosure may have fewer and / or additional components, which may correspond to the process that the electronic device 50 is configured to perform. For example, a device configured to encode a video may not include a speaker or an audio transducer, and may include a microphone, while a device configured to render a decoded video may not include a microphone, and may include a speaker or an audio transducer.

[0080] Reference now Figure 1, the electronic device 50 may include a controller 56, a processor, or a processor circuit system for controlling the device 50. The controller 56 may be connected to a memory 58, which in embodiments of the present invention may store data in the form of image and audio data, and / or may also store instructions for implementation on the controller 56. The controller 56 may also be connected to a codec circuit system 54, which is suitable for performing encoding and / or decoding of audio and / or video data or assisting in encoding and / or decoding performed by the controller.

[0081] The electronic device 50 may also include a card reader 48 and a smart card 46, such as a UICC and a UICC reader, for providing user information and suitable for providing authentication information to authenticate and authorize the user / electronic device 50 at the network. The electronic device 50 may also include an input device 34, such as a keypad, one or more input buttons, or a touch screen input device, for providing information to the controller 56.

[0082] The electronic device 50 may include a radio interface circuit system 52, which is connected to the controller and is suitable for generating wireless communication signals, for example, for communication with a cellular communication network, a wireless communication system or a wireless local area network. The device 50 may also include an antenna 44 connected to the radio interface circuit system 52 for transmitting radio frequency signals generated at the radio interface circuit system 52 to (multiple) other devices and / or for receiving radio frequency signals from (multiple) other devices.

[0083] The electronic device 50 may include a microphone 38, a camera 42, and / or other sensors capable of recording or detecting audio signals, image / video signals, and / or other information about the local / virtual environment, which is then passed to a codec 54 or a controller 56 for processing. The electronic device 50 may receive an audio / image / video signal and / or information about the local / virtual environment from another device before transmission and / or storage for processing. The electronic device 50 may also receive an audio / image / video signal and / or information about the local / virtual environment wirelessly or via a wired connection for encoding / decoding. The structural elements of the electronic device 50 described above represent examples of components for performing corresponding functions.

[0084] The memory 58 may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, flash memory, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The memory 58 may be a non-transitory memory. The memory 58 may be a component for performing a storage function. The controller 56 may be or include one or more processors, which may be of any type suitable for the local technical environment and, as a non-limiting example, may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and a processor based on a multi-core processor architecture. The controller 56 may be a component for performing a function.

[0085] According to an example embodiment of the present disclosure, the electronic device 50 may be configured to perform the capture of a volumetric scene. For example, the electronic device 50 may include a camera 42 or other sensor capable of recording or capturing images and / or video. The electronic device 50 may also include one or more transceivers 52 to enable the transmission of the captured content for processing at another device. Such an electronic device 50 may or may not include Figure 1 All modules shown in .

[0086] According to an example embodiment of the present disclosure, the electronic device 50 may be configured to perform processing of volumetric video content. For example, the electronic device 50 may include a controller 56 for processing images to generate volumetric video content, a controller 56 for processing volumetric video content to project 3D information into 2D information, patches, and auxiliary information, and / or a codec 54 for encoding the 2D information, patches, and auxiliary information into a bitstream for transmission to another device having a radio interface 52. Such an electronic device 50 may or may not include Figure 1 All modules shown in .

[0087] According to an exemplary embodiment of the present disclosure, the electronic device 50 may be configured to perform encoding or decoding of 2D information representing volumetric video content. For example, the electronic device 50 may include a codec 54 for encoding or decoding 2D information representing volumetric video content. Such an electronic device 50 may or may not include Figure 1 All modules shown in .

[0088] According to an exemplary embodiment of the present disclosure, the electronic device 50 may be configured to perform rendering of the decoded 3D volumetric video. For example, the electronic device 50 may include a controller for projecting 2D information to reconstruct the 3D volumetric video, and / or a display 32 for rendering the decoded 3D volumetric video. Such an electronic device 50 may or may not include Figure 1 All modules shown in .

[0089] about Figure 2 , showing an example of a system in which embodiments of the present invention may be utilized. System 10 includes multiple communication devices that can communicate over one or more networks. System 10 may include any combination of wired or wireless networks, including but not limited to wireless cellular telephone networks (such as GSM, UMTS, E-UTRA, LTE, CDMA, 4G, 5G networks, etc.), wireless local area networks (WLANs) such as those defined by any of the IEEE 802.x standards, BLUETOOTH TM Personal Area Network, Ethernet LAN, Token Ring LAN, Wide Area Network, and / or the Internet.

[0090] System 10 may include both wired and wireless communication devices and / or electronic devices suitable for implementing embodiments of the present invention.

[0091] For example, Figure 2 The system shown in shows a representation of the mobile phone network 11 and the Internet 28. The connection to the Internet 28 may include, but is not limited to, long-range wireless connections, short-range wireless connections, and various wired connections, including, but not limited to, telephone lines, cable lines, power lines, and similar communication paths.

[0092] The example communication devices shown in system 10 may include, but are not limited to, a combination of a device 15, a personal digital assistant (PDA) and a mobile phone 14, a PDA 16, an integrated messaging device (IMD) 18, a desktop computer 20, a laptop computer 22, and a head mounted display (HMD) 17. The electronic device 50 may include any one of these example communication devices. In example embodiments of the present disclosure, more than one of these devices, or multiple of one or more of these devices may perform the disclosed process(es). These devices may be connected to the Internet 28 via a wireless connection 2.

[0093] Embodiments may also be implemented in set-top boxes; i.e., digital television receivers that may or may not have a display or wireless capabilities; in tablets or (laptop) personal computers (PCs) with hardware and / or software for processing neural network data; in various operating systems; and in chipsets, processors, DSPs, and / or embedded systems that provide hardware / software based coding. The embodiments may also be implemented in cellular phones (such as smartphones), tablets, personal digital assistants (PDAs) with wireless communication capabilities, portable computers with wireless communication capabilities, image capture devices with wireless communication capabilities (such as digital cameras), gaming devices with wireless communication capabilities, music storage and playback devices with wireless communication capabilities, Internet devices that allow wireless Internet access and browsing, tablets with wireless communication capabilities, and portable units or terminals that incorporate combinations of such functionality.

[0094] Some or additional devices may send and receive calls and messages and communicate with a service provider via a wireless connection 25 to a base station 24, which may be, for example, an eNB, gNB, etc. The base station 24 may be connected to a network server 26 that allows communication between the mobile phone network 11 and the Internet 28. The system may include additional communication devices and various types of communication devices.

[0095] Communication devices may communicate using a variety of transmission technologies, including but not limited to Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Transmission Control Protocol-Internet Protocol (TCP-IP), Short Message Service (SMS), Multimedia Message Service (MMS), Email, Instant Messaging Service (IMS), BLUETOOTH TM , IEEE 802.11, 3GPP narrowband IoT, and any similar wireless communication technology. The communication devices involved in the implementation of various embodiments of the present invention can use various media for communication, including but not limited to radio, infrared, laser, cable connection, and any suitable connection.

[0096] In telecommunications and data networks, a channel can refer to a physical channel or a logical channel. A physical channel can refer to a physical transmission medium such as a wire, while a logical channel can refer to a logical connection through a multiplexed medium that can transmit several logical channels. A channel can be used to transmit an information signal (e.g., a bit stream, which can be an MPEG-I bit stream) from one or more senders (or transmitters) to one or more receivers.

[0097] The features described herein are generally associated with virtual boundaries. The concept of virtual boundaries is introduced in Versatile Video Coding (VVC) (B. Bross, J. Chen, S. Liu, YK Wang's "Versatile Video Coding" (JVET-O2001-vE, June 2020)). In one example, from the perspective of coding dependency, a picture can be divided into different areas by / using virtual boundaries. For example, a virtual boundary can be used to define the boundaries of different faces of a 360° picture in a CMP file format. In another example, a virtual boundary can be used to separate (multiple) refresh areas and (multiple) non-refresh areas of a progressive decoding refresh (GDR) / recovery picture (see L. Wang, S. Hong, and K. Panusopone's "Gradual Decoding Refresh for VVC" (JVET-Q0527, January 2020); S. Hong, L. Wang, and K. Panusopone's "GDR Software" (JVET-T0078, October 2020)). In another example, in Versatile Video Coding (VVC), virtual boundaries may be specified in a sequence parameter set (SPS) and / or a picture header.

[0098] The features described herein may be related to in-loop filters in VVC and the Enhanced Compression Model (ECM), but this is not limiting. There are three in-loop filters in VVC: deblocking, sample adaptive offset (SAO), and adaptive loop filter (ALF). ECM can enhance the in-loop filter through new features, including bilateral (JVET-F0034, JVET-V0094, JVET-X0067), cross-component SAO (CCSAO) (JVET-V0153), longer cross-component ALF (CCALF) (JVET-X0045), alternative band classifiers for ALF (JVET-X0070), and CCSAO EDGE classifiers (JVET-Y0106) (see M. Coban, F. Leannec, MgSarwer, and J. Strom, "Algorithm description of Enhanced Compression Model 3 (ECM 3)" (JVET-X2025, January 2022)).

[0099] The in-loop filter of the current pixel usually needs to use the coded information of its neighbors. Therefore, filtering on one side of a virtual boundary can involve using the coded information on the other side of the virtual boundary. However, for some applications, filtering across a virtual boundary may not be desirable. For example, in GDR, the GDR / restored picture can be divided into refreshed area(s) and non-refreshed area(s) by virtual boundaries(s). To avoid leakage, such as Figure 3 As shown in , the refresh area cannot use any information from the non-refresh area because there is no guarantee that the non-refresh area will be correctly decoded at the decoder. Incorrectly decoded encoded information may contaminate the refresh area, resulting in leakage. Therefore, for GDR / restored pictures, in-loop filtering cannot cross the virtual boundary from the refresh area to the non-refresh area.

[0100] Reference now Figure 3 , illustrates a refresh area 310 separated from a non-refresh area 330 by a virtual boundary 320. As indicated by arrow 340, in-loop filtering of pixels of the refresh area 310 based on the encoding information of the non-refresh area 330 may not be allowed.

[0101] In other cases, it may be possible to filter across virtual boundaries. For example, in the same example of GDR, non-refreshed areas can use information from refreshed areas, such as Figure 4 Therefore, for GDR / restored pictures, in-loop filtering can cross the virtual boundary from the non-refreshed area to the refreshed area.

[0102] Reference now Figure 4 , illustrates a refresh area 410 separated from a non-refresh area 430 by a virtual boundary 420. As indicated by arrow 440, in-loop filtering may be performed on pixels of the non-refresh area 430 based on the encoding information of the refresh area 410.

[0103] In the current version of VVC, the in-loop filter may not be allowed to cross the virtual boundary. In an exemplary embodiment of the present disclosure, the (in-loop) filtering on one side of the virtual boundary may be allowed to use the coding information (pixels, coding mode, motion vector (MV), quarter pixel (QP), etc.) on the other side of the virtual boundary. Additionally or alternatively, in an exemplary embodiment of the present disclosure, the (in-loop) filtering on one side of the virtual boundary may not be allowed to use the coding information (pixels, coding mode, MV, QP, etc.) on the other side of the virtual boundary. The choice of using or not using the coding information on the other side of the virtual boundary may be a default, or may be signaled in a sequence header and / or a picture header and / or a slice header. In other words, it may be determined whether to perform filtering on the pixels on one side based on the pixels on the other side of the virtual boundary. If it is determined that filtering is to be performed, filtering may be performed according to an exemplary embodiment of the present disclosure. If it is determined that filtering is not required, filtering of the pixels on one side may be omitted or not performed. In an exemplary embodiment, the pixels on one side may be filtered based on the pixels on the other side of the virtual boundary only when it is determined that such filtering should be performed based on a default value / setting / configuration or received signaling.

[0104] In an example embodiment of the present disclosure, the filtering operation on one side of the virtual boundary may not use the encoding information on the other side of the virtual boundary, and the filtering operation on the other side of the virtual boundary may not use the encoding information on one side of the virtual boundary. In this case, filtering may not cross the virtual boundary from one side or the other. In other words, there is no option to use the encoding information on the opposite side of the virtual boundary for the filtering operation.

[0105] In an exemplary embodiment of the present disclosure, a filtering operation on one side of a virtual boundary may not use the encoding information on the other side of the virtual boundary, but a filtering operation on the other side of the virtual boundary is allowed to use the encoding information on one side of the virtual boundary. In this case, filtering may not cross the virtual boundary from the one side, but may cross from the other side. In other words, unidirectional use of encoding information only in the first direction is possible.

[0106] In an example embodiment of the present disclosure, filtering operations on one side of a virtual boundary may be allowed to use the encoding information on the other side of the virtual boundary, but filtering operations on the other side of the virtual boundary may not use the encoding information on one side of the virtual boundary. In this case, filtering may cross the virtual boundary from one side, but not from the other side. In other words, unidirectional use of encoding information only in the second direction is possible.

[0107] In an example embodiment of the present disclosure, filtering operations on one side of a virtual boundary may be allowed to use the encoding information on the other side of the virtual boundary, and filtering operations on the other side of the virtual boundary may also be allowed to use the encoding information on one side of the virtual boundary. In this case, filtering may cross the virtual boundary from one side or from the other side. In other words, bidirectional use of encoding information in both the first direction and the second direction is possible.

[0108] In an example embodiment, filtering (in-loop) pixels on one side of a virtual boundary may require coding information on the other side of the virtual boundary. If the coding information on the other side of the virtual boundary is not allowed to be used for pixels on one side of the virtual boundary, an implementation / option may be to disable filtering of pixels on one side of the virtual boundary. Alternatively, an option is to still perform filtering on pixels on one side of the virtual boundary, but replace the coding information on the other side of the virtual boundary with coding information from (filled or derived from) one side of the virtual boundary, or set the coding information to a predetermined value. In other words, filtering can still be performed without using the coding information on the other side of the virtual boundary, but using some other replacement coding information. The replacement coding information may be at least partially different from the coding information on the other side of the virtual boundary.

[0109] In an example embodiment, deblocking filtering may be applied to the (horizontal or vertical) boundary of a block, involving four (horizontal or vertical) pixels on both sides of the block boundary, and other coded information from both sides of the virtual boundary. In an example embodiment, it may be assumed that the block boundary is aligned with the virtual boundary.

[0110] If one side of the virtual boundary is not allowed to use the coding information of the other side of the virtual boundary, deblocking filtering may not be applied within four pixels on one side next to the virtual boundary. Figure 5 As shown in the example of , the refresh area of ​​the GDR / recovery picture may not be allowed to use the coding information of the non-refresh area. Deblocking filtering may be disabled for the four pixels in the refresh area next to the virtual boundary. Alternatively, deblocking filtering may still be applied within the four pixels on one side next to the virtual boundary, but the coding information on the other side is replaced with the coding information from one side or a predetermined value. For example, the four pixels on the other side may be filled from one side (i.e., replaced with at least one of the four pixels on one side). Figure 5 As shown in the example of , deblocking filtering can still be applied to the four pixels in the refresh area next to the virtual boundary, but the four pixels on the non-refresh area are filled with the encoded information from the refresh area.

[0111] Reference now Figure 5, illustrates an example where deblocking filtering may not be applied to a group of pixels 505 of a refresh area 510, or deblocking filtering may still be performed on a group of (filled) pixels 535 of a non-refresh area 530. The refresh area 510 and the non-refresh area 530 may be separated by a virtual boundary 520. It can be noted that although Figure 5 An example is illustrated in which a group of pixels 505 of the refresh area 510 includes four pixels and a group of pixels 535 of the non-refresh area 530 includes four pixels, but this is not limiting; each group of pixels may include more or fewer pixels, and the number of pixels in each group need not be equal.

[0112] In an example embodiment, a sample adaptive offset (SAO) edge filter may be utilized. In VVC, SAO has two parts: band offset, and edge offset. Each coding tree unit (CTU) may choose to use either band offset or edge offset. The choice of band offset or edge offset may be signaled for each CTU. For a CTU, if edge offset is used, a set of parameters (such as Figure 6 , and / or as shown in Figure 7 offsets for the four edge categories shown in ).

[0113] Reference now Figure 6 , illustrates examples of four edge categories. In example 610, pixels a and b are adjacent to pixel c in the horizontal direction. In example 620, pixels a and b are adjacent to pixel c in the vertical direction. In example 630, pixels a and b are adjacent to pixel c along a slope from the upper left corner to the lower right corner. In example 640, pixels a and b are adjacent to pixel c along a slope from the lower left to the upper right.

[0114] Reference now Figure 7 , illustrates an example of four edge categories. In category 1 710, the value of pixel c is lower than the values ​​of pixels a and b. In category 2 720, the values ​​of pixels c and b may be similar, while the value of pixel a may be higher than the values ​​of pixels c and b. Alternatively, the values ​​of pixels a and c may be similar, while the value of pixel b may be higher than the values ​​of pixels a and c. In category 3 730, the values ​​of pixels a and c may be similar, while the value of pixel b may be lower than the values ​​of pixels a and c. Alternatively, the values ​​of pixels c and b may be similar, while the value of pixel a may be lower than the values ​​of pixels c and b. In category 4 740, the value of pixel c may be higher than the values ​​of pixels a and b.

[0115] like Figure 6 and Figure 7 As shown, classifying pixel edges may require the use of neighboring pixels.

[0116] In an example embodiment, if one side of the virtual boundary is not allowed to use the encoding information of the other side of the virtual boundary, the SAO edge offset may be skipped for pixels on one side next to the virtual boundary. Figure 8 As shown in the example of , the refresh area 810 of the GDR / recovery picture may not be allowed to use the coding information of the non-refresh area 830. For pixels 850 in the refresh area 830 that are adjacent to the virtual boundary 820, the SAO edge offset may be disabled.

[0117] Alternatively, the SAO edge offset may still be applied to the pixels on one side (850), but the encoding information on the other side is replaced from one side or set to a predetermined value. For example, the pixels on the other side (860) may be filled from the one side. Figure 8 As shown in the example of FIG. 8 , the SAO edge offset can still be applied to the pixels 850 in the refresh area 810 next to the virtual boundary 820, but the three pixels 860 on the non-refresh area 830 are filled from the refresh area 810 (or set to a predetermined value, such as 2 BD-1 , where BD is the bit depth).

[0118] Reference now Figure 8 , illustrates a refresh region 810 separated from a non-refresh region 830 by a virtual boundary 820. Five pixels 840 in the refresh region 810 may surround a pixel 850 in the refresh region 810. SAO edge offset may not be applied to pixel 850, or may be performed with three pixels 860 in the non-refresh region 830 filled. It should be noted that the number of pixels in each region of 840, 850, and 860 is non-limiting; each group may include pixels corresponding to the pixel 850. Figure 8 Different numbers of pixels are shown in .

[0119] In an example embodiment, an ALF filter may be applied. In VVC, the ALF filter has a 7×7 diamond filter. Fig. 9 An example of an ALF filter on a pixel 950 next to a virtual boundary 920 is shown. If one side of the virtual boundary is not allowed to use the coded information of the other side of the virtual boundary, ALF filtering can be disabled within three pixels on one side next to the virtual boundary. Fig. 9 As shown in the example of , the refresh area 910 of the GDR / recovery picture may not be allowed to use the coding information of the non-refresh area 930. For the pixels 950 in the refresh area 910 adjacent to the virtual boundary 920, ALF may not be performed. Alternatively, ALF may still be applied to the pixels 940 on one side 910 next to the virtual boundary 920, but the coding information of the other side 930 is replaced from the one side 910 or set to a predetermined value. For example, the pixels 960 on the other side 930 may be filled from the one side 910. Fig. 9 As shown in the example of FIG. 1 , ALF can still be performed on pixels 950 in refresh area 910 next to virtual boundary 920, but pixels 960 on non-refresh area 930 are filled from refresh area 910 (or set to a predetermined value, such as 2 BD -1 , where BD is the bit depth).

[0120] Reference now Fig. 9 , illustrates a refresh area 910 separated from a non-refresh area 930 by a virtual border 920. Fifteen pixels 940 may surround one pixel 950 in the refresh area 910 next to the virtual border 920. ALF may not be applied to one pixel 950, or may still be performed if nine pixels 960 of the non-refresh area 930 are filled (e.g., based on pixel 940). It should be noted that the number of pixels in each of 940, 950, and 960 is non-limiting; each group may include the same number of pixels as Fig. 9 Different numbers of pixels are shown in .

[0121] In an example embodiment, a bilateral filter (BIF) for luma may be applied. ECM enhances the features of the in-loop filter of VVC. One of them is a bilateral filter. The bilateral filter may be performed in the sample adaptive offset (SAO) loop filter stage, such as Fig.10 As shown in . Both the bilateral filter (BIF) and SAO can use samples from deblocking as input. Each filter can create offsets for each sample, and these offsets can be added to the input samples and then clipped before continuing with the ALF.

[0122] Reference now Fig.10 , illustrates an example of samples from deblocking 1010 being used by BIF 1020 and SAO 1030, and offsets from BIF 1020 and SAO 1030 being added 1040 to the deblocked output 1050 (which may be clipped 1045).

[0123] A bilateral filter can have a 5×5 diamond filter, such as Fig.11 , where a bilateral filter may or may not be applied to pixels 1150 next to a virtual boundary 1120. Referring now to Fig.11, illustrates an example of a refresh area 1110 separated from a non-refresh area 1130 by a virtual boundary 1120. Eight pixels 1140 surround a pixel 1150 in the refresh area 1110 next to the virtual boundary. BIF may not be applied to pixel 1150, or may still be performed with four pixels 1160 in the non-refresh area 1130 filled or set to a predetermined value. It should be noted that the number of pixels in each of 1140, 1150, and 1160 is non-limiting; each group may include pixels 1150 and 1160 that are not limited; each group may include pixels 1150 and 1160 that are not limited. Fig.11 Different numbers of pixels are shown in .

[0124] In an example embodiment, if one side 1110 of the virtual boundary 1120 is not allowed to use the encoding information of the other side 1130 of the virtual boundary 1120, bilateral filtering may be disabled within two pixels of the one side 1110 next to the virtual boundary 1120. Fig.11 As shown in the example of , the refresh area 1110 of the GDR / recovery picture may not be allowed to use the coded information of the non-refresh area 1130. For the pixels 1150 in the refresh area 1110 adjacent to the virtual boundary 1120, bilateral filtering may not be performed. Alternatively, bilateral filtering may still be performed within the two pixels on one side 1110 next to the virtual boundary 1120, but the coded information on the other side 1130 is replaced from one side 1110 or set to a predetermined value. For example, the pixels 1160 on the other side 1130 may be filled from one side 1110. Fig.11 As shown in the example of FIG. 1 , bilateral filtering can still be applied to pixels 1150 in refresh area 1110 next to virtual boundary 1120, but pixels 1160 on non-refresh area 1130 are filled from refresh area 1110 (or set to a predetermined value, such as 2 BD-1 , where BD is the bit depth).

[0125] In an example embodiment, a bilateral filter for chrominance (BIF chrominance) may be applied. Similar to BIF luma, BIF chrominance may be performed in parallel with SAO and CCSAO processes, such as Fig.12 . BIF chroma (1220), SAO (1230), and CCSAO (1240) can use the same chroma samples produced by the deblocking filter (1210) as input and generate three offsets (per chroma sample) in parallel. These three offsets can then be added to the input chroma samples (1250) to obtain a sum, which can then be clipped to form the final output chroma sample value (1260). BIF chroma can provide an on / off control mechanism at the CTU level and the slice level.

[0126] The filtering process for BIF chroma can be similar to the filtering process for BIF luma. For chroma samples, a 5×5 diamond filter can be used to generate the filter offset, such as Fig.13 As shown, the BIF chrominance may or may not be applied to pixels 1350 next to the virtual boundary 1320. Referring now to Fig.13 , illustrates an example of a refresh area 1310 separated from a non-refresh area 1330 by a virtual boundary 1320. Eight pixels 1340 surround a pixel 1350 in the refresh area 1310 next to the virtual boundary 1320. BIF chrominance may not be applied to pixel 1350, or may still be performed with four pixels 1360 in the non-refresh area 1130 filled or set to a predetermined value. It should be noted that the number of pixels in each of 1340, 1350, and 1360 is non-limiting; each group may include pixels 1350 and 1360 that are not limited; each group may include pixels 1350 and 1360 that are not limited. Fig.13 Different numbers of pixels are shown.

[0127] In an example embodiment, if one side 1310 of the virtual boundary 1320 is not allowed to use the encoding information of the other side 1330 of the virtual boundary 1320, the BIF chroma may be disabled within two pixels of the one side 1310 next to the virtual boundary 1320. Fig.13 As shown in the example of , the refresh area 1310 of the GDR / recovery picture may not be allowed to use the coded information of the non-refresh area 1330. For the pixels 1350 in the refresh area 1310 adjacent to the virtual boundary 1320, the BIF chroma may be skipped. Alternatively, the BIF chroma may still be applied to the two pixels on one side 1310 next to the virtual boundary 1320, but the coded information on the other side 1330 may be replaced from one side 1310 or set to a predetermined value. For example, the pixels 1360 on the other side 1330 may be filled from one side 1310. Fig.13 As shown in the example of FIG. 1 , the BIF chromaticity can still be applied to the pixels 1350 in the refresh area 1310 next to the virtual boundary 1320, but the pixels 1360 on the non-refresh area 1330 are filled from the refresh area 1310 (or set to a predetermined value, such as 2 BD-1 , where BD is the bit depth).

[0128] In an example embodiment, CCSAO may be applied. Cross-component sample adaptive offset (CCSAO) may be used to refine the reconstructed chroma samples. Similar to SAO, CCSAO classifies the reconstructed samples into different categories, derives an offset for each category, and adds the offset to the reconstructed samples in the category. However, unlike SAO, which uses only a single luma / chroma component of the current sample as input, CCSAO utilizes all three components to classify the current sample into different categories. To facilitate parallel processing, the output samples from the deblocking filter may be used as input to CCSAO, such as Fig.12 as shown in .

[0129] In CCSAO, a band offset (BO) classifier or an edge offset (EO) classifier can be used to improve the quality of the reconstructed samples. CCSAO can be applied to luma and chroma components. For a given luma / chroma sample, three candidate samples can be selected to classify the given sample into different categories: a collocated Y sample (1410), a collocated U sample (1420), and a collocated V sample (1430). It can be noted that the collocated Y sample 1410 can be any one of the 9 Y components shown at 1405; Fig.14 The example of is not limiting. The sample values ​​of the three selected samples may then be classified into three different frequency bands, and the joint index may represent the class of a given sample. An offset may be signaled and added to the reconstructed samples belonging to that class.

[0130] For collocated chroma samples, collocated luma samples may be selected from nine candidate positions (1405), such as Fig.14 . Nine candidate positions are illustrated at 1405. The collocated and adjacent sample Y (1410) may be located at position 4. The collocated sample U (1420) may be at least partially collocated at position 4 (i.e., the position of the collocated Y sample). The collocated sample V (1430) may be at least partially collocated at position 4 (i.e., the position of the collocated Y sample).

[0131] In an example embodiment, if one side 1510 of the virtual boundary 1520 is not allowed to use the encoding information of the other side 1530 of the virtual boundary 1520, CCSAO may be disabled within the pixels 1560 of the one side 1510 next to the virtual boundary 1520. Fig.15As shown in the example of , the refresh area 1510 of the GDR / recovery picture may not be allowed to use the coding information of the non-refresh area 1530. For the pixels 1550 in the refresh area 1510 adjacent to the virtual boundary 1520, CCSAO may be skipped. Alternatively, CCSAO may still be applied to the pixels 1550 on one side 1510 next to the virtual boundary 1520, but the coding information on the other side 1530 may be replaced from one side 1510 or set to a predetermined value. For example, the pixels 1560 on the other side 1530 may be filled from one side 1510. Fig.15 As shown in the example of FIG. 1 , CCSAO can still be applied to pixels 1550 in the refresh area 1510 next to the virtual boundary 1520, but pixels 1560 on the non-refresh area 1530 are filled from the refresh area 1510 (or set to a predetermined value, such as 2 BD-1 , where BD is the bit depth).

[0132] Reference now Fig.15 , illustrates an example in which a refresh area 1510 is separated from a non-refresh area 1530 by a virtual boundary 1520. Five pixels 1540 surround one pixel 1550 in the refresh area 1510 next to the virtual boundary 1520. CCSAO may not be applied to the pixel 1550 in the refresh area 1510, or may still be performed with three pixels 1560 in the non-refresh area 1530 filled or set to a predetermined value. It should be noted that the number of pixels in each of 1540, 1550, and 1560 is non-limiting; each group may include pixels corresponding to Fig.15 Different numbers of pixels shown in .

[0133] like Fig.12 As shown in , the SAO, bilateral filter (BIF), and CCSAO offset can be calculated in parallel, added to the reconstructed chroma samples, and clipped jointly.

[0134] In an example embodiment, a longer CCAF may be applied. Fig.16 As shown in , the CCLF process uses a linear filter to filter the luma sample values ​​and generates residual corrections for the chroma samples. Initially, an 8-tap filter was designed for the CCLF process. More recently, a large 25-tap filter can be used in the CCLF process; this is Fig.17 For a given slice, the encoder can collect statistics for the slice, analyze them, and signal up to 16 filters through APS.

[0135] Reference now Fig.16, illustrates a basic example of CCLF. At CTU (Y) 1610, CCLF (Cb) may be applied 1620 to a set of pixels, as shown at 1630. This may be considered as a linear filtering of luma sample values. At CTU (Cb) 1640, ALF chroma may be applied 1650 to a portion of the pixels. This may be considered as filtering of chroma samples. The outputs of 1620 and 1650 may be added 1660 and output as CTB' (Cb) 1670.

[0136] In an example embodiment, if one side 1810 of the virtual boundary 1820 is not allowed to use the encoding information of the other side 1830 of the virtual boundary 1820, a longer CCLF may be disabled within four pixels of one side 1810 next to the virtual boundary 1820. Fig.18 As shown in the example of , the refresh area 1810 of the GDR / recovery picture may not be allowed to use the encoding information of the non-refresh area 1830. For the collocated chroma pixels 1850 in the refresh area 1810 adjacent to the virtual boundary 1820, the longer CCLF may be skipped. Alternatively, the longer CCLF may still be applied to the four pixels on one side 1810 next to the virtual boundary 1820, but the encoding information on the other side 1830 may be replaced from one side 1810 or set to a predetermined value. For example, the pixels 1860 on the other side 1830 may be filled from one side 1810. Fig.18 As shown in the example of FIG. 1 , the longer CCLF may still be applied to the collocated chroma pixels 1850 in the refresh region 1810 next to the virtual boundary 1820, but the pixels 1860 on the non-refresh region 1830 are filled from the refresh region 1810 (or set to a predetermined value, such as 2 BD-1 , where BD is the bit depth).

[0137] Reference now Fig.18 , illustrates an example in which a refresh area 1810 is separated from a non-refresh area 1830 by a virtual boundary 1820. Sixteen pixels 1840 surround a collocated chroma pixel 1850 in the refresh area 1810 next to the virtual boundary 1820. A longer CCLF may not be applied to the collocated chroma pixel 1850, or may still be performed with eight pixels 1860 in the non-refresh area 1830 being filled or set to a predetermined value.

[0138] In an example embodiment, an alternative band classifier for the ALF filter (ABN-ALF) may be applied. The ECM may classify each 2x2 luma block of the ALF using a 13x13 diamond filter.

[0139] In an example embodiment, if one side 1910 of the virtual boundary 1920 is not allowed to use the encoding information of the other side 1930 of the virtual boundary 1920, ABN-ALF may be disabled within six pixels of the one side 1910 next to the virtual boundary 1920. Fig.19 As shown in the example of , the refresh area 1910 of the GDR / recovery picture may not be allowed to use the coding information of the non-refresh area 1930. For the pixel 1950 in the refresh area 1910 adjacent to the virtual boundary 1920, the ABN-ALF may be skipped. Alternatively, the ABN-ALF may still be applied to the six pixels on one side 1910 next to the virtual boundary 1920, but the coding information on the other side 1930 may be replaced from one side 1910 or set to a predetermined value. For example, the pixel 1960 on the other side 1930 may be filled from one side 1910. Fig.19 As shown in the example of FIG. 1 , ABN-ALF can still be applied to pixels 1950 in refresh area 1910 next to virtual boundary 1920, but pixels 1960 on non-refresh area 1930 are filled from refresh area 1910 (or set to a predetermined value, such as 2 BD-1 , where BD is the bit depth).

[0140] Reference now Fig.19 , illustrates an example in which a refresh area 1910 is separated from a non-refresh area 1930 by a virtual boundary 1920. In the refresh area 1910 next to the virtual boundary 1920, 48 pixels 1940 surround one pixel 1950. ANN-ALF may not be applied to pixel 1950, or may still be performed with 36 pixels 1960 in the non-refresh area 1930. It should be noted that the number of pixels in each of 1940, 1950, and 1960 is non-limiting; each group may include the same number of pixels as 1940, 1950, and 1960. Fig.19 Different numbers of pixels are shown in .

[0141] Fig. 20Potential steps of an example method 2000 are illustrated. The example method 2000 may include: selecting at least one first pixel from a first region of a picture, wherein the first region is separated from a different second region by a virtual boundary (2010); determining at least one second pixel for filtering of the at least one first pixel, wherein the at least one second pixel is located in the second region (2020); determining whether to filter the at least one first pixel (2030); based on the determination that the at least one first pixel is to be filtered, filtering the at least one first pixel based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is padded based on the encoding information of the first region (2040). It may be noted that filtering the at least one first pixel is optional; not filtering the at least one first pixel is also an option. Determining whether to filter the at least one first pixel may be based on a default setting / configuration, signaling in a sequence header, signaling in a picture header, and / or signaling in a slice header.

[0142] According to an example embodiment, a device may include: at least one processor; and at least one memory including computer program code; the at least one memory and the computer program code are configured to, together with the at least one processor, enable the device to at least: select at least one first pixel from a first region of a picture, wherein the first region may be separated from a different second region by a virtual boundary; determine at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel may be located in the second region; determine whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filter the at least one first pixel based on at least one of the following: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is filled based on the encoding information of the first region.

[0143] At least one first pixel may be adjacent to the virtual boundary.

[0144] The filtering may include performing at least one in-loop filter, wherein the at least one in-loop filter includes at least one of: a deblocking filter, a sample adaptive offset filter, or an adaptive loop filter.

[0145] The picture may include one of the following: a 360-degree picture, a progressive decoding refresh picture, a recovery picture, or a picture including a sub-picture or a tile.

[0146] The first area may include a refresh area of ​​the picture, wherein the second area may include a non-refresh area of ​​the picture.

[0147] The encoding information associated with the at least one second pixel includes at least one of: pixel information, an encoding mode, a motion vector, or a quarter-pel associated with the at least one second pixel.

[0148] The example apparatus may be further configured to determine whether to filter the at least one first pixel based on at least one of: configuration, signaling in a sequence header, signaling in a picture header, or signaling in a slice header.

[0149] The at least one predetermined value may include a power of two, the power of two including the bit depth of the picture minus one.

[0150] The filtering of the at least one first pixel may include performing deblocking filtering, wherein a block boundary for the deblocking filtering may be aligned with the virtual boundary.

[0151] The filtering of the at least one first pixel may include performing sample adaptive offset edge filtering.

[0152] The filtering of the at least one first pixel may include performing adaptive loop filtering, wherein the adaptive loop filtering may include use of a seven by seven diamond filter.

[0153] The filtering of the at least one first pixel may include performing bilateral filtering for luminance, wherein the bilateral filtering for luminance may include use of a five by five diamond filter.

[0154] The filtering of the at least one first pixel may include performing bilateral filtering for chroma, wherein the bilateral filtering for chroma may include use of a five by five diamond filter.

[0155] The filtering of the at least one first pixel may include performing cross-component sample adaptive offset filtering based on at least one of: a band offset classifier, or an edge offset classifier.

[0156] The filtering of the at least one first pixel may include performing longer cross-component adaptive loop filtering, wherein the longer cross-component adaptive loop filtering may include use of a twenty-five tap large filter.

[0157] The filtering of the at least one first pixel may include performing an alternative band classifier for adaptive loop filtering, wherein the alternative band classifier for adaptive loop filtering may include classifying one or more two by two luma blocks using corresponding thirteen by thirteen diamond filters.

[0158] The example apparatus may also be configured to, based on a determination that filtering of the at least one first pixel is not required, cause the at least one first pixel not to be filtered.

[0159] According to one aspect, an example method may be provided, the example method comprising: selecting at least one first pixel from a first region of a picture, wherein the first region may be separated from a different second region by a virtual boundary; determining at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel may be located in the second region; determining whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filtering the at least one first pixel based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is filled based on the encoding information of the first region.

[0160] According to an example embodiment, a device may include a circuit system configured to perform the following operations: select at least one first pixel from a first region of a picture, wherein the first region may be separated from a different second region by a virtual boundary; determine at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel may be located in the second region; determine whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filter the at least one first pixel based on at least one of the following: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is filled based on the encoding information of the first region.

[0161] According to an example embodiment, a device may include: a processing circuit system; a memory circuit system including a computer program code, the memory circuit system and the computer program code being configured to, together with the processing circuit system, enable the device to: select at least one first pixel from a first region of a picture, wherein the first region may be separated from a different second region by a virtual boundary; determine at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel may be located in the second region; determine whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filter the at least one first pixel based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is filled based on the encoding information of the first region.

[0162] As used in this application, the term "circuitry" may refer to one or more or all of the following: (a) a pure hardware circuit implementation (such as an implementation in analog and / or digital circuitry only), and (b) a combination of hardware circuitry and software, such as (as applicable): (i) a combination of analog and / or digital hardware circuitry and software / firmware, and (ii) any portion of a hardware processor(s) with software, including a digital signal processor(s), software, and memory(s) that work together to enable a device such as a mobile phone or server to perform various functions, and (c) hardware circuitry and / or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s) that requires software (e.g., firmware) to operate, but the software may not be present when not required for operation. This definition of circuitry applies to all uses of the term in this application, including in any claims. As another example, as used in this application, the term circuitry also covers an implementation of only a hardware circuit or processor(s) or a portion of a hardware circuit or processor and its accompanying software and / or firmware. For example, the term circuitry would also include, if applicable to a particular claim element, a baseband integrated circuit or a processor integrated circuit for a mobile device, or a similar integrated circuit in a server, cellular network device, or other computing or networking device.

[0163] According to an example embodiment, a device may include components for performing the following operations: selecting at least one first pixel from a first region of a picture, wherein the first region may be separated from a different second region by a virtual boundary; determining at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel may be located in the second region; determining whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filtering the at least one first pixel based on at least one of the following: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is filled based on the encoding information of the first region.

[0164] According to an example embodiment, a non-transitory computer-readable medium includes program instructions stored thereon, which, when executed by at least one processor, cause the at least one processor to: select at least one first pixel from a first region of a picture, wherein the first region may be separated from a different second region by a virtual boundary; determine at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel may be located in the second region; determine whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filter the at least one first pixel based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is filled based on the encoding information of the first region.

[0165] According to another example embodiment, a non-volatile program storage device readable by a machine may be provided, the device tangibly embodying an instruction program executable by the machine for performing operations, the operations including: selecting at least one first pixel from a first region of a picture, wherein the first region may be separated from a different second region by a virtual boundary; determining at least one second pixel for filtering the at least one first pixel, wherein the at least one second pixel may be located in the second region; determining whether to filter the at least one first pixel; and based on the determination that the at least one first pixel is to be filtered, filtering the at least one first pixel based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or a version of the encoding information associated with the at least one second pixel that is filled based on the encoding information of the first region.

[0166] It should be understood that the above description is illustrative only. Those skilled in the art may design various alternatives and modifications. For example, the features described in the various dependent claims may be combined with each other in any suitable combination. In addition, the features from the above-mentioned different embodiments may be selectively combined into new embodiments. Therefore, this specification is intended to include all such alternatives, modifications, and variations that fall within the scope of the appended claims.

Claims

1. A device comprising: at least one processor; as well as at least one non-transitory memory including computer program code; The at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least: selecting at least one first pixel from a first region of the picture, wherein the first region is separated from a different second region by a virtual boundary; determining at least one second pixel for filtering of the at least one first pixel, wherein the at least one second pixel is located in the second area; determining whether to filter the at least one first pixel; as well as Based on the determination that the at least one first pixel is to be filtered, the at least one first pixel is filtered based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or A version of the encoding information associated with the at least one second pixel that is padded based on the encoding information of the first region. 2 . The apparatus of claim 1 , wherein the at least one first pixel is adjacent to the virtual boundary.

3. The apparatus of claim 1 or 2, wherein the filtering comprises performing at least one in-loop filter, wherein the at least one in-loop filter comprises at least one of: Deblocking filter, Sample Adaptive Offset Filter, or Adaptive loop filter.

4. The apparatus according to any one of claims 1 to 3, wherein the picture comprises one of the following: 360-degree pictures, Progressive decoding refreshes the picture, Recovering pictures, or An image that includes sub-images or tiles. 5 . The apparatus according to claim 1 , wherein the first area comprises a refresh area of ​​the picture, and wherein the second area comprises a non-refresh area of ​​the picture.

6. The apparatus according to any one of claims 1 to 5, wherein the encoding information associated with the at least one second pixel comprises at least one of the following: Pixel information, Encoding mode, motion vector, or A quarter pixel is associated with the at least one second pixel.

7. The apparatus according to any one of claims 1 to 6, wherein the at least one memory and the computer program code are further configured to, together with the at least one processor, cause the apparatus to determine whether to filter the at least one first pixel based on at least one of the following: Configuration, Signaling in the sequence header, signalling in the picture header, or Signaling in the slice header.

8. The apparatus of any one of claims 1 to 7, wherein the at least one predetermined value comprises a power of two, the power of two comprising the bit depth of the picture minus one.

9. The apparatus of any one of claims 1 to 8, wherein the filtering of the at least one first pixel comprises performing deblocking filtering, wherein a block boundary for the deblocking filtering is aligned with the virtual boundary.

10. The apparatus according to any one of claims 1 to 8, wherein the filtering of the at least one first pixel comprises: Perform sample adaptive offset edge filtering.

11. The apparatus according to any one of claims 1 to 8, wherein the filtering of the at least one first pixel comprises: Adaptive loop filtering is performed, wherein the adaptive loop filtering includes the use of a seven by seven diamond filter.

12. The apparatus according to any one of claims 1 to 8, wherein the filtering of the at least one first pixel comprises: Bilateral filtering for luminance is performed, wherein the bilateral filtering for luminance includes the use of a five by five diamond filter.

13. The apparatus according to any one of claims 1 to 8, wherein the filtering of the at least one first pixel comprises: A 2-bilateral filtering for chroma is performed, wherein the bilateral filtering for chroma comprises the use of a five by five diamond filter.

14. The apparatus according to any one of claims 1 to 8, wherein the filtering of the at least one first pixel comprises: Cross-component sample adaptive offset filtering is performed based on at least one of: a band offset classifier, or an edge offset classifier.

15. The apparatus according to any one of claims 1 to 8, wherein the filtering of the at least one first pixel comprises: Longer cross-component adaptive loop filtering is performed, wherein the longer cross-component adaptive loop filtering includes the use of a twenty-five tap large filter.

16. The apparatus according to any one of claims 1 to 8, wherein the filtering of the at least one first pixel comprises: An alternative frequency band classifier for adaptive loop filtering is performed, wherein the alternative frequency band classifier for adaptive loop filtering includes classifying one or more two by two luma blocks using corresponding thirteen by thirteen diamond filters.

17. An apparatus according to any one of claims 1 to 16, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the apparatus to: based on a determination that filtering of the at least one first pixel is not required, not filter the at least one first pixel.

18. A method comprising: selecting at least one first pixel from a first region of the picture, wherein the first region is separated from a different second region by a virtual boundary; determining at least one second pixel for filtering of the at least one first pixel, wherein the at least one second pixel is located in the second area; determining whether to filter the at least one first pixel; as well as Based on the determination that the at least one first pixel is to be filtered, filtering the at least one first pixel based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or A version of the encoding information associated with the at least one second pixel that is padded based on the encoding information of the first region.

19. An apparatus comprising means for performing the following operations: selecting at least one first pixel from a first region of the picture, wherein the first region is separated from a different second region by a virtual boundary; determining at least one second pixel for filtering of the at least one first pixel, wherein the at least one second pixel is located in the second area; determining whether to filter the at least one first pixel; as well as Based on the determination that the at least one first pixel is to be filtered, filtering the at least one first pixel based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or A version of the encoding information associated with the at least one second pixel that is padded based on the encoding information of the first region.

20. A non-transitory computer readable medium comprising program instructions stored thereon, the program instructions, when executed by at least one processor, causing the at least one processor to: selecting at least one first pixel from a first region of the picture, wherein the first region is separated from a different second region by a virtual boundary; determining at least one second pixel for filtering of the at least one first pixel, wherein the at least one second pixel is located in the second area; determining whether to filter the at least one first pixel; as well as Based on the determination that the at least one first pixel is to be filtered, the at least one first pixel is filtered based on at least one of: encoding information associated with the at least one second pixel; at least one predetermined value; or A version of the encoding information associated with the at least one second pixel that is padded based on the encoding information of the first region.

Images