Joint bilateral filter for media coding

Abstract

Various embodiments provide methods, apparatuses, and computer program products. An example apparatus includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: applying a joint bilateral filter within a video codec coding loop on a chroma component of a pixel, wherein a range difference is derived from a luma component of the pixel as a guide image.

Description

JOINT BILATERAL FILTER FOR MEDIA CODINGTECHNICAL FIELD

[0001] The examples and non-limiting embodiments relate generally to video coding and, more particularly to, using joint bilateral filter for media coding.BACKGROUND

[0002] It is known to provide standardized formats for encoding, signaling, or decoding of media data.SUMMARY

[0003] Example 1: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: applying a joint bilateral filter within a video codec coding loop on a chroma component of a pixel, wherein a range difference is derived from a luma component of the pixel as a guide image.

[0004] Example 2: The apparatus of example 1, wherein a spatial difference information of the joint bilateral filter is determined from respective chroma components.

[0005] Example 3: The apparatus of any of examples 1 or 2, wherein a filtered result for the pixel in the chroma component is determined based on at least the following: an original chroma component (U); coordinates of the pixel (x); a filter window (fl) centered at (x); a spatial kernel (fs) for differences in coordinates; a range kernel (fr) for differences in intensities; and the corresponding luma component (Y) to the original chroma component (U).

[0006] Example 4: The apparatus of any of the previous examples, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, one or more parameters to activate the joint bilateral filter per sequence, image, tile, slice, or block.

[0007] Example 5: The apparatus of any of the previous examples, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along the bitstream or receiving from or along the bitstream a size, distribution, and weight assigned to a spatial and a range filter kernel per sequence, image, tile, slice, or block.

[0008] Example 6: The apparatus of any of the previous examples, wherein at least one of elements of the joint bilateral filter for at least one of filter kernels is fixed, and wherein the instructions, whenexecuted by the at least one processor, cause the apparatus at least to perform: signaling, in or along the bitstream or receiving from or along the bitstream, remaining elements , per sequence, image, tile, slice, or block.

[0009] Example 7: The apparatus of any of the previous examples, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving the at least one of elements of the joint bilateral filter for at least one of the filter kernel from a selected prediction mode of a block.

[0010] Example 8: The apparatus of any of the examples 1 to 6, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving the at least one of the elements of the joint bilateral filter for the at least one of the filter kernels from the chroma sampling format of a sequence.

[0011] Example 9: The apparatus of any of the examples 6 to 8, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along the bitstream or receiving from or along the bitstream, the at least one of elements of the joint bilateral filter by using an adaption parameter set.

[0012] Example 10: The apparatus of any of the previous examples, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a prediction mode for the block of samples; and determining a representation for the block to be predicted based on a prediction mode.

[0013] Example 11: The apparatus of example 10, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and the chroma components of the block.

[0014] Example 12: The apparatus of any of the examples 1 to 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on a prediction mode; and blending at least the first and the second initial representations to form a combined prediction block.

[0015] Example 13: The apparatus of example 12, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of theblock; and the quantization parameter difference between the luma and the chroma components of the block.

[0016] Example 14: The apparatus of example 13, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

[0017] Example 15: The apparatus of any of the examples 11, 13, or 14, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along the bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0018] Example 16: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining a representation for the block to be predicted based on the prediction mode; and applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

[0019] Example 17: The apparatus of example 16, wherein instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0020] Example 18: The apparatus of the example 17, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0021] Example 19: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on the prediction mode; blending at least the first and the second initial representations to form a combined prediction block; and applying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image.

[0022] Example 20: The apparatus of example 19, wherein the instructions, when executed by theat least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter is on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0023] Example 21: The apparatus of example 20, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

[0024] Example 22: The apparatus of any of the examples 20 or 21, wherein the instructions, when execute by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0025] Example 23: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a set of prediction parameters for the block of samples; determining a representation for the block to be predicted based at least on the set of prediction parameters; and applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

[0026] Example 24: The apparatus of example 23, wherein the set of prediction parameters comprises one or more of an intra prediction mode, an inter prediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

[0027] Example 25: The apparatus of any of the examples 23 or 24, wherein the representation comprises a sample value for each sample.

[0028] Example 26: The apparatus of any of the examples 23 to 25, wherein instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0029] Example 27: The apparatus of the example 26, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0030] Example 28: An apparatus comprising: at least one processor; and at least one memorystoring instructions that, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a first set of prediction parameters and a second set of prediction parameters for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on the first set of prediction parameters and the second set of prediction parameters; blending at least the first and the second initial representations to form a combined prediction block; and applying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image.

[0031] Example 29: The apparatus of example 28, wherein the first set of prediction parameters and the second set of prediction parameters comprise one or more of an intra prediction mode, an inter prediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

[0032] Example 30: The apparatus of any of the examples 28 or 29, wherein the representation comprises a sample value for each sample.

[0033] Example 31: The apparatus of any of the examples 28 to 30, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0034] Example 32: The apparatus of example 31, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

[0035] Example 33: The apparatus of any of the examples 31 or 32, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0036] Example 34: A method comprising: applying a joint bilateral filter within a video codec coding loop on a chroma component of a pixel, wherein a range difference is derived from a luma component of the pixel as a guide image.

[0037] Example 35: The method of example 34, wherein a spatial difference information of the joint bilateral filter is determined from respective chroma components.

[0038] Example 36: The method of any of examples 34 or 35, wherein a filtered result for the pixel in the chroma component is determined based on at least the following: an original chroma component(U); coordinates of the pixel (x); a filter window (fl) centered at (x); a spatial kernel (fs) for differences in coordinates; a range kernel (fr) for differences in intensities; and the corresponding luma component (Y) to the original chroma component (U).

[0039] Example 37: The method of any of the examples 34 to 36, further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, one or more parameters to activate the joint bilateral filter per sequence, image, tile, slice, or block.

[0040] Example 38: The method of any of the examples 34 to 37 further comprising: signaling, in or along the bitstream or receiving from or along the bitstream a size, distribution, and weight assigned to a spatial and a range filter kernel per sequence, image, tile, slice, or block.

[0041] Example 39: The method of any of the examples 34 to 38, wherein at least one of elements of the joint bilateral filter for at least one of filter kernels is fixed, and wherein the method further comprises: signaling, in or along the bitstream or receiving from or along the bitstream, remaining elements , per sequence, image, tile, slice, or block.

[0042] Example 40: The method of any of the examples 34 to 39 further comprising: deriving the at least one of elements of the joint bilateral filter for at least one of the filter kernel from a selected prediction mode of a block.

[0043] Example 41: The method of any of the examples 34 to 40 further comprising: deriving the at least one of the elements of the joint bilateral filter for the at least one of the filter kernels from the chroma sampling format of a sequence.

[0044] Example 42: The method of any of the examples 39 to 41 further comprising: signaling, in or along the bitstream or receiving from or along the bitstream, the at least one of elements of the joint bilateral filter by using an adaption parameter set.

[0045] Example 43: The method of any of the examples 34 to 42 further comprising: determining a block of samples to be predicted; determining a prediction mode for the block of samples; and determining a representation for the block to be predicted based on a prediction mode.

[0046] Example 44: The method of example 43 further comprising: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and the chroma components of the block.

[0047] Example 45: The method of any of the examples 34 to 42 further comprising: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determiningat least a first and a second initial representations for the block to be predicted based on a prediction mode; and blending at least the first and the second initial representations to form a combined prediction block.

[0048] Example 46: The method of example 45 further comprising: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and the chroma components of the block.

[0049] Example 47: The method of example 46 further comprising: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

[0050] Example 48: The method of any of the examples 44, 46, or 47 further comprising: signaling, in or along the bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0051] Example 49: A method comprising: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining a representation for the block to be predicted based on the prediction mode; and applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

[0052] Example 50: The method of example 49 further comprising: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0053] Example 51: The method of the example 50 further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0054] Example 52: A method comprising: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on the prediction mode; blending at least the first and the second initial representations to form a combined prediction block; and applying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image.

[0055] Example 53: The method of example 52 further comprising: deriving at least one parameter of the joint bilateral filter is on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of theblock.

[0056] Example 54: The method of example 53 further comprising: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

[0057] Example 55: The method of any of the examples 53 or 54 further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0058] Example 56: A method comprising: determining a block of samples to be predicted; determining a set of prediction parameters for the block of samples; determining a representation for the block to be predicted based at least on the set of prediction parameters; and applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

[0059] Example 57: The method of example 56, wherein the set of prediction parameters comprises one or more of an intra prediction mode, an inter prediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

[0060] Example 58: The method of any of the examples 56 or 57, wherein the representation comprises a sample value for each sample.

[0061] Example 59: The method of any of the examples 55 to 58 further comprising: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0062] Example 60: The method of the example 59 further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0063] Example 61: A method comprising: determining a block of samples to be predicted; determining a first set of prediction parameters and a second set of prediction parameters for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on the first set of prediction parameters and the second set of prediction parameters; blending at least the first and the second initial representations to form a combined prediction block; and applying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image.

[0064] Example 62: The method of example 61, wherein the first set of prediction parameters and the second set of prediction parameters comprise one or more of an intra prediction mode, an interprediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

[0065] Example 63: The method of any of the examples 61 or 62, wherein the representation comprises a sample value for each sample.

[0066] Example 64: The method of any of the examples 61 to 63 further comprising: deriving least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0067] Example 65: The method of example 64 further comprising: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

[0068] Example 66: The method of any of the examples 64 or 65 further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0069] Example 67: An apparatus comprising means for performing methods as described in any of the examples 33 to 66.

[0070] Example 68: A computer readable medium comprising program instructions for performing methods as described in any of the examples 33 to 66.

[0071] Example 69: The computer readable medium of example 68, wherein the computer readable medium comprises a non-transitory computer readable medium.BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The foregoing embodiments and other features are explained in the following description, taken in connection with the accompanying drawings, wherein:

[0073] FIG. 1 shows schematically an apparatus employing embodiments of the examples described herein.

[0074] FIG. 2 shows schematically a user equipment suitable for employing embodiments of the examples described herein.

[0075] FIG. 3 further shows schematically electronic devices employing embodiments of the examples described herein connected using wireless and wired network connections.

[0076] FIG. 4 is a block diagram illustrating a system in accordance with an example.

[0077] FIG. 5 is block diagram depicting in-loop filter in versatile video coding.

[0078] FIG. 6 illustrates example filter kernel, in accordance with an embodiment.

[0079] FIG. 7 is an example apparatus, which may be implemented in hardware, and is caused to, implement examples described herein.

[0080] FIG. 8 shows a representation of an example of non-volatile memory media used to store instructions that implement the examples described herein.

[0081] FIG. 9 is an example method performed with an encoder or a decoder, based on the examples described herein.

[0082] FIG. 10 is another example method performed with an encoder or a decoder, based on the examples described herein.

[0083] FIG. 11 is yet another example method performed with an encoder or a decoder, based on the examples described herein.

[0084] FIG. 12 is still another example method performed with an encoder or a decoder, based on the examples described herein.

[0085] FIG. 13 is still another example method performed with an encoder or a decoder, based on the examples described herein.DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0086] The following acronyms and abbreviations that may be found in the specification and / or the drawing figures are defined as follows (the abbreviations may be appended with each other or with other characters using e.g. a hyphen or dash (-), and may be case insensitive):4CC four character code5G fifth generation cellular network technology5GC 5G core network a.k.a. also known asAVC advanced video codingBCW bi-prediction with CU-level weightCABAC context adaptive binary arithmetic coderCCCM: convolutional cross-component modelCIIP combined inter and intra predictionCCRM cross-component residual model / cross-component reconstruction modelCU coding unitDCT discrete cosine transformDSP digital signal processorDST discrete sine transformDU distributed unitECM enhanced compression model eNB (or eNodeB) evolved Node B (for example, an LTE base station)EN-DC E-UTRA-NR dual connectivity en-gNB or En-gNB node providing NR user plane and control plane protocol terminations towards the UE, and acting as secondary node in EN-DCE-UTRA evolved universal terrestrial radio access, for example, the LTE radio access technologyFl or Fl-C interface between CU and DU control interface gNB (or gNodeB) base station for 5G / NR, for example, a node providing NR user plane and control plane protocol terminations towards the UE, and connected via the NG interface to the 5GCHEVC high efficiency video codingIEC International Electrotechnical Commission loT internet of thingsISO International Organization for StandardizationISOBMFF ISO base media file formatJPEG joint photographic experts groupLIC local illumination compensationLTE long-term evolution mdat MediaDataBoxMIME Multipurpose Internet Mail ExtensionMME mobility management entity moov MovieBoxMP4 file format for MPEG-4 Part 14 filesMPEG moving picture experts groupMPEG-2 H.222 / H.262 as defined by the ITUMPEG-4 audio and video coding standard for ISO / IEC 14496 ng or NG new generation ng-eNB or NG-eNB new generation eNBNR new radio (5G radio)N / W or NW networkPDCP packet data convergence protocolPHY physical layerPNG portable network graphicsRAN radio access networkRFC request for commentsRGB red, green, blueREC radio link controlRRC radio resource controlRRH remote radio headRU radio unitRx receiverSDAP service data adaptation protocolSGW serving gatewaySMF session management functionSPS sequence parameter setSVC scalable video codingSI interface between eNodeBs and the EPC trak TrackBoxTx transmitterUE user equipmentUICC Universal Integrated Circuit CardUPF user plane functionURL uniform resource locator vvc Versatile Video CodingWP weighted predictionX2 interconnecting interface between two eNodeBs in LTE networkXn interface between two NG-RAN nodesYUV / YCbCr: a color model based on one luminance and two chrominance / color difference channels (typically used in many video coding applications)

[0087] Some embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all, embodiments may be shown. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. As used herein, the terms ‘data,’ ‘content,’ ‘information,’ and similar terms may be used interchangeably to refer to data capable of being transmitted, received and / or stored in accordance with embodiments of the present invention. Thus, use of any such terms should not be taken to limit the spirit and scope of embodiments.

[0088] Described herein is a method and apparatus for using a joint bilateral filter for media coding, for example, a joint bilateral filter within a video codec coding loop.

[0089] The following describes in detail a suitable apparatus and possible method using jointbilateral filter (e.g., a joint bilateral filter within a video codec coding loop) for media coding according to embodiments. In this regard reference is first made to FIG. 1 and FIG. 2, where FIG. 1 shows an example block diagram of an electronic device or apparatus 100. The apparatus 100 may be an Internet of Things (loT) apparatus configured to perform various functions, such as for example, gathering information by one or more sensors, receiving or transmitting information, analyzing information gathered or received by the apparatus, or the like. The apparatus may comprise a video coding system, which may incorporate a codec. FIG. 2 shows a layout of an apparatus according to an example embodiment. The elements of FIG. 1 and FIG. 2 are explained next.

[0090] The apparatus 100 may for example be a mobile terminal or user equipment of a wireless communication system, a sensor device, a tag, or other lower power device. However, it would be appreciated that embodiments of the examples described herein may be implemented within any electronic device or apparatus which may process data by neural networks.

[0091] The apparatus 100 may comprise a housing 101 for incorporating and protecting the device. The apparatus 100 further may comprise a display 102 in the form of a liquid crystal display. In other embodiments of the examples described herein the display may be any suitable display technology suitable to display an image or video. The apparatus 100 may further comprise a keypad 104. In other embodiments of the examples described herein any suitable data or user interface mechanism may be employed. For example the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.

[0092] The apparatus may comprise a microphone 106 or any suitable audio input which may be a digital or analog signal input. The apparatus 100 may further comprise an audio output device which in embodiments of the examples described herein may be any one of: an earpiece 108, speaker, or an analog audio or digital audio output connection. The apparatus 100 may also comprise a battery (or in other embodiments of the examples described herein the device may be powered by any suitable mobile energy device such as solar cell, fuel cell or clockwork generator). The apparatus 100 may further comprise a camera 109 capable of recording or capturing images and / or video. The apparatus 100 may further comprise an infrared port for short range line of sight communication to other devices. In other embodiments the apparatus 100 may further comprise any suitable short range communication solution such as for example a Bluetooth wireless connection or a USB / firewire wired connection.

[0093] The apparatus 100 may comprise a controller 110, processor or processor circuitry for controlling the apparatus 100. The controller 110 may be connected to memory 112 which in embodiments of the examples described herein may store both data in the form of image and audio data and / or may also store instructions for implementation on the controller 110. The controller 110 may further be connected to codec circuitry 114 suitable for carrying out coding and / or decoding of audioand / or video data or assisting in coding and / or decoding carried out by the controller.

[0094] The apparatus 100 may further comprise a card reader 118 and a smart card 116, for example a UICC and UICC reader for providing user information and being suitable for providing authentication information for authentication and authorization of the user at a network.

[0095] The apparatus 100 may comprise radio interface circuitry 120 connected to the controller and suitable for generating wireless communication signals for example for communication with a cellular communications network, a wireless communications system or a wireless local area network. The apparatus 100 may further comprise an antenna 122 connected to the radio interface circuitry 120 for transmitting radio frequency signals generated at the radio interface circuitry 120 to other apparatus(es) and / or for receiving radio frequency signals from other apparatus(es).

[0096] The apparatus 100 may comprise a camera capable of recording or detecting individual frames which are then passed to the codec circuitry 114 or the controller for processing. The apparatus may receive the video image data for processing from another device prior to transmission and / or storage. The apparatus 100 may also receive either wirelessly or by a wired connection the image for coding / decoding. The structural elements of apparatus 100 described above represent examples of means for performing a corresponding function.

[0097] With respect to FIG. 3, an example of a system within which embodiments of the examples described herein can be utilized is shown. The system 300 comprises multiple communication devices which can communicate through one or more networks. The system 300 may comprise any combination of wired or wireless networks including, but not limited to a wireless cellular telephone network (such as a GSM, UMTS, CDMA, LTE, 4G, 5G network, etc.), a wireless local area network (WLAN) such as defined by any of the IEEE 802.x standards, a Bluetooth personal area network, an Ethernet local area network, a token ring local area network, a wide area network, and the Internet.

[0098] The system 300 may include both wired and wireless communication devices and / or apparatus 100 suitable for implementing embodiments of the examples described herein.

[0099] For example, the system shown in FIG. 3 shows a mobile telephone network 301 and a representation of the internet 302. Connectivity to the internet 302 may include, but is not limited to, long range wireless connections, short range wireless connections, and various wired connections including, but not limited to, telephone lines, cable lines, power lines, and similar communication pathways.

[0100] The example communication devices shown in the system 300 may include, but are not limited to, an electronic device or apparatus 100, a combination of a personal digital assistant (PDA)and a mobile telephone 304, a PDA 306, an integrated messaging device (IMD) 308, a desktop computer 310, a notebook computer 312, or a head-mounted apparatus. The head-mounted apparatus may be a head-mounted display (HMD), or glasses having a device such as a camera configured to encode and / or decode images and / or video. The apparatus 100 may be stationary or mobile when carried by an individual who is moving. The apparatus 100 may also be located in a mode of transport including, but not limited to, a car, a truck, a taxi, a bus, a train, a boat, an airplane, a bicycle, a motorcycle or any similar suitable mode of transport.

[0101] The embodiments may also be implemented in a set-top box; e.g., a digital TV receiver, which may / may not have a display or wireless capabilities, in tablets or (laptop) personal computers (PC), which have hardware and / or software to process neural network data, in various operating systems, and in chipsets, processors, DSPs and / or embedded systems offering hardware / software based coding.

[0102] Some or further apparatus may send and receive calls and messages and communicate with service providers through a wireless connection 314 to a base station 316. The base station 316 may be connected to a network server 318 that allows communication between the mobile telephone network 301 and the internet 302. The system may include additional communication devices and communication devices of various types.

[0103] The communication devices may communicate using various transmission technologies including, but not limited to, code division multiple access (CDMA), global systems for mobile communications (GSM), universal mobile telecommunications system (UMTS), time divisional multiple access (TDMA), frequency division multiple access (FDMA), transmission control protocolinternet protocol (TCP-IP), short messaging service (SMS), multimedia messaging service (MMS), email, instant messaging service (IMS), Bluetooth, IEEE 802.11, 3GPP Narrowband loT and any similar wireless communication technology. A communications device involved in implementing various embodiments of the examples described herein may communicate using various media including, but not limited to, radio, infrared, laser, cable connections, and any suitable connection.

[0104] In telecommunications and data networks, a channel may refer either to a physical channel or to a logical channel. A physical channel may refer to a physical transmission medium such as a wire, whereas a logical channel may refer to a logical connection over a multiplexed medium, capable of conveying several logical channels. A channel may be used for conveying an information signal, for example a bitstream, from one or several senders (or transmitters) to one or several receivers.

[0105] The embodiments may also be implemented in so-called loT devices. The Internet of Things (loT) may be defined, for example, as an interconnection of uniquely identifiable embeddedcomputing devices within the existing Internet infrastructure. The convergence of various technologies has and may enable many fields of embedded systems, such as wireless sensor networks, control systems, home / building automation, etc. to be included in the Internet of Things (loT). In order to utilize the Internet loT devices are provided with an IP address as a unique identifier. loT devices may be provided with a radio transmitter, such as a WLAN or Bluetooth transmitter or a RFID tag. Alternatively, loT devices may have access to an IP-based network via a wired network, such as an Ethernet-based network or a power-line connection (PLC).

[0106] FIG. 4 is a block diagram illustrating a system or apparatus 400 in accordance with several examples. In an example, the encoder 402 is used to encode an image or video from the scene 404, and the encoder 402 is implemented in a transmitting apparatus 406. The encoder 402 produces a bitstream 408 comprising signaling that is received by the receiving apparatus 410, which implements a decoder 412. The encoder 402 sends the bitstream 408 that comprises the herein described signaling. The decoder 412 forms the image or video for the scene 404-1, and the receiving apparatus 410 would present this to the user, e.g., via a smartphone, television, or projector among many other options.

[0107] In some examples, the transmitting apparatus 406 and the receiving apparatus 410 are at least partially within a common apparatus, and for example, are located within a common housing 414. In other examples the transmitting apparatus 406 and the receiving apparatus 410 are at least partially not within a common apparatus and have at least partially different housings. Therefore in some examples, the encoder 402 and the decoder 412 are at least partially within a common apparatus, and for example are located within a common housing 414. For example, the common apparatus comprising the encoder 402 and decoder 412 implements a codec. In other examples, the encoder 402 and the decoder 412 are at least partially not within a common apparatus and have at least partially different housings, but when together still implement a codec.

[0108] In some examples, 3D media from the capture (e.g., volumetric capture) at a viewpoint 416 of the scene 404, which includes a person 418) is converted via projection to a series of 2D representations with occupancy, geometry, attributes and / or displacements. Additional atlas information is also included in the bitstream to enable inverse reconstruction. For decoding, the received bitstream 408 is separated into its components with atlas information; occupancy, geometry, displacement, and attribute 2D representations. A 3D reconstruction is performed to reconstruct the scene 404-1 created looking at the viewpoint 416-1 with a “reconstructed” person 418-1. The “-1” are used to indicate that these are reconstructions of the original. As indicated at 420, the decoder 412 performs an operation(s) or action(s) based on the received signaling.

[0109] Encoding 422 performs encoding of constituent sequence, image, tile, slice or block based on the examples described herein. Decoding 424 performs decoding of sequence, image, tile, slice orblock, based on the examples described herein.

[0110] Having thus introduced a suitable but non-limiting technical context for the practice of the example embodiments of the present disclosure, example embodiments will now be described in detail.

[0111] Various embodiments relate to coding and decoding of digital video material.

[0112] Video codec comprises an encoder that transforms the input video into a compressed representation suited for storage / transmission and a decoder that can decompress the compressed video representation back into a viewable form. Typically, encoder discards some information in the original video sequence to be able to represent the video in a more compact form (that is, at lower bitrate).

[0113] Typical hybrid video codecs, such as H.264 / AVC, H.265 / HEVC and H.266 / VVC. encode the video information in two phases. Firstly, pixel values in a certain picture area (or “block”) are predicted, for example, by motion compensation means (finding and indicating an area in one of the previously coded pictures that corresponds closely to the block being coded) or by spatial means (using the pixel values around the block to be coded in a specified manner). Secondly, the prediction error, e.g., the difference between the predicted block of pixels and the original block of pixels, is coded. This is typically done by transforming the difference in pixel values using a specified transform (e.g., Discrete Cosine Transform (DCT) or a variant of it), quantizing the resulting transform coefficients and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, encoder can control the balance between the accuracy of the pixel representation (picture quality) and size of the resulting coded video representation (file size or transmission bitrate).

[0114] In some video codecs, such as H.265 / HEVC and H.266 / VVC, the video pictures are divided into coding units (CU) covering the area of the picture. A CU comprises one or more prediction units (PU) defining the prediction process for the samples within the CU and one or more transform units (TU) defining the prediction error coding process for the samples in the said CU. Typically, a CU comprise a rectangular block of samples with a size selectable from a predefined set of possible CU sizes. A CU with the maximum allowed size is typically named as LCU (largest coding unit) or CTU (coding tree unit) and the video picture is divided into non-overlapping CTUs. A CTU can be further split into a combination of smaller CUs, e.g. by recursively splitting the CTU and resultant CUs. Each resulting CU typically has at least one PU and at least one TU associated with it. Each PU and TU can be further split into smaller PUs and TUs to increase granularity of the prediction and prediction error coding processes, respectively. Each PU has prediction information associated with it defining what kind of a prediction is to be applied for the pixels within that PU (e.g. motion vector information for inter predicted PUs and intra prediction directionality information for intra predicted PUs). Similarly,each TU is associated with information describing the prediction error decoding process for the samples within the TU (including e.g. DCT coefficient information). It is typically signaled at CU level whether prediction error coding is applied or not for each CU. In the case there is no prediction error residual associated with the CU, it can be considered there are no TUs for the said CU. The division of the image into CUs, and division of CUs into PUs and TUs is typically signaled in the bitstream allowing the decoder to reproduce the intended structure of these units.

[0115] The decoder reconstructs the output video by applying prediction means similar to the encoder to form a predicted representation of the pixel blocks (using the motion or spatial information created by the encoder and stored in the compressed representation) and prediction error decoding (inverse operation of the prediction error coding recovering the quantized prediction error signal in spatial pixel domain). After applying prediction and prediction error decoding means the decoder sums up the prediction and prediction error signals (pixel values) to form the output video frame. The decoder (and encoder) can also apply additional filtering means to improve the quality of the output video before passing it for display and / or storing it as prediction reference for the forthcoming frames in the video sequence.

[0116] Instead, or in addition to approaches utilizing sample value prediction and transform coding for indicating the coded sample values, a color palette based coding can be used. Palette based coding refers to a family of approaches for which a palette, e.g., a set of colors and associated indexes, is defined and the value for each sample within a coding unit is expressed by indicating its index in the palette. Palette based coding can typically achieve good coding efficiency in coding units with a relatively small number of colors (such as image areas which are representing computer screen content, like text or simple graphics). In order to improve the coding efficiency of palette coding different kinds of palette index prediction approaches can be utilized, or the palette indexes can be run-length coded to be able to represent larger homogenous image areas efficiently. Also, in the case the CU contains sample values that are not recurring within the CU, escape coding can be utilized. Escape coded samples are transmitted without referring to any of the palette indexes. Instead, their values are indicated individually for each escape coded sample.

[0117] In typical video codecs the motion information is indicated with motion vectors associated with each motion compensated image block. Each of these motion vectors represents the displacement of the image block in the picture to be coded (in the encoder side) or decoded (in the decoder side) and the prediction source block in one of the previously coded or decoded pictures. To represent motion vectors efficiently those are typically coded differentially with respect to block specific predicted motion vectors. In typical video codecs the predicted motion vectors are created in a predefined way, for example calculating the median of the encoded or decoded motion vectors of the adjacent blocks. Another way to create motion vector predictions is to generate a list of candidate predictions fromadjacent blocks and / or co-located blocks in temporal reference pictures and signaling the chosen candidate as the motion vector predictor. In addition to predicting the motion vector values, the reference index of previously coded / decoded picture can be predicted. The reference index is typically predicted from adjacent blocks and / or or co-located blocks in temporal reference picture. Moreover, typical high efficiency video codecs employ an additional motion information coding / decoding mechanism, often called merging or merge mode, where all the motion field information, which includes motion vector and corresponding reference picture index for each available reference picture list, is predicted and used without any modification / correction. Similarly, predicting the motion field information is carried out using the motion field information of adjacent blocks and / or co-located blocks in temporal reference pictures and the used motion field information is signaled among a list of motion field candidate list filled with motion field information of available adjacent / co-located blocks.

[0118] Typically, video codecs support motion compensated prediction from at least one source image (uni-prediction) and two sources (bi-prediction). In the case of uni-prediction a single motion vector is applied whereas in the case of bi-prediction two motion vectors are determined and the motion compensated predictions from two sources are combined to create the final sample prediction. In the case of weighted prediction the relative weights of the two predictions can be adjusted, or a signaled offset can be added to the prediction signal.

[0119] In addition to applying motion compensation for inter picture prediction, similar approach can be applied to intra picture prediction. In this case the displacement vector indicates where from the same picture a block of samples can be copied to form a prediction of the block to be coded or decoded. This kind of intra block copying (IBC) methods can improve the coding efficiency substantially in presence of repeating structures within the frame, such as, text or other graphics.

[0120] In typical video codecs the prediction residual after motion compensation or intra prediction is first transformed with a transform kernel (like DCT) and then coded. The reason for this is that often there still exists some correlation among the residual and transform can in many cases help reduce this correlation and provide more efficient coding.

[0121] Typical video encoders utilize Lagrangian cost functions to find optimal coding modes, e.g., the desired Macroblock mode and associated motion vectors. This kind of cost function uses a weighting factor X to tie together the (exact or estimated) image distortion due to lossy coding methods and the (exact or estimated) amount of information that is required to represent the pixel values in an image area:

[0122] C=D+' / .R

[0123] Where C is the Lagrangian cost to be minimized, D is the image distortion (e.g. MeanSquared Error) with the mode and motion vectors considered, and R the number of bits needed to represent the required data to reconstruct the image block in the decoder (including the amount of data to represent the candidate motion vectors).

[0124] Typically, video is encoded in YUV or YCbCr color space as that is found to reflect some characteristics of human visual system and allows using lower quality representation for Cb and Cr channels as human perception is less sensitive to the chrominance fidelity those channels represent.

[0125] Partitioning in VVC

[0126] In VVC, each picture is divided into coding tree units (CTUs) similar to HEVC. A picture may also be divided into slices, tiles, bricks and sub-pictures. CTU may be split into smaller CUs using quaternary tree structure. Each CU may be divided using quad-tree and nested multi-type tree including ternary and binary split. There are specific rules to infer partitioning in in picture boundaries. The redundant split patterns are disallowed in nested multi-type partitioning.

[0127] Loop filter in VVC

[0128] The purpose of in-loop filtering is to reduce artifacts and distortions that can occur during the compression process. Compression techniques such as block-based motion compensation and discrete cosine transform (DCT) can introduce artifacts such as blocking, ringing, and blurring in the decoded video. In-loop filtering is designed to reduce these artifacts and improve the perceived visual quality of the video.

[0129] In-loop filters play a critical role in the maintenance of compressed video quality since they can not only improve the quality of the current frame but can also provide a higher quality reference for subsequent frames. Referring to FIG. 5, it depicts the in-loop filters 502 in VVC. Four processing steps, namely a luma mapping with chroma scaling (LMCS) process 504, followed by a deblocking filter (DBF) 506, a sample adaptive offset (SAO) filter 508, and an adaptive loop filter (ALF) 510 are applied to the reconstructed samples before writing them into the decoded picture buffer. The DBF 506 and SAO filter 508 are similar to that of the HEVC standard, whereas LMCS process 504 and ALF block-based are newly introduced in VVC.

[0130] A block-based ALF 510 is used in VVC, which comprises luma ALF, chroma ALF and cross-component ALF (CC-ALF). The ALF filter coefficients are either pre-defined and fixed in both encoder and decoder or adaptively signaled on a picture basis using adaptation parameter set (APS).

[0131] ALF parameter signaling in APS

[0132] ALF filter parameters are signaled in Adaptation Parameter Set (APS). In one APS, up to25 sets of luma filter coefficients and clipping value indexes, and up to eight sets of chroma filter coefficients and clipping value indexes can be signaled. To reduce bits overhead, filter coefficients of different classification for luma component can be merged. In slice header, the indices of the APSs used for the current slice are signaled.

[0133] Clipping value indexes, which are decoded from the APS, allow determining clipping values using a table of clipping values for both luma and Chroma components. These clipping values are dependent of the internal bitdepth. More precisely, the clipping values are obtained by the following formula:

[0134] AlfClip= {round(2B-“*n) for n E [0. . N — 1]}

[0135] with B equal to the internal bitdepth, a is a pre-defined constant value equal to 2.35, and N equal to 4 which is the number of allowed clipping values in VVC. The AlfClip is then rounded to the nearest value with the format of power of 2.

[0136] In slice header, up to 7 APS indices can be signaled to specify the luma filter sets that are used for the current slice. The filtering process can be further controlled at CTB level. A flag is signaled to indicate whether ALF is applied to a luma CTB. A luma CTB can choose a filter set among 16 fixed filter sets and the filter sets from APSs. A filter set index is signaled for a luma CTB to indicate which filter set is applied. The 16 fixed filter sets are pre-defined and hard-coded in both the encoder and the decoder.

[0137] For chroma component, an APS index is signaled in slice header to indicate the chroma filter sets being used for the current slice. At CTB level, a filter index is signaled for each chroma CTB if there is more than one chroma filter set in the APS.

[0138] The filter coefficients are quantized with norm equal to 128. In order to restrict the multiplication complexity, a bitstream conformance is applied so that the coefficient value of the noncentral position shall be in the range of -27 to 27 - 1, inclusive. The central position coefficient is not signaled in the bitstream and is considered as equal to 128.

[0139] For each ALF_APS, in the current draft of VVC there are:Maximum of 25 luma ALF filtersMaximum of 8 chroma ALF filtersMaximum of 4 ccALF filters for Cb componentMaximum of 4 ccALF filters for Cr component

[0140] To limit the required memory for storing ALF_APS at decoder side, currently up to 8 ALF_APSs are allowed.

[0141] APS and ALF APS in VVC

[0142] In current draft of VVC, syntax of APS is as below. There is an APS_ID (aps_adaptation_parameter_set_id) that can take value between 0 to 31. Eight of these IDs are reserved for ALF APSs.

[0143] In the current draft of VVC, syntax of indicating active / used APS in the slice header is as below. A similar syntax is valid when active APSs are signaled in picture header.

[0144] In current release of VVC, syntax of signaling ALF related parameters in CTU level is as below.

[0145] Bilateral Filter and Joint-Bilateral Filter

[0146] A bilateral filter is a non-linear, edge-preserving, and noise-reducing smoothing filter for images. It replaces the intensity of each pixel with a weighted average of intensity values from nearby pixels. This weight typically based on a Gaussian distribution. The weights depend not only on distance of pixel, but also on the range differences, e.g., values of pixel values.

[0147] The joint bilateral filter is an extension of the bilateral filter, incorporating additional information as a ‘guide image’. This guide image provides extra contextual information, e.g., higher resolution content to better identify edges and boundaries, to the filtering process. Instead of considering the spatial distance and range difference between pixels in the input image, the joint bilateral filter considers range difference for corresponding pixels in the guide image. This allows the filter to better preserve edges and details that are important in the guide image, even when they are not as prominent in the input image.

[0148] The joint bilateral filter works by computing weights for each pixel based on both the spatial distance in the input image and the range distance between corresponding pixels in the guide image. These weights are then used to compute a weighted average of pixel values in a local neighborhood, similar to how the traditional bilateral filter operates.

[0149] Typical video codecs, such as H.265 / HEVC and H.266 / VVC standards apply different kinds of means for predicting blocks of samples to be able to compress and represent image date efficiently. Those also use different kinds of processes to smooth the predicted sample surface to attenuate noise and possible unwanted discontinuities in the predicted samples that may arise either from imperfectly coded reference samples or from the block based prediction process that may not always be able to extend texture from one block to another in a continuous manner. Such methodsinclude, for example, intra reference sample smoothing in H.265 / HEVC [1] and H.266 / VVC, as well as position dependent prediction combination (PDPC) in H.266 / VVC.

[0150] Also, in JVET document JVET-AF0199: “Non-EE2: Bilateral Filtering for Intra Prediction”, October 2023, it was proposed to use a bilateral filter for smoothing either intra prediction reference samples or for smoothing intra prediction generated using either filtered or unfiltered reference samples. Bilateral filter in general refers to a filter that adjusts its strength considering the characteristics of the input samples.

[0151] The ECM described in JVET document JVET-AF2025: “Algorithm description of Enhanced Compression Model 11 (ECM 11)”, October 2023 software model is describing different approaches blending multiple intra predictions together. Such approaches include, for example, the decoder side intra mode derivation (DIMD) which can blend up to five directional predictions and a planar prediction; and the fusion for template-based intra mode derivation (TIMD) which can blend up to two directional predictions.

[0152] Video is often represented in a YUV420 format, with chroma components represented at a lower resolution than the luma component. Thus, the chroma components provide a sub-sampled, smoother representation of the information available in the luma component. Applying a joint bilateral filter on the chroma components, taking the range difference information from corresponding pixel of the luma component, can help carry some of the finer detail into the chroma components and thus improve encoding quality.

[0153] In the development of ECM, it is currently discussed to apply bilateral filter on prediction references as described in JVET document JVET-AH2097: “E2-2.14: Bilateral Filtering for Intra Prediction”, April 2024. The bilateral filter is applied on each luma and color component individually. However, video is often represented in a YUV420 format, with chroma components represented at a lower resolution than the luma component. Thus, the range component of a bilateral filter on a chroma component is sub-sampling the full range information that would be available in the luma component.

[0154] A method operating according to various embodiments add a joint bilateral filter on chroma components to the in-loop filtering chain of a video codec, where the range difference is taking from corresponding pixel of the luma component as a ‘guide image’ .

[0155] It is further described how parameters of the joint bilateral filter can be signaled as part of an APS.

[0156] Example embodiment

[0157] In an embodiment a joint bilateral filter is applied as in-loop filter to the chromacomponent samples of a block or image. The spatial difference information of the joint bilateral filter is taken from the respective chroma samples. The range difference information is taken from corresponding pixel of the luma component.

[0158] FIG. 6 illustrates example filter kernel, in accordance with an embodiment. 602 is an example of an spatial filter kernel applied to chroma component that can be used for a joint bilateral filter. 604 is an example of a corresponding range kernel applied to a luma component that can be used for a joint bilateral filter. For YUV420 a 5X5 area in the luma component, covers 9X9 area in the chroma component.

[0159] The filtered result Ueit for a sample at location x in chroma component U is then defined as following:

[0161] Where:

[0162] Uyj(tis the filtered chroma component U;

[0163] U is the original chroma component;

[0164] x are the coordinates of the current sample / pixel to be filtered;

[0165] fl is the filter window centered in x;

[0166] fsis the spatial kernel for differences in coordinates (typically Gaussian kernel);

[0167] fris the range kernel for differences in intensities; and

[0168] Y is the corresponding luma component to chroma component U.

[0169] Additional embodiments

[0170] In an embodiment, a parameter is signaled to activate the joint bilateral in-loop filter per sequence, image, tile, slice or block. Whereas block may refer to CTU, CU, or PU.

[0171] In an embodiment, the size, distribution, and weight assigned to the spatial and range filter kernel are fixed.

[0172] In an embodiment, the size, distribution and weight assigned to the spatial and range filter kernel are signaled per sequence, image, tile, slice or block. Whereas block may refer to CTU, CU, or PU.

[0173] In an embodiment, at least one of the elements of a joint bilateral filter (size, distribution or weight assigned) to at least one of the filter kernel (spatial or range) is fixed. Remaining elements are signaled per sequence, image, tile, slice or block. The block may refer to CTU, CU, or PU.

[0174] In an embodiment, at least one of the elements of a joint bilateral filter (size, distribution or weight assigned) to at least one of the filter kernel (spatial or range) is derived at the selected prediction mode of a block.

[0175] In an embodiment, signaling of joint bilateral filter elements is done using an adaption parameter set (APS).

[0176] In an embodiment, at least one of the elements of a joint bilateral filter (size, distribution or weight assigned) to at least one of the filter kernel (spatial or range) is derived from the chroma sampling format of the sequence.

[0177] Following are some example advantage of embodiments described above:

[0178] Initial testing of shows video coding gains, for example, when used on content with higher frequency content in the chroma planes, such as gaming content.

[0179] ArenaofValorant sequence (gaming content):

[0180] 4people (for comparison)

[0181] In following embodiment, it is further described how parameters of the joint bilateral filter can be adjusted as a function of the number or type of prediction blocks that were included in the blending process or as a function of the selected prediction mode.

[0182] Another example embodiment

[0183] In the another example embodiment, a representation of a block is generated using a set of prediction parameters. A set of prediction parameters include one or more prediction parameters, such as intra prediction mode (e.g., an integer number identifying what kind of a prediction process is used for the block), inter prediction mode, prediction type (e.g., when using planar, DC or directional prediction), prediction direction when directional prediction is used, reference sample filtering mode, and the like. The representation includes a sample value for each sample in the block.

[0184] A j oint bilateral filter is applied to the chroma component samples of the prediction block.The spatial difference information is taken from the respective chroma samples. The range difference information is taken from corresponding pixel of the luma component. The corresponding, or co-located luma sample can be derived using a downsampling filter when necessary. For example, with YUV420 format a 6-tap downsampling filter, such as those used by cross-component tools like CCLM may be used to match the luma sample grid with chroma sample grid. More advanced downsampling filters may also be employed to obtain the desired quality on the co-located luma sample. The selection of the downsampling filter may be a fixed design choice, signaled or inferred. The luma samples may also be filtered with other types of filters, such as high-pass or gradient filters to derive luma representation which amplifies the edge information.

[0185] In another embodiment, at least two initial representations of a block are generated using different sets of prediction parameters, for example, a prediction parameter set SA and a prediction parameter set SB- Each of the initial representations can include a sample value for each sample in the block.

[0186] The at least two initial representations are blended to form a combined prediction block either using constant or variable weights. For example, equal weights for the representations may be used generating an average of the sample values in the initial representations. As an alternative example, unequal weighting can be used to giving more weight for some of the initial representations with respect to others. The weighting can be decided for example by considering the distribution of the elements in a histogram of gradients (e.g., in DIMD) or considering differences in template costs (e.g., in TIMD). Also, a sample-based weighting may be used considering instead or in addition, distances of the predicted samples from different borders of the block.

[0187] A j oint bilateral filter is applied to the chroma component samples of the prediction block.The spatial difference information is taken from the respective chroma samples. The range difference information is taken from corresponding pixel of the luma component.

[0188] As the combined prediction block can be considered to be a smoother representation ofthe samples than what the initial representations were (for random samples following normal distribution and having a variance of v, the variance of average of N samples is v / N), it may be advantageous to determine or adjust the parameters of the joint bilateral filter considering the low variance nature of the combined representation.

[0189] Additional embodiments

[0190] Parameters of the joint bilateral filter may be determined in different ways. For example, quantization parameter (QP) used in residual coding of the sample block can be used to determine parameters of the joint bilateral filter.

[0191] As an additional example, usage of a joint bilateral filter on a prediction block can be conditioned to the intra prediction mode or the number of initial representations that are used to generate the combined prediction by blending the initial representations. For example, the joint bilateral filter can be enabled for a specific set of intra prediction modes, such as the directional prediction modes, but disabled for the DIMD mode which requires blending of multiple initial representations. As another example, joint bilateral filter can be enabled for a determined subset of intra prediction modes and disabled for all the modes that may require blending more than N initial representations to form the combined prediction.

[0192] Decision to adjust parameters of the joint bilateral filter can be done in different ways. For example, the number of initial representations blended to form a prediction block can be determined and one of more parameters can be determined based on that number. Alternatively, a one or more parameter of the joint bilateral filter can be determined based on the selected intra prediction mode; where the mode can further determine the number of initial representations or a range for the number of initial representations that can be blended to form the combined prediction block.

[0193] As an example, for a first intra mode that does not require blending of initial representations, such as a directional intra prediction more, joint bilateral filtering can be performed with a first set of parameters. Whereas, for a second intra prediction mode which require blending of initial representations to form a combined prediction block, the joint bilateral filtering can be performed using a second set of parameters, where the first set and second set of parameters differ from each other.

[0194] An example method includes:

[0195] Determining a block of samples to be predicted;

[0196] Determining a prediction mode for the block of samples;

[0197] Determining a representation for the block to be predicted based on the prediction mode;and

[0198] Applying a joint bilateral filter to the chroma component samples of the prediction block, where the range difference is derived from corresponding pixel of the luma component as a ‘guide image’ .

[0199] The example method may further include, wherein the at least one parameter of a joint bilateral filter is predetermined by a user.

[0200] The example method may further include, wherein the at least one parameter of a joint bilateral filter is derived based on the prediction mode.

[0201] The example method may further include, wherein the at least one parameter of a joint bilateral filter is derived based on a quantization parameter of the block.

[0202] The example method may further include, wherein the at least one parameter of a joint bilateral filter is derived based on the quantization parameter difference between the luma and chroma components of the block.

[0203] The example method may further include, wherein the at least one parameter is used to decide when bilateral filter is applied to the combined prediction block or not.

[0204] The example method may further include, wherein the at least one parameter of a joint bilateral filter signaled in or along a bitstream.

[0205] Another example method includes:

[0206] Determining a block of samples to be predicted;

[0207] Determining a prediction mode for the block of samples;

[0208] Determining at least a first and a second initial representations for the block to be predicted based on the prediction mode;

[0209] Blending at least the first and the second initial representations to form a combined prediction block; and

[0210] Applying a joint bilateral filter to the chroma component samples of the combined prediction block, where the range difference is derived from corresponding pixel of the luma component as a guide image.

[0211] FIG. 7 is an example apparatus 700, which may be implemented in hardware, configuredto implement the examples described herein. The apparatus 700 comprises at least one processor 702 (e.g., an FPGA and / or CPU), at least one memory 704 including computer program code 705, the computer program code 705 having instructions to carry out the methods described herein, wherein the at least one memory 704 and the computer program code 705 are configured to, with the at least one processor 702, cause the apparatus 700 to implement circuitry, a process, component, module, or function (implemented with control module 706) to implement the examples described herein, including using a joint bilateral filter (e.g., a joint bilateral filter within a video codec coding loop) for media coding (e.g., an image and / or video coding). Optionally included encoder 708 of the control module 706 implements encoding based on the examples described herein, and optionally included decoder 710 implements decoding based on the examples described herein. The at least one memory 704 may be a non-transitory memory, a transitory memory, a volatile memory (e.g. RAM), or a non-volatile memory (e.g., ROM).

[0212] The apparatus 700 includes a display and / or I / O interface 712, which includes user interface (UI) circuitry and elements, that may be used to display features or a status of the methods described herein (e.g., as one of the methods is being performed or at a subsequent time), or to receive input from a user such as with using a keypad, camera, touchscreen, touch area, microphone, biometric recognition, one or more sensors, etc. The apparatus 700 includes one or more communication e.g. network (N / W) interfaces (I / F(s)) 714. The communication I / F(s) 714 may be wired and / or wireless and communicate over the Internet / other network(s) via any communication technique including via one or more links 716. The communication I / F(s) 714 may comprise one or more transmitters or one or more receivers.

[0213] The transceiver 718 comprises one or more transmitters 720 and one or more receivers 722. The transceiver 718 and / or communication I / F(s) 714 may comprise standard well-known components such as an amplifier, filter, frequency-converter, (de)modulator, and encoder / decoder circuitries and one or more antennas, such as antennas 724 used for communication over wireless link 726.

[0214] The control module 706 of the apparatus 700 comprises one of or both parts 706-1 and / or 706-2, which may be implemented in a number of ways. The control module 706 may be implemented in hardware as control module 706-1, such as being implemented as part of the at least one processor 702. The control module 706-1 may be implemented also as an integrated circuit or through other hardware such as a programmable gate array. In another example, the control module 706 may be implemented as control module 706-2, which is implemented as computer program code (having corresponding instructions) 705 and is executed by the at least one processor 702. For instance, the at least one memory 704 store instructions that, when executed by the at least one processor 702, cause the apparatus 700 to perform one or more of the operations as described herein. Furthermore, the atleast one processor 702, the at least one memory 704, and example algorithms (e.g., as flowcharts and / or signaling diagrams), encoded as instructions, programs, or code, are means for causing performance of the operations described herein.

[0215] The apparatus 700 to implement the functionality of control module 706 may correspond to any of the apparatuses depicted herein. Alternatively, apparatus 700 and its elements may not correspond to any of the other apparatuses depicted herein, as apparatus 700 may be part of a self- organizing / optimizing network (SON) node or other node, such as a node in a cloud.

[0216] The apparatus 700 may also be distributed throughout the network including within and between apparatus 700 and any network element (such as a base station and / or terminal device and / or user equipment).

[0217] Interface 728 enables data communication and signaling between the various items of apparatus 700, as shown in FIG. 7. For example, the interface 728 may be one or more buses such as address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like. Computer program code (e.g. instructions) 705, including control module 706 may comprise object-oriented software configured to pass data or messages between objects within computer program code 705. The apparatus 700 need not comprise each of the features mentioned, or may comprise other features as well. The various components of apparatus 700 may at least partially reside in a housing 730, or a subset of the various components of apparatus 700 may at least partially be located in different housings, which different housings may include housing 730.

[0218] FIG. 8 shows a schematic representation of non-volatile memory media 800a (e.g. computer / compact disc (CD) or digital versatile disc (DVD)) and 800b (e.g. universal serial bus (USB) memory stick) and 800c (e.g. cloud storage for downloading instructions and / or parameters 802 or receiving emailed instructions and / or parameters 802) storing instructions and / or parameters 802 which when executed by a processor allows the processor to perform one or more of the operations of the methods described herein. Instructions and / or parameters 802 may represent or correspond to a non- transitory computer readable medium.

[0219] FIG. 9 is an example method 900 performed with an encoder or a decoder, based on the examples described herein. At 902, the method 900 includes applying a joint bilateral filter within a video codec coding loop on a chroma component of a pixel, wherein a range difference is derived from a luma component of the pixel as a guide image.

[0220] In an embodiment, a spatial difference information of the joint bilateral filter is determined from respective chroma components.

[0221] In an embodiment, a filtered result for the pixel in the chroma component is determined based on at least the following: an original chroma component (U); coordinates of the pixel (x); a filter window (fl) centered at (x); a spatial kernel (fs) for differences in coordinates; a range kernel (fr) for differences in intensities; and the corresponding luma component (Y) to the original chroma component (U).

[0222] In an embodiment, the method 900 may further include signaling, in or along a bitstream or receiving from or along the bitstream, one or more parameters to activate the joint bilateral filter per sequence, image, tile, slice, or block.

[0223] In an embodiment, the method 900 may further include signaling, in or along the bitstream or receiving from or along the bitstream a size, distribution, and weight assigned to a spatial and a range filter kernel per sequence, image, tile, slice, or block.

[0224] The method 900 may be performed with an encoding apparatus or a decoding apparatus, such as the apparatus 100, 700, apparatuses depicted in FIG. 3 and FIG. 4, for example, the transmitting apparatus 406 with the encoder 402 or the apparatus 400 with the encoder 402; or the receiving apparatus 410 with the decoder 412 or the apparatus 400 with the decoder 412.

[0225] FIG. 10 is another example method 1000 performed with an encoder or a decoder, based on the examples described herein. At 1002, the method 1000 includes determining a block of samples to be predicted. At 1004, the method 1000 includes determining a prediction mode for the block of samples. At 1006, the method 1000 includes determining a representation for the block to be predicted based on the prediction mode. At 1008, the method 1000 includes applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

[0226] In an embodiment, the method 1000 may further include deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0227] In an embodiment, the method 1000 may further include signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0228] The method 1000 may be performed with an encoding apparatus or a decoding apparatus, such as the apparatus 100, 700, apparatuses depicted in FIG. 3 and FIG. 4, for example, the transmitting apparatus 406 with the encoder 402 or the apparatus 400 with the encoder 402; or the receiving apparatus 410 with the decoder 412 or the apparatus 400 with the decoder 412.

[0229] FIG. 11 is yet another example method 1100 performed with an encoder or a decoder, based on the examples described herein. At 1102, the method 1100 includes determining a block of samples to be predicted. At 1104, the method 1100 includes determining a prediction mode for the block of samples. At 1106, the method 1100 includes determining at least a first and a second initial representations for the block to be predicted based on the prediction mode. At 1108, the method 1100 includes blending at least the first and the second initial representations to form a combined prediction block. At 1110, the method 1100 includes applying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image..

[0230] In an embodiment, the method 1100 may further include deriving at least one parameter of the joint bilateral filter is on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block..

[0231] In an embodiment, the method 1100 may further include using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

[0232] In an embodiment, the method 1100 may further include signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter

[0233] The method 1100 may be performed with an encoding apparatus or a decoding apparatus, such as the apparatus 100, 700, apparatuses depicted in FIG. 3 and FIG. 4, for example, the transmitting apparatus 406 with the encoder 402 or the apparatus 400 with the encoder 402; or the receiving apparatus 410 with the decoder 412 or the apparatus 400 with the decoder 412.

[0234] FIG. 12 is still another example method 1200 performed with an encoder or a decoder, based on the examples described herein. At 1202, the method 1200 includes determining a block of samples to be predicted. At 1204, the method 1200 includes determining a set of prediction parameters for the block of samples. At 1206, the method 1200 includes determining a representation for the block to be predicted based at least on the set of prediction parameters. At 1208, the method 1200 includes applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

[0235] In an embodiment, the set of prediction parameters comprises one or more of an intra prediction mode, an inter prediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

[0236] In an embodiment, the representation comprises a sample value for each sample.

[0237] In an embodiment, the method 1200 may further include deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0238] In an embodiment, the method 1200 may further include signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0239] The method 1200 may be performed with an encoding apparatus or a decoding apparatus, such as the apparatus 100, 700, apparatuses depicted in FIG. 3 and FIG. 4, for example, the transmitting apparatus 406 with the encoder 402 or the apparatus 400 with the encoder 402; or the receiving apparatus 410 with the decoder 412 or the apparatus 400 with the decoder 412.

[0240] FIG. 13 is still another example method 1300 performed with an encoder or a decoder, based on the examples described herein. At 1302, the method 1300 includes determining a block of samples to be predicted. At 1304, the method 1300 includes determining a first set of prediction parameters and a second set of prediction parameters for the block of samples. At 1306, the method 1300 includes determining at least a first and a second initial representations for the block to be predicted based on the first set of prediction parameters and the second set of prediction parameters. At 1308, the method 1300 includes, blending at least the first and the second initial representations to form a combined prediction block. At 1310, the method 1300 includes applying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image.

[0241] In an embodiment, the first set of prediction parameters and the second set of prediction parameters comprise one or more of an intra prediction mode, an inter prediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

[0242] In an embodiment, the representation comprises a sample value for each sample.

[0243] In an embodiment, the method 1300 may further include deriving least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

[0244] In an embodiment, the method 1300 may further include using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

[0245] In an embodiment, the method 1300 may further include signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

[0246] The method 1300 may be performed with an encoding apparatus or a decoding apparatus, such as the apparatus 100, 700, apparatuses depicted in FIG. 3 and FIG. 4, for example, the transmitting apparatus 406 with the encoder 402 or the apparatus 400 with the encoder 402; or the receiving apparatus 410 with the decoder 412 or the apparatus 400 with the decoder 412.

[0247] As described above, FIGs. 9 to 13 include flowcharts of an apparatus (e.g. 100, 700, or any other apparatuses described herein), method, and computer program product according to certain example embodiments. It will be understood that each block of the flowcharts, and combinations of blocks in the flowcharts, may be implemented by various means, such as hardware, firmware, processor, circuitry, and / or other devices associated with execution of software including one or more computer program instructions. For example, one or more of the procedures described above may be embodied by computer program instructions. In this regard, the computer program instructions which embody the procedures described above may be stored by a memory (e.g. 58, 125, or 704) of an apparatus employing an embodiment of the present invention and executed by processing circuitry (e.g., 56, 120, or 702) of the apparatus. As will be appreciated, any such computer program instructions may be loaded onto a computer or other programmable apparatus (e.g., hardware) to produce a machine, such that the resulting computer or other programmable apparatus implements the functions specified in the flowchart blocks. These computer program instructions may also be stored in a computer-readable memory that may direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture, the execution of which implements the function specified in the flowchart blocks. The computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operations to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide operations for implementing the functions specified in the flowchart blocks.

[0248] A computer program product is therefore defined in those instances in which the computer program instructions, such as computer -readable program code portions, are stored by at least one non-transitory computer-readable storage medium with the computer program instructions, such as the computer-readable program code portions, being configured, upon execution, to perform the functions described above, such as in conjunction with the flowchart(s) of FIGs. 9 to 13. In other embodiments, the computer program instructions, such as the computer-readable program code portions, need not be stored or otherwise embodied by a non-transitory computer-readable storage medium, but may, instead, be embodied by a transitory medium with the computer program instructions, such as the computer-readable program code portions, still being configured, upon execution, to perform the functions described above.

[0249] Accordingly, blocks of the flowcharts support combinations of means for performing the specified functions and combinations of operations for performing the specified functions for performing the specified functions. It will also be understood that one or more blocks of the flowcharts, and combinations of blocks in the flowcharts, may be implemented by special purpose hardware-based computer systems which perform the specified functions, or combinations of special purpose hardware and computer instructions.

[0250] In some embodiments, certain ones of the operations above may be modified or further amplified. Furthermore, in some embodiments, additional optional operations may be included. Modifications, additions, or amplifications to the operations above may be performed in any order and in any combination.

[0251] Some embodiments have been described in relation to one or more neural networks performing visual temporal extrapolation. It is to be understood that embodiments can be realized with any generative modelling neural networks.

[0252] In the above, some example embodiments have been described with the help of syntax of the bitstream. It needs to be understood, however, that the corresponding structure and / or computer program may reside at the encoder for generating the bitstream and / or at the decoder for decoding the bitstream.

[0253] In the above, where example embodiments have been described with reference to an encoder, it needs to be understood that the resulting bitstream and the decoder have corresponding elements in them. Likewise, where example embodiments have been described with reference to a decoder, it needs to be understood that the encoder has structure and / or computer program for generating the bitstream to be decoded by the decoder.

[0254] Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, although the foregoing descriptions and the associated drawings describe example embodiments in the context of certain example combinations of elements and / or functions, it should be appreciated that different combinations of elements and / or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and / or functions than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Accordingly, the description is intended to embraceall such alternatives, modifications and variances which fall within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0255] It should be understood that the foregoing description is only illustrative. Various alternatives and modifications may be devised by those skilled in the art. For example, features recited in the various dependent claims could be combined with each other in any suitable combination(s). In addition, features from different embodiments described above could be selectively combined into a new embodiment. Accordingly, the description is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.

[0256] References to a ‘computer’ , ‘processor’ , etc. should be understood to encompass not only computers having different architectures such as single / multi-processor architectures and sequential (Von Neumann) / parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGA), application specific circuits (ASIC), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device such as instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device, and the like.

[0257] As used herein, the term ‘circuitry’ may refer to any of the following: (a) hardware circuit implementations, such as implementations in analog and / or digital circuitry, and (b) combinations of circuits and software (and / or firmware), such as (as applicable): (i) a combination of processor(s) or (ii) portions of processor(s) / software including digital signal processor(s), software, and memory(ies) that work together to cause an apparatus to perform various functions, and (c) circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even when the software or firmware is not physically present. This description of ‘circuitry’ applies to uses of this term in this application. As a further example, as used herein, the term ‘circuitry’ would also cover an implementation of merely a processor (or multiple processors) or a portion of a processor and its (or their) accompanying software and / or firmware. The term ‘circuitry’ would also cover, for example and when applicable to the particular element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or another network device.

[0258] Circuitry or Circuit: As used in this application, the term ‘circuitry’ or ‘circuit’ may refer to one or more or all of the following:(a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry); and(b) combinations of hardware circuits and software, such as (as applicable):(i) a combination of analog and / or digital hardware circuit(s) with software / firmware; and(ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions); and(c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0259] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example, and when applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

Claims

CLAIMSWhat is claimed is:

1. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: applying a joint bilateral filter within a video codec coding loop on a chroma component of a pixel, wherein a range difference is derived from a luma component of the pixel as a guide image.

2. The apparatus of claim 1, wherein a spatial difference information of the joint bilateral filter is determined from respective chroma components.

3. The apparatus of any of claims 1 or 2, wherein a filtered result for the pixel in the chroma component is determined based on at least the following: an original chroma component (U); coordinates of the pixel (x); a filter window (fl) centered at (x); a spatial kernel (fs) for differences in coordinates; a range kernel (fr) for differences in intensities; and the corresponding luma component (Y) to the original chroma component (U).

4. The apparatus of any of the previous claims, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, one or more parameters to activate the joint bilateral filter per sequence, image, tile, slice, or block.

5. The apparatus of any of the previous claims, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along the bitstream or receiving from or along the bitstream a size, distribution, and weight assigned to a spatial and a range filter kernel per sequence, image, tile, slice, or block.

6. The apparatus of any of the previous claims, wherein at least one of elements of the joint bilateral filter for at least one of filter kernels is fixed, and wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along the bitstream or receiving from or along the bitstream, remaining elements , per sequence,image, tile, slice, or block.

7. The apparatus of any of the previous claims, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving the at least one of elements of the joint bilateral filter for at least one of the filter kernel from a selected prediction mode of a block.

8. The apparatus of any of the claims 1 to 6, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving the at least one of the elements of the joint bilateral filter for the at least one of the filter kernels from the chroma sampling format of a sequence.

9. The apparatus of any of the claims 6 to 8, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along the bitstream or receiving from or along the bitstream, the at least one of elements of the joint bilateral filter by using an adaption parameter set.

10. The apparatus of any of the previous claims, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a prediction mode for the block of samples; and determining a representation for the block to be predicted based on a prediction mode.

11. The apparatus of claim 10, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and the chroma components of the block.

12. The apparatus of any of the claims 1 to 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining at least a first and a second initial representations for the block to bepredicted based on a prediction mode; and blending at least the first and the second initial representations to form a combined prediction block.

13. The apparatus of claim 12, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and the chroma components of the block.

14. The apparatus of claim 13, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

15. The apparatus of any of the claims 11, 13, or 14, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along the bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

16. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining a representation for the block to be predicted based on the prediction mode; and applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

17. The apparatus of claim 16, wherein instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter based on:an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

18. The apparatus of the claim 17, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

19. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on the prediction mode; blending at least the first and the second initial representations to form a combined prediction block; and applying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image.

20. The apparatus of claim 19, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter is on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

21. The apparatus of claim 20, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

22. The apparatus of any of the claims 20 or 21, wherein the instructions, when execute by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

23. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a set of prediction parameters for the block of samples; determining a representation for the block to be predicted based at least on the set of prediction parameters; and applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

24. The apparatus of claim 23, wherein the set of prediction parameters comprises one or more of an intra prediction mode, an inter prediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

25. The apparatus of any of the claims 23 or 24, wherein the representation comprises a sample value for each sample.

26. The apparatus of any of the claims 23 to 25, wherein instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

27. The apparatus of the claim 26, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

28. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: determining a block of samples to be predicted; determining a first set of prediction parameters and a second set of prediction parameters for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on the first set of prediction parameters and the second set of prediction parameters; blending at least the first and the second initial representations to form a combined prediction block; and applying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image.

29. The apparatus of claim 28, wherein the first set of prediction parameters and the second set of prediction parameters comprise one or more of an intra prediction mode, an inter prediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

30. The apparatus of any of the claims 28 or 29, wherein the representation comprises a sample value for each sample.

31. The apparatus of any of the claims 28 to 30, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: deriving least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

32. The apparatus of claim 31, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

33. The apparatus of any of the claims 31 or 32, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to perform: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

34. A method comprising: applying a joint bilateral filter within a video codec coding loop on a chroma component of a pixel, wherein a range difference is derived from a luma component of the pixel as a guide image.

35. The method of claim 34, wherein a spatial difference information of the joint bilateral filter is determined from respective chroma components.

36. The method of any of claims 34 or 35, wherein a filtered result for the pixel in the chroma component is determined based on at least the following: an original chroma component (U); coordinates of the pixel (x); a filter window (fl) centered at (x); a spatial kernel (fs) for differences in coordinates; a range kernel (fr) for differences in intensities; and the corresponding luma component (Y) to the original chroma component (U).

37. The method of any of the claims 34 to 36, further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, one or more parameters to activate the joint bilateral filter per sequence, image, tile, slice, or block.

38. The method of any of the claims 34 to 37 further comprising: signaling, in or along the bitstream or receiving from or along the bitstream a size, distribution, and weight assigned to a spatial and a range filter kernel per sequence, image, tile, slice, or block.

39. The method of any of the claims 34 to 38, wherein at least one of elements of the joint bilateral filter for at least one of filter kernels is fixed, and wherein the method further comprises: signaling, in or along the bitstream or receiving from or along the bitstream, remaining elements , per sequence, image, tile, slice, or block.

40. The method of any of the claims 34 to 39 further comprising: deriving the at least one of elements of the joint bilateral filter for at least one of the filter kernel from a selected predictionmode of a block.

41. The method of any of the claims 34 to 40 further comprising: deriving the at least one of the elements of the joint bilateral filter for the at least one of the filter kernels from the chroma sampling format of a sequence.

42. The method of any of the claims 39 to 41 further comprising: signaling, in or along the bitstream or receiving from or along the bitstream, the at least one of elements of the joint bilateral filter by using an adaption parameter set.

43. The method of any of the claims 34 to 42 further comprising: determining a block of samples to be predicted; determining a prediction mode for the block of samples; and determining a representation for the block to be predicted based on a prediction mode.

44. The method of claim 43 further comprising: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and the chroma components of the block.

45. The method of any of the claims 34 to 42 further comprising: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on a prediction mode; and blending at least the first and the second initial representations to form a combined prediction block.

46. The method of claim 45 further comprising: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; andthe quantization parameter difference between the luma and the chroma components of the block.

47. The method of claim 46 further comprising: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

48. The method of any of the claims 44, 46, or 47 further comprising: signaling, in or along the bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

49. A method comprising: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining a representation for the block to be predicted based on the prediction mode; and applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

50. The method of claim 49 further comprising: deriving at least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

51. The method of the claim 50 further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

52. A method comprising: determining a block of samples to be predicted; determining a prediction mode for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on the prediction mode; blending at least the first and the second initial representations to form a combined prediction block; andapplying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image.

53. The method of claim 52 further comprising: deriving at least one parameter of the joint bilateral filter is on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

54. The method of claim 53 further comprising: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

55. The method of any of the claims 53 or 54 further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

56. A method comprising: determining a block of samples to be predicted; determining a set of prediction parameters for the block of samples; determining a representation for the block to be predicted based at least on the set of prediction parameters; and applying the joint bilateral filter to chroma component samples of the prediction block, wherein a range difference is derived from corresponding pixel of the luma component as a guide image.

57. The method of claim 56, wherein the set of prediction parameters comprises one or more of an intra prediction mode, an inter prediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

58. The method of any of the claims 56 or 57, wherein the representation comprises a sample value for each sample.

59. The method of any of the claims 55 to 58 further comprising: deriving at least one parameter of the joint bilateral filter based on:an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

60. The method of the claim 59 further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

61. A method comprising: determining a block of samples to be predicted; determining a first set of prediction parameters and a second set of prediction parameters for the block of samples; determining at least a first and a second initial representations for the block to be predicted based on the first set of prediction parameters and the second set of prediction parameters; blending at least the first and the second initial representations to form a combined prediction block; and applying a joint bilateral filter to chroma component samples of the combined prediction block, wherein the range difference is derived from corresponding pixel of the luma component as a guide image.

62. The method of claim 61, wherein the first set of prediction parameters and the second set of prediction parameters comprise one or more of an intra prediction mode, an inter prediction mode, a prediction type, a prediction direction when directional prediction is used, or a reference sample filtering mode.

63. The method of any of the claims 61 or 62, wherein the representation comprises a sample value for each sample.

64. The method of any of the claims 61 to 63 further comprising: deriving least one parameter of the joint bilateral filter based on: an input from a user; the prediction mode; a quantization parameter of the block; and the quantization parameter difference between the luma and chroma components of the block.

65. The method of claim 64 further comprising: using the at least one parameter for deciding when the joint bilateral filter is applied to the combined prediction block.

66. The method of any of the claims 64 or 65 further comprising: signaling, in or along a bitstream or receiving from or along the bitstream, the at least one parameter of the joint bilateral filter.

67. An apparatus comprising means for performing methods as claimed in any of the claims 33 to 66.

68. A computer readable medium comprising program instructions for performing methods as claimed in any of the claims 33 to 66.

69. The computer readable medium of claim 68, wherein the computer readable medium comprises a non-transitory computer readable medium.

Images