Method and device for video processing and medium

Abstract

The embodiment of the invention provides a scheme for video processing. A method for video processing is provided. The method includes performing a conversion between a current video unit of the video and a bitstream of the video, wherein the bitstream includes information of a downsampling filter for obtaining the current video unit.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to video processing techniques, and more particularly to neural network post-processing filters with down-sampling information. Background Art

[0002] Nowadays, digital video capabilities are being applied to all aspects of people's lives. Various types of video compression technologies, such as MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-TH.265 High Efficiency Video Codec (HEVC) standard, and Versatile Video Codec (VVC) standard have been proposed for video encoding / decoding. However, it is generally expected that the encoding and decoding quality of video encoding and decoding technologies will be further improved. Summary of the invention

[0003] An embodiment of the present disclosure provides a solution for video processing.

[0004] In a first aspect, a method for video processing is provided. The method comprises: performing conversion between a current video unit of a video and a bit stream of the video, wherein the bit stream comprises information for obtaining a down sampling filter of the current video unit.

[0005] According to the method of the first aspect of the present disclosure, in the bitstream, information of the downsampling filter for obtaining the current video unit is transmitted by a signal. Compared with the conventional solution in which such information is not transmitted by a signal, the proposed method can advantageously realize the optimization of the post-processing filter based on the information of the downsampling filter for quality improvement, and thus the quality of the video after post-processing can be improved.

[0006] In a second aspect, a device for video processing is provided. The device includes a processor and a non-volatile memory having instructions thereon. The instructions, when executed by the processor, cause the processor to perform the method according to the first aspect of the present disclosure.

[0007] In a third aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions, and the instructions enable a processor to execute the method according to the first aspect of the present disclosure.

[0008] In a fourth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. The method includes: converting a current video unit of the video to a bitstream, wherein the bitstream includes information for obtaining a downsampling filter for the current video unit.

[0009] In a fifth aspect, a method for storing a bitstream of a video is provided, the method comprising: performing conversion between a current video unit of the video and a bitstream, wherein the bitstream comprises information of a downsampling filter for obtaining the current video unit; and storing the bitstream in a non-transitory computer-readable recording medium.

[0010] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become more apparent through the following detailed description with reference to the accompanying drawings.In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0012] Figure 1 A block diagram illustrating an example video encoding and decoding system is shown according to some embodiments of the present disclosure;

[0013] Figure 2 A block diagram illustrating a first example video encoder is shown according to some embodiments of the present disclosure;

[0014] Figure 3 shows a block diagram illustrating an example video decoder according to some embodiments of the present disclosure;

[0015] Figure 4 is an example illustration of the brightness data channel;

[0016] Figure 5 A flowchart showing a method for video processing according to an embodiment of the present disclosure is shown; and

[0017] Figure 6 A block diagram of a computing device is shown in which various embodiments of the present disclosure may be implemented.

[0018] Same or similar reference numbers generally refer to same or similar elements throughout the drawings. DETAILED DESCRIPTION

[0019] The principle of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described only for the purpose of illustrating and helping those skilled in the art to understand and implement the present disclosure, without implying any limitation on the scope of the present disclosure. In addition to the methods described below, the disclosure described herein can also be implemented in various ways.

[0020] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0021] References in this disclosure to "one embodiment," "an embodiment," "an example embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment must include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an example embodiment, it is claimed that such feature, structure, or characteristic, whether or not explicitly described, is within the knowledge of those skilled in the art to affect correlation with other embodiments.

[0022] It should be understood that, although the terms "first" and "second" etc. may be used herein to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish one element from another element. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0023] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "one", "an" and "the" are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "include", "comprises", "has", "has", "includes" and / or "comprising" are used herein to indicate the presence of the features, elements and I or components, etc., but do not exclude the presence or addition of one or more other features, elements, components and / or their combinations. Example Environment

[0024] Figure 1 1 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of the present disclosure. As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0025] The video source 112 may include a source such as a video acquisition device. Examples of a video acquisition device include, but are not limited to, an interface for receiving video data from a video content provider, a computer graphics system for generating video data, and / or a combination thereof.

[0026] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a bit sequence that forms a coded representation of the video data. The bitstream may include coded pictures and associated data. The coded pictures are coded representations of pictures. The associated data may include sequence parameter sets, picture parameter sets, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The coded video data may be directly transmitted to the destination device 120 via the network 130A via the I / O interface 116. The coded video data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0027] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, which is configured to be connected to an external display device interface.

[0028] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other existing and / or future standards.

[0029] Figure 2 is a block diagram showing an example of a video encoder 200 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video encoder 114 in the system 100 is shown.

[0030] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. Figure 2 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0031] In some embodiments, the video encoder 200 may include a segmentation unit 201, a prediction unit 202, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a cache 213 and an entropy coding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0032] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode in which at least one reference picture is a picture in which the current video block is located.

[0033] Furthermore, although some components (such as the motion estimation unit 204 and the motion compensation unit 205) may be integrated, for the purpose of explanation, these components are described in detail below. Figure 2 are shown separately in the example.

[0034] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0035] The mode selection unit 203 may select one of a plurality of encoding modes (intra-frame encoding or inter-frame encoding), for example, based on the error result, and provide the generated intra-frame encoded block or inter-frame encoded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the encoded block for use as a reference picture. In some examples, the mode selection unit 203 may select a combined intra-frame and inter-frame prediction (CIIP) mode in which the prediction is based on an inter-frame prediction signal and an intra-frame prediction signal. In the case of inter-frame prediction, the mode selection unit 203 may also select a resolution for the motion vector (e.g., sub-pixel precision or integer pixel precision) for the block.

[0036] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the cache 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the cache 213 other than the picture associated with the current video block.

[0037] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice. As used herein, an "I slice" may refer to a portion of a picture consisting of macroblocks, all of which are based on macroblocks within the same picture. Furthermore, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of a picture consisting of macroblocks that are independent of macroblocks in the same picture.

[0038] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search the reference pictures of list 0 or list 1 to find the reference video block for the current video block. The motion estimation unit 204 may then generate a reference index and a motion vector, the reference index indicating the reference picture in list 0 or list 1 containing the reference video block, and the motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0039] Alternatively, in other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block. The motion estimation unit 204 may search the reference pictures in list 0 to find a reference video block for the current video block, and may also search the reference pictures in list 1 to find another reference video block for the current video block. The motion estimation unit 204 may then generate a plurality of reference indexes and a plurality of motion vectors, the plurality of reference indexes indicating a plurality of reference pictures in list 0 and list 1 containing a plurality of reference video blocks, and the plurality of motion vectors indicating a plurality of spatial displacements between the plurality of reference video blocks and the current video block. The motion estimation unit 204 may output the plurality of reference indexes and the plurality of motion vectors of the current video block as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the plurality of reference video blocks indicated by the motion information of the current video block.

[0040] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal motion information of a current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that motion information of a current video block is sufficiently similar to motion information of a neighboring video block.

[0041] In one example, motion estimation unit 204 may indicate a value in a syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0042] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0043] As discussed above, the video encoder 200 may signal motion vectors in a predictive manner.Two examples of prediction signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and Merge mode signaling.

[0044] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0045] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the predicted video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample components of samples in the current video block.

[0046] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0047] Transform processing unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0048] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0049] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0050] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0051] The entropy encoding unit 214 may receive data from other functional components of the video encoder 200. When the entropy encoding unit 214 receives the data, the entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.

[0052] Figure 3 is a block diagram showing an example of a video decoder 300 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video decoder 124 in the system 100 is shown.

[0053] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 3 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0054] exist Figure 3 In the example of , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally opposite to the encoding process described with respect to the video encoder 200.

[0055] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 can decode the entropy encoded video data, and the motion compensation unit 302 can determine motion information from the entropy decoded video data, the motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 can determine such information, for example, by performing AMVP and Merge mode. AMVP is used, including deriving several most likely candidates based on data and reference pictures from adjacent PBs. The motion information typically includes horizontal motion vector displacement values ​​and vertical motion vector displacement values, one or two reference picture indexes, and in the case of prediction areas in B strips, also includes an identification of which reference picture list is associated with each index. As used herein, in some aspects, "Merge mode" may refer to deriving motion information from spatial neighboring blocks or temporal neighboring blocks.

[0056] The motion compensation unit 302 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter.Identifiers for the interpolation filters used with sub-pixel precision may be included in the syntax elements.

[0057] The motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of a reference block using interpolation filters used by the video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine the interpolation filters used by the video encoder 200 based on received syntax information, and the motion compensation unit 302 may use the interpolation filters to generate a prediction block.

[0058] The motion compensation unit 302 may use at least part of the syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently of other slices of the same picture in terms of entropy coding and decoding, signal prediction, and residual signal reconstruction. A slice may be an entire picture, or it may be a region of a picture.

[0059] The intra prediction unit 303 may use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 304 inverse quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0060] The reconstruction unit 306 may obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and the buffer 307 also generates the decoded video for presentation on a display device.

[0061] Some exemplary embodiments of the present disclosure will be described in detail below. It should be noted that the section titles used in this document are for ease of understanding, and the embodiments disclosed in the section are not limited to that section. In addition, although some embodiments are described with reference to multifunctional video codecs or other specific video codecs, the disclosed technology is also applicable to other video coding and decoding technologies. In addition, although some embodiments describe the video encoding steps in detail, it should be understood that the corresponding decoding steps of de-encoding will be implemented by the decoder. In addition, the term video processing includes video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another compression format or at different compression bit rates. 1. Brief Overview The present disclosure relates to image / video coding and decoding technology. Specifically, it relates to downsampling filter information for a neural network post-processing filter that is added to a video bitstream and transmitted by a signal. The downsampling information herein may be information describing a neural network-based downsampling model and corresponding parameters. The downsampling information herein may also be information describing a downsampling model that is not based on a neural network (e.g., a bilinear filter or a bicubic filter) and corresponding parameters. The concept can be applied individually or in various combinations to video bitstreams encoded and decoded by any codec (e.g., the Versatile Video Codec (VVC) standard and / or the Versatile SEI Message (VSEI) standard for encoded and decoded video bitstreams). 2. Abbreviations APS Adaptive Parameter Set AU Access Unit CLVS Codec layer video sequence CLVSS Codec layer video sequence starts CRC Cyclic Redundancy Check CVS Codec Video Sequence FIR Finite Impulse Response IRAP Intra-frame random access point NAL Network Abstraction Layer PPS Picture Parameter Set PU Picture Unit RASL Random Access Skip Preamble SEI Supplemental Enhancement Information STSA Stepped Temporal Sublayer Access VCL video codec layer VSEI Versatile Supplementary Enhancement Information (Recommendation ITU-T H.274 | ISO / IEC 23002-7) VUI Video Availability Information VVC Versatile Video Codec (Recommendation ITU-TH.266 | ISO / IEC 23090-3) 3. Introduction 3.1. Video Codec Standards Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Vision, and the two organizations jointly developed H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure, which utilizes temporal prediction plus transform codec. In order to explore future video codec technologies after HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and put them into a reference software called the Joint Exploration Model (JEM). When the Versatile Video Codec (VVC) project officially started, JVET was later renamed the Joint Video Experts Group (JVET). VVC is a new codec standard that aims to reduce bit rate by 50% compared to HEVC. The standard has been finalized by JVET at the 19th meeting that ended on July 1, 2020. The Versatile Video Coding (VVC) standard (ITU-T H.266 | ISO / IEC23090-3) and the associated Versatile Supplementary Enhancement Information (VSEI) standard for coded video bitstreams (ITU-T H.274 | ISO / IEC23002-7) have been designed for the widest range of applications, including traditional uses such as television broadcasting, video conferencing, or playback from storage media, as well as newer and more advanced use cases such as adaptive bitrate streaming, video region extraction, composition and merging of content from multiple coded video bitstreams, multi-view video, scalable layered codecs, and viewport-adaptive 360° immersive media. The Essential Video Codec (EVC) standard (ISO / IEC 23094-1) is another video codec standard that has been recently developed by MPEG. 3.2. General SEI messages and SEI messages in VVC and VSEI SEI messages assist processes related to decoding, display, or other purposes. However, no SEI messages are required for constructing luma or chroma samples through the decoding process. A compliant decoder does not need to process this information for output order consistency. Some SEI messages are required for checking bitstream consistency and output timing decoder consistency. Other SEI messages are not required for checking bitstream consistency. Annex D of VVC specifies syntax and semantics for SEI message payloads for some SEI messages, and specifies the use of SEI messages and VUI parameters for which syntax and semantics are specified in ITU-T H.274 | ISO / IEC 23002-7. 3.3. Signaling of neural network post-processing filters The existing design includes the specification of two SEI messages for signaling of neural network post-processing filters, as follows. 4. Question The current design of SEI message for Neural Network Post-Processing Filter Characterization (NNPFC) has the following problems: 1) In the preprocessing stage, downsampling is a common processing step used in image / video encoding. If a post-processing filter is trained based on a specific downsampling filter, a reconstructed image / video with higher quality can be achieved. In one example, a quality improvement of the reconstructed image / video is desired before upsampling, and only an NNPFC SEI message specifying an NN upsampling filter is available. In this example, the neural network model for quality improvement can be first fine-tuned or optimized based on the corresponding downsampling model, and then the quality-improved model is applied to the reconstructed content. This can achieve higher quality for the reconstructed image / video compared to using a general model for quality improvement. Therefore, it helps if information of the downsampling filter is available for the post-processing filter. 5. Detailed solution In order to solve the above problems, the method summarized as follows is disclosed. The solution should be considered as an example to explain the general concept and should not be interpreted in a narrow way. In addition, these solutions can be applied individually or in combination in any way. In the following description, the term picture can be replaced by any video unit (such as a slice). The term "continuous video units in output / decoding order" can be replaced by "a group of video units with the same parameters (e.g., picture level quantization parameter, picture type, temporal layer id) in output order". 1) To solve problem 1, one or more of the following syntax elements may be signaled in a video message: a. In one example, the video message may be a SEI or a VUI message. b. In one example, downsampling_filter_type may be signaled to indicate the downsampling method used before encoding the associated video unit (eg, picture / slice / block). c. In one example, nnpfc_downsampling_id indicates an identification number (index) for a NN-based downsampling filter. d. In one example, if downsampling_filter_type is equal to a first value, such as 0, it indicates that a neural network based downsampling filter is used to downsample the sequence. i. In one example, nnpfc_downsampling_mode_idc is defined to specify how to interpret the manner in which the neural network based downsampling filter is determined. For example, 1.nnpfc_downsampling_mode_idc equal to 0 specifies that the post-processing filter associated with the nnpfc_downsampling_id value is determined by external means. 2. nnpfc_downsampling_mode_idc equal to 1 specifies that the post-processing filter associated with the nnpfc_downsampling_id value is a neural network represented by the ISO / IEC 15938-17 bitstream contained in this SEI message. 3. nnpfc_downsampling_mode_idc equal to 2 specifies that the post-processing filter associated with the nnpfc_downsampling_id value is a neural network identified by the specified tag uniform resource identifier (URI) (nnpfc_uri_tag[i]) and the neural network information URI (nnpfc_uri[i]). ii. In one example, the following constraints must be applied for the nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples syntaxes: 1. The specified nnpfc_pic_width_in_luma_samples must be consistent with the width of luma samples after applying the post-processing filter identified by nnpfc_id. 2. The specified nnpfc_pic_height_in_luma_samples must be consistent with the height of luma samples after applying the post-processing filter identified by nnpfc_id. e. In one example, if downsampling_filter_type is equal to a second value, such as 1, it indicates that the downsampling filter is a bilinear filter. i. In one example, nnpfc_downsampling_mode_idc is defined to specify the downsampling filter. 1.nnpfc_downsampling_mode_idc equal to 0 specifies that the post-processing filter associated with the nnpfc_downsampling_id value is determined by external means. 2. nnpfc_downsampling_mode_idc equal to 3 specifies that the post-processing filter associated with the nnpfc_downsampling_id value is determined by the filter information nnpfc_downsampling_filter_info[i]. f. In one example, if downsampling_filter_type is equal to a third value, such as 2, it indicates that the downsampling filter is a bicubic filter. i. In one example, nnpfc_downsampling_mode_idc is defined to specify the downsampling filter. 1.nnpfc_downsampling_idc equal to 0 specifies that the post-processing filter associated with the nnpfc_downsampling_id value is determined by external means. 2. nnpfc_downsampling_mode_idc equal to 3 specifies that the post-processing filter associated with the nnpfc_downsampling_id value is determined by the filter information nnpfc_downsampling_filter_info[i]. g. In one example, if downsampling_filter_type is equal to a fourth value, such as 3, it indicates that the downsampling filter is a RPR (reference picture resampling) filter. i. In one example, nnpfc_downsampling_mode_idc is defined to specify the downsampling filter. 1.nnpfc_downsampling_idc equal to 0 specifies that the post-processing filter associated with the nnpfc_downsampling_id value is determined by external means. 2. nnpfc_downsampling_mode_idc equal to 3 specifies that the post-processing filter associated with the nnpfc_downsampling_id value is determined by the filter information nnpfc_downsampling_filter_info[i]. 2) In one example, the following syntax element indicating the complexity of the neural network based downsampling filter may be signaled in the video message: a. In one example, the video message may be a SEI or a VUI message. b. In one example, nnpfc_downsampling_idc is defined to specify a neural network based downsampling filter. c. In one example, nnpfc_downsampling_complexity_idc is defined. i. In one example, nnpfc_downsampling_complexity_idc equal to 0 specifies that there is no syntax element indicating the complexity of the downsampling filter. ii. In one example, nnpfc_downsampling_complexity_idc equal to 1 specifies that there is a syntax element indicating the complexity of the downsampling filter. d. In one example, when nnpfc_downsampling_complexity_idc is equal to 1, the following syntax elements are defined: i.nnpfc_downsampling_parameter_type_idc indicates the type of parameter. 1. In one example, nnpfc_parameter_type_idc equal to 0 indicates that the neural network uses only integer parameters. 2. In one example, nnpfc_parameter_type_idc equal to 1 indicates that the neural network uses only floating point. 3. In one example, nnpfc_parameter_type_idc equal to 2 indicates that the neural network uses only binary parameters. ii.nnpfc_downsampling_log2_parameter_bit_length_minus3 indicates the bit length of the neural network parameter. 1. In one example, nnpfc_downsampling_log2_parameter_bit_length_minus3 is equal to 0, 1, 2, and 3 to indicate that the neural network does not use parameters with bit lengths greater than 8, 16, 32, and 64, respectively. iii.nnpfc_downsampling_num_parameters_idc indicates the maximum number of neural network parameters used for the post-processing filter. iv.nnpfc_downsampling_num_kmac_operations_idc indicates the maximum number of multiplication-accumulation operations per sample point of the downsampling filter. 3) Downsampling is performed at all video unit (e.g., sequence / picture / slice / slice / brick / sub-picture / CTU / CTU row / one or more CUs or CTUs / CTB) level. a. In one example, a downsampling filter is performed for the frame level with the size of its original resolution. 4) In one example, multiple downsampling filters may be applied for different video units. a. In one example, one downsampling filter is applied for all color components and pictures. Therefore, only one nnpfc_downsampling_id is signaled in the SEI message. b. In one example, multiple downsampling filters are applied for different color components. i. In one example, multiple nnpfc_downsampling_ids are signaled in the order of Y, U, and V. ii. In another example, nnpfc_downsampling_id_y, nnpfc_downsampling_id_u, and nnpfc_downsampling_id_v are transmitted through signals separately. c. In one example, multiple downsampling filters are applied for different picture types. i. In one example, multiple nnpfc_downsampling_ids are signaled in the order of I slice, P slice, and B slice. ii. In another example, nnpfc_downsampling_id_I, nnpfc_downsampling_id_P, and nnpfc_downsampling_id_B are transmitted through signals separately. 5) In one example, nnpfc_scaling_factor_width and nnpfc_scaling_factor_height are signaled, which indicate the scaling factor of the upsampling model identified by nnpfc_id. a. In one example, nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples are used in the syntax: i. In one example, nnpfc_scaling_factor_width and nnpfc_scaling_factor_height should be consistent with nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples specified in the SEI message. 1. In one example, nnpfc_scaling_factor_width*width_input is equal to nnpfc_pic_width_in_luma_samples, where width_input represents the width of the image / video before upsampling. 2. In one example, nnpfc_scaling_factor_height*height_input is equal to nnpfc_pic_height_in_luma_samples, where height_input represents the height of the image / video before upsampling. ii. In one example, if nnpfc_scaling_factor_width and nnpfc_scaling_factor_height are not consistent with nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples specified in the SEI message: 1. In one example, to achieve the same resolution as nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples, multiple upsampling using the upsampling model identified by nnpfc_id may be performed. 2. In one example, to achieve the same resolution as nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples, a bilinear or bicubic filter may be applied after / before applying the upsampling model identified by nnpfc_id. b. In one example, nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples are not used in the SEI message. i. In one example, the width and height of the luma sample array of a picture generated by applying the post-processing filter identified by nnpfc_id may be derived according to nnpfc_scaling_factor_width, nnpfc_scaling_factor_height, the corresponding width of the image / video before upsampling, and the corresponding height of the image / video before upsampling. ii. In one example, the width and height of the luma sample array of the picture generated by applying the post-processing filter identified by nnpfc_id are respectively signaled. 1. In one example, luma sample arrays of a picture generated by applying a post-processing filter identified by nnpfc_id are respectively signaled. 2. In one example, nnpfc_scaled_pic_width_in_luma_samples is signaled, which specifies the width of the luma sample array of the picture produced by applying the post-processing filter identified by nnpfc_id to the cropped decoded output picture. nnpfc_scaled_pic_width_in_luma_samples is equal to nnpfc_scaling_factor_width*width_input, where width_input represents the width of the image / video before upsampling. 3. In one example, nnpfc_scaled_pic_height_in_luma_samples is signaled, which specifies the height of the luma sample array of the picture produced by applying the post-processing filter identified by nnpfc_id to the cropped decoded output picture. nnpfc_scaled_pic_height_in_luma_samples is equal to nnpfc_scaling_factor_height*height_input, where width_input represents the width of the image / video before upsampling. 6. Examples Following are some example embodiments for aspects of the solution outlined above in Section V. Most relevant parts that were added or modified were used Underline Indicates that some of the deleted parts are There may be some other changes that are editorial in nature, and therefore not highlighted. 6.1. Example 1 This embodiment is for solution items 1 to 4 and all their sub-items outlined above in Section 5. 6.1.1 Neural Network Post-Processing Filter Characteristics SEI Message Syntax 6.1.2 Neural Network Post-Processing Filter Characteristics SEI Message Semantics This SEI message specifies a neural network that can be used as a post-processing filter. The use of the specified post-processing filter for the specified picture is indicated using a neural network post-processing filter activation SEI message. nnpfc_purpose indicates the purpose of the post-processing filter as specified in Table 2. The value of nnpfc_purpose must be between 0 and 2. 32 nnpfc_purpose not appearing in Table 2 is reserved for future specification by ITU-T|ISO / IEC and MUST NOT be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this specification MUST ignore SEI messages containing reserved values ​​for nnpfc_purpose. Table 2 - Definition of nnpfc_purpose NOTE 1—When the reserved value of nnpfc_purpose is used by ITU-T | ISO / IEC in the future, the syntax of this SEI message may be extended with multiple syntax elements whose presence depends on nnpfc_purpose being equal to the reserved value. When SubWidthC is equal to 1 and SubHeightC is equal to 1, nnpfc_purpose must not be equal to 2 or 4. nnpfc_out_sub_c_flag equal to 1 specifies that outSubWidthC is equal to 1 and outSubHeightC is equal to 1. nnpfc_out_sub_c_flag equal to 0 specifies that outSubWidthC is equal to 2 and outSubHeightC is equal to 1. When nnpfc_out_sub_c_flag is not present, outSubWidthC is inferred to be equal to SubWidthC and outSubHeightC is inferred to be equal to SubHeightC. If SubWidthC is equal to 2 and SubHeightC is equal to 1, then nnpfc_out_sub_c_flag shall not be equal to 0. nnpfc_scaling_factor_width specifies the width of the post-processing filter identified by nnpfc_id. Scaling factor for the width. nnpfc_scaling_factor_height specifies the height of the post-processing filter identified by nnpfc_id. The scaling factor of the height. nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples specify the width and height, respectively, of the luma sample array of the picture produced by applying the post-processing filter identified by nnpfc_id to the cropped decoded output picture. When nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples are not present, they are presumed to be equal to CroppedWidth and CroppedHeight, respectively. nnpfc_pic_width_in_luma_samples is equal to nn p fc_scaling_factor_width*width_ input, where width_input represents the width of the image / video before upsampling. nnpfc_pic_height_in_luma_samples is equal to nn p fc_scaling_ f actor_height* height_in p ut, where width_input represents the width of the image / video before upsampling. nnpfc_component_last_flag equal to 1 specifies that the last dimension of the input tensor inputTensor and the output tensor outputTensor from the post-processing filter is used for channels. nnpfc_component_last_flag equal to 0 specifies that the second dimension of the input tensor inputTensor and the output tensor outputTensor from the post-processing filter is used for channels. NOTE 2 - The first dimension in the input and output tensors is used for batch indexing, which is a practice in some neural network frameworks. Although the semantics of this SEI message uses a batch size equal to 1, it is up to the post-processing implementation to determine the batch size used as input for neural network inference. NOTE 3—A color component is an example of a channel. nnpfc downsampling filter equal to 0 specifies that a neural network-based downsampling filter is used for nnpfc_downsampling_filter is equal to 1 to specify that a bilinear filter is used to downsample the sequence. Downsampling. nnpfc_downsampling_filter equal to 2 specifies that a bicubic filter is used to downsample the sequence. nnpfc_downsampling_id contains an identification number that can be used to identify the post-processing filter. value must Between 0 and 2 32 -2, including the boundary value. nnpfc_downsampling_mode_idc equal to 0 specifies the post-processing filter associated with the nnpfc_id value. Determined by external means not specified in this specification. nnpfc_mode_idc equal to 1 specifies that the post-processing filter associated with the nnpfc_id value is the one contained in this Neural network represented by ISO / IEC 15938-17 bitstream in SEI message. nnpfc_mode_idc equal to 2 specifies that the post-processing filter associated with the nnpfc_id value is the one specified by the label The neural network information URI (nnpfc_uri_tag[i]) and the neural network information URI (nnpfc_uri[i]) identify the Via the network. nnpfc_mode_idc equal to 3 specifies that the post-processing filter associated with the nnpfc_id value is bilinear or bitriangular The downsampling filter is determined by nnpfc_downsampling_filter_parameter_size and nnpfc_downsampling_ fi lter_info [i] Logo. nnpfc_reserved_zero_bit must be equal to 0. nnpfc_uri_tag[i] contains a NULL-terminated UTF-8 string specifying the tag URI. The UTF-8 string contains a URI, with syntax and semantics as specified in IETF RFC4151, that uniquely identifies the format and associated information about the neural network being used as the post-processing filter specified by the nnrpf_uri[i] value. NOTE 4 - The nnrpf_uri_tag[i] element represents a 'tag' URI that allows the format of the neural network data specified by the nnrpf_uri[i] value to be uniquely identified without the need for a central registration authority. nnpfc_uri[i] contains a NULL-terminated UTF-8 string as specified in IETF Internet Standard 63. The UTF-8 string contains a URI, with syntax and semantics as specified in IETF Internet Standard 66, identifying neural network information (e.g., data representation) to be used as a post-processing filter. nnpfc_payload_byte[i] contains the i-th byte of a bitstream conforming to ISO / IEC 15938-17. The byte sequence nnpfc_payload_byte[i] for all present values ​​of i must be a complete bitstream conforming to ISO / IEC 15938-17. nnnfc_downsampling_filter_parameter_size specifies the size of each parameter of the post-processing filter. byte . nnpfc_downsampling_filter_info [ i] contains the i-th byte of the bitstream, which contains the following Parameters of the sampling filter. nnpfc_downsampling_parameter_type_idc is equal to 0 to indicate a neural network based downsampling filter. npfc_parameter_type_flag equal to 1 indicates that the neural network can use only floating point. nnp f c_parameter_ t ype_idc equal to 2 indicates that the neural network uses only binary parameters. npfc_parameter_ type_idc equal to 3 is reserved for future specifications. nnpfc_downsampling_log2_parameter_bit_length_minus3 equals 0, 1, 2, and 3 respectively. Indicates that the neural network does not use parameters with bit lengths greater than 8, 16, 32, and 64. When nnpfc_log2_parameter_bit_length_minus3 is not present, the neural network does not use bits greater than 1 Length parameter. nnpfc_downsampling_num_parameters_idc indicates the number of parameters used for the neural network for the post-processing filter. The maximum number of network parameters, in powers of 2048. nnpfc_num_parameters_idc is equal to 0 to indicate that the neural network parameters are The maximum number of num_parameters_idc values ​​is unspecified. The value of nnpfc_num_parameters_idc must be in the range 0 to 52 (inclusive). Values ​​of nnpfc_num_parameters_idc greater than 52 are reserved for future specification by ITU-T | ISO / IEC, and MUST NOT be present in a bitstream conforming to this version of this specification. Decoders conforming to this version of this specification MUST ignore the packet Contains nnp fc _num_parameters_idc is a reserved value for SEI messages. If nnp fc If the value of _num_parameters_idc is greater than zero, the variable maxNumParameters is derived as Down: maxNumParameters=(2048< <nnpfc_num_parameters_idc)-1 Bitstream consistency requires that the number of neural network parameters of the post-processing filter must be less than or equal to maxNumParameters. nnpfc_num_kmac_operations_idc is greater than 0 and specifies the number of multiplication-accumulation operations per sample of the post-processing filter. The maximum number of operations is less than or equal to nnpfc_num_kmac_operations_idc*1000. operations_idc equal to 0 specifies that the maximum number of multiply-accumulate operations for the network is unspecified. _ num_kmac_ operations_idc must have a value between 0 and 2. 32 -1 range (including the boundary value).

[0062] More details of embodiments of the present disclosure related to a neural network post-processing filter with down-sampling information will be described below. The embodiments of the present disclosure should be considered as examples to explain general concepts and should not be interpreted in a narrow manner. In addition, these embodiments can be applied individually or in combination in any manner.

[0063] As used herein, the term "video unit" may refer to a color component, a sub-picture, a picture, a slice, a codec tree unit (CTU), a CTU row, a CTU group, a codec unit (CU), a prediction unit (PU), a transform unit (TU), a codec tree block (CTB), a codec block (CB), a prediction block (PB), a transform block (TB), a sub-block of a video block, a sub-region within a video block, a video processing unit including a plurality of samples / pixels, etc. A video unit may be rectangular or non-rectangular. In addition, the terms "post-processing filter" and "post-filter" may be used interchangeably.

[0064] Figure 5 FIG. 5 is a flow chart of a method 500 for video processing according to some embodiments of the present disclosure. Figure 5 As shown, at 502, conversion between a current video unit of a video and a bitstream of the video is performed. In some embodiments, the conversion may include encoding the current video unit into the bitstream. Alternatively or additionally, the conversion may include decoding the current video unit from the bitstream.

[0065] In some embodiments, the bitstream includes information of a downsampling filter for obtaining the current video unit. That is, in the preprocessing stage, the downsampling process using the downsampling filter can be applied to the video unit of the original video to obtain the current video unit to be encoded into the bitstream. In one example, the information of the downsampling filter can be included in a supplemental enhancement information (SEI) message. Alternatively, the information of the downsampling filter can be included in a video usability information (VUI) message in the bitstream. With the help of this information of the downsampling filter, the post-processing filter for quality improvement can be first fine-tuned or optimized based on the corresponding downsampling filter information. Then, for the purpose of quality improvement, the post-processing filter can be applied to the reconstructed video.

[0066] In view of the above, information for obtaining a downsampling filter for a current video unit is transmitted by signal in a bitstream. Compared with a conventional solution in which such information is not transmitted by signal, the proposed method can advantageously implement optimization of a post-processing filter for quality improvement based on the information of the downsampling filter, and thus the quality of the video after post-processing can be improved.

[0067] In some embodiments, the information may include a first indication indicating a type of a downsampling filter. By way of example and not limitation, the first indication may be a syntax element downsampling_filter_type. In addition, the information may include a second indication indicating an identification number or index of the downsampling filter. By way of example and not limitation, the second indication may be a syntax element nnpfc_downsampling_id.

[0068] In some embodiments, the first indication equal to the first value (e.g., 0, etc.) may indicate that the downsampling filter is a neural network-based downsampling filter. The first indication equal to the second value (e.g., 1, etc.) may indicate that the downsampling filter is a bilinear filter. The first indication equal to the third value (e.g., 2, etc.) may indicate that the downsampling filter is a bicubic filter, or the first indication equal to the fourth value (e.g., 3, etc.) may indicate that the downsampling filter is a reference picture resampling (RPR) filter. It should be understood that the above description is described for description purposes only. The scope of the present disclosure is not limited in this regard.

[0069] In some embodiments, the information may include a third indication indicating how to determine the downsampling filter. As an example and not limitation, the third indication may be a syntax element nnpfc_downsampling_mode_idc.

[0070] In some embodiments, the third indication being equal to the fifth value (e.g., 0, etc.) may indicate that the downsampling filter is determined based on a predefined scheme. The third indication being equal to the sixth value (e.g., 1, etc.) may indicate that the downsampling filter includes a neural network obtained from a bitstream. For example, the neural network may be represented by an ISO / IEC15938-17 bitstream included in this SEI message. The third indication being equal to the seventh value (e.g., 2, etc.) may indicate that the downsampling filter includes a neural network obtained based on one or more indications in the bitstream. For example, the one or more indications may include at least one of a specified tag uniform resource identifier (URI) (such as nnpfc_uri_tag[i]) or a neural network information URI (such as nnpfc_uri[i]). The third indication being equal to the eighth value (e.g., 3, etc.) may indicate that a parameter set of the downsampling filter is obtained from the bitstream. For example, the parameter set may be indicated by a fourth indication in the bitstream. The fourth indication may be a syntax element nnpfc_downSampling_filter_info. It should be understood that the above description is described for descriptive purposes only. The scope of the present disclosure is not limited in this regard.

[0071] In some embodiments, if the first indication can be equal to the first value, the third indication can be allowed to be equal to one of the fifth value, the sixth value, or the seventh value. Additionally or alternatively, if the first indication can be equal to one of the second value, the third value, or the fourth value, the third indication can be allowed to be equal to one of the fifth value or the eighth value.

[0072] In some embodiments, the value of the first syntax element must be consistent with the width of the luma sample after applying the post-processing filter identified by the second syntax element. The first syntax element indicates the width of the luma sample array of the video unit generated by applying the post-processing filter identified by the second syntax element to the decoded current video unit. The second syntax element indicates an identification number for identifying the post-processing filter, and the value of the third syntax element is consistent with the height of the luma sample after applying the post-processing filter identified by the second syntax element. In addition, the third syntax element indicates the height of the luma sample array of the video unit generated by applying the post-processing filter identified by the second syntax element to the decoded current video unit. By way of example and not limitation, the first syntax element may be the syntax element nnpfc_pic_width_in_luma_samples, the second syntax element may be the syntax element nnpfc_id, and the third syntax element may be the syntax element nnpfc_pic_height_in_luma_samples.

[0073] In some embodiments, the downsampling filter may be a neural network-based downsampling filter. In this case, the bitstream may include a fifth indication for indicating whether at least one indication for indicating the complexity of the neural network-based downsampling filter is present in the bitstream. As an example and not limitation, the fifth indication may be a syntax element nnpfc_downsampling_complexity_idc. For example, the fifth indication being equal to a ninth value (e.g., 0, etc.) indicates that at least one indication is not present in the bitstream. The fifth indication being equal to a tenth value (e.g., 1, etc.) indicates that at least one indication is present in the bitstream. It should be understood that the above description is described for descriptive purposes only. The scope of the present disclosure is not limited in this regard.

[0074] In some embodiments, the information may include a sixth indication for specifying a neural network based downsampling filter. As an example and not limitation, the sixth indication may be a syntax element nnpfc_downsampling_idc.

[0075] In some additional embodiments, at least one indication may include a seventh indication for indicating the type of the parameter of the neural network-based downsampling filter. As an example and not limitation, the seventh indication may be a syntax element nnpfc_downsampling_parameter_type_idc. For example, the seventh indication equal to the eleventh value (e.g., 0, etc.) indicates that the type of the parameter is an integer. The seventh indication equal to the twelfth value (e.g., 1, etc.) indicates that the type of the parameter is a floating point, or the seventh indication equal to the thirteenth value (e.g., 2, etc.) indicates that the type of the parameter is binary. It should be understood that the above description is described for descriptive purposes only. The scope of the present disclosure is not limited in this regard.

[0076] In some embodiments, at least one indication may include an eighth indication indicating a bit length of a parameter of a neural network-based downsampling filter. As an example and not limitation, the eighth indication may be a syntax element nnpfc_downsampling_log2_parameter_bit_length_minus3. For example, the eighth indication equals a fourteenth value (e.g., 0, etc.) indicating that the bit length of the parameter is less than or equal to 8. The eighth indication equals a fifteenth value (e.g., 1, etc.) indicating that the bit length of the parameter may be less than or equal to 16. The eighth indication equals a sixteenth value (e.g., 2, etc.) indicating that the bit length of the parameter is less than or equal to 32. The eighth indication equals a seventeenth value (e.g., 3, etc.) indicating that the bit length of the parameter is less than or equal to 64. It should be understood that the above description is described for descriptive purposes only. The scope of the present disclosure is not limited in this regard.

[0077] In some additional or alternative embodiments, the at least one indication may include a ninth indication for indicating a maximum number of parameters for the neural network based downsampling filter. As an example and not limitation, the ninth indication may be a syntax element nnpfc_downsampling_num_parameters_idc. Additionally or alternatively, the at least one indication may include a tenth indication for indicating a maximum number of multiplication-accumulation operations per sample point for the neural network based downsampling filter. For example, the tenth indication may be a syntax element nnpfc_downsampling_num_kmac_operations_idc.

[0078] In some embodiments, the downsampling process may be performed at one of the following levels: sequence level, frame level, picture level, slice level, tile level, sub-picture level, codec transform unit (CTU) level, CTU row level, one or more codec unit (CU) level, one or more CTU level, or one or more codec transform block (CTB) level. For example, the downsampling process may be performed for all video units of a particular level. Alternatively, the downsampling process may be performed only for a portion of the video units of a particular level. As an example, the downsampling process may be performed at a frame level for a size of the original resolution of a frame of the video.

[0079] In some embodiments, the downsampling filter may be included in a plurality of downsampling filters, and the plurality of downsampling filters may be applied to different video units of a video.

[0080] In some embodiments, the downsampling filter may be applied to all color components and pictures of the video, and only one second indication may be indicated in the bitstream (e.g., only one syntax element nnpfc_downsampling_id). In some alternative embodiments, the downsampling filter may be included in multiple downsampling filters, and multiple downsampling filters may be applied to different color components of the video. For example, different color components may include a first color component, a second color component, and a third color component. As an example, the first color component may be a color component Y, the second color component may be a color component U, and the third color component may be a color component V. It should be understood that the above examples are described for descriptive purposes only. The scope of the present disclosure is not limited in this regard.

[0081] In some embodiments, the plurality of eleventh indications may be signaled in the order of the first color component, the second color component, and the third color component. The eleventh indication is used to indicate an identification number or index of a downsampling filter in the plurality of downsampling filters. As an example and not limitation, the eleventh indication may be a syntax element nnpfc_downsampling_id.

[0082] Alternatively, the twelfth indication, the thirteenth indication and the fourteenth indication are indicated in the bitstream. The twelfth indication is used to indicate the identification number or index of one of the multiple downsampling filters applied to the first color component. The thirteenth indication is used to indicate the identification number or index of one of the multiple downsampling filters applied to the second color component, and the fourteenth indication is used to indicate the identification number or index of one of the multiple downsampling filters applied to the third color component. As an example and not limitation, the twelfth indication may be a syntax element nnpfc_downsampling_id_y, the thirteenth indication may be a syntax element nnpfc_downsampling_id_u, and the fourteenth indication may be a syntax element nnpfc_downsampling_id_v. It should be understood that the above description is described for description purposes only. The scope of the present disclosure is not limited in this regard.

[0083] In some embodiments, multiple downsampling filters may be applied to different picture types of a video. For example, the different picture types may include a first picture type, a second picture type, and a third picture type. For example, the first picture type may be an I slice, the second picture type may be a P slice, and the third picture type may be a B slice. It should be understood that the above examples are described for illustrative purposes only. The scope of the present disclosure is not limited in this regard.

[0084] In some embodiments, the plurality of fifteenth indications may be signaled in the order of the first picture type, the second picture type, and the third picture type. The fifteenth indication is used to indicate an identification number or index of a downsampling filter in the plurality of downsampling filters. As an example and not limitation, the fifteenth indication may be a syntax element nnpfc_downsampling_id.

[0085] Alternatively, the sixteenth indication, the seventeenth indication, and the eighteenth indication are indicated in the bitstream. The sixteenth indication is used to indicate an identification number or index of one of the multiple downsampling filters applied to the first picture type. The seventeenth indication is used to indicate an identification number or index of one of the multiple downsampling filters applied to the second picture type. The eighteenth indication is used to indicate an identification number or index of one of the multiple downsampling filters applied to the third picture type. For example, the sixteenth indication may be a syntax element nnpfc_downSampling_id_I, the seventeenth indication may be a syntax element nnpfc_downsampling_id_P, and the eighteenth indication may be a syntax element nnpfc_downsampling_id_B. It should be understood that the above examples are described for description purposes only. The scope of the present disclosure is not limited in this regard.

[0086] In some embodiments, the bitstream may further include a nineteenth indication and a twentieth indication. The nineteenth indication is used to indicate a width scaling factor of a post-processing filter for processing reconstruction of the current video unit, and the twentieth indication is used to indicate a length scaling factor of the post-processing filter. In one example, the post-processing filter may be identified by a syntax element nnpfc_id. As an example and not limitation, the nineteenth indication may be a syntax element nnpfc_scaling_factor_width, and the twentieth indication may be a syntax element nnpfc_scaling_factor_height.

[0087] In some embodiments, the bitstream may further include a twenty-first indication and a twenty-second indication. The twenty-first indication is used to indicate the width of the luma sample array of the video unit generated by applying the post-processing filter to the decoded current video unit, and the twenty-second indication is used to indicate the height of the luma sample array of the video unit generated by applying the post-processing filter to the decoded current video unit. As an example and not limitation, the twenty-first indication may be a syntax element nnpfc_pic_width_in_luma_samples, and the twenty-second indication may be a syntax element nnpfc_pic_height_in_luma_samples.

[0088] In some embodiments, the value of the 19th indication may be consistent with the value of the 21st indication. For example, the product of the value of the 19th indication and the width of the current video unit may be equal to the value of the 21st indication. Additionally or alternatively, the value of the 20th indication may be consistent with the value of the 22nd indication. For example, the product of the value of the 20th indication and the height of the current video unit may be equal to the value of the 22nd indication.

[0089] In some embodiments, if the value of the 19th indication is inconsistent with the value of the 21st indication and / or the value of the 20th indication is inconsistent with the value of the 22nd indication, multiple upsampling processes may be performed on the reconstruction of the current video unit using a post-processing filter to achieve the same resolution as specified by the 21st indication and the 22nd indication. Alternatively, a bilinear filter or a bicubic filter may be combined with the post-processing filter and applied to the reconstruction of the current video unit to achieve the same resolution as specified by the 21st indication and the 22nd indication.

[0090] In some embodiments, the twenty-first indication and the twenty-second indication may not be present in the bitstream. In this case, the width of the luma sample array may be determined based on the value of the nineteenth indication and the width of the current video unit. Furthermore, the height of the luma sample array may be determined based on the value of the twentieth indication and the height of the current video unit. Additionally or alternatively, the luma sample array may be indicated in the bitstream.

[0091] In some embodiments, the width of the luma sample array and the height of the luma sample array may be indicated in the bitstream. In one example, the width of the luma sample array may be indicated by a twenty-third indication. By way of example and not limitation, the twenty-third indication may be a syntax element nnpfc_scaled_pic_width_in_luma_samples. The value of the twenty-third indication may be equal to the product of the value of the nineteenth indication and the width of the current video unit. In an additional or alternative example, the height of the luma sample array may be indicated by a twenty-fourth indication. By way of example and not limitation, the twenty-fourth indication may be a syntax element nnpfc_scaled_pic_height_in-uma_samples. The value of the twenty-fourth indication may be equal to the product of the value of the twentieth indication and the height of the current video unit.

[0092] It should be understood that any of the above indications may also be a grammatical element represented by any other suitable string different from the above name. The scope of the present disclosure is not limited in this respect.

[0093] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. In the method, a conversion between a current video unit of the video and the bitstream is performed. The bitstream includes information for obtaining a downsampling filter for the current video unit.

[0094] According to some other embodiments of the present disclosure, a method for storing a bitstream of a video is provided. According to the method, a conversion between a current video unit of the video and a bitstream is performed. The bitstream includes information of a downsampling filter for obtaining the current video unit. Moreover, the bitstream is stored in a non-transitory computer-readable recording medium.

[0095] Embodiments of the present disclosure may be described according to the following items, features of which may be combined in any reasonable way.

[0096] Item 1. A method for video processing, comprising: performing a conversion between a current video unit of a video and a bitstream of the video, wherein the bitstream includes information for obtaining a downsampling filter for the current video unit.

[0097] Item 2. A method according to item 1, wherein the information is included in a supplemental enhancement information (SEI) message or a video usability information (VUI) message in the bitstream.

[0098] Item 3. A method according to Item 1, wherein the information comprises a first indication for indicating a type of downsampling filter.

[0099] Clause 4. The method of clause 3, wherein the first indication comprises a syntax element downsampling_filter_type.

[0100] Clause 5. A method according to any one of clauses 1 to 4, wherein the information comprises a second indication indicating an identification number or index of a downsampling filter.

[0101] Clause 6. The method of clause 5, wherein the second indication comprises a syntax element nnpfc_downsampling_id.

[0102] Item 7. A method according to any one of Items 3 to 6, wherein the first indication is equal to the first value indicating that the downsampling filter is a neural network-based downsampling filter, or the first indication is equal to the second value indicating that the downsampling filter is a bilinear filter, or the first indication is equal to the third value indicating that the downsampling filter is a bicubic filter, or the first indication is equal to the fourth value indicating that the downsampling filter is a reference picture resampling (RPR) filter.

[0103] Item 8. A method according to Item 7, wherein the information includes a third indication for indicating how to determine the downsampling filter.

[0104] Clause 9. The method of clause 8, wherein the third indication comprises a syntax element nnpfc_downsampling_mode_idc.

[0105] Item 10. A method according to any one of Items 8 or 9, wherein the third indication is equal to the fifth value indicating that the downsampling filter is determined based on a predefined scheme, or the third indication is equal to the sixth value indicating that the downsampling filter includes a neural network obtained from a bitstream, or the third indication is equal to the seventh value indicating that the downsampling filter includes a neural network obtained based on one or more indications in the bitstream, or the third indication is equal to the eighth value indicating that a parameter set of the downsampling filter is obtained from the bitstream.

[0106] Item 11. A method according to Item 10, wherein the one or more instructions include at least one of a specified tag uniform resource identifier (URI) or a neural network information URI.

[0107] Clause 12. A method according to any of clauses 10 or 11, wherein the parameter set is indicated by a fourth indication in the bitstream.

[0108] Clause 13. The method of clause 12, wherein the fourth indication comprises a syntax element nnpfc_downsampling_filter_info.

[0109] Item 14. A method according to any one of Items 10 to 13, wherein if the first indication is equal to the first value, the third indication is allowed to be equal to one of the fifth value, the sixth value or the seventh value, or if the first indication is equal to one of the second value, the third value or the fourth value, the third indication is allowed to be equal to one of the fifth value or the eighth value.

[0110] Item 15. A method according to any one of Items 1 to 14, wherein the value of the first syntax element is consistent with the width of the luma samples after applying the post-processing filter identified by the second syntax element, the first syntax element indicates the width of the luma sample array of the video unit generated by applying the post-processing filter identified by the second syntax element to the decoded current video unit, and the second syntax element indicates an identification number used to identify the post-processing filter, and the value of the third syntax element is consistent with the height of the luma samples after applying the post-processing filter identified by the second syntax element, the third syntax element indicates the height of the luma sample array of the video unit generated by applying the post-processing filter identified by the second syntax element to the decoded current video unit.

[0111] Item 16. The method of Item 15, wherein the first syntax element comprises a syntax element nnpfc_pic_width_in_luma_samples, the second syntax element comprises a syntax element nnpfc_id, and the third syntax element comprises a syntax element nnpfc_pic_height_in_luma_samples.

[0112] Item 17. A method according to any one of Items 1 to 16, wherein the downsampling filter is a neural network based downsampling filter and the bitstream includes a fifth indication for indicating whether at least one indication for indicating the complexity of the neural network based downsampling filter is present in the bitstream.

[0113] Clause 18. The method of clause 17, wherein the fifth indication comprises a syntax element nnpfc_downsampling_complexity_idc.

[0114] Item 19. A method according to any one of Items 17 to 18, wherein the fifth indication being equal to the ninth value indicates that at least one indication is not present in the bitstream, or the fifth indication being equal to the tenth value indicates that at least one indication is present in the bitstream.

[0115] Item 20. A method according to any one of Items 1 to 19, wherein the information includes a sixth indication for specifying a neural network based downsampling filter.

[0116] Clause 21. The method of clause 20, wherein the sixth indication comprises a syntax element nnpfc_downsampling_idc.

[0117] Item 22. A method according to any one of Items 17 to 21, wherein the at least one indication comprises a seventh indication for indicating a type of parameters of the neural network based downsampling filter.

[0118] Clause 23. The method of clause 22, wherein the seventh indication comprises a syntax element nnpfc_downsampling_parameter_type_idc.

[0119] Item 24. A method according to any one of Items 22 to 23, wherein the seventh indicator is equal to the eleventh value indicating that the type of the parameter is integer, or the seventh indicator is equal to the twelfth value indicating that the type of the parameter is floating point, or the seventh indicator is equal to the thirteenth value indicating that the type of the parameter is binary.

[0120] Item 25. A method according to any one of Items 17 to 25, wherein at least one indication comprises a bit length eighth indication for indicating parameters of a neural network based downsampling filter.

[0121] Item 26. The method of Item 25, wherein the eighth indication comprises a syntax element nnpfc_downsampling_log2_parameter_bit_length_minus3.

[0122] Item 27. A method according to any one of Items 25 to 26, wherein the eighth indication is equal to the bit length of the fourteenth value indicating parameter is less than or equal to 8, or the eighth indication is equal to the bit length of the fifteenth value indicating parameter is less than or equal to 16; or, the eighth indication is equal to the bit length of the sixteenth value indicating parameter is less than or equal to 32; or, the eighth indication is equal to the bit length of the seventeenth value indicating parameter is less than or equal to 64.

[0123] Item 28. A method according to any one of Items 17 to 27, wherein the at least one indication comprises a ninth indication for indicating a maximum number of parameters of the neural network based downsampling filter.

[0124] Item 29. The method of Item 28, wherein the ninth indication comprises a syntax element nnpfc_downsampling_num_parameters_idc.

[0125] Item 30. A method according to any one of Items 17 to 29, wherein the at least one indication includes a tenth indication for indicating a maximum number of multiply-accumulate operations per sample point of the neural network based downsampling filter.

[0126] Clause 31. The method of clause 30, wherein the tenth indication comprises a syntax element nnpfc_downsampling_num_kmac_operations_idc.

[0127] Item 32. A method according to any one of items 1 to 31, wherein the downsampling process is performed at one of the following levels: sequence level, frame level, picture level, slice level, slice level, brick level, sub-picture level, codec transform unit (CTU) level, CTU row level, one or more codec unit (CU) level, one or more CTU level, or one or more codec transform block (CTB) level.

[0128] Item 33. A method according to Item 32, wherein the downsampling process is performed at a frame level at a size of an original resolution for the frames of the video.

[0129] Item 34. A method according to any of Items 5 to 33, wherein the downsampling filter is applied to all color components and pictures of the video and only one second indication is indicated in the bitstream.

[0130] Item 35. A method according to any one of Items 1 to 33, wherein the downsampling filter is included in a plurality of downsampling filters, and the plurality of downsampling filters are applied to different video units of the video.

[0131] Item 36. A method according to any one of Items 1 to 33, wherein the downsampling filter is included in a plurality of downsampling filters, and the plurality of downsampling filters are applied to different color components of the video.

[0132] Item 37. A method according to Item 36, wherein the different color components include a first color component, a second color component, and a third color component, a plurality of eleventh indications are transmitted via a signal in the order of the first color component, the second color component, and the third color component, and the eleventh indication is used to indicate an identification number or index of a downsampling filter among a plurality of downsampling filters.

[0133] Clause 38. The method of clause 37, wherein the eleventh indication comprises a syntax element nnpfc_downsampling_id.

[0134] Item 39. A method according to Item 36, wherein the different color components include a first color component, a second color component, and a third color component, a twelfth indication, a thirteenth indication, and a fourteenth indication are indicated in a bitstream, the twelfth indication is used to indicate an identification number or index of one of a plurality of downsampling filters applied to the first color component, the thirteenth indication is used to indicate an identification number or index of one of a plurality of downsampling filters applied to the second color component, and the fourteenth indication is used to indicate an identification number or index of one of a plurality of downsampling filters applied to the third color component.

[0135] Clause 40. The method of clause 39, wherein the twelfth indication comprises the syntax element nnpfc_downsampling_id_y, the thirteenth indication comprises the syntax element nnpfc_downsampling_id_u, and the fourteenth indication comprises the syntax element nnpfc_downsampling_id_v.

[0136] Item 41. A method according to any one of Items 37 to 40, wherein the first color component is color component Y, the second color component is color component U, and the third color component is color component V.

[0137] Item 42. A method according to any one of Items 1 to 41, wherein the downsampling filter is included in a plurality of downsampling filters, and the plurality of downsampling filters are applied to different picture types of the video.

[0138] Item 43. A method according to Item 42, wherein the different picture types include a first picture type, a second picture type, and a third picture type, a plurality of fifteenth indications are transmitted by a signal in the order of the first picture type, the second picture type, and the third picture type, and the fifteenth indication is used to indicate an identification number or index of a downsampling filter among a plurality of downsampling filters.

[0139] Item 44. The method of Item 43, wherein the fifteenth indication comprises a syntax element nnpfc_downsampling_id.

[0140] Item 45. A method according to Item 42, wherein the different picture types include a first picture type, a second picture type, and a third picture type, a sixteenth indication, a seventeenth indication, and an eighteenth indication are indicated in a bitstream, the sixteenth indication is used to indicate an identification number or index of one of a plurality of downsampling filters applied to the first picture type, the seventeenth indication is used to indicate an identification number or index of one of a plurality of downsampling filters applied to the second picture type, and the eighteenth indication is used to indicate an identification number or index of one of a plurality of downsampling filters applied to the third picture type.

[0141] Item 46. The method of Item 45, wherein the sixteenth indication comprises the syntax element nnpfc_downsampling_id_I, the seventeenth indication comprises the syntax element nnpfc_downsampling_id_P, and the eighteenth indication comprises the syntax element nnpfc_downsampling_id_B.

[0142] Item 47. A method according to any one of items 43 to 46, wherein the first picture type is an I slice, the second picture type is a P slice, and the third picture type is a B slice.

[0143] Item 48. A method according to any one of Items 1 to 47, wherein the bitstream further includes a nineteenth indication and a twentieth indication, the nineteenth indication being used to indicate a width scaling factor of a post-processing filter used to process reconstruction of the current video unit, and the twentieth indication being used to indicate a length scaling factor of the post-processing filter.

[0144] Item 49. The method of Item 48, wherein the nineteenth indication comprises a syntax element nnpfc_scaling_factor_width and the twentieth indication comprises a syntax element nnpfc_scaling_factor_height.

[0145] Clause 50. A method according to any of Clauses 48 to 49, wherein the post-processing filter is identified by a syntax element nnpfc_id.

[0146] Item 51. A method according to any one of Items 48 to 50, wherein the bitstream further includes a twenty-first indication and a twenty-second indication, the twenty-first indication being used to indicate the width of the luma sample array of the video unit generated by applying the post-processing filter to the decoded current video unit, and the twenty-second indication being used to indicate the height of the luma sample array of the video unit generated by applying the post-processing filter to the decoded current video unit.

[0147] Item 52. The method of Item 51, wherein the twenty-first indication comprises a syntax element nnpfc_pic_width_in_luma_samples, and the twenty-second indication comprises a syntax element nnpfc_pic_height_in_luma_samples.

[0148] Item 53. The method according to any one of Items 51 to 52, wherein the nineteenth indicated value is consistent with the twenty-first indicated value, and the twentieth indicated value is consistent with the twenty-second indicated value.

[0149] Item 54. The method of Item 53, wherein the product of the value of the nineteenth indication and the width of the current video unit is equal to the value of the twenty-first indication.

[0150] Item 55. A method according to any one of Items 53 to 54, wherein the product of the value of the twentieth indication and the height of the current video unit is equal to the value of the twenty-second indication.

[0151] Item 56. A method according to any one of items 51 to 52, wherein if the value indicated by the nineteenth is inconsistent with the value indicated by the twenty-first and the value indicated by the twentieth is inconsistent with the value indicated by the twenty-second, performing multiple upsampling processes on the reconstruction of the current video unit by using a post-processing filter.

[0152] Item 57. A method according to any one of items 51 to 52, wherein if the value indicated by the nineteenth is inconsistent with the value indicated by the twenty-first and the value indicated by the twentieth is inconsistent with the value indicated by the twenty-second, a bilinear filter or a bicubic filter is combined with a post-processing filter and applied to the reconstruction of the current video unit.

[0153] Item 58. A method according to any one of Items 48 to 50, wherein a twenty-first indication and a twenty-second indication are not present in the bitstream, the twenty-first indication being used to indicate the width of the luma sample array of the video unit generated by applying a post-processing filter to the decoded current video unit, and the twenty-second indication being used to indicate the height of the luma sample array of the video unit generated by applying the post-processing filter to the decoded current video unit.

[0154] Item 59. The method of item 58, wherein the twenty-first indication comprises a syntax element nnpfc_pic_width_in_luma_samples, and the twenty-second indication comprises a syntax element nnpfc_pic_height_in_luma_samples.

[0155] Item 60. A method according to any one of Items 58 to 59, wherein a width of the luma sample array is determined based on a value of the nineteenth indication and a width of the current video unit, and a height of the luma sample array is determined based on a value of the twentieth indication and a height of the current video unit.

[0156] Item 61. A method according to any one of Items 58 to 60, wherein the array of luma samples is indicated in the bitstream.

[0157] Item 62. The method of any one of Items 58 to 61, wherein the width of the luma sample array and the height of the luma sample array are indicated in the bitstream.

[0158] Item 63. The method of Item 62, wherein a width of the array of luma samples is indicated by a twenty-third indication.

[0159] Item 64. The method of item 63, wherein the twenty-third indication comprises a syntax element nnpfc_scaled_pic_width_in_luma_samples.

[0160] Item 65. A method according to any one of Items 63 to 64, wherein the value of the twenty-third indication is equal to the product of the value of the nineteenth indication and the width of the current video unit.

[0161] Item 66. The method of any one of Items 62 to 65, wherein a height of the array of luma samples is indicated by a twenty-fourth indication.

[0162] Item 67. The method of item 66, wherein the twenty-fourth indication comprises a syntax element nnpfc_scaled_pic_height_in_luma_samples.

[0163] Item 68. A method according to any one of Items 66 to 67, wherein the value of the twenty-fourth indication is equal to the product of the value of the twentieth indication and the height of the current video unit.

[0164] Item 69. A method according to any of Items 1 to 68, wherein the conversion includes encoding the current video unit into a bitstream.

[0165] Item 70. A method according to any one of items 1 to 68, wherein converting comprises decoding the current video unit from a bitstream.

[0166] Item 71. An apparatus for video processing, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of Items 1 to 70.

[0167] Item 72. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform a method according to any one of Items 1 to 70.

[0168] Item 73. A non-transitory computer-readable recording medium storing a bitstream of a video, the bitstream being generated by a method executed by an apparatus for video processing, the method comprising: performing a conversion between a current video unit of the video and the bitstream, the bitstream comprising information for obtaining a downsampling filter for the current video unit.

[0169] Item 74. A method for storing a bitstream of a video, comprising: performing conversion between a current video unit of the video and a bitstream, the bitstream including information for obtaining a downsampling filter for the current video unit; and storing the bitstream in a non-transitory computer-readable recording medium. Example Device

[0170] Figure 6 A block diagram of a computing device 600 in which various embodiments of the present disclosure may be implemented is shown. The computing device 600 may be implemented as a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300), or may be included in a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300).

[0171] It should be understood that Figure 6 The computing device 600 shown in FIG. 6 is for illustrative purposes only and is not intended to in any way imply any limitation on the functionality and scope of the embodiments of the present disclosure.

[0172] like Figure 6 As shown, computing device 600 comprises a general computing device 600. Computing device 600 may include at least one or more processors or processing units 610, memory 620, storage unit 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660.

[0173] In some embodiments, the computing device 600 can be implemented as any user terminal or server terminal with computing power. The server terminal can be a server, a large computing device, etc. provided by a service provider. The user terminal can be, for example, any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet computer, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof, including the accessories and peripherals of these devices, or any combination thereof. It is conceivable that the computing device 600 can support any type of interface to the user (such as a "wearable" circuit device, etc.).

[0174] The processing unit 610 may be a physical processor or a virtual processor and may implement various processes based on a program stored in the memory 620. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to increase the parallel processing capability of the computing device 600. The processing unit 610 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0175] The computing device 600 typically includes various computer storage media. Such media can be any media accessible by the computing device 600, including but not limited to volatile media and non-volatile media, or removable media and non-removable media. The memory 620 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (such as a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM) or flash memory) or any combination thereof. The storage unit 630 can be any removable or non-removable medium, and can include machine-readable media, such as a memory, a flash drive, a disk, or other media that can be used to store information and / or data and can be accessed in the computing device 600.

[0176] The computing device 600 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Figure 6 Although not shown in the figure, a disk drive for reading from and / or writing to a removable nonvolatile disk and an optical drive for reading from and / or writing to a removable nonvolatile optical disk may be provided. In this case, each drive may be connected to the bus (not shown) via one or more data medium interfaces.

[0177] The communication unit 640 communicates with another computing device via a communication medium. In addition, the functions of the components in the computing device 600 can be implemented by a single computing cluster or multiple computing machines, which can communicate via a communication connection. Therefore, the computing device 600 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general network nodes.

[0178] The input device 650 may be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. The output device 660 may be one or more of various output devices, such as a display, a speaker, a printer, etc. With the aid of the communication unit 640, the computing device 600 may also communicate with one or more external devices (not shown), such as storage devices and display devices, and the computing device 600 may also communicate with one or more devices that enable a user to interact with the computing device 600, or, if necessary, the computing device 600 may also communicate with any device (e.g., a network card, a modem, etc.) that enables the computing device 600 to communicate with one or more other computing devices. Such communication may be performed via an input / output (I / O) interface (not shown).

[0179] In some embodiments, some or all components of the computing device 600 may also be arranged in a cloud computing architecture rather than being integrated in a single device. In a cloud computing architecture, components may be provided remotely and work together to implement the functions described in the present disclosure. In some embodiments, cloud computing provides computing, software, data access and storage services, which will not require the end user to know the physical location or configuration of the system or hardware that provides these services. In various embodiments, cloud computing provides services via a wide area network (such as the Internet) using a suitable protocol. For example, a cloud computing provider provides an application via a wide area network, which can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture and the corresponding data may be stored on a server at a remote location. The computing resources in a cloud computing environment may be merged or distributed at the location of a remote data center. Cloud computing infrastructure can provide services through a shared data center, although they appear as a single access point to the user. Therefore, the cloud computing architecture may be used to provide the components and functions described herein from a service provider at a remote location. Alternatively, the components and functions described herein may be provided by a conventional server, or may be installed on a client device directly or otherwise.

[0180] In an embodiment of the present disclosure, the computing device 600 may be used to implement video encoding / decoding. The memory 620 may include one or more video encoding / decoding modules 625 having one or more program instructions. These modules are accessible and executable by the processing unit 610 to perform the functions of the various embodiments described herein.

[0181] In an example embodiment performing video encoding, input device 650 may receive video data as input 670 to be encoded. The video data may be processed, for example, by video codec module 625 to generate an encoded bitstream. The encoded bitstream may be provided as output 680 via output device 660.

[0182] In an example embodiment performing video decoding, input device 650 may receive an encoded bitstream as input 670. The encoded bitstream may be processed, for example, by video codec module 625 to generate decoded video data. The decoded video data may be provided as output 680 via output device 660.

[0183] Although the present disclosure has been specifically shown and described with reference to the preferred embodiments of the present disclosure, it will be appreciated by those skilled in the art that various changes may be made in form and detail without departing from the spirit and scope of the present application as defined by the appended claims. These modifications are intended to be encompassed by the scope of the present application. Therefore, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. A method for video processing, comprising: A conversion between a current video unit of a video and a bitstream of the video is performed, wherein the bitstream includes information of a downsampling filter for obtaining the current video unit.

2. The method of claim 1, wherein the information is included in a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) message in the bitstream. The method according to claim 1 , wherein the information comprises a first indication for indicating a type of the down-sampling filter. The method according to claim 3 , wherein the first indication comprises a syntax element downsampling_filter_type. 5 . The method according to claim 1 , wherein the information comprises a second indication for indicating an identification number or index of the down-sampling filter. The method of claim 5 , wherein the second indication comprises a syntax element nnpfc_downsampling_id.

7. The method according to any one of claims 3 to 6, wherein the first indication is equal to a first value indicating that the downsampling filter is a neural network based downsampling filter, or The first indication being equal to the second value indicates that the downsampling filter is a bilinear filter, or The first indication being equal to the third value indicates that the downsampling filter is a bicubic filter, or The first indication being equal to the fourth value indicates that the downsampling filter is a reference picture resampling (RPR) filter. The method according to claim 7 , wherein the information comprises a third indication for indicating how to determine the down-sampling filter. 9 . The method of claim 8 , wherein the third indication comprises a syntax element nnpfc_downsampling_mode_idc.

10. The method according to any one of claims 8 or 9, wherein the third indication is equal to the fifth value indicating that the downsampling filter is determined based on a predefined scheme, or The third indication being equal to the sixth value indicates that the downsampling filter comprises a neural network obtained from the bitstream, or The third indication being equal to the seventh value indicates that the downsampling filter comprises a neural network obtained based on one or more indications in the bitstream, or, The third indication being equal to the eighth value indicates that the parameter set of the downsampling filter is obtained from the bitstream.

11. The method of claim 10, wherein the one or more indications include at least one of a specified tag uniform resource identifier (URI) or a neural network information URI.

12. The method according to any one of claims 10 or 11, wherein the parameter set is indicated by a fourth indication in the bitstream. The method of claim 12 , wherein the fourth indication comprises a syntax element nnpfc_downsampling_filter_info.

14. The method according to any one of claims 10 to 13, wherein if the first indication is equal to the first value, the third indication is allowed to be equal to one of the fifth value, the sixth value or the seventh value, or If the first indication is equal to one of the second value, the third value, or the fourth value, the third indication is allowed to be equal to one of the fifth value or the eighth value.

15. The method according to any one of claims 1 to 14, wherein a value of a first syntax element is consistent with a width of luma samples after applying a post-processing filter identified by a second syntax element, the first syntax element indicating a width of a luma sample array of a video unit generated by applying the post-processing filter identified by the second syntax element to a decoded current video unit, and the second syntax element indicating an identification number for identifying the post-processing filter, and A value of a third syntax element is consistent with the height of luma samples after applying the post-processing filter identified by the second syntax element, the third syntax element indicating the height of the luma sample array of the video unit produced by applying the post-processing filter identified by the second syntax element to the decoded current video unit. 16 . The method of claim 15 , wherein the first syntax element comprises a syntax element nnpfc_pic_width_in_luma_samples, the second syntax element comprises a syntax element nnpfc_id, and the third syntax element comprises a syntax element nnpfc_pic_height_in_luma_samples.

17. The method according to any one of claims 1 to 16, wherein the downsampling filter is a neural network based downsampling filter, and the bitstream comprises a fifth indication for indicating whether at least one indication for indicating the complexity of the neural network based downsampling filter is present in the bitstream. The method of claim 17 , wherein the fifth indication comprises a syntax element nnpfc_downsampling_complexity_idc.

19. The method according to any one of claims 17 to 18, wherein the fifth indication being equal to the ninth value indicates that the at least one indication is not present in the bitstream, or The fifth indication being equal to the tenth value indicates that the at least one indication is present in the bitstream.

20. The method according to any one of claims 1 to 19, wherein the information comprises a sixth indication for specifying the neural network based downsampling filter. The method of claim 20 , wherein the sixth indication comprises a syntax element nnpfc_downsampling_idc.

22. The method according to any one of claims 17 to 21, wherein the at least one indication comprises a seventh indication for indicating a type of a parameter of the neural network based downsampling filter.

23. The method of claim 22, wherein the seventh indication comprises a syntax element nnpfc_downsampling_parameter_type_idc.

24. A method according to any one of claims 22 to 23, wherein the seventh indication is equal to an eleventh value indicating that the type of the parameter is an integer, or The seventh indication is equal to the twelfth value indicating that the type of the parameter is floating point, or The seventh indication being equal to the thirteenth value indicates that the type of the parameter is binary.

25. The method according to any one of claims 17 to 25, wherein the at least one indication comprises a bit length eighth indication for indicating parameters of the neural network based downsampling filter. 26 . The method of claim 25 , wherein the eighth indication comprises a syntax element nnpfc_downsampling_log2_parameter_bit_length_minus3.

27. A method according to any one of claims 25 to 26, wherein the eighth indication is equal to the fourteenth numerical value indicating that the bit length of the parameter is less than or equal to 8, or The eighth indication being equal to the fifteenth value indicates that the bit length of the parameter is less than or equal to 16; or, The eighth indication being equal to the sixteenth value indicates that the bit length of the parameter is less than or equal to 32; or, The eighth indication being equal to the seventeenth value indicates that the bit length of the parameter is less than or equal to 64.

28. The method according to any one of claims 17 to 27, wherein the at least one indication comprises a ninth indication for indicating a maximum number of parameters of the neural network based downsampling filter.

29. The method of claim 28, wherein the ninth indication comprises a syntax element nnpfc_downsampling_num_parameters_idc.

30. The method of any one of claims 17 to 29, wherein the at least one indication comprises a tenth indication for indicating a maximum number of multiply-accumulate operations per sample of the neural network based downsampling filter.

31. The method of claim 30, wherein the tenth indication comprises a syntax element nnpfc_downsampling_num_kmac_operations_idc.

32. The method according to any one of claims 1 to 31, wherein the downsampling process is performed at one of the following levels: Sequence level, Frame level, Picture level, Stripe level, Film level, Brick level, Sub-picture level, Codec Transform Unit (CTU) level, CTU row level, The level of one or more codec units (CUs), The level of one or more CTUs, or A level of one or more codec transform blocks (CTBs).

33. The method of claim 32, wherein the downsampling process is performed at the frame level at a size of an original resolution for frames of the video.

34. The method of any one of claims 5 to 33, wherein the down-sampling filter is applied to all color components and pictures of the video and only one second indication is indicated in the bitstream.

35. The method of any one of claims 1 to 33, wherein the down-sampling filter is included in a plurality of down-sampling filters, and the plurality of down-sampling filters are applied to different video units of the video.

36. The method of any one of claims 1 to 33, wherein the down-sampling filter is included in a plurality of down-sampling filters, and the plurality of down-sampling filters are applied to different color components of the video.

37. A method according to claim 36, wherein the different color components include a first color component, a second color component and a third color component, a plurality of eleventh indications are transmitted by a signal in the order of the first color component, the second color component and the third color component, and the eleventh indication is used to indicate an identification number or index of a down-sampling filter among the plurality of down-sampling filters.

38. The method of claim 37, wherein the eleventh indication comprises a syntax element nnpfc_downsampling_id.

39. A method according to claim 36, wherein the different color components include a first color component, a second color component and a third color component, a twelfth indication, a thirteenth indication and a fourteenth indication are indicated in the bitstream, the twelfth indication is used to indicate an identification number or index of one of the multiple downsampling filters applied to the first color component, the thirteenth indication is used to indicate an identification number or index of one of the multiple downsampling filters applied to the second color component, and the fourteenth indication is used to indicate an identification number or index of one of the multiple downsampling filters applied to the third color component.

40. The method of claim 39, wherein the twelfth indication comprises a syntax element nnpfc_downsampling_id_y, the thirteenth indication comprises a syntax element nnpfc_downsampling_id_u, and the fourteenth indication comprises a syntax element nnpfc_downsampling_id_v.

41. The method of any one of claims 37 to 40, wherein the first color component is color component Y, the second color component is color component U, and the third color component is color component V.

42. The method of any one of claims 1 to 41, wherein the down-sampling filter is included in a plurality of down-sampling filters, and the plurality of down-sampling filters are applied to different picture types of the video.

43. The method of claim 42, wherein the different picture types include a first picture type, a second picture type, and a third picture type, a plurality of fifteenth indications are transmitted by a signal in the order of the first picture type, the second picture type, and the third picture type, and the fifteenth indication is used to indicate an identification number or index of one of the plurality of down-sampling filters.

44. The method of claim 43, wherein the fifteenth indication comprises a syntax element nnpfc_downsampling_id.

45. The method according to claim 42, wherein the different picture types include a first picture type, a second picture type, and a third picture type, and a sixteenth indication, a seventeenth indication, and an eighteenth indication are indicated in the bitstream, the sixteenth indication is used to indicate an identification number or index of one of the multiple down-sampling filters applied to the first picture type, the seventeenth indication is used to indicate an identification number or index of one of the multiple down-sampling filters applied to the second picture type, and the eighteenth indication is used to indicate an identification number or index of one of the multiple down-sampling filters applied to the third picture type.

46. ​​The method of claim 45, wherein the sixteenth indication comprises a syntax element nnpfc_downsampling_id_I, the seventeenth indication comprises a syntax element nnpfc_downsampling_id_P, and the eighteenth indication comprises a syntax element nnpfc_downsampling_id_B.

47. The method according to any one of claims 43 to 46, wherein the first picture type is an I slice, the second picture type is a P slice, and the third picture type is a B slice.

48. A method according to any one of claims 1 to 47, wherein the bitstream further includes a nineteenth indication and a twentieth indication, the nineteenth indication being used to indicate a width scaling factor of a post-processing filter used to process the reconstruction of the current video unit, and the twenty-third indication being used to indicate a length scaling factor of the post-processing filter.

49. The method of claim 48, wherein the nineteenth indication comprises a syntax element nnpfc_scaling_factor_width and the twentieth indication comprises a syntax element nnpfc_scaling_factor_height.

50. The method according to any one of claims 48 to 49, wherein the post-processing filter is identified by a syntax element nnpfc_id.

51. The method according to any one of claims 48 to 50, wherein the bitstream further includes a twenty-first indication and a twenty-second indication, the twenty-first indication being used to indicate a width of a luma sample array of a video unit generated by applying the post-processing filter to the decoded current video unit, and the twenty-second indication being used to indicate a height of the luma sample array of the video unit generated by applying the post-processing filter to the decoded current video unit.

52. The method of claim 51, wherein the twenty-first indication comprises a syntax element nnpfc_pic_width_in_luma_samples and the twenty-second indication comprises a syntax element nnpfc_pic_height_in_luma_samples.

53. The method of any one of claims 51 to 52, wherein the value of the nineteenth indication coincides with the value of the twenty-first indication, and the value of the twenty-th indication coincides with the value of the twenty-second indication.

54. The method of claim 53, wherein the product of the value of the nineteenth indication and a width of the current video unit is equal to the value of the twenty-first indication.

55. The method of any one of claims 53 to 54, wherein the product of the value of the twentieth indication and the height of the current video unit is equal to the value of the twenty-second indication.

56. A method according to any one of claims 51 to 52, wherein if the value of the nineteenth indication is inconsistent with the value of the twenty-first indication and the value of the twenty-second indication is inconsistent with the value of the twenty-second indication, multiple upsampling processes are performed on the reconstruction of the current video unit by using the post-processing filter.

57. A method according to any one of claims 51 to 52, wherein if the value of the nineteenth indication is inconsistent with the value of the twenty-first indication and the value of the twenty-th indication is inconsistent with the value of the twenty-second indication, a bilinear filter or a bicubic filter is combined with the post-processing filter and applied to the reconstruction of the current video unit.

58. The method according to any one of claims 48 to 50, wherein a twenty-first indication and a twenty-second indication are not present in the bitstream, the twenty-first indication being used to indicate a width of a luma sample array of a video unit generated by applying the post-processing filter to a decoded current video unit, and the twenty-second indication being used to indicate a height of the luma sample array of the video unit generated by applying the post-processing filter to the decoded current video unit.

59. The method of claim 58, wherein the twenty-first indication comprises a syntax element nnpfc_pic_width_in_luma_samples and the twenty-second indication comprises a syntax element nnpfc_pic_height_in_luma_samples.

60. The method of any one of claims 58 to 59, wherein the width of the luma sample array is determined based on the value of the nineteenth indication and a width of the current video unit, and the height of the luma sample array is determined based on the value of the twentieth indication and a height of the current video unit.

61. The method of any one of claims 58 to 60, wherein the array of luma samples is indicated in the bitstream.

62. The method of any one of claims 58 to 61, wherein the width of the luma sample array and the height of the luma sample array are indicated in the bitstream.

63. The method of claim 62, wherein the width of the array of luma samples is indicated by a twenty-third indication.

64. The method of claim 63, wherein the twenty-third indication comprises a syntax element nnpfc_scaled_pic_width_in_luma_samples.

65. The method of any one of claims 63 to 64, wherein the value of the twenty-third indication is equal to the product of the value of the nineteenth indication and the width of the current video unit.

66. The method of any one of claims 62 to 65, wherein the height of the luma sample array is indicated by a twenty-fourth indication.

67. The method of claim 66, wherein the twenty-fourth indication comprises a syntax element nnpfc_scaled_pic_height_in_luma_samples.

68. The method of any one of claims 66 to 67, wherein the value of the twenty-fourth indication is equal to the product of the value of the twentieth indication and the height of the current video unit.

69. The method of any one of claims 1 to 68, wherein the converting comprises encoding the current video unit into the bitstream.

70. The method of any one of claims 1 to 68, wherein the converting comprises decoding the current video unit from the bitstream.

71. An apparatus for video processing, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of claims 1 to 70.

72. A non-transitory computer readable storage medium storing instructions that cause a processor to perform the method according to any one of claims 1 to 70.

73. A non-transitory computer-readable recording medium storing a bit stream of a video, the bit stream being generated by a method performed by an apparatus for video processing, the method comprising: A conversion is performed between a current video unit of the video and the bitstream, the bitstream including information for obtaining a downsampling filter for the current video unit.

74. A method for storing a bitstream of a video, comprising: performing conversion between a current video unit of the video and the bitstream, the bitstream comprising information for obtaining a downsampling filter for the current video unit; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images