Conditional bilateral filter for intra prediction

Abstract

Various implementations described herein include methods and systems for coding a video. In one aspect, the method includes receiving a video bitstream including a current picture, the current picture including a plurality of blocks. The plurality of blocks includes a current block. The method includes performing intra prediction on the current block of the plurality of coded blocks. The method includes applying a bilateral filter to the current block. The method also includes reconstructing the current block based on the intra prediction and an output of the bilateral filter.

Description

Related applications

[0001] This application is a continuation of U.S. Patent Application No. 19 / 035,679, filed January 23, 2025, entitled “Conditional Bilateral Filter for Intra Prediction,” which claims priority to U.S. Provisional Patent Application No. 63 / 625,260, filed January 25, 2024, entitled “Conditional Bilateral Filter for Intra Prediction,” each of which is incorporated herein by reference in its entirety. Technical Field

[0002] The disclosed implementations generally relate to video encoding and decoding, including but not limited to systems and methods for applying bilateral filtering to intra-frame prediction. Background Technology

[0003] Digital video is supported by various electronic devices such as digital televisions, laptop or desktop computers, tablet computers, digital camera devices, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, video streaming devices, etc. Electronic devices send and receive or otherwise transmit digital video data across communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video encoding can be used to compress video data according to one or more video encoding standards before transmission or storage. Video encoding can be performed by hardware and / or software on servers or electronic / client devices providing cloud services.

[0004] Video coding typically uses prediction methods that leverage the inherent redundancy in video data (e.g., inter-frame prediction, intra-frame prediction, etc.). Video coding aims to compress video data into a form using a lower bitrate while avoiding or minimizing video quality degradation. Several video codec standards have been developed. For example, High-Efficiency Video Coding (HEVC / H.265) is a video compression standard designed as part of the MPEG-H (Moving Picture Experts Group-H) project. The ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) and ISO / IEC (International Organization for Standardization / International Electrotechnical Commission) released the HEVC / H.265 standard in 2013 (Revision 1), 2014 (Revision 2), 2015 (Revision 3), and 2016 (Revision 4). Versatile Video Coding (VVC / H.266) is a video compression standard designed as a successor to HEVC. The ITU-T and ISO / IEC released the VVC / H.266 standard in 2020 (Revision 1) and 2022 (Revision 2). AOMedia Video 1 (Alliance for Open MediaVideo 1, AV1) is an open video coding format designed as an alternative to HEVC. On January 8, 2019, a validation version 1.0.0 with errata table 1 was released. Enhanced Compression Model (ECM) is a video coding standard currently under development. ECM aims to significantly improve compression efficiency beyond existing standards such as HEVC / H.265 and VVC, fundamentally enabling higher quality video at lower bitrates. Summary of the Invention

[0005] Among other things, this disclosure describes a set of methods for video (image) compression, and more specifically, relates to conditionally applying bilateral filtering to intra-frame prediction. In intra-frame prediction, reference samples from neighboring reconstructed regions can be used directly as predictors, or they can be filtered, for example, using one or more predefined filters. Bilateral filters are nonlinear, edge-preserving, and noise-reducing smoothing filters for images. Bilateral filtering involves replacing the intensity of a corresponding pixel with a weighted average of the intensity values ​​from nearby pixels (e.g., based on a distribution such as a Gaussian distribution). Applying filters in this way can improve video quality (e.g., removing and / or reducing artifacts in the reconstructed video). Furthermore, selectively applying filters where they are most beneficial (e.g., in a specific intra-frame mode) can improve coding efficiency. Some aspects of this disclosure describe the conditional use of bilateral filters for intra-frame prediction to further improve compression efficiency.

[0006] According to some embodiments, a method for video decoding is provided. The method includes: (i) receiving a video bitstream (e.g., an encoded video sequence) including a current image, the current image including a plurality of blocks, the plurality of blocks including a current block; (ii) performing intra-frame prediction on the current block among the plurality of encoded blocks; (iii) applying a bilateral filter to the current block; and (iv) reconstructing the current block based on the output of the intra-frame prediction and the bilateral filter.

[0007] According to some embodiments, a video encoding method is provided. The method includes: (i) receiving video data (e.g., a source video sequence) including a current image, the current image including a plurality of blocks, the plurality of blocks including the current block; (ii) encoding the current block by applying intra-frame prediction to the current block; (iii) signaling the current block in the video bitstream; and (iv) signaling in the video bitstream whether a bilateral filter is applied to the current block.

[0008] According to some embodiments, a video encoding method is provided. The method includes: (i) receiving video data including a current image, the current image comprising a plurality of blocks, the plurality of blocks including the current block; and (ii) encoding the current block by applying intra-frame prediction to the current block. The video bitstream includes encoded information of the current block and an indicator indicating whether a bilateral filter is applied to the encoded information of the current block.

[0009] According to some implementations, a method for processing visual media data includes: (i) obtaining a source video sequence comprising multiple frames; and (ii) performing a conversion between the source video sequence and a video bitstream of the visual media data according to a format rule, wherein (a) the video bitstream includes a current block; and (b) the format rule specifies: (1) performing intra-frame prediction on the current block; (2) applying a bilateral filter to the current block; and (3) reconstructing the current block based on the output of the intra-frame prediction and the bilateral filter.

[0010] According to some embodiments, a computing system is provided, such as a streaming system, a server system, a personal computer system, or other electronic device. The computing system includes a control circuitry system and a memory storing one or more sets of instructions. The one or more sets of instructions include instructions for performing any of the methods described herein. In some embodiments, the computing system includes an encoder component and a decoder component (e.g., a transcoder).

[0011] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more sets of instructions for execution by a computing system. The one or more sets of instructions include instructions for performing any of the methods described herein.

[0012] Therefore, methods, apparatus, and systems for encoding and decoding video are disclosed. Such methods, apparatus, and systems can supplement or replace conventional methods, apparatus, and systems for video encoding / decoding.

[0013] The features and advantages described in this specification are not necessarily all included, and in particular, some additional features and advantages will be apparent to those skilled in the art in light of the accompanying drawings, specification, and claims provided in this disclosure. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes and is not necessarily intended to depict or limit the subject matter described herein. Attached Figure Description

[0014] To provide a more detailed understanding of this disclosure, a more specific description can be made by referring to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the drawings only show relevant features of this disclosure and are therefore not necessarily intended to be limiting, as those skilled in the art will understand upon reading this disclosure that other valid features may be permissible.

[0015] Figure 1 This is a block diagram illustrating an example communication system according to some implementations.

[0016] Figure 2AThis is a block diagram illustrating example elements of an encoder component according to some embodiments.

[0017] Figure 2B This is a block diagram illustrating example elements of a decoder component according to some embodiments.

[0018] Figure 3 This is a block diagram illustrating an example server system according to some implementation methods.

[0019] Figure 4A Example transform unit blocks and filter apertures according to some implementations are shown.

[0020] Figure 4B An example method for generating prediction samples according to some implementations is shown.

[0021] Figure 5 An example processing for applying bilateral filtering to intra-frame prediction samples is shown according to some implementations.

[0022] Figure 6A An example video decoding process according to some implementation methods is shown.

[0023] Figure 6B An example video encoding process according to some implementation methods is shown.

[0024] By convention, the various features shown in the accompanying drawings are not necessarily drawn to scale, and similar reference numerals may be used to indicate similar features throughout the specification and the drawings. Detailed Implementation

[0025] This disclosure describes video / image compression techniques that include applying bilateral filtering to intra-frame prediction. The disclosed bilateral filtering technique includes: performing intra-frame prediction on the current block of a plurality of coded blocks of a video bitstream; applying a bilateral filter to the current block; and reconstructing the current block based on the intra-frame prediction and the output of the bilateral filter. As discussed in detail below, bilateral filtering can be applied conditionally (e.g., based on the size of the current block and / or the prediction mode of the current block). In this way, bilateral filtering can be applied only where it is most effective / beneficial. Applying a bilateral filter to intra-frame prediction can improve encoding / decoding accuracy compared to current implementations (e.g., intra-frame prediction without bilateral filtering). For example, simulation results performed under general test conditions based on ECM v.14 show that applying a bilateral filter to intra-frame prediction samples improves the Y component by 0.12%, the U component by 0.08%, and the V component by 0.09%.

[0026] Example systems and devices

[0027] Figure 1This is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to 120-m) communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is a streaming system, for example, for use with video-enabled applications such as video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0028] Source device 102 includes a video source 104 (e.g., a camera device component or media storage device) and an encoder component 106. In some embodiments, the video source 104 is a digital camera device (e.g., configured to create an uncompressed video sample stream). The encoder component 106 generates one or more encoded video bitstreams from the video stream. The video stream from video source 104 can have a higher data volume compared to the encoded video bitstream 108 generated by encoder component 106. Because the encoded video bitstream 108 has a lower data volume (less data) compared to the video stream from video source 104, it requires less bandwidth to transmit and less storage space to store compared to the video stream from video source 104. In some embodiments, source device 102 does not include encoder component 106 (e.g., configured to transmit uncompressed video to network 110).

[0029] One or more networks 110 represent any number of networks that transmit information between source device 102, server system 112, and / or electronic device 120, including, for example, wired (connected) and / or wireless communication networks. One or more networks 110 may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet.

[0030] One or more networks 110 include server systems 112 (e.g., distributed / cloud computing systems). In some embodiments, server system 112 is a streaming server or includes streaming servers (e.g., configured to store and / or distribute video content, such as encoded video streams from source device 102). Server system 112 includes codec components 114 (e.g., configured to encode and / or decode video data). In some embodiments, codec components 114 include encoder components and / or decoder components. In various embodiments, codec components 114 are instantiated as hardware, software, or a combination of hardware and software. In some embodiments, codec components 114 are configured to decode encoded video bitstream 108 and re-encode video data using different encoding standards and / or methods to generate encoded video data 116. In some embodiments, server system 112 is configured to generate multiple video formats and / or encodings based on encoded video bitstream 108. In some embodiments, server system 112 is used as a Media-Aware Network Element (MANE). For example, server system 112 can be configured to trim the encoded video bitstream 108 to customize potentially different bitstreams for one or more electronic devices 120. In some implementations, the MANE is provided separately from server system 112.

[0031] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode encoded video data 116 to generate an outgoing video stream that can be rendered on a display or other type of rendering device. In some embodiments, one or more of the electronic devices 120 do not include a display component (e.g., communicatively coupled to an external display device and / or includes a media storage device). In some embodiments, electronic device 120 is a streaming client. In some embodiments, electronic device 120 is configured to access server system 112 to obtain encoded video data 116.

[0032] The source device and / or multiple electronic devices 120 are sometimes referred to as “terminal devices” or “user devices”. In some implementations, one or more of the electronic devices 120 and / or the source device 102 are instances of server systems, personal computers, portable devices (e.g., smartphones, tablets, or laptops), wearable devices, video conferencing equipment, and / or other types of electronic devices.

[0033] In an example operation of communication system 100, source device 102 transmits an encoded video bitstream 108 to server system 112. For example, source device 102 may encode a stream of images captured by the source device. Server system 112 receives the encoded video bitstream 108 and may decode and / or encode the encoded video bitstream 108 using codec component 114. For example, server system 112 may apply encoding to the video data that is optimized for network transmission and / or storage. Server system 112 may transmit encoded video data 116 (e.g., one or more encoded video bitstreams) to one or more electronic devices 120. Each electronic device 120 may decode the encoded video data 116 and optionally display video images.

[0034] Figure 2A This is a block diagram illustrating example elements of an encoder component 106 according to some embodiments. The encoder component 106 receives video data (e.g., a source video sequence) from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives the video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder component 106). The video source 104 may provide the source video sequence as a digital video sample stream, which may have any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 YCrCB or RGB), and any suitable sampling structure (e.g., YCrCb 4:2:0 or YCrCb 4:4:4). In some embodiments, the video source 104 is a storage device storing previously captured / prepared video. In some embodiments, the video source 104 is a camera device that captures local image information as a video sequence. Video data can be provided as multiple individual images that are given motion when viewed sequentially. Each image can be organized as a spatial array of pixels, where, depending on the sampling structure, color space, etc., each pixel may include one or more samples. The relationship between pixels and samples will be readily understood by those skilled in the art.

[0035] Encoder component 106 is configured to encode and / or compress images of a source video sequence into an encoded video sequence 216 in real time or under other time constraints required by the application. In some embodiments, encoder component 106 is configured to perform a conversion between the source video sequence and a visual media data bitstream (e.g., a video bitstream). Implementing an appropriate encoding rate is a function of controller 204. In some embodiments, controller 204 controls and is functionally coupled to other functional units as described below. Parameters set by controller 204 may include rate control-related parameters (e.g., image skipping, quantizer, and / or rate-distortion optimization techniques). This includes parameters such as image size, group of pictures (GOP) layout, and maximum motion vector search range. Other functions of the controller 204 can be readily identified by those skilled in the art, as these functions may belong to the encoder component 106 optimized for a specific system design.

[0036] In some implementations, encoder component 106 is configured to operate within an encoding / decoding loop. In a simplified example, the encoding / decoding loop includes a source encoder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder 210. Decoder 210 reconstructs the symbols in a manner similar to that of the (remote) decoder to create sample data (assuming lossless compression between the symbols and the encoded video bitstream). The reconstructed sample stream (sample data) is input to a reference image memory 208. Since decoding the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of the reference image memory 208 are also bit-accurate between the local and remote encoders. In this way, the encoder's prediction portion interprets the same sample values ​​as the sample values ​​that the decoder interprets during prediction as reference image samples.

[0037] The operation of decoder 210 can be combined with a remote decoder, for example, as shown below. Figure 2B The operation of the decoder component 122 is the same as described in the detailed description. However, a brief reference is provided. Figure 2B Since the symbols are available and the encoding of the symbols into an encoded video sequence by the entropy encoder 214 and the decoding of the symbols by the parser 254 can be lossless, the entropy decoding portion of the decoder component 122, including the buffer memory 252 and the parser 254, does not need to be fully implemented in the local decoder 210.

[0038] Besides parsing / entropy decoding, the decoder techniques described in this paper can exist in the corresponding encoders in essentially the same form. For this reason, the subject matter focuses on decoder operations. Additionally, the description of the encoder techniques can be simplified, as the encoder techniques are inverses of the decoder techniques.

[0039] As part of the operation of the source encoder 202, the source encoder 202 can perform motion-compensated predictive coding, which predictively encodes the input frame with reference to one or more previously encoded frames from the video sequence designated as reference frames. In this way, the encoding engine 212 encodes the differences between pixel blocks of the input frame and pixel blocks of the reference frame, which can be selected as the prediction reference for the input frame. The controller 204 can manage the encoding operations of the source encoder 202, including, for example, setting parameters and subgroup parameters for encoding the video data.

[0040] Decoder 210 decodes encoded video data based on symbols created by source encoder 202 that can be designated as reference frames. The operation of encoding engine 212 can advantageously handle lossy processing. When encoded video data is processed by video decoder (… Figure 2A When decoded at a location (not shown), the reconstructed video sequence can be a copy of the source video sequence with some errors. Decoder 210 replicates the decoding process performed on the reference frame by a remote video decoder, and the reconstructed reference frame can be stored in reference image memory 208. In this way, encoder component 106 locally stores a copy of the reconstructed reference frame that shares common content (no transmission errors) with the reconstructed reference frame to be obtained by the remote video decoder.

[0041] Predictor 206 can perform a prediction search against encoding engine 212. That is, for a new frame to be encoded, predictor 206 can search the reference image memory 208 for sample data (as candidate reference pixel blocks) or specific metadata such as reference image motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new image. Predictor 206 can operate pixel-by-pixel based on the sample blocks to find appropriate prediction references. As determined by the search results obtained by predictor 206, the input image can have prediction references obtained from multiple reference images stored in reference image memory 208.

[0042] The outputs of all the functional units mentioned above can undergo entropy encoding in the entropy encoder 214. The entropy encoder 214 converts these symbols, such as those generated by the various functional units, into an encoded video sequence by lossless compression according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, and / or arithmetic coding).

[0043] In some implementations, the output of entropy encoder 214 is coupled to a transmitter. The transmitter can be configured to buffer encoded video sequences, such as those created by entropy encoder 214, in preparation for transmission via communication channel 218, which can be a hardware / software link to a storage device storing the encoded video data. The transmitter can be configured to combine encoded video data from source encoder 202 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown). In some implementations, the transmitter can transmit additional data along with the encoded video. Source encoder 202 can include such data as part of the encoded video sequence. Additional data can include temporal / spatial / SNR (Signal-to-Noise Ratio, SNR) enhancement layers, other forms of redundant data such as redundant images and slices, Supplementary Enhancement Information (SEI) messages, fragments of Visual Usability Information (VUI) parameter sets, etc.

[0044] Controller 204 can manage the operation of encoder component 106. During encoding, controller 204 can assign a specific encoding picture type to each encoded picture, which may affect the encoding technique applied to the corresponding picture. For example, pictures can be assigned as intra-pictures (I-pictures), predictive pictures (P-pictures), or bidirectional predictive pictures (B-pictures). Intra-pictures can be encoded and decoded without using any other frames in the sequence as prediction sources. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are familiar with those variations of I-pictures and their corresponding applications and characteristics, and therefore they will not be repeated here. Predictive pictures can be encoded and decoded using inter-frame prediction or intra-frame prediction that uses at most one motion vector and reference index to predict sample values ​​for each block. Bidirectional predictive pictures can be encoded and decoded using inter-frame prediction or intra-frame prediction that uses at most two motion vectors and reference indexes to predict sample values ​​for each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0045] Source images can typically be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and encoded on a block-by-block basis. These blocks can be predictively encoded with reference to other (already encoded) blocks, determined by the coding assignments of the corresponding images applied to the blocks. For example, blocks of image I can be unpredictably encoded, or blocks of image I can be predictively encoded (spatial prediction or intra-frame prediction) with reference to already encoded blocks of the same image. Pixel blocks of image P can be unpredictably encoded with reference to a previously encoded reference image via spatial prediction or temporal prediction. Blocks of image B can be unpredictably encoded with reference to one or two previously encoded reference images via spatial prediction or temporal prediction.

[0046] Video can be captured as multiple source images (video images) in a time-series manner. Intra-frame image prediction (often simply called intra-prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In the example, a specific image during encoding / decoding—referred to as the current image—is segmented into blocks. Where a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference images.

[0047] Encoder component 106 can perform encoding operations according to any predetermined video coding technique or standard described herein. In operation, encoder component 106 can perform various compression operations, including predictive coding operations utilizing temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0048] Figure 2B This is a block diagram illustrating example elements of a decoder component 122 according to some embodiments. Figure 2B The decoder component 122 is coupled to channel 218 and display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to loop filter 256 and configured to transmit data to display 124 (e.g., via a wired or wireless connection).

[0049] In some embodiments, decoder component 122 includes a receiver coupled to channel 218 and configured to receive data from channel 218 (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more encoded video sequences to be decoded by decoder component 122. In some embodiments, the decoding of each encoded video sequence is independent of other encoded video sequences. Each encoded video sequence can be received from channel 218, which may be a hardware / software link to a storage device storing the encoded video data. The receiver may receive encoded video data along with other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective user entities (not depicted). The receiver may separate the encoded video sequences from other data. In some embodiments, the receiver receives additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. Decoder component 122 may use the additional data to decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0050] According to some embodiments, the decoder component 122 includes a buffer memory 252, a parser 254 (sometimes also referred to as an entropy decoder), a scaler / inverse transform unit 258, an intra-frame image prediction unit 262, a motion compensation prediction unit 260, an aggregator 268, a loop filter unit 256, a reference image memory 266, and a current image memory 264. In some embodiments, the decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuit systems. The decoder component 122 can be implemented at least partially in software.

[0051] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., to combat network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, a separate buffer memory is provided outside decoder component 122 (e.g., to combat network jitter) in addition to buffer memory 252 inside decoder component 122 (e.g., configured to handle broadcast timing). Buffer memory 252 may not be necessary or may be small when receiving data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network. Buffer memory 252 may be required to make the best use of packet networks such as the Internet. Buffer memory 252 may be relatively large and / or have an adaptive size and may be implemented at least partially in an operating system or similar component outside decoder component 122.

[0052] Parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. Symbols may include, for example, information for managing the operation of decoder component 122, and / or information for controlling rendering devices such as display 124. Control information for the rendering device may be in the form of, for example, Supplemental Enhancement Information (SEI) messages or Video Availability Information (VUI) parameter set fragments (not depicted). Parser 254 parses (entropy decodes) the encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow principles known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. Parser 254 may extract a subgroup parameter set from the encoded video sequence for pixels in a subgroup of subgroups for use in the video decoder, based on at least one parameter corresponding to a group. Subgroups may include picture groups (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser 254 can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0053] Depending on the type of encoded video picture or a portion thereof (e.g., inter-frame and intra-frame pictures, inter-frame and intra-frame blocks) and other factors, the reconstruction of symbol 270 may involve multiple different units. Which units are involved and how they are involved can be controlled by parser 254 through subgroup control information parsed from the encoded video sequence. For clarity, the flow of such subgroup control information between parser 254 and the following multiple units is not depicted.

[0054] The decoder component 122 can be conceptually subdivided into multiple functional units, and in some implementations, these units interact closely with each other and can be at least partially integrated with each other. However, for clarity, this document retains a conceptual subdivision of the functional units.

[0055] The scaler / inverse transform unit 258 receives quantized transform coefficients as symbols 270 and control information (e.g., which transform to use, block size, quantization factor, and / or quantization scaling matrix) from the parser 254. The scaler / inverse transform unit 258 can output blocks including sample values, which can be input to the aggregator 268. In some cases, the output samples of the scaler / inverse transform unit 258 belong to intra-frame encoded blocks; that is, blocks that do not use predictive information from previously reconstructed images, but can use predictive information from previously reconstructed portions of the current image. Such predictive information can be provided by the intra-frame image prediction unit 262. The intra-frame image prediction unit 262 can use surrounding reconstructed information obtained from the current (partially reconstructed) image from the current image memory 264 to generate blocks of the same size and shape as the blocks in the reconstruction. The aggregator 268 can add the prediction information already generated by the intra-frame image prediction unit 262 to the output sample information provided by the scaler / inverse transform unit 258 based on each sample.

[0056] In other cases, the output samples of the scaler / inverse transform unit 258 belong to inter-frame encoded blocks and potentially to motion-compensated blocks. In this case, the motion compensation prediction unit 260 can access the reference image memory 266 to obtain samples for prediction. After motion compensation of the obtained samples according to the symbol 270 belonging to the block, these samples can be added by the aggregator 268 to the output of the scaler / inverse transform unit 258 (referred to in this case as residual samples or residual signals) to generate output sample information. The address in the reference image memory 266 from which the motion compensation prediction unit 260 obtains the predicted samples can be controlled by motion vectors. Motion vectors can be used by the motion compensation prediction unit 260 in the form of symbol 270, which may have, for example, an X component, a Y component, and a reference image component. Motion compensation may also include, for example, interpolation of sample values ​​obtained from the reference image memory 266 when using subsampled precise motion vectors, and motion vector prediction mechanisms.

[0057] The output samples of aggregator 268 can undergo various loop filtering techniques in loop filter unit 256. Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video bitstream and available to loop filter unit 256 as symbols 270 from parser 254. However, video compression techniques may also respond to metadata obtained during decoding of previous (in decoding order) portions of the encoded picture or encoded video sequence, and to sample values ​​obtained from previous reconstruction and loop filtering. The output of loop filter unit 256 may be a sample stream, which can be output to a rendering device such as display 124, and stored in reference picture memory 266 for use in future inter-frame picture prediction.

[0058] Once reconstructed, certain encoded images can be used as reference images for future predictions. Once an encoded image has been reconstructed and has been identified as a reference image (e.g., by parser 254), the current reference image can become part of the reference image memory 266, and a new current image memory can be reallocated before reconstructing subsequent encoded images begins.

[0059] Decoder component 122 can perform decoding operations according to a predetermined video compression technique that may be recorded in any standard such as those described herein. As specified in a video compression technique document or standard, and particularly in a configuration file therein, an encoded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard. Furthermore, the complexity of the encoded video sequence may be within a range defined by the hierarchy of the video compression technique or standard in order to conform to some video compression techniques or standards. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured, for example, megasamples per second), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further restricted by the Hypothetical Reference Decoder (HRD) specification and metadata managed by the HRD buffer, which is signaled in the encoded video sequence.

[0060] Figure 3This is a block diagram illustrating a server system 112 according to some embodiments. The server system 112 includes a control circuitry system 302, one or more network interfaces 304, a memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuitry system 302 includes one or more processors (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and / or a DPU (Data Processing Unit)). In some embodiments, the control circuitry system includes a field-programmable gate array, a hardware accelerator, and / or an integrated circuit (e.g., an application-specific integrated circuit).

[0061] Network interface 304 can be configured to interface with one or more communication networks (e.g., wireless networks, wired networks, and / or optical networks). Communication networks can be local, wide area, metropolitan area, vehicle-mounted and industrial, real-time, latency-tolerant, etc. Examples of communication networks include: local area networks such as Ethernet, wireless LANs; cellular networks including GSM (Global System for Mobile Communications), 3G (the Third Generation), 4G (the Fourth Generation), 5G (the Fifth Generation), LTE (Long Term Evolution), etc.; wired or wireless wide area digital networks for TV including cable TV, satellite TV, and terrestrial broadcast TV; vehicle and industrial networks including CANBus (Controller Area Network-BUS), etc. Such communication can be one-way receiving (e.g., broadcast TV), one-way transmitting (e.g., to a CANBus device), or bidirectional (e.g., to other computer systems using local area digital networks or wide area digital networks). Such communication can include communication across one or more cloud computing networks.

[0062] User interface 306 includes one or more output devices 308 and / or one or more input devices 310. Input devices 310 may include one or more of the following: keyboard, mouse, touchpad, touch screen, data glove, joystick, microphone, scanner, camera device, etc. Output devices 308 may include one or more of the following: audio output devices (e.g., speakers), visual output devices (e.g., displays or monitors), etc.

[0063] Memory 314 may include high-speed random access memory (e.g., DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), DDR RAM (Double Data Rate Random Access Memory), and / or other random access solid-state memory devices) and / or non-volatile memory (e.g., one or more disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state memory devices). Memory 314 may optionally include one or more storage devices located remotely from the control circuitry system 302. Memory 314, or alternatively, the non-volatile solid-state memory devices within memory 314 include non-transitory computer-readable storage media. In some embodiments, memory 314 or the non-transitory computer-readable storage media of memory 314 stores programs, modules, instructions, and data structures, or subsets or supersets thereof:

[0064] • Operating system 316, which includes procedures for handling various basic system services and for performing hardware-related tasks;

[0065] • Network communication module 318, which is used to connect server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired and / or wireless connections);

[0066] • Encoding / decoding module 320, which performs various functions related to encoding and / or decoding data, such as video data. In some embodiments, encoding / decoding module 320 is an example of codec component 114. Encoding / decoding module 320 includes, but is not limited to, one or more of the following:

[0067] ◦ Decoding module 322, which performs various functions related to decoding encoded data, such as those previously described with respect to decoder component 122; and

[0068] ◦ Encoding module 340, which performs various functions related to encoding data, such as those previously described with respect to encoder component 106; and

[0069] • For example, an image memory 352 for storing images and image data, used in conjunction with the encoding / decoding module 320. In some embodiments, the image memory 352 includes one or more of the following: a reference image memory 208, a buffer memory 252, a current image memory 264, and a reference image memory 266.

[0070] In some implementations, the decoding module 322 includes a parsing module 324 (e.g., configured to perform various functions previously described with respect to the parser 254), a transformation module 326 (e.g., configured to perform various functions previously described with respect to the scaler / inverse transformation unit 258), a prediction module 328 (e.g., configured to perform various functions previously described with respect to the motion compensation prediction unit 260 and / or the intra-frame image prediction unit 262), and a filter module 330 (e.g., configured to perform various functions previously described with respect to the loop filter 256).

[0071] In some embodiments, the encoding module 340 includes a code module 342 (e.g., configured to perform various functions previously described with respect to source encoder 202 and / or encoding engine 212) and a prediction module 344 (e.g., configured to perform various functions previously described with respect to predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 include... Figure 3 A subset of the modules shown. For example, a prediction module shared by both decoding module 322 and encoding module 340.

[0072] Each of the modules identified above and stored in memory 314 corresponds to a set of instructions for performing the functions described herein. The modules identified above (e.g., instruction sets) do not need to be implemented as separate software programs, processes, or modules, and therefore various subsets of these modules can be combined or otherwise rearranged in various implementations. For example, the codec module 320 may optionally not include separate decoding and encoding modules, but instead use the same set of modules to perform both sets of functions. In some implementations, memory 314 stores a subset of the modules and data structures identified above. In some implementations, memory 314 stores additional modules and data structures not described above.

[0073] Although Figure 3 A server system 112 according to some embodiments is shown, but Figure 3 This is intended more as a functional description of various features that can exist in one or more server systems than as a structural diagram of the implementation described herein. In practice, items shown individually may be combined and some items may be separated. For example, Figure 3 Some items shown individually can be implemented on a single server, and a single item can be implemented by one or more servers. The actual number of servers used to implement server system 112 and how features are distributed among said servers will vary depending on the implementation method, and optionally, it depends in part on the amount of data traffic processed by the server system during peak usage periods and during average usage periods.

[0074] Example encoding techniques

[0075] The encoding processes and techniques described below can be performed on the devices and systems described above (e.g., source device 102, server system 112, and / or electronic device 120). According to some embodiments, methods for applying bilateral filtering are described below.

[0076] Intra-frame prediction is one of the main techniques used to improve coding efficiency. In intra-frame prediction, samples from neighboring reconstructed regions can be used as reference samples to generate the predictor for the current block. For example, the reference sample can be directly copied and used as the predictor, or the reference sample can be filtered, for example, using a simple [1,2,1] / 4 filter and then copied as the predictor. However, the final predictor may still contain reconstruction noise or may have over-smoothing applied.

[0077] According to some implementations, bilateral filters can be conditionally applied to intra-frame prediction. Bilateral filters are non-linear techniques that can blur an image while preserving strong edges. In some implementations, bilateral filters can replace conventional intra-frame prediction filters or operate serially / parallel with a conventional design. For example, in serial operation, bilateral filters can be introduced as an additional filtering step to generate the final predictor. In parallel operation, bilateral filters and conventional intra-frame prediction filters are executed exclusively, and the codec can use bilateral filters or existing filters (or some combination of both).

[0078] In some implementations, bilateral filters are applied to intra-frame prediction in a manner similar to their application to transform blocks. Bilateral filters can be applied to avoid unwanted oversmoothing for pixels in edge regions. Bilateral filtering involves weighting neighboring samples while considering the pixel value itself, to give greater weight to pixels with similar brightness or chromaticity values. For example, those located in… Samples at a location use their neighboring samples Perform filtering. Weights This is a sample The assigned weights are used to assign weights to the samples. Filtering is performed, and the weights are defined as shown in Equation 1 below.

[0079]

[0080] Formula 1

[0081] In Equation 1, and These are samples and The intensity value. It is a spatial parameter, and It is a range parameter. Filtering and its components... The filtered sample value can be defined as shown in Equation 2 below.

[0082]

[0083] Formula 2

[0084] The characteristics (or strength) of a bilateral filter can be controlled by these two parameters. For example, samples closer to the sample to be filtered and those with smaller intensity differences from the sample to be filtered have greater weights than samples farther away and with larger intensity differences from the sample to be filtered. In some implementations, each sample is filtered using only its immediate neighbors' reconstructed samples.

[0085] Figure 4A This is a diagram of an example 8×8 cell 402, illustrating a filter comprising a positive-shaped filter aperture 404 centered on the sample to be filtered. In such an example, the settings are based on the cell size. It is defined by Equation 3 below.

[0086]

[0087] Formula 3

[0088] In some implementations, bilateral filtering is applied only to luma blocks. In other implementations, bilateral filtering is applied only to blocks with values ​​higher than a predefined threshold (e.g., greater than zero).

[0089] In some implementations, intra-predicted samples are generated based on intra-reference samples. Typically, the generated intra-predicted samples may contain noise. In some implementations, bilateral filtering can be applied to reduce coding noise within the prediction, as bilateral filtering can achieve edge-preserving denoising. In some implementations, the generated intra-predicted samples are further filtered using bilateral filters to reduce noise levels. In some implementations, the bilateral filters are applied in a manner similar to that used for loop filtering stages. In some implementations, the filter length of the bilateral filters is 2.

[0090] Figure 5 An example processing 500 for applying a bilateral filter to intra-frame prediction samples, according to some embodiments, is shown. Figure 5In the example, intra-prediction sample 506 can be generated directly from unfiltered intra-reference sample 502, or indirectly from unfiltered intra-reference sample 502 via 1-2-1 filter 504 (or other types of filters). In this example, intra-prediction sample 506 undergoes bilateral filtering, where a 2-D bilateral filter 508 is applied to these intra-prediction samples before they are used in later processing 510 (e.g., later reconstruction processing). In some implementations, the bilateral filter used for intra-prediction filtering has the same properties as the bilateral filter used for in-loop filtering. For example, the filter length can remain 2 and the filtering logic can remain unchanged. In some implementations, for the current block / sample, the bilateral filter uses the available upper neighbor reconstructed sample and left neighbor reconstructed sample. As an example, for a block employing Template-Based Intra Prediction Mode (TIMD), a 6-tap cubic interpolation filter can be selected.

[0091] Figure 6A This is a flowchart illustrating a method 600 for decoding video according to some embodiments. Method 600 can be performed at a computing system (e.g., server system 112, source device 102, or electronic device 120) having a control circuitry and a memory storing instructions for execution by the control circuitry. In some embodiments, method 600 is performed by executing instructions stored in the memory of the computing system (e.g., memory 314).

[0092] The system receives (602) a video bitstream (e.g., an encoded video sequence) comprising a current image, which includes multiple blocks, including the current block. The system performs (604) intra-frame prediction on the current block among the multiple encoded blocks. The system applies (606) a bilateral filter (e.g., a 2-D bilateral filter 508) to the current block. The system reconstructs (608) the current block based on the intra-frame prediction and the output of the bilateral filter. In this way, the bilateral filter can be used before or after other processing steps during intra-frame prediction. When used before other processing steps, the bilateral filter is used to filter reference samples reconstructed from neighboring regions, which can be further processed and used to generate a predictor. When used after other processing steps, the bilateral filter is used to filter processed reference samples to generate the final predictor.

[0093] In some implementations, bilateral filtering is conditionally applied to intra-frame prediction based on the prediction mode of the corresponding block. For example, if the current prediction mode is DC (Direct Current), then a bilateral filter is not applied. As another example, if the current prediction mode is planar, then a bilateral filter is not applied. In another example, if the current prediction mode is either DC or planar, then a bilateral filter is not applied. In yet another example, if the current prediction mode is a purely vertical or purely horizontal mode, then a bilateral filter is not applied. In some implementations, if the current prediction mode is a near-vertical or near-horizontal mode (e.g., two adjacent modes of purely vertical or purely horizontal modes), then a bilateral filter is not applied. In some implementations, if the current prediction mode is an angular mode (e.g., neither planar nor DC), then a bilateral filter is not applied.

[0094] In some implementations, if the current mode directly uses reference samples falling into integer positions, such as 0 (PLANAR_IDX), -14, -12, -10, -6, 2, 34, 66, 72, 76, 78, or 80, then a bilateral filter is not applied. In some implementations, if Position Dependent Prediction Combination (PDPC) is enabled, then a bilateral filter is not applied. In some implementations, if an unconventional intra-prediction mode (including but not limited to decoder-side intra-prediction, template-based intra-prediction, intra-prediction fusion, and MIP (Matrix-based IntraPrediction, MIP)) is enabled, then a bilateral filter is not applied. In some implementations, if the intra-predictor is a fused signal predicted from multiple reference rows, then a bilateral filter is not applied.

[0095] In some implementations, a bilateral filter is applied to a portion of the predicted samples within the current prediction block.

[0096] In some implementations, a bilateral filter is not applied when samples are copied from reference samples instead of predicted by interpolation. In some implementations, samples in different rows within the current block will have different reference sample predictors, depending on the intra-frame mode with a fixed reference row position. For example, Figure 4B This illustrates how, according to some implementation methods, two rows of samples are copied from a reference sample while other two rows of samples are generated using interpolation to produce predicted samples. Figure 4B In the example, the bilateral filter can be applied to two rows of samples generated via interpolation, but not to two rows of samples copied from the reference sample.

[0097] In some implementations, bilateral filtering is conditionally applied to intra-frame prediction based on the current block size. For example, if the current block size is too small (e.g., less than 32 samples), bilateral filtering may not be applied. In some cases, applying bilateral filtering to smaller blocks can introduce artifacts. In another example, if the current block size is too large (e.g., greater than 32 samples), bilateral filtering is not applied. For example, the accuracy of bilateral filtering may decrease when the block size increases beyond a certain limit.

[0098] In some implementations, bilateral filters can be used before or after other processing steps preceding intra-frame prediction. Bilateral filters can also be used both before and after other processing steps preceding intra-frame prediction. When used before other processing steps, bilateral filters can be used to filter reference samples reconstructed from neighboring regions, which can then be further processed and used to generate the predictor. When used after other processing steps, bilateral filters can be used to filter processed reference samples to generate the final predictor.

[0099] In some implementations, a bilateral filter is used before or after Location-Related Prediction Combination (PDPC). It can also be used both before and after PDPC. When used before PDPC, the bilateral filter can be used to filter the predictor built during the prediction phase. When used after PDPC, the bilateral filter can be used to filter the predictor to which PDPC has been applied and generate the final predictor.

[0100] In some implementations, the proposed method can signal, enable, or disable parameters in the Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Application Parameter Set (APS), slice header, and / or block header (e.g., CTU (Coding Tree Unit), CU, PU, ​​or TU).

[0101] Figure 6BThis is a flowchart illustrating a method 650 for encoding video according to some embodiments. Method 650 can be executed at a computing system (e.g., server system 112, source device 102, or electronic device 120) having a control circuitry and memory storing instructions for execution by the control circuitry. In some embodiments, method 650 is executed by executing instructions stored in memory (e.g., memory 314) of the computing system. In some embodiments, method 650 is executed by the same system as method 600 described above.

[0102] The system receives (652) video data (e.g., a source video sequence) including a current image, which comprises multiple blocks. The multiple blocks include the current block. The system encodes the current block by applying intra-frame prediction to it (654). The system signals (656) the current block in the video bitstream. The system signals (658) in the video bitstream whether a bilateral filter is applied to the current block. As previously described, the encoding process can mirror the decoding process described herein (e.g., conditional bilateral filtering for intra-frame prediction). For brevity, these details are not repeated here.

[0103] Although Figure 6A and Figure 6B Multiple logical stages are shown in a specific order, but stages that are not in order can be reordered, and other stages can be combined or split. Some reorderings or other groupings not specifically mentioned will be obvious to those skilled in the art, and therefore the orderings and groupings presented herein are not exhaustive. Furthermore, it should be understood that the stages can be implemented in hardware, firmware, software, or any combination thereof.

[0104] Now let's turn to some example implementations.

[0105] (A1) In one aspect, some implementations include a method for video decoding (e.g., method 600). In some implementations, the method is performed at a computing system (e.g., server system 112) having memory and control circuitry. In some implementations, the method is performed at an encoding / decoding module (e.g., encoding / decoding module 320). In some implementations, the method is performed at a source coding component (e.g., source encoder 202), a coding engine (e.g., coding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes: (i) receiving a video bitstream (e.g., an encoded video sequence) comprising a current image, the current image comprising a plurality of blocks, the plurality of blocks including a current block; (ii) performing intra-frame prediction on the current block among the plurality of coded blocks; (iii) applying a bilateral filter to the current block; and (iv) reconstructing the current block based on the intra-frame prediction and the output of the bilateral filter. Some implementations include conditionally using a bilateral filter for intra-frame prediction. For example, the bilateral filter may replace different types of filters or operate serially or in parallel with different types of filters. When operating serially with another filter, a bilateral filter can be applied as an additional filtering step to generate the final predictor. When operating in parallel with another filter, the bilateral filter and the other filter can be executed exclusively (e.g., the codec can select a bilateral filter or an existing filter based on encoding information). In some implementations, the video bitstream includes an indicator indicating whether a bilateral filter should be applied to the current block. The indicator can be signaled in high-level syntax or at the block level. For example, a bilateral filter can be applied before and / or after other processing steps. When used before other processing steps, a bilateral filter can be used to filter reference samples reconstructed from neighboring regions, which can be further processed and used to generate the predictor. When used after other processing steps, a bilateral filter can be used to filter processed reference samples to generate the final predictor.

[0106] (A2) In some implementations of A1, a bilateral filter is applied before intra-frame prediction is performed. For example, a bilateral filter may be applied to a reference sample of the current block. In some implementations, a bilateral filter is applied after intra-frame prediction is performed. For example, a bilateral filter may be applied to the prediction block of the current block. In some implementations, the video bitstream includes an indicator indicating whether the bilateral filter is performed before or after another processing step (e.g., another filter or intra-frame prediction). In some implementations, the indicator is signaled using high-level syntax (e.g., in a sequence parameter set, video parameter set, picture parameter set, application parameter set, or slice header). In some implementations, the indicator is signaled at the block level (e.g., in a block header), such as at the CTU level, CU level, PU level, or TU level.

[0107] (A3) In some implementations of A1 or A2, performing intra-frame prediction includes applying a location-related prediction combination (PDPC) technique. For example, a bilateral filter may be used before and / or after the PDPC.

[0108] (A4) In some implementations of A3, a bilateral filter is applied after the PDPC technique is performed. For example, when used after PDPC, the bilateral filter can be used to filter the predictor to which PDPC has been applied and generate the final predictor. In some implementations, the bilateral filter is signaled in the video bitstream (e.g., in high-level syntax) to indicate whether it is applied before or after the PDPC technique.

[0109] (A5) In some implementations of A3, a bilateral filter is applied before performing the PDPC technique. For example, when used before PDPC, the bilateral filter can be used to filter the predictor built during the prediction phase.

[0110] (A6) In some embodiments of A1 through A5, a bilateral filter is applied to the current block when the current block has a block size that satisfies one or more criteria. For example, the bilateral filter is conditionally applied to intra-frame prediction based on the current block size. In some embodiments, the bilateral filter is not applied to the current block when the current block has a block size that does not satisfy one or more criteria. In some embodiments, the bilateral filter is applied to the current block based on the determination that the current block size satisfies one or more criteria. The block size may refer to block height, block width, block area, and / or block perimeter. In some embodiments, one or more criteria are fixed (e.g., hard-coded in the coding unit). In some embodiments, one or more criteria are based on encoded information and / or signaled in the video bitstream.

[0111] (A7) In some implementations of A6, a bilateral filter is applied to the current block when the current block has a block size greater than a predetermined threshold. For example, if the current block size is too small, the bilateral filter may not be applied. As an example, if the block size is less than 32 samples (pixels), the bilateral filter may not be applied (e.g., to avoid introducing artifacts into the reconstructed video). As an example, the predetermined threshold may be the area of ​​32×32 samples.

[0112] (A8) In some implementations of A6, a bilateral filter is applied to the current block when the current block has a block size less than a predetermined threshold. For example, if the current block size is too large, the bilateral filter may not be applied. As an example, if the block size is greater than 32 samples, the bilateral filter may not be applied.

[0113] (A9) In some embodiments of A1 through A8, a bilateral filter is applied to the current block when the intra-prediction mode used for intra-prediction satisfies one or more criteria. For example, the bilateral filter is conditionally applied to intra-prediction based on the prediction mode of the corresponding block. In some embodiments, one or more criteria are fixed. In some embodiments, one or more criteria are based on encoded information and / or signaled in the video bitstream. In some embodiments, the bilateral filter is not applied to the current block when the intra-prediction mode of the current block does not satisfy one or more criteria. In some embodiments, the bilateral filter is applied to the current block based on the determination that the intra-prediction mode used for intra-prediction satisfies one or more criteria. In some embodiments, the bilateral filter is selectively applied based on whether the intra-prediction mode used for intra-prediction satisfies one or more criteria. In some embodiments, one or more criteria include criteria that the intra-prediction mode is not an unconventional intra-prediction mode (e.g., not involving decoder-side intra-mode derivation (DIMD), template-based intra-mode derivation, intra-prediction fusion, matrix-based intra-prediction (MIP), etc.). For example, if one of the unconventional intra-prediction modes is enabled, the bilateral filter is not applied. In some implementations, one or more criteria include the criterion that the intra-prediction mode does not use reference samples located at integer positions. For example, if the current mode directly uses reference samples falling at integer positions, such as 0 (PLANAR_IDX), -14, -12, -10, -6, 2, 34, 66, 72, 76, 78, or 80, the bilateral filter is not applied.

[0114] (A10) In some implementations of A9, one or more criteria include a criterion that the intra-prediction mode is not a DC mode. For example, if the current prediction mode is DC, a bilateral filter may not be applied.

[0115] (A11) In some implementations of A9 or A10, one or more criteria include criteria that the intra-prediction mode is not a planar mode. For example, if the current prediction mode is planar, a bilateral filter may not be applied.

[0116] (A12) In some implementations of any of A9 through A11, one or more criteria include: a first criterion that the intra-prediction mode is not a planar mode, and a second criterion that the intra-prediction mode is not a DC mode. For example, if the current prediction mode is DC or planar, a bilateral filter may not be applied. In some implementations, a bilateral filter is applied only when the intra-prediction mode is an angular prediction mode.

[0117] (A13) In some implementations of any of A9 to A12, one or more criteria include: a first criterion that the intra-prediction mode is not a pure vertical mode, and a second criterion that the intra-prediction mode is not a pure horizontal mode. For example, if the current prediction mode is a pure vertical mode or a pure horizontal mode, a bilateral filter may not be applied.

[0118] (A14) In some implementations of any of A9 to A13, one or more criteria include: a first criterion that the intra-prediction mode is not a pure vertical mode or a neighboring mode, and a second criterion that the intra-prediction mode is not a pure horizontal mode or a neighboring mode. For example, if the current prediction mode is a near-vertical or horizontal mode (e.g., two adjacent modes of a pure vertical or a pure horizontal mode), then a bilateral filter may not be applied.

[0119] (A15) In some implementations of any of A9 through A14, one or more criteria include: the intra-prediction mode is not an angular mode. For example, if the current prediction mode is an angular mode (e.g., not planar and DC), a bilateral filter may not be applied. In some implementations, a bilateral filter is applied only when the intra-prediction mode is a non-directional intra-prediction mode.

[0120] (A16) In some embodiments of any of A1 to A15, a bilateral filter is applied to the current block when PDPC processing is not applied to the current block. For example, if PDPC is enabled, a bilateral filter is not applied. In some embodiments, a bilateral filter is not applied to the current block based on the determination that PDPC processing is applied to the current block. In some embodiments, a bilateral filter is applied to the current block when PDPC processing is not applied to the current block.

[0121] (A17) In some embodiments of any of A1 to A16, a bilateral filter is applied to the current block when a single reference line is used for intra-prediction. For example, if the intra-predictor is a fused signal predicted from multiple reference lines, no bilateral filter is applied. In some embodiments, a bilateral filter is applied to the current block based on the determination that a single reference line is used for intra-prediction. In some embodiments, a bilateral filter is not applied to the current block based on the determination that multiple reference lines are used for intra-prediction.

[0122] (A18) In some implementations of any of A1 through A17, applying a bilateral filter to the current block includes applying the bilateral filter only to a subset of the samples within the current block. For example, applying the bilateral filter to a portion of the predicted samples within the current prediction block. As an example, a bilateral filter may be applied to samples obtained via interpolation, but not to samples copied from a reference sample (or vice versa). As an example, no bilateral filter is applied when samples are copied from a reference sample instead of predicted using interpolation. Depending on the intra-frame mode with a fixed reference row position, samples from different rows within the current block will have different reference sample predictors. For the example 4×4 block, two rows of samples may be copied from the reference sample, while the other two rows of samples are generated using interpolation to produce predicted samples.

[0123] (B1) In another aspect, some implementations include a method for video encoding (e.g., method 650). In some implementations, the method is performed at a computing system (e.g., server system 112) having memory and control circuitry. In some implementations, the method is performed at an encoding / decoding module (e.g., encoding / decoding module 320). The method includes: (i) receiving video data (e.g., a source video sequence) including a current image comprising a plurality of blocks, the plurality of blocks including the current block; (ii) encoding the current block by applying intra-frame prediction to the current block; (iii) signaling the current block in the video bitstream; and (iv) signaling in the video bitstream whether a bilateral filter is applied to the current block.

[0124] (C1) In another aspect, some implementations include a method for processing visual media data. In some implementations, the method is performed at a computing system (e.g., server system 112) having memory and control circuitry. In some implementations, the method is performed at a codec module (e.g., codec module 320). The method includes: (i) obtaining a source video sequence comprising multiple frames; and (ii) performing a conversion between the source video sequence and a video bitstream of visual media data according to format rules. The video bitstream includes a current block. The format rules specify: (a) performing intra-frame prediction on the current block; (b) applying a bilateral filter to the current block; and (c) reconstructing the current block based on the output of the intra-frame prediction and the bilateral filter.

[0125] On the other hand, some implementations include a computing system (e.g., server system 112) that includes a control circuitry system (e.g., control circuitry system 302) and a memory (e.g., memory 314) coupled to the control circuitry system, the memory storing one or more sets of instructions configured to be executed by the control circuitry system, the set of one or more sets of instructions including instructions for performing any of the methods described herein (e.g., A1 to A18, B1 and C1 above).

[0126] In another aspect, some implementations include a non-transitory computer-readable storage medium that stores one or more sets of instructions for execution by a control circuitry of a computing system, the set of instructions including instructions for performing any of the methods described herein (e.g., A1 to A18, B1 and C1 above).

[0127] Unless otherwise stated, any syntax element (e.g., indicator) described herein can be a High-Level Syntax (HLS). As used herein, the HLS is signaled at a level higher than the block level. For example, the HLS can correspond to the sequence level, frame level, slice level, or tile level. As another example, HLS elements can be signaled in the Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), slice header, picture header, tile header, and / or CTU header.

[0128] It should be understood that while the terms “first,” “second,” etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the claims. As used in the description of embodiments and appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or,” as used herein, refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0129] As used herein, the term "if" may be interpreted, depending on the context, as meaning "when the prerequisite is true," "after the prerequisite is true," "in response to determining that the prerequisite is true," "based on the determination that the prerequisite is true," or "in response to detecting that the prerequisite is true." Similarly, the phrases "if it is determined [the prerequisite is true]," "if [the prerequisite is true]," or "when [the prerequisite is true]" may be interpreted, depending on the context, as meaning "after determining that the prerequisite is true," "in response to determining that the prerequisite is true," "based on the determination that the prerequisite is true," "after detecting that the prerequisite is true," or "in response to detecting that the prerequisite is true."

[0130] For illustrative purposes, the foregoing description has been described with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described to best illustrate the principles of operation and practical application, thereby enabling others skilled in the art to implement them.

Claims

1. A method for video decoding performed at a computing system having a memory and one or more processors, the method comprising: Receive a video bitstream including the current image, wherein the current image comprises multiple blocks, and the multiple blocks include the current block; Intra-frame prediction is performed on the current block of the plurality of coded blocks; Apply a bilateral filter to the current block; as well as The current block is reconstructed based on the intra-frame prediction and the output of the bilateral filter.

2. The method according to claim 1, wherein, The bilateral filter is applied before performing the intra-frame prediction.

3. The method according to claim 1, wherein, Performing the intra-frame prediction includes applying the location-related prediction combination (PDPC) technique.

4. The method according to claim 3, wherein, The bilateral filter is applied after the PDPC technique is performed.

5. The method according to claim 3, wherein, The bilateral filter is applied before performing the PDPC technique.

6. The method according to claim 1, wherein, When the current block has a block size that meets one or more criteria, the bilateral filter is applied to the current block.

7. The method according to claim 6, wherein, When the current block has a block size greater than a predetermined threshold, the bilateral filter is applied to the current block.

8. The method according to claim 6, wherein, When the current block has a block size less than a predetermined threshold, the bilateral filter is applied to the current block.

9. The method according to claim 1, wherein, The bilateral filter is applied to the current block when the intra-prediction mode used for the intra-prediction meets one or more criteria.

10. The method according to claim 9, wherein, The one or more criteria include those where the intra-frame prediction mode is not a DC mode.

11. The method according to claim 9, wherein, The one or more criteria include criteria that the intra-prediction mode is not a planar mode.

12. The method according to claim 9, wherein, The one or more criteria include: a first criterion that the intra-prediction mode is not a planar mode, and a second criterion that the intra-prediction mode is not a DC mode.

13. The method according to claim 9, wherein, The one or more criteria include: a first criterion that the intra-prediction mode is not a pure vertical mode, and a second criterion that the intra-prediction mode is not a pure horizontal mode.

14. The method according to claim 9, wherein, The one or more criteria include: a first criterion that the intra-prediction mode is not a pure vertical mode or a neighboring mode, and a second criterion that the intra-prediction mode is not a pure horizontal mode or a neighboring mode.

15. The method according to claim 9, wherein, The one or more criteria include criteria where the intra-prediction mode is not an angle mode.

16. The method according to claim 1, wherein, When PDPC processing is not applied to the current block, the bilateral filter is applied to the current block.

17. The method according to claim 1, wherein, When a single reference line is used for the intra-frame prediction, the bilateral filter is applied to the current block.

18. The method according to claim 1, wherein, Applying a bilateral filter to the current block includes applying the bilateral filter only to a subset of samples within the current block.

19. A method for video encoding performed at a computing system having a memory and one or more processors, the method comprising: Receive video data including the current image, wherein the current image comprises multiple blocks, and the multiple blocks include the current block; The current block is encoded by applying intra-frame prediction to the current block; The current block is notified by a signal in the video bitstream; as well as The video bitstream is signaled to indicate whether a bilateral filter should be applied to the current block.

20. A non-transitory computer-readable storage medium storing a video bitstream generated by a video coding method, the video coding method comprising: Receive video data including a current image, the current image comprising multiple blocks, the multiple blocks including the current block; and The current block is encoded by applying intra-frame prediction to the current block; as well as The video bitstream includes encoded information of the current block and an indicator indicating whether a bilateral filter is applied to the encoded information of the current block.

Images