Picture filtering method and system and computer readable medium

Abstract

The invention provides a picture filtering system and method. The method comprises the following steps: applying a time filter to a current picture; and encoding the current picture after applying the temporal filter. The application of the temporal filter to the current picture comprises: obtaining an exponential function; obtaining a weight of the at least one neighboring picture by multiplying the scaling function with the exponential function; obtaining a filtering sample value of the current picture based on the sample value of the current picture and the weight of the at least one adjacent picture; and replacing the sample value of the current picture with the filtered sample value. According to the invention, simplification of time filtering is realized.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. 63 / 034,042, filed on June 3, 2020, and U.S. Application No. 17 / 240,376, filed on April 26, 2021, the disclosures of which are incorporated herein by reference in their entireties. Technical Field

[0002] The present application relates to video encoding technology, and more specifically, to a method, system and computer-readable medium for image filtering. Background Art

[0003] AOMedia Video 1 (AV1) is an open, royalty-free video coding format designed for video transmission over the Internet, developed by the Alliance for Open Media (AOMedia) as the successor to VP9. AOMedia is an alliance established in 2015 that includes semiconductor companies, video-on-demand providers, video content producers, software development companies, and web browser vendors.

[0004] Libaom is the reference implementation of AV1. Scalable Video Technology AV1 (SVT-AV1) includes an open source encoder and decoder first released by Intel in February 2019.

[0005] Since the temporal filtering process is performed only once for each block, compared to the multiple coding modes and transform types tested during encoding, the total time spent on temporal filtering is only a small fraction of the total encoding time. However, when approaching a real-time encoder, most coding modes are not tested or skipped using the early decision process, and while temporal filtering continues to take roughly the same amount of time as before, it now becomes the majority of the total time. Most of the time spent in the temporal filtering process comes from the floating point calculations of the weights of the collocated pixels, which may also involve finding the value of an exponential function, which is a time-consuming process.

[0006] Therefore, since the previous weight calculation involves many floating-point calculations, the hardware implementation of the encoder is quite expensive and inefficient. Summary of the invention

[0007] Implementations of the present disclosure may provide solutions to the above-mentioned problems and / or other problems.

[0008] According to one or more embodiments, the present application provides a method for filtering a picture. The method is performed by an encoder, including: applying a temporal filter to a current picture, including: obtaining an exponential function; obtaining a weight of at least one adjacent picture by multiplying a scaling function with the exponential function; obtaining a filtered sample value of the current picture based on a sample value of the current picture and a weight of at least one adjacent picture; and replacing the sample value of the current picture with the filtered sample value. The method also includes encoding the current picture after applying the temporal filter.

[0009] According to an embodiment, obtaining the exponential function comprises: obtaining the exponential function by calculating the exponent of the exponential function as a fixed point value or by using at least one lookup table, wherein the exponential function comprises an exponent having a numerator and a denominator, the numerator having at least one first factor, and the denominator having at least one second factor.

[0010] According to an embodiment, obtaining the exponential function comprises calculating an exponent of the exponential function as a fixed point value.

[0011] According to an embodiment, calculating a fixed-point value includes representing the fixed-point value as 2 n bits, where n is an integer.

[0012] According to an embodiment, calculating the fixed point value comprises scaling a floating point value representation of a factor of the exponential function by the scaling factor.

[0013] According to an embodiment, obtaining the exponential function comprises using at least one look-up table.

[0014] According to an embodiment, the at least one lookup table comprises a first lookup table and a second lookup table, and the first lookup table comprises an integer part of an exponential function and the second lookup table comprises a fractional part of an exponential function.

[0015] According to an embodiment, at least one look-up table comprises fixed point values ​​of the exponential function.

[0016] According to an embodiment, the method further comprises converting factors of the scaling function into fixed point representation, wherein obtaining the weight of at least one neighboring picture comprises multiplying the fixed point representation of the scaling function with an exponential function.

[0017] According to an embodiment, the method further includes iteratively executing a pixel-by-pixel loop multiple times, wherein each iteration of the multiple iterations includes obtaining a filtered sample value of the current image for a corresponding pixel in the pixels of the current image; and before executing the multiple iterations of the pixel-by-pixel loop, converting to a fixed-point representation based on a factor in a scaling function or an exponential function that does not change for the pixel.

[0018] According to an embodiment, the scaling function comprises a numerator having at least one third factor and a denominator having at least one fourth factor, the at least one third factor comprising a noise level of the current picture, and the at least one fourth factor comprising a picture order count distance of at least one neighboring picture from the current picture.

[0019] According to one or more embodiments, the present application provides a picture filtering system. The system includes: at least one memory storing computer code; and at least one processor configured to access the computer code and operate according to the instructions of the computer code, the computer code including: a time filter code configured to enable at least one processor to apply a time filter to the current picture, the time filter code including: an exponential function acquisition code configured to enable at least one processor to obtain an exponential function; a weight acquisition code configured to enable at least one processor to obtain the weight of at least one adjacent picture by multiplying a scaling function with an exponential function; a filter sample value acquisition code configured to enable at least one processor to obtain a filter sample value of the current picture based on the sample value of the current picture and the weight of at least one adjacent picture; and a sample value replacement code configured to enable at least one processor to replace the sample value of the current picture with the filter sample value. The computer code may also include an encoding code configured to enable at least one processor to encode the current picture after applying the time filter.

[0020] According to an embodiment, an exponential function obtaining code is configured to cause at least one processor to obtain an exponential function by calculating the exponent of the exponential function as a fixed-point value or by using at least one lookup table, wherein the exponential function includes an exponent having a numerator and a denominator, the numerator having at least one first factor, and the denominator having at least one second factor.

[0021] According to an embodiment, the exponential function obtaining code is configured to cause at least one processor to calculate an exponent of the exponential function as a fixed point value.

[0022] According to an embodiment, the exponential function acquisition code is configured to cause at least one processor to represent the fixed point value as 2 n bits, where n is an integer.

[0023] According to an embodiment, the exponential function obtaining code is configured to cause at least one processor to scale floating point value representations of factors of the exponential function by the scaling factors.

[0024] According to an embodiment, the exponential function obtaining code is configured to cause at least one processor to obtain the exponential function by using at least one lookup table.

[0025] According to an embodiment, the at least one lookup table comprises a first lookup table and a second lookup table, and the first lookup table comprises an integer part of an exponential function and the second lookup table comprises a fractional part of an exponential function.

[0026] According to an embodiment, at least one look-up table comprises fixed point values ​​of the exponential function.

[0027] According to an embodiment, the computer code further includes: a conversion code configured to cause at least one processor to convert a factor of a scaling function into a fixed-point representation, wherein the weight acquisition code is configured to cause at least one processor to obtain a weight of at least one adjacent picture by multiplying the fixed-point representation of the scaling function with an exponential function.

[0028] According to an embodiment, the computer code also includes: a loop code, which includes a filtered sample value obtaining code and is configured to cause at least one processor to iteratively execute a pixel-by-pixel loop multiple times, each of the multiple iterations including obtaining the filtered sample value of the current image for a corresponding pixel in the pixels of the current image; and a conversion code, which is configured to cause at least one processor to convert to a fixed-point representation based on a factor in a scaling function or an exponential function that does not change for a pixel before executing multiple iterations of the pixel-by-pixel loop.

[0029] According to one or more embodiments, a non-transitory computer readable medium storing computer code is provided. The computer code is configured to, when executed by at least one processor, cause the at least one processor to perform any of the above methods.

[0030] It can be seen that the present application provides a method, system and computer-readable medium for image filtering. The method for image filtering includes: applying a time filter to the current image; and encoding the current image after applying the time filter. Applying the time filter to the current image includes: obtaining an exponential function; obtaining the weight of at least one adjacent image by multiplying a scaling function with the exponential function; obtaining a filtered sample value of the current image based on the sample value of the current image and the weight of at least one adjacent image; and replacing the sample value of the current image with the filtered sample value. In the present application, simplification of time filtering is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Further features, nature, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings, in which:

[0032] Figure 1 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment of the present application;

[0033] Figure 2 is a schematic diagram of a simplified block diagram of a communication system according to another embodiment of the present application;

[0034] Figure 3 is a schematic diagram of a simplified block diagram of a decoder according to an embodiment of the present application;

[0035] Figure 4 is a schematic diagram of a simplified block diagram of an encoder according to an embodiment of the present application;

[0036] Figure 5 is a diagram of a computer code according to an embodiment of the present application; and

[0037] Figure 6 is a diagram of a computer system suitable for embodiments of the present application. DETAILED DESCRIPTION

[0038] In this disclosure, the term block may refer to a prediction block, a coding block, or a coding unit (CU).

[0039] Figure 1 A simplified block diagram of a communication system (100) according to an embodiment of the present disclosure is shown. The system (100) may include at least two terminals (110, 120) interconnected via a network (150). For unidirectional data transmission, a first terminal (110) may encode video data at a local location for transmission to another terminal (120) via the network (150). A second terminal (120) may receive encoded video data from another terminal from the network (150), decode the encoded data and display the recovered video data. Unidirectional data transmission is common in media service applications and the like.

[0040] Figure 1 A second pair of terminals (130, 140) is shown, which are provided to support bidirectional transmission of encoded video, such as may occur during a video conference. For bidirectional data transmission, each terminal (130, 140) can encode video data captured at a local location for transmission to the other terminal via a network (150). Each terminal (130, 140) can also receive encoded video data sent by the other terminal, can decode the encoded data, and can display the recovered video data at a local display device.

[0041] exist Figure 1In the embodiment of the present invention, the terminals (110 to 140) can be shown as servers, personal computers and smart phones, and / or any other type of terminals. For example, the terminals (110 to 140) can be laptop computers, tablet computers, media players and / or dedicated video conferencing equipment. The network (150) represents any number of networks that transmit coded video data between the terminals (110 to 140), including, for example, wired and / or wireless communication networks. The communication network (150) can exchange data in circuit switching and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks and / or the Internet. For the purpose of this discussion, unless explained below, the architecture and topology of the network (150) may be unimportant for the operation of the present disclosure.

[0042] Figure 2 The placement of a video encoder and decoder in a streaming media environment is shown as an example of application of the disclosed subject matter. The disclosed subject matter can be equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0043] like Figure 2 As shown in , a streaming media system (200) may include a capture subsystem (213), which may include a video source (201) and an encoder (203). The video source (201) may be, for example, a digital video camera and may be configured to create an uncompressed video sample stream (202). The uncompressed video sample stream (202) may provide a high data volume when compared to an encoded video bitstream and may be processed by an encoder (203) coupled to the camera (201). The encoder (203) may include hardware, software, or a combination thereof to implement or implement various aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream (204) may include a lower data volume when compared to a sample stream and may be stored on a streaming media server (205) for future use. One or more streaming media clients (206) may access the streaming media server (205) to retrieve a video bitstream (209) that may be a copy of the encoded video bitstream (204).

[0044] In an embodiment, the streaming server (205) may also function as a Media-Aware Network Element (MANE). For example, the streaming server (205) may be configured to prune the encoded video bitstream (204) for use in customizing a potentially different bitstream for one or more of the streaming clients (206). In an embodiment, the MANE may be provided separately from the streaming server (205) in the streaming system (200).

[0045] The streaming client (206) may include a video decoder (210) and a display (212). The video decoder (210) may, for example, decode a video bitstream (209) that is an incoming copy of an encoded video bitstream (204) and create an outgoing video sample stream (211) that may be rendered on a display (212) or another rendering device (not depicted). In some streaming systems, the video bitstreams (204, 209) may be encoded according to certain video encoding / compression standards. Examples of such standards include, but are not limited to, ITU-T Recommendation H.265. Under development is a video coding standard informally referred to as Versatile Video Coding (VVC). Implementations of the present disclosure may be used in the context of VVC.

[0046] Figure 3 An example functional block diagram of a video decoder (210) attached to a display (212) is shown in accordance with an embodiment of the present disclosure.

[0047] The video decoder (210) may include a channel (312), a receiver (310), a buffer memory (315), an entropy decoder / parser (320), a scaler / inverse transform unit (351), an intra-frame prediction unit (352), a motion compensated prediction unit (353), an aggregator (355), a loop filter unit (356), a reference picture memory (357), and a current picture memory (358). In at least one embodiment, the video decoder (210) may include an integrated circuit, a series of integrated circuits, and / or other electronic circuit systems. The video decoder (210) may also be implemented in part or in whole in software running on one or more CPUs with associated memory.

[0048] In this and other embodiments, the receiver (310) may receive one or more encoded video sequences to be decoded by the decoder (210) one at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequence may be received from a channel (312), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (310) may receive the encoded video data and other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective consuming entities (not depicted). The receiver (310) may separate the encoded video sequence from the other data. To combat network jitter, a buffer memory (315) may be coupled between the receiver (310) and an entropy decoder / parser (320) (hereinafter referred to as the "parser"). When the receiver (310) receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous synchronous network, the buffer (315) may not be used or may be very small. For use over a best-effort packet network such as the Internet, a buffer (315) may be required, which may be relatively large and may have an adaptive size.

[0049] The video decoder (210) may include a parser (320) to reconstruct symbols (321) from an entropy coded video sequence. The types of these symbols include, for example, information for managing the operation of the decoder (210) and possibly controlling a rendering device such as Figure 2, which can be coupled to a display (212) of a decoder. The control information of the rendering device can be in the form of, for example, a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not depicted). The parser (320) can parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence can be based on a video coding technology or standard, and can follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (320) can extract a set of subgroup parameters for at least one of the pixel subgroups in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. The subgroup can include Groups of Pictures (GOP), pictures, blocks, slices, macroblocks, Coding Units (CU), blocks, Transform Units (TU), Prediction Units (PU), etc. The parser (320) may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0050] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer (315), thereby creating symbols (321).

[0051] Depending on the type of coded video pictures or parts thereof (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors, the reconstruction of the symbol (321) may involve a number of different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the coded video sequence by the parser (320). For clarity, the flow of such subgroup control information between the parser (320) and the following multiple units is not depicted.

[0052] In addition to the functional blocks already mentioned, decoder 210 can be conceptually subdivided into a number of functional units, as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purpose of describing the disclosed subject matter, it is appropriate to conceptually subdivide into the following functional units.

[0053] One unit may be a scaler / inverse transform unit (351). The scaler / inverse transform unit (351) may receive quantized transform coefficients and control information including which transform to use, block size, quantization factor, quantization scaling matrix, etc. as symbols (321) from the parser (320). The scaler / inverse transform unit (351) may output a block including sample values, which may be input into an aggregator (355).

[0054] In some cases, the output samples of the scaler / inverse transform (351) may belong to an intra-coded block, i.e., a block that does not use prediction information from a previously reconstructed picture, but may use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) uses surrounding already reconstructed information obtained from the current (partially reconstructed) picture from the current picture memory (358) to generate a block of the same size and shape as the block being reconstructed. In some cases, the aggregator (355) adds the prediction information already generated by the intra-prediction unit (352) to the output sample information as provided by the scaler / inverse transform unit (351) on a per-sample basis.

[0055] In other cases, the output samples of the scaler / inverse transform unit (351) may belong to an inter-frame coded and possibly motion compensated block. In this case, the motion compensated prediction unit (353) may access the reference picture memory (357) to obtain samples for prediction. After motion compensation of the obtained samples according to the block-related symbols (321), these samples may be added by the aggregator (355) to the output of the scaler / inverse transform unit (351) (in this case referred to as residual samples or residual signal), thereby generating output sample information. The address within the reference picture memory (357) from which the motion compensated prediction unit (353) obtains the predicted samples may be controlled by a motion vector. The motion vector may be available to the motion compensated prediction unit (353) in the form of a symbol (321) that may have, for example, X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​such as those obtained from the reference picture memory (357) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, and the like.

[0056] The output samples of the aggregator (355) may be subjected to various loop filtering techniques in a loop filter unit (356). The video compression techniques may include in-loop filter techniques controlled by parameters included in the coded video bitstream and provided to the loop filter unit (356) as symbols (321) from the parser (320), but may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of a coded picture or coded video sequence, and to previously reconstructed and loop filtered sample values.

[0057] The output of the loop filter unit (356) may be a sample stream that may be output to a rendering device such as a display (212) and stored in a reference picture memory (357) for use in future inter-picture prediction.

[0058] Certain coded pictures, once fully reconstructed, can be used as reference pictures for future predictions. Once a coded picture is fully reconstructed and the coded picture has been identified as a reference picture (e.g., by the parser (320)), the current reference picture can become part of the reference picture memory (357) and a new current picture memory can be reallocated before starting to reconstruct subsequent coded pictures.

[0059] The video decoder (210) may perform decoding operations according to a predetermined video compression technique, which may be documented in a standard such as ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax specified by the video compression technique or standard being used, in the sense that it adheres to the syntax of the video compression technique or standard as specified in the video compression technique document or standard, particularly in the configuration document therein. In addition, in order to conform to certain video compression techniques or standards, the complexity of the encoded video sequence may be within a range as defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (e.g., measured in megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the level can be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management that is signaled in the encoded video sequence.

[0060] In an embodiment, a receiver (310) may receive additional (redundant) data and the encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (210) to correctly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0061] Figure 4 An example functional block diagram of a video encoder (203) associated with a video source (201) is shown according to an embodiment of the present disclosure.

[0062] The video encoder (203) may include, for example, an encoder, i.e., a source encoder (430), a coding engine (432), a (local) decoder (433), a reference picture memory (434), a predictor (435), a transmitter (440), an entropy encoder (445), a controller (450), and a channel (460).

[0063] The encoder (203) may receive video samples from a video source (201) (which is not part of the encoder) that may capture video images to be encoded by the encoder (203).

[0064] The video source (201) can provide a source video sequence to be encoded by the encoder (203) in the form of a digital video sample stream, which can be any suitable bit depth (e.g., 8 bits, 10 bits, 12 bits, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...) and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (201) can be a storage device that stores previously prepared videos. In a video conferencing system, the video source (201) can be a camera that captures local image information as a video sequence. The video data can be provided as multiple separate pictures that convey motion when viewed in sequence. The pictures themselves can be organized as a spatial array of pixels, where each pixel can include one or more samples depending on the sampling structure, color space, etc. used. The relationship between pixels and samples can be easily understood by those skilled in the art. The following description focuses on samples.

[0065] According to an embodiment, the encoder (203) can encode and compress the pictures of the source video sequence into an encoded video sequence in real time or under any other time constraints as required by the application. Implementing an appropriate encoding speed is a function of the controller (450). The controller (450) can also control other functional units as described below and can be functionally coupled to these units. For clarity, the coupling is not depicted. The parameters set by the controller (450) may include parameters related to rate control (picture jump, quantizer, λ (lambda) value of rate distortion optimization technology, ...), picture size, picture group (group of pictures, GOP) layout, maximum motion vector search range, etc. Those skilled in the art can easily identify other functions of the controller (450) because these functions may be related to the video encoder (203) optimized for a specific system design.

[0066] Some video encoders operate in a manner that is readily recognizable to those skilled in the art as a "coding loop". As an overly simplified description, the coding loop may consist of the encoding portion of a source encoder (430) (responsible for creating symbols based on the input picture to be encoded and the reference picture) and a (local) decoder (433) embedded in the encoder (203), which reconstructs the symbols to create sample data, and the (remote) decoder will also create sample data when the compression between the symbols and the encoded video bitstream is lossless in certain video compression techniques. This reconstructed sample stream may be input to a reference picture memory (434). Since the decoding of the symbol stream results in bit-accurate results that are independent of the decoder location (local or remote), the reference picture memory contents are also bit-accurate between the local encoder and the remote encoder. In other words, the prediction portion of the encoder "sees" the reference picture samples as exactly the same sample values ​​that the decoder will "see" when using prediction during decoding. This basic principle of reference picture synchronization (and drift if synchronization cannot be maintained, such as due to channel errors) is known to those skilled in the art.

[0067] The operation of the "local" decoder (433) can be combined with the above Figure 3 The operation of the "remote" decoder (210) described in detail is the same. However, since the symbols are available and the symbol encoding / decoding of the encoded video sequence by the entropy encoder (445) and the parser (320) may be lossless, the entropy decoding portion of the decoder (210) including the channel (312), the receiver (310), the buffer (315) and the parser (320) may not be fully implemented in the local decoder (433).

[0068] An observation that can be made at this point is that, in addition to the parsing / entropy decoding present in the decoder, any decoder technology may need to be present in a substantially identical functional form in the corresponding encoder. For this reason, the disclosed subject matter focuses on the encoder operation. The description of the decoder technology may be abbreviated because it may be contrary to the fully described encoder technology. A more detailed description is only required in certain areas and is provided below.

[0069] As part of its operation, the source encoder (430) may perform motion compensated predictive coding, which predictively encodes an input frame with reference to one or more previously encoded frames from a video sequence, which are designated as "reference frames." In this manner, the encoding engine (432) encodes the differences between pixel blocks of the input frame and pixel blocks of a reference frame, which may be selected as a prediction reference for the input frame.

[0070] The local video decoder (433) can decode the encoded video data of the frame that can be designated as the reference frame based on the symbols created by the source encoder (430). The operation of the encoding engine (432) can advantageously be a lossy process. When the encoded video data can be decoded at the video decoder ( Figure 4 When decoded at a local video decoder (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (433) replicates the decoding process that may be performed by the video decoder on the reference frame and may cause the reconstructed reference frame to be stored in the reference picture memory (434). In this way, the encoder (203) may store a copy of the locally reconstructed reference frame with common content as the reconstructed reference frame that will be obtained by the remote video decoder (in the absence of transmission errors).

[0071] The predictor (435) may perform a prediction search for the encoding engine (432). That is, for a new frame to be encoded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or certain metadata such as reference picture motion vectors, block shapes, etc. that may be used as appropriate prediction references for the new picture. The predictor (435) may operate on a sample block-pixel block basis to find an appropriate prediction reference. In some cases, as determined by the search results obtained by the predictor (435), the input picture may have prediction references taken from multiple reference pictures stored in the reference picture memory (434).

[0072] The controller (450) can manage encoding operations of the video encoder (430), including, for example, setting parameters and sub-group parameters for encoding video data.

[0073] The outputs of all the aforementioned functional units may be entropy encoded in an entropy encoder (445). The entropy encoder converts the symbols, as generated by the various functional units, into a coded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc.

[0074] The transmitter (440) can buffer the encoded video sequence as created by the entropy encoder (445) in preparation for transmission via a communication channel (460), which can be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (440) can combine the encoded video data from the video encoder (430) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0075] The controller (450) may manage the operation of the encoder (203). During encoding, the controller (450) may assign a specific encoding picture type to each encoding picture, which may affect the encoding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned as an intra picture (I picture), a predicted picture (P picture), or a bidirectionally predicted picture (B picture).

[0076] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of those variations of I pictures and their respective applications and features.

[0077] A prediction picture (P picture) may be a picture that may be encoded and decoded using intra prediction or inter prediction that predicts sample values ​​of each block using at most one motion vector and a reference index.

[0078] Bidirectional prediction pictures (B pictures) can be pictures that can be encoded and decoded using intra prediction or inter prediction using up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple prediction pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0079] A source picture may typically be spatially subdivided into a plurality of blocks of samples (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and encoded on a block-by-block basis. A block may be predictively encoded with reference to other (already encoded) blocks as determined by the coding assignment applied to the block's corresponding picture. For example, a block of an I picture may be non-predictively encoded, or it may be predictively encoded with reference to already encoded blocks of the same picture (spatial prediction or intra prediction). A pixel block of a P picture may be non-predictively encoded with reference to one previously encoded reference picture via spatial prediction or via temporal prediction. A block of a B picture may be non-predictively encoded with reference to one or two previously encoded reference pictures via spatial prediction or via temporal prediction.

[0080] The video encoder (203) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T Recommendation H.265. In its operation, the video encoder (203) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard being used.

[0081] In an embodiment, the transmitter (440) may send additional data along with the encoded video. The video encoder (430) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0082] Temporal filtering is a technique that attempts to denoise the current frame using temporal redundancy with neighboring frames. It is applied to Versatile Video Coding and Test Model 6 (VTM6) as test models for the Versatile Video Coding (VVC) codec (as described in JVET-O0549), and to libaom and SVT-AV1 as encoders for the AV1 codec. The exact implementation of the filter varies from encoder to encoder.

[0083] During mode decision, the encoder may try to minimize the rate*λ+distortion metric for each block.

[0084] If the current picture is followed by the same subsequent picture, the coding block in the current picture can be skipped. In this case, the rate *λ+2* distortion can be minimized. VTM approximates this value for random access (RA) by setting different quantization parameter (QP) values ​​for the temporal layers.

[0085] If the subsequent pictures are only slightly different, the coded block will still likely be skipped. In this case, rate*λ+distortion1+distortion2 can be minimized. Instead of computing the two distortions separately, the distortion between the coded block and the average of the original versions of the two pictures can be computed. This is one motivation for using temporally filtered original pictures. If the original images are different, the filter strength may drop rapidly, since it will no longer be possible to reuse the decoded sample values ​​of the first picture for the second picture.

[0086] The general application process of the temporal filter in an encoder may be as follows.

[0087] Step 1: The encoder reads the image.

[0088] Step 2: If the picture is low enough in the coding hierarchy, it is filtered before encoding. Otherwise, the picture is encoded without filtering.

[0089] Step 3: If it is determined that the temporal filter is to be applied to the current frame, motion estimation and motion compensation methods are applied to the neighboring pictures. The purpose of this step is to find the closest match to the current block in each of the neighboring frames.

[0090] Step 4: Once all similar blocks from each of the adjacent frames are collected, all sample values ​​of the original picture are replaced by the filtered sample values ​​I by the encoder before encoding by using the following equation (1): n : Among them, I o is the original sample value, I r (i) is the co-located sample value in the adjacent picture i after motion compensation, and w r (i, a) is the weight of neighbor picture i when the number of available neighbor pictures is equal to a.

[0091] The weight w is calculated by the encoder using the following equation (2): r (i, a): Here, e can be an exponential constant.

[0092] ΔI(i) is calculated by using the following equation (3): ΔI(i)=I r (i)-I o Equation (3)

[0093] The function f calculates the factor in the exponential based on ΔI, and the specific function varies between implementations. n and x d are other factors that modify the weights, which may depend on other pixel-level features, such as the local mean ME error, or may depend on block / frame-level features, such as QP. Finally, S is a scaling factor for the final weight calculation.

[0094] Although the temporal filter implementation may be performed as described above, the details of the implementation may vary. The implementation varies primarily in the following points:

[0095] (A) Applicable frames: Although implementations can apply temporal filtering to lower layers in the Group of Pictures (GOP) hierarchy, there are slight variations in how it is determined. For example, in VTM, Random-Access (RA) pictures with picture order count (POC) % 8 == 0 and Low-Delay (LD) pictures with POC % 4 == 0 are filtered. All-Intra (AI) pictures are never filtered. In addition, in Libaom and SVT-AV1, all key frames and ALTREF frames are temporally filtered.

[0096] (B) Number of reference frames: The number of adjacent frames used for the temporal filtering process may vary from encoder to encoder. VTM can use 4 frames in RA coding conditions and 2 reference frames in LD coding configuration. The libaom and SVT encoders make the number of reference frames configurable. However, the default value can be set to 7 reference frames.

[0097] (C) The size of the block used: VTM can perform motion estimation (ME), motion compensation (MC) and filtering on an 8x8 block basis, while libaom and SVT can perform processing on a 32x32 block basis. If the decoder detects that the ME error is too large, both Libaom and SVT may switch to 16x16 level processing.

[0098] (D) Motion Estimation: Implementations may differ in the number of hierarchical motion estimation levels, whether the search starts from a subsampled reference frame and the current frame, the minimum fraction of pixels for which motion estimation is performed, and the filters used for motion compensation.

[0099] (E) Weight calculation method: The weight assigned to each of the juxtaposed pixels before averaging is the main area where implementations may differ. The weight calculation can be performed as a function of various factors. Although the motion estimation error of the current pixel is a commonly used factor in implementations, more factors can be used to change the resulting weights. For example, in VVC, the ME error and QP of the current frame can be used to determine the weights. In Libaom, the ME error of the window around the current pixel, the noise level of the current frame, and the amplitude of the motion vector can be used to determine the weight. In SVT-AV1, the ME error of the window around the current pixel and the noise level of the current frame can be used to determine the weight.

[0100] Although there are differences in the factors used, implementations can determine the weights using equation (4) below:

[0101] The terms used in the equations have been described previously. Since at least one of the factors (e.g., ME error of the pixel / average ME error of the window around the current pixel) varies from pixel to pixel, the weight for each pixel is unique and may need to be calculated repeatedly.

[0102] The embodiments of the present disclosure may be used alone or in any order. In addition, each of the embodiments (e.g., methods, encoders, and decoders) may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transient computer-readable medium. The embodiments of the present disclosure may implement various aspects of the temporal filtering described above and below.

[0103] Implementations of the present disclosure may implement not only the techniques described in the present disclosure for temporal filtering, but also the techniques described in the present disclosure for any method including exponential calculation as described below.

[0104] Embodiments of the present disclosure may implement a weight derivation equation for calculating a weight w. The general form of the weight derivation equation for calculating a weight w is shown below in equation (5):

[0105] where e can be an exponential constant, and s n1 、s n2 、s d1、s d2 、x n1 、x n2 、x d1 、x d2 is a real number. Depending on the implementation, the number of factors may vary. For example, in an implementation, only s is included in the equation n1 、s d1 、x n1 、x d1 In another embodiment, s may be included in the equation n1 、s n2 、s n3 、s d1 、s d2 、s d3 、x n1 、x n2 、x n3 、x d1 、x d2 、x d3 .

[0106] The reasons for having the general form in this way are as follows:

[0107] (A) The term in the numerator of an exponential (e.g., x n1 、x n2 , ...) are factors that, if they become larger, reduce the final weight of the calculation. For example, one such factor may be the motion estimation error. If the motion estimation error is larger, the weight should be smaller, so the motion estimation error related factor is placed in the numerator. There may be multiple such factors, so there may be multiple factors in the numerator of the general form.

[0108] (B) The term in the denominator of an exponential (e.g., x d1 、x d2 , ...) are factors that increase the final weight of the calculation if they become larger. For example, one such factor may be a quantization parameter (QP). If the quantization parameter (QP) is larger, the weight should be larger, so the QP factor related parameters are placed in the denominator. There may be multiple such factors, so there may be multiple factors in the denominator in the general form.

[0109] (C) Numerator term s n1 、s n2, ... may contain the final scaling factor. The final weight may be scaled because an exponential function may result in values ​​less than 1, and scaling may need to be performed in order to operate with fixed-point precision. These terms may also contain parameters that may linearly affect the calculated final weight. For example, one parameter may be the noise level of the frame being filtered, where the weight may be increased proportionally to the amount of noise. Since there may be multiple such parameters and since the scaling factor is also included in the numerator, the general form may include multiple factors in the numerator.

[0110] (D) The term s in the denominator d1 、s d2 , ... may contain factors that may inversely affect the final weight. Such factors may include, for example, the POC distance of the reference frame from the current frame. In this case, the larger the POC distance, the lower the weight that can be assigned to the reference frame. Since there may be multiple such parameters, the denominator may include multiple factors in the general form.

[0111] According to an embodiment, the encoder may convert the exponent of the exponential function (eg, the term x) into n1 、x n2 , … and x d1 、x d2 , …) are computed as fixed-point values ​​rather than floating-point values.

[0112] Depending on the implementation, fixed-point representation can use 2 n bits, where n is an integer. The value n can be chosen so that the fixed-point value does not overflow 2 n .

[0113] In another embodiment, the fixed-point representation may use a scaling factor so that the base floating-point value can be obtained by dividing by the scaling factor. The scaling factor may also be selected as a power of 2 so that the scaling operation can be performed by a simple binary logical shift operation. For example, the scaling factor may be equal to 2. 5 . We can calculate x n1_固定 =x n1 ×2 5 and round it to the nearest integer to convert the floating point value x n1 Written as a fixed-point value. x n1 With another integer x n2 The multiplication can be performed as (x n1_固定 × n2 )>>5.

[0114] In one embodiment, the scaling factor may be selected such that scaling up and down to obtain the underlying floating point value does not result in a significant loss of precision in the value.

[0115] In another embodiment, the scaling factors can be selected so that upscaling operations of floating points to obtain fixed-point representations and performing calculations in the fixed-point representations does not significantly degrade the performance of the encoder. For example, each of the factors used to calculate the exponent of the exponential function (e.g., average motion estimation error, noise level of a frame, and magnitude of motion vectors) can itself be converted to fixed-point representations.

[0116] According to an embodiment, the encoder may use a lookup table to obtain the exponential function instead of using the above calculations.

[0117] According to an embodiment, two lookup tables may be used to calculate the exponential function, one for the fractional portion of the input value and one for the integer portion of the input value, since e 整数+小数 =e 整数 ×e 小数 .

[0118] In one embodiment, the lookup table may contain fixed point values ​​obtained by appropriately scaling the floating point values.

[0119] In one embodiment, e x can be equivalently converted to 2 x.log2e , which can also be written as 2 整数 2 小数 . 2 小数 part can be calculated using a lookup table, while 2 整数 Parts can be obtained using logical shift operations.

[0120] In an embodiment, each value stored in the lookup table may be no more than N bits wide, so that a single instruction / multiple data (SIMD) implementation of the lookup operation may be performed efficiently. For example, N may be equal to 8, and for Advanced Vector Extensions (AVX) and Advanced Vector Extensions 2 (AVX2) instruction sets, SIMD lookups may be performed efficiently.

[0121] In an embodiment, the size of the lookup table may be no more than N values ​​long, so that a SIMD implementation of the lookup operation may be performed efficiently. For example, N may be equal to 16, and for AVX and AVX2 instruction sets, a SIMD lookup may be performed efficiently.

[0122] According to an embodiment, the encoder may multiply / divide the value s by an exponential function before the multiplication to obtain the final weight value s for a given collocated pixel. n1 、s n2 , ... and s d1 、sd2 , …are converted to fixed-point values ​​in order to avoid floating-point operations. Converting the points of these values ​​during calculations may change the final performance.

[0123] In an embodiment, the scaling factor used to convert a floating point value to a fixed point representation may be a power of 2, such that the scaling operation may be performed by a simple binary logical shift operation.

[0124] In an embodiment, the value s n1 、s n2 , ... and s d1 、s d2 , ... can be directly incorporated into the lookup table of the exponential function so that no further multiplication is required after looking up the value from the table. As an example, this can be done by factoring The merge into the lookup table is performed by multiplying the existing value in the lookup table. In the example, the value in the lookup table can be merged into n1 、s n2 , ... and s d1 、s d2 , …are then converted to fixed-point values.

[0125] According to an embodiment, the encoder can be used by using s n1 、s n2 , ... and s d1 、s d2 , ... to calculate the effective value and then s n1 、s n2 , ... and s d1 、s d2 , … to fixed point. This fixed point value can be used directly during the weight calculation for each pixel, rather than being converted to fixed point later during the weight calculation for each pixel. Converting the point of these values ​​during calculation may change the final performance. Converting these values ​​to fixed point outside of the per-pixel loop saves redundant calculations and is therefore more efficient.

[0126] In an embodiment, the exponential factor (x) of the exponential function that remains constant for the pixel block may be pre-calculated. n1 、x n2 , … and x d1 、x d2 , ...) and can store the resulting values. When calculating the weight of each pixel, x cannot be n1 、x n2 , … and x d1 、x d2 , … are multiplied / divided by each of them, and only those that change for each pixel can be multiplied / divided by the combined value of the invariant factors.

[0127] In an embodiment, the scaling factor used to convert a floating point value to a fixed point representation may be a power of 2, such that the scaling operation may be performed by a simple binary logical shift operation.

[0128] In an embodiment, by using x n1 、x n2 , … and x d1 、x d2 , ... to calculate the effective value and then convert x n1 、x n2 , … and x d1 、x d2 , … are converted to fixed point. This fixed point value can be used directly during the weight calculation for each pixel, rather than being converted to fixed point later during the weight calculation for each pixel. Converting the point of these values ​​during calculation may change the final performance. Converting these values ​​to fixed point outside of the per-pixel loop saves redundant calculations and is therefore more efficient.

[0129] The embodiments of the present disclosure may include at least one processor and a memory storing computer code. The computer code may be configured to enable at least one processor to perform the functions of the embodiments of the present disclosure when executed by at least one processor. The encoder (or decoder) may include at least one processor and a memory. The encoder (or decoder) may perform the functions of the embodiments described herein.

[0130] For example, refer to Figure 5 , the encoder (500) of the present disclosure may include at least one processor and a memory storing computer code. The computer instructions may include a temporal filter code (510) and an encoding code (520).

[0131] The temporal filter code (510) may be configured to cause at least one processor to apply a temporal filter to the current picture, as described in relation to one or more embodiments of the present disclosure. As an example, the temporal filter code (510) may include an exponential function acquisition code (511), a scaling function acquisition code (512), a weight acquisition code (513), a filtered sample value acquisition code (514), and a sample value replacement code (515).

[0132] The exponential function obtaining code (511) may be configured to cause at least one processor to obtain an exponential function by calculating an exponent of the exponential function as a fixed point value and / or by using at least one lookup table, the exponential function including an exponent having a numerator and a denominator, the numerator having at least one first factor (e.g., x n1 、x n2 , ...), the denominator has at least one second factor (e.g., x d1 、xd2 , ...), as described with respect to one or more embodiments of the present disclosure.

[0133] The scaling function obtaining code (512) may be configured to cause at least one processor to obtain a scaling function by calculating factors of the scaling function as fixed point values ​​and / or by using at least one lookup table, the scaling function including an exponent having a numerator and a denominator, the numerator having at least one third factor (e.g., s n1 、s n2 , ...), the denominator has at least one fourth factor (e.g., s d1 、s d2 , ...), as described with respect to one or more embodiments of the present disclosure.

[0134] The weight obtaining code (513) may be configured to cause at least one processor to obtain the weight of at least one neighboring picture by multiplying a scaling function by an exponential function (eg, refer to equation (5)), as described with respect to one or more embodiments of the present disclosure.

[0135] The filtered sample value obtaining code (514) can be configured to cause at least one processor to obtain the filtered sample value of the current picture based on the sample value of the current picture and the weight of at least one adjacent picture, as described with respect to one or more embodiments of the present disclosure (e.g., with reference to equation (1)).

[0136] The sample value replacement code (515) may be configured to cause at least one processor to replace sample values ​​of the current picture with filtered sample values, as described with respect to one or more embodiments of the present disclosure.

[0137] According to an embodiment, the temporal filter code (510) may include a loop code configured to iteratively execute one or more of the exponential function acquisition code (511), the scaling function acquisition code (512), the weight acquisition code (513), the filtered sample value acquisition code (514), and the sample value replacement code (515) for each pixel in the current picture in whole or in part. According to one or more embodiments, one or more of the exponential function acquisition code (511) and the scaling function acquisition code (512) may include a conversion code for converting the value of the exponential function and / or the scaling function from a floating point representation to a fixed point representation, and the conversion code may be included inside or outside the loop code.

[0138] The encoding code (520) may be configured to cause at least one processor to encode the current picture after applying the temporal filter, as described with respect to one or more embodiments of the present disclosure.

[0139] According to one or more embodiments, the computer code may also include a lookup table code (530) that may be configured to cause at least one processor to obtain one or more lookup tables, as described with respect to embodiments of the present disclosure. For example, according to an embodiment, the lookup table code (530) may be configured to cause at least one processor to receive a lookup table and / or construct a lookup table including values ​​therein.

[0140] The techniques of the embodiments of the present disclosure described above may be implemented as computer software that uses computer-readable instructions and is physically stored in one or more computer-readable media. For example, Figure 6 A computer system (900) suitable for implementing embodiments of the disclosed subject matter is shown.

[0141] Computer software may be encoded using any suitable machine code or computer language that may be assembled, compiled, linked or similarly constructed to create code comprising instructions that may be executed directly by a computer central processing unit (CPU), graphics processing unit (GPU), etc., or through interpretation, microcode execution, etc.

[0142] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0143] Figure 6 The components for the computer system (900) shown in the example are exemplary in nature and are not intended to imply any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Nor should the configuration of components be interpreted as having any dependency or requirement related to any one or combination of components shown in the exemplary embodiment of the computer system (900).

[0144] The computer system (900) may include certain human interface input devices. Such human interface input devices may be responsive to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), olfactory input (not depicted). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0145] Input human interface devices may include one or more of the following (only one of each is depicted): keyboard (901), mouse (902), trackpad (903), touch screen (910), data gloves, joystick (905), microphone (906), scanner (907), and camera (908).

[0146] The computer system (900) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate one or more senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen (910), a data glove, or a joystick (905), but there may also be tactile feedback devices that are not used as input devices). For example, such devices may be audio output devices (e.g., speakers (909), headphones (not depicted)), visual output devices (e.g., screens (910) including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which are capable of outputting two-dimensional visual output or more than three-dimensional output through devices such as stereo output, virtual reality glasses (not depicted), holographic displays, and smoke generators (not depicted)), and printers (not depicted).

[0147] The computer system (900) may also include human- and machine-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (920) with CD / DVD and other media (921), thumb drives (922), removable hard drives or solid-state drives (923), traditional magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD-based devices such as security dongles (not depicted), etc.

[0148] Those skilled in the art should also understand that the term "computer-readable media" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0149] The computer system (900) may also include an interface to one or more communication networks. For example, the network may be wireless, wired, optical. The network may also be local, wide area, urban, vehicle and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicle and industrial networks including CANBus, etc. Some networks typically require an external network interface adapter attached to some common data port or peripheral bus (949) (e.g., a USB port of the computer system (900)); other networks are typically integrated into the core of the computer system 900 by attaching to a system bus as described below (e.g., an Ethernet interface in a PC computer system or a cellular network interface in a smart phone computer system). Using any of these networks, the computer system (900) can communicate with other entities. Such communications may be one-way receive-only (e.g., broadcast TV), one-way send-only (e.g., CANbus to certain CANbus devices), or two-way, such as to other computer systems using a local or wide area digital network. Such communications may include communications to a cloud computing environment (955). Certain protocols and protocol stacks may be used on each of these networks and network interfaces as described above.

[0150] The aforementioned human interface devices, human-accessible storage devices, and network interfaces (954) may be attached to the core (940) of the computer system (900).

[0151] The core (940) may include one or more central processing units (CPUs) (941), graphics processing units (GPUs) (942), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (943), hardware accelerators for certain tasks (944), and the like. These devices, along with read-only memory (ROM) (945), random access memory (946), internal mass storage devices such as internal non-user accessible hard drives, SSDs, and the like (947), may be connected via a system bus (948). In some computer systems, the system bus (948) may be accessed in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, and the like. Peripheral devices may be attached to the core's system bus (948) directly or via a peripheral bus (949). The architecture of the peripheral bus includes PCI, USB, and the like. A graphics adapter 950 may be included in the core 940.

[0152] The CPU (941), GPU (942), FPGA (943), and accelerator (944) can execute certain instructions, which can be combined to form the above-mentioned computer code. The computer code can be stored in ROM (945) or RAM (946). Transition data can also be stored in RAM (946), while permanent data can be stored in, for example, an internal mass storage device (947). Fast storage and retrieval of any of the memory devices can be achieved by using a cache memory, which can be closely associated with one or more CPUs (941), GPUs (942), mass storage devices (947), ROMs (945), RAMs (946), etc.

[0153] The computer readable medium may have computer codes for executing various computer-implemented operations. The media and computer codes may be those specially designed and constructed for the purpose of the present disclosure, or they may be those well known and available to those skilled in the art of computer software.

[0154] As an example and not by way of limitation, a computer system having an architecture (900), particularly a core (940), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software contained in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage device introduced above, as well as certain storage devices of the core (940) having a non-transient nature, such as a core internal mass storage device (947) or ROM (945). Software implementing various embodiments of the present disclosure can be stored in such a device and executed by the core (940). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can enable the core (940), particularly the processor therein (including a CPU, GPU, FPGA, etc.) to perform a specific process or a specific part of a specific process described herein, including defining a data structure stored in the RAM (946) and modifying such a data structure according to a process defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (944)), which logic may operate in place of or in conjunction with software to perform a particular process or particular portion of a particular process described herein. Where appropriate, references to software may include logic, and conversely, references to logic may include software. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry embodying logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0155] Although the present disclosure has described several non-limiting example embodiments, there are changes, permutations, and various substitute equivalents that fall within the scope of the present disclosure. Therefore, it should be understood that those skilled in the art will be able to devise many systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope.

Claims

1. A method for image filtering, characterized in that: The method comprises: Apply temporal filters to the current image, including: obtaining an exponential function including a numerator having one or more factors corresponding to a motion estimation error and a denominator associated with a quantization parameter; Obtaining a weight of at least one adjacent picture by multiplying a scaling function by the exponential function; Obtaining a filtered sample value of the current picture based on the sample value of the current picture and a weight of the at least one adjacent picture; and replacing the sample values ​​of the current picture with the filtered sample values; and The current picture is encoded or decoded after applying the temporal filter.

2. The method according to claim 1, characterized in that: The obtaining of the exponential function comprises: The exponential function is obtained by calculating an exponent of the exponential function as a fixed point value or by using at least one lookup table, the exponential function comprising an exponent having a numerator and a denominator, the numerator having at least one first factor and the denominator having at least one second factor.

3. The method according to claim 2, characterized in that The obtaining of the exponential function comprises: The exponent of the exponential function is calculated as a fixed-point value.

4. The method according to claim 3, characterized in that The calculating the fixed-point value comprises: Represent the fixed point value as 2 n bits, where n is an integer.

5. The method according to claim 3, characterized in that: The calculating the fixed-point value comprises: The floating point value representation of the factor of the exponential function is scaled by the scaling factor.

6. The method according to claim 1, characterized in that The obtaining of the exponential function comprises: At least one lookup table is used.

7. The method according to claim 6, characterized in that The at least one lookup table includes a first lookup table and a second lookup table, and The first lookup table includes an integer portion of the exponential function and the second lookup table includes a fractional portion of the exponential function.

8. The method according to claim 6, characterized in that The at least one lookup table includes fixed point values ​​for the exponential function.

9. The method according to claim 1, characterized in that: The method further comprises: converting the factors of the scaling function into fixed point representation, Wherein, obtaining the weight of the at least one adjacent picture comprises multiplying the fixed-point representation of the scaling function by the exponential function.

10. The method according to claim 1, characterized in that The method further comprises: Performing a pixel-by-pixel loop for multiple iterations, each of the multiple iterations comprising: obtaining, for corresponding pixels in the pixels of the current picture, filtered sample values ​​of the current picture, and replacing sample values ​​of the current picture with the filtered sample values; and Prior to performing a plurality of iterations of the pixel-by-pixel loop, the sample values ​​are converted to a fixed point representation based on a factor in the scaling function or the exponential function that does not vary across pixels.

11. The method according to claim 1, characterized in that The scaling function includes a numerator having at least one first factor and a denominator having at least one second factor, The at least one first factor comprises a noise level of the current picture, and The at least one second factor comprises a picture order count distance between the at least one neighboring picture and the current picture.

12. A system for image filtering, characterized in that: The system comprises: at least one memory storing computer code; and At least one processor is configured to access the computer code and execute the method according to any one of claims 1 to 11 according to the instructions of the computer code.

13. A non-transitory computer readable medium storing computer code, characterized in that: The computer code is configured to, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 11.

Images