Video encoding method and apparatus, electronic device, and non-transitory computer-readable storage medium

Abstract

The present disclosure provides a video encoding method and apparatus, an electronic device, and a non-transitory computer-readable storage medium. The video encoding method includes performing motion estimation and motion compensation on a current block; calculating a block error between an extended block of the current block and an extended block of a motion compensated block; determining a filtering weight based on the block error; temporally filtering the current block by performing a weighted sum of the current block and the motion compensated block based on the determined filtering weight; and encoding the current block based on a result of the temporal filtering. The present disclosure can consider the relationship between a block-level filtering weight factor and surrounding pixels in a motion compensation-based temporal filtering process, and utilize the surrounding pixels to correct the result of the block-level error term in the temporal filtering weight, thereby improving the temporal filtering effect and improving the encoding efficiency.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to video coding and compression. More specifically, the present disclosure relates to a video coding method and apparatus thereof, an electronic device, and a non-transitory computer-readable storage medium. BACKGROUND

[0002] Various electronic devices (e.g., digital television, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video gaming consoles, smartphones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices transmit and receive or otherwise communicate digital video data over a communication network, and / or store digital video data on a storage device. Because of the limited bandwidth capacity of communication networks and the limited storage resources of storage devices, video data can be compressed prior to being communicated or stored according to one or more video coding standards. For example, video coding standards include Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), Moving Picture Experts Group (MPEG) codecs, etc. Video coding typically employs prediction methods (e.g., inter-prediction, intra-prediction, etc.) that take advantage of inherent redundancies in video data. Video coding aims to compress video data into a form that uses a lower bit rate, while avoiding or minimizing loss in video quality.

[0003] Motion Compensated Temporal Filter (MCTF) is an encoder-only video filtering technique that utilizes the temporal correlation of video, filters the reference frame in the temporal domain using inter-block reference relationship, to reduce the temporal redundancy information generated in the video block reference process, thereby improving the overall coding efficiency.

[0004] However, MCTF does not consider the motion relationship between the current block and the surrounding blocks, resulting in poor temporal filtering effect and affecting the coding efficiency. SUMMARY

[0005] Examples of the present disclosure provide a video coding method and apparatus thereof, an electronic device, and a non-transitory computer-readable storage medium.

[0006] According to a first aspect of the present disclosure, a video coding method is provided, comprising: performing motion estimation and motion compensation on a current block to obtain a motion compensated block of the current block; calculating a block error between an extended block of the current block and an extended block of the motion compensated block, wherein the extended block of the current block is a block obtained by extending a surrounding pixel region of the current block, and the extended block of the motion compensated block is a block obtained by extending a surrounding pixel region of the motion compensated block; determining a filter weight based on the calculated block error; performing temporal filtering on the current block by performing a weighted sum of the current block and the motion compensated block based on the determined filter weight; and encoding the current block based on a result of the temporal filtering.

[0007] Optionally, the calculating the block error between the extended block of the current block and the extended block of the motion compensated block comprises: calculating a sum of squared errors between the extended block of the current block and the extended block of the motion compensated block; calculating a variance between the extended block of the current block and the extended block of the motion compensated block; and calculating the block error based on the calculated sum of squared errors and the calculated variance.

[0008] Optionally, the calculating the sum of squared errors between the extended block of the current block and the extended block of the motion compensated block further comprises: scaling the sum of squared errors, and using the scaled sum of squared errors as a final sum of squared errors for calculating the block error.

[0009] Optionally, the scaling the sum of squared errors comprises: scaling the sum of squared errors based on a ratio of an area of the current block to an area of the extended block of the current block.

[0010] Optionally, the scaling the sum of squared errors based on the ratio of the area of the current block to the area of the extended block of the current block comprises: scaling the sum of squared errors based on the ratio of the area of the current block to the area of the extended block of the current block and a preset scaling factor.

[0011] Optionally, the scaling the sum of squared errors comprises: scaling the sum of squared errors based on a content characteristic of a surrounding pixel region of the current block in the extended block of the current block and the extended block of the motion compensated block.

[0012] Optionally, the extended block of the current block is obtained by extending a predetermined number of pixels around the current block, and the extended block of the motion compensated block is obtained by extending the predetermined number of pixels around the motion compensated block.

[0013] According to a second aspect of the present disclosure, a video encoding apparatus is provided, comprising: a motion estimation and compensation module configured to perform motion estimation and motion compensation on a current block to obtain a motion compensated block of the current block; a block error calculation module configured to calculate a block error between an extended block of the current block and an extended block of the motion compensated block, wherein the extended block of the current block is a block obtained by extending a surrounding pixel region of the current block, and the extended block of the motion compensated block is a block obtained by extending a surrounding pixel region of the motion compensated block; a filter weight determination module configured to determine a filter weight based on the calculated block error; a temporal filtering module configured to perform temporal filtering on the current block by performing weighted summation on the current block and the motion compensated block based on the determined filter weight; and an encoding module configured to encode the current block based on a result of the temporal filtering.

[0014] Optionally, the block error calculation module is configured to: calculate a sum of squared errors between the extended block of the current block and the extended block of the motion compensated block; calculate a variance between the extended block of the current block and the extended block of the motion compensated block; and calculate the block error based on the calculated sum of squared errors and the calculated variance.

[0015] Optionally, the block error calculation module is configured to: scale the sum of squared errors, and use the scaled sum of squared errors as a final sum of squared errors for calculating the block error.

[0016] Optionally, the block error calculation module is configured to scale the sum of squared errors based on a ratio of an area of the current block to an area of the extended block of the current block.

[0017] Optionally, the block error calculation module is configured to scale the sum of squared errors based on a ratio of an area of the current block to an area of the extended block of the current block and a preset scaling factor.

[0018] Optionally, the block error calculation module is configured to scale the sum of squared errors based on a content characteristic of a surrounding pixel region of the current block in the current block and the extended block of the current block.

[0019] Optionally, the extended block of the current block is obtained by extending a predetermined number of pixels around the current block, and the extended block of the motion compensated block is obtained by extending the predetermined number of pixels around the motion compensated block.

[0020] According to a third aspect of the present disclosure, there is provided an apparatus for video coding, comprising: one or more processors; and a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors, wherein the one or more processors, upon execution of the instructions, are configured to perform the video coding method according to the present disclosure.

[0021] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the video coding method according to the present disclosure.

[0022] According to a fifth aspect of the present disclosure, there is provided a computer program product having instructions for storing a bitstream, wherein the bitstream comprises: video data generated by the video coding method according to the present disclosure.

[0023] According to a sixth aspect of the present disclosure, there is provided a computer program product comprising computer instructions that, when executed by at least one processor, implement the video coding method according to the present disclosure.

[0024] According to a seventh aspect of the present disclosure, there is provided a method for generating a bitstream, comprising: generating a bitstream according to the video coding method according to the present disclosure.

[0025] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0026] According to the video coding method of the present disclosure, in the motion compensation based temporal filtering process, when calculating the block level error, the current block used for calculating the block level error is expanded to a larger block around it for calculating the block level error, and based on this, the relationship between the block level filtering weight factor and the surrounding pixels is considered, and the surrounding pixels are used to correct the result of the block level error term in the temporal filtering weight. That is, the present disclosure considers the spatial correlation between the current block and the adjacent block in the motion compensation based temporal filtering process, calculates the block level error in the MCTF filtering weight using the overlapping block idea, which can ensure to improve the temporal filtering effect and improve the coding efficiency.

[0027] It will be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate examples consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0029] Figure 1is a block diagram illustrating an exemplary system for encoding and decoding a video block, in accordance with some embodiments of the present disclosure.

[0030] Figure 2 is a block diagram illustrating an exemplary video encoder, in accordance with some embodiments of the present disclosure.

[0031] Figure 3 is a block diagram illustrating an exemplary video decoder, in accordance with some embodiments of the present disclosure.

[0032] Figure 4 is a schematic diagram illustrating a spatial domain filtering method employing a Gaussian filter kernel in the related art.

[0033] Figure 5 is a flowchart illustrating a video encoding method, in accordance with exemplary embodiments of the present disclosure.

[0034] Figure 6 is a schematic diagram illustrating motion estimation and motion compensation, in accordance with exemplary embodiments of the present disclosure.

[0035] Figure 7 is a schematic diagram illustrating padding for a block to obtain a padded block, in accordance with exemplary embodiments of the present disclosure.

[0036] Figure 8 is a block diagram illustrating a video encoding apparatus, in accordance with exemplary embodiments of the present disclosure.

[0037] Figure 9 is a diagram illustrating a computing environment coupled with a user interface, in accordance with some embodiments of the present disclosure. DETAILED DESCRIPTION

[0038] Reference will now be made in detail to specific implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth in order to help provide an understanding of the subject matter presented herein. But the subject matter presented herein can be practiced without the exact details, and can

[0039] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present disclosure, as well as in the drawings, are used to distinguish between similar objects and are not necessarily used to describe a particular sequential or chronological order. It will be understood that the use of such terms as "first", "second", and the like, is arbitrary and is merely intended to distinguish between two or more similar objects or actions. It will be understood that the data so designated can be interchanged, under appropriate circumstances, so that the embodiments of the present disclosure described herein can be practiced in other sequences than the one described in the figures or the one described in the present disclosure.

[0040] Figure 1This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0041] In some implementations, the target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14.

[0042] In some other implementations, the encoded video data can be sent from the output interface 22 to the storage device 32. Subsequently, the target device 14 can access the encoded video data in the storage device 32 via the input interface 28.

[0043] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0044] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback.

[0045] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0046] Video encoder 20 and video decoder 30 can operate according to a proprietary standard or industry standard, such as VVC, HEVC, MPEG-4, Part 10, AVC, or extensions of such standards. It should be understood that the application is not limited to a specific video coding / decoding standard and can be applicable to other video coding / decoding standards. It is generally contemplated that video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally contemplated that video decoder 30 of destination device 14 can be configured to decode video data according to any of these current or future standards.

[0047] Video encoder 20 and video decoder 30 can be implemented as any of a variety of suitable encoder and / or decoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuitry, software, hardware, firmware or any combinations thereof. When implemented partially in software, an electronic device can store instructions for the software in a suitable, non- transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video coding / decoding operations disclosed in the present disclosure. Each of video encoder 20 and video decoder 30 can be included in one or more encoders or decoders, either of which can be integrated as part of a combined encoder / decoder (CODEC) in a respective device.

[0048] Figure 2 is a block diagram illustrating an example video encoder 20, in accordance with some embodiments described in the present application. Video encoder 20 can perform intra-prediction encoding and inter-prediction encoding on video blocks within a video frame. Intra-prediction encoding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-prediction encoding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video coding, the term “frame” can be used as a synonym for the term “image” or “picture.”

[0049] As Figure 2As shown in FIG. 1, video encoder 20 includes video data memory 40, prediction processing unit 41, decoded picture buffer (DPB) 64, summer 50, transform processing unit 52, quantization unit 54, and entropy encoding unit 56. Prediction processing unit 41 further includes motion estimation unit 42, motion compensation unit 44, partition unit 45, intra-prediction processing unit 46, and intra block copy (BC) unit 48. In some implementations, video encoder 20 also includes inverse quantization unit 58, inverse transform processing unit 60, and summer 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between summer 62 and DPB 64 to filter block boundaries to remove blockiness artifacts from reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of summer 62. In some examples, the loop filters can be omitted, and the decoded video blocks can be provided directly from summer 62 to DPB 64. Video encoder 20 can take the form of a fixed or programmable hardware encoder, or can be dispersed into one or more of the illustrated fixed or programmable hardware encoders.

[0050] Video data memory 40 can store video data to be encoded by the components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18 as shown in FIG. 1. Video data memory 40 can store code streams generated by entropy encoding unit 56, or other intermediate results generated by other components of video encoder 20, including motion vectors, residual data, and so on. Figure 1 DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) for use in encoding video data by video encoder 20 (e.g., in intra- or inter-coding modes).

[0051] As shown in FIG. 1, video encoder 20 receives video data from video source 18. The video data can represent a series of pictures organized in a series of frames. Video encoder 20 can encode the video data to produce a coded video bitstream (CVBS) for communication or storage. Figure 2 As shown in FIG. 1, video encoder 20 receives video data from video source 18. The video data can represent a series of pictures organized in a series of frames. Video encoder 20 can encode the video data to produce a coded video bitstream (CVBS) for communication or storage.

[0052] Prediction processing unit 41 can select one of a plurality of possible predictive encoding modes, e.g., one of a plurality of intra-predictive encoding modes or one of a plurality of inter-predictive encoding modes, for the current video block based on the error results (e.g., coding rate and level of distortion). Prediction processing unit 41 can provide the resulting intra- or inter-predicted block to summer 50 to generate a residual block, and to summer 62 to reconstruct the encoded block for use as part of a reference frame at a later time. Prediction processing unit 41 also provides syntax elements, e.g., motion vectors, intra-mode indicators, partitioning information, and other such syntax information, to entropy encoding unit 56.

[0053] To select an appropriate intra-predictive encoding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 can perform intra-predictive encoding of the current video block in relation to one or more neighboring blocks in the same frame as the current block being encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-predictive encoding of the current video block in relation to one or more predictive blocks in one or more reference frames to provide temporal prediction. Video encoder 20 can perform multiple encoding passes, e.g., to select a suitable encoding mode for each block of video data.

[0054] In some implementations, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating motion vectors according to a predetermined pattern within a sequence of video frames, the motion vectors indicating displacement of video blocks within the current video frame relative to predictive blocks within a reference video frame. Motion estimation performed by motion estimation unit 42 is a process of generating motion vectors that estimate motion for video blocks. For example, a motion vector can indicate displacement of a video block within a current video frame or picture relative to a predictive block within a reference frame that is related to a current block being encoded within the current frame. The predetermined pattern can designate video frames in the sequence as P-frames or B-frames. Intra-BC unit 48 can determine vectors for intra-BC encoding (e.g., block vectors) in a similar manner as motion vectors determined by motion estimation unit 42 for inter-prediction, or can utilize block vectors determined by motion estimation unit 42.

[0055] Whether the predictive block is from the same frame according to intra-prediction or a different frame according to inter-prediction, video encoder 20 can form pixel difference values by subtracting pixel values of the predictive block from pixel values of the current video block being encoded, thereby forming a residual video block. The pixel difference values forming the residual video block can include both luma component differences and chroma component differences.

[0056] Intra prediction processing unit 46 can encode the current block using various intra prediction modes, e.g., during a separate encoding pass, and can select a suitable intra prediction mode to use from the tested intra prediction modes (or, in some examples, the mode selection unit). Intra prediction processing unit 46 can provide information indicating the selected intra prediction mode for the block to entropy encoding unit 56. Entropy encoding unit 56 can encode information indicating the selected intra prediction mode in the bitstream.

[0057] After prediction processing unit 41 determines a prediction block for the current video block via either inter prediction or intra prediction, summer 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block can be included in one or more TUs and is provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform, e.g., a discrete cosine transform (DCT) or a conceptually similar transform.

[0058] Transform processing unit 52 can send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting a quantization parameter. In some examples, quantization unit 54 can then perform a scan of the matrix including the quantized transform coefficients. Alternatively, entropy encoding unit 56 can perform the scan.

[0059] After quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients using, e.g., context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy encoding methodology or technique, into a video bitstream. The encoded bitstream can then be transmitted to video decoder 30, as shown in FIG. 3, or archived, as shown in FIG. 4, for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 can also entropy encode motion vectors and other syntax elements for the current video frame being encoded. Figure 1 Figure 1

[0060] ​​The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for generating a reference block used to predict other video blocks. As noted above, the motion compensation unit 44 can generate a motion compensated prediction block from one or more reference blocks of a frame stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values for use in motion estimation.

[0061] The summer 62 adds the reconstructed residual block to the motion compensated prediction block produced by the motion compensation unit 44 to produce a reference block for storage in the DPB 64. The reference block can then be used by the intra BC unit 48, the motion estimation unit 42, and the motion compensation unit 44 as a prediction block to inter predict another video block in a subsequent video frame.

[0062] Figure 3 FIG. 1 shows a block diagram of an example video decoder 30 in accordance with some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, a summer 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra prediction unit 84, and an intra BC unit 85. The video decoder 30 can perform decoding processes substantially reciprocal to those described above in connection with the video encoder 20. Figure 2 The decoding processes described in relation to the video encoder 20 are substantially reciprocal. For example, the motion compensation unit 82 can generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra prediction unit 84 can generate prediction data based on intra prediction mode indicators received from the entropy decoding unit 80.

[0063] In some examples, embodiments of the present disclosure can be dispersed among one or more of the units of the video decoder 30. For example, the intra BC unit 85 can perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 can not include the intra BC unit 85, and the functionality of the intra BC unit 85 can be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0064] The video data memory 79 can store video data, such as an encoded video bitstream, to be decoded by the other components of the video decoder 30. The video data stored in the video data memory 79 can be obtained, for example, from the storage device 32, from a local video source, such as a camera, via wired or wireless network communication of video data, or by accessing a physical data storage medium, such as a flash drive or hard disk.

[0065] Video

[0066] During the decoding process, video decoder 30 receives an encoded video bitstream that represents encoded video frames and associated syntax elements of video blocks. Entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors or intra prediction mode indicators, and other syntax elements. Entropy decoding unit 80 then forwards the motion vectors or intra prediction mode indicators, and other syntax elements to prediction processing unit 81.

[0067] When a video frame is coded as an intra-predicted coded (I) frame or an intra coded prediction block in other types of frames, intra-prediction unit 84 of prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode signaled and reference data from previously decoded blocks of the current frame.

[0068] When a video frame is coded as an inter-predicted coded (i.e., B or P) frame, motion compensation unit 82 of prediction processing unit 81 produces one or more prediction blocks for a video block of the current video frame based on the motion vectors and other syntax elements received from entropy decoding unit 80. Each of the prediction blocks can be produced from a reference frame within one of the reference frame lists. Video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, using default construction techniques based on reference frames stored in DPB 92.

[0069] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 produces a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block can be within a reconstructed region of the same picture as the current video block, as defined by video encoder 20.

[0070] Motion compensation unit 82 and / or intra BC unit 85 determine the prediction information for a video block of the current video frame by parsing the motion vectors and other syntax elements, and then use the prediction information to produce a prediction block for the current video block being decoded.

[0071] Motion compensation unit 82 can also perform interpolation to calculate interpolated values for sub-integer pixels of reference blocks using interpolation filters as used by video encoder 20 during encoding of the video blocks. In this case, motion compensation unit 82 can determine the interpolation filters used by video encoder 20 from the syntax elements received, and use these interpolation filters to produce the prediction blocks.

[0072] Inverse quantization unit 86 inverse quantizes quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80 using the same quantization parameter calculated by video encoder 20 for each video block in the video frame to determine a degree of quantization. Inverse transform processing unit 88 applies an inverse transform, e.g., an inverse DCT, an inverse integer transform or a conceptually similar inverse transform process, to the transform coefficients in order to reconstruct the residual blocks in the pixel domain.

[0073] After motion compensation unit 82 or intra BC unit 85 generates the prediction block for the current video block based on the vectors and other syntax elements, adder 90 reconstructs the decoded video block for the current video block by adding the residual block from inverse transform processing unit 88 to the corresponding prediction block generated by motion compensation unit 82 and intra BC unit 85. In-loop filter 91, e.g., a de-blocking filter, a SAO filter, a CCSAO filter and / or an ALF, can be located between adder 90 and DPB 92 to further process the decoded video block. In some examples, in-loop filter 91 can be omitted and the decoded video block can be directly provided by adder 90 to DPB 92. The decoded video blocks in a given frame are then stored in DPB 92, which stores reference frames for subsequent motion compensation of video blocks that follow. DPB 92 or a memory device separate from DPB 92 can also store decoded video for later presentation on a display device (e.g., display device 34 of FIG. 1). Figure 1

[0074] As previously mentioned, Motion Compensated Temporal Filter (MCTF) is an encoder-only video filtering technique that exploits the temporal correlation of video, and exploits the inter-block reference relationship to temporally filter the reference frames to reduce the temporal redundancy information generated in the video block reference process, thereby improving the overall coding efficiency.

[0075] ​The MCTF technique can be mainly divided into two parts, i.e., a "motion estimation compensation part" and a "filtering part". In the "motion estimation compensation part", the MCTF performs block-level motion estimation (ME) first. Specifically, the current image can be uniformly divided into a plurality of blocks of m x m size, for example, a plurality of blocks of 8 x 8, and a motion vector (MV) can be searched for each block. Then, a classical block-based motion compensation can be performed on each 8 x 8 block by using the MV, and a motion frame can be directly spliced by using all the 8 x 8 motion compensated image blocks. In this way, the motion compensation process can be performed for each adjacent frame of the current frame, and a plurality of motion compensation frames corresponding to the plurality of adjacent frames of the current frame can be obtained. In the "filtering part", each block of the current frame and a plurality of motion compensation blocks corresponding to the adjacent frames are weighted according to a certain proportion to perform bilateral filtering, and a final time domain filtering result is generated.

[0076] For the "filtering part", the bilateral filtering weight of each pixel can be generated according to the motion compensation frame, wherein the bilateral filtering weight of each pixel can be composed of three parts: a frame-level weight factor, a block-level weight factor, and a pixel-level weight factor, i.e., the final weight of each pixel can be determined by the three factors. Specifically, the "frame-level weight factor" can be a basic weight set according to the properties of the current frame, for example, a reference level, a frame type, etc.; the "block-level weight factor" is to calculate the error and variance information between the compensation image block and the original image block for each 8 x 8 block, and then to obtain the "block-level weight factor" based on the calculated error and variance; and the "pixel-level weight factor" can be to calculate the error of each pixel, and then to obtain the "pixel-level weight factor" based on the error of each pixel. Next, each pixel in the current frame and the corresponding motion compensation pixel in the adjacent frame can be weighted and subjected to bilateral filtering to obtain the final time domain filtering result.

[0077] Specifically, for the filtering part of the MCTF, a bilateral filter is adopted. The luminance is filtered in units of 8 x 8 blocks, and the chrominance is filtered in units of 4 x 4 blocks. In the filtering formula, I n represents the filtered pixel value, I o is the original pixel value, I r (i) is the pixel value of the best matching block obtained by motion compensation in a certain reference frame, i.e., the pixel value of the motion compensation block, w r (i, a) is the weight corresponding to each reference frame. Therefore, the filtering process is actually a weighted sum process of the reference blocks and the original block, i.e., the filtering process is actually a weighted sum process of the motion compensation blocks and the current block.

[0078] Filtering in block unit: 8x8 block for luma, 4x4 block for chroma (YUV420)

[0079] pixel value of the current frame after temporal filtering;

[0080] I o pixel value of the original frame;

[0081] I r (i), pixel value of the best matching block in the i-th reference frame, i.e., the pixel value of the motion compensated block;

[0082] w r (i, a), weight of the i-th reference frame, where there are a reference frames available;

[0083]

[0084] ΔI(i) = I r (i) - I o

[0085] where s o (n) denotes the overall filtering strength of the n-th frame (i.e., the current frame), s r (i, a) denotes the reference frame weight, which depends on the distance between the reference frame and the original frame, σ w , σ c , σ l parameters are related to the quantization parameter (QP), where σ l denotes the σ parameter for luma component, σ c denotes the σ parameter for chroma component. ΔI(i) 2 is the square of the difference between the compensated pixel and the original pixel, which is related to each pixel. The coefficient 0.4 corresponds to the weight parameter of the luma component, and the coefficient 0.55 corresponds to the weight parameter of the chroma component.

[0086] It can be seen that the weight value w r (i, a) is related to multiple indicators, including: sequence frame level parameters (e.g., s o (n), s r (i, a)), block level parameters (e.g., w a , σ w , σ c , σ l ), pixel level parameters (e.g., ΔI(i) 2 ), etc.

[0087] For the 8x8 block level parameters w a and σ w , the block frequency F and the block error E are used for calculation:

[0088]

[0089] wherein,

[0090]

[0091] SSD represents the sum of squared error between the current block and the motion compensated block, V represents the variance between the current block and the motion compensated block, S 2 represents the size of the current block. For example, for a current block of 8x8, the size of the current block is: S 2 = 8x8 = 64.

[0092] However, the block error E in the prior art algorithm is only calculated using the pixels inside the current 8x8 block, without considering the surrounding information. It can be seen that the prior MCTF does not consider the motion relationship between the current block and the surrounding blocks, resulting in poor filter weights and affecting the coding efficiency.

[0093] In addition, in the related art, an overlapping block motion compensation based temporal filtering technology is proposed. The spatial correlation is considered in the motion estimation, motion compensation and filtering process in the temporal filtering, i.e. the spatial correlation related overlapping block idea is used to improve the traditional motion estimation, motion compensation and filtering process to reduce the blocking effect of the frame after temporal filtering, thereby improving the coding performance of the MCTF tool. Specifically, the overlapping block motion compensation based temporal filtering technology can first perform spatial filtering on the filter weights of the block edges, for example, a spatial filtering method using a Gaussian filter kernel can be used to perform spatial filtering on the filter weights of the block edges.

[0094] Figure 4 is a schematic diagram showing the spatial filtering method using a Gaussian filter kernel in the related art. Referring to Figure 4 For the weight of each edge pixel above the current block, the weight information of the adjacent block above can be used to perform spatial filtering using a Gaussian filter kernel to obtain the new filter weight of each edge pixel above. For example, a Gaussian filter kernel with a size of 3x3 can be used for spatial filtering. Then, the weights of the left edge, the lower edge and the right edge can be selected for spatial filtering, and the final temporal filtering weight can be obtained. Next, the final temporal filtering weight can be used for temporal bilateral filtering to obtain the final filtered frame.

[0095] However, in the filtering process of the overlapping block motion compensation based temporal filtering technology, only the filter weights of the surrounding pixels are considered for the filter weights of each pixel in the block, and the information of the surrounding blocks is not considered at the level of the weight parameters of the block, resulting in limited improvement of the filtering performance.

[0096] To solve the above problems in the related art, the video encoding method and device, electronic device and non-transitory computer-readable storage medium provided by the present disclosure expand the current block for calculating the block-level error to a larger block for calculating the block-level error when calculating the block-level error in the motion compensation-based time domain filtering process, consider the relationship between the block-level filtering weight factor and the surrounding pixels, and use the surrounding pixels to correct the result of the block-level error term in the time domain filtering weight. That is, the present disclosure considers the spatial correlation between the current block and the adjacent block in the motion compensation-based time domain filtering process, calculates the block-level error in the MCTF filtering weight using the overlapping block idea, and can ensure to improve the time domain filtering effect and improve the encoding efficiency.

[0097] In the following, the implementation process of the video encoding method in the present disclosure will be described in detail with reference to the accompanying drawings. Figures 5 to 9

[0098] Referring to Figure 5 , in step 501, motion estimation and motion compensation can be performed on the current block to obtain a motion compensation block of the current block.

[0099] For the part of "motion estimation": assuming that the current frame is I t , i.e. it needs to be filtered for the current frame I t , then it needs to perform block-level motion estimation on the current frame I t in turn, for example, for each of the four adjacent frames. Assuming that the number of adjacent frames N = 8, they can be respectively denoted as: I t-4 , I t-3 , I t-2 , I t-1 , I t+1 , I t+2 , I t+3 , I t+4 . As shown in Figure 6 , the schematic diagram showing the motion estimation and motion compensation according to the example embodiment of the present disclosure is shown in Figure 6 , the first four frames of the current frame I t are respectively: I t-4 , I t-3 , I t-2 , I t-1 , and the last four frames of the current frame I t are respectively: I t+1 , I t+2 , I t+3 , I t+4 . In this way, the most similar block corresponding to each 8x8 block in the current frame I t in each of the adjacent 8 frames can be searched out, and the position offset of the similar block relative to the current block can be represented by a motion vector MV. ​

[0100] For example, in order to accelerate the motion estimation process, a pyramid motion estimation form can be adopted, i.e. a fast estimation can be performed on down-sampled images, and then a motion estimation (ME) can be performed based on the down-sampled ME results to obtain the original resolution ME. Specifically, for the original resolution, an integer-pixel ME can be performed first, and then a sub-pixel ME can be performed based on the integer-pixel ME results to obtain one MV corresponding to each block.

[0101] For the "motion compensation" part: for the current 8x8 block, in each neighboring frame, a corresponding MV searched can be directly used to perform a classic block-level motion compensation to obtain a motion-compensated image block corresponding to the neighboring frame. In this way, performing motion compensation in each of the 8 neighboring frames can obtain 8 best-matched image blocks, i.e. 8 motion-compensated blocks, corresponding to the 8 neighboring frames respectively.

[0102] In step 502, a block error between the extended block of the current block and the extended block of the motion-compensated block can be calculated, wherein the extended block of the current block can be a block obtained by extending a surrounding pixel region of the current block, and the extended block of the motion-compensated block can be a block obtained by extending a surrounding pixel region of the motion-compensated block.

[0103] According to an exemplary embodiment of the present disclosure, the extended block of the current block can be obtained by extending a predetermined number a of pixels around the current block respectively, and the extended block of the motion-compensated block can also be obtained by extending a predetermined number a of pixels around the motion-compensated block respectively.

[0104] For example, a can be, but is not limited to, 2, i.e. 2 pixels can be filled around the current block 8x8 to obtain a 12x12 extended block of the current block. At this time, the obtained extended block is a square block. Of course, the number of extensions a is variable, and the obtained extended block can also be non-square. Figure 7 FIG. 1 is a schematic diagram showing an extension of a block to obtain an extended block according to an exemplary embodiment of the present disclosure. Referring to FIG. 1, a current block 10 of m x m size can be extended around the four sides to obtain an extended block 20 of n x n size. Figure 7 The current block of m x m size can be extended around the four sides to obtain an extended block of n x n size.

[0105] It should be noted that the number of pixels filled around the four sides of a certain block can be the same or different. If the number of pixels filled around the four sides of a certain block is the same, the obtained extended block is a square block; if the number of pixels filled around the four sides of a certain block is different, the obtained extended block is a non-square block. For example, 1 pixel can be filled at the left edge of a block 8x8, 3 pixels can be filled at the right edge of the block, 2 pixels can be filled at the upper edge of the block, and 4 pixels can be filled at the lower edge of the block, and thus a non-square extended block 12x14 is obtained.

[0106] According to an example embodiment of the present disclosure, the sum of squared differences (SSD) between the extended block of the current block and the extended block of the motion compensation block can be calculated, and the variance (V) between the extended block of the current block and the extended block of the motion compensation block can be calculated. Then, the block error E can be calculated based on the calculated sum of squared differences SSD and the calculated variance V:

[0107]

[0108] It should be noted that in the related art, when calculating the block error E, the sum of squared differences SSD and the variance V are calculated only using the internal pixels contained in the current block 8x8, and then the block error E is calculated based on the calculated sum of squared differences SSD and the calculated variance V.

[0109] In the present disclosure, when calculating the block error E, the surrounding pixel region of the current block is further considered. That is, the current block m*m can be extended to the four sides to obtain an extended block n*n, and then the sum of squared differences SSD_pad and the variance V_pad can be calculated based on the pixels in the extended block n*n.

[0110] According to an example embodiment of the present disclosure, the sum of squared differences of the extended block can be scaled, and the scaled sum of squared differences can be used as the final sum of squared differences for calculating the block error E.

[0111] According to an example embodiment of the present disclosure, the sum of squared differences can be scaled based on the ratio of the area of the current block to the area of the extended block of the current block. That is, the scaled sum of squared differences SSD' can be represented by the following formula:

[0112] SSD' = SSD_pad * (m*m) / (n*n)

[0113] wherein SSD_pad represents the sum of squared differences calculated using the pixels in the extended block n*n, m*m represents the size of the block before extension, and n*n represents the size of the obtained extended block after extension.

[0114] In addition, when calculating the block error E, the variance V_pad of the padded block can be directly used. Then, based on the scaled error sum of squares SSD' and the variance V_pad of the padded block, the filter weight can be calculated.

[0115] According to the exemplary embodiments of the present disclosure, the error sum of squares can also be scaled based on the ratio of the current block to the area of the padded block of the current block and a preset scaling factor. The preset scaling factor can be flexibly set according to actual needs, and the present disclosure does not limit the specific value of the preset scaling factor. At this time, the scaled error sum of squares SSD' can be represented by the following formula:

[0116] SSD' = a * SSD_pad * (m*m) / (n*n)

[0117] Wherein, a is the preset scaling factor mentioned above.

[0118] According to the exemplary embodiments of the present disclosure, the error sum of squares can also be scaled based on the content characteristics of the surrounding pixel region of the current block in the current block and the padded block of the current block.

[0119] For example, the content contained in the current block can be relatively complex, for example, the texture can be relatively complex; and the filling content of the surrounding region of the current block can be relatively simple, for example, it can be only a single background content. At this time, the feature value in terms of content between the current block and the surrounding region of the current block can be calculated, that is, the feature value in terms of content between the center block and the surrounding block can be calculated, and then the scaling weight of the error sum of squares can be adjusted based on the calculated feature value in terms of content. The features in terms of content can include but are not limited to mean, variance, etc.

[0120] In step 503, the filter weight can be determined based on the calculated block error E. For example, as described above, the weight w r (i, a) corresponding to each reference frame can be determined based on the calculated block error E.

[0121] In step 504, the current block can be temporally filtered by weighted sum of the current block and the motion compensation block based on the determined filter weight. That is, the current block can be temporally filtered by weighted sum of the current block and the motion compensation block based on the determined weight w r (i, a) corresponding to each reference frame. For example, as described above, the current block can be temporally filtered by the following formula:

[0122]

[0123] In step 505, the current block can be encoded based on the time domain filtering result. For example, information related to the time domain filtering result can be encoded into a bitstream, and thus signaled to a decoder, so that the decoder can decode the current block based on the information related to the time domain filtering result parsed from the bitstream.

[0124] Figure 8 is a block diagram illustrating a video encoding apparatus 800 according to an example embodiment of the present disclosure.

[0125] Referring to Figure 8 The video encoding apparatus 800 can include a motion estimation and compensation module 801, a block error calculation module 802, a filter weight determination module 803, a time domain filtering module 804, and an encoding module 805.

[0126] The motion estimation and compensation module 801 can perform motion estimation and motion compensation on the current block to obtain a motion compensated block of the current block.

[0127] The block error calculation module 802 can calculate a block error between an extended block of the current block and an extended block of the motion compensated block, wherein the extended block of the current block can be a block obtained by extending a surrounding pixel region by a predetermined number a of pixels from the current block, and the extended block of the motion compensated block can be a block obtained by extending a surrounding pixel region by a predetermined number a of pixels from the motion compensated block.

[0128] According to an example embodiment of the present disclosure, the extended block of the current block can be obtained by extending a predetermined number a of pixels from each of four sides of the current block, and the extended block of the motion compensated block can be obtained by extending a predetermined number a of pixels from each of four sides of the motion compensated block.

[0129] According to an example embodiment of the present disclosure, the block error calculation module 802 can calculate a sum of squared differences (SSD) between the extended block of the current block and the extended block of the motion compensated block, and the block error calculation module 802 can calculate a variance (V) between the extended block of the current block and the extended block of the motion compensated block. Then, the block error calculation module 802 can calculate the block error E based on the calculated sum of squared differences SSD and the calculated variance V.

[0130] According to an example embodiment of the present disclosure, the block error calculation module 802 can scale the sum of squared differences of the extended blocks, and can use the scaled sum of squared differences as a final sum of squared differences for calculating the block error E.

[0131] According to an example embodiment of the present disclosure, the block error calculation module 802 can scale the sum of squared differences based on a ratio of an area of the current block to an area of the extended block of the current block to obtain a scaled sum of squared differences SSD'

[0132] In calculating the block error E, the variance V_pad of the padded block can be directly used. Then, based on the scaled error sum of squares SSD' and the variance V_pad of the padded block, the filter weight can be calculated.

[0133] According to exemplary embodiments of the present disclosure, the block error calculation module 802 can further scale the error sum of squares based on a ratio of the current block to an area of the padded block of the current block and a preset scaling factor. The preset scaling factor can be flexibly set according to actual needs, and the present disclosure does not limit the specific value of the preset scaling factor.

[0134] According to exemplary embodiments of the present disclosure, the block error calculation module 802 can further scale the error sum of squares based on a content characteristic of a surrounding pixel region of the current block in the current block and the padded block of the current block.

[0135] The filter weight determination module 803 can determine the filter weight based on the calculated block error E. For example, as described above, the weight w(i, a) corresponding to each reference frame can be determined based on the calculated block error E. r (i, a).

[0136] The temporal filtering module 804 can perform temporal filtering on the current block by weighted summing the current block and the motion compensated block based on the determined filter weight. That is, the current block can be temporally filtered by weighted summing the current block and the motion compensated block based on the weight w(i, a) corresponding to each reference frame. r (i, a).

[0137] The encoding module 805 can encode the current block based on the temporal filtering result. For example, the encoding module 805 can encode information related to the temporal filtering result into a bitstream, and thus signal to a decoder, so that the decoder can decode the current block based on the information related to the temporal filtering result parsed from the bitstream.

[0138] Figure 9 A computing environment 910 coupled to a user interface 950 is shown. The computing environment 910 can be part of a data processing server. The computing environment 910 includes a processor 920, a memory 930, and an input / output (I / O) interface 940.

[0139] Processor 920 typically controls the overall operation of computing environment 910, such as operations associated with display, data acquisition, data communication, and image processing. Processor 920 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 920 may include one or more modules that facilitate interaction between processor 920 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0140] Memory 930 is configured to store various types of data to support the operation of computing environment 910. Memory 930 may include predefined software 932. Examples of such data include instructions for any application or method operating on computing environment 910, video datasets, image data, etc. Memory 930 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0141] I / O interface 940 provides an interface between processor 920 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 940 can be coupled to encoders and decoders.

[0142] In an embodiment, a non-transitory computer-readable storage medium is also provided, including, for example, a plurality of programs in a memory 930 and / or a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method. The plurality of programs can be executed by a processor 920 in a computing environment 910 to perform the above-described methods. In one example, the plurality of programs can be executed by a processor 920 in a computing environment 910 to (e.g., from...) Figure 2 The video encoder 20 in the computing environment 910 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 920 in the computing environment 910 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 920 in the computing environment 910 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 920 in the computing environment 910 to (e.g., to...) Figure 3transmitted by the encoder (e.g., video encoder 20 in FIG. 1) to the decoder (e.g., video decoder 30 in FIG. 1). Alternatively, the non-transitory computer-readable storage medium can store the bitstream or data stream that includes the encoded video information generated by the encoder (e.g., video encoder 20 in FIG. 1) for use in decoding video data by the decoder (e.g., video decoder 30 in FIG. 1). Figure 2 transmitted by the encoder (e.g., video encoder 20 in FIG. 1) to the decoder (e.g., video decoder 30 in FIG. 1). Alternatively, the non-transitory computer-readable storage medium can store the bitstream or data stream that includes the encoded video information generated by the encoder (e.g., video encoder 20 in FIG. 1) for use in decoding video data by the decoder (e.g., video decoder 30 in FIG. 1). Figure 3 transmitted by the encoder (e.g., video encoder 20 in FIG. 1) to the decoder (e.g., video decoder 30 in FIG. 1). Alternatively, the non-transitory computer-readable storage medium can store the bitstream or data stream that includes the encoded video information generated by the encoder (e.g., video encoder 20 in FIG. 1) for use in decoding video data by the decoder (e.g., video decoder 30 in FIG. 1).

[0143] In embodiments, a bitstream generated by the above-described encoding method or to be decoded by the above-described decoding method is provided. In embodiments, a bitstream including encoded video information generated by the above-described encoding method or to be decoded by the above-described decoding method is provided.

[0144] In embodiments, a computing device is also provided, including one or more processors (e.g., processor 920) and a non-transitory computer-readable storage medium or memory 930 having stored therein a plurality of programs that can be executed by the one or more processors, wherein the one or more processors, when executing the programs, are configured to perform the above-described methods.

[0145] In embodiments, a computer program product having instructions for storing or transmitting a bitstream including encoded video information generated by the above-described encoding method or to be decoded by the above-described decoding method is also provided. In embodiments, a computer program product including a plurality of programs, for example, in memory 930, that can be executed by processor 920 in computing environment 910 for performing the above-described methods is also provided. For example, the computer program product can include a non-transitory computer- readable storage medium.

[0146] In embodiments, computing environment 910 can be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components, for performing the above-described methods.

[0147] In embodiments, a method of storing a bitstream is also provided, including storing the bitstream on a digital storage medium, wherein the bitstream includes encoded video information generated by the above-described encoding method or to be decoded by the above-described decoding method.

[0148] In an embodiment, there is also provided a method for transmitting the bitstream generated by the above-described encoder. In an embodiment, there is also provided a method for receiving the bitstream to be decoded by the above-described decoder.

[0149] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the disclosure as set forth in the foregoing description and the associated drawings are presented for purposes of illustration and should not be interpreted to limit the scope of the disclosure. Numerous modifications, variations, and alternative implementations will be apparent to persons skilled in the art from the teachings of the foregoing description and associated drawings.

[0150] Unless otherwise specifically noted, the order of steps of the methods according to the present disclosure is merely intended to be illustrative and the steps of the methods according to the present disclosure are not limited to the specific order described above, but can be changed according to the actual situation. In addition, at least one of the steps of the methods according to the present disclosure can be adjusted, combined or deleted according to the actual needs.

[0151] The examples are chosen and described so that others skilled in the art can understand the principles of the present disclosure and its practical application. The best mode of the present disclosure is thus made apparent, and to enable others skilled in the art to carry out the present disclosure with various modifications as suitable for particular uses, each of the various embodiments is best made of the principles described herein and the various modifications appropriate to the particular application. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A method of video coding, the method comprising: The method comprises: performing motion estimation and motion compensation on a current block to obtain a motion compensated block of the current block; calculating a block error between an extended block of the current block and an extended block of the motion compensated block, wherein the extended block of the current block is a block obtained by extending a surrounding pixel region of the current block, and the extended block of the motion compensated block is a block obtained by extending a surrounding pixel region of the motion compensated block; determining a filter weight based on the calculated block error; performing temporal filtering on the current block by performing weighted summation on the current block and the motion compensated block based on the determined filter weight; encoding the current block based on a result of the temporal filtering.

2. The encoding method of claim 1, wherein, The calculating a block error between an extended block of the current block and an extended block of the motion compensated block comprises: calculating a sum of squared errors between the extended block of the current block and the extended block of the motion compensated block; calculating a variance between the extended block of the current block and the extended block of the motion compensated block; calculating the block error based on the calculated sum of squared errors and the calculated variance.

3. The encoding method of claim 2, wherein, The calculating a sum of squared errors between the extended block of the current block and the extended block of the motion compensated block further comprises: scaling the sum of squared errors, and using the scaled sum of squared errors as a final sum of squared errors for calculating the block error.

4. The encoding method of claim 3, wherein, The scaling the sum of squared errors comprises: scaling the sum of squared errors based on a ratio of an area of the current block to an area of the extended block of the current block.

5. The encoding method of claim 4, wherein, The scaling the sum of squared errors based on the ratio of the area of the current block to the area of the extended block of the current block comprises: scaling the sum of squared errors based on the ratio of the area of the current block to the area of the extended block of the current block and a preset scaling factor.

6. The encoding method of claim 3, wherein, The scaling the sum of squared errors comprises: scaling the sum of squared errors based on a content characteristic of a surrounding pixel region of the current block in the current block and the extended block of the current block.

7. The encoding method of claim 1, wherein, The extended block of the current block is obtained by extending a predetermined number of pixels around the current block, and the extended block of the motion compensated block is obtained by extending the predetermined number of pixels around the motion compensated block.

8. A video encoding apparatus, comprising: The method comprises: a motion estimation and compensation module configured to perform motion estimation and motion compensation on a current block to obtain a motion compensated block of the current block; a block error calculation module configured to calculate a block error between an extended block of the current block and an extended block of the motion compensated block, wherein the extended block of the current block is a block obtained by extending a surrounding pixel region of the current block, and the extended block of the motion compensated block is a block obtained by extending a surrounding pixel region of the motion compensated block; a filter weight determination module configured to determine a filter weight based on the calculated block error; a temporal filtering module configured to perform temporal filtering on the current block by performing weighted summation on the current block and the motion compensated block based on the determined filter weight; an encoding module configured to encode the current block based on a result of the temporal filtering.

9. An apparatus for video encoding, comprising: one or more processors; and a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors, wherein the one or more processors, upon execution of the instructions, are configured to perform the video encoding method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the video encoding method according to any one of claims 1-7. the bitstream comprises:

11. A computer program product having instructions for storing a bitstream, wherein, video data generated by the video encoding method according to any one of claims 1-7. the computer instructions, when executed by at least one processor, implement the video encoding method according to any one of claims 1 to 7.

12. A computer program product comprising computer instructions, characterized in that, 13. A method of generating a bitstream, comprising: generating a bitstream according to the video encoding method of any one of claims 1 to 7. ​

Images