Video encoding methods and apparatuses, electronic devices, storage media and software products

Abstract

This disclosure provides a video coding method, apparatus, electronic device, storage medium, and program product thereof. The video coding method includes: dividing a current frame into multiple first blocks according to a first partitioning method, and dividing the current frame into multiple second blocks according to a second partitioning method; performing motion estimation and motion compensation on the multiple first blocks and multiple second blocks respectively; superimposing the multiple first motion compensation blocks and multiple second motion compensation blocks; performing temporal filtering on the current frame based on the multiple superimposed motion compensation blocks; and encoding the current frame based on the temporal filtering result. Thus, by performing weighted filtering based on the motion compensation results obtained from blocks divided by two different methods, and using continuous region pixels in one motion compensation result to correct discontinuous region pixels in another motion compensation result, the block artifacts in the motion-compensated frame of MCTF can be reduced.

Description

Technical Field

[0001] This application relates to video encoding / decoding and compression. More specifically, this application relates to video coding methods and apparatuses, electronic devices, storage media, and program products. Background Technology

[0002] Various electronic devices (such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, video streaming devices, etc.) support digital video. Electronic devices send and receive, or otherwise transmit, digital video data via communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited storage resources of storage devices, video data can be compressed using one or more video codec standards before it is transmitted or stored. For example, video codec standards include Universal Video Codec (VVC), Joint Explore Test Model (JEM), High Efficiency Video Codec (HEVC / H.265), Advanced Video Codec (AVC / H.264), Moving Picture Experts Group (MPEG) codec, etc. Video codecs typically employ prediction methods that utilize the inherent redundancy in video data (e.g., inter-frame prediction, intra-frame prediction, etc.). Video codecs aim to compress video data to a form using a lower bitrate while avoiding or minimizing degradation in video quality.

[0003] Motion Compensated Temporal Filter (MCTF) is a pre-coding filtering algorithm widely used in many coding standards (e.g., HEVC, VVC, AVS, AV1, AV2, etc.). This algorithm is used to remove high-frequency and random noise from video before encoding, thereby improving the overall compression efficiency of the video. The motion estimation process in MCTF is performed in 8x8 blocks, and the motion estimation of each block is independent. Therefore, if motion estimation is performed on multiple adjacent blocks in a video frame, multiple interpolation blocks with discontinuous boundaries may be found in adjacent frames because the motion estimation is performed independently. This results in discontinuous boundaries in the interpolated frame (i.e., the motion-compensated frame) formed by the motion-compensated interpolation blocks (i.e., the motion-compensated blocks), meaning that the motion-compensated frame used for filtering will exhibit block artifacts, thus affecting the objective compression efficiency of MCTF. Summary of the Invention

[0004] Examples of video coding methods, apparatuses, electronic devices, storage media, and program products disclosed herein are provided.

[0005] According to a first aspect of this disclosure, a video coding method is provided, comprising: dividing a current frame into multiple first blocks according to a first division method, and dividing the current frame into multiple second blocks according to a second division method, wherein non-boundary pixel regions in each second block partially overlap with boundary pixel regions of a corresponding plurality of first blocks; performing motion estimation and motion compensation on each of the multiple first blocks to obtain multiple first motion compensation blocks, and performing motion estimation and motion compensation on each of the multiple second blocks to obtain multiple second motion compensation blocks; superimposing the multiple first motion compensation blocks and the multiple second motion compensation blocks to obtain multiple superimposed motion compensation blocks; performing temporal filtering on the current frame based on the multiple superimposed motion compensation blocks, and encoding the current frame based on the temporal filtering result of the current frame.

[0006] Optionally, the first partitioning method is a conventional partitioning method, which divides the current frame into N×N first blocks, where N is an integer greater than 1; the second partitioning method is a partitioning method with a right-lower offset of 1 / 2 block size, which divides the current frame into (N-1)×(N-1) second blocks; wherein the block size of the first block and the second block is the same.

[0007] Optionally, the step of superimposing the plurality of first motion compensation blocks and the plurality of second motion compensation blocks to obtain a plurality of superimposed motion compensation blocks includes: performing position-adaptive weighted superposition of the plurality of first motion compensation blocks and the plurality of second motion compensation blocks to obtain the plurality of superimposed motion compensation blocks, wherein the weight of pixels in the first motion compensation blocks and the second motion compensation blocks that are closer to their respective boundary pixel positions is smaller.

[0008] Optionally, the overlapping region between the second motion compensation block and the corresponding plurality of first motion compensation blocks includes multiple pixel pairs, wherein each pixel pair includes a first pixel in the first motion compensation block and a second pixel in the second motion compensation block that overlaps with the first pixel; the overlapping region is divided into multiple pixel regions according to the distance between the first pixel and the second pixel in the pixel pair and their respective boundary pixel positions, and a weight is set for each pixel region, wherein the weight of the first pixel in the pixel region that is closer to the corresponding boundary pixel position is smaller, and the weight of the second pixel in the pixel region that is closer to the corresponding boundary pixel position is smaller, wherein the sum of the weights of the first pixel and the second pixel in each pixel pair is 1.

[0009] Optionally, the plurality of pixel regions include a first pixel region, a second pixel region, and a third pixel region; wherein, the first pixel region includes a region where the first pixel is a boundary pixel and the second pixel is a non-boundary pixel; the second pixel region includes a region where both the first pixel and the second pixel are non-boundary pixels and a region where both the first pixel and the second pixel are boundary pixels; the third pixel region includes a region where the first pixel is a non-boundary pixel and the second pixel is a boundary pixel; wherein, the weight of the first pixel in the first pixel region is set to be less than the weight of the second pixel; the weight of the first pixel in the second pixel region is set to be equal to the weight of the second pixel; and the weight of the first pixel in the third pixel region is set to be greater than the weight of the second pixel.

[0010] According to a second aspect of this disclosure, a video encoding apparatus is provided, comprising: a block partitioning module configured to partition a current frame into a plurality of first blocks according to a first partitioning method, and to partition the current frame into a plurality of second blocks according to a second partitioning method, wherein each second block includes boundary pixels of a corresponding plurality of first blocks; a motion estimation and compensation module configured to perform motion estimation and motion compensation on each of the plurality of first blocks to obtain a plurality of first motion compensation blocks, and to perform motion estimation and motion compensation on each of the plurality of second blocks to obtain a plurality of second motion compensation blocks; a block overlay module configured to overlay the plurality of first motion compensation blocks and the plurality of second motion compensation blocks to obtain a plurality of overlay motion compensation blocks; a temporal filtering module configured to perform temporal filtering on the current frame based on the plurality of overlay motion compensation blocks; and an encoding module configured to encode the current frame based on the temporal filtering result of the current frame.

[0011] Optionally, the first partitioning method is a conventional partitioning method, which divides the current frame into N×N first blocks, where N is an integer greater than 1; the second partitioning method is a partitioning method with a right-lower offset of 1 / 2 block size, which divides the current frame into (N-1)×(N-1) second blocks; wherein the block size of the first block and the second block is the same.

[0012] Optionally, the block overlay module is configured to: perform position-adaptive weighted overlay of the plurality of first motion compensation blocks and the plurality of second motion compensation blocks to obtain the plurality of overlay motion compensation blocks, wherein the weight of pixels in the first motion compensation blocks and the second motion compensation blocks that are closer to their respective boundary pixel positions is smaller.

[0013] Optionally, the overlapping region between the second motion compensation block and the corresponding plurality of first motion compensation blocks includes multiple pixel pairs, wherein each pixel pair includes a first pixel in the first motion compensation block and a second pixel in the second motion compensation block that overlaps with the first pixel; the overlapping region is divided into multiple pixel regions according to the distance between the first pixel and the second pixel in the pixel pair and their respective boundary pixel positions, and a weight is set for each pixel region, wherein the weight of the first pixel in the pixel region that is closer to the corresponding boundary pixel position is smaller, and the weight of the second pixel in the pixel region that is closer to the corresponding boundary pixel position is smaller, wherein the sum of the weights of the first pixel and the second pixel in each pixel pair is 1.

[0014] Optionally, the plurality of pixel regions include a first pixel region, a second pixel region, and a third pixel region; wherein, the first pixel region includes a region where the first pixel is a boundary pixel and the second pixel is a non-boundary pixel; the second pixel region includes a region where both the first pixel and the second pixel are non-boundary pixels and a region where both the first pixel and the second pixel are boundary pixels; the third pixel region includes a region where the first pixel is a non-boundary pixel and the second pixel is a boundary pixel; wherein, the weight of the first pixel in the first pixel region is set to be less than the weight of the second pixel; the weight of the first pixel in the second pixel region is set to be equal to the weight of the second pixel; and the weight of the first pixel in the third pixel region is set to be greater than the weight of the second pixel.

[0015] According to a third aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors, wherein the one or more processors are configured to perform a video encoding method according to this disclosure when executing the instructions.

[0016] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform a video coding method according to this disclosure.

[0017] According to a fifth aspect of this disclosure, a computer program product is provided, including computer instructions that, when executed by at least one processor, implement a video encoding method according to this disclosure.

[0018] According to a sixth aspect of this disclosure, a method for generating a bitstream is provided, comprising: generating a bitstream according to a video coding method of this disclosure.

[0019] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0020] This disclosure proposes an edge block motion estimation-based MCTF, which independently performs motion estimation and motion compensation using two types of staggered blocks, and weights the resulting two motion compensation blocks. Since the positions of the two motion compensation blocks are also staggered, continuous region pixels (e.g., non-boundary pixels or pixels far from the boundary) in one motion compensation block can be used to correct discontinuous region pixels (i.e., boundary pixels or pixels close to the boundary) in the other motion compensation block during the weighting process. In this way, block artifacts in the motion compensation frame of MCTF can be reduced, thereby improving the coding compression efficiency and compression performance of MCTF.

[0021] It will be understood that the above general description and the following detailed description are merely examples and do not limit this disclosure. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate examples according to this disclosure and, together with this description, serve to explain the principles of this disclosure.

[0023] Figure 1 This is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0024] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0025] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0026] Figure 4 This is a schematic diagram of motion-compensated temporal filtering in related technologies.

[0027] Figure 5 This is a schematic diagram illustrating a specific implementation example of MCTF in the related technology.

[0028] Figure 6 This is a schematic diagram illustrating the causes of discontinuous boundaries in related technologies.

[0029] Figure 7 This is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0030] Figure 8 This is a schematic diagram illustrating the process of MCTF based on edge block motion estimation according to an exemplary embodiment of the present disclosure.

[0031] Figure 9 This is a schematic diagram illustrating position-adaptive weighted superposition according to an exemplary embodiment of the present disclosure.

[0032] Figure 10 This is a schematic diagram illustrating a plurality of pixel regions according to an exemplary embodiment of the present disclosure.

[0033] Figure 11 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment of the present disclosure.

[0034] Figure 12 This is a diagram illustrating a computing environment coupled to a user interface according to some embodiments of the present disclosure. Detailed Implementation

[0035] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. Numerous non-limiting details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this disclosure are used to distinguish objects and are not used to describe any specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in sequences other than those shown in the drawings or described in this disclosure.

[0037] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0038] In some implementations, the target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14.

[0039] In some other implementations, the encoded video data can be sent from the output interface 22 to the storage device 32. Subsequently, the target device 14 can access the encoded video data in the storage device 32 via the input interface 28.

[0040] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0041] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback.

[0042] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0043] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0044] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0045] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding on video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0046] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of the adder 62. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or it can be distributed among one or more of the fixed or programmable hardware units described.

[0047] The video data storage device 40 can store video data encoded by the components of the video encoder 20. For example, it can store data from... Figure 1 The video source 18 shown obtains video data from the video data storage 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) used by the video encoder 20 (e.g., in intra-frame or inter-frame predictive coding mode) when encoding the video data.

[0048] like Figure 2 As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., sets of video blocks) or other larger coding units (CUs) according to a predefined splitting structure (e.g., a quadtree (QT) structure) associated with the video data. It should be noted that the term "block" or "video block" as used herein can be a portion of a frame or image, particularly a rectangular (square or non-square) portion. Referring to, for example, HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0049] The prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0050] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0051] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of a video block within the current video frame or picture relative to a prediction block within a reference frame associated with the current block being encoded in the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0052] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0053] Intra-prediction processing unit 46 can encode the current block using various intra-prediction modes, for example, during individual encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) can select a suitable intra-prediction mode from the tested intra-prediction modes for use. Intra-prediction processing unit 46 can provide information indicating the intra-prediction mode selected for the block to entropy coding unit 56. Entropy coding unit 56 can encode the information indicating the selected intra-prediction mode into the bitstream.

[0054] After prediction processing unit 41 determines the prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0055] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0056] After quantization, the entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... Figure 1 The video decoder 30 shown, or archived in, for example Figure 1 The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame being encoded.

[0057] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0058] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0059] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0060] In some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame prediction (BC) unit 85 may perform embodiments of this application individually or in combination with other units of the video decoder 30 (e.g., motion compensation unit 82, intra-frame prediction unit 84, and entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame prediction (BC) unit 85, and the functionality of the intra-frame prediction (BC) unit 85 may be performed by other components of the prediction processing unit 81 (e.g., motion compensation unit 82).

[0061] The video data storage device 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of the video decoder 30. The video data stored in the video data storage device 79 can be obtained, for example, from the storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0062] See

[0063] During the decoding process, the video decoder 30 receives a encoded video bitstream representing video blocks and associated syntax elements of encoded video frames. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0064] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0065] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0066] In some examples, when a video block is encoded according to the intra-frame BC mode described herein, the intra-frame BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within a reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0067] The motion compensation unit 82 and / or the intra-frame BC unit 85 determine the prediction information for the video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0068] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0069] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0070] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be directly provided to the DPB 92 by the adder 90. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., ...). Figure 1 On the display device 34).

[0071] MCTF is a technique that uses adjacent left and right frames to filter the current frame to be encoded. Figure 4 This is a schematic diagram illustrating motion-compensated temporal filtering in related technologies. (Refer to...) Figure 4 Assuming we use the current frame to be encoded, I t The two adjacent frames on the left t-2 and I t-1 and the two adjacent frames I on the right t+1 and I t+2 For the current frame to be encoded I t Perform filtering. Filtering can be performed in units of, for example, but not limited to, 8×8 blocks, and the positions of the filtered blocks on adjacent frames can be obtained through motion estimation.

[0072] MCTF can consist of three parts: "motion estimation," "motion compensation," and "weighted filtering." "Motion estimation" refers to finding the most similar matching block (e.g., the block with the smallest SSE) from neighboring frames of the current frame to be encoded. The position of the matching block can be represented by a motion vector (MV). For example, refer to... Figure 4 The current frame to be encoded, I t The current block on image frame It-1 The motion vector on the image frame is MV1, and the current block is in image frame I. t-2 The motion vector on the image frame is MV0, and the current block is in image frame I. t+1 The motion vector on the image frame is MV2, and the current block is in image frame I. t+2 The motion vector on the image is MV3. "Motion compensation" refers to the process of obtaining interpolated blocks (also called motion-compensated blocks) from adjacent frames of the current frame to be encoded based on MV; "weighted filtering" refers to the process of weighted superposition of the current block and the interpolated blocks. For example, refer to... Figure 4 When weighted superimposing the current block and the interpolation block, the weight corresponding to the current block can be W. T Image frame I T-1 The corresponding weighting weight can be W. T-1 Image frame I T-2 The corresponding weighting weight can be W. t-2 Image frame I t+1 The corresponding weighting weight can be W. t+1 Image frame I t+2 The corresponding weighting weight can be W. t+2 .

[0073] Below, we will refer to Figure 5 The three components of MCTF—"motion estimation," "motion compensation," and "weighted filtering"—are explained in detail. Figure 5 This is a schematic diagram illustrating a specific implementation example of MCTF in related technologies. (Refer to...) Figure 5 Assuming the current frame is I t That is, it is necessary to target the current frame I. t To perform filtering, several frames before and after the current frame are needed. For example, four frames before and after the current frame are filtered sequentially. t Perform block-level motion estimation. Assuming the number of adjacent frames N = 8, they can be denoted as: I t-4 I t-3 I t-2 I t-1 I t+1 I t+2 I t+3 I t+4 .

[0074] Motion estimation

[0075] For the current frame I respectively t Block-level motion estimation is performed with the eight adjacent frames before and after it, with the aim of searching for the current frame I. t In the context of each 8×8 block, the most similar corresponding block in the eight reference frames is the motion vector (MV) representing the position offset of the similar block relative to the current block. Since there are a total of eight reference frames, the current frame I...t Each 8×8 block in the code corresponds to 8 MVs, namely MV t-4 MV t-3、 MV t-2 MV t-1 MV t+1 MV t+2、 MV t+3 MV t+4 Furthermore, in this disclosure, a hierarchical pyramid architecture can also be used to quickly derive the MV of each 8x8 sub-block.

[0076] Exercise compensation

[0077] For the current 8×8 block, in each adjacent frame, the corresponding MV found can be directly used for classic block-level motion compensation to obtain the motion compensation block corresponding to that adjacent frame. In this way, by performing motion compensation in each of the 8 adjacent frames, we can obtain 8 best-matched interpolation blocks, i.e., 8 motion compensation blocks, corresponding one-to-one with the 8 adjacent frames.

[0078] Weighted filtering

[0079] For the current frame I t Each 8×8 block can be derived from 8 adjacent reference frames, resulting in 8 interpolation blocks. Then, for the current frame I... t The included 8×8 block is weighted proportionally with the aforementioned 8 interpolation blocks to obtain the time-domain filtering result, i.e., the block after filtering the 8×8 block. Furthermore, when weighting and superimposing the current block with the interpolation blocks, the weight corresponding to the current block can be W. t Image frame I t-1 The weighted weights corresponding to the interpolation blocks on the above can be W. t-1 Image frame I t-2 The weighted weights corresponding to the interpolation blocks on the above can be W. t-2 Image frame I t-3 The weighted weights corresponding to the interpolation blocks on the above can be W. t-3 Image frame I t-4 The weighted weights corresponding to the interpolation blocks on the above can be W. t-4 Image frame I t+1 The weighted weights corresponding to the interpolation blocks on the above can be W. t+1 Image frame I t+2 The weighted weights corresponding to the interpolation blocks on the above can be W. T+2 Image frame I t+3 The weighted weights corresponding to the interpolation blocks on the above can be W. t+3 Image frame I t+4 The weighted weights corresponding to the interpolation blocks on the above can be W. t+4Then, the filtered block can be used to replace the current frame I. t The original block in the current frame I, i.e., the block that replaces the current frame I. t The original 8×8 block before filtering.

[0080] Figure 4 and Figure 5 This demonstrates the process of performing temporal filtering on a single block. If filtering is performed on multiple adjacent blocks within the same video frame, block artifacts may appear at the boundaries between blocks. Figure 6 This is a schematic diagram illustrating the causes of discontinuous boundaries in related technologies. (Refer to...) Figure 6 Suppose motion estimation is performed on three blocks in the current frame in adjacent frames. Since the motion estimation of each block is independent of each other, three interpolated blocks with discontinuous boundaries may be found in adjacent frames. Subsequently, discontinuous boundaries will appear on the interpolated frame (also known as the motion-compensated frame) after motion compensation, resulting in block artifacts. These block artifacts will affect the compression performance of temporal filtering.

[0081] To address the aforementioned problems in related technologies, the video coding method, apparatus, electronic device, storage medium, and program products disclosed herein propose MCTF based on edge block motion estimation. This method independently performs motion estimation and motion compensation using two types of staggered blocks, and weights the resulting two motion-compensated blocks. Since the positions of the two motion-compensated blocks are also staggered, during the weighting process, continuous region pixels (e.g., non-boundary pixels or pixels far from the boundary) in one motion-compensated block can be used to correct discontinuous region pixels (i.e., boundary pixels or pixels near the boundary) in the other motion-compensated block. This reduces block artifacts in the motion-compensated frames of MCTF, thereby improving the coding compression efficiency and compression performance of MCTF.

[0082] Below, we will refer to Figures 7 to 12 This disclosure provides a detailed description of the video coding methods, apparatus, electronic devices, storage media, and program products disclosed herein.

[0083] Figure 7 This is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0084] Reference Figure 7 In step 701, the current frame can be divided into multiple first blocks according to the first division method, and the current frame can be divided into multiple second blocks according to the second division method, wherein the non-boundary pixel regions in each second block partially overlap with the boundary pixel regions of the corresponding several first blocks.

[0085] According to exemplary embodiments of this disclosure, the first partitioning method described above can be a conventional partitioning method, that is, the current frame can be divided into N×N first blocks, where N can be an integer greater than 1. For example, N can be equal to 4, in which case the current frame can be divided into N×N = 4×4 = 16 first blocks. The second partitioning method can be a partitioning method with a right-lower offset of 1 / 2 block size, that is, (N-1)×(N-1) second blocks can be obtained from the current frame. For example, when N = 4, (N-1)×(N-1) = (4-1)×(4-1) = 9 second blocks can be obtained from the current frame, where the block size of the first block and the second block can be the same. That is, according to the second partitioning method, the remaining portion of the image boundary region cannot form a minimum MCTF unit, and such remaining portion can be left unprocessed.

[0086] Figure 8 This is a schematic diagram illustrating a process of MCTF based on edge block motion estimation according to an exemplary embodiment of the present disclosure. (Refer to...) Figure 8 The frame to be filtered (i.e., the current frame) can be divided into N×N blocks as shown in number 1 according to the first division method, and into (N-1)×(N-1) blocks as shown in number 2 according to the second division method. For example, when N is 4, the conventional division method yields 16 blocks of size H×H, which are the blue solid line blocks in number 1; by shifting the division starting point to the lower right by H / 2, 9 blocks of size H×H can be obtained, which are the red solid line blocks in number 2. The blue dashed line blocks in number 2 represent the 16 blocks obtained by the conventional division method. The positional relationship of the blocks obtained by the two division methods can be visually seen through the red solid line blocks and blue dashed blocks shown in number 2. Clearly, the second partitioning method will result in parts that cannot form a complete H×H block (i.e., the parts outside the red solid line block in sequence 2). These parts will not be processed. In other words, the second partitioning method can only produce (N-1)×(N-1)=(4-1)×(4-1)=9 blocks.

[0087] Return to reference Figure 7 In step 702, motion estimation and motion compensation can be performed on each of the multiple first blocks to obtain multiple first motion compensation blocks, and motion estimation and motion compensation can be performed on each of the multiple second blocks to obtain multiple second motion compensation blocks.

[0088] Specifically, such as Figure 8 As shown, motion estimation and motion compensation can be performed on each of the N×N=4×4=16 filter blocks in sequence 1, thereby obtaining the original interpolation block in the form of sequence 3.

[0089] For the filtering block partitioning form of sequence number 2, the frame to be filtered can be shifted to the lower right by 1 / 2 block size, that is, shifted to the lower right by H / 2 size. This will divide the frame to be filtered into (N-1)×(N-1)=(4-1)×(4-1)=9 second blocks. Then, motion estimation and motion compensation can be performed on each of these 9 second blocks to obtain the boundary interpolation block of the form of sequence number 4.

[0090] It should be noted that since the positions of the original interpolation block shown in number 3 and the interpolation block shown in number 4 overlap, that is, since each interpolation block in number 4 is exactly at the boundary position of the original interpolation block shown in number 3, the interpolation block shown in number 4 can be called the "boundary interpolation block".

[0091] In step 703, multiple first motion compensation blocks and multiple second motion compensation blocks can be superimposed to obtain multiple superimposed motion compensation blocks.

[0092] Return to reference Figure 8 By weighted superimposing the original interpolation block shown in number 3 and the boundary interpolation block shown in number 4, the superimposed motion compensation block in number 5, i.e., the weighted superimposed block, can be obtained. Furthermore, the weighting weight of the original interpolation block shown in number 3 can be w0, and the weighting weight of the boundary interpolation block shown in number 4 can be w1. Thus, during the weighting process, the boundary interpolation block shown in number 5, i.e., the continuous region of the shaded area, will correct the discontinuous boundary regions in the original interpolation block, thereby reducing discontinuous boundaries in the MCTF interpolated frame, i.e., reducing block artifacts in the interpolated frame.

[0093] According to an exemplary embodiment of this disclosure, multiple first motion compensation blocks and multiple second motion compensation blocks can be adaptively weighted and superimposed to obtain multiple superimposed motion compensation blocks. The weight of pixels closer to their respective boundary pixels in the first and second motion compensation blocks can be smaller. It should be noted that by using the position-adaptive weighted superposition method, larger weights can be applied to pixels closer to continuous regions and smaller weights to pixels farther from continuous regions. This further enhances the correction effect of continuous regions on discontinuous regions.

[0094] According to an exemplary embodiment of this disclosure, the overlapping area between the second motion compensation block and the corresponding plurality of first motion compensation blocks may include multiple pixel pairs, wherein each pixel pair may include a first pixel in the first motion compensation block and a second pixel in the second motion compensation block that overlaps with the position of the first pixel.

[0095] Furthermore, overlapping regions can be divided into multiple pixel regions based on the distances between the first and second pixels in a pixel pair and their respective boundary pixels. Weights can be assigned to pixel pairs within each pixel region. Specifically, the weight of the first pixel in a pixel region closer to its corresponding boundary pixel can be smaller; similarly, the weight of the second pixel in a pixel region closer to its corresponding boundary pixel can be smaller. The sum of the weights of the first and second pixels in each pixel pair can be 1.

[0096] Figure 9 This is a schematic diagram illustrating position-adaptive weighted superposition according to an exemplary embodiment of the present disclosure. (Refer to...) Figure 9 Sequence number 5 represents the weighted superposition interpolation block, i.e., the weighted superposition motion compensation block. Sequence numbers 6 and 7 illustrate the position-adaptive weighting process, whose design logic is to increase the weight of continuous regions and decrease the weight of discontinuous regions. That is, the closer to the boundary of the first compensation block, the smaller the weight of the first pixel and the larger the weight of the second pixel; the closer to the boundary of the second compensation block, the larger the weight of the first pixel and the smaller the weight of the second pixel. In this way, it can be ensured that the weight of continuous regions is always large, and thus the degree of correction can always be large, thereby effectively reducing discontinuous boundaries, i.e., effectively reducing block artifacts in the interpolated frame.

[0097] Thus, in this disclosure, by using a position-adaptive weighted superposition method for the first motion compensation block and the second motion compensation block—that is, by applying a larger weight to pixels closer to the continuous region and a smaller weight to pixels farther from the continuous region—the correction effect of the continuous region on the discontinuous region can be further enhanced. Compared to the average weighting method, the position-adaptive weighted superposition method can better remove discontinuous boundaries, thereby more effectively reducing block artifacts in the interpolated frame.

[0098] According to an exemplary embodiment of this disclosure, the plurality of pixel regions may include a first pixel region, a second pixel region, and a third pixel region.

[0099] The first pixel region may include a region where the first pixel is a boundary pixel and the second pixel is a non-boundary pixel; the second pixel region may include a region where both the first pixel and the second pixel are non-boundary pixels and a region where both the first pixel and the second pixel are boundary pixels; the third pixel region may include a region where the first pixel is a non-boundary pixel and the second pixel is a boundary pixel.

[0100] In addition, the weight of the first pixel in the first pixel region can be set to be less than the weight of the second pixel; the weight of the first pixel in the second pixel region can be set to be equal to the weight of the second pixel; and the weight of the first pixel in the third pixel region can be set to be greater than the weight of the second pixel.

[0101] Figure 10 This is a schematic diagram illustrating a plurality of pixel regions according to an exemplary embodiment of the present disclosure. (Refer to...) Figure 10 The overlapping area between the second motion compensation block and the corresponding four first motion compensation blocks may include the first pixel region A, the second pixel region B, and the third pixel region C.

[0102] For each pixel pair in the first pixel region A, the first pixel in the pair belongs to the boundary pixels that are relatively close to the edges (e1, e2, e3, e4) of the first compensation block; while the second pixel in the pair belongs to the non-boundary pixels that are relatively far from the edges (f1, f2, f3, f4) of the second compensation block. Therefore, the weighting weight of the first pixel that is relatively close to the boundary (e1, e2, e3, e4) can be set to be less than the weighting weight of the second pixel that is relatively far from the boundary (f1, f2, f3, f4). For example, the weighting weight of the first pixel can be set to 3 / 8, and the weighting weight of the second pixel can be set to 5 / 8. This can enhance the correction effect of continuous regions on discontinuous regions.

[0103] The second pixel region B can further include two types of sub-regions: b1 and b2. In sub-region b1, the first and second pixels of each pixel pair are non-boundary pixels; in sub-region b2, the first and second pixels of each pixel pair are boundary pixels. That is, for the second pixel region B, in each of its multiple pixel pairs, the first and second pixels are either both boundary pixels or both non-boundary pixels. In this case, the continuity of the first and second pixels is equivalent, meaning their contributions to content continuity are roughly equal. Therefore, the weight of the first pixel can be set equal to the weight of the second pixel. For example, the weighted weight of the first pixel can be set to 4 / 8, and the weighted weight of the second pixel can also be set to 4 / 8.

[0104] For the third pixel region C, the first pixel in each pixel pair within this region is a non-boundary pixel that is relatively far from the edge (e1, e2, e3, e4) of the first compensation block; while the second pixel in the same pixel pair is a boundary pixel that is relatively close to the edge (f1, f2, f3, f4) of the second compensation block. Therefore, the weighting weight of the first pixel that is relatively far from the boundary (e1, e2, e3, e4) can be set to be greater than the weighting weight of the second pixel that is relatively close to the boundary (f1, f2, f3, f4). For example, the weighting weight of the first pixel can be set to 5 / 8, and the weighting weight of the second pixel can be set to 3 / 8. This enhances the correction effect of continuous regions on discontinuous regions.

[0105] It should be noted that, Figure 10 The multiple pixel regions included in the overlapping area shown are only one specific method of region division. Other methods of region division can also be used, which will not be elaborated here. The above implementation is only an example.

[0106] Return to reference Figure 7 In step 704, temporal filtering can be performed on the current frame based on multiple superimposed motion compensation blocks.

[0107] In step 705, the current frame can be encoded based on the temporal filtering result of the current frame.

[0108] Figure 11 This is a block diagram illustrating a video encoding apparatus 1100 according to an exemplary embodiment of the present disclosure.

[0109] Reference Figure 11 The video encoding device 1100 may include a block partitioning module 1101, a motion estimation compensation module 1102, a block overlay module 1103, a temporal filtering module 1104, and an encoding module 1105.

[0110] The segmentation module 1101 can divide the current frame into multiple first blocks according to the first segmentation method, and can divide the current frame into multiple second blocks according to the second segmentation method, wherein the non-boundary pixel regions in each second block partially overlap with the boundary pixel regions of the corresponding several first blocks.

[0111] According to exemplary embodiments of this disclosure, the first partitioning method described above can be a conventional partitioning method, that is, the current frame can be divided into N×N first blocks, where N can be an integer greater than 1. For example, N can be equal to 4, in which case the current frame can be divided into N×N = 4×4 = 16 first blocks. The second partitioning method can be a partitioning method with a right-lower offset of 1 / 2 block size, that is, (N-1)×(N-1) second blocks can be obtained from the current frame. For example, when N = 4, (N-1)×(N-1) = (4-1)×(4-1) = 9 second blocks can be obtained from the current frame, where the block size of the first block and the second block can be the same. That is, according to the second partitioning method, the remaining portion of the image boundary region cannot form a minimum MCTF unit, and such remaining portion can be left unprocessed.

[0112] The motion estimation and compensation module 1102 can perform motion estimation and motion compensation on each of the multiple first blocks to obtain multiple first motion compensation blocks, and can perform motion estimation and motion compensation on each of the multiple second blocks to obtain multiple second motion compensation blocks.

[0113] The block overlay module 1103 can overlay multiple first motion compensation blocks and multiple second motion compensation blocks to obtain multiple overlay motion compensation blocks.

[0114] According to an exemplary embodiment of the present disclosure, the block overlay module 1103 can perform position-adaptive weighted overlay of multiple first motion compensation blocks and multiple second motion compensation blocks to obtain multiple overlay motion compensation blocks, wherein the weight of pixels in the first motion compensation block and the second motion compensation block that are closer to their respective boundary pixel positions can be smaller.

[0115] According to an exemplary embodiment of this disclosure, the overlapping area between the second motion compensation block and the corresponding plurality of first motion compensation blocks may include multiple pixel pairs, wherein each pixel pair may include a first pixel in the first motion compensation block and a second pixel in the second motion compensation block that overlaps with the position of the first pixel.

[0116] Furthermore, overlapping regions can be divided into multiple pixel regions based on the distances between the first and second pixels in a pixel pair and their respective boundary pixels. Weights can be assigned to pixel pairs within each pixel region. Specifically, the weight of the first pixel in a pixel region closer to its corresponding boundary pixel can be smaller; similarly, the weight of the second pixel in a pixel region closer to its corresponding boundary pixel can be smaller. The sum of the weights of the first and second pixels in each pixel pair can be 1.

[0117] Thus, in this disclosure, by using a position-adaptive weighted superposition method for the first motion compensation block and the second motion compensation block—that is, by applying a larger weight to pixels closer to the continuous region and a smaller weight to pixels farther from the continuous region—the correction effect of the continuous region on the discontinuous region can be further enhanced. Compared to the average weighting method, the position-adaptive weighted superposition method can better remove discontinuous boundaries, thereby more effectively reducing block artifacts in the interpolated frame.

[0118] According to an exemplary embodiment of this disclosure, the plurality of pixel regions may include a first pixel region, a second pixel region, and a third pixel region.

[0119] The first pixel region may include a region where the first pixel is a boundary pixel and the second pixel is a non-boundary pixel; the second pixel region may include a region where both the first pixel and the second pixel are non-boundary pixels and a region where both the first pixel and the second pixel are boundary pixels; the third pixel region may include a region where the first pixel is a non-boundary pixel and the second pixel is a boundary pixel.

[0120] In addition, the weight of the first pixel in the first pixel region can be set to be less than the weight of the second pixel; the weight of the first pixel in the second pixel region can be set to be equal to the weight of the second pixel; and the weight of the first pixel in the third pixel region can be set to be greater than the weight of the second pixel.

[0121] The temporal filtering module 1104 can perform temporal filtering on the current frame based on multiple superimposed motion compensation blocks.

[0122] The encoding module 1105 can encode the current frame based on the temporal filtering result of the current frame.

[0123] Figure 12 A computing environment 1210 coupled to a user interface 1250 is shown. The computing environment 1210 may be part of a data processing server. The computing environment 1210 includes a processor 1220, a memory 1230, and an input / output (I / O) interface 1240.

[0124] Processor 1220 typically controls the overall operation of computing environment 1210, such as operations associated with display, data acquisition, data communication, and image processing. Processor 1220 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 1220 may include one or more modules that facilitate interaction between processor 1220 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0125] Memory 1230 is configured to store various types of data to support the operation of computing environment 1210. Memory 1230 may include predefined software 1232. Examples of such data include instructions for any application or method operating on computing environment 1210, video datasets, image data, etc. Memory 1230 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0126] I / O interface 1240 provides an interface between processor 1220 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 1240 can be coupled to encoders and decoders.

[0127] In an embodiment, a non-transitory computer-readable storage medium is also provided, including, for example, a plurality of programs in a memory 1230 and / or a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method. The plurality of programs can be executed by a processor 1220 in a computing environment 1210 to perform the above-described methods. In one example, the plurality of programs can be executed by a processor 1220 in a computing environment 1210 to (e.g., from...) Figure 2 The video encoder 20 in the computing environment 1210 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1220 in the computing environment 1210 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 1220 in the computing environment 1210 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 1220 in the computing environment 1210 to (e.g., to...) Figure 3 The video decoder 30 in the middle sends the bitstream or data stream. Alternatively, a non-transitory computer-readable storage medium may store data generated by the encoder (e.g., Figure 2 The video encoder 20 in the video encoder (e.g., the one described above) generates the video for the decoder (e.g., the one described above). Figure 3The video decoder 30 in the video decoder uses a bitstream or data stream that includes encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.) when decoding video data. Non-transitory computer-readable storage media can be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage devices, etc.

[0128] In one embodiment, a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method is provided. In another embodiment, a bitstream comprising encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method is provided.

[0129] In one embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 1220); and a non-transitory computer-readable storage medium or memory 1230 therein storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0130] In one embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, the bitstream including encoded video information generated by the encoding method described above or encoded video information to be decoded by the decoding method described above. In another embodiment, a computer program product including, for example, a plurality of programs in a memory 1230 is also provided, the plurality of programs being executable by a processor 1220 in a computing environment 1210 to perform the methods described above. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0131] In an embodiment, the computing environment 1210 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0132] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a digital storage medium, wherein the bitstream includes encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method.

[0133] In one embodiment, a method for transmitting a bitstream generated by the encoder described above is also provided. In another embodiment, a method for receiving a bitstream to be decoded by the decoder described above is also provided.

[0134] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0135] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0136] The examples chosen and described are intended to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. A video encoding method, characterized in that, include: The current frame is divided into multiple first blocks according to the first division method, and into multiple second blocks according to the second division method, wherein the non-boundary pixel regions in each second block partially overlap with the boundary pixel regions of the corresponding several first blocks; Motion estimation and motion compensation are performed on each of the plurality of first blocks to obtain a plurality of first motion compensation blocks, and motion estimation and motion compensation are performed on each of the plurality of second blocks to obtain a plurality of second motion compensation blocks; The plurality of first motion compensation blocks and the plurality of second motion compensation blocks are superimposed to obtain a plurality of superimposed motion compensation blocks; Based on the multiple superimposed motion compensation blocks, temporal filtering is performed on the current frame; The current frame is encoded based on the temporal filtering result of the current frame; The step of superimposing the plurality of first motion compensation blocks and the plurality of second motion compensation blocks to obtain a plurality of superimposed motion compensation blocks includes: The plurality of first motion compensation blocks and the plurality of second motion compensation blocks are adaptively weighted and superimposed to obtain the plurality of superimposed motion compensation blocks, wherein the pixels in the first motion compensation blocks and the second motion compensation blocks that are closer to their respective boundary pixel positions have smaller weights.

2. The video encoding method according to claim 1, characterized in that, The first partitioning method is a conventional partitioning method, which divides the current frame into N×N first blocks, where N is an integer greater than 1; The second division method is a division method with a right-lower offset of 1 / 2 block size, which divides the current frame into (N-1) × (N-1) second blocks; The first and second blocks are the same size.

3. The video encoding method according to claim 1, characterized in that, The overlapping area between the second motion compensation block and the corresponding plurality of first motion compensation blocks includes multiple pixel pairs, wherein each pixel pair includes a first pixel in the first motion compensation block and a second pixel in the second motion compensation block that overlaps with the position of the first pixel; The overlapping area is divided into multiple pixel regions based on the distance between the first and second pixels in the pixel pair and their respective boundary pixels. A weight is set for each pixel pair in each pixel region. The weight of the first pixel in a pixel region that is closer to the corresponding boundary pixel is smaller, and the weight of the second pixel in a pixel region that is closer to the corresponding boundary pixel is smaller. The sum of the weights of the first and second pixels in each pixel pair is 1.

4. The video encoding method according to claim 3, characterized in that, The plurality of pixel regions includes a first pixel region, a second pixel region, and a third pixel region; The first pixel region includes a region where the first pixel is a boundary pixel and the second pixel is a non-boundary pixel; the second pixel region includes a region where both the first pixel and the second pixel are non-boundary pixels and a region where both the first pixel and the second pixel are boundary pixels; the third pixel region includes a region where the first pixel is a non-boundary pixel and the second pixel is a boundary pixel. In the first pixel region, the weight of the first pixel is set to be less than the weight of the second pixel; in the second pixel region, the weight of the first pixel is set to be equal to the weight of the second pixel; and in the third pixel region, the weight of the first pixel is set to be greater than the weight of the second pixel.

5. A video encoding device, characterized in that, include: The block division module is configured to divide the current frame into multiple first blocks according to a first division method, and divide the current frame into multiple second blocks according to a second division method, wherein each second block includes the boundary pixels of a corresponding number of first blocks; The motion estimation and compensation module is configured to perform motion estimation and motion compensation on each of the plurality of first blocks to obtain a plurality of first motion compensation blocks, and to perform motion estimation and motion compensation on each of the plurality of second blocks to obtain a plurality of second motion compensation blocks; The block stacking module is configured to stack the plurality of first motion compensation blocks and the plurality of second motion compensation blocks to obtain a plurality of stacked motion compensation blocks; The temporal filtering module is configured to perform temporal filtering on the current frame based on the plurality of superimposed motion compensation blocks; The encoding module is configured to encode the current frame based on the temporal filtering result of the current frame; The block overlay module is configured as follows: The plurality of first motion compensation blocks and the plurality of second motion compensation blocks are adaptively weighted and superimposed to obtain the plurality of superimposed motion compensation blocks, wherein the pixels in the first motion compensation blocks and the second motion compensation blocks that are closer to their respective boundary pixel positions have smaller weights.

6. An electronic device, comprising: One or more processors; as well as A memory, coupled to the one or more processors and configured to store instructions executable by the one or more processors. The one or more processors are configured to perform the video encoding method according to any one of claims 1-4 when executing the instructions.

7. A non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform the video encoding method according to any one of claims 1-4.

8. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by at least one processor, they implement the video encoding method according to any one of claims 1 to 4.

9. A method for generating a bitstream, comprising: The video encoding method according to any one of claims 1 to 4 generates a bitstream.

Images