Full pixel-based and group-based motion compensated picture boundary filling
Through the full-pixel motion compensation filling technology, the problems of low motion compensation filling efficiency and high computational complexity at image boundaries in the prior art are solved. The motion vector and sample replication method with integer accuracy are adopted to improve the efficiency of video encoding and decoding.
Patent Information
- Application Number
- CN202380018205.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-08
- Filing Date
- 2023-06-14
- Publication Date
- 2024-11-08
AI Technical Summary
When existing video encoding technology deals with motion compensation filling at image boundaries, there are problems of low efficiency and high computational complexity. Especially in motion compensation operations with fractional accuracy, the direct memory access method is not applicable, resulting in memory bandwidth access. and the computational complexity is significant.
Using full-pixel motion compensation filling technology, by cropping or rounding the fractional accuracy part of the initial motion vector, the final motion vector with integer accuracy is generated, the samples in the reference picture are directly copied as the fill pixel, and offset and filtering are applied if necessary. processing to improve efficiency.
It improves the efficiency and computational complexity of image boundary filling, reduces memory bandwidth and computing requirements, and realizes a more efficient video encoding and decoding process.
Smart Images

Figure CN118923114A_ABST
Abstract
Description
Incorporated by Reference
[0001] This application claims the benefit of priority to U.S. Patent Application No. 18 / 207,541, filed on June 8, 2023, for "FULL-PIXELBASED AND GROUP-BASED MOTION COMPENSATED PICTURE BOUNDARY PADDING," which claims the benefit of priority to U.S. Provisional Application No. 63 / 388,598, filed on July 12, 2022, for "Picture Boundary Fractional Motion Compensation Free Padding and GroupingBased MC Padding." The disclosure of the prior application is incorporated herein by reference in its entirety. Technical Field
[0002] This disclosure describes implementations generally related to video encoding. Background Art
[0003] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. To the extent the work is described in this background section, the work of the presently named inventors and aspects of the description that may not otherwise be qualified as prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art against the present disclosure.
[0004] Image / video compression can help to transmit image / video files across different devices, storages, and networks with minimal quality degradation. In some examples, video codec technology can compress video based on spatial redundancy and temporal redundancy. In an example, a video codec can use a technique called intra-frame prediction, which can compress images based on spatial redundancy. For example, intra-frame prediction can use reference data from the current picture being reconstructed for sample prediction. In another example, a video codec can use a technique called inter-frame prediction, which can compress images based on temporal redundancy. For example, inter-frame prediction can use motion compensation to predict samples in the current picture based on previously reconstructed pictures. Motion compensation is usually indicated by a motion vector (MV). Summary of the invention
[0005] Aspects of the present disclosure provide methods and apparatus for video and / or picture encoding / decoding. The apparatus for video decoding includes a processing circuit system that receives a bitstream including multiple pictures. An initial motion vector (MV) associated with a motion compensation padding (MCP) block is determined based on motion information of a boundary block at a picture boundary of a picture in the multiple pictures. The boundary block is located in the picture. The MCP block is located in an MCP region outside the picture and adjacent to the picture boundary. The initial MV has fractional precision. A final MV with integer precision associated with the MCP block is generated based on the initial MV. At least samples in the MCP block are reconstructed based on the generated final MV.
[0006] In an embodiment, an MV associated with an MCP block is determined based on multiple pieces of motion information of corresponding multiple boundary blocks at picture boundaries of pictures in multiple pictures. The multiple boundary blocks are located in the picture, and the MCP block is located in an MCP region outside the picture and adjacent to the picture boundary. Each of the multiple boundary blocks is a spatially adjacent block of a boundary block in the multiple boundary blocks. The multiple boundary blocks are adjacent to the MCP block. At least a sample in the MCP block is determined based on the determined MV.
[0007] In an embodiment, an MV associated with an MCP block is determined based on motion information of a boundary block located at a picture boundary of a picture. The boundary block is located in the picture. The MCP block is located in an MCP region outside the picture and adjacent to the picture boundary of the picture. Motion compensated padding is used to determine samples in the MCP block based on the determined MV. Samples in the MCP block are filtered based on at least one sample within the picture within N1 rows of the picture boundary of the picture.
[0008] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions, which, when executed by a computer for video encoding / decoding, cause the computer to perform a method for video encoding / decoding. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Additional features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:
[0010] Figure 1 is a schematic illustration of an exemplary block diagram of a communication system (100).
[0011] Figure 2 is a schematic illustration of an exemplary block diagram of a decoder.
[0012] Figure 3 is a schematic illustration of an exemplary block diagram of an encoder.
[0013] FIG. 4A to FIG. 4B An example of a reference picture with an extended area is shown.
[0014] Figure 5 An example showing motion compensated border filling is shown.
[0015] Figures 6 to 8 An example showing group-based motion compensated boundary filling is shown.
[0016] Fig. 9 An exemplary flow chart outlining the encoding process is shown.
[0017] Figures 10 to 12 An exemplary flow chart outlining the decoding process is shown.
[0018] Fig.13 is a schematic illustration of a computer system according to an embodiment. DETAILED DESCRIPTION
[0019] Figure 1 A block diagram of a video processing system (100) in some examples is shown. The video processing system (100) is an example of an application of the disclosed subject matter, a video encoder and a video decoder in a streaming environment. The disclosed subject matter can be equally applied to other video-enabled applications including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.
[0020] The video processing system (100) includes a capture subsystem (113) that may include a video source (101), such as a digital camera, that creates, for example, an uncompressed video picture stream (102). In an example, the video picture stream (102) includes samples captured by the digital camera. The video picture stream (102) is depicted as a thick line to emphasize the high amount of data when compared to the encoded video data (104) (or encoded video bitstream), which may be processed by an electronic device (120) coupled to the video source (101) including a video encoder (103). The video encoder (103) may include hardware, software, or a combination thereof to implement or embody aspects of the disclosed subject matter as described in more detail below. The encoded video data (104) (or encoded video bitstream) is depicted as a thin line to emphasize the lower data volume when compared to the video picture stream (102), which can be stored on the streaming server (105) for future use. One or more streaming client subsystems such as Figure 1The client subsystems (106) and (108) in the streaming server (105) can access the streaming server (105) to retrieve copies (107) and (109) of the encoded video data (104). The client subsystem (106) can include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and creates an output video picture stream (111) that can be presented on a display (112) (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (104), (107) and (109) (e.g., a video bitstream) can be encoded according to certain video encoding / compression standards. Examples of these standards include ITU-T Recommendation H.265. In the example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed subject matter can be used in the context of VVC.
[0021] Note that the electronic devices (120) and (130) may include other components (not shown). For example, the electronic device (120) may include a video decoder (not shown), and the electronic device (130) may also include a video encoder (not shown).
[0022] Figure 2 An exemplary block diagram of a video decoder (210) is shown. The video decoder (210) may be included in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., receiving circuitry). The video decoder (210) may be used to replace Figure 1 A video decoder (110) is shown in an example.
[0023] The receiver (231) can receive one or more encoded video sequences to be decoded by the video decoder (210). In an embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequence can be received from a channel (201), which can be a hardware / software link to a storage device storing the encoded video data. The receiver (231) can receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams, which can be forwarded to their respective use entities (not depicted). The receiver (231) can separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (215) can be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other applications, the buffer memory (215) can be external to the video decoder (210) (not depicted). In still other applications, a buffer memory (not depicted) may be present outside the video decoder (210) to, for example, prevent network jitter, and another buffer memory (215) may be present inside the video decoder (210) to, for example, handle playout timing. When the receiver (231) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous network, the buffer memory (215) may not be required, or the buffer memory (215) may be smaller. In order to make the most of a packet network such as the Internet, a buffer memory (215) may be required, which may be relatively large and may advantageously have an adaptive size, and may be implemented at least in part in an operating system or similar element (not depicted) external to the video decoder (210).
[0024] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the encoded video sequence. The categories of these symbols include information for managing the operation of the video decoder (210) and potential information for controlling a presentation device such as a presentation device (212) (e.g., a display screen) that is not part of the electronic device (230) but can be coupled to the electronic device (230), such as Figure 2As shown. The control information for the rendering device may be in the form of a supplementary enhancement information (SEI) message or a video usability information (VUI) parameter set fragment (not depicted). The parser (220) may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (220) may extract a subgroup parameter set of at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a group of pictures (Group of Pictures, GOP), a picture, a tile, a slice, a macroblock, a coding unit (Coding Unit, CU), a block, a transform unit (Transform Unit, TU), a prediction unit (Prediction Unit, PU), etc. The parser (220) may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.
[0025] The parser (220) may perform entropy decoding / parsing operations on a video sequence received from the buffer memory (215) to create symbols (221).
[0026] Depending on the type of coded video picture or part of a coded video picture (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors, the reconstruction of the symbol (221) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (220). For the sake of brevity, such subgroup control information flow between the parser (220) and the following multiple units is not depicted.
[0027] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into a number of functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purposes of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.
[0028] The first unit is a sealer / inverse transform unit (251). The sealer / inverse transform unit (251) receives quantized transform coefficients as symbols (221) from the parser (220) and control information including which transform to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (251) can output a block including sample values, which can be input into an aggregator (255).
[0029] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to an intra-coded block. An intra-coded block may be a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of a current picture. Such predictive information may be provided by an intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) uses surrounding already reconstructed information obtained from a current picture buffer (258) to generate a block of the same size and shape as the block being reconstructed. For example, the current picture buffer (258) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (255) adds the prediction information already generated by the intra-prediction unit (252) to the output sample information as provided by the scaler / inverse transform unit (251) on a per-sample basis.
[0030] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to an inter-frame coded block and potentially to a motion compensated block. In this case, the motion compensated prediction unit (253) may access the reference picture memory (257) to obtain samples for prediction. After motion compensation of the obtained samples according to the symbols (221) belonging to the block, these samples may be added to the output of the scaler / inverse transform unit (251) (in this case referred to as residual samples or residual signals) by the aggregator (255) to generate output sample information. The address within the reference picture memory (257) from which the motion compensated prediction unit (253) obtains the predicted samples may be controlled by a motion vector, which may be obtained by the motion compensated prediction unit (253) in the form of a symbol (221), which may have, for example, an X component, a Y component and a reference picture component. Motion compensation may also include interpolation of sample values obtained from the reference picture memory (257) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.
[0031] The output samples of the aggregator (255) may be subjected to various loop filtering techniques in a loop filter unit (256). The video compression techniques may include in-loop filter techniques controlled by parameters included in the coded video sequence (also referred to as the coded video bitstream) and available to the loop filter unit (256) as symbols (221) from the parser (220). The video compression may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of a coded picture or coded video sequence, and to previously reconstructed and loop filtered sample values.
[0032] The output of the loop filter unit (256) may be a sample stream that may be output to a rendering device (212) and stored in a reference picture memory (257) for use in future inter-picture prediction.
[0033] Once fully reconstructed, certain coded pictures can be used as reference pictures for future predictions. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture has been identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) can become part of the reference picture memory (257) and a new current picture buffer can be reallocated before starting to reconstruct a subsequent coded picture.
[0034] The video decoder (210) may perform decoding operations according to a standard or predetermined video compression technology such as ITU-T H.265 Recommendation. The coded video sequence may conform to the syntax specified by the video compression technology or standard used in the sense that the coded video sequence follows both the syntax of the video compression technology or standard and the profile recorded in the video compression technology or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technology or standard as tools available only under the profile. For compliance, the complexity of the coded video sequence is also required to be within a range as defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further defined by the Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.
[0035] In an embodiment, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The video decoder (210) may use the additional data to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.
[0036] Figure 3 An exemplary block diagram of a video encoder (303) is shown. The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., a transmission circuit system). The video encoder (303) can be used to replace Figure 1 A video encoder (103) in an example.
[0037] The video encoder (303) can be obtained from a video source (301) (not Figure 3 In another example, the video source (301) is a part of the electronic device (320).
[0038] The video source (301) may provide a source video sequence in the form of a digital video sample stream to be encoded by a video encoder (303), the digital video sample stream may have any suitable bit depth (e.g., 8 bits, 10 bits, 12 bits, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (301) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The pictures themselves may be organized as a spatial pixel array, wherein each pixel may include one or more samples, depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be easily understood by those skilled in the art. The following description focuses on samples.
[0039] According to an embodiment, the video encoder (303) can encode and compress the pictures of the source video sequence into an encoded video sequence (343) in real time or under any other time constraints required. Implementing the appropriate encoding speed is a function of the controller (350). In some embodiments, the controller (350) controls other functional units as described below and is functionally coupled to the other functional units. For the sake of brevity, the coupling is not depicted. The parameters set by the controller (350) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology, ...), picture size, picture group (Group of Picture, GOP) layout, maximum motion vector search range, etc. The controller (350) can be configured to have other suitable functions belonging to the video encoder (303) optimized for a specific system design.
[0040] In some embodiments, the video encoder (303) is configured to operate in a decoding loop. As a simplified description, in an example, the decoding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols and creates sample data in a manner similar to that which the (remote) decoder will also create. The reconstructed sample stream (sample data) is input to a reference picture memory (334). Since the decoding of the symbol stream produces a bit-accurate result that is independent of the decoder location (local or remote), the contents of the reference picture memory (334) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction part of the encoder are exactly the same as the sample values that the decoder will "see" when using prediction during decoding. The basic principles of reference picture synchronization (and drift caused when synchronization cannot be maintained, such as due to channel errors) are also used in some related technologies.
[0041] The operation of the "local" decoder (333) can be combined with the "remote" decoder such as has been described above. Figure 2 The operation of the video decoder (210) described in detail is the same. However, reference is also briefly made to Figure 2 , since the symbols are available and the encoding of the symbols into a coded video sequence by the entropy encoder (345) and the decoding of the symbols by the parser (220) can be lossless, the entropy decoding portion of the video decoder (210) including the buffer memory (215) and the parser (220) may not be fully implemented in the local decoder (333).
[0042] In an embodiment, decoder techniques other than parsing / entropy decoding present in a decoder are present in a corresponding encoder in the same or substantially the same functional form. Therefore, the disclosed subject matter focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are contrary to the decoder techniques described comprehensively. A more detailed description is provided below in some parts.
[0043] In some examples, during operation, the source encoder (330) may perform motion compensated predictive coding, which predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures." In this manner, the encoding engine (332) encodes the differences between pixel blocks of the input picture and pixel blocks of a reference picture, which may be selected as a prediction reference for the input picture.
[0044] The local video decoder (333) can decode the encoded video data of the picture that can be designated as the reference picture based on the symbols created by the source encoder (330). The operation of the encoding engine (332) can advantageously be a lossy process. When the encoded video data can be decoded at the video decoder ( Figure 3 When the video encoder (303) is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture memory (334). In this way, the video encoder (303) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the remote video decoder.
[0045] The predictor (335) may perform a prediction search for the encoding engine (332). That is, for a new picture to be encoded, the predictor (335) may search the reference picture memory (334) for sample data (as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors, block shapes, etc. that may be used as appropriate prediction references for the new picture. The predictor (335) may operate on a pixel block by pixel block basis to find an appropriate prediction reference. In some cases, as determined by the search results obtained by the predictor (335), the input picture may have prediction references taken from multiple reference pictures stored in the reference picture memory (334).
[0046] The controller (350) can manage encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for encoding video data.
[0047] The outputs of all the above mentioned functional units may be subjected to entropy coding in an entropy encoder (345). The entropy encoder (345) converts the symbols generated by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.
[0048] The transmitter (340) can buffer the encoded video sequence created by the entropy encoder (345) in preparation for transmission via a communication channel (360), which can be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (340) can combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).
[0049] The controller (350) may manage the operation of the video encoder (303). During encoding, the controller (350) may assign a certain encoding picture type to each encoded picture, which may affect the encoding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned one of the following picture types:
[0050] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of those variations of I pictures and their corresponding applications and features.
[0051] A predictive picture (P picture) may be a picture that may be encoded and decoded using intra prediction or inter prediction for predicting sample values of each block using at most one motion vector and a reference index.
[0052] Bidirectional predictive pictures (B pictures) can be pictures that can be encoded and decoded using intra-prediction or inter-prediction that predicts sample values for each block using up to two motion vectors and reference indices. Similarly, multi-predictive pictures can use more than two reference pictures and associated metadata for reconstructing a single block.
[0053] The source picture may typically be spatially subdivided into a number of blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples, respectively), and encoded on a block-by-block basis. The blocks may be predictively coded with reference to other (coded) blocks, which are determined by the coding allocation applied to the corresponding picture of the block. For example, blocks of an I picture may be non-predictively coded, or may be predictively coded (spatial prediction or intra-prediction) with reference to coding blocks of the same picture. Pixel blocks of a P picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B picture may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.
[0054] The video encoder (303) may perform encoding operations according to a standard or predetermined video encoding technique such as ITU-T H.265 Recommendation. In the operation of the video encoder (303), the video encoder (303) may perform various compression operations, including predictive encoding operations that exploit temporal redundancy and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.
[0055] In an embodiment, the transmitter (340) may transmit additional data along with the encoded video. The source encoder (330) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.
[0056] Video can be captured as multiple source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often referred to simply as intra-picture prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In an example, a particular picture in encoding / decoding, which is referred to as the current picture, is divided into blocks. In the case where a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture can be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case of using multiple reference pictures, the motion vector may have a third dimension that identifies the reference picture.
[0057] In some embodiments, a bidirectional prediction technique may be used in inter-picture prediction. According to the bidirectional prediction technique, two reference pictures are used, such as a first reference picture and a second reference picture that are both before the current picture in the video in decoding order (but may be in the past and in the future in display order, respectively). A block in the current picture may be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block may be predicted by a combination of the first reference block and the second reference block.
[0058] In addition, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.
[0059] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, a picture in a video picture sequence is divided into coding tree units (CodingTree Unit, CTU) for compression, and the CTUs in the picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Typically, a CTU includes three coding tree blocks (Coding Tree Block, CTB), each of which is a luminance CTB and two chrominance CTBs. Each CTU can be recursively divided into one or more coding units (Coding Unit, CU) in a quadtree. For example, a 64×64 pixel CTU can be divided into a 64×64 pixel CU, or 4 32×32 pixel CUs, or 16 16×16 pixel CUs. In the example, each CU is analyzed to determine the prediction type for the CU, such as an inter-frame prediction type or an intra-frame prediction type. The CU is divided into one or more prediction units (Prediction Unit, PU) according to temporal and / or spatial predictability. Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In an embodiment, the prediction operation in decoding (encoding / decoding) is performed in units of prediction blocks. Using a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values (e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.
[0060] Note that the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using any suitable technology. In an embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more processors executing software instructions.
[0061] Various aspects of the present disclosure provide techniques for motion compensated picture boundary filling (also referred to as motion compensated boundary filling or motion compensated filling (MCP)). Techniques for motion compensated picture boundary filling may include using full pixel (or full sample) motion compensation data for filling operations performed at picture boundaries of a picture. By using full pixel motion compensation, direct memory access (DMA) (e.g., simple DMA) can be used to obtain full pixel data from a corresponding reference picture as filling pixel data in an MCP region at a picture boundary without any fractional interpolation operations. The MCP region is outside the picture. Motion vector (MV) grouping methods can also be used to improve the access efficiency of a memory bus.
[0062] Various inter prediction modes can be used in VVC. For an inter-predicted CU, motion parameters may include motion vectors, one or more reference picture indices, a reference picture list usage index, and additional information for certain coding features of inter-prediction sample generation. Motion parameters may be signaled explicitly or implicitly. When a CU is encoded in skip mode, the CU may be associated with a PU and may have no significant residual coefficients, no encoded motion vector increments or MV differences (e.g., MVD), or reference picture indices. A merge mode may be specified in which the motion parameters of the current CU are obtained from neighboring CUs, including spatial and / or temporal candidates, and optional additional information introduced, for example, in VVC. The merge mode may be applied to inter-predicted CUs, not just for skip mode. In an example, an alternative to the merge mode is explicit transmission of motion parameters, in which, for each CU, motion information including, for example, MVs, corresponding reference picture indices for each reference picture list, reference picture list usage flags, and other information is explicitly signaled.
[0063] In an embodiment, for example in VVC, the VVC test model (VTM) reference software includes one or more refined inter-frame prediction coding tools, and the one or more refined inter-frame prediction coding tools include: extended merge prediction, merge motion vector difference (MMVD) mode, adaptive motion vector prediction (AMVP) mode with symmetric MVD signaling, affine motion compensation prediction, subblock-based temporal motion vector prediction (SbTMVP), adaptive motion vector resolution (AMVR), motion field storage (1 / 16 luminance sample MV storage and 8×8 motion field compression), bi-prediction with CU-level weights (BCW), bi-directional optical flow (BDOF), prediction refinement using optical flow (PROF), decoder side motion vector refinement (DEF), decoder side motion vector refinement, DMVR), combined inter-frame and intra-frame prediction (CIIP), geometric partitioning mode (GPM), etc.
[0064] Examples of inter-frame prediction and related methods (e.g., refined inter-frame prediction coding tools) are described below. In order to use TMVP in merge mode and / or AMVP mode, in some examples, such as in VVC, the motion data is stored in a collocated reference frame (or collocated reference picture). In the example, the motion data is stored on an 8×8 block basis. For example, in VVC, in order to limit the storage of MVs used for temporal prediction, a 10-bit mantissa exponent representation method can be used to compress the stored MVs. In some embodiments, such as in ECM5, the granularity of motion storage is reduced to a 4×4 block basis.
[0065] Techniques for picture boundary padding can be used. The padded portion outside the picture boundary can be used to predict other pictures. For example, in VVC and ECM-4.0, the extended picture area (also referred to as the extended area) is an area surrounding the picture with a size of (maximum CU width + 16) in each direction of the picture boundary. Pixels in the extended picture area (also referred to as repeated padding pixels) can be derived by repeated boundary padding. When the reference block is partially or completely outside the picture boundary (OOB), the repeated padding pixels in the extended area can be used for motion compensation similar to the pixels in the reference picture.
[0066] Figure 4A An example of a reference picture (401) having an extended region (also referred to as a repeated fill region or fill region) (410) is shown, and in some examples (e.g., ECM-4.0), the extended region is filled according to repeated boundary filling. In the example, in a first round of repeated boundary filling, a first pixel in the extended region that is an immediate neighbor of a pixel within the picture boundary of the reference picture (401) is filled based on a pixel within the picture boundary of the reference picture (401); in a second round of repeated boundary filling, a second pixel in the extended region that is an immediate neighbor of the first pixel is filled based on the first pixel; and repeated boundary filling can continue until all pixels in the extended region are filled.
[0067] In a related example, a technique for motion compensated boundary filling may be used. For example, samples outside of a picture boundary are derived by motion compensation rather than using only repetitive filling. Figure 4B An example of a reference picture with an extended region (also referred to as a padding region) filled according to motion compensated boundary filling and repeated boundary filling is shown. Figure 4B In the example, Figure 4A Compared to the padding area (410) in , the total padding area size can be increased by a predefined value L (e.g., 64). Figure 4B The extended area in may include a first portion (470) that may be filled according to motion compensation boundary filling. The first portion (470) may also be referred to as a motion compensation (MC) filling area (or MCP area). Figure 4B The extended region in may include a repeated fill region (480) based on, for example, Figure 4A The MCP region described in is filled by repeated filling.
[0068] Reference FIG. 4A to FIG. 4B When motion compensation is performed with an MV pointing to a block outside a frame boundary (or picture boundary) of a reference picture (e.g., (401) or (451)), the repeated filling area (410) or Figure 4BThe padding pixels in the extended area (eg, including the repeated padding area (480) and the MCP area (470)) in the image processing unit may be used as reference pixels.
[0069] Figure 5 An example of motion compensated (MC) border filling is shown. In MC border filling, MV (561) can be derived based on a boundary block (BBlk) (e.g., a 4×4 boundary block) (511) in a picture (551). For example, MV (561) is determined based on motion information of the boundary block (511). In the case where the boundary block (511) is intra-coded, the motion information of the boundary block (511) is not available and zero MV can be used. For example, MV (561) is set to zero MV.
[0070] In the case where the boundary block (511) is encoded with bidirectional inter-frame prediction, the motion information of the boundary block (511) may include two MVs, such as a first MV pointing to a first pixel position in a first reference picture and a second MV pointing to a second pixel position in a second reference picture. In the example, only one MV (e.g., the first MV or the second MV) is used in MC boundary filling. The MV used in MC boundary filling may point to a pixel position farther from a picture boundary in a reference picture of picture (551). For example, if the first pixel position is farther from a first picture boundary in a first reference picture of picture (551) than the second pixel position is from a second picture boundary in a second reference picture of picture (551), the first MV is used (e.g., MV (561) is the first MV). Otherwise, the second MV may be used (e.g., MV (561) is the second MV).
[0071] The MV (561) can be used to derive a padding block (or MCP block) (e.g., an L×4 or 4×L padding block) (521) in an MC padding region (570) of a picture (551). For example, the MV (561) points to a reference block (also referred to as a reference boundary block or Ref BBlk) (531) in a reference picture (501) of the picture (551). An MCP reference block (541) adjacent to the reference block (531) in the reference picture (501) can be used to determine the MCP block (521). The size of the MCP reference block (541) can be equal to the size of the MCP block (521). In the example, the size of the reference block (531) is equal to the size of the boundary block (511).
[0072] In some examples, pixels in an MC padding block (e.g., MCP block (521)) are modified (e.g., corrected) with an offset (e.g., a DC offset or a DC offset value). Figure 5The samples in the boundary block (511) in the reference picture (501) and the corresponding reference block (e.g., Figure 5 The offset is determined by using a reference sample in Ref BBlk (531) in the reference picture (501). The offset may be equal to a difference between a DC value (e.g., an average pixel value) of a reconstructed boundary block (e.g., boundary block (511)) in the picture (551) and a corresponding reference block (e.g., Ref BBlk (531)) in the reference picture (501).
[0073] Figure 5 The MC boundary filling method described in can be combined with the repeated filling method to determine the extended area of the picture (551), such as Figure 4B The value L may indicate a desired frame boundary extension (or a desired picture boundary extension) to determine an extension region of the reference picture (451) as described in.
[0074] In the example, two different padding area sizes are implemented, such as 64 ( Figure 4B ) and 16. In the case where MV (561) points to a position inside the reference picture boundary, L may be set to at least equal to 4. In the example, in the case where L is less than 64, this may be accomplished by using, for example Figure 4A The repetitive padding described in fills the remainder of the MC padding area (570) with repetitive padding samples.
[0075] In a related example, fractional motion compensation is used to determine an MCP block (e.g., an L×4 block) such as the MCP block (521) in the picture (551), for example, where the MV (561) has a fractional precision portion. In some examples, when fractional motion compensation is used to determine the L×4 MCP block, more than L×4 pixels are to be retrieved from memory for MC padding. When fractional motion compensation is used, the computational complexity and memory bandwidth access may be significant. Due to the complex motion compensation operation, a simple DMA method may not be applied to fractional motion compensation.
[0076] According to an embodiment of the present disclosure, an MV with integer sample (or integer pixel) precision (e.g., 1-pixel) may be used in the MC filling process of a picture. When the MV associated with a block to be MC filled (e.g., an MCP block or an MC filling block) is integer precision, the MCP reference block pointed to by the integer MV may be directly copied from the reference picture without fractional motion compensation. When the MV of the MCP block to be MC filled (or the MV associated with the MCP block) is fractional precision, an integer pixel MV associated with the MCP block may be generated based on the MV associated with the MCP block with fractional precision. In an embodiment, the MV associated with the MCP block (e.g., MV (561)) may be determined based on motion information of a boundary block (e.g., boundary block (511)), e.g. Figure 5 As described in. A boundary block may be adjacent to an MCP block. A boundary block and an MCP block may be separated by a picture boundary of a picture, a boundary block may be within a picture, and an MCP block may be outside a picture.
[0077] In an embodiment, an MV associated with an MCP block (e.g., MV (561) of MCP block (521)) has fractional precision, and the MV associated with the MCP block may be clipped (e.g., by a simple clipping operation) to generate a final MV with integer precision. The clipping operation may be applied to the values of the components of the MV associated with the MCP block. For example, the value of the X component of the MV is clipped to keep only the integer sample precision portion, and the fractional precision portion of the value of the X component of the MV is discarded. For example, the value of the Y component of the MV is clipped to keep only the integer sample precision portion, and the fractional precision portion of the value of the Y component of the MV is discarded.
[0078] The MV associated with the MCP block (eg, the MV (561) of the MCP block (521)) has fractional precision, and the MV associated with the MCP block may be rounded to generate a final MV having integer precision.
[0079] In an embodiment, the MV associated with the MCP block may be rounded toward zero. For example, if the X component of the MV has a fractional precision portion, the value of the X component of the MV may be rounded toward 0. For example, if the Y component of the MV has a fractional precision portion, the value of the Y component of the MV may be rounded toward 0.
[0080] In an embodiment, the MV associated with the MCP block may be rounded toward positive infinity or negative infinity to obtain a final MV associated with the MCP block. In an example, the MV associated with the MCP block is rounded toward positive infinity, for example, in the case where the X component of the MV has a fractional precision portion, the value of the X component of the MV is rounded toward positive infinity. For example, in the case where the Y component of the MV has a fractional precision portion, the value of the Y component of the MV is rounded toward positive infinity.
[0081] In an example, an MV associated with an MCP block is rounded toward negative infinity, for example, if the X component of the MV has a fractional precision portion, the value of the X component of the MV may be rounded toward negative infinity. For example, if the Y component of the MV has a fractional precision portion, the value of the Y component of the MV may be rounded toward negative infinity.
[0082] In an embodiment, sign-based rounding may be applied to the MV associated with the MCP block to generate a final MV with integer precision. In the case where a component of the MV (e.g., an X component or a Y component) is positive, the positive component of the MV may be rounded toward positive infinity, for example, if the positive component of the MV has a fractional precision portion. In the case where a component of the MV is negative, the negative component of the MV may be rounded toward negative infinity, for example, if the negative component of the MV has a fractional precision portion.
[0083] Reference Figure 5 , the reference samples (or MCP reference samples) in the MCP reference block (e.g., MCP reference block (541)) in the reference picture (e.g., reference picture (501)) pointed to by the final MV (or integer MV) can be used to determine the MCP block (e.g., MCP block (521)) in the MC filling area (e.g., MC filling area (570)) of the picture (e.g., picture (551)). Figure 5 , in the case where the MV (561) is a final MV with integer precision, only L×4 samples are used to determine the MCP block (521). In the case where the MV (561) is an MV with fractional precision, more than L×4 samples are used to determine the MCP block (521), for example due to interpolation. Therefore, performing full pixel motion compensated padding with the final MV may be more efficient (e.g., with less memory bandwidth and less computational complexity) than performing fractional motion compensated padding with the MV. DMA may be used in full pixel motion compensated padding.
[0084] In an embodiment, reference samples in an MCP reference block (e.g., MCP reference block (541)) may be copied from a reference picture (e.g., reference picture (501)), and the reference samples in the MCP reference block (e.g., MCP reference block (541)) may be directly used as MC padding pixels. For example, MC padding pixels in an MCP block (e.g., MCP block (521)) may be directly copied from the MCP reference block.
[0085] The reference samples in the MCP reference block (e.g., MCP reference block (541)) may be calibrated, for example, with an offset (or offset value), and the calibrated reference samples (or reference samples with calibration) may be used as MC filling pixels. For example, the MC filling pixels in the MCP block (e.g., MCP block (521)) are based on the reference samples in the MCP reference block (e.g., MCP reference block (541)) and the offset.
[0086] In an example, an offset such as a DC offset is used in a calibration process. For example, the DC offset is determined based on a difference between a first DC value (e.g., an average pixel value) of a boundary block (e.g., boundary block (511)) and a second DC value (e.g., an average pixel value) of a reference block (e.g., reference block (531)) of the boundary block in a reference picture. In an example, the first DC value is an average of pixel values of pixels (or samples) in the boundary block (e.g., boundary block (511)), and the second DC value is an average of pixel values of reference pixels (or reference samples) in the reference block (e.g., reference block (531)) in the reference picture.
[0087] The reference block may be determined based on the MV or final MV (integer MV) of the MCP block. The DC offset may be applied to the reference samples in the MCP reference block. For example, the DC offset is added to the reference samples in the MCP reference block to determine the MC padding pixels in the MCP block.
[0088] In an embodiment, the offset may be signaled for an MCP block or a boundary block associated with an MCP block. For example, the offset may be signaled for each MCP block or each boundary block associated with a respective MCP block.
[0089] In an embodiment, the offset may be signaled for a group of MCP blocks or a group of border blocks associated with the group of MCP blocks. In an example, a group of border blocks includes a plurality of border blocks within a picture and adjacent to a picture boundary. A group of MCP blocks may include MCP blocks in an MCP padding region of a picture and adjacent to a picture boundary. In an example, each border block in a group of border blocks is adjacent to a corresponding MCP block in a group of MCP blocks. For example, the offset is signaled for each group of MCP blocks or each group of border blocks.
[0090] The other MCP blocks in the MCP region (570) may be determined using the full pixel motion compensated padding described for the MCP block (521). Thus, the MCP region (570) may be determined using full pixel motion compensated padding which is more efficient than fractional motion compensated padding. An extended region of a picture (551) may include the MCP region (570). A picture (and an extended region) may be used as a reference picture for encoding (e.g., reconstructing) another picture.
[0091] According to an embodiment of the present disclosure, the MV used in motion compensation padding may be determined based on multiple pieces of motion information of a group of blocks such as continuous blocks (e.g., continuous 4×4 blocks). In an embodiment, multiple MVs of continuous blocks are grouped into MVs. For example, n continuous blocks (e.g., continuous 4×4 blocks) along a picture boundary are merged into a 4n×4 block (e.g., in a row) or a 4×4n block (e.g., in a column). n may be an integer greater than 1. The n continuous blocks may include n boundary blocks within a picture that are adjacent to a picture boundary.
[0092] The MV of one of the consecutive blocks may be determined, for example, as Figure 5 As described in . In an embodiment, for MC padding, the MV associated with the 4n×4 block or 4×4n block may be determined based on multiple MVs of the corresponding continuous blocks. For example, the MV associated with the 4n×4 block or 4×4n block may be determined based on the average or weighted average of multiple MVs of the corresponding continuous blocks. For example, when n is greater than 2, the MV associated with the 4n×4 block or 4×4n block may be determined based on median filtering of multiple MVs of the corresponding continuous blocks. The MV associated with the 4n×4 block or 4×4n block may indicate an MCP reference block in a reference picture of the picture. The MCP block in the MC padding area may be determined based on the reference samples in the MCP reference block. The MC padding area may be outside the picture and adjacent to the picture boundary. The MCP block may be adjacent to the 4n×4 block or 4×4n block. In the example, the MCP block and the MCP reference block have a size of 4n×L block or L×4n. L may be a suitable integer such as 4, 16, or 64.
[0093] In an example, the MV associated with the 4n×4 block or the 4×4n block has fractional precision and is used to determine the MCP reference block, and then the fractional motion compensation padding is used to determine the MCP block. In an example, the MV associated with the 4n×4 block or the 4×4n block has fractional precision and is thus clipped or rounded to obtain an MV with integer precision. The MV associated with the 4n×4 block or the 4×4n block with integer precision may be used to determine the MCP reference block, and then the full pixel motion compensation padding is used to determine the MCP block.
[0094] The consecutive blocks (eg, consecutive 4×4 blocks) may include consecutive blocks in a column (eg, left column, right column) of a picture, such as Figure 6 The consecutive blocks may include consecutive blocks in a row (e.g., top row, bottom row) of a picture, such as Figure 7 The continuous blocks may include a corner block (or corner boundary block) of the picture and at least a neighboring block (or neighboring boundary block) of the corner block, for example Figure 8 As shown in .
[0095] In an embodiment, the DC offset value of the MC padding for each MCP block (e.g., a 4n×L block or an L×4n block) may be determined based on the difference between the average pixel value of the 4n×4 block or the 4×4n block in the picture and the average pixel value of the corresponding reference block (e.g., a 4n×4 block or a 4×4n block) in the reference picture.
[0096] The consecutive blocks or n consecutive blocks may include boundary blocks in the same column (eg, left column or right column) in the picture, such as Figure 6 As shown in . Figure 6 , the boundary blocks (611) to (612) in the picture (651) are in the left column of the picture (651). The boundary block (611) may have a first MV, and the boundary block (612) may have a second MV. The boundary blocks (611) to (612) may be merged into the block (610). The MV (661) associated with the block (610) may be determined based on an average of the first MV and the second MV, for example, a weighted average of the first MV and the second MV. The MV (661) may indicate an MCP reference block (641) and a reference block (631) in a reference picture (601) of the picture (651). The MCP block (621) in the MC padding area (670) may be determined based on a reference sample in the MCP reference block (641). The MC padding area (670) may be outside the picture (651) and adjacent to the picture boundary (691). The MCP block (621) may be adjacent to the block (610). The reference block (631) may correspond to the block (610). In an example, the size of each of the boundary blocks (611) to (612) is 4×4. Therefore, the block (610) and the reference block (631) include 4 columns and 8 rows, and the MCP block (621) and the MCP reference block (641) include L columns and 8 rows. L may be a suitable integer, such as 4, 16, or 64.
[0097] The consecutive blocks or n consecutive blocks may include boundary blocks in the same row (eg, top row or bottom row) in the picture, such as Figure 7 As shown in . Figure 7, the boundary blocks (613) to (614) in the picture (652) are in the top row of the picture (652). The boundary block (613) may have a third MV, and the boundary block (614) may have a fourth MV. The boundary blocks (613) to (614) may be merged into a block (615). The MV (662) associated with the block (615) may be determined based on an average of the third MV and the fourth MV, for example, a weighted average of the third MV and the fourth MV. The MV (662) may indicate an MCP reference block (642) and a reference block (632) in a reference picture (602) of the picture (652). The MCP block (622) in the MC padding area (672) may be determined based on a reference sample in the MCP reference block (642). The MC padding area (672) may be outside the picture (652) and adjacent to the picture boundary (692). The MCP block (622) may be adjacent to the block (615). The reference block (632) may correspond to the block (615). In the example, the size of each of the boundary blocks (613) to (614) is 4×4. Therefore, the block (615) and the reference block (632) include 8 columns and 4 rows, and the MCP block (622) and the MCP reference block (642) include 8 columns and L rows. L may be a suitable integer, such as 4, 16, or 64.
[0098] In an embodiment, the above reference Figure 7 The description may be applied to the top row and / or bottom row of a picture. In an embodiment, n consecutive blocks (e.g., n consecutive 4×4 blocks) along a portion of or the entire top row of a picture boundary are merged into a 4n×4 block (e.g., a top 4n×4 block). In an embodiment, consecutive blocks along a portion of or the entire bottom row of a picture boundary are merged into a 4n×4 block (e.g., a bottom 4n×4 block). The MV associated with the top 4n×4 block (e.g., block (615)) may point to a corresponding 4n×4 block (e.g., reference block (632)) in a reference picture (e.g., picture (602)) and a first MCP reference block (e.g., MCP reference block (642)) in the reference picture. Reference samples within the first MCP reference block are used as padding pixels for the top row along the picture boundary. The MV associated with the bottom 4n×4 block may point to a corresponding 4n×4 block and a second MCP reference block in the reference picture. Reference samples within the second MCP reference block are used as padding pixels for the bottom row along the picture boundary. In an example, along the left picture boundary and the right picture boundary, the motion information (e.g., MV) of each 4×4 boundary block in the picture can be used to obtain MC padding pixels, which are padded outside the picture boundary to the left side of the picture boundary and to the right side of the picture boundary, e.g. Figure 5 Described in .
[0099] The continuous block or n continuous blocks may include a corner block (or corner boundary block) in the picture, e.g. Figure 8 As shown in . Figure 8 , the continuous blocks (616) to (617) in the picture (653) include the corner block (616) and the adjacent boundary block (617). The continuous blocks (616) to (617) can be merged into a block (618). The MV (663) associated with the block (618) can be determined based on the MVs of the corresponding continuous blocks (616) to (617), for example, with reference to Figure 6 to Figure 7 Described. MV (663) may indicate a reference block (633) in a reference picture (603) of an MCP reference block (643) and a picture (653). An MCP block (623) in an MC padding area (673) may be determined based on a reference sample in an MCP reference block (643). The MC padding area (673) may be outside the picture (653) and adjacent to a picture boundary (693). An MCP block (623) may be adjacent to a block (618). A reference block (633) may correspond to a block (618). In an example, a size of each of consecutive blocks (616) to (617) is 4×4. Thus, block (618) and reference block (633) include 8 columns and 4 rows. Each of an MCP block (623) and an MCP reference block (643) may include a top portion of 8 columns and L rows and a left portion of L columns and 4 rows. L may be a suitable integer, such as 4, 16, or 64.
[0100] According to an embodiment of the present disclosure, filtering may be applied to one or more padding samples in an MC padding region (e.g., (570), (670), (672), or (673)). In an embodiment, boundary sample filtering is applied to padding samples in an MC padding region using spatially neighboring samples located within a picture (e.g., (551), (651), (652), or (653)).
[0101] Boundary sample filtering may be applied as a weighted average of padding samples outside the picture and adjacent reconstructed samples within the picture boundary. In an embodiment, padding samples in the MC padding region (or current padding samples to be filtered) are filtered based on the padding samples and one or more reconstructed samples within the picture (e.g., at least one sample within the N1 row of the picture boundary of the picture within the picture). For example, the filtered padding samples are a weighted average of padding samples in the MC padding region and one or more reconstructed samples within the picture.
[0102] The weight of one of the one or more reconstructed samples used in the boundary sample filtering may depend on the distance between the current filling sample and one of the one or more reconstructed samples, one of which is within the picture and is used as input for filtering the current filling sample. For example, the current filling sample to be filtered is on the left side of the picture (e.g., in column -1), and the one or more reconstructed samples used in the boundary sample filtering are in the same row as the current filling sample and are located in consecutive columns 0 to (N1-1) in the picture. N1 is greater than 0. The consecutive columns may include the left column of the picture (e.g., column 0). The weights W0 to W1 of one or more reconstructed samples in the corresponding consecutive columns 0 to (Nl-1) are N1 can decrease with the number of columns. For example, in the weights W0 to W N1 In the example, weight W0 is the largest and weight W N1 Minimum.
[0103] In an embodiment, the MC padding region includes a first padding sample and a second padding sample. The first padding sample is within the N2 rows of the picture boundary, and the second padding sample is outside the N2 rows of the picture boundary. The first padding sample can be filtered as described above. The second padding sample is not filtered.
[0104] For example, only padding samples outside the picture but closer to the picture boundary are filtered. For example, padding samples within N2 rows of the picture boundary (e.g., padding samples within N2 rows above the picture and within N2 rows below the picture) and padding samples within N2 columns of the picture boundary (e.g., padding samples within N2 columns on the left side of the picture and within N2 columns on the right side of the picture) can be filtered. N2 can be a positive integer. The value of N2 can include but is not limited to 1, 2, 3, 4, etc.
[0105] Fig. 9 A flow chart summarizing a process (900) according to an embodiment of the present disclosure is shown. The process (900) may be used in a video encoder. In various embodiments, the process (900) is performed by a processing circuit system, such as a processing circuit system that performs the functions of a video encoder (103), a processing circuit system that performs the functions of a video encoder (303). In some embodiments, the process (900) is implemented as software instructions, so that when the processing circuit system executes the software instructions, the processing circuit system performs the process (900). In an example, the process (900) implements full-pixel motion compensated padding or full-pixel MC padding. The process starts at (S901) and proceeds to (S910).
[0106] At (S910), an initial motion vector (MV) associated with a motion compensated padding (MCP) block may be determined based on motion information of a boundary block located at a picture boundary of a picture, e.g. Figure 5The boundary block may be located in the picture, and the MCP block may be located in the MCP region outside the picture and adjacent to the picture boundary of the picture. The initial MV may have fractional precision.
[0107] At (S920), a final MV associated with the MCP block may be determined based on the initial MV associated with the MCP block. The final MV may have integer precision.
[0108] In an embodiment, a final MV associated with the MCP block is determined by clipping an initial MV to integer precision. In an embodiment, the final MV associated with the MCP block is determined by rounding the initial MV toward zero. In an embodiment, the final MV associated with the MCP block is determined by rounding the initial MV toward positive infinity or negative infinity. In an embodiment, the initial MV includes an X component and a Y component, and the final MV is determined by rounding the initial MV. For each of the X component and the Y component having fractional precision, in the case where the component has a positive value, the component may be rounded toward positive infinity. In the case where the component has a negative value, the component may be rounded toward negative infinity.
[0109] At (S930), at least samples in the MCP block may be determined based on the determined final MV with integer precision. For example, the samples in the MCP block are determined using full-pixel MC padding. In an embodiment, the MCP reference block of the MCP block is determined based on the final MV associated with the MCP block. The MCP reference block is in a reference picture of the picture. The MCP block may be determined based on the MCP reference block. In an example, the samples in the MCP block may be determined by directly copying the reference samples in the MCP reference block. In an example, an offset (e.g., a DC offset value) is determined based on the difference between the average pixel value of the boundary block and the average pixel value of the reference block of the boundary block in the reference picture. The offset may be applied to at least samples in the MCP block. For example, the offset may be added to samples in the MCP block, which are determined by directly copying the reference samples in the MCP reference block to obtain the final samples in the MCP block.
[0110] In an example, one or more samples in the MCP block are filtered based on at least one sample within the picture that is within N1 rows of a picture boundary of the picture.
[0111] In an example, the above described offset and filtering are applied to at least samples in an MCP block.
[0112] Then, the process proceeds to (S999) and terminates. The process (900) may be appropriately adjusted. The steps in the process (900) may be modified and / or omitted. Additional steps may be added. Any suitable implementation order may be used. In an example, the encoding information indicating the offset is encoded. The encoded encoding information may be included in the bitstream.
[0113] Fig.10 A flow chart summarizing a process (1000) according to an embodiment of the present disclosure is shown. The process (1000) may be used in a video decoder. In various embodiments, the process (1000) is performed by a processing circuit system, such as a processing circuit system that performs the functions of a video decoder (110), a processing circuit system that performs the functions of a video decoder (210). In some embodiments, the process (1000) is implemented as software instructions, so when the processing circuit system executes the software instructions, the processing circuit system performs the process (1000). The process (1000) starts at (S1001) and proceeds to (S1010). In an example, the process (1000) implements full pixel motion compensation padding or full pixel MC padding.
[0114] At (S1010), a bitstream including a plurality of pictures is received.
[0115] At (S1020), an initial motion vector (MV) associated with a motion compensation padding (MCP) block may be determined based on motion information of a boundary block located at a picture boundary of a picture in a plurality of pictures, e.g. Figure 5 The boundary block may be located in the picture, and the MCP block may be located in the MCP region outside the picture and adjacent to the picture boundary of the picture. The initial MV may have fractional precision.
[0116] At (S1030), a final MV associated with the MCP block may be determined based on the initial MV. The final MV may have integer precision for full-pixel MC padding. In an embodiment, the final MV is determined by clipping the initial MV to integer precision. In an embodiment, the final MV is determined by rounding the initial MV toward zero. In an embodiment, the final MV is determined by rounding the initial MV toward positive infinity or negative infinity. In an embodiment, the initial MV includes an X component and a Y component, and the final MV is determined by rounding the initial MV. For each of the X component and the Y component having fractional precision, in the case where the component has a positive value, the component may be rounded toward positive infinity. In the case where the component has a negative value, the component may be rounded toward negative infinity.
[0117] At (S1040), at least samples in the MCP block may be reconstructed according to the determined final MV with integer precision. For example, full-pixel MC padding is used to reconstruct the samples in the MCP block.
[0118] In an embodiment, an MCP reference block of an MCP block is determined based on a final MV associated with the MCP block. The MCP reference block is located in a reference picture of a picture. The MCP block may be reconstructed based on the MCP reference block. In an example, a sample in an MCP block may be determined by directly copying a reference sample in an MCP reference block. In an example, an offset (e.g., a DC offset value) is determined based on a difference between an average pixel value of a boundary block and an average pixel value of a reference block of a boundary block in a reference picture. In an example, coding information is signaled and received by a decoder (e.g., a video decoder), and the coding information may indicate an offset. The offset may be applied to at least a sample in an MCP block. For example, an offset may be added to a sample in an MCP block that is determined by directly copying a reference sample in an MCP reference block to obtain a final sample in the MCP block.
[0119] In an example, one or more samples in the MCP block are filtered based on at least one sample within the picture that is within N1 rows of a picture boundary of the picture.
[0120] In an example, the above described offset and filtering are applied to at least samples in an MCP block.
[0121] Then, the process proceeds to (S1099) and terminates. Process (1000) may be adjusted appropriately. Steps in process (1000) may be modified and / or omitted. Additional steps may be added. Any suitable implementation order may be used. The MCP region may be determined, for example, by using the steps in process (1000) to determine the MCP blocks in the MCP region. The extended region of the picture may include the MCP region. The picture (and the extended region) may be used as a reference picture for encoding (e.g., reconstructing) another picture. In an example, the extended region of the picture includes a repeated fill region (e.g., (480)), such as Figure 4B Described in .
[0122] Fig.11A flow chart summarizing a process (1100) according to an embodiment of the present disclosure is shown. The process (1100) may be used in a video / image encoder, such as a video decoder. In various embodiments, the process (1100) is performed by a processing circuit system, such as a processing circuit system that performs the functions of a video decoder (110), a processing circuit system that performs the functions of a video decoder (210), etc. In some embodiments, the process (1100) is implemented as software instructions, so when the processing circuit system executes the software instructions, the processing circuit system performs the process (1100). The process (1100) starts at (S1101) and proceeds to (S1110).
[0123] At (S1110), a bitstream including a plurality of pictures is received.
[0124] At (S1120), a motion vector (MV) associated with a motion compensation padding (MCP) block may be determined based on a plurality of pieces of motion information of a corresponding plurality of boundary blocks at a picture boundary of a picture in a plurality of pictures, for example, with reference to Figures 6 to 8 As described. Multiple boundary blocks may be located in a picture. The MCP block may be located in an MCP region outside the picture and adjacent to a picture boundary of the picture. Each of the multiple boundary blocks may be a spatially adjacent block of a boundary block in the multiple boundary blocks, and the multiple boundary blocks may be spatially adjacent to the MCP block.
[0125] In an embodiment, the plurality of pieces of motion information include a plurality of MVs of the corresponding plurality of boundary blocks. In an example, the MV associated with the MCP block is determined by averaging (e.g., weighted averaging) the plurality of MVs of the corresponding plurality of boundary blocks, for example, referring to Figure 6 to Figure 7 In an example, the MV associated with the MCP block is determined by applying a median filter to the plurality of MVs, for example, when the number of the plurality of MVs is greater than two.
[0126] In an example, the picture boundaries include a top picture boundary, a bottom picture boundary, a left picture boundary, and a right picture boundary. In an example, a plurality of boundary blocks are located at the top picture boundary. In an example, a plurality of boundary blocks are located at the bottom picture boundary. In an example, a size of each of the plurality of boundary blocks is 4×4, a number of the plurality of boundary blocks is n, and a size of the MCP block is 4n×L. L may be a positive integer, such as 4, 16, or 64.
[0127] At (S1130), at least samples in the MCP block may be determined according to the determined MV. In an embodiment, an MCP reference block of the MCP block is determined according to the determined MV, and the MCP reference block is in a reference picture of the picture. The MCP block may be determined according to the MCP reference block, such as described above.
[0128] In an example, the offset is determined based on samples in a plurality of boundary blocks and reference samples in a plurality of reference blocks corresponding to the plurality of boundary blocks. In an example, the offset is determined based on a difference between an average pixel value of the plurality of boundary blocks and an average pixel value of the plurality of reference blocks. The plurality of reference blocks may be in a reference picture and may be indicated by the determined MV. The offset may be applied to at least samples in the MCP block. In an example, the offset and filtering described above are applied to at least samples in the MCP block.
[0129] In an embodiment, the determined MV associated with the MCP has fractional precision, and a final MV associated with the MCP block with integer precision is determined based on the determined MV. The MCP block may be determined according to the determined final MV with integer precision.
[0130] Then, the process proceeds to (S1199) and terminates. The process (1100) may be appropriately adjusted. The steps in the process (1100) may be modified and / or omitted. Additional steps may be added. Any suitable implementation order may be used. The process (1100) may be adapted for use in an encoder (e.g., a video encoder or an image encoder). For example, steps (S1120) and (S1130) may be used in an encoder.
[0131] Fig.12 A flow chart summarizing a process (1200) according to an embodiment of the present disclosure is shown. The process (1200) may be used in a video / image encoder, such as a video decoder. In various embodiments, the process (1200) is performed by a processing circuit system, such as a processing circuit system that performs the functions of a video decoder (110), a processing circuit system that performs the functions of a video decoder (210), etc. In some embodiments, the process (1200) is implemented as software instructions, so when the processing circuit system executes the software instructions, the processing circuit system performs the process (1200). The process (1200) starts at (S1201) and proceeds to (S1210).
[0132] At (S1210), a bitstream including a plurality of pictures is received.
[0133] At (S1220), a motion vector (MV) associated with the MCP block may be determined according to motion information of a boundary block located at a picture boundary of a picture among the plurality of pictures. The boundary block is located in the picture, and the MCP block is located in an MCP region outside the picture and adjacent to the picture boundary of the picture.
[0134] At (S1230), samples in the MCP block may be determined according to the determined MV using motion compensation padding. In the case where the determined MV has fractional precision, the motion compensation padding may be fractional motion compensation padding. In the case where the determined MV (i) has integer precision or (ii) is clipped or rounded to integer precision, the motion compensation padding may be full-pixel motion compensation padding. In an example, as described above, the offset is also applied to the samples in the MCP block.
[0135] At (S1240), samples in the MCP block may be filtered based on at least one sample within the picture that is within N1 rows of a picture boundary of the picture. In an example, the samples in the MCP block are filtered based on a weighted average of the sample and the at least one sample within the picture. The weight of the sample in the at least one sample within the picture is based on a distance between (i) the sample in the MCP block and (ii) the sample in the at least one sample within the picture. In an example, the samples in the MCP block are within N2 rows of a picture boundary, and samples in the MCP block that are outside the N2 rows of the picture boundary are not filtered.
[0136] Then, the process proceeds to (S1299) and terminates. The process (1200) may be appropriately adjusted. The steps in the process (1200) may be modified and / or omitted. Additional steps may be added. Any suitable implementation order may be used. The process (1200) may be adapted for use in an encoder (e.g., a video encoder or an image encoder). For example, steps (S1220), (S1230), and (S1240) may be used in an encoder.
[0137] The embodiments in the present disclosure may be used alone or in combination in any order. In addition, each of the method (or embodiment), encoder, and decoder may be implemented by a processing circuit system (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium.
[0138] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. Fig.13 A computer system (1300) suitable for implementing certain embodiments of the disclosed subject matter is shown.
[0139] Computer software may be encoded using any suitable machine code or computer language, which may be subjected to mechanisms such as assembly, compilation, linking, etc. to create code comprising instructions that may be executed directly by one or more computer central processing units (CPU), graphics processing units (GPU), etc., or by interpretation, microcode execution, etc.
[0140] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.
[0141] Fig.13 The components for the computer system (1300) shown in the example are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Nor should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components shown in the exemplary embodiment of the computer system (1300).
[0142] The computer system (1300) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input by one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, tapping), visual input (e.g., gestures), olfactory input (not depicted). The human-machine interface devices may also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).
[0143] Input human-machine interface devices may include one or more of the following (only one of each is depicted): keyboard (1301), mouse (1302), touchpad (1303), touch screen (1310), data gloves (not shown), joystick (1305), microphone (1306), scanner (1307), camera (1308).
[0144] The computer system (1300) may also include certain human interface output devices. Such human interface output devices may stimulate one or more senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: tactile output devices (e.g., tactile feedback via a touch screen (1310), a data glove (not shown), or a joystick (1305), but there may also be tactile feedback devices that do not function as input devices); audio output devices (e.g., speakers (1309), headphones (not depicted)); visual output devices (e.g., screens (1310), including CRT (Cathode Ray Tube, CRT) screens, LCD (Liquid Crystal Display, LCD) screens, plasma screens, OLED (Organic Light Emitting Diode, OLED) screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which may be capable of outputting two-dimensional visual output or more than three-dimensional output by means such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and cigarette cans (not depicted)); and printers (not depicted).
[0145] The computer system (1300) may also include storage devices and their associated media accessible to humans, such as optical media including CD / DVD ROM (Read Only Memory, ROM) / RW (1320) with CD / DVD and other media (1321), thumb drives (1322), removable hard drives or solid-state drives (1323), legacy magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD-based devices such as security dongles (not depicted), etc.
[0146] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.
[0147] The computer system (1300) may also include an interface (1354) to one or more communication networks (1355). The network may be, for example, wireless, wired, optical. The network may also be local, wide, metropolitan, vehicle and industrial, real-time, delay-tolerant, etc. Examples of networks include: local area networks such as Ethernet; wireless LAN (Local Area Network, LAN); cellular networks including GSM (Global System for Mobile Communications, GSM), 3G (the Third Generation, 3G), 4G (the Fourth Generation, 4G), 5G (the Fifth Generation, 5G), LTE (Long Term Evolution, LTE), etc.; TV wired or wireless wide area digital networks including cable TV, satellite TV and terrestrial broadcast TV; vehicles and industries including CANBus, etc. Some networks typically require an external network interface adapter attached to some general data port or peripheral bus (1349) (such as, for example, a USB port of the computer system (1300)); other networks are typically integrated into the core of the computer system (1300) by attaching to a system bus as described below (e.g., to an Ethernet interface in a PC computer system or to a cellular network interface in a smartphone computer system). Using any of these networks, the computer system (1300) can communicate with other entities. Such communications can be one-way receive only (e.g., broadcast TV), one-way send only (e.g., a CAN bus to some CAN bus devices), or two-way, for example, to other computer systems using a local area digital network or a wide area digital network. Certain protocols and protocol stacks can be used on each of these networks and network interfaces as described above.
[0148] The above-mentioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core ( 1340 ) of the computer system ( 1300 ).
[0149] The core (1340) may include one or more central processing units (CPUs) (1341), graphics processing units (GPUs) (1342), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (1343), hardware accelerators for certain tasks (1344), graphics adapters (1350), etc. These devices, along with read-only memory (ROM) (1345), random access memory (1346), internal mass storage devices (1347) such as internal non-user accessible hard drives, SSDs (Solid State Drives, SSDs), etc., may be connected via a system bus (1348). In some computer systems, the system bus (1348) may be accessible in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, etc. Peripheral devices may be attached to the core's system bus (1348) directly or via a peripheral bus (1349). In an example, a screen (1310) may be connected to a graphics adapter (1350). The architecture of the peripheral bus includes PCI (Peripheral Component Interconnect / Interface, PCI), USB (Universal Serial Bus, USB), etc.
[0150] The CPU (1341), GPU (1342), FPGA (1343) and accelerator (1344) can execute certain instructions, which can be combined to form the computer code mentioned above. The computer code can be stored in ROM (1345) or RAM (Random Access Memory, RAM) (1346). Transient data can also be stored in RAM (1346), while permanent data can be stored in, for example, an internal mass storage device (1347). Fast storage and retrieval of any storage device in the storage device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1341), GPUs (1342), mass storage devices (1347), ROMs (1345), RAMs (1346), etc.
[0151] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or the medium and computer code may be of a type well known and available to those skilled in the art of computer software.
[0152] As an example and not limitation, a computer system (1300) having an architecture and in particular a core (1340) can provide functionality due to a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software contained in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage device as described above, as well as certain storage devices of the core (1340) having non-transitory properties, for example, a core internal mass storage device (1347) or a ROM (1345). Software implementing various embodiments of the present disclosure can be stored in such a device and executed by the core (1340). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can enable the core (1340) and in particular the processor therein (including a CPU, GPU, FPGA, etc.) to perform specific processing or specific parts of specific processing described herein, including defining data structures stored in RAM (1346) and modifying such data structures according to processing defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of hard-wired logic or otherwise embodied in circuitry (e.g., accelerator (1344)) that may replace or operate in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and references to logic may also include software. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry implementing logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.
[0153] The use of "at least one of" or "one of" in the present disclosure is intended to include any one or combination of the elements. For example, references to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A through C are intended to include only A, only B, only C, or any combination thereof. References to one of A or B and one of A and B are intended to include A or B or (A and B). Where applicable, such as when the elements are not mutually exclusive, the use of "one of" does not exclude any combination of the elements.
[0154] Although the present disclosure has described several exemplary embodiments, there are changes, permutations, and various alternative equivalents that fall within the scope of the present disclosure. It will therefore be appreciated that those skilled in the art will be able to conceive of many systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.
Claims
1. A method for video decoding in a video encoder, the method comprising: receiving a bitstream comprising a plurality of pictures; determining an initial motion vector (MV) with fractional precision associated with a motion compensated padding (MCP) block, wherein the initial MV is determined based on motion information of a boundary block located at a picture boundary of a picture of the plurality of pictures, the boundary block being located in the picture, the MCP block being located in an MCP region outside the picture and adjacent to the picture boundary of the picture; generating a final MV having integer precision based on the initial MV associated with the MCP block, wherein the final MV is associated with the MCP block; and At least samples in the MCP block are reconstructed according to the generated final MV having the integer precision.
2. The method according to claim 1, wherein: Generating the final MV includes: The final MV associated with the MCP block is generated by clipping the initial MV associated with the MCP block to the integer precision.
3. The method according to claim 1, wherein: Generating the final MV includes: The final MV associated with the MCP block is generated by rounding the initial MV associated with the MCP block towards zero.
4. The method according to claim 1, wherein: Generating the final MV includes: The final MV associated with the MCP block is generated by rounding the initial MV associated with the MCP block toward positive infinity or negative infinity.
5. The method according to claim 1, wherein The initial MV associated with the MCP block includes an X component and a Y component, For each of the X component and the Y component having the fractional precision, In response to a component being a positive value, generating the final MV includes rounding the component toward positive infinity; and In response to the component being a negative value, generating the final MV includes rounding the component toward negative infinity.
6. The method according to claim 1, wherein: The reconstruction includes: Determine an MCP reference block for the MCP block according to the final MV, the MCP reference block being in a reference picture of the picture; and The MCP block is reconstructed based on the MCP reference block.
7. The method according to claim 6, wherein: Reconstructing the MCP block according to the MCP reference block comprises: determining an offset based on a difference between an average pixel value of the boundary block and an average pixel value of a reference block of the boundary block in the reference picture; and The offset is applied to the at least samples in the MCP block.
8. The method according to claim 6, wherein The method includes receiving encoded information indicating an offset; and Reconstructing the MCP block from the MCP reference block comprises applying the offset to the at least samples in the MCP block.
9. The method according to claim 6, wherein: Reconstructing the MCP block according to the MCP reference block comprises: Samples in the MCP block are filtered based on at least one sample within the picture within N1 rows of the picture boundary of the picture.
10. A method for video decoding in a video encoder, comprising: receiving a bitstream comprising a plurality of pictures; determining a motion vector (MV) associated with a motion compensated padding (MCP) block according to a plurality of pieces of motion information of a corresponding plurality of boundary blocks located at a picture boundary of a picture of the plurality of pictures, the plurality of boundary blocks being located in the picture, the MCP block being located in an MCP region outside the picture and adjacent to the picture boundary of the picture, each of the plurality of boundary blocks being a spatially adjacent block of a boundary block of the plurality of boundary blocks, the plurality of boundary blocks being adjacent to the MCP block; At least samples in the MCP block are determined based on the determined MV.
11. The method according to claim 10, wherein The plurality of pieces of motion information include a plurality of MVs of the corresponding plurality of boundary blocks; and Determining the MV includes determining the MV associated with the MCP block by averaging the plurality of MVs of the corresponding plurality of boundary blocks.
12. The method according to claim 10, wherein The plurality of pieces of motion information include a plurality of MVs of the corresponding plurality of boundary blocks; and Determining the MV includes determining the MV associated with the MCP block by applying a median filter to the plurality of MVs of the corresponding plurality of boundary blocks.
13. The method according to claim 10, wherein The picture borders include a top picture border, a bottom picture border, a left picture border and a right picture border, Each of the plurality of boundary blocks has a size of 4×4, The number of the plurality of boundary blocks is n, The size of the MCP block is 4n×L, L is a positive integer, and The plurality of boundary blocks are located at the top picture boundary or the bottom picture boundary.
14. The method according to claim 10, wherein: Determining the at least sample in the MCP block comprises: Determine an MCP reference block of the MCP block according to the determined MV, wherein the MCP reference block is located in a reference picture of the picture; and The MCP block is determined based on the MCP reference block.
15. The method according to claim 14, wherein: Determining the MCP block includes: determining an offset based on a difference between an average pixel value of the plurality of boundary blocks and an average pixel value of a plurality of reference blocks corresponding to the plurality of boundary blocks, the plurality of reference blocks being in the reference picture; and The offset is applied to the at least samples in the MCP block.
16. The method according to claim 10, wherein: Determining the at least sample in the MCP block comprises: In response to determining that the MV associated with the MCP has fractional accuracy, determining a final MV associated with the MCP block based on the determined MV, the final MV having integer precision; and The MCP block is determined according to the determined final MV having the integer precision.
17. A method for video decoding in a video encoder, comprising: receiving a bitstream comprising a plurality of pictures; determining a motion vector (MV) associated with a motion compensated padding (MCP) block according to motion information of a boundary block located at a picture boundary of a picture of the plurality of pictures, the boundary block being located in the picture, the MCP block being located in an MCP region outside the picture and adjacent to the picture boundary of the picture; Determining samples in the MCP block according to the determined MV using motion compensated padding; as well as Samples in the MCP block are filtered based on at least one sample within the picture within N1 rows of the picture boundary of the picture.
18. The method according to claim 17, wherein: The filtering includes: The samples in the MCP block are filtered based on a weighted average of the samples and the at least one sample within the picture.
19. The method according to claim 18, wherein A weight of a sample of the at least one sample within the picture is based on a distance between (i) the sample in the MCP block and (ii) a sample of the at least one sample within the picture.
20. The method of claim 17, wherein The samples in the MCP block are within N2 rows of the picture boundary, and Samples in the MCP block that are outside of N2 rows of the picture boundary are not filtered.
