Method for Decoding Video Data, Computing System, and Program

Abstract

The various embodiments described in this application include a video coding method and system. In one aspect, the method includes receiving a current frame including a current coding block. The current coding block has a plurality of sub-blocks. The plurality of sub-blocks are associated with different motion vectors and include a first sub-block located at the boundary of the current coding block. The method further includes determining a motion vector of the current coding block, determining a first motion vector of the first sub-block, and identifying a coding prediction block based on the motion vector of the current coding block and a first prediction block based on the first motion vector of the first sub-block, and generating motion compensation data for the first sub-block by combining the coding prediction block and the first prediction block to generate motion compensation data for the first sub-block based on the motion vector of the current coding block and the first motion vector of the first sub-block.

Description

Technical Field

[0001] The disclosed embodiments generally relate to video coding including, but not limited to, systems and methods that combine sub-block motion compensation and overlap block motion compensation.

Background Art

[0002] Digital video is supported by various electronic devices such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconferencing devices, video streaming devices, and the like. The electronic devices transmit and receive or otherwise communicate digital video data via a communication network and / or store the digital video data in a storage device. Due to the limited bandwidth capacity of the communication network and the limited memory resources of the storage device, video coding may be used to compress video data according to one or more video coding standards before the video data is communicated or stored.

[0003] Multiple video codec standards have been developed. For example, video coding standards include AOMedia Video 1 (AV1), Versatile Video Coding (VVC), Joint Exploration test Model (JEM), High-Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.265), and Moving Picture Expert Group (MPEG) coding. Video coding generally utilizes prediction methods (e.g., inter prediction, intra prediction, etc.) that exploit the redundancy inherent in video data. Video coding aims to compress video data into a form that uses a lower bitrate while avoiding or minimizing degradation of video quality.

[0004] High Efficiency Video Coding (HEVC), also known as H.265, is a video compression standard designed as part of the MPEG-H project. ITU-T and ISO / IEC published HEVC / H.265 in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). Versatile Video Coding (VVC), also known as H.266, is a video compression standard intended as a successor to HEVC. ITU-T and ISO / IEC published the VVC / H.266 standard in 2020 (version 1) and 2022 (version 2). AV1 is an open video coding format designed as an alternative to HEVC. On January 8, 2019, an effective version 1.0.0 including errata sheet 1 of the specification was released.

Summary of the Invention

[0005] As described above, encoding (compression) reduces the bandwidth and / or storage space requirements. As will be described in detail later, both reversible compression and irreversible compression can be used. Reversible compression refers to a technique in which an exact copy of the original signal can be reconstructed from the compressed original signal by a decoding process. Irreversible compression refers to a coding / decoding process in which the original video information is not sufficiently preserved during coding and is not sufficiently recovered during decoding. When using irreversible compression, the reconstructed signal may not be the same as the original signal, but the distortion between the original signal and the reconstructed signal is small enough to make the reconstructed signal useful for its intended purpose. The amount of acceptable distortion depends on the application. For example, a user of a particular consumer video streaming application may tolerate higher distortion than a user of a movie or television broadcast application. The compression ratio available with a particular coding algorithm generally enables a coding algorithm that results in higher loss and higher compression ratio.

[0006] The present disclosure describes applying overlapped block motion compensation (OBMC) at sub-block boundaries when coding blocks with different motion vectors at the sub-block level. Overlapped block motion compensation is applied, for example, in AV1, at block boundaries of blocks coded by translational motion modes based on block-level motion vectors. Further, in some embodiments, overlapped block motion compensation is applied when there is discontinuity in motion information at block boundaries and / or sub-block boundaries, thereby improving the prediction accuracy levels of blocks and sub-blocks in video coding. Overlapped block motion compensation is applied to refine sub-block-based motion compensation including, but not limited to, local and global warp motion, bidirectional optical flow, and temporal interpolation prediction modes.

[0007] According to some embodiments, a method of video decoding is provided. The method includes receiving a current frame including a current coding block. The current coding block has a plurality of sub-blocks. The plurality of sub-blocks are associated with different motion vectors and include a first sub-block located at the boundary of the current coding block. The current coding block has a plurality of sub-blocks including the first sub-block. The method further includes determining a motion vector of the current coding block, determining a first motion vector of the first sub-block, and generating motion compensation data for the first sub-block based on the motion vector of the current coding block and the first motion vector of the first sub-block. In some embodiments, the method further includes identifying one or more adjacent sub-blocks. The method further includes determining an adjacent motion vector for each of the one or more adjacent sub-blocks. The motion compensation data for the first sub-block is generated based on the motion vector of the current coding block, the first motion vector of the first sub-block, and one or more adjacent motion vectors of the one or more adjacent sub-blocks. In some embodiments, each adjacent sub-block includes a proximity sub-block directly adjacent to the first sub-block.

[0008] In some embodiments, the step of generating motion compensation data for the first sub-block further includes identifying a coding prediction block based on the motion vector of the current coding block, identifying a first prediction block based on the first motion vector of the first sub-block, and combining the coding prediction block and the first prediction block to generate the motion compensation data for the first sub-block. Further, in some embodiments, the method further includes determining a first weight associated with the motion vector of the current coding block and determining a second weight associated with the first motion vector of the first sub-block. The step of combining the coding prediction block and the first prediction block further includes weighted-averaging the coding prediction block and the first prediction block based on the first weight and the second weight to generate the motion compensation data for the first sub-block.

[0009] According to some embodiments, a computing system such as a streaming system, a server system, a personal computer system, or other electronic device is provided. The computing system includes a control circuit and a memory storing one or more sets of instructions. The one or more sets of instructions include instructions for performing any of the methods described herein. In some embodiments, the computing system includes an encoder component and / or a decoder component.

[0010] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more sets of instructions executable by a computer system. The one or more sets of instructions include instructions for performing any of the methods described herein.

[0011] Thus, devices and systems are disclosed along with a method of coding video. Such methods, devices, and systems may complement or replace conventional methods, devices, and systems for video coding.

[0012] The features and advantages described in the specification are not necessarily all inclusive, and in particular, some additional features and advantages will be apparent to those skilled in the art in view of the drawings, specification, and claims provided in this disclosure. Further, note that the language used herein has been selected primarily for readability and instructional purposes and not necessarily for the purpose of describing or limiting the subject matter described herein.

[0013] For a more detailed understanding of the present disclosure, a more specific description may be made with reference to the features of various embodiments. Some embodiments are represented in the accompanying drawings. It should be noted that the accompanying drawings merely represent the relevant features of the present disclosure and should not necessarily be regarded as limiting, and other effective features can be recognized for the purpose of explanation as would be understood by those skilled in the art when reading the present disclosure.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0015] According to general convention, the various features shown in the drawings are not necessarily drawn to a fixed scale, and the same reference numerals can be used throughout the specification and drawings to indicate the same features.

[0016] The present disclosure describes applying overlap block motion compensation (OBMC) at sub-block boundaries when coding blocks with different motion vectors at the sub-block level. Overlap block motion compensation is applied, for example, in AV1, at the block boundaries of blocks coded in a translational motion mode based on block-level motion vectors. In some embodiments, overlap block motion compensation improves the prediction accuracy level when there are discontinuities in motion information at block boundaries and / or sub-block boundaries. Overlap block motion compensation is applied to refine sub-block-based motion compensation including, but not limited to, local and global warp motion, bidirectional optical flow, and temporal interpolation prediction modes.

[0017] [Exemplary Systems and Devices] FIG. 1 is a block diagram depicting a communication system 100 according to some embodiments. Communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to electronic devices 120-m) communicatively coupled to each other via one or more networks. In some embodiments, communication system 100 is a streaming system used by video-enabled applications such as, for example, video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0018] The source device 102 includes a video source 104 (e.g., a camera component or a media storage) and an encoder component 106. In some embodiments, the video source 104 is a digital camera (e.g., configured to generate an uncompressed video sample stream). The encoder component 106 generates one or more encoded video bitstreams from the video stream. The video stream from the video source 104 may have a high data volume compared to the encoded video bitstream 108 generated by the encoder component 106. Since the encoded video bitstream 108 has a low data volume (less data) compared to the video stream from the video source, the encoded video bitstream 108 requires less bandwidth for transmission and less storage space for storage compared to the video stream from the video source 104. In some embodiments, the source device 102 does not include the encoder component 106 (e.g., configured to transmit uncompressed video data over a network).

[0019] One or more networks 110 represent any number of networks that carry information between the source device 102, the server system 112, and / or the electronic device 120, including, for example, wireline (wired) and / or wireless communication networks. One or more networks 110 may exchange data over circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and / or the Internet.

[0020] One or more networks 110 include a server system (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is or includes a streaming server (e.g., configured to store and / or distribute video content such as an encoded video stream from the source device 102). The server system 112 includes an encoder component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the encoder component 114 includes an encoder component and / or a decoder component. In various embodiments, the encoder component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the encoder component 114 is configured to re-encode video data using different encoding standards and / or method logics to decode the encoded video bitstream 108 and generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings from the encoded video bitstream 108.

[0021] In some embodiments, the server system 112 functions as a Media-Aware Network Element (MANE). For example, the server system 112 may be configured to prune the encoded video bitstream 108 to match potentially different bitstreams to one or more electronic devices 120. In some embodiments, the MANE is provided separately from the server system 112.

[0022] The electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode the encoded video data 116 to generate an output video stream that can be rendered on a display or other type of rendering device. In some embodiments, one or more electronic devices 120 do not include a display component (e.g., are communicatively coupled to an external display device and / or include a media stream). In some embodiments, the electronic device 120 is a streaming client. In some embodiments, the electronic device 120 is configured to access the server system 112 to obtain the encoded video data 116.

[0023] The source device 102 and / or the plurality of electronic devices 120 are sometimes referred to as "terminal devices" or "user devices". In some embodiments, the source device 102 and / or one or more electronic devices 120 are instances of a server system, a personal computer, a portable device (e.g., a smartphone, a tablet, or a laptop), a wearable device, a video conferencing device, and / or other types of electronic devices.

[0024] In an example of the operation of communication system 100, source device 102 sends an encoded video bitstream 108 to server system 112. For example, source device 102 may code a stream of pictures captured by the source device. Server system 112 receives encoded video bitstream 108 and may decode and / or encode encoded video bitstream 108 by coder component 114. For example, server system 112 may apply encoding to video data that is more optimal for network transmission and / or storage. Server system 112 may send encoded video data 116 (e.g., one or more coded video bitstreams) to one or more electronic devices 120. Each electronic device 120 may decode encoded video data 116 to recover video pictures and optionally display them.

[0025] In some embodiments, the above transmission is unidirectional data transmission. Unidirectional data transmission is sometimes used in media serving applications and the like. In some embodiments, the above transmission is bidirectional data transmission. Bidirectional data transmission is sometimes used in video conferencing applications and the like. In some embodiments, encoded video bitstream 108 and / or encoded video data 116 are encoded and / or decoded according to any of the video coding / compression standards described herein, such as HEVC, VVC, and / or AV1.

[0026] FIG. 2A is a block diagram representing exemplary elements of an encoder component 106 according to some embodiments. The encoder component 106 receives a source video sequence from a video source 104. In some embodiments, the encoder component 106 includes a receiver (e.g., transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives a video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder component 106). The video source 104 can supply the source video sequence in the form of a digital video sample stream that can be of any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 YCrCb, or RGB), and any suitable sampling structure (e.g., YCrCb 4:2:0 or YCrCb 4:4:4). In some embodiments, the video source 104 is a storage device storing previously captured / prepared video. In some embodiments, the video source 104 is a camera that captures local image information as a video sequence. The video data may be supplied as a plurality of individual pictures that impart motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, and each pixel can include one or more samples depending on the sampling structure, color space, etc. used. One skilled in the art can easily understand the relationship between pixels and samples. The following description focuses on samples.

[0027] The encoder component 106 is configured to encode and / or compress pictures of a source video sequence into a coded video sequence 216 in real time or under other timing constraints required by an application. Enforcing an appropriate coding speed is a function of the controller 204. In some embodiments, the controller 204 controls and is functionally coupled to other functional units described hereinafter. Parameters set by the controller 204 may include rate control related parameters (e.g., picture skip, quantizer, and / or lambda value of rate distortion optimization techniques), picture size, group of pictures (GOP) layout, maximum motion vector search range, and the like. Those skilled in the art can readily identify other functions of the controller 204 in that they may relate to the encoder component 106 that is optimized for a particular system design.

[0028] In some embodiments, encoder component 106 is configured to operate in a coding loop. In a simplified example, the coding loop involves a source coder 202 (e.g., generating symbols such as a symbol stream based on an input picture and reference pictures to be coded), and a (local) decoder 210. The decoder 210 reconstructs symbols to generate sample data, similar to a (remote) decoder (when the compression between the symbols and the coded video bitstream is reversible). The reconstructed sample stream (sample data) is an input to the reference picture memory 208. Since the decoding of the symbol stream results in a bit-exact result regardless of the location of the decoder (local or remote), the content in the reference picture memory 208 is also bit-exact between a local encoder and a remote encoder. Thus, the prediction part of the encoder interprets the same sample values as reference picture samples as would be interpreted by the decoder when using the prediction during decoding. This basic principle of the synchronization of reference pictures (and the resulting drift if the synchronization cannot be maintained, e.g., due to channel errors) is known to those skilled in the art.

[0029] The operation of decoder 210 can be the same as that of a remote decoder, such as decoder component 122, which is described in detail below in relation to FIG. 2B. Briefly referring to FIG. 2B, however, since symbols are available and symbol encoding / decoding for the coded video sequence by entropy encoder 214 and parser 254 can be reversible, the entropy decoding part of decoder component 122, including buffer 252 and parser 254, may not be fully implemented in local decoder 210.

[0030] The observations obtained at this point are that any decoder technology, except for the parsing / entropy decoding present in the decoder, must necessarily exist in substantially the same functional form in the corresponding encoder. Therefore, the disclosed subject matter focuses on the operation of the decoder. The description of encoder technology can be omitted as it is the reverse of the decoder technology described comprehensively. Only in a specific range is more detailed description required and is provided below.

[0031] As part of its operation, source coder 202 may perform motion-compensated predictive coding. This predictively codes the input frame by referring to one or more coded frames from the video sequence designated as the reference frame. In this way, coding engine 212 codes the difference between a pixel block of the reference frame that can be selected as a prediction reference for the input frame and a pixel block of the input frame. Controller 204 can manage the coding operation of source coder 202, including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0032] Decoder 210 decodes the coded video data of the frame that can be designated as the reference frame based on the symbols generated by source coder 202. The operation of coding engine 212 may advantageously be an irreversible process. When the coded video data is decoded by a video decoder (not shown in FIG. 2A), the reconstructed video sequence is a reproduction of the source video sequence with some errors. Decoder 210 may reproduce the decoding process that can be performed by a video decoder remote from the reference frame and cause the reconstructed reference frame to be stored in reference picture memory 208. In this way, encoder component 106 can locally store a copy of the reconstructed reference frame that has the same content as the reconstructed reference frame that would be obtained by a remote video decoder (without transmission errors).

[0033] The predictor 206 may perform a prediction search for the coding engine 212. That is, in order for a new picture to be coded, the predictor 206 may look for specific metadata such as reference picture motion vectors, block shapes, etc. or sample data (as candidate reference pixel blocks) in the reference picture memory 208 that can serve as appropriate prediction criteria for that new picture. The predictor 206 may operate on a sample block-by-pixel block basis to find appropriate prediction criteria. In some cases, the input picture may have prediction criteria derived from a plurality of reference pictures stored in the reference picture memory 208, as determined by the search results obtained by the predictor 206.

[0034] The outputs of all the above functional units may undergo entropy coding in the entropy encoder 214. The entropy encoder 214 converts the symbols generated by the various functional units into a coded video sequence by reversibly compressing the symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable length coding, and / or arithmetic coding).

[0035] In some embodiments, the output of the entropy encoder 214 is coupled to a transmitter. The transmitter may be configured to buffer the coded video sequences generated by the entropy encoder 214 and prepare them for transmission over a communication channel 218, which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter may be configured to merge the coded video data from the source coder 202 with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown). In some embodiments, the transmitter may transmit additional data along with the encoded video. The source coder 202 may include such data as part of the coded video sequence. The additional data may include, for example, temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, and the like.

[0036] Controller 204 may manage the operation of video component 106. During coding, controller 204 may assign to each coded picture a particular coding picture type that may affect the coding technique applied to each picture. For example, a picture may be assigned as an Intra Picture (I picture), a Predictive Picture (P picture), or a Bi-directionally Predictive Picture (B picture). An Intra Picture may be encoded and decoded without using any other picture in the sequence as a source of prediction. Some video codecs allow various types of Intra Pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will be aware of such variations of I pictures and their respective applications and characteristics, and thus they are not repeated here. A Predictive Picture may be encoded and decoded by intra prediction or inter prediction using at most one motion vector and a reference index to predict the sample values of each block. A Bi-directionally Predictive Picture may be encoded and decoded using intra prediction or inter prediction that uses at most two motion vectors and reference indices to predict the sample values of each block. Similarly, multiple-predictive picture(s) may be able to use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0037] The source picture may be spatially subdivided into a plurality of sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples respectively), and coded block by block. The blocks may be predicted coded by referring to other (already coded) blocks determined by the coding assignment applied to each picture of the block. For example, blocks of an I picture may be non-predictively coded, or they may be predictively coded by referring to already coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P picture may be coded non-predictively, or by spatial prediction or temporal prediction by referring to a previously coded reference picture. Blocks of a B picture may be coded non-predictively, or by spatial prediction or temporal prediction by referring to one or two previously coded reference pictures.

[0038] Video may be captured as a plurality of source pictures (video pictures) in a time sequence. Intra picture prediction (often abbreviated as intra prediction) utilizes the spatial correlation within a given picture, and inter picture prediction utilizes the (temporal or other) correlation between pictures. In one example, a particular picture being encoded / decoded, called the current picture, is partitioned into blocks. When a block within the current picture resembles a reference block within a reference picture that has been previously coded and is still buffered within the video, that block within the current picture may be coded by a vector called a motion vector. The motion vector indicates the reference block within the reference picture and may have a third dimension that specifies the reference picture when multiple reference pictures are being used.

[0039] The encoder component 106 may perform a coding operation according to a predetermined video coding technique or standard, such as any of those described herein. In that operation, the encoder component 106 may perform various compression operations, including a predictive coding operation that exploits temporal and spatial redundancies within the input video sequence. Accordingly, the coded video data may conform to a syntax specified by the video coding technique or standard being used.

[0040] FIG. 2B is a block diagram representing exemplary elements of a decoder component 122 according to some embodiments. The decoder component 122 of FIG. 2B is coupled to a channel 218 and a display 124. In some embodiments, the decoder component 122 includes a transmitter configured to be coupled to a loop filter unit 256 and transmit data to the display 124 (e.g., via a wired or wireless connection).

[0041] In some embodiments, decoder component 122 includes a receiver configured to be coupled to a channel and receive data from the channel (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more coded video sequences to be decoded by decoder component 122. In some embodiments, the decoding of each coded video sequence is independent of other coded video sequences. Each coded video sequence may be received from channel 218, which may be a hardware / software link to a storage device storing the encoded video data. The receiver may receive the encoded video data together with other data, such as coded audio data and / or auxiliary data streams, which may be transferred to their respective usage entities (not shown). The receiver may separate the coded video sequences from the other data. In some embodiments, the receiver receives additional (redundant) data together with the encoded video. The additional data may be used by decoder component 122 to decode the data and / or to more accurately reconstruct the original video data. The additional data may take forms such as, for example, temporal / spatial / SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, and the like.

[0042] According to some embodiments, decoder component 122 includes buffer memory 252, parser 254 (which may also be referred to as an entropy decoder), scaler / inverse transform unit 258, intra-picture prediction unit 262, motion compensation prediction unit 260, aggregator 268, loop filter unit 256, reference picture memory 266, and current picture memory 264. In some embodiments, decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuits. In some embodiments, decoder component 122 is implemented at least partially in software.

[0043] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., to counter network jitter). In some embodiments, buffer memory 252 is separated from decoder component 122. In some embodiments, the separated buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, the separated buffer memory is provided outside of decoder component 122 (e.g., to counter network jitter) in addition to buffer memory 252 within decoder component 122 (e.g., configured to handle playback timing). Buffer memory 252 may not be required, or may be small, when receiving data from a sufficient bandwidth and controllable memory / transfer device or from an isosynchronous network. For use in a best effort packet network such as the Internet, buffer memory 252 may be required, may be relatively large, and advantageously may be of an adaptable size and may be implemented, at least in part, in an operating system or similar element (not shown) outside of decoder component 122.

[0044] Parser 254 is configured to reconstruct symbol 270 from the coded video sequence. The symbol may include, for example, information used to manage the operation of decoder component 122 and / or information for controlling a rendering device such as display 124. The control information for the rendering device may be in the form of an SEI message or a VUI parameter set fragment (not shown). Parser 254 parses (entropy decodes) the coded video sequence. The coding of the coded video sequence may conform to a video coding technology or standard, and may follow various principles including variable length coding, Huffman coding, context-dependent or independent arithmetic coding, etc. Parser 254 may extract a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to that group. The subgroups may include, for example, group of pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. Parser 254 may also extract information such as transform coefficients, quantization parameter values, motion vectors, etc. from the coded video sequence.

[0045] The reconstruction of symbol 270 can have a number of different units depending on the type of the coded video picture or a portion thereof (e.g., inter and intra pictures, inter and intra blocks) and other factors. How the units are included can be controlled by subgroup control information parsed by parser 254 from the coded video sequence. The flow of such subgroup control information between parser 254 and the following multiple units is not shown for clarity.

[0046] Beyond the function blocks already described, the decoder component 122 can conceptually be subdivided into a number of functional units described below. In an actual implementation operating under commercial constraints, many of these units interact closely with each other and can be at least partially incorporated into each other. However, for the purpose of explaining the disclosed subject matter, the conceptual subdivision into functional units below is supported.

[0047] The scaler / inverse transform unit 258 receives from the parser 254, as symbols 270, the quantized transform coefficients together with control information (e.g., which transform to use, block size, quantization coefficients, and / or quantization scaling matrix, etc.). The scaler / inverse transform unit 258 can output a block containing sample values that can be input to the aggregator 268.

[0048] In some cases, the output samples of the scaler / inverse transform unit 258 are related to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information can be supplied by the intra-picture prediction unit 262. The intra-picture prediction unit 262 can generate a block of the same size and shape as the block being reconstructed, using the surrounding already-reconstructed information fetched from the current (partially reconstructed) picture in the current picture memory 264. The aggregator 268 can add, for each sample, the prediction information generated by the intra-picture prediction unit 262 to the output sample information supplied by the scaler / inverse transform unit 258.

[0049] In other cases, the output samples of the scaler / inverse transform unit 258 are related to inter-coded and potentially motion-compensated blocks. In such cases, the motion compensation prediction unit 260 can access the reference picture memory 266 to fetch the samples used for prediction. According to the symbols 270 related to the block, after motion-compensating the fetched samples, these samples can be added by the aggregator 268 to the output of the scaler / inverse transform unit 258 (in this case, referred to as residual samples or a residual signal) to generate output sample information. The address in the reference picture memory 266 where the motion compensation prediction unit 260 fetches the prediction samples can be controlled by a motion vector. The motion vector can be utilized by the motion compensation prediction unit 260 in the form of symbols 270 that can have, for example, X, Y, and reference picture components. Motion compensation can also include interpolation of the sample values fetched from the reference picture memory 266 when an exact sub-sample motion vector is used, a motion vector prediction mechanism, and the like.

[0050] The output samples of the aggregator 268 can undergo various loop filtering techniques in the loop filter unit 256. Video compression techniques can include in-loop filter techniques. This technique is included in the coded video bitstream and is controlled by the parameters made available to the loop filter unit 256 as symbols 270 from the parser 254, but can also respond to meta information obtained during the decoding of the previous part (in the decoding order) of the coded picture or coded video sequence, and further can respond to previously configured loop filter processed sample values.

[0051] The output of the loop filter unit 256 can be an output to a rendering device such as the display 124 and further can be a sample stream that can be stored in the reference picture memory 266 for use in future inter-picture prediction.

[0052] When a particular coded picture is completely reconstructed, it can be used as a reference picture for future prediction. When the coded picture is completely reconstructed and the coded picture is identified as a reference picture (e.g., by parser 254), the current reference picture can become part of reference picture memory 266, and the unused current picture memory can be reallocated before starting the reconstruction of subsequent coded pictures.

[0053] Decoder component 122 may perform a decoding operation according to a predetermined video compression technique that may be documented by a standard such as any of the standard specifications described herein. The coded video sequence may conform to the syntax defined by the video compression technique or standard in use, in the sense that it follows the syntax of the video compression technique or standard, specifically in the profile document thereof. Also, for compliance with some video compression techniques or standards, the complexity of the coded video sequence can be within the bounds defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level can, in some cases, be further restricted through the Hypothetical Reference Decoder (HRD) specification and the metadata for HRD buffer management signaled in the coded video sequence.

[0054] FIG. 3 is a block diagram representing a server system 112 according to some embodiments. The server system 112 includes a control circuit 302, one or more network interfaces 304, a memory 314, a user interface 306, and one or more communication buses 312 interconnecting these components. In some embodiments, the control circuit 302 includes one or more processors (e.g., CPU, GPU, and / or DPU). In some embodiments, the control circuit 302 includes one or more field programmable gate arrays (FPGA), hardware accelerators, and / or one or more integrated circuits (e.g., application specific integrated circuit).

[0055] The network interface 304 may be configured to interface with one or more communication networks (e.g., wireless, wired, and / or optical networks). The communication network can be local, wide area, metropolitan, vehicle and industrial, real-time, delay tolerant, etc. Examples of communication networks include local area networks such as Ethernet (registered trademark) and wireless LAN; cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; TV wired or wireless wide area digital networks including cable TV, satellite TV, terrestrial broadcast TV; vehicle and industrial networks including CANBus, etc. Such communication can be unidirectional receive-only (e.g., broadcast TV), unidirectional transmit-only (e.g., CANBus to a specific CANBus device), or bidirectional (e.g., to another computer system using a local or wide area digital network). Such communication can include communication to one or more cloud computing networks.

[0056] The user interface 306 includes one or more output devices 308 and / or one or more input devices 310. The input device 310 may include one or more of a keyboard, a mouse, a trackpad, a touch screen, a data glove, a joystick, a microphone, a scanner, a camera, etc. The output device 308 may include one or more of an audio output device (e.g., a speaker), a visual output device (e.g., a display or a monitor), etc.

[0057] The memory 314 may include high-speed random access memory (e.g., DRAM, SRAM, DDR RAM, and / or other random access solid state memory devices) and / or non-volatile memory (e.g., one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid state storage devices). The memory 314 may optionally include one or more storage devices that are remote from the control circuit 302. The memory 314, or alternatively, the non-volatile solid state memory device within the memory 314, includes a non-transitory computer-readable storage medium. In some embodiments, the memory 314, or the non-transitory computer-readable storage medium of the memory 314, stores the following programs, modules, instructions, and data structures, or subsets or supersets thereof: ● An operating system 316 that includes procedures for handling various basic system services and performing hardware-dependent tasks; ● A network communication module 318 used to connect the server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired and / or wireless connections); ● A coding module 320 that performs various functions related to encoding and / or decoding data such as video data. In some embodiments, the coding module 320 is an instance of the coder component 114. The coding module 320: A decoding module 322 that executes various functions related to decoding of encoded data, such as the functions described above with respect to the decoder component 122; and One or more of, but not limited to, an encoding module 340 that executes various functions related to encoding of data, such as the functions described above with respect to the encoder component 106; ● A picture memory 352 that stores pictures and picture data for use, for example, by the coding module 320. In some embodiments, the picture memory 352 includes one or more of a reference picture memory 208, a buffer memory 252, a current picture memory 264, and a reference picture memory 266.

[0058] In some embodiments, the decoding module 322 includes a parsing module 324 (configured to execute various functions described above with respect to, for example, the parser 254), a conversion module 326 (configured to execute various functions described above with respect to, for example, the scanner / inverse conversion unit 258), a prediction module 328 (configured to execute various functions described above with respect to, for example, the motion compensation prediction unit 260 and / or the intra-picture prediction unit 262), and a filter module 330 (configured to execute various functions described above with respect to, for example, the loop filter 256).

[0059] In some embodiments, the encoding module 340 includes an encoding module 342 (configured to execute various functions described above with respect to, for example, the source coder 202 and / or the coding engine 212), and a prediction module 344 (configured to execute various functions described above with respect to, for example, the predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 includes a subset of the modules shown in FIG. 3. For example, a shared prediction module is used by both the decoding module 322 and the encoding module 340.

[0060] Each of the above-described modules stored in the memory 314 corresponds to a set of instructions for performing the functions described herein. The above-described modules (e.g., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus, various subsets of these modules may be combined or otherwise rearranged in various embodiments. For example, the coding module 320 optionally does not include separate decoding and encoding modules, but rather uses the same set of modules that perform the set of both functions. In some embodiments, the memory 314 stores a subset of the modules and data structures described above. In some embodiments, the memory 314 stores additional modules and data structures not described above, such as an audio processing module.

[0061] In some embodiments, the server system 112 includes web pages and applications implemented using a web or Hypertext Transfer Protocol (HTTP) server, File Transfer Protocol (FTP) server, and Common Gateway Interface (CGI) scripts, PHP Hypertext Preprocessor (PHP), Active Server Pager (ASP), Hyper Text Markup Language (HTML), Extensible Markup Language (XML), Java, JavaScript (registered trademark), Asynchronous JavaScript and XML (AJAX), XHP, Javelin, Wireless Universal Resource File (WURFL), and the like.

[0062] FIG. 3 depicts a server system 112 according to some embodiments, but FIG. 3 is intended as a functional description of various features that may exist in one or more server systems rather than a structural diagram of the embodiments described herein. In fact, as will be appreciated by those skilled in the art, the items shown separately may be combined and some items may be separated. For example, some of the items shown separately in FIG. 3 may be implemented on a single server, and a single item may be implemented by one or more servers. The actual number of servers used to implement the server system 112 and how the features are allocated among them may vary from implementation to implementation and optionally depend in part on the amount of data traffic that the server system handles during peak usage periods and average usage periods.

[0063] FIG. 4 represents a current coding block 400 including a plurality of sub-blocks according to some embodiments. A group of pictures (GOP) includes a sequence of image frames. The plurality of image frames includes a current image frame that further includes the current coding block 400. The current coding block 400 is encoded based on prediction data of one or more coding blocks (e.g., 410) of one or more reference images within the GOP. The current coding block 400 is divided into a plurality of sub-blocks (e.g., 4×4 sub-blocks). The upper left sub-block 402 is located at the sub-block position (0,0) on the current coding block 400, and the lower right sub-block 404 is located at, for example, the sub-block position (3,3) on the current coding block 400. The plurality of sub-blocks includes a first sub-block 406 located at the sub-block (1,1) on the current coding block 400. The current coding block 400 has a block-level motion vector MV current and the motion vector MV current is configured to identify a reference coding prediction block 410 within the block reference image frame for the current coding block 400. The first sub-block 406 has a first motion vector MV at the sub-block level 1,1having a first motion vector MV 1,1 is configured to identify a reference first prediction block 416 within a sub-block reference image frame for the first sub-block 406. The first motion vector MV of the first sub-block 406 1,1 The sub-block reference image frame identified by may optionally be the same as or different from the block reference image frame identified by the motion vector MV of the current coding block 400 current . Each of the block reference image frame and the sub-block reference image frame may optionally precede or follow the current image frame including the current coding block 400 within the GOP.

[0064] The motion compensation data for the first sub-block 406 is based on the motion vector MV of the current coding block 400 current and the first motion vector MV of the first sub-block 406 1,1 is generated. In some embodiments, the coding prediction block 410 is identified based on the motion vector MV of the current coding block 400 current , for example, within the block reference image frame. The first prediction block 416 is identified based on the first motion vector MV of the first sub-block 406 1,1 , for example, within the sub-block reference image frame. The coding prediction block 410 and the first prediction block 416 are combined to generate the motion compensation data for the first sub-block 406. Further, in some embodiments, the first weight w1 is determined in relation to the motion vector MV of the current coding block 400 current , and the second weight w2 is determined in relation to the first motion vector MV of the first sub-block 406 1,1 . The coding prediction block 410 and the first prediction block 416 are combined in a weighted manner, for example, by weighted averaging the coding prediction block 410 and the first prediction block 416 based on the first weight w1 and the second weight w2, thereby generating the motion compensation data for the first sub-block 406.

[0065] Furthermore, in some embodiments, the second weight w2 is the same for all samples of different sub - blocks within the coding prediction block 410, regardless of the position of the corresponding sub - block containing the sample. For example, the second weight w2 of the samples within the first sub - block 406 is determined based on the distance between the center of the current coding block 400 and the center of the first sub - block 406. Alternatively, in some embodiments, the second weight w2 depends on the position of each sample within the current coding block 400. For example, the second weight w2 of the samples within the first sub - block 406 is determined based on the distance between each sample of the current coding block 400 and the center of the first sub - block 406. The sum of the first weight w1 and the second weight w2 is equal to a predefined value (e.g., 64). In one example, the second weight w2 is determined to be 16 based on the distance between the center of the current coding block 400 and the center of the first sub - block 406, and the first weight w1 is equal to 48.

[0066] In some embodiments, a mode data item is applied to define the NEWMV mode. The encoded video bitstream 108 (FIG. 1) includes the mode data item and is sent to the electronic device 120. During encoding, the motion vector MV of the current coding block 400 current is encoded into the mode data item. During decoding, the electronic device 120 identifies or extracts the motion vector MV of the current coding block 400 from the mode data item. Alternatively, in some embodiments, the mode data item is applied to define the NEAR or NEAREST mode. The encoded video bitstream 108 includes the mode data item and is sent to the electronic device 120. During encoding, the motion vector MV of the current coding block 400 current is used to generate the mode data item. During decoding, the electronic device 120 derives the motion vector MV of the current coding block 400 from the mode data item. current current

[0067] ​​In some embodiments, a sub-block overlap block motion compensation (OBMC) indicator is associated with the current coding block 400. The encoded video bitstream 108 (FIG. 1) includes an OBMC indicator associated with the current coding block 400. Upon receiving the encoded video bitstream 108, the electronic device 120 determines, based on the OBMC indicator, that the sub-block OBMC mode is effective for the current coding block 400. The motion vector MV of the current coding block 400 current and the first motion vector of the first sub-block are determined to generate the motion compensation data of the first sub-block 406. Further, in some embodiments, in accordance with the determination that the sub-block OBMC mode is effective for the current coding block 400, the motion compensation data of the first sub-block 406 is based on the motion vector MV current , the first motion vector, and one or more adjacent motion vectors of one or more adjacent sub-blocks of the first sub-block 406. Further details regarding OBMC that requires adjacent sub-blocks are described below with reference to FIGS. 5-7.

[0068] In some embodiments, the first motion vector MV of the first sub-block 406 1,1includes a forward coding motion vector corresponding to the forward prediction block 416 and a backward coding motion vector corresponding to the backward prediction block 412. The method 700 further includes determining a sub-block OBMC parameter indicating whether each of the forward prediction block 416 and the backward prediction block 412 is applied to predict a plurality of sub-blocks of the current coding block 400 including the first sub-block 406. According to the determination that the sub-block OBMC parameter satisfies a predefined criterion (e.g., indicating that the blocks 412 and 416 are applied to predict the first sub-block 406), the first prediction block of the first sub-block is determined based on the average of the forward prediction block 416 and the backward prediction block 412. Further, in some embodiments, the first sub-block 406 is interpolated from the forward prediction block 416 and the backward prediction block 412. The forward coding motion vector and the backward coding motion vector are respectively based on the distances of the first sub-block 406 to the forward prediction block 416 and the backward prediction block 412.

[0069] FIG. 5 depicts a current coding block 400 including a first sub-block 406 surrounded by a plurality of adjacent sub-blocks 506 within the current coding block 400, according to some embodiments. The current coding block 400 is divided into a plurality of sub-blocks (e.g., 4×4 sub-blocks). The upper left sub-block 402 is located at the sub-block position (0, 0) on the current coding block 400, and the lower right sub-block 404 is located at, for example, the sub-block position (3, 3) on the current coding block 400. The plurality of sub-blocks includes the first sub-block 406 located at the sub-block (1, 1) on the current coding block 400. The current coding block 400 has a block-level motion vector MV current and the first sub-block 406 has a first motion vector MV at the sub-block level 1,1It has. The first sub-block 406 has one or more adjacent (e.g., directly adjacent, proximate) sub-blocks 506. Referring to FIG. 5, the one or more adjacent sub-blocks 506 are completely located within the same current coding block 400 as the first sub-block 406. The motion compensation data of the first sub-block 406 is the motion vector MV of the current coding block 400 current , the first motion vector MV of the first sub-block 406 1,1 , and is generated based on one or more adjacent motion vectors of the one or more adjacent sub-blocks 506.

[0070] In some embodiments, the one or more adjacent sub-blocks 506 include one or more of the upper adjacent sub-block 506T, the lower adjacent sub-block 506B, the left adjacent sub-block 506L, and the right adjacent sub-block 506R. Each adjacent sub-block 506 has an adjacent motion vector configured to identify a reference sub-block within the respective reference picture frame of each adjacent sub-block 506. The adjacent sub-blocks 506T, 506B, 506L, or 506R have sub-block positions (0,1), (2,1), (1,0), or (1,2), and the motion vectors MV 0,1 , MV 2,1 , MV 1,0 , or MV 1,2 correspond respectively. In some embodiments, the motion compensation data of the first sub-block 406 is the motion vector MV of the current coding block 400 current , the first motion vector MV of the first sub-block 406 1,1 , and is generated based on the motion vector MV of the current coding block 400 0,1 , MV 2,1 , MV 1,0 , and MV 1,2 and a subset or all of. In one example, only one adjacent motion vector (e.g., MV 0,1 ) is used together with the motion vectors MV current and MV 1,1 to generate the motion compensation data of the first sub-block 406. In another example, all the adjacent motion vectors MV 0,1 , MV 2,1 , MV 1,0 , and MV1,2 is used together with motion vector MV to generate motion compensation data for the first sub-block 406 current and MV 1,1 In yet another example, two adjacent motion vectors MV 0,1 and MV 1,0 are used to generate motion compensation data for the first sub-block 406

[0071] The prediction blocks of the current coding block 400, the first sub-block 406, and one or more adjacent sub-blocks 506 are respectively specified based on the motion vector MV current and the first motion vector MV of the first sub-block 406 1,1 and the adjacent motion vectors MV 0,1 and MV 2,1 and MV 1,0 and MV 1,2 and a subset or all of MV 0,1 and MV 2,1 and MV 1,0 and MV 1,2 and are combined to generate motion compensation data for the first sub-block 406. In some embodiments, these prediction blocks are combined in a weighted manner using, for example, the distance between each position of the samples within the first sub-block and the boundary of the adjacent sub-block and a subset or all of the adjacent motion vectors MV 0,1 and MV 2,1 and MV 1,0 and MV 1,2 Specifically, each of the adjacent motion vectors of the adjacent sub-block 506 has a third weight w3 determined based on the distance between each position of the samples within the first sub-block and the boundary of the adjacent sub-block. In some embodiments, the third weight for the adjacent sub-block is predefined and fixed. Equal weighting coefficients are used among all adjacent sub-blocks of the current coding block 400. The coding prediction block, the first prediction block, and one or more adjacent prediction blocks are averaged in a weighted manner based on the first weight w1, the second weight w2, and the third weight w3 of each adjacent sub-block 506 to generate motion compensation data for the first sub-block 406

[0072] Furthermore, in some embodiments, one or more adjacent sub-blocks 506 of the first sub-block 406 include one or more of the upper left adjacent sub-block 402, the upper right adjacent sub-block 506RT, the lower right adjacent sub-block 506RB, and the lower left adjacent sub-block 502LB, in addition to one or more of the adjacent sub-blocks 502T, 506B, 506L, and 506R. The adjacent sub-block 402, 506RT, 506RB, or 506LB has a sub-block position of (0,0), (0,2), (2,2), or (2,0) and a motion vector MV 0,0 , MV 0,2 , MV 2,2 , or MV 2,0 respectively. The motion compensation data of the first sub-block 406 is based on the motion vector MV current of the current coding block 400, the first motion vector MV 1,1 of the first sub-block 406, and a subset or all of the adjacent motion vectors MV 0,1 , MV 2,1 , MV 1,0 , MV 1,2 , MV 0,0 , MV 0,2 , MV 2,2 , and MV 2,0 . In some embodiments, each of the sub-blocks 506T, 506B, 506L, and 506R shares a side with the first sub-block 406 and is referred to as a neighboring sub-block. In some embodiments, the sub-blocks 402, 506RT, 506RB, 506LB, 506L, 506T, 506B, and 506R share a corner node or a side with the first sub-block 406 and are referred to as neighboring sub-blocks. In some embodiments, one or more adjacent sub-blocks 506 of the first sub-block 406 broadly include one or more of the sub-blocks that are directly adjacent to the neighboring sub-blocks.

[0073] In some embodiments, the first sub-block 406 includes the lower-right sub-block 404. One or more adjacent sub-blocks of the sub-block 404 include the upper-adjacent sub-block 508T, the left-adjacent sub-block 508L, and the upper-left adjacent sub-block 508LT. Each adjacent sub-block 508T, 508L, or 508LT has an adjacent motion vector MV 2,3 , MV 3,2 , or MV 2,2 configured to identify a reference sub-block within an adjacent reference picture frame in the GOP for each respective adjacent sub-block 508. The motion compensation data for the sub-block 404 is generated based on the motion vector MV current of the current coding block 400, the first motion vector MV 1,1 of the sub-block 404, and a subset or all of the adjacent motion vectors of the adjacent sub-blocks 508T, 508L, and 508LT.

[0074] In some embodiments not shown, the first sub-block 406 includes sub-blocks located at sub-block positions (1,3), (2,3), (3,1), (3,2), (1,2), (2,1), or (2,2). The motion compensation data for the first sub-block 406 is determined based on the motion vectors of one or more adjacent sub-blocks of the first sub-block 406, and the one or more adjacent sub-blocks are located within the current coding block 400.

[0075] FIG. 6 represents a portion of a current image frame 600 including a current coding block 400 and neighboring coding blocks 602, 604, and 606 according to some embodiments. The current image frame 600 includes a first sub-block 406 adjacent to a subset of adjacent sub-blocks 506 outside the current coding block 400. The current coding block 400 is divided into a plurality of sub-blocks (e.g., 4×4 sub-blocks). The upper left sub-block 402 is located at the sub-block position (0, 0) on the current coding block 400, and the lower right sub-block 404 is located at, for example, the sub-block position (3, 3) on the current coding block 400. The plurality of sub-blocks includes a first sub-block 406 located at the boundary of the current image frame 600. That is, at least one side of the first sub-block 406 overlaps with one of the boundaries of the current image frame 600. In this example, the first sub-block 406 is located at the sub-block position (0, 0) and includes the upper left sub-block 402 of the current coding block 400. The current coding block 400 has a block-level motion vector MV current and the first sub-block 406 has a first motion vector MV 0,0 at the sub-block level. The first sub-block 406 has one or more adjacent (e.g., directly adjacent, neighboring) sub-blocks 506. The motion compensation data of the first sub-block 406 is generated based on the motion vector MV current of the current coding block 400, the first motion vector MV 0,0 of the first sub-block 406, and one or more adjacent motion vectors of one or more adjacent sub-blocks 506.

[0076] In some embodiments, one or more adjacent sub-blocks 506 (e.g., 506R, 506B, 506RB, or combinations thereof) of the first sub-block 406 are completely located within the same current coding block 400 as the first sub-block 406. The motion compensation data of the first sub-block 406 is the motion vector MV current and the first motion vector MV 0,0and one or more adjacent motion vectors of one or more adjacent sub - blocks 506 within the current coding block 400 (e.g., MV 1,1 , MV 0,1 , MV 1,0 ), and is generated based on them.

[0077] Referring to FIG. 6, in some embodiments, one or more adjacent sub - blocks 506 of the first sub - block 406 (e.g., 506T, 506L, 506RT, 506LT, 506LB, or combinations thereof) are outside the current coding block 400 where the first sub - block 406 is located. For example, one or more adjacent sub - blocks 506 include one or both of sub - blocks 506T and 506RT and are included in the upper coding block 602 located immediately above the current coding block 400. In other examples, one or more adjacent sub - blocks 506 include sub - block 506LT and are at least partially included in the upper - left coding block 604 directly connected to the upper - left corner of the current coding block 400. In still other examples, one or more adjacent sub - blocks 506 include one or both of sub - blocks 506L and 506LB and are at least partially included in the left coding block 606 located immediately to the left of the current coding block 400.

[0078] In some embodiments, the motion - compensated data of the first sub - block 406 is based on one or more adjacent motion vectors of one or more adjacent sub - blocks 506 outside the current coding block 400 (e.g., MV -1,1 , MV 0,-1 , MV -1,0 , MV 1,-1 , MV -1,1 ). In one example, the motion - compensated data of the first sub - block 406 is the motion vector MV current , the first motion vector MV 0,0 , and the adjacent motion vectors MV -1,0 and MV 0,-1 of the adjacent sub - blocks 506T and 506LIt is generated based on. Alternatively, in some embodiments, for each adjacent sub-block 506 outside the current coding block 400, the adjacent motion vector of the corresponding proximity coding block is used as the motion vector of each adjacent sub-block 506 to determine the motion compensation data of the first sub-block 406. For example, the motion compensation data of the first sub-block 406 is based on the motion vector MV current and the first motion vector MV 0,0 and the adjacent motion vectors of the proximity coding blocks 602 and 606.

[0079] Furthermore, in some embodiments, the first sub-block 406 corresponds to a subset of the first adjacent sub-blocks located in the proximity coding blocks 602, 604, or 606 (i.e., outside the current coding block 400) and a subset of the second adjacent sub-blocks located within the current coding block 400. For example, the subset of the first adjacent sub-blocks includes one or both of the sub-block 506T located in the upper coding block 602 and the other sub-block 506L located in the left coding block 606, and the subset of the second adjacent sub-blocks includes one or both of the sub-blocks 506R and 506B.

[0080] In some embodiments not shown, the first sub-block 406 is located at the upper boundary (e.g., at sub-block position (0, 1)), and one or more adjacent sub-blocks 506 include a subset or all of three sub-blocks located at the last row of the upper coding block 602 located immediately above the current coding block 400 (e.g., located at positions (-1, 0), (-1, 1), and (-1, 2)). It should be noted that the motion compensation data of the first sub-block 406 is generated based on the motion vector of the upper coding block 602 or the motion vectors of the adjacent sub-blocks at the last row of the upper coding block 602. In some embodiments not shown, the first sub-block 406 is located at the left boundary (e.g., at sub-block position (2, 0)), and one or more adjacent sub-blocks 506 include a subset or all of three sub-blocks located at the rightmost column of the left coding block 606 located immediately to the left of the current coding block 400 (e.g., located at positions (1, -1), (2, -1), and (2, -1)). The motion compensation data of the first sub-block 406 is generated based on the motion vector of the left coding block 606 or the motion vectors of the adjacent sub-blocks at the rightmost column of the left coding block 606.

[0081] The first sub-block 406 corresponds to eight directly adjacent sub-blocks 506 that are connected to the first sub-block 406 at least by corners, and another sixteen adjacent sub-blocks that are separated from the first sub-block 406 only by the eight directly adjacent sub-blocks 506. In some embodiments, a subset or all of the eight directly adjacent sub-blocks 506 are applied to determine the motion compensation data of the first sub-block 406. In some embodiments, a subset or all of the sixteen adjacent sub-blocks are also applied to determine the motion compensation data of the first sub-block 406. In some embodiments, one or more adjacent sub-block identifiers are defined for the first sub-block 406 at the sub-block level, select one or more motion vectors of one or more adjacent sub-blocks 506 of the first sub-block 406, and are used to determine the motion compensation data of the first sub-block 406. Alternatively, in some embodiments, one or more adjacent sub-block identifiers are defined at the frame level and applied to all sub-blocks of the current image frame 600 including the first sub-block 406 of the current coding block 400. Alternatively, in some embodiments, one or more adjacent sub-block identifiers are defined at the coding block level and applied to a plurality of sub-blocks of the current coding block 400 including the first sub-block 406.

[0082] Furthermore, in some embodiments, an adjacent sub-block selection rule is applied. According to the adjacent sub-block selection rule, one or more adjacent sub-blocks 506 are selected for the first sub-block 406 based on the position of the first sub-block 406 within the current coding block 400, the position of the current coding block 400 within the current image frame 600, or both. For example, referring to FIG. 5, when the first sub-block 406 is located at sub-block position (1,1), the adjacent sub-blocks 506T, 506L, 506B, and 506R are applied to determine the motion compensation data of the first sub-block 406. When the first sub-block 406 is located at sub-block position (3,3), the adjacent sub-blocks 508T, 508L, and 508LT are applied to determine the motion compensation data of the first sub-block 406.

[0083] FIG. 7 is a flowchart of an exemplary method 700 for processing (e.g., decoding) video data according to some embodiments. The method 700 may be executed in a computing system (e.g., server system 112, source device 102, or electronic device 120) comprising a control circuit and a memory storing instructions executed by the control circuit. In some embodiments, the method 700 is executed by executing instructions (e.g., coding module 320 of memory 314) stored in the memory of the computing system. The computer system receives (702) a current image frame including a current coding block 400. The current coding block 400 has a plurality of sub-blocks. The plurality of sub-blocks are associated with different motion vectors and include a first sub-block 406 located at the boundary of the current coding block. The computer system determines (704) the motion vector MV of the current coding block 400, determines (706) the first motion vector of the first sub-block 406, and generates (708) motion compensation data for the first sub-block 406 based on the motion vector of the current coding block 400 and the first motion vector of the first sub-block 406. current and generates (708) motion compensation data for the first sub-block 406 based on the motion vector of the current coding block 400 and the first motion vector of the first sub-block 406.

[0084] In some embodiments, the warping motion mode is applied as one of a plurality of coding modes to generate sub-block motion vectors within the current coding block 400. The current coding block 400 is divided into a plurality of sub-blocks. Each sub-block is associated with its own motion vector for motion compensation. Optionally, for every two sub-blocks of the current coding block 400, they have the same motion vector or different motion vectors. Overlapped block motion compensation is applied at the sub-block boundaries of the current coding block 400. Further, in some embodiments, the current coding block 400 is associated with a motion vector and includes a first sub-block 406 having a first motion vector. In one example, the current coding block 400 is divided into 16 sub-blocks (see, e.g., FIG. 4) and is associated with a motion vector MV current In some embodiments, the motion vector MV current of the current coding block 400 is signaled using the NEWMV mode. Alternatively, in some embodiments, the motion vector MV current of the current coding block 400 is derived using the NEAR or NEAREST mode, and for each sub-block of the current coding block 400, an individual motion vector is derived using the warping motion mode. For example, the sub-block motion vector MV 1,1 is associated with a first sub-block 406 located at the sub-block position (1, 1) of the current coding block 400. The current coding block 400 is coded by the local warping motion mode, and for the first sub-block 406 located at the sub-block position (1, 1), overlapped block motion compensation is applied to generate motion compensation data for the first sub-block 406 based on the motion vector MV current of the current coding block 400 and the first motion vector MV 1,1 of the first sub-block 406. In one example, the motion compensation data for the first sub-block 406 located at the sub-block position (1, 1) is the motion vector MV currentThe coding prediction block 410 specified (710) thereby and the first motion vector MV 1,1 The first prediction block 416 specified (712) thereby are combined (714), for example, by using a weighted average of the coding prediction block 410 and the first prediction block 416, to be generated.

[0085] In some embodiments, regardless of the sub-block position of the first sub-block 406 within the current coding block 400, the motion vector used for overlap block motion compensation is the motion vector MV related to the current coding block 400 current , one or more adjacent motion vectors of one or more adjacent sub-blocks 506 (FIG. 5) of the first sub-block 406, and the motion vectors of the adjacent coding blocks 602, 604, or 606, including one or more of them. In one example, the current coding block 400 is divided into 16 sub-blocks, and the motion vector MV current is associated therewith. In some embodiments, the motion vector MV of the current coding block 400 current is signaled using the NEWMV mode. Alternatively, in some embodiments, the motion vector MV of the current coding block 400 current is derived using the NEAR or NEAREST mode, and for each sub-block of the current coding block 400, an individual motion vector is derived using the local warping motion mode. For each sub-block of the current coding block 400, each motion vector is derived using the local warping motion mode. For example, the first motion vector MV 1,1 is related to the first sub-block 406 located at the sub-block position (1,1) of the current coding block 400.

[0086] In some embodiments, the first sub-block 406 located at the sub-block position (1,1) has four motion vectors MV 1,0 , MV 0,1 , MV 1,2 , and MV 2,1It has four adjacent sub - blocks located at sub - block positions (1,0), (0,1), (1,2), and (2,1) respectively (716). When the current coding block 400 is coded in the local warping motion mode and overlap block motion compensation is applied to the current coding block 400, the motion compensation data of the first sub - block 406 located at the sub - block position (1,1) of the current coding block 400 is the first motion vector M of the first sub - block 406 1,1 and a subset or all of the adjacent motion vectors MV 1,0 of the adjacent sub - blocks, MV 0,1 MV 1,2 and MV 2,1 are used to generate it (720). In some embodiments, each adjacent sub - block 506 includes a proximity sub - block that is directly adjacent to the first sub - block 406 (722).

[0087] Furthermore, each current sample within the first sub - block 406 corresponds to predicted samples having different weights and related to the adjacent motion vectors MV 1,0 MV 0,1 MV 1,2 and MV 2,1 . The predicted sample of the current sample related to the motion vector MV refers to the sample in the reference picture by adding the motion MV to the position of the current sample. The corresponding predicted samples related to the adjacent motion vectors MV 1,0 MV 0,1 MV 1,2 and MV 2,1 are combined based on different weights so as to generate each current sample within the first sub - block 406. For the current sample of the first sub - block 406, the weight of each predicted sample specified by the adjacent motion vectors MV 1,0 MV 0,1 MV 1,2 and MV 2,1 is determined based on the relative distance from the current sample of the first sub - block 406 to each adjacent sub - block 506L, 506T, 506R, or 506B.

[0088] In some embodiments, the motion compensation data of the first sub-block 406 of the current coding block 400 (e.g., located at the sub-block position (1, 1)) is the motion vector of the current coding block 400, the first motion vector of the first sub-block 406, and the adjacent motion vectors MV 1,0 , MV 0,1 , MV 1,2 and MV 2,1 and is generated using one or more of them. In some embodiments, an indicator is applied to indicate whether the motion vector of the current coding block 400 or the adjacent motion vectors of the adjacent sub-blocks are used. The indicator is explicitly encoded in the data stream for the current coding block 400 and is decoded by the video decoder 122 to determine whether the motion vectors of the first sub-block and the adjacent sub-blocks are used to generate motion compensation data at the sub-block level. Alternatively, in some embodiments, an indicator is not explicitly encoded in the data stream for the current coding block 400 to indicate whether the motion vector MV of the current coding block 400 and the adjacent motion vectors of the adjacent sub-blocks are used. The data stream received by the video decoder implicitly contains such information and is thus analyzed to determine whether the motion vectors of the first sub-block and the adjacent sub-blocks are used to generate motion compensation data at the sub-block level.

[0089] In some embodiments, the first sub-block 406 is located directly adjacent to the boundary of the current coding block 400 (at the sub-block position (0, 1)), and the motion compensation data of the first sub-block 406 is the motion vector MV of the current coding block 400 currentIt is generated using a subset of the motion vectors of the neighboring coding block and / or the adjacent motion vectors of the adjacent sub-blocks. In some embodiments, an indicator is applied to indicate whether the motion vector of the current coding block 400, or the neighboring coding block, or the adjacent sub-block is used. The indicator is explicitly encoded in the data stream for the current coding block 400 and is decoded by the video decoder 122 to determine which motion vectors are used to generate motion compensation data at the sub-block level. Alternatively, in some embodiments, the indicator is not explicitly encoded in the data stream for the current coding block 400. The data stream is sent to the video decoder 122 and is analyzed to determine which motion vectors are used to generate motion compensation data at the sub-block level.

[0090] In some embodiments, based on the position of the first sub-block 406 within the current coding block 400, the motion vectors of the adjacent sub-blocks within the adjacent coding block are applied to determine the motion compensation data of the first sub-block 406 of the current coding block 400. For example, in accordance with the determination that the first sub-block 406 is located at the boundary (e.g., the upper boundary or the left boundary) of the current coding block 400, the motion vector of the adjacent coding block, the motion vector of the adjacent sub-block of the adjacent coding block, or both are used to determine the motion compensation data of the first sub-block 406 of the current coding block 400. Conversely, in accordance with the determination that the first sub-block 406 is not located at the boundary of the current coding block 400, the motion vectors of the adjacent sub-blocks located within the same current coding block 400 are used to determine the motion compensation data of the first sub-block 406 of the current coding block 400. In some embodiments, in accordance with the determination that the first sub-block 406 is located at the lower or right boundary of the current coding block 400, the motion vector of the adjacent coding block is not used, while one or more motion vectors of one or more adjacent sub-blocks located within the same current coding block 400 are used to determine the motion compensation data of the first sub-block 406 of the current coding block 400.

[0091] In some embodiments, in a bidirectional optical flow (OPFL) (e.g., CWG-B041), overlap block motion compensation is applied at sub-block boundaries. A sub-block refers to a block to which an individual motion vector is assigned by the use of OPFL. In one example, the sub-block size is 4×4 or 8×8 pixels. Overlap block motion compensation is applied on an OPFL-coded block to derive a predicted block related to one or both reference frames. In some embodiments, a Temporal Interpolated Prediction (TIP) mode (e.g., in CWG-C007) is applied. Overlap block motion compensation is applied at sub-block boundaries. A sub-block refers to a block on which a motion field is generated. In one example, the sub-block size is 8×8 pixels. Overlap block motion compensation is applied on a TIP-coded block to derive a predicted block related to one or both reference frames.

[0092] Specifically, in some embodiments, the motion compensation data of sub - blocks of a coding block (e.g., 410 and 416 in FIG. 4) within a reference frame is determined based on a combination of the motion vector of the coding block and the first motion vector of the corresponding sub - block. Further, in some embodiments, the motion compensation data of sub - blocks of a coding block (e.g., 410 and 416 in FIG. 4) within a reference frame is determined based on one or more motion vectors of one or more adjacent sub - blocks within the same coding block. Alternatively, in some embodiments, the motion compensation data of sub - blocks of a coding block (e.g., 410 and 416 in FIG. 4) within a reference frame is determined based on one or more motion vectors of one or more adjacent sub - blocks within one or more adjacent coding blocks within the reference frame, e.g., based on the motion vector of the upper sub - block of the upper coding block and the motion vector of the left sub - block of the left coding block. In some embodiments, an indicator is applied to indicate whether the motion vector of a coding block, or a neighboring coding block, or an adjacent sub - block is used for coding of the reference frame. The indicator is explicitly encoded in the data stream for coding blocks of the reference frame (e.g., 410 and 416 in FIG. 4) and is decoded by the video decoder 122 to determine which motion vectors are used to generate motion compensation data at the sub - block level for the reference frame. Alternatively, in some embodiments, the indicator is not explicitly encoded in the data stream for coding blocks of the reference frame. The data stream received by the video decoder 122 implicitly contains such information and is thus analyzed to determine which motion vectors are used to generate motion compensation data at the sub - block level for coding blocks of the reference frame.

[0093] FIG. 7 represents a number of logical steps in a particular order, but the order-independent steps may be rearranged and other steps may be combined or decomposed. Since some rearrangements or other groupings not specifically described will be apparent to those skilled in the art, the order and groupings presented herein are not inclusive. Further, it should be recognized that the steps may be implemented in hardware, firmware, software, or any combination thereof.

[0094] Reference will now be made to some exemplary embodiments.

[0095] (A1) In one aspect, some embodiments include a method of processing video data (e.g., method 700). Method 700 is executed by a computing system (e.g., server system 112) that includes a control circuit (e.g., control circuit 302) and a memory (e.g., memory 314) coupled to the control circuit. Method 700 includes receiving (702) a current frame including a current coding block, the current coding block having a plurality of sub-blocks, the plurality of sub-blocks being associated with different motion vectors and including a first sub-block located at a boundary of the current coding block. Method 700 further includes determining (704) a motion vector of the current coding block, determining (706) a first motion vector of the first sub-block, and generating (708) motion compensation data for the first sub-block based on the motion vector of the current coding block and the first motion vector of the first sub-block.

[0096] (A2)In some embodiments of A1, the step of generating motion compensation data for the first sub-block based on the motion vector of the current coding block and the first motion vector of the first sub-block includes a step (710) of identifying a coding prediction block based on the motion vector of the current coding block, a step (712) of identifying a first prediction block based on the first motion vector of the first sub-block, and a step (714) of combining the coding prediction block and the first prediction block to generate motion compensation data for the first sub-block.

[0097] (A3)In some embodiments of A2, the method further includes a step of determining a first weight associated with the coding prediction block of the current coding block, and a step of determining a second weight associated with the first prediction block based on the distance between the center of the current coding block and each sample within the first sub-block. The step of combining the coding prediction block and the first prediction block further includes a step of averaging the coding prediction block and the first prediction block based on the first weight and the second weight in a weighted manner to generate motion compensation data for the first sub-block.

[0098] (A4)In some embodiments of any of A1 - A3, the method 700 further includes a step (716) of identifying one or more adjacent sub-blocks of the first sub-block. The method 700 further includes a step (718) of determining an adjacent motion vector for each adjacent sub-block. The motion compensation data for the first sub-block is generated based on the motion vector of the current coding block, the first motion vector of the first sub-block, and one or more adjacent motion vectors of one or more adjacent sub-blocks (720).

[0099] (A5) In some embodiments of A4, the method further includes determining a third weight for each adjacent sub-block, and identifying a coding prediction block, a first prediction block, and one or more adjacent prediction blocks based on the motion vector of the current coding block, the first motion vector of the first sub-block, and one or more adjacent motion vectors. Method 700 further includes averaging the coding prediction block, the first prediction block, and one or more adjacent prediction blocks in a weighted manner based on the first weight, the second weight, and the third weight of each adjacent sub-block to generate motion compensation data for the first sub-block.

[0100] (A6) Further, in some embodiments of A4 or A5, the method further includes determining a first weight associated with the coding prediction block and determining a second weight associated with the first prediction block based on the distance between the center of the current coding block and each sample in the first sub-block. For each sample in the first sub-block, the third weight of each prediction block is determined based on the distance of the sample of the first sub-block from each adjacent sub-block.

[0101] (A7) In some embodiments of A4 - A6, the first sub-block is located at the boundary of the current coding block. For example, the first sub-block is within the current coding block and one or two sides of the first sub-block 406 overlap one or two boundaries of the current coding block. The one or more adjacent sub-blocks include a subset of first adjacent sub-blocks located in adjacent coding blocks (e.g., the upper coding block 602, the left coding block 606, and the upper - left coding block 604 in FIG. 6), and the adjacent coding blocks are directly adjacent to the current coding block.

[0102] (A8) In some embodiments of A7, the adjacent coding block is located immediately above or immediately to the left of the current coding block.

[0103] (A9)In some embodiments of any one of A4 to A8, one or more adjacent sub-blocks include a subset of second adjacent sub-blocks located together with the first sub-block within the current coding block, and the subset of second adjacent sub-blocks includes one or more of a left sub-block, an upper sub-block, a lower sub-block, and a right sub-block. Each of the left sub-block, the upper sub-block, the lower sub-block, and the right sub-block is respectively connected to one of each of the left side, the upper side, the lower side, and the right side of the first sub-block.

[0104] (A10)In some embodiments of any one of A4 to A9, the method further includes identifying the one or more adjacent sub-blocks based on one or more adjacent sub-block identifiers associated with the first sub-block.

[0105] (A11)In some embodiments of any one of A4 to A10, the method further includes identifying one or more adjacent sub-blocks based on the position of the first sub-block within the current coding block according to an adjacent sub-block selection rule.

[0106] (A12)In some embodiments of any one of A4 to A11, each adjacent sub-block includes a proximity sub-block that is directly adjacent to the first sub-block (722).

[0107] (A13)In some embodiments of any one of A1 to A12, the mode data item is applied to define the NEWMV mode. Determining the motion vector of the current coding block further includes identifying the motion vector of the current coding block using the mode data item.

[0108] (A14)Alternatively, in some embodiments of any one of A1 to A12, the mode data item is applied to define the NEAR or NEAREST mode. Determining the motion vector of the current coding block further includes deriving the motion vector of the current coding block using the mode data item.

[0109] (A15)In some embodiments of any one of A1 to A14, determining the first motion vector of the first sub-block further includes deriving the first motion vector of the first sub-block using a local warping motion mode.

[0110] (A16)In some embodiments of any one of A1 to A15, method 700 further includes identifying a sub-block overlap block motion compensation (OBMC) indicator associated with the current coding block. According to the determination that the sub-block OBMC mode is valid based on the OBMC indicator, the motion vector of the current coding block and the first motion vector of the first sub-block are determined to generate motion compensation data for the first sub-block.

[0111] (A17)In some embodiments of any one of A1 to A16, the first motion vector of the first sub-block includes a forward coding motion vector corresponding to a forward prediction block and a backward coding motion vector corresponding to a backward prediction block. Method 700 further includes determining a sub-block OBMC parameter indicating whether each of the forward prediction block and the backward prediction block is applied to predict a plurality of sub-blocks of the current coding block including the first sub-block. According to the determination that the sub-block OBMC parameter satisfies a predefined criterion, the first prediction block of the first sub-block is determined based on the average of the forward prediction block and the backward prediction block.

[0112] (A18)Furthermore, in some embodiments of A17, the first sub-block is interpolated from the forward prediction block and the backward prediction block. Method 700 further includes determining the forward coding motion vector and the backward coding motion vector respectively based on the distance of the first sub-block to the forward prediction block and the backward prediction block.

[0113] In other aspects, some embodiments include a computing system (e.g., server system 112) that includes a control circuit (e.g., control circuit 302) and a memory (e.g., memory 314) coupled to the control circuit, the memory storing one or more programs configured to be executed by the control circuit, the one or more programs including instructions for performing any of the methods described herein (e.g., A1 - A18 above).

[0114] In yet other aspects, some embodiments include a non - transitory computer - readable storage medium storing a set of one or more programs configured to be executed by a control circuit of a computing system, the set of one or more programs including instructions for performing any of the methods described herein (e.g., A1 - A18 above).

[0115] The terms "first", "second", etc. may be used herein to describe various elements, but it will be understood that these elements should not be limited by these terms. These terms are only used to distinguish one element from another.

[0116] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be a limitation of the claims. As used in the description of embodiments and the appended claims, the singular forms ( "a", "an", and "the") are intended to include the plural forms as well, unless the context clearly dictates otherwise. The term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. Further, the terms "comprises" and / or "comprising", when used herein, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features.

[0117] As used herein, the term "if" can be interpreted, depending on the context, to mean "when" or "upon" or "in response to determining" or "in accordance with a determination" or "in response to detecting" that the condition recited above it is true. Similarly, "when it is determined that" or "when" or "if" [the condition recited above it is true] can be interpreted, depending on the context, to mean "upon determining that" or "in response to determining" or "in accordance with a determination" or "upon detecting" or "in response to detecting" that the condition recited above it is true.

[0118] The foregoing description has been presented for purposes of illustration and is described with reference to specific embodiments. However, the above examples of discussion are not intended to be exhaustive or to limit the scope of the claims to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of operation and the practical application and to enable others skilled in the art to best utilize the technology.

[0119] [Related Applications] This application claims the priority of U.S. Provisional Patent Application No. 63 / 346,067, filed on May 26, 2022, under the title of "Combination Between Subblock Motion And Overlapped Block Motion Compensation", and is a continuation application of U.S. Patent Application No. 18 / 142,192, filed on May 2, 2023, under the title of "Systems and Methods for Combining Subblock Motion Compensation and Overlapped Block Motion Compensation", and claims the priority thereof. The entire text of the previous U.S. application is incorporated herein by reference.

Claims

1. A method for decoding video data in a computing system, comprising: Receiving a current frame including a current coding block, the current coding block having a plurality of sub-blocks, the plurality of sub-blocks being associated with different motion vectors, and including a first sub-block located at a boundary of the current coding block; Determining a motion vector of the current coding block; Determining a first motion vector of the first sub-block; Generating motion compensation data for the first sub-block based on the motion vector of the current coding block and the first motion vector of the first sub-block; The method comprising the steps of.

2. The step of generating motion compensation data for the first sub-block based on the motion vector of the current coding block and the first motion vector of the first sub-block comprises: Identifying a coding prediction block based on the motion vector of the current coding block; Identifying a first prediction block based on the first motion vector of the first sub-block; Combining the coding prediction block and the first prediction block to generate the motion compensation data for the first sub-block; The method according to claim 1, further comprising: The method according to claim 1.

3. Determining a first weight associated with the coding prediction block of the current coding block; Determining a second weight associated with the first prediction block based on a distance between the center of the current coding block and each sample within the first sub-block; The method further comprising: The step of combining the coding prediction block and the first prediction block further comprises averaging the coding prediction block and the first prediction block based on the first weight and the second weight in a weighted manner to generate the motion compensation data for the first sub-block; The method according to claim 2.

4. Identifying one or more adjacent sub-blocks of the first sub-block; Determining an adjacent motion vector for each adjacent sub-block; The method further comprising: The motion compensation data of the first sub-block is generated based on the motion vector of the current coding block, the first motion vector of the first sub-block, and one or more adjacent motion vectors of the one or more adjacent sub-blocks. The method according to claim 1.

5. Determining a third weight for each adjacent sub-block; Identifying a coding prediction block, a first prediction block, and one or more adjacent prediction blocks based on the motion vector of the current coding block, the first motion vector of the first sub-block, and the one or more adjacent motion vectors; Averaging the coding prediction block, the first prediction block, and the one or more adjacent prediction blocks in a weighted manner based on a first weight, a second weight, and the third weight of each adjacent sub-block so as to generate the motion compensation data of the first sub-block. The method according to claim 4, further comprising:

6. Determining the first weight associated with the coding prediction block; Determining the second weight associated with the first prediction block based on the distance between the center of the current coding block and each sample in the first sub-block. The method further comprising: For each sample of the first sub-block, the third weight of each prediction block is determined based on the distance of the sample of the first sub-block from each of the adjacent sub-blocks. The method according to claim 5.

7. The first sub-block is located at the boundary of the current coding block. The one or more adjacent sub-blocks include a subset of first adjacent sub-blocks located in adjacent coding blocks. The adjacent coding block is directly adjacent to the current coding block. The method according to claim 4.

8. The adjacent coding block is located directly above the current coding block or directly to the left of the current coding block. The method according to claim 7.

9. The one or more adjacent sub-blocks include a subset of second adjacent sub-blocks located together with the first sub-block within the current coding block. The subset of the second adjacent sub-blocks includes one or more of a left sub-block, an upper sub-block, a lower sub-block, and a right sub-block. Each of the left sub-block, the upper sub-block, the lower sub-block, and the right sub-block is respectively connected to one of the left side, the upper side, the lower side, and the right side of the first sub-block. The subset of the second adjacent sub-blocks includes (1) all of the adjacent sub-blocks when the first sub-block is not a boundary sub-block, or (2) a part that does not include all of the adjacent sub-blocks when the first sub-block is a boundary sub-block. The method according to claim 4.

10. Further comprising the step of identifying the one or more adjacent sub-blocks based on one or more adjacent sub-block identifiers associated with the first sub-block. The method according to claim 4.

11. Further comprising the step of identifying the one or more adjacent sub-blocks according to an adjacent sub-block selection rule based on the position of the first sub-block in the current coding block. The method according to claim 4.

12. Each adjacent sub-block includes a proximity sub-block that is directly adjacent to the first sub-block. The method according to claim 4.

13. A control circuit; A memory storing one or more programs configured to be executed by the control circuit. And having The one or more programs Include an instruction to receive a current frame including a current coding block, the current coding block having a plurality of sub-blocks, the plurality of sub-blocks being associated with different motion vectors, and including a first sub-block located at the boundary of the current coding block, the receiving instruction; An instruction to determine a motion vector of the current coding block; An instruction to determine a first motion vector of the first sub-block; And an instruction to generate motion compensation data for the first sub-block based on the motion vector of the current coding block and the first motion vector of the first sub-block. A computing system.

14. The mode data item is applied to define the NEWMV mode, and the instruction to determine the motion vector of the current coding block Further includes specifying the motion vector of the current coding block using the mode data item. The computing system according to claim 13.

15. A mode data item is applied to define a NEAR or NEAREST mode, and the instruction for determining the motion vector of the current coding block further comprises deriving the motion vector of the current coding block using the mode data item. The computing system according to claim 13. **Claim 16** Determining the first motion vector of the first sub-block further comprises deriving the first motion vector of the first sub-block using a local warping motion mode. The computing system according to claim 13. **Claim 17** A non-transitory computer-readable storage medium storing one or more programs executed by a control circuit of a computing system, wherein the one or more programs include an instruction for receiving a current frame including a current coding block, the current coding block having a plurality of sub-blocks, the plurality of sub-blocks being associated with different motion vectors, and including a first sub-block located at a boundary of the current coding block, the receiving instruction; an instruction for determining a motion vector of the current coding block; an instruction for determining a first motion vector of the first sub-block; and an instruction for generating motion compensation data for the first sub-block based on the motion vector of the current coding block and the first motion vector of the first sub-block. A non-transitory computer-readable storage medium having the above. **Claim 18** The one or more programs further include an instruction for identifying a sub-block overlap block motion compensation (OBMC) indicator associated with the current coding block, and according to a determination that the sub-block OBMC mode is effective based on the OBMC indicator, the motion vector of the current coding block and the first motion vector of the first sub-block are determined to generate the motion compensation data for the first sub-block. The non-transitory computer-readable storage medium according to claim 17. **Claim 19** The first motion vector of the first sub-block includes a forward coding motion vector corresponding to a forward prediction block and a backward coding motion vector corresponding to a backward prediction block. The one or more programs Further comprising a step of determining a sub-block OBMC parameter indicating whether each of the forward prediction block and the backward prediction block is applied to predict a plurality of sub-blocks of the current coding block including the first sub-block, According to the determination that the sub-block OBMC parameter satisfies a predefined criterion, the first prediction block of the first sub-block is determined based on the average of the forward prediction block and the backward prediction block, The non-transitory computer-readable storage medium according to claim 17.

20. The first sub-block is interpolated from the forward prediction block and the backward prediction block, The one or more programs, Further comprising instructions for determining the forward coding motion vector and the backward coding motion vector respectively based on the distance of the first sub-block to the forward prediction block and the backward prediction block, The non-transitory computer-readable storage medium according to claim 19.

Images