Chain-based motion vector refinement

Abstract

Some aspects of the disclosure provide a method of video decoding. In one example, a coded video bitstream is received. The coded video bitstream includes coded information of a current block in a current picture. It is determined that the coded information indicates that the current block is predicted using a chain vector. The chain vector includes at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block. At least one vector offset for refining the chain vector is determined based on at least the first reference block and the second reference block. The current block is reconstructed based on the chain vector and the vector offset.

Description

Incorporation

[0001] This application claims priority to U.S. Patent Application No. 19 / 172,516, filed April 7, 2025, which claims priority to U.S. Provisional Application No. 63 / 631,413, filed April 8, 2024, and U.S. Provisional Application No. 63 / 638,396, filed April 24, 2024. The entire disclosure of these earlier applications is incorporated herein by reference in its entirety. Technical Field

[0002] This invention describes various aspects generally related to video encoding and decoding. Background Technology

[0003] The background description provided in this application is intended to generally present the context of the invention. To the extent that the work described in this background section is intended, the work of the currently named inventors and aspects of the description that may not have otherwise qualified as prior art at the time of filing are neither explicitly nor implicitly considered as prior art to the invention.

[0004] Image / video compression facilitates the transfer of image / video data across different devices, storage devices, and networks with minimal quality loss. In some examples, video codecs can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-frame prediction, which compresses images based on spatial redundancy. For example, intra-frame prediction can use reference data from the current image being reconstructed to predict samples. In another example, a video codec can use a technique called inter-frame prediction, which compresses images based on temporal redundancy. For example, inter-frame prediction can utilize motion compensation to predict samples in the current image based on previously reconstructed images. Motion compensation can be indicated by a motion vector (MV). Summary of the Invention

[0005] Various aspects of the present invention include bitstreams, methods, and apparatus for video encoding / decoding. In some examples, the apparatus for video encoding / decoding includes processing circuitry.

[0006] Some aspects of the present invention provide a video decoding method. In one example, an encoded video stream is received. The encoded video stream includes encoded information of a current block in a current image. The encoded information is determined to indicate prediction of the current block using a chained vector. The chained vector includes at least a first sub-vector and a second sub-vector, the first sub-vector pointing relative to the current block to a first reference block, and the second sub-vector pointing relative to the first reference block to a second reference block. At least one vector offset for refining the chained vector is determined, based at least on the first and second reference blocks. The current block is reconstructed based on the chained vector and the vector offset.

[0007] Some aspects of the present invention provide a method for video encoding. In one example, a refined chained vector is determined to be applied to a current block in a current image. The chained vector includes at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block. Based on the first and second reference blocks, at least one vector offset is determined. The current block is encoded based on the chained vector and the vector offset.

[0008] Some aspects of the present invention provide a video decoding method. In one example, an encoded video stream is received. The encoded video stream includes encoded information of a current block in a current image. The encoded information is determined to indicate that the current block is predicted using a chained vector, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing relative to the current block to a first reference block, and the second sub-vector pointing relative to the first reference block to a second reference block. A prediction filter is determined based on the first and second reference blocks. The current block is reconstructed based on the chained vector and the prediction filter.

[0009] Some aspects of the present invention provide a method for video coding. In one example, a predictive filtering evaluation is performed on a chained vector of a current block in a current image. The chained vector includes at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block. A predictive filter is determined based on the first and second reference blocks. The current block is encoded based on the chained vector and the predictive filter.

[0010] Some aspects of the present invention provide a method for processing visual media data. In this method, a conversion between a visual media file and a visual media data bitstream is performed according to format rules. In one example, the bitstream includes encoded information of a current block in a current image, the encoded information of the current block indicating prediction of the current block based on a chained vector, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing relative to the current block to a first reference block, and the second sub-vector pointing relative to the first reference block to a second reference block. The format rules specify that at least one vector offset is determined based on the first and second reference blocks; and the current block is reconstructed based on the chained vector and the vector offset.

[0011] Some aspects of the present invention provide a method for processing visual media data. In this method, a conversion between a visual media file and a visual media data bitstream is performed according to format rules. In one example, the bitstream includes encoded information of a current block in a current image, the encoded information indicating prediction of the current block using a chained vector, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block. The format rules specify that a prediction filter is determined based on the first and second reference blocks; and the current block is reconstructed based on the chained vector and the prediction filter. Other aspects of the present invention also provide an apparatus for video coding. The apparatus for video coding includes processing circuitry configured to implement any of the described methods for video coding.

[0012] The present invention also provides an apparatus for video decoding. The apparatus for video decoding includes processing circuitry configured to implement any of the described methods for video decoding.

[0013] Aspects of the present invention also provide an apparatus for video encoding. The apparatus for video encoding includes processing circuitry configured to implement any of the described methods for video encoding.

[0014] Aspects of the present invention also provide a non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform any of the described methods for video decoding / encoding. Attached Figure Description

[0015] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0016] Figure 1 An example block diagram of a communication system is shown schematically.

[0017] Figure 2An example block diagram of a decoder is shown schematically.

[0018] Figure 3 An example block diagram of an encoder is shown schematically.

[0019] Figure 4 The diagram shows some examples of chain motion vectors.

[0020] Figure 5 A diagram showing bidirectional chain motion vectors in some examples is provided.

[0021] Figure 6 A diagram showing some examples of chained block vectors is provided.

[0022] Figure 7 A diagram showing the refined motion vector of a unidirectional chain motion vector according to an embodiment of the present invention is provided.

[0023] Figure 8 A diagram showing the refined motion vector of a bidirectional chain motion vector according to an embodiment of the present invention is provided.

[0024] Figure 9 A diagram illustrating the refined motion vector of a bidirectional motion vector including a chain motion vector according to an embodiment of the present invention is shown.

[0025] Figure 10 A diagram illustrating the refined motion vector of a bidirectional motion vector including a chain motion vector according to an embodiment of the present invention is shown.

[0026] Figure 11 A technical diagram of a prediction filter for determining a chain motion vector according to an embodiment of the present invention is shown.

[0027] Figure 12 A technical diagram of a predictive filter for determining a bidirectional chain motion vector is shown according to an embodiment of the present invention.

[0028] Figure 13 A technical diagram of a predictive filter for determining a bidirectional chain motion vector is shown according to an embodiment of the present invention.

[0029] Figure 14 A technical diagram of a prediction filter for determining a chain motion vector according to an embodiment of the present invention is shown.

[0030] Figure 15 A flowchart outlining the decoding process according to some aspects of the present invention is shown.

[0031] Figure 16 A flowchart outlining the encoding process according to some aspects of the present invention is shown.

[0032] Figure 17A flowchart outlining the decoding process according to some aspects of the present invention is shown.

[0033] Figure 18 A flowchart outlining the encoding process according to some aspects of the present invention is shown.

[0034] Figure 19 It is a schematic diagram of a computer system based on one aspect. Detailed Implementation

[0035] Figure 1 Block diagrams of some example video processing systems (100) are shown. The video processing system (100) is an example application of the disclosed subject matter, namely, video encoders and video decoders in a streaming media environment. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming media services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0036] The video processing system (100) includes an acquisition subsystem (113), which may include a video source (101), such as a digital camera, for creating, for example, an uncompressed video image stream (102). In one example, the video image stream (102) includes samples captured by a digital camera. The video image stream (102) is depicted as a thick line compared to the encoded video data (104) (or encoded video stream) to emphasize the high data volume of the video image stream, which may be processed by an electronic device (120) including a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to enable or implement aspects of the disclosed subject matter described in more detail below. The encoded video data (104) (or encoded video stream) is depicted as a thin line compared to the video image stream (102) to emphasize the low data volume of the encoded video data, which may be stored on a streaming media server (105) for future use. At least one streaming media client subsystem, such as Figure 1Client subsystems (106) and (108) can access a streaming media server (105) to retrieve copies (107) and (109) of encoded video data (104). Client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and creates an outgoing video image stream (111) that can be presented on a display (112) (e.g., a screen) or other display device (not depicted). In some streaming media systems, the encoded video data (104), (107), and (109) (e.g., a video stream) may be encoded according to certain video coding / compression standards. For example, those standards include the ITU-T H.265 Recommendation. In one example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed topics are applicable in the context of VVC.

[0037] It should be noted that electronic devices (120) and (130) may include other components (not shown). For example, electronic device (120) may also include a video decoder (not shown), while electronic device (130) may also include a video encoder (not shown).

[0038] Figure 2 An example block diagram of a video decoder (210) is shown. The video decoder (210) may be included in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., receiving circuitry). The video decoder (210) may replace... Figure 1 The video decoder (110) in the example is used.

[0039] The receiver (231) can receive, for example, at least one encoded video sequence included in the bitstream, which is decoded by the video decoder (210). In one aspect, one encoded video sequence is received at a time, and the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequences can be received from a channel (201), which can be a hardware / software link connected to a storage device storing the encoded video data. The receiver (231) can receive the encoded video data and other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective user entities (not depicted). The receiver (231) can separate the encoded video sequences from other data. To combat / mitigate network jitter, a buffer memory (215) can be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other applications, the buffer memory (215) may be external to the video decoder (210) (not depicted). In other applications, a buffer memory (not depicted) may exist external to the video decoder (210), for example, to combat / mitigate network jitter, and another buffer memory (215) may exist internally to the video decoder (210), for example, to handle playback timing. The buffer memory (215) may not be necessary or may be made smaller when the receiver (231) receives data from a store / forward device with sufficient bandwidth and controllability or from an isosynchronous network. For use on best-effort packet networks such as the Internet, a buffer memory (215) may be required. The buffer memory (215) may be relatively large and may advantageously have an adaptive size, and may be implemented at least partially in a similar component (not depicted) external to the operating system or the video decoder (210).

[0040] The video decoder (210) may include a parser (220) for reconstructing symbols (221) from the encoded video sequence. Categories of those symbols include information for managing the operation of the video decoder (210) and may include information for controlling a display device, such as a component of a non-electronic device (230) but which may be coupled to the electronic device (230) (e.g., a display screen). Figure 2As shown in the diagram. Control information for one or more display devices may be in the form of Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not depicted). The parser (220) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (220) may extract a subgroup parameter set from the encoded video sequence for at least one pixel subgroup / subgroup in the video decoder based on at least one parameter corresponding to a group. Subgroups may include Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (220) can also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0041] The parser (220) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) in order to create symbols (221).

[0042] Depending on the type of the encoded video image or its portions (e.g., inter-frame and intra-frame images, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (221) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (220). For clarity, the flow of such subgroup control information between the parser (220) and the multiple units described below is not depicted.

[0043] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In practical implementations under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0044] The first unit is the scaler / inverse transform unit (251). The scaler / inverse transform unit (251) receives from the parser (220) quantization transform coefficients as one or more symbols (221) and control information, including the transform to be used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (251) can output blocks containing sample values, which can be input into the aggregator (255).

[0045] In some cases, the output samples of the scaler / inverse transform unit (251) may relate to intra-coded blocks. Intra-coded blocks do not use predictive information from previously reconstructed images, but instead can use predictive information from previously reconstructed portions of the current image. Such predictive information can be provided by the intra-image prediction unit (252). In some cases, the intra-image prediction unit (252) uses surrounding reconstructed information obtained from the current image buffer (258) to generate blocks of the same size and shape as the blocks in the reconstruction. For example, the current image buffer (258) is used to buffer partially reconstructed current images and / or fully reconstructed current images. In some cases, the aggregator (255) adds the predictive information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a per-sample basis.

[0046] In other cases, the output samples of the scaler / inverse transform unit (251) may involve inter-frame coded and possibly motion-compensated blocks. In this case, the motion compensation prediction unit (253) can access the reference image memory (257) to obtain samples for prediction. After motion compensation of the obtained samples according to the block-related symbols (221), these samples can be added to the output of the scaler / inverse transform unit (251) (referred to in this case as residual samples or residual signals) by the aggregator (255) to generate output sample information. The address of the predicted samples obtained by the motion compensation prediction unit (253) in the reference image memory (257) can be controlled by motion vectors used by the motion compensation prediction unit (253) in the form of symbols (221), which may have, for example, X, Y, and reference image components. Motion compensation may also include interpolation of sample values ​​obtained from the reference image memory (257) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0047] The output samples of the aggregator (255) can be processed in the loop filter unit (256) using various loop filtering techniques. Video compression techniques may include loop filtering techniques controlled by parameters included in the encoded video sequence (also known as the encoded video stream) and available as symbols (221) from the parser (220) for use in the loop filter unit (256). Video compression may also respond to metadata obtained during the decoding of a previous (in decoding order) portion of the encoded image or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0048] The output of the loop filter unit (256) can be a sample stream, which can be output to the display device (212) and stored in the reference image memory (257) for future inter-frame image prediction.

[0049] Some encoded images can be used as reference images for future predictions after they have been fully reconstructed. For example, after the encoded image corresponding to the current image has been fully reconstructed and the encoded image has been identified as a reference image (by, for example, a parser (220)), the current image buffer (258) can become part of the reference image memory (257), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0050] The video decoder (210) can perform decoding operations according to a predetermined video compression technology or standard, such as ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax of the video compression technology or standard and the configuration file documented therein if the encoded video sequence follows both the syntax of the video compression technology or standard and the configuration file documented therein. Specifically, the configuration file may select certain tools from all available tools in the video compression technology or standard as the only tools available under that configuration file. For compliance, the complexity of the encoded video sequence must also be within the limits defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer indicated by signaling in the encoded video sequence.

[0051] In one aspect, the receiver (231) may receive supplemental (redundant) data along with the encoded video. This supplemental data may be part of one or more encoded video sequences. The supplemental data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The supplemental data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0052] Figure 3 An example block diagram of a video encoder (303) is shown. The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., transmission circuitry). The video encoder (303) may replace Figure 1 The video encoder (103) in the example is used.

[0053] The video encoder (303) can obtain data from the video source (301) (non-video source). Figure 3 In one example, an electronic device (320) receives video samples, and a video source (301) can capture one or more video images to be encoded by a video encoder (303). In another example, the video source (301) is part of the electronic device (320).

[0054] A video source (301) can provide a sequence of source video samples in the form of a digital video sample stream to be encoded by a video encoder (303). This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (301) can be a storage device storing previously prepared video. In a video conferencing system, the video source (301) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual images that, when viewed sequentially, produce an animated effect. The images themselves can be organized as spatial pixel arrays, where each pixel can include at least one sample, depending on the sampling structure, color space, etc., used. Samples are described in detail below.

[0055] According to one aspect, the video encoder (303) can encode and compress images of a source video sequence into an encoded video sequence (343) in real time or as needed, according to any other time constraints. Implementing an appropriate encoding rate is a function of the controller (350). In some aspects, the controller (350) controls and is functionally coupled to other functional units described below. For clarity, the coupling is not depicted. Parameters set by the controller (350) may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, GOP layout, maximum motion vector search range, etc. The controller (350) can be configured to have other suitable functions related to the video encoder (303) optimized for a specific system design.

[0056] In some respects, the video encoder (303) is configured to operate in an encoding loop. Simply put, in one example, the encoding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and one or more reference images), and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to that created by the (remote) decoder. The reconstructed sample stream (sample data) is input to a reference image memory (334). Since the decoding of the symbol stream produces a bit-by-bit accurate result unaffected by the decoder's location (local or remote), the contents of the reference image memory (334) are also bit-by-bit accurate between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction section are exactly the same as the sample values ​​"seen" by the decoder during decoding when using the prediction. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related fields.

[0057] The operation of the “local” decoder (333) can be the same as that of a “remote” decoder such as a video decoder (210), which has already been described above. Figure 2 A detailed description has been provided. However, a brief additional reference is also available. Figure 2 When symbols are available and the entropy encoder (345) and parser (220) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (210), including the buffer (215) and parser (220), may not be fully implemented in the local decoder (333).

[0058] In one respect, decoder techniques other than parsing / entropy decoding, which exist in the decoder, exist in the corresponding encoder with the same or substantially the same functional form. Therefore, the disclosed subject focuses on decoder operation. The description of encoder techniques can be simplified, as encoder techniques are the reverse operation of the comprehensively described decoder techniques. More detailed descriptions are only required in certain areas, and these descriptions are provided below.

[0059] In some examples, during operation, the source encoder (330) may perform motion-compensated predictive coding, i.e., predictively coding the input image against at least one previously encoded image designated as a “reference image” in the reference video sequence. In this way, the coding engine (332) encodes the differences between pixel blocks of the input image and pixel blocks of one or more reference images, which may be selected as one or more predictive references for the input image.

[0060] The local video decoder (333) can decode encoded video data of an image that can be designated as a reference image, based on symbols created by the source encoder (330). Advantageously, the operation of the encoding engine (332) can be a lossy process. When the encoded video data can be decoded by the video decoder (333), Figure 3 During decoding at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process, which can be performed by the video decoder on the reference image, and can store the reconstructed reference image in the reference image memory (334). In this way, the video encoder (303) can store a copy of the reconstructed reference image locally, which shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0061] The predictor (335) can perform a prediction search against the encoding engine (332). That is, for a new image to be encoded, the predictor (335) can search in the reference image memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as suitable prediction references for the new image. The predictor (335) can operate on a pixel-by-pixel basis based on sample blocks to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (335), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (334).

[0062] The controller (350) can manage the encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for video data encoding.

[0063] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder (345). The entropy encoder (345) converts these symbols into an encoded video sequence by applying lossless compression to the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, and arithmetic coding.

[0064] The transmitter (340) may buffer one or more encoded video sequences created by the entropy encoder (345) in preparation for transmission via a communication channel (360), which may be a hardware / software link connected to a storage device that will store the encoded video data. The transmitter (340) may combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0065] The controller (350) can manage the operation of the video encoder (303). During encoding, the controller (350) can assign a specific encoded image type to each encoded image, which may affect the encoding techniques applicable to the corresponding image. For example, one of the following image types can typically be assigned to an image.

[0066] Intraframe images (I-images) are images that are encoded and decoded without using any other images in the sequence as prediction sources. Some video codecs allow different types of intraframe images, including, for example, Independent Decoder Refresh (IDR) images.

[0067] Predictive images (P-images) can be images that are encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using motion vectors and reference indices.

[0068] A bidirectional predictive image (B-image) can be an image encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using two motion vectors and a reference index. Similarly, a multi-predictive image can reconstruct a single block using more than two reference images and associated metadata.

[0069] The source image can typically be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples each) and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignment applied to the corresponding images of the blocks. For example, blocks of an I-image can be non-predictively coded, or these blocks can be predictively coded with reference to already coded blocks of the same image (spatial prediction or intra-frame prediction). Pixel blocks of a P-image can be predictively coded via spatial prediction or via temporal prediction with reference to a previously coded reference image. Blocks of a B-image can be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference images.

[0070] The video encoder (303) can perform encoding operations according to predetermined video coding techniques or standards, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0071] In one respect, the transmitter (340) may transmit additional data along with the encoded video. The source encoder (330) may transmit such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0072] Video can be captured as multiple source images (video images) in a time-series manner. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In one example, the specific image being encoded / decoded is called the current image, which is divided into blocks. Where a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image and, in the case of multiple reference images, may have a third dimension identifying the reference images.

[0073] In some aspects, inter-frame image prediction can utilize bidirectional prediction techniques. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly representing past and future images in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. A block can be predicted using a combination of the first and second reference blocks.

[0074] Furthermore, merging mode techniques can be used to improve coding efficiency in inter-frame image prediction.

[0075] According to some aspects of the invention, prediction is performed on a block-by-block basis, such as inter-frame image prediction and intra-frame image prediction. For example, according to the HEVC standard, images in a video image sequence are divided into coding tree units (CTUs) for compression. A single CTU in an image has the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU comprises three coding tree blocks (CTBs): one luma coding tree block and two chroma coding tree blocks. Each CTU can be recursively quadtree-based into one or more coding units. For example, a 64×64 pixel CTU can be split into one 64×64 pixel coding unit, or four 32×32 pixel coding units, or sixteen 16×16 pixel coding units. In one example, each coding unit is analyzed to determine its prediction type, such as inter-frame prediction or intra-frame prediction. Coding units are split into at least one prediction unit based on temporal and / or spatial predictability. Typically, each prediction unit comprises one luma prediction block (PB) and two chroma prediction blocks. In one aspect, prediction operations in encoding / decoding are performed on a block-by-block basis. Using the luma prediction block as an example, a prediction block comprises a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0076] It should be noted that the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using any suitable technology. In one aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using at least one integrated circuit (IC). In another aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using at least one processor that executes software instructions.

[0077] Various aspects of this invention provide techniques (e.g., methods, embodiments, encoders, decoders) for chained vector-based video coding, such as chained motion vector refinement, chained vector-based predictive filtering, etc. The techniques of this invention can be used individually or in any combination in any order. Furthermore, each technique can be implemented by processing circuitry (e.g., at least one processor or at least one integrated circuit). In one example, at least one processor executes a program stored in a non-transitory computer-readable storage medium. In some examples, the prediction of the current block is based on a chained vector comprising at least a first sub-vector and a second sub-vector, the first sub-vector pointing relative to the current block to a first reference block, and the second sub-vector pointing relative to the first reference block to a second reference block. The encoder / decoder can determine at least one vector offset for refining the chained vector, at least based on the first and second reference blocks; and reconstruct the current block based on the chained vector and the vector offset. In some examples, the encoder / decoder can determine a predictive filter based on the first and second reference blocks; and reconstruct the current block based on the chained vector and the predictive filter.

[0078] Video encoding and decoding are widely used in many applications. Video codecs can employ video encoding and decoding standards such as H.264, H.265, H.266 (VVC), AV1, and AVS.

[0079] Video codecs typically comprise multiple modules, including intra / inter-frame prediction, transform coding, quantization, entropy coding, and loop filtering. Some aspects of this invention provide a set of video compression methods for inter-frame predictive coding. In inter-frame predictive coding, motion vectors can be refined. In some examples, optical flow is used to further refine the bidirectional prediction blocks. The technique of refining bidirectional prediction blocks using optical flow is called bidirectional optical flow (BDOF). BDOF is used to refine the motion vectors or predicted signals constructed through bidirectional prediction.

[0080] In some examples, these techniques can be used in scenarios involving chained vectors, such as chained motion vectors, chained block vectors, etc. A chained motion vector refers to a chain of motion vectors comprising at least two motion vectors in a certain order, such as a first motion vector and a second motion vector in a certain order. A chained block vector refers to a chain of block vectors comprising at least two block vectors in a certain order, such as a first block vector and a second block vector in a certain order.

[0081] Figure 4 A diagram showing chain motion vectors in some examples is provided. Figure 4 In the example, the current block (411) in the current image (410) of the video has a first reference block A1 in the first reference image (420), which is composed of a first motion vector based on the current block (411). ) points to; and the first reference block A1 in the first reference image (420) has a second reference block A2 in the second reference image (430), the second reference block A2 being pointed to by a second motion vector based on the first reference block A1 ( ) points to. In Figure 4 In the example, the first motion vector ( ) and second motion vector ( The combination of ) forms a chain of motion vectors, and the first motion vector ( ) and second motion vector ( This is referred to as a sub-motion vector of the chained motion vector. In some examples, the second reference block A2 is used as the prediction block for the current block based on the chained motion vector. In some examples, the current block is predicted based on a combination of the first reference block A1 and the second reference block A2 based on the chained motion vector.

[0082] exist Figure 4 In the example, the first reference image (420) and the second reference image (430) are both located on the same side of the current image (410) in the display order, and the sub-motion vectors of the chain motion vector point in the same direction in the display order; therefore, the chain motion vector is called a unidirectional chain motion vector. For example, the current image (410), the first reference image (420), and the second reference image (430) are displayed in either the order of the current image or in reverse order. In one example, the first reference image (420) is displayed before the current image (410), and the second reference image (430) is displayed before the first reference image (420). In another example, the first reference image (420) is displayed after the current image (410), and the second reference image (430) is displayed after the first reference image (420).

[0083] It should be noted that a chain motion vector can be a unidirectional chain motion vector (e.g., such as...). Figure 4 (As shown), it can also be a bidirectional chain motion vector.

[0084] Figure 5 A diagram showing bidirectional chain motion vectors is provided for some examples. Figure 5 In the example, the current block (511) in the current image (510) of the video has a first reference block A1 in the first reference image (520), which is composed of a first motion vector based on the current block (511). ) points to; and the first reference block A1 in the first reference image (520) has a second reference block A2 in the second reference image (530), the second reference block A2 being pointed to by a second motion vector based on the first reference block A1 ( ) points to. In Figure 5 In the example, the first motion vector ( ) and second motion vector ( The combination of ) forms a chain of motion vectors, and the first motion vector ( ) and second motion vector ( This is referred to as a sub-motion vector of the chained motion vector. In some examples, the second reference block A2 is used as the prediction block for the current block based on the chained motion vector. In some examples, the current block is predicted based on a combination of the first reference block A1 and the second reference block A2 based on the chained motion vector.

[0085] exist Figure 5 In the example, the first reference image (520) and the second reference image (530) are located on different sides of the current image (510) in the display order, and the sub-motion vectors of the chain motion vector point in different directions in the display order; therefore, the chain motion vector is called a bidirectional chain motion vector. Figure 5 In the example, the first reference image (520) is displayed before the current image (510), and the second reference image (530) is displayed after the current image (510). Figure 5 The chain motion vector in the example is called a bidirectional chain motion vector.

[0086] Figure 6 A diagram showing some examples of chained block vectors is provided. Figure 6 In the example, the current block (611) in the current image (610) of the video has a first reference block A1 in the current image (610), which is composed of a first block vector based on the current block (611). ) points to; and the first reference block A1 in the current image (610) has a second reference block A2 in the current image (610), the second reference block A2 being determined by a second block vector based on the first reference block A1 ( ) points to. In Figure 6 In the example, the first block vector ( ) and the second vector ( The combination of ) forms a chain of block vectors, and the first block vector ( ) and the second block vector ( This is called a sub-block vector of the chained block vector. In some examples, the second reference block A2 is used as the predicted block for the current block, based on the chained block vector. In some examples, the current block is predicted based on a combination of the first reference block A1 and the second reference block A2, based on the chained block vector.

[0087] In this invention, although some examples are described based on chain motion vectors, the techniques used in these examples can be appropriately modified for chain block vectors.

[0088] According to one aspect of the invention, when using a chain motion vector, the current block is similar to both the first reference block and the second reference block, and the current block can be refined using bilateral matching with the first and second reference blocks of the chain motion vector.

[0089] According to some aspects of the invention, the chain motion vector of the current block is refined using motion vector refinement (e.g., unidirectional motion vector refinement, bidirectional motion vector refinement, etc.). When the motion vector of the current block is a chain motion vector, bilateral matching is performed using reference blocks (e.g., A1 and A2) from the sub-motion vectors (pointed to by the sub-motion vectors) of the chain motion vector.

[0090] In some examples, for one-way bilateral matching, a block search process is performed on, for example, two reference blocks using motion (motion offset) in the same direction. The motion offset can be searched within the search range to find the optimal cost motion vector offset (mv) between the two reference blocks. off Final motion vector (mv) c ') is achieved by offsetting the motion vector (mv) off ) and chain motion vector (mv c It is constructed by adding together )

[0091] In some respects, the current block may have at least one chained motion vector. In one example, the current block has one chained motion vector. In another example, the current block is a bidirectional prediction block and has a first chained motion vector in a first direction of the display order and a second chained motion vector in a second direction of the display order.

[0092] In some respects, a chained motion vector includes more than one motion vector as a sub-motion vector. In some examples, a chained motion vector has two sub-motion vectors. In some examples, a chained motion vector has more than two sub-motion vectors.

[0093] In some examples, when the sub-motion vector points in one direction, a block search process can be performed on two reference blocks using unidirectional motion to find the optimal matching block by utilizing bilateral matching between the reference blocks. A motion vector offset corresponding to the optimal matching block is derived. This motion vector offset is then added to the chained motion vector.

[0094] Figure 7 A diagram showing the refined motion vector of a unidirectional chain motion vector according to an embodiment of the present invention is provided.

[0095] exist Figure 7 In the example, the current block (711) in the current image (710) of the video has a first reference block A1 in the first reference image (720), which is composed of a first motion vector based on the current block (711). ) points to; and the first reference block A1 in the first reference image (720) has a second reference block A2 in the second reference image (730), the second reference block A2 being pointed to by a second motion vector based on the first reference block A1 ( ) points to. In Figure 7 In the example, the first motion vector ( ) and second motion vector ( The combination of these elements forms a chain of motion vectors. Figure 7 In the example, the sub-motion vectors (mv1 and mv2) point in the same direction, so this chain motion vector is called a unidirectional chain motion vector.

[0096] In addition, Figure 7 In the example, the search range is defined around the first reference block A1 and the second reference block A2, and a block search process is performed on the first reference block A1 and the second reference block A2 using unidirectional motion (candidate motion offset). In one example, motion offsets can be searched within the search range to find the motion vector offset (mv) with the optimal cost between the two reference blocks A1 and A2. off In one example, for a candidate motion offset, two candidate refined reference blocks A1' and A2' can be identified, and the cost associated with the candidate motion offset, such as the sum of absolute differences (SAD), is calculated based on the sample difference between these two candidate refined reference blocks A1' and A2'. Among the cost associated with the candidate motion offset, the lowest cost can be found, and the candidate motion offset associated with this lowest cost can be determined as the motion vector offset (mv) with the optimal cost. off Final motion vector (mv) c ') is achieved by offsetting the motion vector (mv) off ) and chain motion vector (mvc It is constructed by adding together )

[0097] like Figure 7 As shown in the example, when the sub-motion vectors (mv1 and mv2) point in one direction, when using a motion vector offset (mv... off During the block search process, a bilateral block match is performed between reference blocks (A1 and A2) to find the best matching block. This is achieved by offsetting a motion vector (mv). off ) and chain motion vector (mv c The refined motion vector (mv) is derived by adding the two vectors together. c ').

[0098] In some examples, when the sub-motion vector points bidirectionally, the block search process can be performed using mirrored direction motion on both reference blocks to find the best-matching block using bilateral matching between the reference blocks. A motion vector offset corresponding to the best-matching block is derived. This motion vector offset is then added to the chained motion vector.

[0099] Figure 8 A diagram illustrating the refined motion vector of a bidirectional chain motion vector according to an embodiment of the present invention is shown. Figure 8 In the example, the current block (811) in the current image (810) of the video has a first reference block A1 in the first reference image (820), which is composed of a first motion vector based on the current block (811). ) points to; and the first reference block A1 in the first reference image (820) has a second reference block A2 in the second reference image (830), the second reference block A2 being pointed to by a second motion vector based on the first reference block A1 ( ) points to. In Figure 8 In the example, the first motion vector ( ) and second motion vector ( The combination of these elements forms a chain motion vector.

[0100] exist Figure 8 In the example, the first reference image (820) and the second reference image (830) are located on different sides of the current image (810) in the display order, and the sub-motion vectors of the chain motion vector point in different directions in the display order; therefore, the chain motion vector is called a bidirectional chain motion vector. Figure 8 In the example, the first reference image (820) is displayed before the current image (810), and the second reference image (830) is displayed after the current image (810). Figure 8 The chain motion vector in the example is called a bidirectional chain motion vector.

[0101] In addition, Figure 8 In the example, the search range is defined around a first reference block A1 and a second reference block A2, and a block search process is performed on the first reference block A1 and the second reference block A2 using mirror-direction motion (e.g., candidate motion offsets) (e.g., when applying a candidate motion offset to the first reference block A1 and applying the opposite value of that candidate motion offset to the second reference block A2). In one example, motion offsets can be searched within the search range to find the motion vector offset (mv) with the optimal cost between the two reference blocks A1 and A2. off In one example, for a candidate motion offset, two candidate refined reference blocks A1' (e.g., the candidate motion offset is applied to obtain candidate refined reference block A1') and A2' (e.g., the inverse of the candidate motion offset is applied to obtain candidate refined reference block A2') can be identified. Based on the sample difference between these two candidate refined reference blocks A1' and A2', the cost associated with the candidate motion offset, such as SAD, is calculated. Among the cost values ​​associated with the candidate motion offsets, the lowest cost value can be found, and the candidate motion offset associated with the lowest cost value can be determined as the motion vector offset (mv) with the optimal cost. off Final motion vector (mv) c ') is achieved by offsetting the motion vector by the inverse value (-mv) off ) and chain motion vector (mv c It is constructed by adding together )

[0102] exist Figure 8 In the example, the sub-motion vectors (mv1 and mv2) point in both directions, using a motion vector offset (mv off During the block search process, bilateral block matching is performed between reference blocks (A1 and A2) to find the best matching block, where the motion vector offset (mv) is... off ) is used for the 'A1' reference block, while the inverse value of this motion vector offset (-mv) off ) is used for the 'A2' reference block. This is achieved by offsetting a motion vector (-mv) off ) and chain motion vector (mv c The refined motion vector (mv) is derived by adding the two vectors together. c ').

[0103] In some aspects, the current block is predicted based on bidirectional motion vectors, where the motion vectors point to reference images in two directions, and at least one of these motion vectors is a unidirectional chain motion vector. A motion vector offset corresponding to each unidirectional chain motion vector can be applied.

[0104] Figure 9A diagram illustrating the refined motion vector of a bidirectional motion vector including a chain motion vector according to an embodiment of the present invention is shown.

[0105] exist Figure 9 In the example, the current block (911) in the current image (910) of the video is predicted based on a bidirectional motion vector consisting of two chained motion vectors. These two chained motion vectors are unidirectional chained motion vectors, but they point in different directions in the display order.

[0106] Specifically, the current block (911) has a first chain motion vector. Second chain motion vector .

[0107] For the first chain motion vector The current block (911) has a first reference block A1 in the first reference image (920), which is composed of a first motion vector based on the current block (911). ) points to; and the first reference block A1 in the first reference image (920) has a second reference block A2 in the second reference image (930), the second reference block A2 being pointed to by a second motion vector based on the first reference block A1 ( ) points to.

[0108] For the second chain motion vector The current block (911) has a third reference block B1 in the third reference image (940), which is composed of a third motion vector based on the current block (911). ) points to; and the third reference block B1 in the third reference image (940) has a fourth reference block B2 in the fourth reference image (950), the fourth reference block B2 being determined by a fourth motion vector based on the third reference block B1 ( ) points to.

[0109] exist Figure 9 In the example, for each directed chain motion vector, unidirectional motion vector refinement can be applied. Motion vector offset (mv) off a and MV off b The values ​​are derived by performing double-sided block matching on the reference block corresponding to each directed chain motion vector. The derived motion vector offsets are then added to each chain motion vector.

[0110] Specifically, for the first chain motion vector ,exist Figure 9In the example, the search range is defined around the first reference block A1 and the second reference block A2, and the block search process is performed with motion (candidate motion offset) in the same direction on the first reference block A1 and the second reference block A2. In one example, motion offsets can be searched within the search range to find the motion vector offset (mv) with the optimal cost between the two reference blocks A1 and A2. off a In one example, for a candidate motion offset, two candidate refined reference blocks A1' and A2' can be identified, and the cost associated with the candidate motion offset, such as SAD, is calculated based on the sample difference between these two candidate refined reference blocks A1' and A2'. Among the cost values ​​associated with the candidate motion offsets, the lowest cost value can be found, and the candidate motion offset associated with this lowest cost value can be determined as the motion vector offset (mv) with the optimal cost. off a The final first motion vector (mv) c a ') is achieved by offsetting the motion vector (mv) off a ) and the first chain motion vector (mv c a It is constructed by adding together )

[0111] For the second chain motion vector ,exist Figure 9 In the example, the search range is defined around the third reference block B1 and the fourth reference block B2, and the block search process is performed with motion (candidate motion offset) in the same direction on the third reference block B1 and the fourth reference block B2. In one example, motion offsets can be searched within the search range to find the motion vector offset (mv) with the optimal cost between the two reference blocks B1 and B2. off b In one example, for a candidate motion offset, two candidate refined reference blocks B1' and B2' can be identified, and the cost associated with the candidate motion offset, such as SAD, is calculated based on the sample difference between these two candidate refined reference blocks B1' and B2'. Among the cost values ​​associated with the candidate motion offsets, the lowest cost value can be found, and the candidate motion offset associated with this lowest cost value can be determined as the motion vector offset (mv) with the optimal cost. off b The final second motion vector (mv) c b ') is achieved by offsetting the motion vector (mv) off b ) and the second chain motion vector (mv c b It is constructed by adding together )

[0112] In some respects, when the current block has a chain of bidirectional motion vectors and at least one of the motion vectors is a chain of motion vectors, an offset of a motion vector with the same magnitude but a different direction can be applied to the bidirectional chain of motion vectors.

[0113] Figure 10 A diagram illustrating the refined motion vector of a bidirectional motion vector including a chain motion vector according to an embodiment of the present invention is shown.

[0114] exist Figure 10 In the example, the current block (1011) in the current image (1010) of the video is predicted based on a bidirectional motion vector consisting of two chained motion vectors. These two chained motion vectors are unidirectional chained motion vectors, but they point in different directions in the display order.

[0115] Specifically, the current block (1011) has a first chain motion vector. Second chain motion vector .

[0116] For the first chain motion vector The current block (1011) has a first reference block A1 in the first reference image (1020), which is composed of a first motion vector based on the current block (1011). ) points to; and the first reference block A1 in the first reference image (1020) has a second reference block A2 in the second reference image (1030), the second reference block A2 being pointed to by a second motion vector based on the first reference block A1 ( ) points to.

[0117] For the second chain motion vector The current block (1011) has a third reference block B1 in the third reference image (1040), which is composed of a third motion vector based on the current block (1011). ) points to; and the third reference block B1 in the third reference image (1040) has a fourth reference block B2 in the fourth reference image (1050), which is determined by a fourth motion vector based on the third reference block B1 ( ) points to.

[0118] exist Figure 10 In the example, for two directed chain motion vectors, a motion vector offset (mv) is derived by performing bilateral block matching using complete reference blocks (including reference blocks A1, A2, B1, and B2) of all sub-motion vectors of the chain motion vectors. off To find the optimal cost block, the 'A' reference block uses a motion vector offset (mv). offThe 'B' reference block uses the opposite motion vector offset (-mv), while the 'B' reference block uses the opposite motion vector offset (-mv). off ).

[0119] Specifically, in Figure 10 In the example, the search range is defined around the first reference block A1, the second reference block A2, the third reference block B1, and the fourth reference block B2. A block search process is performed by applying motion (candidate motion offset) in a first direction on the first and second reference blocks A1 and A2, and motion in a second direction on the third and fourth reference blocks B1 and B2. The first and second directions are opposite to each other. In one example, motion offsets can be searched within the search range to find the motion vector offset (mv) with the optimal cost for reference blocks A1, A2, B1, and B2. off In one example, for a candidate motion offset, four candidate refined reference blocks A1', A2', B1', and B2' can be identified, and the cost associated with the candidate motion offset is calculated based on these four candidate refined reference blocks A1', A2', B1', and B2'. In one example, the cost is the sum of the first sub-value and the second sub-value. The first sub-value is calculated based on the sample difference between the two candidate refined reference blocks A1' and A2', and the second sub-value is calculated based on the sample difference between the two candidate refined reference blocks B1' and B2'. Among the cost values ​​associated with the candidate motion offset, the lowest cost value can be found, and the candidate motion offset associated with the lowest cost value can be determined as the motion vector offset (mv) with the optimal cost. off Final motion vector (mv) c a 'and MV c b ') can be achieved by offsetting the motion vector (mv) off ) and the first chain motion vector (mv c a Add them together and offset the opposite motion vector (-mv) off ) and the second chain motion vector (mv c b They are added together to construct the structure.

[0120] In one aspect, chain motion refinement can be applied based on block shape, block size, and other encoded information. For example, when the block shape of the current block meets the requirements, the refinement of the chain motion vector can be applied to the current block. In another example, when the block size of the current block meets the requirements, the refinement of the chain motion vector can be applied to the current block.

[0121] In one aspect, it can be determined whether to apply chain motion vector refinement when the difference between reference blocks is less than a specified threshold. In one example, chain motion vector refinement (refinement of the chain motion vector) is not applied when the SAD value between the two reference blocks (pointed to by the first and second sub-motion vectors within the chain motion vector) is less than a specified threshold.

[0122] Some aspects of the invention also provide techniques for deriving prediction filters for a current block used for prediction using a chained vector. The chained vector has multiple reference blocks. The chained vector can be a chained motion vector comprising multiple sub-motion vectors, or a chained block vector comprising multiple sub-block vectors. In some examples, prediction filters are derived from at least two prediction blocks associated with the chained vector, and the derived prediction filters are applied to reference blocks used to generate the prediction block for the current block.

[0123] It is important to note that when using a chained vector in the current block, such as... Figure 4 , Figure 5 and Figure 6 As shown, there are at least two reference blocks. In some aspects, a prediction filter can be determined from the filter set based on the cost calculated between the reference blocks. The prediction filter with the optimal cost is applied to a reference block, which is then used as the prediction block for the current block.

[0124] In some examples, the filter set may include at least one prediction filter.

[0125] In some examples, the filter set may include at least one linear filter. In some examples, the filter set may include at least one nonlinear filter. In some examples, the filter set may include predefined filters, such as sharpening filters, smoothing filters, bilateral filters, etc.

[0126] In some examples, the filter set may include regression-based filters that are derived from the reference block using regression techniques.

[0127] According to some aspects of the invention, a predictive filter can be determined based on the cost calculated between an unfiltered / filtered reference block and an unfiltered reference block.

[0128] Figure 11 A technical diagram of a prediction filter for determining a chain motion vector is shown according to an embodiment of the present invention.

[0129] exist Figure 11 In the example, the current block (1111) in the current image (1110) is based on including the first sub-motion vector ( ) and second sub-motion vector ( The chain motion vectors of the current block (1111) in the current image (1110) are predicted. The current block (1111) in the current image (1110) has a first reference block A1 in the first reference image (1120), which is predicted by the first sub-motion vector of the current block (1111). ) points to; and the first reference block A1 in the first reference image (1120) has a second reference block A2 in the second reference image (1130), the second reference block A2 being determined by a second sub-motion vector based on the first reference block A1 ( ) points to.

[0130] In some examples, the prediction filter is selected from the filter set based on the cost associated with the filter in the filter set. In some examples, the cost is calculated between an unfiltered / filtered second reference block and an unfiltered first reference block.

[0131] For example, such as Figure 11 As shown, the filter set includes a first filter (e.g., a sharpening filter) denoted by f1 and a second filter (e.g., a smoothing filter) denoted by f2. In one example, an unfiltered cost, such as the SAD between sample values ​​of the unfiltered second reference block A2 and the unfiltered first reference block A1, is calculated between them. This unfiltered cost is denoted as C. unfiltered .

[0132] Furthermore, the filtered cost is calculated between the filtered reference block A2, represented by f(A2), and the unfiltered reference block A1. For example, a first filtered cost (denoted by C) is calculated between the second reference block filtered by the first filter f1(A2) and the unfiltered first reference block A1. f1 (represented by), for example, the SAD between the sample values ​​of the filtered second reference block f1(A2) and the sample values ​​of the unfiltered first reference block A1; calculating the second filtered cost (by C) between the second reference block filtered by the second filter f2(A2) and the unfiltered first reference block A1. f2 (represented by) for example, the SAD between the sample values ​​of the filtered second reference block f2 (A2) and the sample values ​​of the unfiltered first reference block A1.

[0133] In addition, Figure 11 In the example, the unfiltered cost is compared with two filtered costs, and the optimal cost (lowest cost) is determined. When the optimal cost comes from the unfiltered cost, no prediction filter is needed. When the optimal cost comes from the first unfiltered cost C... f1 When the optimal cost comes from the second filtered cost C, the first filter is determined as the prediction filter for the current block. f2At this point, the second filter is determined as the prediction filter for the current block. In some examples, a second reference block is used to generate the prediction block for the current block, and the prediction filter is applied to the second reference block. In one example, the filtered second reference block is used for further motion compensation.

[0134] In some examples, the current block has bidirectional chained motion vectors, where at least one motion vector is a chained motion vector, and a prediction filter can be determined for each motion vector separately.

[0135] In one example, when the motion vector of the current block is a bidirectional chained motion vector, the bidirectional chained motion vector includes two chained motion vectors, and two prediction filters can be determined for each of these two chained motion vectors.

[0136] Figure 12 A technical diagram of a predictive filter for determining a bidirectional chain motion vector is shown according to an embodiment of the present invention.

[0137] exist Figure 12 In the example, the current block (1211) in the current image (1210) of the video is predicted based on a bidirectional motion vector that includes two chained motion vectors. These two chained motion vectors are unidirectional chained motion vectors, but they point in different directions in the display order.

[0138] Specifically, the current block (1211) has a first chain motion vector. Second chain motion vector .

[0139] For the first chain motion vector The current block (1211) has a first reference block A1 in the first reference image (1220), which is composed of a first motion vector based on the current block (1211). ) points to; and the first reference block A1 in the first reference image (1220) has a second reference block A2 in the second reference image (1230), the second reference block A2 being determined by a second motion vector based on the first reference block A1 ( ) points to.

[0140] For the second chain motion vector The current block (1211) has a third reference block B1 in the third reference image (1240), which is composed of a third motion vector based on the current block (1211). ) points to; and the third reference block B1 in the third reference image (1240) has a fourth reference block B2 in the fourth reference image (1250), which is determined by a fourth motion vector based on the third reference block B1 ( ) points to.

[0141] exist Figure 12 In the example, a prediction filter is determined for each unidirectional chain motion vector.

[0142] Specifically, for the first chain motion vector ,exist Figure 12 In the example, the filter set includes a first filter f1 and a second filter f2. In one example, the unfiltered cost C is calculated based on the unfiltered second reference block A2 and the unfiltered first reference block A1. a unfiltered .

[0143] Furthermore, the relationship between the second reference block (A2) filtered by the first filter f1 and the unfiltered first reference block A1 is calculated using C. a f1 The first filtered cost is represented; the second filtered cost C between the second reference block (A2) filtered by the second filter f2 and the unfiltered first reference block A1 is calculated. a f2 .

[0144] exist Figure 12 In the example, the first filtered cost C a f1 Lower than the unfiltered cost C a unfiltered The second filtered cost C a f2 The first filter is selected as the first chain motion vector. The prediction filter. In some examples, based on the first chain motion vector. The second reference block is used to generate the prediction block for the current block, and the first filter is applied to the second reference block. In one example, the filtered second reference block is used for further motion compensation.

[0145] In addition, Figure 12 In the example, for the second chain motion vector The filter set includes a first filter f1 and a second filter f2. In one example, the unfiltered cost C is calculated based on the unfiltered fourth reference block B2 and the unfiltered third reference block B1. b unfiltered .

[0146] Furthermore, the relationship between the fourth reference block (B2) filtered by the first filter f1 and the unfiltered third reference block B1 is calculated using C. b f1The value of the third filtered block is represented; the value of the fourth filtered block (B2) after being filtered by the second filter f2 and the unfiltered third reference block B1 is calculated as C. b f2 .

[0147] exist Figure 12 In the example, the fourth filtered cost C b f2 Lower than the unfiltered cost C b unfiltered And the third filtered cost C b f1 The second filter is selected as the second chain motion vector. The prediction filter. In some examples, based on the second chain motion vector. The fourth reference block is used to generate the prediction block for the current block, and the second filter is applied to the fourth reference block. In one example, the filtered fourth reference block is used for further motion compensation.

[0148] In some respects, when a chain vector has two motion vectors, a predictive filter can be derived for these two chain motion vectors.

[0149] In one example, when the motion vector of the current block is a bidirectional chained motion vector, the bidirectional chained motion vector includes two chained motion vectors, and a prediction filter can be determined for these two chained motion vectors.

[0150] Figure 13 A technical diagram of a predictive filter for determining a bidirectional chain motion vector is shown according to an embodiment of the present invention.

[0151] exist Figure 13 In the example, the current block (1311) in the current image (1310) of the video is predicted based on a bidirectional motion vector that includes two chained motion vectors. These two chained motion vectors are unidirectional chained motion vectors, but they point in different directions in the display order.

[0152] Specifically, the current block (1311) has a first chain motion vector. Second chain motion vector .

[0153] For the first chain motion vector The current block (1311) has a first reference block A1 in the first reference image (1320), which is composed of a first motion vector based on the current block (1311). ) points to; and the first reference block A1 in the first reference image (1320) has a second reference block A2 in the second reference image (1330), the second reference block A2 being determined by a second motion vector based on the first reference block A1 ( ) points to.

[0154] For the second chain motion vector The current block (1311) has a third reference block B1 in the third reference image (1340), which is composed of a third motion vector based on the current block (1311). ) points to; and the third reference block B1 in the third reference image (1340) has a fourth reference block B2 in the fourth reference image (1350), which is determined by a fourth motion vector based on the third reference block B1 ( ) points to.

[0155] exist Figure 13 In the example, for a bidirectional chain motion vector that includes two unidirectional chain motion vectors, a prediction filter is determined for the two unidirectional chain motion vectors.

[0156] like Figure 13 As shown, the filter set includes a first filter (e.g., a sharpening filter) f1 and a second filter (e.g., a smoothing filter) f2. The unfiltered cost is calculated between unfiltered reference blocks. The filtered cost is calculated between two reference blocks (f(A2), f(B2)) filtered by the filter set, where the same filters are applied to both reference blocks (A2, B2) and the unfiltered reference blocks (A1, B1).

[0157] Specifically, in one example, the unfiltered cost C is calculated based on the unfiltered first reference block A1, the unfiltered second reference block A2, the unfiltered third reference block B1, and the unfiltered fourth reference block B2. unfiltered For example, the unfiltered cost C unfiltered It is the sum of the values ​​of the first unfiltered offspring and the second unfiltered offspring. For example, the value of the first unfiltered offspring is calculated as the SAD of the sample difference between the unfiltered second reference block A2 and the unfiltered first reference block A1; the value of the second unfiltered offspring is calculated as the SAD of the sample difference between the unfiltered fourth reference block B2 and the unfiltered third reference block A1.

[0158] In addition, the two filtered costs associated with the first and second filters in the filter set are calculated.

[0159] In one example, the first filtered cost C associated with the first filter. f1It is calculated based on the unfiltered first reference block A1, the second reference block f1(A2) filtered by the first filter, the unfiltered third reference block B1, and the fourth reference block f1(B2) filtered by the first filter. For example, the first filtered value is C. f1 It is the sum of the values ​​of the first filtered child and the second filtered child. For example, the value of the first filtered child is calculated as the SAD of the sample difference between the filtered second reference block f1 (A2) and the unfiltered first reference block A1; the value of the second unfiltered child is calculated as the SAD of the sample difference between the filtered fourth reference block f1 (B2) and the unfiltered third reference block B1.

[0160] Furthermore, in one example, the second filtered cost C associated with the second filter. f2 It is calculated based on the unfiltered first reference block A1, the second reference block f2(A2) filtered by the second filter, the unfiltered third reference block B1, and the fourth reference block f2(B2) filtered by the second filter. For example, the second filtered value is C. f2 It is the sum of the values ​​of the third filtered child and the fourth filtered child. For example, the value of the third filtered child is calculated as the SAD of the sample difference between the filtered second reference block f2 (A2) and the unfiltered first reference block A1; the value of the fourth unfiltered child is calculated as the SAD of the sample difference between the filtered fourth reference block f2 (B2) and the unfiltered third reference block B1.

[0161] exist Figure 13 In the example, the unfiltered cost C unfiltered and two filtered costs C f1 and C f2 Compare and determine the optimal cost (lowest cost). When the optimal cost comes from the unfiltered cost C... unfiltered At this time, a predictive filter is not needed. When the optimal cost comes from the first unfiltered cost C... f1 At that time, the first filter is determined as the prediction filter for these two chain motion vectors. When the optimal cost comes from the second filtered cost C... f2 At that time, the second filter is determined as the prediction filter for these two chain motion vectors. In some examples, the second reference block A2 and the fourth reference block B2 are used to generate the prediction block for the current block, and the prediction filter is applied to the second reference block A2 and the fourth reference block B2.

[0162] It is important to note that when using a chained vector in the current block, such as... Figure 4 , Figure 5 and Figure 6 As shown, there are at least two reference blocks. The prediction filter (model) can be derived using the reference blocks, and the prediction filter can be applied to the reference blocks.

[0163] In some examples, the prediction filter is a linear or nonlinear filter, and it is derived using a linear regression-based method to derive the linear or nonlinear equations from two reference blocks, such as a first reference block A1 and a second reference block A2. The derived prediction filter can then be applied to the reference blocks used to generate the prediction block for the current block.

[0164] Figure 14 A technical diagram of a prediction filter for determining a chain motion vector according to an embodiment of the present invention is shown.

[0165] exist Figure 14 In the example, the current block (1411) in the current image (1410) is based on including the first sub-motion vector ( ) and second sub-motion vector ( The chain motion vectors of the current block (1411) in the current image (1410) are predicted. The current block (1411) in the current image (1410) has a first reference block A1 in the first reference image (1420), which is predicted by the first sub-motion vector of the current block (1411). ) points to; and the first reference block A1 in the first reference image (1420) has a second reference block A2 in the second reference image (1430), the second reference block A2 being determined by a second sub-motion vector based on the first reference block A1 ( ) points to.

[0166] In one example, the prediction filter f(x) is determined by... The linear filter (also called a linear model or linear function) is represented by, where and These are the filter coefficients. The filter coefficients can be derived using a linear regression-based method, based on a first reference block A1 and a second reference block A2. For example, a linear filter (also called a linear model or linear function) with the filter coefficients to be determined is applied to the second reference block to obtain a filtered second reference block. The difference (e.g., SAD) between the filtered second reference block and the first reference block is calculated, and the filter coefficients are determined using a linear regression-based method to minimize this difference. Then, the derived prediction filter f(x) with the determined filter coefficients can be applied to the second reference block, which is used to generate the prediction block for the current block.

[0167] In some respects, the derived prediction filter can be scaled based on the distance between the reference image and the current image.

[0168] exist Figure 14 In the example, the distance between the first reference block A1 and the second reference block A2 is represented as The distance between the second reference block A2 and the current block is expressed as In one example, it can be based on the distance ratio (e.g., For prediction filters Scaling is performed to derive the scaled filter. In another example, the prediction filter offset No scaling is applied, and only the scaling factor is applied. Based on the distance ratio (e.g., The filter is scaled, and the scaled filter is... express.

[0169] In some examples, when the motion vector of the current block is a bidirectional chained motion vector, the bidirectional chained motion vector includes two chained motion vectors, and two prediction filters can be derived for each of these two chained motion vectors.

[0170] In some examples, when the motion vector of the current block is a bidirectional chained motion vector, the bidirectional chained motion vector includes two chained motion vectors, and a prediction filter can be derived for these two chained motion vectors.

[0171] In some respects, the prediction filter can be applied before or after motion compensation. In one example, after a prediction block is generated based on a reference block, a prediction filter is applied to the prediction block, and then further motion compensation is applied. In another example, after a prediction block is generated based on a reference block, specific motion compensation is applied to the prediction block, and then the prediction filter is applied after that motion compensation. Motion compensation includes, but is not limited to, DMVR, BDOF, etc.

[0172] In some respects, the prediction filter can be applied before or after the interpolation filter for each reference block in each reference list. In one example, the prediction filter can be applied before the interpolation filter to generate the prediction block for the score position. In another example, the prediction filter can be applied after the interpolation filter to generate the prediction block for the score position.

[0173] Figure 15 A flowchart illustrating an overview process (1500) according to one aspect of the invention is shown. Process (1500) can be used with a video decoder. In various aspects, process (1500) is executed by processing circuitry, such as processing circuitry executing the functions of video decoder (110), processing circuitry executing the functions of video decoder (210), etc. In some aspects, process (1500) is implemented in the form of software instructions, so that when processing circuitry executes software instructions, processing circuitry executes process (1500). The process begins at (S1501) and continues to (S1510).

[0174] In (S1510), the encoded video stream is received. The encoded video stream includes the encoded information of the current block in the current image.

[0175] At (S1520), it is determined that the encoded information indicates that the current block is predicted using a chained vector. The chained vector includes at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block.

[0176] In (S1530), at least one vector offset for refining the chain vector is determined based on at least the first reference block and the second reference block.

[0177] At (S1540), the current block is reconstructed based on the chained vector and vector offset.

[0178] In some respects, a chain vector is at least one of a chain motion vector and a chain block vector.

[0179] According to one aspect of the invention, the chained vector is a chained motion vector comprising at least a first sub-motion vector and a second sub-motion vector, the first sub-motion vector pointing relative to the current block to a first reference block in a first reference image, and the second sub-motion vector pointing relative to the first reference block to a second reference block in a second reference image. In some examples, a search range for candidate motion vector offsets can be searched. A cost associated with a candidate motion vector offset is calculated, at least based on the first reference block in the first reference image and the second reference block in the second reference image. Then, a motion vector offset is determined based on the cost associated with the candidate motion vector offset. For example, the candidate motion vector with the lowest cost value is selected as the motion vector offset.

[0180] In some examples, the first and second reference images are located on the same side of the current image in the display order. In one example, to calculate the cost value, a candidate motion vector offset is applied to the first reference block to obtain a first candidate refined reference block in the first reference image, and the candidate motion vector offset is applied to the second reference block to obtain a second candidate refined reference block in the second reference image. The cost value associated with the candidate motion vector offset is calculated based on the difference between the first and second candidate refined reference blocks.

[0181] In some examples, the first and second reference images are located on different sides of the current image in the display order. In one example, to calculate the cost value, a candidate motion vector offset is applied to the first reference block to obtain a first candidate refined reference block in the first reference image, and the inverse value of that candidate motion vector offset is applied to the second reference block to obtain a second candidate refined reference block in the second reference image. The cost value associated with the candidate motion vector offset is calculated based on the difference between the first and second candidate refined reference blocks.

[0182] In some respects, the encoded information indicates bidirectional prediction of the current block using a first motion vector and a second motion vector, and at least one of the first motion vector and the second motion vector is a chain motion vector.

[0183] In some examples, the first motion vector is a first chained motion vector in a first direction in the display order, and the second motion vector is a second chained motion vector in a second direction in the display order. The first chained motion vector points to a first reference block in a first reference image and a second reference block in a second reference image, and the second chained motion vector points to a third reference block in a third reference image and a fourth reference block in a fourth reference image. Furthermore, based on the first reference block in the first reference image and the second reference block in the second reference image, a first motion vector offset of the first chained motion vector is determined; based on the third reference block in the third reference image and the fourth reference block in the fourth reference image, a second motion vector offset of the second chained motion vector is determined. In one example, based on the first chained motion vector with the first motion vector offset, a first prediction for the current block is generated; based on the second chained motion vector with the second motion vector offset, a second prediction for the current block is generated. Based on the first prediction and the second prediction, the current block is reconstructed.

[0184] In some examples, a cost associated with a candidate motion vector offset is calculated based on a first reference block in a first reference image, a second reference block in a second reference image, a third reference block in a third reference image, and a fourth reference block in a fourth reference image. A motion vector offset is then selected from the candidate motion vector offsets based on this cost value. A first prediction for the current block is generated based on a first chain of motion vectors having this motion vector offset. A second prediction for the current block is generated based on a second chain of motion vectors having the opposite value of this motion vector offset. The current block is then reconstructed based on the first and second predictions.

[0185] In one example, to calculate the cost value, for one of these candidate motion vector offsets, the candidate motion vector offset is applied to a first reference block and a second reference block respectively to obtain a first refined reference block and a second refined reference block, and a first sub-value is calculated based on the difference between the first and second refined reference blocks. Furthermore, the inverse value of the candidate motion vector offset is applied to a third and a fourth reference block respectively to obtain a third refined reference block and a fourth refined reference block, and a second sub-value is calculated based on the difference between the third and fourth refined reference blocks. The cost value associated with this candidate motion vector offset is calculated as the sum of the first and second sub-values.

[0186] In some examples, a refined chain vector is determined when at least one of the block shape and block size meets the requirements. In some examples, a refined chain vector is determined when the difference between the first reference block and the second reference block meets the requirements.

[0187] The process then continues to (S1599) and terminates.

[0188] The process (1500) can be adjusted as appropriate. One or more steps in the process (1500) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used.

[0189] Figure 16 A flowchart illustrating an overview process (1600) according to one aspect of the present invention is shown. Process (1600) can be used with a video encoder. In various aspects, process (1600) is executed by processing circuitry, such as processing circuitry that performs the functions of a video encoder (103), processing circuitry that performs the functions of a video encoder (303), etc. In some aspects, process (1600) is implemented in the form of software instructions, so that when processing circuitry executes software instructions, processing circuitry executes process (1600). The process begins at (S1601) and continues to (S1610).

[0190] In (S1610), refinement is applied to the chained vector of the current block in the current image. The chained vector includes at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block.

[0191] In (S1620), at least one vector offset is determined based on the first reference block and the second reference block.

[0192] In (S1630), the current block is encoded based on the chained vector and vector offset.

[0193] It should be noted that a chain vector is at least one of a chain motion vector and a chain block vector.

[0194] In some aspects, the chained vector is a chained motion vector that includes at least a first sub-motion vector and a second sub-motion vector, the first sub-motion vector pointing relative to the current block to a first reference block in a first reference image, and the second sub-motion vector pointing relative to the first reference block to a second reference block in a second reference image. In some examples, to determine the vector offset, a search range of candidate motion vector offsets can be searched, at least based on the first reference block in the first reference image and the second reference block in the second reference image, calculating the cost associated with each candidate motion vector offset respectively. Based on the cost associated with the candidate motion vector offsets, a motion vector offset is determined.

[0195] In some examples, the first and second reference images are located on the same side of the current image in the display order. In one example, to calculate the cost value, a candidate motion vector offset is applied to the first reference block to obtain a first candidate refined reference block in the first reference image, and the candidate motion vector offset is applied to the second reference block to obtain a second candidate refined reference block in the second reference image. The cost value associated with the candidate motion vector offset is calculated based on the difference between the first and second candidate refined reference blocks.

[0196] In some examples, the first and second reference images are located on different sides of the current image in the display order. In one example, to calculate the cost value, a candidate motion vector offset is applied to the first reference block to obtain a first candidate refined reference block in the first reference image, and the inverse value of that candidate motion vector offset is applied to the second reference block to obtain a second candidate refined reference block in the second reference image. The cost value associated with the candidate motion vector offset is calculated based on the difference between the first and second candidate refined reference blocks.

[0197] In some examples, it is determined that the current block is encoded by bidirectional prediction using a first motion vector and a second motion vector, and at least one of the first motion vector and the second motion vector is a chained motion vector.

[0198] In some examples, the first motion vector is a first chained motion vector in a first direction in the display order, and the second motion vector is a second chained motion vector in a second direction in the display order. The first chained motion vector points to a first reference block in a first reference image and a second reference block in a second reference image, and the second chained motion vector points to a third reference block in a third reference image and a fourth reference block in a fourth reference image. In some examples, a first motion vector offset of the first chained motion vector is determined based on the first reference block in the first reference image and the second reference block in the second reference image, and a second motion vector offset of the second chained motion vector is determined based on the third reference block in the third reference image and the fourth reference block in the fourth reference image. A first prediction for the current block is generated based on the first chained motion vector with the first motion vector offset, and a second prediction for the current block is generated based on the second chained motion vector with the second motion vector offset. Bidirectional prediction for the current block is performed based on the first prediction and the second prediction.

[0199] In some examples, a cost associated with a candidate motion vector offset is calculated based on a first reference block in a first reference image, a second reference block in a second reference image, a third reference block in a third reference image, and a fourth reference block in a fourth reference image. Based on the cost value, a motion vector offset is selected from the candidate motion vector offsets. A first prediction for the current block is generated based on a first chain of motion vectors having that motion vector offset. A second prediction for the current block is generated based on a second chain of motion vectors having the opposite value of that motion vector offset. Based on the first and second predictions, a bidirectional prediction for the current block is performed.

[0200] In some examples, to calculate the cost value, for one of these candidate motion vector offsets, the candidate motion vector offset is applied to a first reference block and a second reference block respectively to obtain a first refined reference block and a second refined reference block, and a first sub-value is calculated based on the difference between the first refined reference block and the second refined reference block. Furthermore, the inverse value of the candidate motion vector offset is applied to a third reference block and a fourth reference block respectively to obtain a third refined reference block and a fourth refined reference block, and a second sub-value is calculated based on the difference between the third refined reference block and the fourth refined reference block. The cost value associated with the candidate motion vector offset is then calculated as the sum of the first sub-value and the second sub-value.

[0201] In one example, a refined chain vector is determined when at least one of the block shape and block size meets the requirements. In another example, a refined chain vector is determined when the difference between the first reference block and the second reference block meets the requirements.

[0202] The process then continues to (S1699) and terminates.

[0203] The process (1600) can be adjusted as appropriate. One or more steps in the process (1600) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used.

[0204] According to one aspect of the present invention, a method for processing visual media data is provided. In this method, a conversion between a visual media file and a visual media data bitstream is performed according to format rules. For example, the bitstream may be a bitstream decoded / encoded using any of the decoding and / or encoding methods described in this application. The format rules may specify at least one constraint on the bitstream and / or at least one process to be performed by the decoder and / or encoder.

[0205] In one example, the bitstream includes encoded information of the current block in the current image, indicating prediction of the current block based on a chained vector. This chained vector includes at least a first sub-vector and a second sub-vector, where the first sub-vector points to a first reference block relative to the current block, and the second sub-vector points to a second reference block relative to the first reference block. The format rules specify that at least one vector offset is determined based on the first and second reference blocks; and the current block is reconstructed based on the chained vector and the vector offset.

[0206] Figure 17 A flowchart illustrating an overview process (1700) according to one aspect of the invention is shown. Process (1700) can be used with a video decoder. In various aspects, process (1700) is executed by processing circuitry, such as processing circuitry executing the functions of video decoder (110), processing circuitry executing the functions of video decoder (210), etc. In some aspects, process (1700) is implemented in the form of software instructions, so that when processing circuitry executes software instructions, processing circuitry executes process (1700). The process begins at (S1701) and continues to (S1710).

[0207] At (S1710), the encoded video stream is received. The encoded video stream includes the encoded information of the current block in the current image.

[0208] In (S1720), it is determined that the encoded information indicates that the current block is predicted using a chain vector, the chain vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block.

[0209] In (S1730), a prediction filter is determined based on the first reference block and the second reference block.

[0210] At (S1740), the current block is reconstructed based on the chained vector and prediction filter.

[0211] In some aspects, the filter cost associated with multiple predefined filters in a predefined filter set is calculated. Based on the filter cost, the predicted filter is selected from the predefined filter set.

[0212] In some examples, the predefined filter set includes at least one of the following: linear filter, nonlinear filter, sharpening filter, smoothing filter, bilateral filter; and / or regression-based filter derived from the first reference block and the second reference block.

[0213] In some examples, without applying multiple predefined filters, the no-filter cost is calculated based on the difference between the first and second reference blocks. If the no-filter cost is lower than the individual filter costs, it is determined that the multiple predefined filters are not applied.

[0214] In some examples, in order to calculate the filter cost, for one of the plurality of predefined filters, the predefined filter is applied to a second reference block to generate a filtered second reference block, and the filter cost associated with the predefined filter is calculated based on the difference between the first reference block and the filtered second reference block.

[0215] In some aspects, the encoded information indicates bidirectional prediction of the current block using a first chain of motion vectors and a second chain of motion vectors, the first chain of motion vectors pointing to a first reference block in a first reference image and a second reference block in a second reference image, and the second chain of motion vectors pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image. In some examples, a first prediction filter is determined based on the first and second reference blocks, and a second prediction filter is determined based on the third and fourth reference blocks. The current block is reconstructed based on the first and second prediction filters.

[0216] In one example, to determine a first predictive filter, a first filter cost associated with a plurality of predefined filters is calculated based on a first reference block and a second reference block. Based on the first filter cost associated with the plurality of predefined filters, a first predictive filter is selected from the plurality of predefined filters. In another example, to determine a second predictive filter, a second filter cost associated with the plurality of predefined filters is calculated based on a third reference block and a fourth reference block, and a second predictive filter is selected from the plurality of predefined filters based on the second filter cost associated with the plurality of predefined filters.

[0217] In some examples, filter costs associated with multiple predefined filters are calculated based on a first reference block, a second reference block, a third reference block, and a fourth reference block. Based on these filter costs associated with the predefined filters, a prediction filter is selected from the predefined filters. The current block is then reconstructed based on this prediction filter.

[0218] In some examples, to calculate the filter cost value, for one of the plurality of predefined filters, the predefined filter is applied to a second reference block to generate a filtered second reference block, and a first-generation value is calculated based on the difference between the first reference block and the filtered second reference block. Furthermore, the predefined filter is applied to a fourth reference block to generate a filtered fourth reference block, and a second-generation value is calculated based on the difference between the third reference block and the filtered fourth reference block. The sum of the first-generation value and the second-generation value is calculated as the filter cost value associated with the predefined filter.

[0219] In some respects, linear regression is used to derive the prediction filter based on the first and second reference blocks.

[0220] In some examples, a first prediction filter is derived using linear regression based on a first reference block in a first reference image and a second reference block in a second reference image. The first prediction filter is scaled based on the ratio of a first distance (the distance between the first and second reference images) to obtain the prediction filter.

[0221] In one example, to reconstruct the current block, motion compensation is applied to a second reference block to obtain a motion-compensated block, and the prediction filter is applied to that motion-compensated block to generate a prediction block for the current block.

[0222] In one example, to reconstruct the current block, the prediction filter is applied to a second reference block to obtain a filtered block of the current block, and motion compensation is applied to the filtered block to generate a predicted block of the current block.

[0223] In one example, to reconstruct the current block, an interpolation filter is applied to a second reference block to obtain an intermediate block, and the prediction filter is applied to that intermediate block to generate a prediction block for the current block.

[0224] In one example, to reconstruct the current block, a prediction filter is applied to a second reference block to obtain an intermediate block of the current block, and an interpolation filter is applied to that intermediate block to generate a prediction block of the current block.

[0225] The process then continues to (S1799) and terminates.

[0226] The process (1700) can be adjusted as appropriate. One or more steps in the process (1700) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used.

[0227] Figure 18 A flowchart illustrating an overview process (1800) according to one aspect of the invention is shown. Process (1800) can be used with a video encoder. In various aspects, process (1800) is executed by processing circuitry, such as processing circuitry that performs the functions of a video encoder (103), processing circuitry that performs the functions of a video encoder (303), etc. In some aspects, process (1800) is implemented in the form of software instructions, so that when processing circuitry executes software instructions, processing circuitry executes process (1800). The process begins at (S1801) and continues to (S1810).

[0228] At (S1810), a predictive filtering evaluation is performed on the chained vector of the current block in the current image. The chained vector includes at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block.

[0229] In (S1820), the prediction filter is determined based on the first reference block and the second reference block.

[0230] In (S1830), the current block is encoded based on the chained vector and prediction filter.

[0231] In some aspects, the filter cost associated with multiple predefined filters in a predefined filter set is calculated. Based on the filter cost, the predicted filter is selected from the predefined filter set.

[0232] In some examples, the predefined filter set includes at least one of the following: linear filter, nonlinear filter, sharpening filter, smoothing filter, bilateral filter, and / or regression-based filter derived from the first reference block and the second reference block.

[0233] In some examples, without applying multiple predefined filters, the no-filter cost is calculated based on the difference between the first and second reference blocks. If the no-filter cost is lower than the individual filter costs, it is determined that the multiple predefined filters are not applied.

[0234] In some examples, in order to calculate the filter cost, for one of the plurality of predefined filters, the predefined filter is applied to a second reference block to generate a filtered second reference block, and the filter cost associated with the predefined filter is calculated based on the difference between the first reference block and the filtered second reference block.

[0235] In some aspects, the current block is determined to be encoded via bidirectional prediction using a first chain of motion vectors and a second chain of motion vectors, the first chain of motion vectors pointing to a first reference block in a first reference image and a second reference block in a second reference image, and the second chain of motion vectors pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image. In some examples, a first prediction filter is determined based on the first and second reference blocks, and a second prediction filter is determined based on the third and fourth reference blocks. The current block is then encoded based on the first and second prediction filters.

[0236] In one example, to determine a first predictive filter, a first filter cost associated with a plurality of predefined filters is calculated based on a first reference block and a second reference block. Based on the first filter cost associated with the plurality of predefined filters, a first predictive filter is selected from the plurality of predefined filters. In another example, to determine a second predictive filter, a second filter cost associated with the plurality of predefined filters is calculated based on a third reference block and a fourth reference block, and a second predictive filter is selected from the plurality of predefined filters based on the second filter cost associated with the plurality of predefined filters.

[0237] In some examples, filter costs associated with multiple predefined filters are calculated based on a first reference block, a second reference block, a third reference block, and a fourth reference block. A prediction filter is then selected from these predefined filters based on the filter costs associated with them. The current block is then encoded based on this prediction filter.

[0238] In some examples, to calculate the filter cost value, for one of the plurality of predefined filters, the predefined filter is applied to a second reference block to generate a filtered second reference block, and a first-generation value is calculated based on the difference between the first reference block and the filtered second reference block. Furthermore, the predefined filter is applied to a fourth reference block to generate a filtered fourth reference block, and a second-generation value is calculated based on the difference between the third reference block and the filtered fourth reference block. The sum of the first-generation value and the second-generation value is calculated as the filter cost value associated with the predefined filter.

[0239] In some respects, linear regression is used to derive the prediction filter based on the first and second reference blocks.

[0240] In some examples, a first prediction filter is derived using linear regression based on a first reference block in a first reference image and a second reference block in a second reference image. The first prediction filter is scaled based on the ratio of a first distance (the distance between the first and second reference images) to obtain the prediction filter.

[0241] In one example, in order to encode the current block, motion compensation is applied to a second reference block to obtain a motion-compensated block, and the prediction filter is applied to the motion-compensated block to generate a prediction block for the current block.

[0242] In one example, in order to encode the current block, the prediction filter is applied to a second reference block to obtain a filtered block of the current block, and motion compensation is applied to the filtered block to generate a prediction block of the current block.

[0243] In one example, in order to encode the current block, an interpolation filter is applied to a second reference block to obtain an intermediate block, and the prediction filter is applied to that intermediate block to generate a prediction block for the current block.

[0244] In one example, to encode the current block, a prediction filter is applied to a second reference block to obtain an intermediate block of the current block, and an interpolation filter is applied to that intermediate block to generate a prediction block of the current block.

[0245] The process then continues to (S1899) and terminates.

[0246] The process (1800) can be adjusted as appropriate. One or more steps in the process (1800) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used.

[0247] According to one aspect of the present invention, a method for processing visual media data is provided. In this method, a conversion between a visual media file and a visual media data bitstream is performed according to format rules. For example, the bitstream may be a bitstream decoded / encoded using any of the decoding and / or encoding methods described in this application. The format rules may specify at least one constraint on the bitstream and / or at least one process to be performed by the decoder and / or encoder.

[0248] In one example, the bitstream includes encoded information of the current block in the current image, which indicates prediction of the current block using a chained vector. This chained vector includes at least a first sub-vector and a second sub-vector, where the first sub-vector points to a first reference block relative to the current block, and the second sub-vector points to a second reference block relative to the first reference block. The format rules specify that a prediction filter is determined based on the first and second reference blocks; and the current block is reconstructed based on the chained vector and the prediction filter.

[0249] The above-described technologies can be implemented in the form of computer software using computer-readable instructions and physically stored in at least one computer-readable medium. For example, Figure 19 A computer system suitable for implementing certain aspects of the disclosed subject matter is shown (1900).

[0250] Computer software can be written using any suitable machine code or computer language. This machine code or computer language may be processed by mechanisms such as assembly, compilation, and linking to generate code containing instructions that can be executed directly by at least one computer central processing unit (CPU), graphics processing unit (GPU), or through interpretation, microcode execution, or other methods.

[0251] These instructions can be executed on various types of computers or their components, including personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, and so on.

[0252] Figure 19 The components of the computer system (1900) shown are merely examples and are not intended to imply any limitation on the scope or functionality of computer software used to implement various aspects of the invention. The configuration of the components should also not be construed as having any dependency or requirement on any component or combination of components shown in the example aspects of the computer system (1900).

[0253] Computer systems (1900) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from at least one human user via, for example, tactile input (e.g., key presses, swipes, data glove movements), audio input (e.g., voice, clapping sounds), visual input (e.g., gestures), and olfactory input (not depicted). Human-machine interface devices may also be used to acquire media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0254] Human-machine interface input devices may include at least one of the following (only one of each is depicted): keyboard (1901), mouse (1902), touchpad (1903), touch screen (1910), data glove (not shown), joystick (1905), microphone (1906), scanner (1907), and camera (1908).

[0255] The computer system (1900) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of at least one human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback provided by a touchscreen (1910), a data glove (not shown), or a joystick 1905, but there may also be tactile feedback devices that do not act as input devices), audio output devices (e.g., speakers (1909), headphones (not depicted)), visual output devices (e.g., screens (1910), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input functionality, each with or without tactile feedback functionality—some of which may be able to output two-dimensional or more than three-dimensional visual outputs through stereoscopic output, virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted), etc.), and printers (not depicted).

[0256] The computer system (1900) may also include user-accessible storage devices and their associated media, such as optical media or similar media (1921) including CD / DVD ROM / RW (1920) equipped with CD / DVD, flash drives (1922), removable hard disk drives or solid-state drives (1923), conventional magnetic media such as magnetic tapes and floppy disks (not depicted), and dedicated ROM / ASIC / PLD devices such as security dongles (not depicted).

[0257] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not cover transmission media, carrier waves, or other transient signals.

[0258] The computer system (1900) may also include an interface (1954) connected to at least one communication network (1955). The network may be, for example, a wireless network, a wired network, or an optical network. The network may also be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), an in-vehicle and industrial network, a real-time network, a latency-tolerant network, etc. Examples of networks include: local area networks such as Ethernet and wireless LANs; cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; in-vehicle and industrial networks including CAN buses, etc. Some networks typically require external network interface adapters (e.g., the USB port of the computer system (1900)) attached to certain general-purpose data ports or peripheral buses (1949); other networks are typically integrated into the core of the computer system (1900) via a system bus described below (e.g., integrated into a PC computer system via an Ethernet interface, or integrated into a smartphone computer system via a cellular network interface). Using any of these networks, the computer system (1900) can communicate with other entities. This communication can be one-way receiving (e.g., broadcasting TV), one-way transmitting (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as connecting to other computer systems using a local area or wide area digital network. Certain protocols and protocol stacks can be used on each of the networks and network interfaces described above.

[0259] The aforementioned human-machine interface devices, user-accessible storage devices, and network interfaces can be attached to the core (1940) of the computer system (1900).

[0260] The core (1940) may include at least one CPU (1941), GPUs (1942), a dedicated programmable processing unit in the form of a Field Programmable Gate Array (FPGA) (1943), a hardware accelerator for certain tasks (1944), a graphics adapter (1950), etc. These devices, along with read-only memory (ROM) (1945), random-access memory (RAM) (1946), and internal mass storage devices such as internal non-user-accessible hard disk drives, SSDs, etc. (1947), can be connected via a system bus (1948). In some computer systems, the system bus (1948) may be accessed in the form of at least one physical plug, allowing for expansion via additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (1948) or connected to the core's system bus (1948) via a peripheral bus (1949). In one example, a screen (1910) may be connected to a graphics adapter (1950). Peripheral bus architectures include PCI, USB, etc.

[0261] CPUs (1941), GPUs (1942), FPGAs (1943), and accelerators (1944) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (1945) or RAM (1946). Temporary data can also be stored in RAM (1946), while permanent data can be stored, for example, in an internal mass storage device (1947). Fast storage and retrieval of any of these storage devices can be achieved by using a cache memory, which can be closely associated with at least one CPU (1941), GPU (1942), mass storage device (1947), ROM (1945), RAM (1946), etc.

[0262] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be designed and constructed specifically for the purposes of this invention, or may belong to a class well-known and available to those skilled in the art of computer software.

[0263] For example, but not as a limitation, a computer system having an architecture (1900), particularly a core (1940), can provide functionality generated when one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) execute software embodied in at least one tangible computer-readable medium. Such a computer-readable medium can be a medium associated with certain storage devices (e.g., internal core mass storage (1947) or ROM (1945)) of the non-transitory nature of the core (1940) as described above. Software implementing various aspects of the invention can be stored in such a device and executed by the core (1940). Depending on specific needs, the computer-readable medium may include at least one storage device or chip. The software can cause the core (1940), particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute a particular process or a particular portion of a particular process described in this application, including defining data structures stored in RAM (1946) and modifying such data structures according to the process defined by the software. Alternatively or as an alternative, a computer system may provide functionality generated by hard-wired logic or otherwise embodied in circuitry (e.g., an accelerator (1944)), which may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may cover logic, and vice versa. Where appropriate, references to computer-readable media may cover circuitry (e.g., integrated circuits) storing software for execution, circuitry embodying logic for execution, or both. This invention covers any suitable combination of hardware and software.

[0264] The terms “at least one of…” or “one of…” as used in this invention are intended to include any one or a combination of the listed elements. For example, mentioning at least one of A, B, or C, at least one of A, B, and C, at least one of A, B, and / or C, and at least one of A to C are intended to include only A, only B, only C, or any combination thereof. Mentioning one of A or B and one of A and B are intended to include either A or B or (A and B). Where applicable, such as when the elements are not mutually exclusive, the use of “one of…” does not exclude any combination of the listed elements.

[0265] Although several examples of various aspects have been described in this invention, modifications, substitutions, and various alternative equivalents exist that fall within the scope of this invention. Therefore, it should be understood that those skilled in the art will be able to design many systems and methods that, while not explicitly stated or described herein, embody the principles of the invention and are therefore within its spirit and scope.

[0266] The aforementioned disclosure also includes the features mentioned below. These features can be combined in various ways, and are not limited to the combinations mentioned below.

[0267] (1). A video decoding method, comprising: receiving an encoded video stream, the encoded video stream including encoded information of a current block in a current image; determining that the encoded information indicates prediction of the current block using a chained vector, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block; determining at least one vector offset for refining the chained vector based at least on the first reference block and the second reference block; and reconstructing the current block based on the chained vector and the vector offset.

[0268] (2). According to the method of feature (1), wherein the chain vector is at least one of chain motion vector and chain block vector.

[0269] (3). A method according to any one of features (1) to (2), wherein the chain vector is a chain motion vector, the chain motion vector includes at least a first sub-motion vector and a second sub-motion vector, the first sub-motion vector pointing relative to the current block to a first reference block in a first reference image, the second sub-motion vector pointing relative to the first reference block to a second reference block in a second reference image, and determining the vector offset includes: searching within a search range of candidate motion vector offsets; calculating a cost associated with the candidate motion vector offset based at least on the first reference block in the first reference image and the second reference block in the second reference image; and determining a motion vector offset based on the cost associated with the candidate motion vector offset.

[0270] (4). A method according to any one of features (1) to (3), wherein the first reference image and the second reference image are located on the same side of the current image in the display order, the calculation comprising: applying a candidate motion vector offset to the first reference block to obtain a first candidate refined reference block in the first reference image; applying the candidate motion vector offset to the second reference block to obtain a second candidate refined reference block in the second reference image; and calculating a cost associated with the candidate motion vector offset based on the difference between the first candidate refined reference block and the second candidate refined reference block.

[0271] (5). A method according to any one of features (1) to (4), wherein the first reference image and the second reference image are located on different sides of the current image in the display order, the calculation comprising: applying a candidate motion vector offset to the first reference block to obtain a first candidate refined reference block in the first reference image; applying the opposite value of the candidate motion vector offset to the second reference block to obtain a second candidate refined reference block in the second reference image; and calculating a cost associated with the candidate motion vector offset based on the difference between the first candidate refined reference block and the second candidate refined reference block.

[0272] (6). The method according to any one of features (1) to (5), wherein determining that the encoded information indicates prediction of the current block using the chain vector further includes: determining that the encoded information indicates prediction of the current block using a first motion vector and a second motion vector, and at least one of the first motion vector and the second motion vector is a chain motion vector.

[0273] (7). A method according to any one of features (1) to (6), wherein the first motion vector is a first chain motion vector in a first direction in a display order, and the second motion vector is a second chain motion vector in a second direction in the display order, the first chain motion vector pointing to a first reference block in a first reference image and a second reference block in a second reference image, the second chain motion vector pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image, the method comprising: determining a first motion vector offset of the first chain motion vector based on the first reference block in the first reference image and the second reference block in the second reference image; determining a second motion vector offset of the second chain motion vector based on the third reference block in the third reference image and the fourth reference block in the fourth reference image; generating a first prediction of the current block based on the first chain motion vector having the first motion vector offset; generating a second prediction of the current block based on the second chain motion vector having the second motion vector offset; and reconstructing the current block based on the first prediction and the second prediction.

[0274] (8). A method according to any one of features (1) to (7), wherein the first motion vector is a first chain motion vector in a first direction in the display order, and the second motion vector is a second chain motion vector in a second direction in the display order, the first chain motion vector pointing to a first reference block in a first reference image and a second reference block in a second reference image, the second chain motion vector pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image, the method comprising: calculating a cost associated with a candidate motion vector offset based on the first reference block in the first reference image, the second reference block in the second reference image, the third reference block in the third reference image and the fourth reference block in the fourth reference image; determining a motion vector offset from the candidate motion vector offsets based on the cost; generating a first prediction of the current block based on the first chain motion vector having the motion vector offset; generating a second prediction of the current block based on the second chain motion vector having the opposite value of the motion vector offset; and reconstructing the current block based on the first prediction and the second prediction.

[0275] (9). A method according to any one of features (1) to (8), wherein the calculation of the sub-value comprises: for one of the candidate motion vector offsets: applying the one candidate motion vector offset to the first reference block and the second reference block respectively to obtain a first candidate refined reference block and a second candidate refined reference block respectively; calculating a first sub-value based on the difference between the first candidate refined reference block and the second candidate refined reference block; applying the opposite value of the one candidate motion vector offset to the third reference block and the fourth reference block respectively to obtain a third candidate refined reference block and a fourth candidate refined reference block respectively; calculating a second sub-value based on the difference between the third candidate refined reference block and the fourth candidate refined reference block; and calculating the sub-value associated with the one candidate motion vector offset as the sum of the first sub-value and the second sub-value.

[0276] (10). The method according to any one of features (1) to (9) further includes at least one of the following: determining the refinement of the chain vector when at least one of the block shape and block size meets the requirements; and / or determining the refinement of the chain vector when the difference between the first reference block and the second reference block meets the requirements.

[0277] (11). A video coding method, comprising: determining a chained vector applied to a current block in a current image, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block; determining at least one vector offset based on the first reference block and the second reference block; and encoding the current block based on the chained vector and the vector offset.

[0278] (12). According to the method of feature (11), wherein the chain vector is at least one of chain motion vector and chain block vector.

[0279] (13). A method according to any one of features (11) to (12), wherein the chained vector is a chained motion vector, the chained motion vector including at least a first sub-motion vector and a second sub-motion vector, the first sub-motion vector pointing relative to the current block to a first reference block in a first reference image, the second sub-motion vector pointing relative to the first reference block to a second reference block in a second reference image, and determining the vector offset includes: searching within a search range of candidate motion vector offsets; calculating a cost associated with the candidate motion vector offset based at least on the first reference block in the first reference image and the second reference block in the second reference image; and determining a motion vector offset based on the cost associated with the candidate motion vector offset.

[0280] (14). The method according to any one of features (11) to (13), wherein the first reference image and the second reference image are located on the same side of the current image in the display order, the calculation includes: applying a candidate motion vector offset to the first reference block to obtain a first candidate refined reference block in the first reference image; applying the candidate motion vector offset to the second reference block to obtain a second candidate refined reference block in the second reference image; and calculating a cost associated with the candidate motion vector offset based on the difference between the first candidate refined reference block and the second candidate refined reference block.

[0281] (15). The method according to any one of features (11) to (14), wherein the first reference image and the second reference image are located on different sides of the current image in the display order, the calculation includes: applying a candidate motion vector offset to the first reference block to obtain a first candidate refined reference block in the first reference image; applying the opposite value of the candidate motion vector offset to the second reference block to obtain a second candidate refined reference block in the second reference image; and calculating a cost associated with the candidate motion vector offset based on the difference between the first candidate refined reference block and the second candidate refined reference block.

[0282] (16). The method according to any one of features (11) to (15) further includes: determining that the current block is encoded by bidirectional prediction using a first motion vector and a second motion vector, and that at least one of the first motion vector and the second motion vector is a chain motion vector.

[0283] (17). A method according to any one of features (11) to (16), wherein the first motion vector is a first chain motion vector in a first direction in a display order, and the second motion vector is a second chain motion vector in a second direction in the display order, the first chain motion vector pointing to a first reference block in a first reference image and a second reference block in a second reference image, the second chain motion vector pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image, the method comprising: determining a first motion vector offset of the first chain motion vector based on the first reference block in the first reference image and the second reference block in the second reference image; determining a second motion vector offset of the second chain motion vector based on the third reference block in the third reference image and the fourth reference block in the fourth reference image; generating a first prediction of the current block based on the first chain motion vector having the first motion vector offset; generating a second prediction of the current block based on the second chain motion vector having the second motion vector offset; and performing the bidirectional prediction of the current block based on the first prediction and the second prediction.

[0284] (18). A method according to any one of features (11) to (17), wherein the first motion vector is a first chain motion vector in a first direction in a display order, and the second motion vector is a second chain motion vector in a second direction in the display order, the first chain motion vector pointing to a first reference block in a first reference image and a second reference block in a second reference image, the second chain motion vector pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image, the method comprising: calculating a cost associated with a candidate motion vector offset based on the first reference block in the first reference image, the second reference block in the second reference image, the third reference block in the third reference image and the fourth reference block in the fourth reference image; determining a motion vector offset from the candidate motion vector offsets based on the cost; generating a first prediction for the current block based on the first chain motion vector having the motion vector offset; generating a second prediction for the current block based on the second chain motion vector having the opposite value of the motion vector offset; and performing the bidirectional prediction of the current block based on the first prediction and the second prediction.

[0285] (19). The method according to any one of features (11) to (18), wherein the calculation of the substitution value comprises: for one of the candidate motion vector offsets: applying the one candidate motion vector offset to the first reference block and the second reference block respectively to obtain a first candidate refined reference block and a second candidate refined reference block respectively; calculating a first sub-sub-value based on the difference between the first candidate refined reference block and the second candidate refined reference block; applying the opposite value of the one candidate motion vector offset to the third reference block and the fourth reference block respectively to obtain a third candidate refined reference block and a fourth candidate refined reference block respectively; calculating a second sub-sub-value based on the difference between the third candidate refined reference block and the fourth candidate refined reference block; and calculating the substitution value associated with the one candidate motion vector offset as the sum of the first sub-sub-value and the second sub-sub-value.

[0286] (20). The method according to any one of features (11) to (19) further includes at least one of the following: determining the refinement of the chain vector when at least one of the block shape and block size meets the requirements; and / or determining the refinement of the chain vector when the difference between the first reference block and the second reference block meets the requirements.

[0287] (21). A video decoding method, comprising: receiving an encoded video stream, the encoded video stream including encoded information of a current block in a current image; determining that the encoded information indicates prediction of the current block using a chained vector, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block; determining a prediction filter based on the first reference block and the second reference block; and reconstructing the current block based on the chained vector and the prediction filter.

[0288] (22). The method according to feature (21), wherein determining the predictive filter includes: calculating the filter cost associated with a plurality of predefined filters in a predefined filter set; and selecting the predictive filter from the predefined filter set based on the filter cost.

[0289] (23). The method according to any one of features (21) to (22), wherein the predefined filter set includes at least one of the following: a linear filter; a nonlinear filter; a sharpening filter; a smoothing filter; a bilateral filter; and / or a regression-based filter derived from the first reference block and the second reference block.

[0290] (24). The method according to any one of features (21) to (23) further includes: calculating the filterless cost based on the difference between the first reference block and the second reference block when the plurality of predefined filters are not applied; and determining that the plurality of predefined filters are not applied when the filterless cost is lower than the respective filter costs.

[0291] (25). The method according to any one of features (21) to (24), wherein the calculation includes: for one of the plurality of predefined filters: applying the predefined filter to the second reference block to generate a filtered second reference block; and calculating the filter cost associated with the predefined filter based on the difference between the first reference block and the filtered second reference block.

[0292] (26). A method according to any one of features (21) to (25), wherein the encoded information indicates bidirectional prediction of the current block using a first chain motion vector and a second chain motion vector, the first chain motion vector pointing to a first reference block in a first reference image and a second reference block in a second reference image, the second chain motion vector pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image, and the method comprising: determining a first prediction filter based on the first reference block and the second reference block; determining a second prediction filter based on the third reference block and the fourth reference block; and reconstructing the current block based on the first prediction filter and the second prediction filter.

[0293] (27). A method according to any one of features (21) to (26), wherein: determining the first predictive filter comprises: calculating a first filter cost associated with a plurality of predefined filters based on the first reference block and the second reference block; and selecting the first predictive filter from the plurality of predefined filters based on the first filter cost associated with the plurality of predefined filters; and determining the second predictive filter comprises: calculating a second filter cost associated with the plurality of predefined filters based on the third reference block and the fourth reference block; and selecting the second predictive filter from the plurality of predefined filters based on the second filter cost associated with the plurality of predefined filters.

[0294] (28). A method according to any one of features (21) to (27), wherein the encoded information indicates bidirectional prediction of the current block using a first chain motion vector and a second chain motion vector, the first chain motion vector pointing to a first reference block in a first reference image and a second reference block in a second reference image, the second chain motion vector pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image, and the method includes: calculating a filter cost associated with a plurality of predefined filters based on the first reference block, the second reference block, the third reference block and the fourth reference block; selecting the prediction filter from the plurality of predefined filters based on the filter cost associated with the plurality of predefined filters; and reconstructing the current block based on the prediction filter.

[0295] (29). A method according to any one of features (21) to (28), wherein calculating the filter cost includes: for one of the plurality of predefined filters: applying the predefined filter to the second reference block to generate a filtered second reference block; calculating a first generation value based on the difference between the first reference block and the filtered second reference block; applying the predefined filter to the fourth reference block to generate a filtered fourth reference block; calculating a second generation value based on the difference between the third reference block and the filtered fourth reference block; and calculating the sum of the first generation value and the second generation value as the filter cost associated with the predefined filter.

[0296] (30). The method according to any one of features (21) to (29), wherein determining the prediction filter includes: deriving the prediction filter using linear regression based on the first reference block and the second reference block.

[0297] (31). A method according to any one of features (21) to (30), wherein determining the prediction filter comprises: deriving a first prediction filter using linear regression based on a first reference block in a first reference image and a second reference block in a second reference image; and scaling the first prediction filter based on a ratio of a first distance to a second distance to obtain the prediction filter, the first distance being the distance between the first reference image and the second reference image, and the second distance being the distance between the current image and the second reference image.

[0298] (32). The method according to any one of features (21) to (31), wherein the reconstruction of the current block includes: applying motion compensation to the second reference block to obtain a motion-compensated block; and applying the prediction filter to the motion-compensated block to generate a prediction block of the current block.

[0299] (33). The method according to any one of features (21) to (32), wherein the reconstruction of the current block includes: applying the prediction filter to the second reference block to obtain a filtered block of the current block; and applying motion compensation to the filtered block to generate the prediction block of the current block.

[0300] (34). The method according to any one of features (21) to (33), wherein the reconstruction of the current block includes: applying an interpolation filter to the second reference block to obtain an intermediate block; and applying the prediction filter to the intermediate block to generate a prediction block of the current block.

[0301] (35). The method according to any one of features (21) to (34), wherein the reconstruction of the current block includes: applying the prediction filter to the second reference block to obtain an intermediate block of the current block; and applying an interpolation filter to the intermediate block to generate a prediction block of the current block.

[0302] (36). A video coding method, comprising: determining a predictive filtering evaluation of a chained vector of a current block in a current image, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block; determining a predictive filter based on the first reference block and the second reference block; and coding the current block based on the chained vector and the predictive filter.

[0303] (37). The method according to feature (36), wherein determining the predictive filter includes: calculating the filter cost associated with a plurality of predefined filters in a predefined filter set; and selecting the predictive filter from the predefined filter set based on the filter cost.

[0304] (38). The method according to any one of features (36) to (37), wherein the predefined filter set includes at least one of the following: a linear filter; a nonlinear filter; a sharpening filter; a smoothing filter; a bilateral filter; and / or a regression-based filter derived from the first reference block and the second reference block.

[0305] (39). The method according to any one of features (36) to (38) further includes: calculating the filterless cost based on the difference between the first reference block and the second reference block when the plurality of predefined filters are not applied; and determining that the plurality of predefined filters are not applied when the filterless cost is lower than the respective filter costs.

[0306] (40). The method according to any one of features (36) to (39), wherein the calculation includes: for one of the plurality of predefined filters: applying the predefined filter to the second reference block to generate a filtered second reference block; and calculating the filter cost associated with the predefined filter based on the difference between the first reference block and the filtered second reference block.

[0307] (41). A method according to any one of features (36) to (40), wherein the current block is encoded by bidirectional prediction using a first chain motion vector and a second chain motion vector, the first chain motion vector pointing to a first reference block in a first reference image and a second reference block in a second reference image, the second chain motion vector pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image, and the method includes: determining a first prediction filter based on the first reference block and the second reference block; determining a second prediction filter based on the third reference block and the fourth reference block; and performing the bidirectional prediction of the current block based on the first prediction filter and the second prediction filter.

[0308] (42). A method according to any one of features (36) to (41), wherein: determining the first predictive filter comprises: calculating a first filter cost associated with a plurality of predefined filters based on the first reference block and the second reference block; and selecting the first predictive filter from the plurality of predefined filters based on the first filter cost associated with the plurality of predefined filters; and determining the second predictive filter comprises: calculating a second filter cost associated with the plurality of predefined filters based on the third reference block and the fourth reference block; and selecting the second predictive filter from the plurality of predefined filters based on the second filter cost associated with the plurality of predefined filters.

[0309] (43). A method according to any one of features (36) to (42), wherein the current block is encoded by bidirectional prediction using a first chain motion vector and a second chain motion vector, the first chain motion vector pointing to a first reference block in a first reference image and a second reference block in a second reference image, the second chain motion vector pointing to a third reference block in a third reference image and a fourth reference block in a fourth reference image, and the method includes: calculating a filter cost associated with a plurality of predefined filters based on the first reference block, the second reference block, the third reference block and the fourth reference block; selecting the prediction filter from the plurality of predefined filters based on the filter cost associated with the plurality of predefined filters; and performing the bidirectional prediction of the current block based on the prediction filter.

[0310] (44). A method according to any one of features (36) to (43), wherein calculating the filter cost includes: for one of the plurality of predefined filters: applying the predefined filter to the second reference block to generate a filtered second reference block; calculating a first generation value based on the difference between the first reference block and the filtered second reference block; applying the predefined filter to the fourth reference block to generate a filtered fourth reference block; calculating a second generation value based on the difference between the third reference block and the filtered fourth reference block; and calculating the sum of the first generation value and the second generation value as the filter cost associated with the predefined filter.

[0311] (45). The method according to any one of features (36) to (44), wherein determining the prediction filter includes: deriving the prediction filter using linear regression based on the first reference block and the second reference block.

[0312] (46). A method according to any one of features (36) to (45), wherein determining the prediction filter comprises: deriving a first prediction filter using linear regression based on a first reference block in a first reference image and a second reference block in a second reference image; and scaling the first prediction filter based on a ratio of a first distance to a second distance to obtain the prediction filter, the first distance being the distance between the first reference image and the second reference image, and the second distance being the distance between the current image and the second reference image.

[0313] (47). The method according to any one of features (36) to (46), wherein encoding the current block includes: applying motion compensation to the second reference block to obtain a motion-compensated block; applying the prediction filter to the motion-compensated block to generate a prediction block of the current block; and encoding the current block based on the prediction block.

[0314] (48). The method according to any one of features (36) to (47), wherein encoding the current block includes: applying the prediction filter to the second reference block to obtain a filtered block of the current block; applying motion compensation to the filtered block to generate a prediction block of the current block; and encoding the current block based on the prediction block.

[0315] (49). The method according to any one of features (36) to (48), wherein encoding the current block includes: applying an interpolation filter to the second reference block to obtain an intermediate block; applying the prediction filter to the intermediate block to generate a prediction block of the current block; and encoding the current block based on the prediction block.

[0316] (50). The method according to any one of features (36) to (49), wherein encoding the current block includes: applying the prediction filter to the second reference block to obtain an intermediate block of the current block; applying an interpolation filter to the intermediate block to generate a prediction block of the current block; and encoding the current block based on the prediction block.

[0317] (51). A method for processing visual media data, the method comprising: processing a bitstream of visual media data according to a format rule, wherein: the bitstream includes encoded information of a current block in a current image, the encoded information of the current block indicating prediction of the current block based on a chained vector, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block; and the format rule specifies: determining at least one vector offset based on the first reference block and the second reference block; and reconstructing the current block based on the chained vector and the vector offset.

[0318] (52). A method for processing visual media data, the method comprising: processing a bitstream of visual media data according to a format rule, wherein: the bitstream includes encoded information of a current block in a current image, the encoded information indicating prediction of the current block using a chained vector, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block; and the format rule specifies: determining a prediction filter based on the first reference block and the second reference block; and reconstructing the current block based on the prediction filter.

[0319] (53). An apparatus for video decoding, comprising a processing circuit configured to perform a method according to any one of features (1) to (10).

[0320] (54). An apparatus for video encoding, comprising processing circuitry configured to perform a method according to any one of features (11) to (20).

[0321] (55). An apparatus for video decoding, comprising processing circuitry configured to perform a method according to any one of features (21) to (35).

[0322] (56). An apparatus for video encoding, comprising processing circuitry configured to perform a method according to any one of features (36) to (50).

[0323] (57). A non-transitory computer-readable storage medium storing instructions which, when executed by at least one processor, cause the at least one processor to perform a method according to any one of features (1) to (52).

Claims

1. A video decoding method, characterized in that, include: Receive an encoded video stream, wherein the encoded video stream includes the encoded information of the current block in the current image; The encoded information indicates that the current block is predicted using a chained vector, the chained vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block; At least one vector offset for refining the chain vector is determined based on at least the first reference block and the second reference block; as well as The current block is reconstructed based on the chained vector and the vector offset.

2. The method according to claim 1, characterized in that, The chain vector is at least one of a chain motion vector and a chain block vector.

3. The method according to any one of claims 1 to 2, characterized in that, The chained vector is a chained motion vector, which includes at least a first sub-motion vector and a second sub-motion vector. The first sub-motion vector points relative to the current block to the first reference block in the first reference image, and the second sub-motion vector points relative to the first reference block to the second reference block in the second reference image. Determining the vector offset includes: The search is performed within the search range of candidate motion vector offsets; Based at least on the first reference block in the first reference image and the second reference block in the second reference image, calculate the cost associated with the candidate motion vector offset; and A motion vector offset is determined based on the cost associated with the candidate motion vector offset.

4. The method according to claim 3, characterized in that, The first reference image and the second reference image are located on the same side of the current image in the display order, and the calculation includes: A candidate motion vector offset is applied to the first reference block to obtain a first candidate refined reference block in the first reference image; The candidate motion vector offset is applied to the second reference block to obtain a second candidate refined reference block in the second reference image; and Based on the difference between the first candidate refined reference block and the second candidate refined reference block, the cost associated with the candidate motion vector offset is calculated.

5. The method according to claim 3, characterized in that, The first reference image and the second reference image are located on different sides of the current image in the display order, and the calculation includes: A candidate motion vector offset is applied to the first reference block to obtain a first candidate refined reference block in the first reference image; The inverse value of the candidate motion vector offset is applied to the second reference block to obtain a second candidate refined reference block in the second reference image; and Based on the difference between the first candidate refined reference block and the second candidate refined reference block, the cost associated with the candidate motion vector offset is calculated.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining that the encoded information indicates the prediction of the current block using the chained vector further includes: The encoded information indicates that the current block is predicted bidirectionally using a first motion vector and a second motion vector, and at least one of the first motion vector and the second motion vector is a chain motion vector.

7. The method according to claim 6, characterized in that, The first motion vector is a first chain motion vector in a first direction in the display sequence, and the second motion vector is a second chain motion vector in a second direction in the display sequence. The first chain motion vector points to a first reference block in a first reference image and a second reference block in a second reference image, and the second chain motion vector points to a third reference block in a third reference image and a fourth reference block in a fourth reference image. The method includes: Based on the first reference block in the first reference image and the second reference block in the second reference image, determine the first motion vector offset of the first chain motion vector; Based on the third reference block in the third reference image and the fourth reference block in the fourth reference image, the second motion vector offset of the second chain motion vector is determined; Based on the first chained motion vector with the first motion vector offset, a first prediction for the current block is generated; Based on the second chained motion vector with the second motion vector offset, a second prediction for the current block is generated; and The current block is reconstructed based on the first prediction and the second prediction.

8. The method according to claim 6, characterized in that, The first motion vector is a first chain motion vector in a first direction in the display sequence, and the second motion vector is a second chain motion vector in a second direction in the display sequence. The first chain motion vector points to a first reference block in a first reference image and a second reference block in a second reference image, and the second chain motion vector points to a third reference block in a third reference image and a fourth reference block in a fourth reference image. The method includes: Based on the first reference block in the first reference image, the second reference block in the second reference image, the third reference block in the third reference image, and the fourth reference block in the fourth reference image, calculate the cost associated with the candidate motion vector offset; Based on the cost value, a motion vector offset is determined from the candidate motion vector offsets; Based on the first chained motion vector with the motion vector offset, generate the first prediction for the current block; A second prediction for the current block is generated based on a second chained motion vector having the opposite value of the motion vector offset; and The current block is reconstructed based on the first prediction and the second prediction.

9. The method according to claim 8, characterized in that, The calculation of the cost value includes: For one of the candidate motion vector offsets: The candidate motion vector offsets are applied to the first reference block and the second reference block respectively to obtain the first candidate refined reference block and the second candidate refined reference block respectively; The first offspring value is calculated based on the difference between the first candidate refined reference block and the second candidate refined reference block; The inverse values ​​of the candidate motion vector offsets are applied to the third reference block and the fourth reference block, respectively, to obtain the third candidate refined reference block and the fourth candidate refined reference block, respectively. The second offspring value is calculated based on the difference between the third candidate refined reference block and the fourth candidate refined reference block; and The cost associated with the candidate motion vector offset is calculated as the sum of the first sub-sub-value and the second sub-sub-value.

10. The method according to any one of claims 1 to 9, characterized in that, It also includes at least one of the following: When at least one of the block shape and block size meets the requirements, the refined chain vector is determined; and / or When the difference between the first reference block and the second reference block meets the requirements, the chain vector is refined.

11. A video encoding method, characterized in that, include: A fine-grained chain vector is applied to the current block in the current image, the chain vector including at least a first sub-vector and a second sub-vector, the first sub-vector pointing to a first reference block relative to the current block, and the second sub-vector pointing to a second reference block relative to the first reference block; Based on the first reference block and the second reference block, at least one vector offset is determined; as well as The current block is encoded based on the chained vector and the vector offset.

12. The method according to claim 11, characterized in that, The chain vector is at least one of a chain motion vector and a chain block vector.

13. The method according to any one of claims 11 to 12, characterized in that, The chained vector is a chained motion vector, which includes at least a first sub-motion vector and a second sub-motion vector. The first sub-motion vector points relative to the current block to the first reference block in the first reference image, and the second sub-motion vector points relative to the first reference block to the second reference block in the second reference image. Determining the vector offset includes: The search is performed within the search range of candidate motion vector offsets; Based at least on the first reference block in the first reference image and the second reference block in the second reference image, calculate the cost associated with the candidate motion vector offset, respectively; and A motion vector offset is determined based on the cost associated with the candidate motion vector offset.

14. The method according to claim 13, characterized in that, The first reference image and the second reference image are located on the same side of the current image in the display order, and the calculation includes: A candidate motion vector offset is applied to the first reference block to obtain a first candidate refined reference block in the first reference image; The candidate motion vector offset is applied to the second reference block to obtain a second candidate refined reference block in the second reference image; and Based on the difference between the first candidate refined reference block and the second candidate refined reference block, the cost associated with the candidate motion vector offset is calculated.

15. A method for processing visual media data, characterized in that, The method includes: The bitstream of visual media data is processed according to format rules, where: The bitstream includes encoded information of the current block in the current image. This encoded information indicates prediction of the current block based on a chained vector, where the chained vector includes at least a first sub-vector and a second sub-vector. The first sub-vector points to a first reference block relative to the current block, and the second sub-vector points to a second reference block relative to the first reference block. The format rules stipulate that: Based on the first reference block and the second reference block, at least one vector offset is determined; and The current block is reconstructed based on the chained vector and the vector offset.

Images