Cross-component prediction with geometric partition mode and geometric partition weight adaptation

Abstract

The present disclosure provides methods and apparatuses for video decoding and video encoding, and methods of processing visual media data. A method of video decoding includes receiving coded information indicating that a current block is coded using GPM with multiple sets of blending widths; determining a set of blending widths to be applied to the current block from the multiple sets of blending widths based on block size information and GPM information of the current block; determining a blending width from the determined set of blending widths; and reconstructing the current block according to the GPM and the determined blending width.

Description

[0001] Related applications This application claims priority to U.S. Patent Application No. 19 / 182,375, filed April 17, 2025, entitled “Cross-component prediction in geometric partition mode for chromablock,” which claims priority to U.S. Provisional Application No. 63 / 635,974, filed April 18, 2024, entitled “Cross-component prediction in geometric partition mode for chromablock,” and U.S. Provisional Application No. 63 / 635,974, filed April 20, 2024, entitled “Geometric partition weight adaptation based on geometric partition mode information and block information.” Priority is claimed in U.S. Provisional Application No. 63 / 636,780, entitled "Affine motion vector predictor and merge candidate construction by using intra template-matching," filed April 24, 2024. The entire disclosure of these prior applications is incorporated herein by reference. Technical Field

[0002] This invention describes various aspects generally related to video encoding and decoding. Background Technology

[0003] The background description provided in this application is intended to generally present the context of the invention. To the extent that the work described in this background section is intended, the work of the currently named inventors and aspects of the description that may not have otherwise qualified as prior art at the time of filing are neither explicitly nor implicitly considered as prior art to the invention.

[0004] Image / video compression can facilitate the transfer of image / video data across different devices, storage devices, and networks with minimal quality loss. In some examples, video codec techniques can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-frame prediction, which can compress images based on spatial redundancy. For example, intra-frame prediction can use reference data from the currently reconstructed image to predict samples. In another example, a video codec can use a technique called inter-frame prediction, which can compress images based on temporal redundancy. For example, inter-frame prediction can utilize motion compensation to predict samples in the current image based on previously reconstructed images. Motion compensation can be indicated by a motion vector (MV). Summary of the Invention

[0005] Various aspects of the present invention include methods and apparatus for video encoding / decoding.

[0006] Various aspects of the present invention provide a video decoding method, wherein: encoded information is received, indicating that the current block is encoded using a geometric partition mode (GPM) with multiple sets of blend widths; based on the block size information and GPM information of the current block, a blend width set to be applied to the current block is determined from the multiple blend width sets; a blend width is determined from the determined blend width set; and the current block is reconstructed based on the GPM and the determined blend width.

[0007] Aspects of the present invention also provide a video decoding method, wherein encoded information is received. The encoded information instructs chroma blocks to be encoded based on corresponding luma blocks using cross-component prediction (CCP) and a multi-model filter, wherein the luma blocks are predicted by GPM and include a first partition and a second partition separated by geometrical partitioning edges of the GPM. A multi-model filter for the chroma blocks is determined. The chroma blocks include a first partition and a second partition separated by geometrical partitioning edges. The multi-model filter includes one of the following: (i) a first filter coefficient determined based on the first partition of the luma block and the first partition of the chroma block, and a second filter coefficient determined based on the second partition of the luma block and the second partition of the chroma block; and (ii) a first filter coefficient determined based on a first luma template of the luma block and a first chroma template of the chroma block, and a second filter coefficient determined based on a second luma template of the luma block and a second chroma template of the chroma block. The luma template of the luma block is adjacent to the luma block and includes reconstructed luma samples. The luma template includes a first luma template and a second luma template separated by extensions of geometrical partitioning edges extending into the luma template. The chroma template of the chroma block is adjacent to the chroma block and includes reconstructed chroma samples. The chroma template includes a first chroma template and a second chroma template separated by extensions extending into the chroma template by geometric dividing edges. The video decoding method includes reconstructing the chroma block based on a multi-model filter.

[0008] Various aspects of the present invention also provide a video decoding method, wherein encoded information is received, indicating that a current block in a current image is encoded using an affine pattern. A template-matching (TM) process is applied to determine a reference block in the current image. Based on motion vector information associated with multiple corners of the reference block, the control point motion vector (CPMV) of the current block is determined. Based on the determined CPMVs of the current block, the current block is reconstructed.

[0009] The present invention also provides an apparatus for video decoding. The apparatus for video decoding includes processing circuitry configured to implement any of the described methods for video decoding.

[0010] The present invention also provides a video coding method, wherein a set of mixing widths to be applied to the current block is determined from multiple sets of mixing widths based on block size information and GPM information of the current block. A mixing width is determined from the determined set of mixing widths, and the current block is encoded according to the GPM and the determined mixing width.

[0011] The present invention also provides a video coding method in which a multi-model filter for chroma blocks is determined. A chroma block includes a first partition and a second partition separated by geometrical segmentation edges. The chroma block is encoded based on a corresponding luma block using CCP and the multi-model filter, and the luma block is predicted by GPM and includes a first partition and a second partition separated by geometrical segmentation edges of the GPM. The multi-model filter includes one of the following: (i) first filter coefficients determined based on the first partition of the luma block and the first partition of the chroma block, and second filter coefficients determined based on the second partition of the luma block and the second partition of the chroma block; and (ii) first filter coefficients determined based on a first luma template of the luma block and a first chroma template of the chroma block, and second filter coefficients determined based on a second luma template of the luma block and a second chroma template of the chroma block. The luma template of the luma block is adjacent to the luma block and includes reconstructed luma samples. The luma template includes a first luma template and a second luma template separated by extensions extending into the luma template by geometrical segmentation edges. The chroma template of the chroma block is adjacent to the chroma block and includes reconstructed chroma samples. The chroma template includes a first chroma template and a second chroma template, separated by extensions formed by geometric dividing edges. The video coding method includes encoding chroma blocks according to a multi-model filter.

[0012] The present invention also provides a video coding method in which a TM process is applied to determine a reference block in the current image for a current block in the current image. Based on motion vector information associated with multiple corners of the reference block, the CPMVs of the current block are determined. Based on the determined CPMVs of the current block, the current block is encoded using an affine pattern.

[0013] Aspects of the present invention also provide an apparatus for video encoding. The apparatus for video encoding includes processing circuitry configured to implement any of the described methods for video encoding.

[0014] Aspects of the present invention also provide a non-transitory computer-readable storage medium for storing instructions that, when executed by a computer, cause the computer to perform any of the described methods for video decoding / encoding. Attached Figure Description

[0015] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which: Figure 1 An example block diagram of a communication system (100) is shown schematically.

[0016] Figure 2 An example block diagram of a decoder is shown schematically.

[0017] Figure 3An example block diagram of an encoder is shown schematically.

[0018] Figure 4A An example of GPM applied to a block according to one aspect of the present invention is shown.

[0019] Figure 4B This illustrates a predetermined number of angles distributed in the range of 0° to 360° for GPM according to one aspect of the invention.

[0020] Figure 4C A plurality of angle-corresponding dividing edges for GPM are shown according to an embodiment of the present invention.

[0021] Figure 4D An example of a GPM mixing process applied to the current block according to an embodiment of the present invention is shown.

[0022] Figure 5 An example of a CCP according to one aspect of the present invention is shown.

[0023] Figure 6 An example of CCP on a geometrically partitioned block according to one aspect of the invention is shown.

[0024] Figure 7 An example of CCP on a geometrically partitioned block according to one aspect of the invention is shown.

[0025] Figure 8 An example of decoder side motion vector refinement (DMVR) based on bilateral matching (BM) according to one aspect of the invention is shown.

[0026] Figure 9A An affine motion field of a block (902) described by motion information (4 parameters) from two control points is shown according to one aspect of the invention.

[0027] Figure 9B An affine motion field of a block (904) described by three CPMVs (6 parameters) according to one aspect of the invention is shown.

[0028] Figures 10 to 11 An example of using an affine model of TM in an encoded region of a current image according to one aspect of the invention is shown.

[0029] Figure 12 An example of CPMV derivation for the current block based on motion information at the corner of a reference block, according to one aspect of the invention, is shown.

[0030] Figure 13 A flowchart outlining the decoding process according to some aspects of the present invention is shown.

[0031] Figure 14 A flowchart outlining the decoding process according to one aspect of the present invention is shown.

[0032] Figure 15 A flowchart outlining the decoding process according to one aspect of the present invention is shown.

[0033] Figure 16 It is a schematic diagram of a computer system based on one aspect. Detailed Implementation

[0034] Figure 1 Block diagrams of some example video processing systems (100) are shown. The video processing system (100) is an example application of the disclosed subject matter, namely, video encoders and video decoders in a streaming media environment. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming media services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0035] The video processing system (100) includes an acquisition subsystem (113) that may include a video source (101), such as a digital camera, for creating, for example, an uncompressed video image stream (102). In one example, the video image stream (102) includes samples captured by a digital camera. The video image stream (102) is depicted as a thick line compared to encoded video data (104) (or encoded video bitstream) to emphasize the high data volume of the video image stream, which may be processed by an electronic device (120) including a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to enable or implement aspects of the disclosed subject matter described in more detail below. Compared to the video image stream (102), the encoded video data (104) (or encoded video bitstream) is depicted as a thin line to emphasize the lower data volume of the encoded video data, which can be stored on a streaming media server (105) for future use. At least one streaming media client subsystem, such as... Figure 1Client subsystems (106) and (108) can access a streaming media server (105) to retrieve copies (107) and (109) of encoded video data (104). Client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and creates an outgoing video image stream (111) that can be displayed on a display (112) (e.g., a screen) or other display device (not depicted). In some streaming media systems, the encoded video data (104), (107), and (109) (e.g., video streams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In one example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed topics are applicable in the context of VVC.

[0036] It should be noted that electronic devices (120) and (130) may include other components (not shown). For example, electronic device (120) may also include a video decoder (not shown), while electronic device (130) may also include a video encoder (not shown).

[0037] Figure 2 An example block diagram of a video decoder (210) is shown. The video decoder (210) may be included in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., receiving circuitry). The video decoder (210) may replace... Figure 1 The video decoder (110) in the example is used.

[0038] The receiver (231) can receive, for example, at least one encoded video sequence included in the bitstream, which is decoded by the video decoder (210). In one aspect, one encoded video sequence is received at a time, and the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequences can be received from a channel (201), which can be a hardware / software link connected to a storage device storing the encoded video data. The receiver (231) can receive the encoded video data and other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective user entities (not depicted). The receiver (231) can separate the encoded video sequences from other data. To combat network jitter, a buffer memory (215) can be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other applications, the buffer memory (215) may be external to the video decoder (210) (not depicted). In other applications, a buffer memory (not depicted) may exist external to the video decoder (210), for example, to combat network jitter, and another buffer memory (215) may exist internally to the video decoder (210), for example, to handle playback timing. The buffer memory (215) may not be necessary or may be small when the receiver (231) receives data from a store / forward device with sufficient bandwidth and controllability or from an isosynchronous network. For use on best-effort packet networks such as the Internet, a buffer memory (215) may be required, which may be relatively large and advantageously have an adaptive size, and may be at least partially implemented in the operating system or in a similar element (not depicted) external to the video decoder (210).

[0039] The video decoder (210) may include a parser (220) for reconstructing symbols (221) from the encoded video sequence. Categories of those symbols include information for managing the operation of the video decoder (210) and may include information for controlling a display device, such as a component of a non-electronic device (230) but which may be coupled to the electronic device (230) (e.g., a display screen). Figure 2As shown in the diagram. Control information for one or more display devices may be in the form of Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not depicted). The parser (220) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (220) may extract a subgroup parameter set from the encoded video sequence for at least one pixel subgroup in the video decoder based on at least one parameter corresponding to a group. The subgroup may include Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (220) can also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0040] The parser (220) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) in order to create symbols (221).

[0041] Depending on the type of the encoded video image or its portions (e.g., inter-frame and intra-frame images, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (221) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (220). For clarity, the flow of such subgroup control information between the parser (220) and the multiple units described below is not depicted.

[0042] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In practical implementations under commercial constraints, many of these units interact closely with each other and may be partially integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0043] The first unit is the scaler / inverse transform unit (251). The scaler / inverse transform unit (251) receives from the parser (220) quantization transform coefficients as one or more symbols (221) and control information, including the transform to be used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (251) can output blocks containing sample values, which can be input into the aggregator (255).

[0044] In some cases, the output samples of the scaler / inverse transform unit (251) may relate to intra-coded blocks. Intra-coded blocks do not use predictive information from previously reconstructed images, but instead may use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-image prediction unit (252). In some cases, the intra-image prediction unit (252) uses surrounding reconstructed information obtained from the current image buffer (258) to generate blocks of the same size and shape as the block being reconstructed. For example, the current image buffer (258) is used to buffer partially reconstructed current images and / or fully reconstructed current images. In some cases, the aggregator (255) adds the predictive information already generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a per-sample basis.

[0045] In other cases, the output samples of the scaler / inverse transform unit (251) may involve inter-frame coded and possibly motion-compensated blocks. In this case, the motion compensation prediction (MCP) unit (253) can access the reference image memory (257) to obtain samples for prediction. After motion compensation of the obtained samples according to the block-related symbols (221), these samples can be added to the output of the scaler / inverse transform unit (251) (referred to in this case as residual samples or residual signals) by the aggregator (255) to generate output sample information. The address of the predicted samples obtained by the MCP unit (253) in the reference image memory (257) can be controlled by motion vectors available to the MCP unit (253) in the form of symbols (221), which may have, for example, X, Y, and reference image components. Motion compensation may also include interpolation of sample values ​​obtained from the reference image memory (257) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0046] The output samples of the aggregator (255) can be employed by various loop filtering techniques in the loop filter unit (256). Video compression techniques may include loop filtering techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video stream) and available as symbols (221) from the parser (220) for use in the loop filter unit (256). Video compression may also respond to metadata obtained during the decoding of a previous (in decoding order) portion of the encoded image or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0047] The output of the loop filter unit (256) can be a sample stream, which can be output to the display device (212) and stored in the reference image memory (257) for future inter-frame image prediction.

[0048] Some encoded images can be used as reference images for future predictions after they have been fully reconstructed. For example, after the encoded image corresponding to the current image has been fully reconstructed and the encoded image has been identified as a reference image (by, for example, a parser (220)), the current image buffer (258) can become part of the reference image memory (257), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0049] The video decoder (210) can perform decoding operations according to a predetermined video compression technology or standard, such as ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax of the video compression technology or standard as well as the configuration file documented therein, if the encoded video sequence follows both the syntax of the video compression technology or standard and the configuration file documented therein. Specifically, the configuration file may select certain tools from all available tools in the video compression technology or standard as the only tools available under that configuration file. For compliance, the complexity of the encoded video sequence must also be within the limits defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the HRD buffer management metadata indicated by signaling in the encoded video sequence.

[0050] In one aspect, the receiver (231) may receive supplemental (redundant) data along with the encoded video. This supplemental data may be part of one or more encoded video sequences. The supplemental data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The supplemental data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0051] Figure 3 An example block diagram of a video encoder (303) is shown. The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., transmission circuitry). The video encoder (303) may replace Figure 1 The video encoder (103) in the example is used.

[0052] The video encoder (303) can obtain data from the video source (301) (non- Figure 3 In one example, an electronic device (320) receives video samples, and a video source (301) can capture one or more video images to be encoded by a video encoder (303). In another example, the video source (301) is part of the electronic device (320).

[0053] A video source (301) can provide a sequence of source videos to be encoded by a video encoder (303) in the form of a digital video sample stream. This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 YCrCb, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (301) can be a storage device storing previously prepared video. In a video conferencing system, the video source (301) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual images that, when viewed sequentially, produce an animated effect. The images themselves can be organized as spatial pixel arrays, where each pixel can include at least one sample, depending on the sampling structure, color space, etc., used. Samples are described in detail below.

[0054] According to one aspect, the video encoder (303) can encode and compress images of a source video sequence into an encoded video sequence (343) in real time or as needed, according to any other time constraints. Implementing an appropriate encoding rate is a function of the controller (350). In some aspects, the controller (350) controls and is functionally coupled to other functional units described below. For clarity, the coupling is not depicted. Parameters set by the controller (350) may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, GOP layout, maximum motion vector search range, etc. The controller (350) can be configured to have other suitable functions related to the video encoder (303) optimized for a specific system design.

[0055] In some respects, the video encoder (303) is configured to operate in an encoding loop. Simply put, in one example, the encoding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and one or more reference images), and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to that created by the (remote) decoder. The reconstructed sample stream (sample data) is input to a reference image memory (334). Since decoding the symbol stream produces a bit-by-bit accurate result independent of the decoder's location (local or remote), the contents of the reference image memory (334) are also bit-by-bit accurate between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction section are exactly the same as the sample values ​​"seen" by the decoder during decoding when using the prediction. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related fields.

[0056] The operation of the “local” decoder (333) can be the same as that of a “remote” decoder such as a video decoder (210), which has already been described above. Figure 2 A detailed description has been provided. However, a brief additional reference is also available. Figure 2 When symbols are available and the entropy encoder (345) and parser (220) are able to encode / decode the symbols into an encoded / decoded video sequence without loss, the entropy decoding portion of the video decoder (210), including the buffer (215) and parser (220), may not be fully implemented in the local decoder (333).

[0057] In one respect, decoder techniques other than parsing / entropy decoding, which exist in the decoder, exist in the corresponding encoder with the same or substantially the same functional form. Therefore, the disclosed subject focuses on decoder operation. The description of encoder techniques can be simplified, as encoder techniques are the reverse operation of the comprehensively described decoder techniques. More detailed descriptions are only required in certain areas, and these descriptions are provided below.

[0058] In some examples, during operation, the source encoder (330) may perform motion-compensated predictive coding, i.e., predictively coding the input image against at least one previously encoded image designated as a “reference image” in the reference video sequence. In this way, the coding engine (332) encodes the differences between pixel blocks of the input image and pixel blocks of one or more reference images, which may be selected as one or more predictive references for the input image.

[0059] The local video decoder (333) can decode encoded video data of an image that can be designated as a reference image, based on symbols created by the source encoder (330). Advantageously, the operation of the encoding engine (332) can be a lossy process. When the encoded video data can be decoded by the video decoder (333), Figure 3 During decoding at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process, which can be performed by the video decoder on the reference image, and can store the reconstructed reference image in the reference image memory (334). In this way, the video encoder (303) can store copies of the reconstructed reference images locally that share the same content (no transmission errors) as the reconstructed reference images to be obtained by the remote video decoder.

[0060] The predictor (335) can perform a prediction search against the encoding engine (332). That is, for a new image to be encoded, the predictor (335) can search in the reference image memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as suitable prediction references for the new image. The predictor (335) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (335), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (334).

[0061] The controller (350) can manage the encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for video data encoding.

[0062] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder (345). By applying lossless compression to the symbols generated by the various functional units according to techniques such as Huffman coding, variable-length coding, and arithmetic coding, the entropy encoder (345) converts these symbols into an encoded video sequence.

[0063] The transmitter (340) may buffer one or more encoded video sequences created by the entropy encoder (345) in preparation for transmission via a communication channel (360), which may be a hardware / software link connected to a storage device that will store the encoded video data. The transmitter (340) may combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0064] The controller (350) can manage the operation of the video encoder (303). During encoding, the controller (350) can assign a specific encoded image type to each encoded image, which may affect the encoding techniques applicable to the corresponding image. For example, one of the following image types can typically be assigned to an image.

[0065] Intraframe images (I-images) are images that are encoded and decoded without using any other images in the sequence as prediction sources. Some video codecs support different types of intraframe images, including, for example, Independent Decoder Refresh (IDR) images.

[0066] Predictive images (P-images) can be images that are encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using motion vectors and reference indices.

[0067] A bidirectional predictive image (B-image) can be an image encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using two motion vectors and a reference index. Similarly, a multi-predictive image can reconstruct a single block using more than two reference images and associated metadata.

[0068] The source image can typically be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples each) and encoded block by block. A block can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignments of the corresponding images applied to the block. For example, blocks of an I-image can be non-predictively coded, or these blocks can be predictively coded with reference to already coded blocks of the same image (spatial prediction or intra-frame prediction). Pixel blocks of a P-image can be predictively coded via spatial prediction or via temporal prediction with reference to a previously coded reference image. Blocks of a B-image can be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference images.

[0069] The video encoder (303) can perform encoding operations according to predetermined video coding techniques or standards, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0070] In one respect, the transmitter (340) may transmit additional data along with the encoded video. The source encoder (330) may transmit such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0071] Video can be captured as multiple source images (video images) in a time-series manner. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In one example, the specific image being encoded / decoded is called the current image, which is divided into blocks. Where a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image and, in the case of multiple reference images, may have a third dimension identifying the reference images.

[0072] In some aspects, inter-frame image prediction can utilize bidirectional prediction techniques. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly representing past and future images in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. A block can be predicted using a combination of the first and second reference blocks.

[0073] Furthermore, merging mode techniques can be used to improve coding efficiency in inter-frame image prediction.

[0074] According to some aspects of the invention, prediction is performed on a block-by-block basis, such as inter-frame image prediction and intra-frame image prediction. For example, according to the HEVC standard, images in a video image sequence are divided into coding tree units (CTUs) for compression. CTUs in an image have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Typically, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree-divided into one or more coding units. For example, a 64×64 pixel CTU can be divided into one 64×64 pixel coding unit, four 32×32 pixel coding units, or sixteen 16×16 pixel coding units. In one example, each coding unit is analyzed to determine the prediction type of that coding unit, such as inter-frame prediction or intra-frame prediction. Based on temporal and / or spatial predictability, coding units are divided into at least one prediction unit. Typically, each prediction unit comprises one luma prediction block (PB) and two chroma prediction blocks. In one aspect, prediction operations in encoding / decoding are performed on a block-by-block basis. Using the luma prediction block as an example, a prediction block comprises a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0075] It should be noted that the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using any suitable technology. In one aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using at least one integrated circuit (IC). In another aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using at least one processor that executes software instructions.

[0076] Video coding has been widely used in many applications such as broadcasting, video recording, and video streaming. Various emerging video coding standards, such as H.264, H.265 / HEVC, H.266 / VVC, and AV1, are applied to video applications. A hybrid video codec may include coding modules, intra-frame prediction, inter-frame prediction, transform coding, quantization, entropy coding, and post-loop filters.

[0077] In one aspect, geometric partitioning prediction is a prediction model that uses geometric dividing edges to form two different geometric partitions. Geometric partitioning prediction can be referred to as GPM. Figure 4A An example of GPM applied to a block (401) according to one aspect of the invention is shown. The block (401) can be divided into partitions (411)-(412) by a geometric segmentation edge (402). Each partition can be encoded using an intra-prediction mode, an inter-prediction mode, a block vector (BV)-based prediction mode (such as an intra-block copy (IBC) mode), an intra-template matching prediction (IntraTMP) mode, etc. In the BV-based prediction mode, the current block in the current image is predicted. A reference block within the reconstructed region in the current image can be determined, and the BV can indicate (e.g., point to) the position of the reference block relative to the current block. In one example, the geometric segmentation edge (402) extends into the template (421) of the block (401). In one example, signaling in the bitstream can indicate a combination of the partitioning modes of the two partitions (411)-(412) with a prediction mode (e.g., an intra-prediction mode). In one example, signaling can indicate three syntaxes to indicate which partitioning mode and which prediction modes (e.g., intra-frame prediction modes) are applied to the two partitions (411)-(412).

[0078] GPM can be used for inter-frame prediction, such as in VVC. In one example, GPM is applied only to coding units of 8×8 or larger. In one aspect, when using GPM, the coding unit can be split into two geometric partitions (also called geometric partitions or partitions) by using a predetermined number (e.g., 64) of different partitioning schemes or one of the GPM partitioning modes. A geometric partition index (or GPM partitioning mode index) can be used to indicate the partitioning scheme or GPM partitioning mode, such as one of 64 different partitioning schemes. Different partitioning schemes can be distinguished by a predefined number (e.g., 24) of angles relative to the center of the coding unit (e.g., non-uniform quantization between 0° and 360°) and at most a predefined number (e.g., 4) of partitioning edges (also called geometric partitioning edges), such as... Figures 4B to 4CAs shown.

[0079] According to an embodiment of the present invention, Figure 4B A predefined number (or multiple angles) distributed between 0° and 360° is shown, such as the 24 angles supported in VVC. These multiple angles can be indicated by corresponding angle indices 0-23. According to an embodiment of the invention, Figure 4C This illustrates multiple partition edges corresponding to one of several angles (e.g., four partition edges indicated by corresponding indices (idx) 0-3), such as the possible partition edges supported for angle index 3. A set of GPM partitioning patterns that can be used to encode blocks can be based on... Figures 4B to 4C The shown is an appropriate combination of multiple angles and one or more associated dividing edges.

[0080] In one embodiment, each geometric partition in the coding unit performs inter-frame prediction using corresponding motion information. In one example, each partition can only perform unidirectional prediction; for example, each partition has a corresponding motion vector and a corresponding reference index. Unidirectional prediction motion constraints can be applied to ensure that only two motion-compensated predictions are used for each coding unit, which is the same as when applying bidirectional prediction to the entire coding unit. In some examples, bidirectional prediction is applied to partitions within the coding unit.

[0081] In one embodiment, inter-frame prediction and another prediction (e.g., intra-frame prediction) are used for geometric partitions in the coding unit.

[0082] If GPM is used for a coding unit, signaling can indicate information about the geometric partitioning index of the coding unit and the prediction information for the two geometric partitions within the coding unit. In one example, if inter-frame prediction is performed for each geometric partition in the coding unit, the prediction information includes two merge indices. If inter-frame and intra-frame prediction are used for the geometric partitions in the coding unit, the prediction information may include a merge index and an index indicating the intra-frame prediction mode.

[0083] After predicting two geometric partitions, a blending process with adaptive weights (or GPM blending process) can be used to adjust the sample values ​​within the blended region (including samples along the partition edges). The size of the blended region can be determined by... Figure 4C The shown mixing width θ Indicator. Mixed width θ It can be the width of the mixed zone measured perpendicular to the dividing edge (452).

[0084] In one embodiment, the mixed width θIt is fixed, for example, for encoding units with different content, such as natural content, screen content, a mixture of one or more natural contents and one or more screen contents, etc. In one embodiment, the mixing width is... θ It is adaptive, for example, selected from a mixed width set (also known as a width candidate list).

[0085] Figure 4C An exemplary GPM mixing process applied to an encoding unit (450) according to an embodiment of the present invention is illustrated. The encoding unit (450) can be divided into geometric partitions (461)-(462) by dividing edges (452). In one example, a first prediction mode and a second prediction mode are applied to predict samples in the encoding unit (450) as P0 and P1, respectively. The first prediction mode and the second prediction mode may include one or more appropriate prediction modes, such as one or more inter-frame prediction modes, intra-frame prediction modes, IBC modes, IntraTMP modes, and / or similar modes.

[0086] The partition edge (452) can be indexed with the angle (e.g., Figure 4B The angle index (10 or 22) in the code corresponds to the angle orientation. The partition edge (452) and the angle index can correspond to the geometric partition index of the coding unit (450).

[0087] The blending region or blending area (451) may include samples along the dividing edge (452). The blending region (451) may include samples at a distance (or blending width) from the dividing edge (452). θ The samples are within the range of the boundary (471)-(472) of the mixed region (451). The boundary (471)-(472) of the mixed region (451) is parallel to the dividing edge (452) and is separated from the dividing edge (452) by a distance. θ In one example, the blending region (451) includes a first blending region (465) and a second blending region (466) separated by a dividing edge (452). The first blending region (465) may be located within the partition (461), and the second blending region (466) may be located within the partition (462).

[0088] The samples in the coding unit (450) can be determined based on a mixing process, for example, by a weighted sum P.

[0089] P = (1-W)×P1 + W×P0 (Formula 1) P0 and P1 can represent the predicted values ​​of samples based on the first and second prediction modes, respectively. The displacement of the sample in the coding unit (450) relative to the partitioning edge (452) is also considered. d ( x c ,y cThe weights (W) of the samples in the coding unit (450) can be determined.

[0090] A hybrid mask can be applied to the coding unit (450), and the weights (or weighted values) in the hybrid mask... ω xc,yc This can be given by the following ramp function. In one example, W is... ω xc,yc / 8 .

[0091] , Formula 2 When the sample ( x c ,y c When the displacement is located within partition (462) and outside the second mixing region (466), d ( x c ,y c Less than or equal to - θ ,and ω xc,yc And W is 0. Therefore, samples located outside the mixed region (451) in partition (462) can be predicted as P1 based on the second prediction mode.

[0092] When the sample ( x c ,y c When the displacement is located within partition (461) and outside the first mixing region (465), d ( x c ,y c (greater than or equal to) θ ,and ω xc,yc The value is 8 and W is 1. Therefore, samples located outside the mixing region (451) in partition (461) can be predicted as P0 based on the first prediction mode. In one example, mixing is not used for samples outside the mixing region (451).

[0093] When the sample ( x c ,y c When located in the mixed region (451), the displacement d ( x c ,y c Between - θ and θ Between, and ωxc,yc Based on displacement d ( x c ,y c The sample in the mixed region (451) can be predicted as a weighted sum of P0 and P1, as described in Equation 1.

[0094] In one example Fixed at 2 pixels (pel), for example in the current VVC design, the ramp function ω xc,yc Quantifiable as ω m,n .

[0095] Formula 3 In one example, d(m, n) is 16× d ( x c ,y c ).

[0096] The mixture result P (e.g., the predicted sample value of the coding unit (450)) may include the predicted signal for the coding unit (450) (e.g., the entire coding unit (450)). As in other prediction modes, transformation and quantization processes may be applied to the coding unit (450). The motion field of the coding unit (450) predicted using GPM may be stored.

[0097] The partitioning pattern in GPM can indicate the geometric split edges and the geometric partitions formed by using these edges. A set of blend widths, including multiple blend widths, can be used. Signaling indices can be used to indicate which blend width from the blend width set should be used to blend two samples from two partitions within the blending region along the geometric split edge.

[0098] The blend width set can include any number of blend widths, for example, 5 blend widths { θ 1, θ 2, θ 3, θ 4, θ 5}. In one example, the mixed width set includes In one example, the mixed width set includes { τ / 4, τ / 2, τ ,2 τ , 4 τ},in, τ It is a positive number. When τ When the value is 2, the mixed width set includes .

[0099] According to one aspect of the invention, more than one or more mixed width sets can be used in GPM. Based on information such as block width, block height, and GPM information (e.g., partitioning mode), a mixed width set to be applied to the current block encoded in GPM can be determined from multiple mixed width sets. In one aspect, based on block size information such as indicating block width, block height, and aspect ratio between block width and block height, and GPM information such as indicating partitioning mode, a mixed width set to be applied to the current block can be determined from multiple mixed width sets. A mixed width can be determined from the determined mixed width set, and the current block can be reconstructed based on the GPM and the determined mixed width.

[0100] Multiple blend width sets can include any number of blend width sets, and each of the multiple blend width sets can include any number of blend widths. In one example, two blend width sets are used for GPM. For example, the multiple blend width sets include a first blend width set (e.g., blend width set #1) and a second blend width set (e.g., blend width set #2). In one aspect, the minimum blend width in the first blend width set is less than the minimum blend width in the second blend width set. In one example, the first blend width set is {τ / 4, τ / 2, τ, 2τ, 4τ}, and the second blend width set is { τ / 2, τ , 2 τ , 4 τ , 8 τ Furthermore, the minimum blend width in the first blend width set is τ / 4, which is less than the minimum blend width τ / 2 in the second blend width set. In one aspect, the maximum blend width in the first blend width set (e.g., 4τ) is less than the maximum blend width in the second blend width set (e.g., 8τ).

[0101] In one aspect, when the block width and block height satisfy a first condition, the mixed width set of the current block can be determined as mixed width set #1. When the block width and block height satisfy a second condition, the mixed width set of the current block can be determined as mixed width set #2. When the block width and block height do not satisfy either the first or the second condition, based on the partitioning mode in GPM and the aspect ratio between the block height and block width, the mixed width set can be determined as either the first mixed width set or the second mixed width set. In one example, the aspect ratio is the ratio of the block width to the block height.

[0102] In one respect, the first and second conditions can be based on i) a comparison between block height and block width and ii) a threshold T.

[0103] In one example, the first condition includes: both the block width and the block height are less than or equal to the threshold T, and the second condition includes: both the block width and the block height are greater than or equal to the threshold T.

[0104] In one example, the first condition includes: both the block width and the block height are less than or equal to the threshold T, and the second condition includes: both the block width and the block height are greater than or equal to the threshold T.

[0105] In one example, the first condition includes: each of the block width and block height is less than or equal to a threshold T, and the second condition includes: each of the block width and block height is greater than a threshold T.

[0106] In one example, the first condition includes: each of the block width and block height is less than a threshold T, and the second condition includes: each of the block width and block height is greater than or equal to the threshold T.

[0107] More specifically, in other cases, such as when the block width and block height do not satisfy the first condition and the second condition, a mixed width set #2 may be selected if one of the following two conditions is met: i) the aspect ratio is greater than 1 and a near-vertical partitioning pattern (also known as near-vertical GPM) is used, and ii) the aspect ratio is less than 1 and a near-horizontal partitioning pattern (also known as near-horizontal GPM) is used. Otherwise, for example, when neither condition (i) nor (ii) is met, a mixed width set #1 may be selected.

[0108] In one example, the block width and block height do not satisfy either the first or second condition, and the aspect ratio is the ratio of the block width to the block height. The two conditions can include: i) an aspect ratio greater than 1, and the partitioning pattern in the GPM is a near-vertical partitioning pattern; and ii) an aspect ratio less than 1, and the partitioning pattern in the GPM is a near-horizontal partitioning pattern. When either condition (i) or (ii) is met, the blended width set of the current block can be determined as the second blended width set (e.g., { τ / 2, τ , 2 τ , 4 τ , 8 τ}), and when neither of the two conditions is met, the mixed width set can be determined as the first mixed width set (e.g., {τ / 4, τ / 2, τ, 2τ, 4τ}).

[0109] In one example, a near-horizontal partitioning mode indicates that the geometrical division angle between the geometrical dividing edge and the horizontal line is less than and / or equal to a first predefined angle. In one example, the partitioning mode in GPM is a near-horizontal partitioning mode when the geometrical division angle between the geometrical dividing edge and the horizontal line is less than or equal to the first predefined angle.

[0110] In one example, the near-vertical partitioning mode indicates that the geometrical division angle between the geometrical dividing edge and the vertical line is less than and / or equal to a second predefined angle. In one example, the partitioning mode in GPM is the near-vertical partitioning mode when the geometrical division angle between the geometrical dividing edge and the vertical line is less than or equal to the second predefined angle.

[0111] In one example, the first and second predefined angles can be the same for all block sizes.

[0112] In one example, the first predefined angle and the second predefined angle can vary depending on the block size.

[0113] In one example, the first predefined angle and the second predefined angle can differ based on the aspect ratio r (r is the ratio of the block width to the block height) or 1 / r. In another example, for the same aspect ratio r or 1 / r, the first predefined angle and the second predefined angle are the same.

[0114] According to one aspect of the invention, two blended width sets are used for GPM. In one example, the first blended width set, or blended width set #1, is { τ / 4, τ / 2, τ , 2 τ , 4 τ}, and the second mixed width set or mixed width set #2 is { τ / 2, τ , 2 τ , 4 τ , 8 τ Two thresholds, T1 and T2, can be used, with T1 being less than T2. ​​When both the block width and block height are less than and / or equal to threshold T1 (e.g., when each of the block width and block height is less than or equal to threshold T1), a mixed width set #1 is used. When both the block width and block height are greater than and / or equal to threshold T2 (e.g., when each of the block width and block height is greater than or equal to threshold T2), a mixed width set #2 is used. In other cases, the mixed width set is determined based on the partitioning pattern and the aspect ratio of the block width to the block height, as described above.

[0115] In one example, when the third condition is met—that each of the block height and block width is less than or equal to a threshold T1—the current block's blended width set is determined as the first blended width set (e.g., {τ / 4, τ / 2, τ, 2τ, 4τ}), and when the fourth condition is met—that each of the block height and block width is greater than or equal to a threshold T2—the blended width set is determined as the second blended width set (e.g., {τ / 4, τ / 2, τ, 2τ, 4τ}). τ / 2, τ, 2 τ , 4 τ , 8 τ}). T2 is greater than T1. When neither the third nor the fourth condition is met, the mixed width set is determined as either the first mixed width set or the second mixed width set, based on the partitioning pattern in the GPM and the aspect ratio between the block height and the block width, as described above. For example, the third and fourth conditions are not met, and the aspect ratio is the ratio of the block width to the block height. The two conditions include: i) the aspect ratio is greater than 1, and the partitioning pattern in the GPM is a near-vertical partitioning pattern; and ii) the aspect ratio is less than 1, and the partitioning pattern in the GPM is a near-horizontal partitioning pattern. When either of the two conditions is met, the mixed width set is determined as the second mixed width set (e.g., { τ / 2, τ , 2 τ , 4 τ , 8 τ When neither of the two conditions is met, the set of mixed widths is determined as the first set of mixed widths (e.g., {τ / 4, τ / 2, τ, 2τ, 4τ}).

[0116] This invention describes CCP under GPM for chroma blocks. In intra-frame predictive coding, CCP can be employed to predict chroma prediction blocks based on luma prediction blocks using linear filters (e.g., linear filtering formulas) or nonlinear polynomial filters (e.g., nonlinear polynomial filtering formulas). One or more coefficients of the linear or nonlinear polynomial formula can be derived from the correlation between luma and chroma templates, which may include spatially adjacent reconstructed samples from the upper and / or left regions of the current block. Figure 5 An example of a CCP according to one aspect of the invention is shown. A luminance template (511) of luminance block Y (501) and chrominance templates (512)-(513) of the corresponding chrominance blocks Cb (502) and Cr (503) are used to derive one or more filter coefficients for the filters of the corresponding chrominance blocks (502)-(503). The derived filters can be applied to luminance block (501) to predict chrominance blocks (502)-(503) and generate chrominance prediction data for Cb and Cr.

[0117] exist Figure 5In the example shown, the luminance template (511) and chrominance templates (512)-(513) are L-shaped. Each derived filter (e.g., filter 521 applied to Cb or filter 522 applied to Cr) can be a single-model filter or a multi-model filter. An example of deriving a multi-model filter can be done by using the average sample value of the template (e.g., luminance template (511)) as a classifier. Samples with values ​​higher than the average sample value can be used to derive the first filter, while other samples with values ​​lower than the average sample value can be used to derive the second filter. The multi-model filter can include both a first filter and a second filter.

[0118] According to one aspect of the invention, a CCP and a multi-model filter can be used to encode a chroma block based on a corresponding luma block (e.g., a luma block co-located with a chroma block, wherein the luma block and chroma block reside in the same coding unit). In one example, a GPM is used to predict the luma block. The luma block can be a luma prediction block or a luma reconstruction block.

[0119] The luma block may include a first partition (also called partition A) and a second partition (also called partition B) separated by geometric dividing edges of the GPM. The chroma block may also include a first partition (also called partition A) and a second partition (also called partition B) separated by geometric dividing edges. The multi-model filter used for the chroma block may, for example, be based on templates adjacent to the luma block and the chroma block respectively (e.g., luma template and chroma template) (such as...). Figure 6 (as shown) or prediction blocks based on luma and chroma blocks (such as...) Figure 7 (as shown) to determine. In one example, the geometric dividing edge separating the luma block and the geometric dividing edge separating the chroma block are based on the same partitioning pattern of GPM (e.g., reference). Figures 4A to 4C The geometric segmentation pattern described is used to determine this. For example, the geometric segmentation edges separating the luma blocks have the same angle as the geometric segmentation edges separating the chroma blocks (e.g., determined by the GPM segmentation pattern described). Figure 4B Angle indices 0-23 indicate the angle (one of which is indicated in the index) and the dividing edge (e.g., by...). Figure 4C (One of the indices 0-3 in the index).

[0120] A luminance template for a luminance block may be adjacent to the luminance block and includes reconstructed luminance samples. The luminance template may include a first luminance template and a second luminance template separated by extensions extending into the luminance template via geometric dividing edges. In one example, the first and second luminance templates are referred to as two dividing templates. A chrominance template for a chrominance block may be adjacent to the chrominance block and includes reconstructed chrominance samples. The chrominance template may include a first chrominance template and a second chrominance template separated by extensions extending into the chrominance template via geometric dividing edges. In one example, the first and second chrominance templates are referred to as two dividing templates.

[0121] A multi-model filter may include a first filter having first filter coefficients and a second filter having second filter coefficients. In one aspect, the first filter coefficients are determined based on a first partition of the luma block and a first partition of the chroma block, and the second filter coefficients are determined based on a second partition of the luma block and a second partition of the chroma block. In another aspect, the first filter coefficients are determined based on a first luma template of the luma block and a first chroma template of the chroma block, and the second filter coefficients are determined based on a second luma template of the luma block and a second chroma template of the chroma block. In one example, the chroma block is reconstructed according to the multi-model filter.

[0122] In one aspect, the first filter coefficients of the multi-model filter are determined based on a first luminance template of the luminance block and a first chrominance template of the chrominance block, and the second filter coefficients of the multi-model filter are determined based on a second luminance template of the luminance block and a second chrominance template of the chrominance block, such as... Figure 6 As shown.

[0123] Figure 6 An example of a CCP on a geometrically partitioned block according to one aspect of the invention is shown. Luminosity block Y (601) includes a first partition separated by a geometrically partitioned edge 671 of the GPM (Gross Partition). Figure 6 Partition A and Partition B in the middle) Figure 6 Partition B in the diagram). Luminance block (601) can be a luminance prediction block that includes predicted samples, or a luminance reconstruction block that includes reconstructed samples and is determined using GPM. Chroma blocks (e.g., chroma block Cb (602) or chroma block Cr (603)) can be predicted using CCP and a multi-model filter.

[0124] The chroma block (602) may include a first partition separated by geometric dividing edges 672. Figure 6 Partition A and Partition B in the middle) Figure 6 Partition B in the chroma block (603) may include a first partition separated by geometric dividing edges 673. Figure 6 Partition A and Partition B in the middle) Figure 6 In one example, the partitioning pattern of the luma block (601) is the same as that of each partitioning pattern of the chroma blocks (602)-(603). Therefore, the geometric dividing edge 671 of the luma block (601) has the same angle as the geometric dividing edge (672)-(673) of the chroma blocks (602)-(603) (e.g., by...). Figure 4B Angle indices 0-23 (one of which indicates the angle) and the same dividing edge (e.g., by...) Figure 4C (One of the indices 0-3 in the index).

[0125] The multi-model filter for the chroma block (602) can be determined based on the luma template (611) and the chroma template (612). The multi-model filter for the chroma block (602) may include a first filter (including first filter coefficients) 641 and a second filter (including second filter coefficients) 642, wherein the first filter is determined based on the first luma template (621) of the luma block (601) and the first chroma template (623) of the chroma block (602), and the second filter is determined based on the second luma template (622) of the luma block (601) and the second chroma template (624) of the chroma block (602). Reference Figure 6 In the filter derivation process 631 for partition A of Cb, a first filter 641 is determined, and in the filter derivation process 632 for partition B of Cb, a second filter 642 is determined.

[0126] A luminance template (611) of a luminance block (601) may be adjacent to the luminance block (601) and includes reconstructed luminance samples. The luminance template (611) may include a first luminance template (621) and a second luminance template (622) separated by an extension extending from a geometric dividing edge (671) into the luminance template (611). A chrominance template (612) of a chrominance block (602) may be adjacent to the chrominance block (602) and includes reconstructed chrominance samples. The chrominance template (612) may include a first chrominance template (623) and a second chrominance template (624) separated by an extension extending from a geometric dividing edge (672) into the chrominance template (612).

[0127] Similarly, the multi-model filter for the chroma block (603) can be determined based on the luma template (611) and the chroma template (613). The multi-model filter for the chroma block (603) may include a first filter (including first filter coefficients) 643 and a second filter (including second filter coefficients) 644, the first filter being determined based on the first luma template (621) of the luma block (601) and the first chroma template (625) of the chroma block (603), and the second filter being determined based on the second luma template (622) of the luma block (601) and the second chroma template (626) of the chroma block (603). Reference Figure 6 In the filter derivation process 633 for partition A of Cr, the first filter 643 is determined, and in the filter derivation process 634 for partition B of Cr, the second filter 644 is determined.

[0128] The chroma template (613) of the chroma block (603) may be adjacent to the chroma block (603) and include reconstructed chroma samples. The chroma template (613) may include a first chroma template (625) and a second chroma template (626) separated by an extension in the chroma template (613) by a geometric dividing edge 673 extending into the extension.

[0129] refer to Figure 6 The chroma block (602) can be predicted using GPM and a multi-model including a first filter 641 and a second filter 642. In one example, the first filter 641 is applied to the first partition (partition A) of the luma block (e.g., a luma prediction block or a luma reconstruction block) to obtain prediction data 651 for the first partition (partition A) of Cb. The second filter 642 is applied to the second partition (partition B) of the luma block (e.g., a luma prediction block or a luma reconstruction block) to obtain prediction data 652 for the second partition (partition B) of Cb.

[0130] In another example, a first filter 641 is applied to a luminance block (e.g., a luminance prediction block or a luminance reconstruction block) to obtain first prediction data for Cb, and a second filter 642 is applied to a luminance block (e.g., a luminance prediction block or a luminance reconstruction block) to obtain second prediction data for Cb. The first and second prediction data for Cb can be combined (or mixed) using a GPM mixing process, such as a reference... Figure 4D As described.

[0131] refer to Figure 6 The chroma block (603) can be predicted using GPM and a multi-model including a first filter 643 and a second filter 644. In one example, the first filter 643 is applied to the first partition (partition A) of the luminance block (e.g., a luminance prediction block or a luminance reconstruction block) to obtain prediction data 653 for the first partition (partition A) of Cr. The second filter 644 is applied to the second partition (partition B) of the luminance block (e.g., a luminance prediction block or a luminance reconstruction block) to obtain prediction data 654 for the second partition (partition B) of Cr.

[0132] In another example, a first filter 643 is applied to a luminance block (e.g., a luminance prediction block or a luminance reconstruction block) to obtain first prediction data for Cr, and a second filter 644 is applied to the luminance block (e.g., a luminance prediction block or a luminance reconstruction block) to obtain second prediction data for Cr. The first and second prediction data for Cr can be combined (or mixed) using a GPM mixing process, such as a reference... Figure 4D As described.

[0133] Indicates the partitioning pattern (e.g., including) Figure 4B The angles shown are Figure 4CThe geometric partitioning edge (shown) can be signaled or derived at the block level (e.g., at the luma block level or chroma block level). In one aspect, the geometric partitioning edge indicating the partitioning mode can be signaled or derived (derived from the luma block) at the chroma block or chroma prediction block (e.g., chroma block (602) or (603)). In another aspect, for example, when a single-tree partitioning structure is used for luma block (601) and chroma blocks (602)-(603), the geometric partitioning edge indicating the partitioning mode can be signaled or derived at the luma block (e.g., luma block (601)) (such as the luma prediction block).

[0134] In one example, each partition in the luma block (601) may be predicted using inter-frame prediction, intra-frame prediction, BV-based prediction, or similar methods, but is not limited to these.

[0135] In one example, when the geometric dividing edge (e.g., (671)) does not cross the template (e.g., (611)), the use of a multi-model filter determined by GPM (such as the one mentioned above) is not permitted. Figure 6 (As described) to carry out CCP.

[0136] In two partition templates (e.g., a first luminance template and a second luminance template (621)-(622)) separated by a geometric partition edge (e.g., (671)), the smaller template has a size such as the number of samples in that smaller template. The size ratio can be the ratio of the size of the smaller template to the size of the template. In one example, the use of a multi-model filter (such as the one referenced above) determined by GPM is not allowed when the size of the smaller template is less than a threshold (e.g., a predefined threshold) or the size ratio of the smaller template is less than a threshold ratio (e.g., a predefined ratio). Figure 6 (As described) to carry out CCP.

[0137] Including and such Figure 6 The multi-model filter (e.g., a multi-model filter for Cb comprising two filters (641)-(642)) associated with the described GPM pattern can be combined with other multi-model filtering methods to derive multi-model filters comprising more than two filters. In one aspect, a second multi-model filter can be further derived for geometric partitions (e.g., one of partitions A and B of Cb) by using the average sample value within the associated partition template as a classifier. In one example, a second multi-model filter is derived for each geometric partition by using the average sample value within the associated partition template as a classifier. When a second multi-model filter comprising two filters is derived for each geometric partition of Cb by using the average sample value within the associated partition template as a classifier, the multi-model filter for Cb comprises four filters.

[0138] refer to Figure 6 For the first partition (partition A) of Cb, a second multi-model filter, including a third filter and a fourth filter, is derived. The first average sample value of the first luminance template (621) is used as the first classifier. Based on the comparison between the value of the luminance sample and the first average sample value, the luminance samples in the first luminance template (621) can be classified into a first luminance sample subset and a second luminance sample subset. The chrominance samples in the first chrominance template (623) can be classified into a first chrominance sample subset and a second chrominance sample subset, wherein the first chrominance sample subset is co-located with the first luminance sample subset, and the second chrominance sample subset is co-located with the second luminance sample subset. Therefore, the third filter is derived based on the first chrominance sample subset and the first luminance sample subset, and the fourth filter is derived based on the second chrominance sample subset and the second luminance sample subset.

[0139] In one example, similar to the description above for the first partition of Cb, the second multi-model filter is derived for the second partition (partition B) of Cb.

[0140] In one example, similar to the description above for the first partition of Cb, the second multi-model filter is derived for the first partition (partition A) of Cr or the second partition (partition B) of Cr.

[0141] In one example, a signaling indication flag is provided for each partition (also called each geometry partition) to indicate whether a multi-model filter (e.g., a second multi-model filter) is used for the partition. When the flag is true, the second multi-model filter is derived for the geometry partition. Otherwise, a single-model filter is derived for the geometry partition.

[0142] In one example, a signaling flag is used at the block level to indicate whether a second multi-model filter is derived for both geometric partitions. When the flag is true, the second multi-model filter can be determined using the associated average sample value classifier and applied to each of the geometric partitions. Otherwise, a single-model filter without an average sample value classifier is used for each geometric partition.

[0143] Multi-model filters can be linear filters, such as linear formulas, or nonlinear filters, such as nonlinear polynomial formulas.

[0144] In one respect, the following formula can be used. Chromaticity prediction data is predicted based on luminance prediction data or luminance reconstruction data. and These are the filter coefficients, and and This refers to the predicted chromaticity sample and the corresponding predicted luminance sample or reconstructed luminance sample in the luminance block. (Reference) Figure 6 In one example, the filter coefficients of filters 641 to 642 and It can have different values.

[0145] In one respect, it can be used with n The following polynomial formula with coefficients Chromaticity prediction data is predicted based on luminance prediction data or luminance reconstruction data. These are the filter coefficients, and , and This includes the predicted chromaticity sample, the corresponding predicted luminance sample or reconstructed luminance sample, and the surrounding luminance samples of the corresponding predicted / reconstructed luminance sample within the luminance block. (Reference) Figure 6 In one example, the filter coefficients of filters 641 to 642 They can have different values.

[0146] In one respect, the following polynomial formula with nonlinear terms can be used. Chromaticity prediction data is predicted based on luminance prediction data or luminance reconstruction data. , and These are the filter coefficients, and , and (For example, and This refers to the predicted chromaticity sample, the corresponding predicted luminance sample or reconstructed sample, and the surrounding luminance samples of the corresponding predicted / reconstructed luminance sample within the luminance block. (Reference) Figure 6 In one example, the filter coefficients of filters 641 to 642 , and They can have different values.

[0147] In one respect, more than two different filtering formulas can be used, and a signaling instruction syntax can be used to indicate which filtering formula has been selected.

[0148] According to one aspect of the invention, the first filter coefficients of the multi-model filter can be determined based on a first predicted luminance sample in a first partition of the luminance block and a first predicted chrominance sample in a first partition of the chrominance block, and the second filter coefficients of the multi-model filter can be determined based on a second predicted luminance sample in a second partition of the luminance block and a second predicted chrominance sample in a second partition of the chrominance block, such as... Figure 7 As shown.

[0149] Figure 7An example of a CCP for GPM according to one aspect of the invention is shown. In one example, a luma prediction block is encoded in the GPM, wherein a first partition employs inter-frame prediction, and a second partition employs inter-frame prediction. A derived filter (e.g., a multi-model filter) is applied to the corresponding geometric partition of the luma reconstruction block to predict the chroma prediction block. The CCP can be applied in conjunction with the GPM when at least one geometric partition employs inter-frame prediction or BV-based prediction coding. The filter is derived using the method described above and applied to the geometric partition.

[0150] The luminance block can be a luminance prediction block (701A) or a luminance reconstruction block (701B). The luminance prediction block (701A) includes a first partition separated by a geometric dividing edge 771 of the GPM (Gross Partition Mapping). Figure 7 Partition A and Partition B in the middle) Figure 7 Partition B in the diagram). The luminance reconstruction block (701B) includes a first partition separated by geometric dividing edges 771. Figure 7 Partition A and Partition B in the middle) Figure 7 Partition B in the diagram). The luminance prediction block (701A) uses GPM for prediction. The luminance reconstruction block (701B) is determined based on the luminance prediction block (701A) and includes luminance reconstruction samples.

[0151] The chromaticity prediction block Cb (702), which is the prediction block for chromaticity block Cb, and the chromaticity prediction block Cr (703), which is the prediction block for chromaticity block Cr, can be predicted using GPM. The chromaticity prediction block Cb (702) can include a first partition separated by geometric dividing edges 772. Figure 7 Partition A and Partition B in the middle) Figure 7 Partition B in the chromaticity prediction block Cr (703) may include a first partition separated by geometric dividing edges 773. Figure 7 Partition A and Partition B in the middle) Figure 7 In one example, the partitioning pattern of the luminance block is the same as that of each partitioning pattern of the chrominance blocks Cb and Cr. Therefore, the geometric dividing edge 771 of the luminance block is the same as the geometric dividing edge (772)-(773) of the chrominance blocks Cb and Cr.

[0152] Each geometric partition can be predicted using inter-frame prediction or BV-based prediction (e.g., IBC mode or IntraTMP mode) and can be used to derive filter models from the corresponding prediction blocks. In one example, each partition of the luma prediction block (701A), chroma prediction block Cb (702), or chroma prediction block Cr (703) can be predicted using inter-frame prediction or BV-based prediction (e.g., IBC mode or IntraTMP mode). In one example, no partition of the luma prediction block (701A), chroma prediction block Cb (702), and chroma prediction block Cr (703) is predicted using intra-frame prediction.

[0153] In one example, refer to Figure 7 The lumen prediction block (701A) is encoded in the GPM, where the first partition uses a first inter-frame prediction mode and the second partition uses a second inter-frame prediction mode. In one example, it is not necessary to use inter-frame prediction or BV-based prediction (e.g., IBC mode or IntraTMP mode) for both partitions. For example, the lumen prediction block (701A) is encoded in the GPM, where one of the first and second partitions uses an inter-frame prediction mode, and the other of the first and second partitions uses a BV-based prediction mode. In one example, the lumen prediction block (701A) is encoded in the GPM, where the first partition uses a first BV prediction mode and the second partition uses a second BV-based prediction mode.

[0154] A luminance prediction block (701A) and a chrominance prediction block (e.g., chrominance prediction block Cb (702) or chrominance prediction block Cr (703)) can be used to determine a multi-model filter for the chrominance block (e.g., chrominance block Cb or Cr). The multi-model filter for the chrominance block Cb can be determined based on the luminance prediction block (701A) and the chrominance prediction block Cb (702). The multi-model filter for the chrominance block Cb may include a first filter (including first filter coefficients) 741 and a second filter (including second filter coefficients) 742, the first filter being determined based on a first partition of the luminance prediction block (701A) and a first partition of the chrominance prediction block (702), and the second filter being determined based on a second partition of the luminance prediction block (701A) and a second partition of the chrominance prediction block (702). Reference Figure 7 In the filter derivation process 731 for partition A of Cb, a first filter 741 is determined, and in the filter derivation process 732 for partition B of Cb, a second filter 742 is determined.

[0155] Similarly, the multi-model filter for the chromaticity block Cr can be determined based on the luminance prediction block (701A) and the chromaticity prediction block Cr (703). The multi-model filter for the chromaticity block Cr can include a first filter (including first filter coefficients) 743 and a second filter (including second filter coefficients) 744, the first filter being determined based on a first partition of the luminance prediction block (701A) and a first partition of the chromaticity prediction block (703), and the second filter being determined based on a second partition of the luminance prediction block (701A) and a second partition of the chromaticity prediction block (703). Reference Figure 7 In the filter derivation process 733 for partition A of Cr, the first filter 743 is determined, and in the filter derivation process 734 for partition B of Cr, the second filter 744 is determined.

[0156] refer to Figure 7 The chromaticity block Cb can be predicted using GPM and a multi-model including a first filter 741 and a second filter 742. In one example, the first filter 741 is applied to the first partition (partition A) of the luminance reconstruction block (701B) to obtain prediction data 751 for the first partition (partition A) of Cb. The second filter 742 is applied to the second partition (partition B) of the luminance reconstruction block (701B) to obtain prediction data 752 for the second partition (partition B) of Cb.

[0157] In another example, a first filter 741 is applied to the luminance reconstruction block (701B) to obtain first prediction data for Cb, and a second filter 742 is applied to the luminance reconstruction block (701B) to obtain second prediction data for Cb. The first and second prediction data for Cb can be combined (or mixed) using a GPM mixing process, such as a reference... Figure 4D As described.

[0158] refer to Figure 7 The chromaticity block Cr can be predicted using GPM and a multi-model including a first filter 743 and a second filter 744. In one example, the first filter 743 is applied to the first partition (partition A) of the luminance reconstruction block (701B) to obtain prediction data 753 for the first partition (partition A) of Cr. The second filter 744 is applied to the second partition (partition B) of the luminance reconstruction block (701B) to obtain prediction data for the second partition (partition B) of Cr.

[0159] In another example, a first filter 743 is applied to the luminance reconstruction block (701B) to obtain first prediction data for Cr, and a second filter 744 is applied to the luminance reconstruction block (701B) to obtain second prediction data for Cr. The first and second prediction data for Cr can be combined (or mixed) using a GPM mixing process, such as a reference... Figure 4D As described.

[0160] In one example Figures 6 to 7 The methods described can be combined to derive filter coefficients for geometrically partitioned prediction modes. For example, prediction data (651)-(654) can be mixed with corresponding prediction data (751)-(754).

[0161] In one example, for a geometric partition (e.g., partition A of Cb) predicted via inter-frame prediction or BV-based prediction, a multi-model filter can be derived, similar to... Figure 6 As described in the document. In one example, the average sample value, such as the average sample value of the corresponding partition in the brightness prediction block (701A), can be a classifier derived from a multi-model filter.

[0162] In one example, a filter can be derived between luminance prediction and chrominance prediction, for example, filter 741 can be derived between partition A of luminance prediction block (701A) and partition A of chrominance prediction block (702). The derived filter 741 can be applied to luminance prediction or luminance reconstruction (e.g., partition A of luminance reconstruction block (701B)) to generate final chrominance prediction data.

[0163] In one example, the final chromaticity prediction data can be a weighted sum of prediction data (e.g., in the absence of a blending process, prediction data refers to at least one of (751)-(754) shown in the right column) and the original prediction blocks (e.g., (702)-(703)) shown in the left column). The weights can be predefined weights. In one example, the weight of the original prediction blocks (e.g., (702) or (703)) is 0.25.

[0164] In one example, when the luma block is coded using GPM and there is no intra-prediction for the geometric partition, the signaling instructs the transform unit syntax to indicate whether to apply it. Figure 7 The described method. In one example, the transform unit syntax is implicitly false when the encoded block flag is not true.

[0165] Figure 8 An exemplary schematic diagram of a BM-based DMVR is shown. For example... Figure 8As shown, the current image (802) may include the current block (808). The current image may have a first reference image (804) from a reference image list L0 and a second reference image (806) from a reference image list L1. For the current block (808), a pair of reference blocks are identified in the first and second reference images based on initial motion vectors MV0 and MV1. For example, an initial reference block (812) in the first reference image (804) can be located based on the initial motion vector MV0, and an initial reference block (814) in the second reference image (806) can be located based on the initial motion vector MV1. A search process can be performed around the initial MV0 in the first reference image (804) and the initial MV1 in the second reference image (806). For example, MV0 can be adjusted. diff The initial MV0 and MV1 are applied in the opposite direction to obtain motion vector candidates, such as MV0' and MV1'. Based on the motion vector candidates, a pair of candidate reference blocks are identified in the first and second reference images. For example, a candidate reference block (810) can be identified in the first reference image (804) based on MV0', and a candidate reference block (816) can be identified in the second reference image (806) based on MV1'. In some examples, BM refers to the operation of calculating a distortion metric between a pair of reference blocks in the corresponding reference images of the current image, such as using the sum of absolute differences (SAD) between a pair of reference blocks as the distortion metric for that pair of reference blocks. For example, the BM method calculates the initial SAD between the pair of initial reference blocks (812) and (814) and the second SAD between the pair of candidate reference blocks (810) and (816). The initial SAD is associated with the initial motion vectors (e.g., MV0 and MV1), and the second SAD is associated with the motion vector candidates (e.g., MV0' and MV1'). Similarly, the BM method can compute the SADs of multiple motion vector candidates around the initial motion vector. The motion vector candidate with the minimum SAD can become the corrected motion vector and is used to generate a bidirectional prediction signal to predict the current block (808).

[0166] It is important to note that the corrected motion vectors derived through the DMVR process are used to generate inter-frame prediction samples and can be used for temporal motion vector prediction in future image coding. In some examples, the original motion vectors are used in the deblocking process and also for spatial motion vector prediction in future coding units.

[0167] In DVMR, the search point revolves around the initial motion vector, and the motion vector offset follows the motion vector difference (MVD) mirror rule. Any point checked by DMVR and represented by a candidate motion vector pair (MV0', MV1') satisfies this rule. and .in, , such as MV diff , representing the corrected offset between the initial motion vector (e.g., MV0, MV1) and the corrected motion vector of one of the reference images.

[0168] In some examples (e.g., VVC), DMVR is applied to coding units encoded in a regular merging mode. The motion vector pair obtained from the regular merging candidate is used as input to the DMVR process. DMVR applies BM to correct the input motion vector pair {MV0, MV1} and converts the corrected motion vector pair {MV... refinedL0 MV refinedL1 This is used for motion compensation prediction of both the luminance and chrominance components. The output motion vector of the DMVR can be called the corrected motion vector pair, and can be expressed by Equation 4: Formula 4 By using the MVD mirror property, MVD Δmv is applied to the input motion vector pair to obtain the corrected motion vector pair, since the input motion vector pair points to two different reference images that have the same difference in picture order count (POC) from the current image and are in different time directions.

[0169] In some examples, DMVR can be applied at the sub-block level, where the luma-coded block is divided into 16×16 sub-blocks for the motion vector correction process. Δmv is derived independently for each sub-block.

[0170] In one aspect, in inter-frame predictive coding, the final motion vector can be derived based on (i) spatial and / or temporal information or (ii) the sum of the signaling-indicated MVD and the derived or selected motion vector predictor (MVP). When the final motion vector is derived based on spatial and / or temporal information, inter-frame predictive coding can be called a merging mode, and the final motion vector can be called a merged motion vector. The merged motion vector can be derived from the motion vectors of adjacent coded blocks, the motion vectors of non-adjacent coded blocks, the motion vectors of co-located coded blocks in the reference image, historical motion information from previously coded blocks, etc. During the merging candidate list construction process, the positions of the coded blocks can be fixed and predefined. When the final motion vector is the sum of the signaling-indicated MVD and the derived or selected MVP, inter-frame predictive coding or the coded mode can be called an advanced motion vector prediction (AMVP) mode. In AMVP mode, the MVP can be derived or selected from a candidate list. The candidate list can have at least one MVP in the candidate list. The candidate list construction for the AMVP pattern can be similar to the merge candidate list construction in the merge pattern.

[0171] In HEVC, translational motion is used for MCP (Motion Capability Programming). Translational motion refers to the uniform movement of all points on a moving object along the same straight line or in the same direction; therefore, motion vectors can be used to define the motion of the object. In real-world scenarios, various types of motion may exist, such as zooming in / out, rotation, perspective motion, and other irregular motions. Non-translational motion refers to the motion in which the points on a moving object do not move uniformly. Some motion models may include translational components for translational motion and non-translational components for non-translational motion.

[0172] Block-based affine transformations (MCPs) can be used, for example in VTM. Figure 9A The affine motion field of a block (902) described by motion information (4 parameters) from two control points is shown. Figure 9B The affine motion field of the block (904) is shown, which is described by three CPMVs (6 parameters).

[0173] like Figure 9A As shown, in the 4-parameter affine motion model, the sample position in block (902) is ( x, y The motion vector at point () can be derived from the following formula 5: Formula 5 in, It can be a motion vector in the first direction (or the X direction), and This can be a motion vector in the second direction (or the Y direction). The motion vector can also be described in Equation 6: Formula 6 like Figure 9B As shown, in the 6-parameter affine motion model, the sample position in block (904) is ( x, y The motion vector at point () can be derived from the following formula 7: Formula 7 The 6-parameter affine motion model can also be described in Equation 8: Formula 8 As shown in formulas 5 and 7, ( mv 0x , mv 0y () can be the motion vector of the top-left control point. mv 1x , mv 1y () can be the motion vector of the upper right control point. mv 2x , mv 2y () can be the motion vector of the lower left control point.

[0174] The above descriptions of the merge mode and AMVP mode can be used in coded blocks encoded using an affine model. In some examples, such as the VVC standard, a 4-parameter affine model (e.g.) Figure 9A (as shown) and 6-parameter affine model (as shown) Figure 9B (As shown) can be constructed using two CPMVs (such as V0 and V1) and three CPMVs (such as V0, V1, and V2), respectively. In one example, we can derive an 8-parameter perspective model using four CPMVs at the four corresponding corners of the coded block.

[0175] In affine merge prediction, for example, the affine merge (AF_MERGE) mode can be applied to coding units with a width and height greater than or equal to 8. The CPMVs of the current coding unit can be generated based on the motion information of spatially adjacent coding units. In one example, up to five CPMVP candidates can be applied to the affine merge prediction, and a signaling index can be used to indicate which of the five CPMVP candidates can be used for the current coding unit. In affine merge prediction, at least one of several types of CPMV candidates can be used to form the affine merge candidate list: (1) inherited affine merge candidates extrapolated from the CPMVs of adjacent coding units, (2) constructed affine merge candidates with CPMVPs derived using the translational motion vectors of adjacent coding units, (3) zero motion vectors, and / or similar items.

[0176] In affine AMVP prediction, for example, the affine AMVP pattern can be applied to coding units with a width and height greater than or equal to 16. A coding unit-level affine flag can be signaled in the bitstream to indicate whether the affine AMVP pattern is used, and then another flag can be signaled to indicate whether a 4-parameter affine or a 6-parameter affine is applied. In affine AMVP prediction, the difference between the predicted values ​​of the CPMVs and CPMVPs of the current coding unit can be signaled in the bitstream. In one example, the size of the affine AMVP candidate list can be 2, and the affine AMVP candidate list can be generated using at least one of several types of CPMV candidates, for example, in the following order: (1) inherited affine AMVP candidates extrapolated from the CPMVs of neighboring coding units, (2) constructed affine AMVP candidates with CPMVPs derived using translational motion vectors of neighboring coding units, (3) translational motion vectors from neighboring coding units, (4) zero motion vectors, and / or similar items.

[0177] According to one aspect of the invention, intra-frame TM can be applied to affine MVP and affine merge candidate construction. In one example, affine MVP or affine merge candidate is determined based on intra-frame TM. In one aspect, the current block in the current image is encoded in an affine mode (e.g., using an affine model). A TM process (also called intra-frame TM) can be applied to determine a reference block in the current image. The CPMVs of the current block can be determined based on motion vector information associated with multiple corners of the reference block. The current block can be reconstructed based on the determined CPMVs of the current block.

[0178] In one example, the affine merge candidates used to construct the affine candidate list are derived based on the CPMVs of the current block, and the current block is reconstructed based on one of the affine merge candidates in the affine candidate list. In another example, for instance, the affine MVP of the current block is derived based on the CPMVs of the current block, and the current block is reconstructed based on the affine MVP.

[0179] Figure 10 An example of using an affine model of TM in an encoded region (e.g., a reconstructed region (1100)) of the current image (1000) is shown. The current image (1000) is being encoded (e.g., being reconstructed). In one example, the encoded region (1100) has already been encoded (e.g., reconstructed). TM processing can be applied to the reconstructed region (1100) in the current image (1000) to determine at least one reference block C' (1002), which may be adjacent to and / or not adjacent to the current block (1001). In one example, the current block (1001) is being encoded (e.g., being reconstructed). The reference block C' (1002) is determined by the TM process. For example, the TM cost is determined based on the reference template of the block in the reconstructed region (1100) and the current template (1011) of the current block (1001). The reference block (1002) can be determined as one of the blocks with the minimum TM cost among the TM costs. The minimum TM cost is determined based on the reference template (1012) of the reference block (1002) and the current template (1011).

[0180] In one example, the TM procedure is executed in a predefined search region, such as in a reconstruction region.

[0181] In one example, the predefined search area is limited to the N CTU columns to the left of the current CTU column and the M CTU rows above the current CTU row, where N and M are positive numbers. The current block (1001) is located within the current CTU. The current CTU column is the CTU column that includes the current CTU. The current CTU row is the CTU row that includes the current CTU.

[0182] In one example, the TM process is performed via subsampling operations or subsampling to reduce the number of samples used in the TM process, thereby reducing complexity.

[0183] In one example, BV (1021) indicates the reference block (1002) of the current block (1001). For example, BV (1021) indicates the positional difference between the reference block (1002) and the current block (1001).

[0184] The reference block (1002) may have motion vector information and / or BV information, and the motion vector information or BV information can be used to derive affine models, such as 4-parameter models, 6-parameter models, or perspective 8-parameter models. In one aspect, the motion vectors around at least two of the four corners of the reference block C′ (1002) can be used to determine the CPMVs of the current block (1001). In one example, the motion vectors around at least two of the four corners of the reference block C′ (1002) are directly used as the corresponding CPMVs for the affine model derivation of the current block (1001).

[0185] In one aspect, such as Figure 11 As shown, when all four CPMVs are available, it is possible to use the four CPMVs of the current block (1001) (including...). CPMV LT , CPMV RT , CPMV LB and CPMV RB This is used to derive the 8-parameter affine / perspective transformation model of the current block (1001). The subscripts L, R, T, and B in LT, RT, LB, and RB indicate the left, right, top, and bottom positions of the current block (1001), respectively. Figure 11 The positions of the corresponding CPMVs are shown in the diagram. LT, RT, LB, and RB indicate the top-left, top-right, bottom-left, and bottom-right corners of the current block (1001), respectively. (See reference...) Figure 11 , CPMV LT , CPMV RT , CPMV LB and CPMV RB These are the CPMVs at the corresponding corners LT, RT, LB and RB of the current block (1001).

[0186] In one example, when at least two or three of the four CPMVs are available, a 4-parameter affine model or a 6-parameter affine model can be derived from the four CPMVs. In another example, the 4-parameter or 6-parameter affine model can be derived by selecting two or three motion vectors from the available CPMVs respectively. For example, when... CPMV LT , CPMV RT and CPMV LB When available, use two CPMVs (e.g., (i)). CPMV LT and CPMV RT, or (ii) CPMV LT and CPMV LB To derive the 4-parameter affine model.

[0187] In one example, the size of the reference block C′ (1002) is the same as the size of the current block C (1001).

[0188] In one example, at least one motion vector at each corner of the reference block C' (1002) (e.g., corresponding to CPMV, such as...) CPMV LT , CPMV RT , CPMV LB and CPMV RB One of them is derived from one or more motion vectors around the corresponding corner of the reference block C'(1002) according to a predefined scan order. Each corner can have its own scan order. Figure 12 An example of CPMV derivation at the corners (e.g., the four corners) of a reference block C' (1002) according to one aspect of the invention is shown. The numbers at each corner of the reference block C' (1002), such as 0, 1, 2, and 3, indicate the block at the corresponding corner. These numbers can indicate the scanning order of the corresponding corner. In one example, when the first number is less than the second number, the block indicated by the first number is scanned first, followed by the block indicated by the second number.

[0189] refer to Figure 12 At each corner of reference block C' (1002), the scanning order is 0, 1, 2, and 3. For example, at the LT corner of reference block C' (1002), the block located at the upper left position is scanned first, followed by the block located at the lower right position, the block located at the lower left position, and the block located at the upper right position. In one example, the block located at the upper left position (indicated by "0") has a first motion vector associated with that block ( Figure 12 The first motion vector shown (1) st MV), and the first motion vector associated with block "0" is determined as the motion vector at the LT angle of reference block C' (1002). Reference Figure 12 For the angles LT, RT, LB, and RB of the reference block (1002), determine the first motion vector and the second motion vector (e.g., Figure 12 The second motion vector shown (2) nd MV), third motion vector (e.g., Figure 12 The third motion vector shown (3) rd MV)) and the fourth motion vector (e.g., Figure 12The fourth motion vector shown (4) th MV). In one example, the first motion vector, the second motion vector, the third motion vector, and the fourth motion vector are used to determine... CPMV LT , CPMV RT , CPMV LB and CPMV RB .

[0190] In one example, multiple blocks (e.g., blocks indicated by “0”, “1”, “2”, and “3”) located at one of the corners (e.g., LT) of a reference block (1002) are scanned in a predefined scan order (e.g., from 0 to 3) to determine the motion vector associated with one of the multiple blocks. The motion information associated with the multiple corners of the reference block includes the determined motion vector associated with one of the multiple blocks (e.g., the determined motion vector is directly used as the CPMV). LT In one example, the first motion vector is directly used as the CPMV of the current block (1001). LT .

[0191] The motion vector information associated with multiple corners of the reference block includes the motion vectors associated with those corners. The CPMVs of the current block can be determined as the vector sum of the corresponding motion vectors associated with the multiple corners and the BV. In one example, the CPMVs of the current block (1001) LT It is determined based on the first motion vector. For example, Figure 10 The BV(1021) derived via the TM process shown can be used as an offset, added to each motion vector derived at each corner of the reference block (1002). For each corner of the current block (1001), the final CPMV (e.g., CPMV) is calculated. LT ) can be equal to the sum of the motion vector derived at the corner of the reference block (1002) and BV (1021). For example, the CPMV of the LT corner of the current block (1001). LT = First motion vector of the LT angle of reference block (1002) + BV (1021). For example, CPMV of the RT angle of the current block (1001). RT =Second motion vector + BV (1021).

[0192] In one example, each CPMV of the current block (1001) is derived from a predefined location or a list of predefined locations. In one example, such as... Figure 12 As shown, the four CPMVs of the current block (1001) are derived at each of the four predefined corner positions. For example, the CPMVs of the current block (1001)LT It is derived from the LT position of reference block (1002). In one example, the CPMV at each corner is determined according to the reference above. Figure 12 The predefined scanning order described is derived from the motion vectors around the corresponding angles.

[0193] In one example, a linear regression method (such as regression-based affine merging) is applied to multiple locations around the corners of selected CPMVs to derive affine parameters. In another example, motion vectors from multiple locations (e.g., LT, RT, LB, and / or RB), such as four motion vectors associated with LT, four associated with RT, four associated with LB, and four associated with RB, are used to solve a linear regression formula to derive the parameters used in the affine pattern. In one example, for regression-based affine parameter derivation, the number of multiple locations around each corner of the selected CPMVs (e.g., four) is the same.

[0194] In one example, when using selected CPMVs to derive an affine model (such as a 4-parameter affine model, a 6-parameter affine model, or a perspective model (e.g., an 8-parameter affine model)), all selected CPMVs point to the same reference image.

[0195] In one example, time scaling can be applied to the selected CPMV when the reference image pointed to by the selected CPMV is different from the target reference image of the current block C (1001). A suitable method can be used to determine the target reference image. In one example, the target reference image is determined to be the reference image pointed to by a majority of CPMVs. In another example, the target reference image is a specific reference image, such as a reference image with reference index 0.

[0196] In one example, when all selected CPMVs have the same prediction orientation (e.g., a first reference image list L0 or a second reference image list L1) and point to the same reference image, the selected CPMVs are used to construct affine merge candidates, for example, for affine merge mode. The number of selected CPMVs can be 2, 3, or 4. In this case, the constructed affine merge candidates can be inserted into the affine merge candidate list for the current block (1001). An affine merge candidate can be selected from the affine merge candidate list and used in affine mode to predict the current block (1001).

[0197] In one example, when the number of selected CPMVs is 2, the selected CPMVs can be from... CPMV LT , CPMV RT , CPMV LB and CPMV RB One of the following combinations of the two CPMVs selected, for example (i) CPMV LT and CPMV RT (ii) CPMV LT and CPMV LB (iii) CPMV RT and CPMV RB , and (iv) CPMV LB and CPMV RB .

[0198] In one example, when the number of selected CPMVs is 3, the selected CPMVs can be from... CPMV LT , CPMV RT , CPMV LB and CPMV RB One of the following combinations of the three CPMVs selected, for example [ CPMV LT , CPMV RT , CPMV LB ]、[ CPMV LT , CPMV RT , CPMV RB ]、[ CPMV LT , CPMV LB , CPMV RB ]as well as[ CPMV RT , CPMV LB , CPMV RB ].

[0199] In one example, when all selected CPMVs have the same prediction direction and point to the same reference image, the selected CPMVs are used to construct an affine MVP, such as for an affine AMVP pattern. The number of selected CPMVs can be 2, 3, or 4. The constructed affine MVP can be used to predict the current block (1001).

[0200] In one example, the number of CPMVs selected is the same for both reference directions (or for two reference lists L0 and L1).

[0201] In one example, the CPMV derivation is used to derive the affine merge candidate for constructing the list of affine merge candidates, or the affine MVP for predicting the current block (1001).

[0202] In one example, when the derived affine merge candidate or the derived affine MVP is a bidirectional predicted affine motion vector, a BM process such as BM-based DMVR can be applied.

[0203] The term "IBC mode" can refer to the IBC mode or a variant thereof. The term "IntraTMP mode" can refer to the IntraTMP mode or a variant thereof. The term "GPM" can refer to the GPM mode or a variant thereof. The term "affine mode" can refer to the affine mode or a variant thereof.

[0204] Figure 13 A flowchart illustrating an overview process (1300) according to one aspect of the present invention is shown. The method (1300) can be used in devices such as video decoders. In various aspects, the process (1300) is executed by processing circuitry, such as processing circuitry executing the functions of video decoder (110), processing circuitry executing the functions of video decoder (210), etc. In some aspects, the process (1300) is implemented in the form of software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry executes the process (1300). The process begins at (S1301) and continues to (S1310).

[0205] At (S1310), encoded information is received, which indicates that the current block uses GPM encoding with multiple mixed width sets.

[0206] At (S1320), based on the block size information and GPM information of the current block, a set of blend widths to be applied to the current block is determined from multiple blend width sets.

[0207] In one example, multiple blended width sets include a first blended width set {τ / 4, τ / 2, τ, 2τ, 4τ} and a second blended width set { τ / 2, τ , 2 τ , 4 τ 8 τ}, and τ is a positive value.

[0208] In one example, block size information indicates the block height and block width of the current block. When the block width and block height satisfy a first condition, a first mixed width set is determined based on a comparison between (i) both the block height and block width and (ii) a threshold T. When the block width and block height satisfy a second condition, the mixed width set is determined as a second mixed width set. The first and second conditions are based on a comparison between (i) both the block height and block width and (ii) the threshold T. When the block width and block height do not satisfy either the first or the second condition, the mixed width set is determined as either the first or the second mixed width set based on the partitioning pattern in the GPM and the aspect ratio between the block height and block width.

[0209] In one example, the block width and block height do not satisfy either the first or the second condition, and the aspect ratio is the ratio of the block width to the block height. The two conditions are: (i) the aspect ratio is greater than 1, and the partitioning pattern in the GPM is a near-vertical partitioning pattern; and (ii) the aspect ratio is less than 1, and the partitioning pattern in the GPM is a near-horizontal partitioning pattern. When one of the two conditions is met, the mixed width set is determined as the second mixed width set. When neither condition is met, the mixed width set is determined as the first mixed width set.

[0210] In one example, the partitioning pattern in GPM is a near-horizontal partitioning pattern when the geometrical partitioning angle between the geometrical partitioning edge and the horizontal line is less than or equal to a predefined angle.

[0211] In one example, the partitioning pattern in GPM is a near-vertical partitioning pattern when the geometric partitioning angle between the geometric partitioning edge and the vertical line is less than or equal to a predefined angle.

[0212] In one example, the block size information indicates the block height and block width of the current block. The minimum blend width in the first blend width set is less than the minimum blend width in the second blend width set. The blend width set is determined as the first blend width set when a third condition is met, where each of the block height and block width is less than or equal to a threshold T1. The blend width set is determined as the second blend width set when a fourth condition is met, where each of the block height and block width is greater than or equal to a threshold T2. T2 is greater than T1. When neither the third nor the fourth condition is met, the blend width set is determined as either the first or second blend width set based on the partitioning pattern in the GPM and the aspect ratio between the block height and block width.

[0213] In one example, neither the third nor the fourth condition is met, and the aspect ratio is the ratio of block width to block height. The two conditions are: (i) the aspect ratio is greater than 1, and the partitioning pattern in the GPM is a near-vertical partitioning pattern; and (ii) the aspect ratio is less than 1, and the partitioning pattern in the GPM is a near-horizontal partitioning pattern. When either of the two conditions is met, the blended width set is determined as the second blended width set. When neither condition is met, the blended width set is determined as the first blended width set.

[0214] At (S1330), the mixing width is determined from the established mixing width set.

[0215] At (S1340), the current block is reconstructed based on the GPM and the determined blend width.

[0216] The process then continues to (S1399) and terminates.

[0217] The process (1300) can be adjusted as appropriate. One or more steps in the process (1300) can be modified and / or omitted. Additional steps can be added. Any suitable implementation order can be used.

[0218] Video coding methods may include, as described above, determining a set of blend widths to be applied to the current block from multiple sets of blend widths based on the block size information and GPM information of the current block. Video coding methods may also include: determining a blend width from the determined set of blend widths, and encoding the current block according to the GPM and the determined blend width.

[0219] Configuring multiple mixed width sets (e.g., a first mixed width set and a second mixed width set) for the GPM and determining one of the multiple mixed width sets based on encoded information such as the block size information of the current block and GPM information can reduce the number of bits used to indicate the mixed width, thereby improving coding efficiency. For example, the determination of one of the multiple mixed width sets is performed implicitly, so no additional signaling bits are required to determine one of the multiple mixed width sets.

[0220] Figure 14 A flowchart illustrating an overview process (1400) according to one aspect of the invention is shown. The method (1400) can be used in devices such as video decoders. In various aspects, the process (1400) is executed by processing circuitry, such as processing circuitry executing the functions of video decoder (110), processing circuitry executing the functions of video decoder (210), etc. In some aspects, the process (1400) is implemented as software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry executes the process (1400). The process begins at (S1401) and continues to (S1410).

[0221] At (S1410), encoded information is received, which instructs the chroma blocks to be encoded based on the corresponding luma blocks using CCP and a multi-model filter. The luma blocks are predicted using GPM and include a first partition and a second partition separated by the geometric dividing edges of the GPM.

[0222] At (1420), a multi-model filter for a chroma block is determined, the chroma block comprising a first partition and a second partition separated by geometric dividing edges. The multi-model filter comprises one of the following: (i) first filter coefficients determined based on the first partition of the luma block and the first partition of the chroma block, and second filter coefficients determined based on the second partition of the luma block and the second partition of the chroma block; and (ii) first filter coefficients determined based on the first luma template of the luma block and the first chroma template of the chroma block, and second filter coefficients determined based on the second luma template of the luma block and the second chroma template of the chroma block.

[0223] The luminance template of the luminance block is adjacent to the luminance block and includes a reconstructed luminance sample. The luminance template includes a first luminance template and a second luminance template separated by an extension extending into the luminance template by a geometric dividing edge. The chrominance template of the chrominance block is adjacent to the chrominance block and includes a reconstructed chrominance sample. The chrominance template includes a first chrominance template and a second chrominance template separated by an extension extending into the chrominance template by a geometric dividing edge.

[0224] In one example, the first filter coefficients of the multi-model filter are determined based on the first luma template of the luma block and the first chroma template of the chroma block. The second filter coefficients of the multi-model filter are determined based on the second luma template of the luma block and the second chroma template of the chroma block.

[0225] At (S1430), the chroma block is reconstructed based on the multi-model filter.

[0226] The process then continues to (S1499) and terminates.

[0227] The process (1400) can be adjusted as appropriate. One or more steps in the process (1400) can be modified and / or omitted. Additional steps can be added. Any suitable implementation order can be used.

[0228] In one example, geometric dividing edges (e.g., separating chroma blocks) are signaled or deduced at the chroma block.

[0229] In one example, when a single-tree partitioning structure is used for luma and chroma blocks, the geometric dividing edges separating the chroma blocks are signaled or deduced at the luma block.

[0230] In one example, the multi-model filter is a linear filter.

[0231] In one example, the multi-model filter is a nonlinear polynomial filter.

[0232] In one example, a multi-model filter and GPM are applied to either the predicted or reconstructed samples of the luminance block to reconstruct the chrominance block.

[0233] In one example, the first filter coefficients of the multi-model filter are determined based on the first predicted luminance sample in the first partition of the luminance block and the first predicted chrominance sample in the first partition of the chrominance block. The second filter coefficients of the multi-model filter are determined based on the second predicted luminance sample in the second partition of the luminance block and the second predicted chrominance sample in the second partition of the chrominance block.

[0234] A video coding method may include: determining a multi-model filter for a chroma block, the chroma block comprising a first partition and a second partition separated by geometrical segmentation edges. The chroma block is encoded based on a corresponding luma block using CCP and the multi-model filter, and the luma block is predicted by GPM and comprises a first partition and a second partition separated by geometrical segmentation edges of the GPM. The multi-model filter comprises one of: (i) first filter coefficients determined based on the first partition of the luma block and the first partition of the chroma block, and second filter coefficients determined based on the second partition of the luma block and the second partition of the chroma block; and (ii) first filter coefficients determined based on a first luma template of the luma block and a first chroma template of the chroma block, and second filter coefficients determined based on a second luma template of the luma block and a second chroma template of the chroma block. The luma template of the luma block is adjacent to the luma block and includes reconstructed luma samples. The luma template comprises a first luma template and a second luma template separated by extensions extending into the luma template by geometrical segmentation edges. The chroma template of the chroma block is adjacent to the chroma block and includes reconstructed chroma samples. The chroma template includes a first chroma template and a second chroma template, separated by extensions formed by geometric dividing edges. Video coding methods may include encoding chroma blocks according to a multi-model filter.

[0235] In some related techniques, single-model filters are applied to two partitions in GPM. (See reference...) Figure 6 In the method described in section 7, since the two partitions are implicitly known based on GPM, a multi-model filter is applied to these two partitions without signaling overhead. Therefore, more accurate predictions can be made without signaling overhead.

[0236] Figure 15A flowchart illustrating an overview process (1500) according to one aspect of the invention is shown. The method (1500) can be used in devices such as video decoders. In various aspects, the process (1500) is executed by processing circuitry, such as processing circuitry executing the functions of video decoder (110), processing circuitry executing the functions of video decoder (210), etc. In some aspects, the process (1500) is implemented in the form of software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry executes the process (1500). The process begins at (S1501) and continues to (S1510).

[0237] At (S1510), encoded information is received, which indicates that the current block in the current image is encoded using an affine mode.

[0238] At (S1520), the TM procedure is applied to determine the reference block in the current image.

[0239] At (S1530), the CPMVs of the current block are determined based on motion vector information associated with multiple corners of the reference block.

[0240] At (S1540), the current block is reconstructed based on the determined CPMVs of the current block.

[0241] The process then continues to (S1599) and terminates.

[0242] The process (1500) can be adjusted as appropriate. One or more steps in the process (1500) can be modified and / or omitted. Additional steps can be added. Any suitable implementation order can be used.

[0243] In one example, multiple blocks located at one of the corners of a reference block are scanned in a predefined scan order to determine the motion vector associated with one of the blocks. The motion information associated with the multiple corners of the reference block includes the determined motion vector associated with one of the blocks.

[0244] In one example, BV indicates the position of the reference block relative to the current block. Motion vector information associated with multiple angles of the reference block includes motion vectors associated with those angles. The CPMVs of the current block are determined as the vector sum of the corresponding motion vectors associated with the multiple angles and the BV.

[0245] In one example, the affine merge candidate used to construct the affine candidate list is derived based on the CPMVs of the current block, and the current block is reconstructed based on one of the affine merge candidates in the affine candidate list. In another example, the affine MVP of the current block is derived, and the current block is reconstructed based on the affine MVP.

[0246] In one aspect, the video coding method includes: applying a TM procedure to determine a reference block in the current image for the current block in the current image; determining CPMVs of the current block based on motion vector information associated with multiple corners of the reference block; and encoding the current block based on the determined CPMVs of the current block using affine patterns.

[0247] Compared to related techniques using affine modes, the method using intra-frame TM described in this invention can obtain more accurate CPMVs, thereby enabling more accurate predictions without increasing signaling overhead.

[0248] In one aspect, a method for processing visual media data includes processing a bitstream of the visual media data according to format rules. For example, the bitstream may be a bitstream decoded / encoded using any of the decoding and / or encoding methods described in this application. The format rules may specify at least one constraint on the bitstream and / or at least one process to be performed by the decoder and / or encoder.

[0249] In one aspect, the bitstream includes receiving encoded information that indicates the current block uses GPM encoding with multiple sets of mixed widths. The format rules specify that: based on the block size information and GPM information of the current block, a set of mixed widths to be applied to the current block is determined from multiple sets of mixed widths; a mixed width is determined from the determined set of mixed widths; and the current block is reconstructed based on the GPM and the determined mixed width.

[0250] In one aspect, the bitstream includes receiving encoded information instructing chroma blocks to be encoded based on corresponding luma blocks using CCPs and multi-model filters. The luma blocks are predicted using GPMs and include a first partition and a second partition separated by geometrical partitioning edges of the GPMs. Format rules specify that a multi-model filter is determined for the chroma blocks, which include the first and second partitions separated by geometrical partitioning edges. The multi-model filter includes one of the following: (i) first filter coefficients determined based on the first partition of the luma block and the first partition of the chroma block, and second filter coefficients determined based on the second partition of the luma block and the second partition of the chroma block; and (ii) first filter coefficients determined based on the first luma template of the luma block and the first chroma template of the chroma block, and second filter coefficients determined based on the second luma template of the luma block and the second chroma template of the chroma block. The luma template of the luma block is adjacent to the luma block and includes reconstructed luma samples. The luma template includes a first luma template and a second luma template separated by extensions of geometrical partitioning edges extending into the luma template. The chroma template of the chroma block is adjacent to the chroma block and includes reconstructed chroma samples. The chroma template comprises a first chroma template and a second chroma template, separated by extensions formed by geometric dividing edges. The format rules further specify that the chroma block is reconstructed based on a multi-model filter.

[0251] In one aspect, the bitstream includes receiving encoded information indicating that the current block in the current image is encoded using an affine mode. The format rules specify that: a TM process is applied to determine a reference block in the current image; the CPMVs of the current block are determined based on motion vector information associated with multiple corners of the reference block; and the current block is reconstructed based on the determined CPMVs.

[0252] The methods, aspects, and / or examples in this invention can be used individually or in any combination in any order. For example, some aspects and / or examples executed by the decoder can be executed by the encoder, and vice versa. Each method (or aspect), encoder, and decoder can be implemented by processing circuitry (e.g., at least one processor or at least one IC). In one example, at least one processor executes a program stored in a non-transitory computer-readable storage medium.

[0253] The above-described technologies can be implemented in the form of computer software using computer-readable instructions and physically stored in at least one computer-readable medium. For example, Figure 16 A computer system (1600) suitable for implementing certain aspects of the disclosed subject matter is shown.

[0254] Computer software can be written using any suitable machine code or computer language. This machine code or computer language may be processed by mechanisms such as assembly, compilation, and linking to generate code containing instructions. These instructions can be executed directly by at least one computer central processing unit (CPU), graphics processing unit (GPU), or through methods such as interpretation or microcode execution.

[0255] These instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0256] Figure 16 The components of the computer system (1600) shown are merely examples and are not intended to imply any limitation on the scope or functionality of computer software used to implement aspects of the present invention. The configuration of the components should also not be construed as having any dependency or requirement on any component or combination of components shown in the example aspects of the computer system (1600).

[0257] The computer system (1600) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from at least one human user via, for example, tactile input (e.g., key presses, swipes, data glove movements), audio input (e.g., voice, clapping sounds), visual input (e.g., gestures), or olfactory input (not depicted). The human-machine interface device may also be used to acquire media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0258] Human-machine interface input devices may include at least one of the following (only one of each is depicted): keyboard (1601), mouse (1602), touchpad (1603), touch screen (1610), data glove (not shown), joystick (1605), microphone (1606), scanner (1607), and camera (1608).

[0259] The computer system (1600) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of at least one human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback provided by touch screens (1610), data gloves (not shown), or joysticks (1605), but there may also be tactile feedback devices that do not act as input devices), audio output devices (e.g., speakers (1609), headphones (not depicted)), visual output devices (e.g., screens (1610), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input functionality, each with or without tactile feedback functionality—some of which are capable of outputting two-dimensional or more than three-dimensional visual outputs through means such as stereoscopic output; virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted)), and printers (not depicted).

[0260] The computer system (1600) may also include user-accessible storage devices and their associated media, such as optical media or similar media (1621) including CD / DVD ROM / RW (1620) equipped with CD / DVD, flash drives (1622), removable hard disk drives or solid state drives (SSDs) (1623), conventional magnetic media such as magnetic tapes and floppy disks (not depicted), and dedicated devices based on ROM / ASIC / PLD such as security dongles (not depicted).

[0261] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not cover transmission media, carrier waves, or other transient signals.

[0262] The computer system (1600) may also include an interface (1654) connected to at least one communication network (1655). The network may be, for example, a wireless network, a wired network, or an optical network. The network may also be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), an in-vehicle and industrial network, a real-time network, a latency-tolerant network, etc. Examples of networks include: local area networks such as Ethernet and wireless LANs; cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; in-vehicle and industrial networks including CAN buses, etc. Some networks typically require an external network interface adapter (e.g., a USB port on the computer system (1600)) attached to some general-purpose data port or peripheral bus (1649); other networks are typically integrated into the core of the computer system (1600) via a system bus described below (e.g., integrated into a PC computer system via an Ethernet interface, or integrated into a smartphone computer system via a cellular network interface). Using any of these networks, the computer system (1600) can communicate with other entities. This communication can be one-way receiving (e.g., broadcasting TV), one-way transmitting (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as connecting to other computer systems using a local area or wide area digital network. Certain protocols and protocol stacks can be used on each of the networks and network interfaces described above.

[0263] The aforementioned human-machine interface device, user-accessible storage device, and network interface can be attached to the core (1640) of the computer system (1600).

[0264] The core (1640) may include at least one CPU (1641), GPUs (1642), a dedicated programmable processing unit in the form of a Field Programmable Gate Array (FPGA) (1643), a hardware accelerator (1644) for certain tasks, a graphics adapter (1650), etc. These devices, along with read-only memory (ROM) (1645), random-access memory (RAM) (1646), and internal mass storage devices (1647) such as internal non-user-accessible hard disk drives, SSDs, etc., can be connected via a system bus (1648). In some computer systems, the system bus (1648) may be accessed in the form of at least one physical plug, allowing for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (1648) or connected to the core's system bus (1648) via a peripheral bus (1649). In one example, a screen (1610) may be connected to a graphics adapter (1650). Peripheral bus architectures include PCI, USB, etc.

[0265] CPUs (1641), GPUs (1642), FPGAs (1643), and accelerators (1644) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (1645) or RAM (1646). Temporary data can also be stored in RAM (1646), while permanent data can be stored, for example, in an internal mass storage device (1647). Fast storage and retrieval of any of the storage devices can be achieved by using a cache memory, which can be closely associated with at least one CPU (1641), GPU (1642), mass storage device (1647), ROM (1645), RAM (1646), etc.

[0266] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be designed and constructed specifically for the purposes of this invention, or may belong to a class well-known and available to those skilled in the art of computer software.

[0267] For example, but not as a limitation, a computer system having an architecture (1600), particularly a core (1640), can provide functionality generated when one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) execute software embodied in at least one tangible computer-readable medium. Such a computer-readable medium can be a medium associated with certain storage devices (e.g., internal core mass storage (1647) or ROM (1645)) that are user-accessible mass storage devices as described above and with the non-volatile nature of the core (1640). Software implementing various aspects of the invention can be stored in such devices and executed by the core (1640). Depending on specific needs, the computer-readable medium may include at least one storage device or chip. The software can cause the core (1640), particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute a particular process or a particular portion of a particular process described in this application, including defining data structures stored in RAM (1646) and modifying such data structures according to the process defined by the software. Alternatively or as an alternative, the computer system may provide functionality generated by hard-wired logic or otherwise embodied in circuitry (e.g., an accelerator (1644)) that operates in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may cover logic, and vice versa. Where appropriate, references to computer-readable media may cover circuitry (e.g., an IC) storing software for execution, circuitry embodying logic for execution, or both. This invention covers any suitable combination of hardware and software.

[0268] The terms “at least one of…” or “one of…” as used in this invention are intended to include any one or a combination of the listed elements. For example, mentioning at least one of A, B, or C, at least one of A, B, and C, at least one of A, B, and / or C, and at least one of A to C are intended to include only A, only B, only C, or any combination thereof. Mentioning one of A or B and one of A and B are intended to include either A or B or (A and B). Where applicable, such as when the elements are not mutually exclusive, the use of “one of…” does not exclude any combination of the listed elements.

[0269] Although several examples of various aspects have been described in this invention, modifications, substitutions, and various alternative equivalents exist that fall within the scope of this invention. Therefore, it should be understood that those skilled in the art will be able to design many systems and methods that, while not explicitly stated or described herein, embody the principles of the invention and are therefore within its spirit and scope.

[0270] The aforementioned disclosure also includes the following features. These features can be combined in various ways, and are not limited to the following combinations.

[0271] (1) A video decoding method, the method comprising: receiving encoded information indicating that a current block is encoded using GPM encoding with a plurality of mixed width sets; determining a set of mixed widths to be applied to the current block from the plurality of mixed width sets based on block size information and GPM information of the current block; determining a mixed width from the determined set of mixed widths; and reconstructing the current block according to the GPM and the determined mixed width.

[0272] (2) According to the method of feature (1), wherein the multiple mixed width sets include a first mixed width set {τ / 4, τ / 2, τ, 2τ, 4τ} and a second mixed width set { τ / 2, τ , 2 τ , 4 τ 8 τ}, and τ is a positive value.

[0273] (3) According to the method of feature (1) or (2), wherein the block size information indicates the block height and block width of the current block, and the plurality of mixed width sets include a first mixed width set and a second mixed width set; and determining the mixed width set includes: when the block width and block height satisfy a first condition, determining the mixed width set as the first mixed width set; when the block width and block height satisfy a second condition, determining the mixed width set as the second mixed width set, the first condition and the second condition being based on a comparison between (i) both the block height and the block width and (ii) a threshold T; and when the block width and block height do not satisfy the first condition and do not satisfy the second condition, determining the mixed width set as one of the first mixed width set and the second mixed width set based on the partitioning pattern in GPM and the aspect ratio between the block height and the block width.

[0274] (4) According to the method of feature (3), wherein the block width and block height do not satisfy the first condition and do not satisfy the second condition, and the aspect ratio is the ratio of the block width to the block height; the two conditions include: (i) the aspect ratio is greater than 1 and the partitioning pattern in the GPM is a near-vertical partitioning pattern, and (ii) the aspect ratio is less than 1 and the partitioning pattern in the GPM is a near-horizontal partitioning pattern; and determining the mixed width set as one of the first mixed width set and the second mixed width set includes: when one of the two conditions is satisfied, the mixed width set is determined as the second mixed width set; and when neither of the two conditions is satisfied, the mixed width set is determined as the first mixed width set.

[0275] (5) According to the method of feature (4), when the geometric dividing angle between the geometric dividing edge of the dividing pattern and the horizontal line is less than or equal to the predefined angle, the dividing pattern in GPM is a near-horizontal dividing pattern.

[0276] (6) According to the method of feature (4), when the geometric dividing angle between the geometric dividing edge of the dividing pattern and the vertical line is less than or equal to the predefined angle, the dividing pattern in GPM is a near-vertical dividing pattern.

[0277] (7) According to the method of feature (1), wherein the block size information indicates the block height and block width of the current block, and a plurality of mixed width sets include a first mixed width set and a second mixed width set; the minimum mixed width in the first mixed width set is less than the minimum mixed width in the second mixed width set; and determining the mixed width set includes: when a third condition is met that each of the block height and block width is less than or equal to a threshold T1, the mixed width set is determined as the first mixed width set, and when a fourth condition is met that each of the block height and block width is greater than or equal to a threshold T2, the mixed width set is determined as the second mixed width set, where T2 is greater than T1; and when neither the third nor the fourth condition is met, the mixed width set is determined as one of the first mixed width set and the second mixed width set based on the partitioning pattern in the GPM and the aspect ratio between the block height and the block width.

[0278] (8) According to the method of feature (7), wherein the third condition is not satisfied, the fourth condition is not satisfied, and the aspect ratio is the ratio of block width to block height; the two conditions include: i) the aspect ratio is greater than 1 and the partitioning pattern in GPM is a near-vertical partitioning pattern, and ii) the aspect ratio is less than 1 and the partitioning pattern in GPM is a near-horizontal partitioning pattern; and determining the mixed width set as one of the first mixed width set and the second mixed width set includes: when one of the two conditions is satisfied, the mixed width set is determined as the second mixed width set; and when neither of the two conditions is satisfied, the mixed width set is determined as the first mixed width set.

[0279] (9) A video decoding method, the method comprising: receiving encoded information indicating that a chroma block is encoded based on a corresponding luma block using CCP and a multi-model filter, the luma block being predicted by GPM and including a first partition and a second partition separated by geometrical partitioning edges of the GPM; determining a multi-model filter for the chroma block, the chroma block including the first partition and the second partition separated by geometrical partitioning edges, the multi-model filter including one of: (i) first filter coefficients determined based on the first partition of the luma block and the first partition of the chroma block, and second filter coefficients determined based on the second partition of the luma block and the second partition of the chroma block; and (i) (i) (ii ... ii) First filter coefficients determined based on a first luminance template of a luminance block and a first chroma template of a chroma block, and second filter coefficients determined based on a second luminance template of a luminance block and a second chroma template of a chroma block, wherein the luminance template of the luminance block is adjacent to the luminance block and includes reconstructed luminance samples, the luminance template including a first luminance template and a second luminance template separated by an extension extending into the luminance template by a geometric segmentation edge, the chroma template of the chroma block being adjacent to the chroma block and including reconstructed chroma samples, the chroma template including a first chroma template and a second chroma template separated by an extension extending into the chroma template by a geometric segmentation edge; and reconstructing the chroma block according to a multi-model filter.

[0280] (10) According to the method of feature (9), wherein determining the multi-model filter includes: determining the first filter coefficients of the multi-model filter based on the first luminance template of the luminance block and the first chrominance template of the chrominance block, and determining the second filter coefficients of the multi-model filter based on the second luminance template of the luminance block and the second chrominance template of the chrominance block.

[0281] (11) The method according to any one of features (9) to (10), wherein the geometric dividing edge of the chroma block is signaled or deduced at the chroma block.

[0282] (12) According to any one of the features (9) to (10), wherein when a single tree partitioning structure is used for the luma block and the chroma block, the geometric partitioning edge separating the chroma block is signaled or deduced at the luma block.

[0283] (13) The method according to any one of the features (9) to (12), wherein the multi-model filter is a linear filter.

[0284] (14) The method according to any one of the features (9) to (12), wherein the multi-model filter is a nonlinear polynomial filter.

[0285] (15) The method according to any one of features (9) to (14), wherein reconstructing the chroma block includes: applying a multi-model filter and GPM to a predicted or reconstructed sample of the luma block to reconstruct the chroma block.

[0286] (16) According to the method of feature (9), wherein determining the multi-model filter includes: determining the first filter coefficients of the multi-model filter based on the first predicted luminance sample in the first partition of the luminance block and the first predicted chrominance sample in the first partition of the chrominance block; and determining the second filter coefficients of the multi-model filter based on the second predicted luminance sample in the second partition of the luminance block and the second predicted chrominance sample in the second partition of the chrominance block.

[0287] (17) A video decoding method, the method comprising: receiving encoded information indicating that a current block in a current image is encoded using an affine mode; and applying a TM procedure to determine a reference block in the current image; Based on motion vector information associated with multiple corners of the reference block, determine the CPMVs of the current block; and based on the determined CPMVs of the current block, reconstruct the current block.

[0288] (18) The method according to feature (17) further includes: scanning multiple blocks located at one of the multiple corners of a reference block in a predefined scanning order to determine a motion vector associated with one of the multiple blocks, the motion information associated with the multiple corners of the reference block including the determined motion vector associated with one of the multiple blocks.

[0289] (19) According to the method of feature (17) or (18), wherein BV indicates the position of the reference block relative to the current block; motion vector information associated with multiple angles of the reference block includes motion vectors associated with multiple angles; and determining the CPMVs of the current block includes determining the CPMVs of the current block as the vector sum of the corresponding motion vectors associated with multiple angles and BV.

[0290] (20) The method according to any one of features (17) to (19), wherein reconstructing the current block includes one of the following: deriving an affine merge candidate for constructing an affine candidate list based on the CPMVs of the current block, and reconstructing the current block based on one of the affine merge candidates in the affine candidate list; or deriving an affine MVP of the current block, and reconstructing the current block based on the affine MVP.

[0291] (21) An apparatus for processing a video stream, comprising processing circuitry configured to perform a method according to any one of features (1) to (20).

[0292] (22) A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a method according to any one of features (1) to (20).

Claims

1. A video decoding method, characterized in that, The method includes: Receive encoded information, which indicates that the current block uses GPM encoding with multiple mixed width sets; Based on the block size information and GPM information of the current block, determine the set of blend widths to be applied to the current block from the multiple blend width sets; Determine the mixing width from the established set of mixing widths; and The current block is reconstructed based on the GPM and the determined blending width.

2. The method according to claim 1, characterized in that, The plurality of mixed width sets include a first mixed width set {τ / 4, τ / 2, τ, 2τ, 4τ} and a second mixed width set { τ / 2, τ , 2 τ , 4 τ 8 τ }, and τ is a positive value.

3. The method according to claim 1 or 2, characterized in that, The block size information indicates the block height and block width of the current block, and the plurality of mixed width sets include a first mixed width set and a second mixed width set; and Determining the set of mixed widths includes: When the block width and the block height satisfy the first condition, the mixed width set is determined as the first mixed width set; When the block width and the block height satisfy a second condition, the mixed width set is determined as the second mixed width set, wherein the first condition and the second condition are based on a comparison between (i) both the block height and the block width and (ii) a threshold T; and When the block width and the block height do not satisfy the first condition and do not satisfy the second condition, the mixed width set is determined to be one of the first mixed width set and the second mixed width set based on the partitioning mode in the GPM and the aspect ratio between the block height and the block width.

4. The method according to claim 3, characterized in that, The two conditions include: (i) the aspect ratio is greater than 1, and the division pattern in the GPM is a near-vertical division pattern; and (ii) the aspect ratio is less than 1, and the division pattern in the GPM is a near-horizontal division pattern; and Determining the blended width set as one of the first blended width set and the second blended width set includes: When one of the two conditions is met, the mixed width set is determined as the second mixed width set; and When neither of the two conditions is met, the mixed width set is determined as the first mixed width set.

5. The method according to claim 4, characterized in that, When the geometric dividing angle between the geometric dividing edge of the dividing pattern and the horizontal line is less than or equal to a predefined angle, the dividing pattern in the GPM is the near-horizontal dividing pattern.

6. The method according to claim 4, characterized in that, When the geometric dividing angle between the geometric dividing edge of the dividing pattern and the vertical line is less than or equal to a predefined angle, the dividing pattern in the GPM is the near-vertical dividing pattern.

7. The method according to claim 1, characterized in that, The block size information indicates the block height and block width of the current block, and the plurality of mixed width sets include a first mixed width set and a second mixed width set; The minimum blend width in the first blend width set is less than the minimum blend width in the second blend width set; and Determining the set of mixed widths includes: When a third condition is met, where each of the block height and the block width is less than or equal to a threshold T1, the mixed width set is determined as the first mixed width set, and When the fourth condition is met that each of the block height and the block width is greater than or equal to the threshold T2, the mixed width set is determined as the second mixed width set, where T2 is greater than T1; as well as When neither the third nor the fourth condition is met, the mixed width set is determined to be one of the first mixed width set and the second mixed width set based on the partitioning mode in the GPM and the aspect ratio between the block height and the block width.

8. A video decoding method, characterized in that, The method includes: Receive encoded information, the encoded information indicating that the chroma block is encoded based on the corresponding luma block using CCP and multi-model filter, the luma block is predicted by GPM, and includes a first partition and a second partition separated by the geometric segmentation edge of the GPM; A multi-model filter is determined for the chroma block, the chroma block comprising a first partition and a second partition separated by a geometric segmentation edge, the multi-model filter comprising one of: (i) first filter coefficients determined based on the first partition of the luminance block and the first partition of the chroma block, and second filter coefficients determined based on the second partition of the luminance block and the second partition of the chroma block; and (ii) the first filter coefficients determined based on a first luminance template of the luminance block and a first chroma template of the chroma block, and the second filter coefficients determined based on a second luminance template of the luminance block and a second chroma template of the chroma block, the luminance template of the luminance block being adjacent to the luminance block and comprising reconstructed luminance samples, the luminance template comprising a first luminance template and a second luminance template separated by an extension of the geometric segmentation edge extending into the luminance template, the chroma template of the chroma block being adjacent to the chroma block and comprising reconstructed chroma samples, the chroma template comprising a first chroma template and a second chroma template separated by an extension of the geometric segmentation edge extending into the chroma template; and The chroma block is reconstructed based on the multi-model filter.

9. The method according to claim 8, characterized in that, Determining the multi-model filter includes: The first filter coefficients of the multi-model filter are determined based on the first luminance template of the luminance block and the first chrominance template of the chrominance block, and the second filter coefficients of the multi-model filter are determined based on the second luminance template of the luminance block and the second chrominance template of the chrominance block.

10. The method according to claim 8 or 9, characterized in that, The geometric dividing edge separating the chroma block is signaled or deduced at the chroma block.

11. The method according to claim 8 or 9, characterized in that, When a single-tree partitioning structure is used for the luma block and the chroma block, the geometric dividing edge separating the chroma block is signaled or deduced at the luma block.

12. The method according to any one of claims 8 to 11, characterized in that, The reconstruction of the chroma block includes: The multi-model filter and the GPM are applied to the predicted or reconstructed samples of the luminance block to reconstruct the chrominance block.

13. The method according to any one of claims 8, characterized in that, Determining the multi-model filter includes: Based on the first predicted luminance sample in the first partition of the luminance block and the first predicted chrominance sample in the first partition of the chrominance block, the first filter coefficients of the multi-model filter are determined; and The second filter coefficients of the multi-model filter are determined based on the second predicted luminance sample in the second partition of the luminance block and the second predicted chrominance sample in the second partition of the chrominance block.

14. A video decoding method, characterized in that, The method includes: Receive encoded information, which indicates that the current block in the current image is encoded using an affine mode; The TM procedure is applied to determine the reference block in the current image; Based on motion vector information associated with multiple angles of the reference block, the CPMVs of the current block are determined; and Reconstruct the current block based on the identified CPMVs of the current block.

15. The method according to claim 14, characterized in that, Also includes: Multiple blocks located at one of the multiple corners of the reference block are scanned in a predefined scanning order to determine a motion vector associated with one of the multiple blocks. The motion information associated with the multiple corners of the reference block includes the determined motion vector associated with one of the multiple blocks.

Images