Block partitioning method for video coding and decoding

Abstract

The present disclosure provides systems and methods for processing video content. The method may include: dividing a plurality of blocks associated with an image into a first partition and a second partition along a division edge; performing inter prediction on the plurality of blocks to generate a first prediction signal of the first partition and a second prediction signal of a second partition; a first prediction signal and a second prediction signal of an edge block associated with the divided edge are mixed.

Description

[0001] This application is a divisional application of Chinese patent application No. 2020800571764, filed on July 22, 2020, entitled "Blocking Method for Video Encoding and Decoding".

[0002] Cross-reference to related applications This disclosure claims priority to U.S. Provisional Application No. 62 / 887,039, filed August 15, 2019, and U.S. Provisional Application No. 62 / 903,970, filed September 23, 2019, both of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure relates generally to video processing, and more specifically to methods and systems for performing motion prediction using triangulation or geometric partitioning. Background Technology

[0004] Video is a set of still images (or "frames") that capture visual information. To reduce storage memory and transmission bandwidth, video can be compressed before storage or transmission and decompressed before display. The compression process is usually called encoding, and the decompression process is usually called decoding. There are various video coding formats that use standardized video coding techniques, the most common being based on prediction, transform, quantization, entropy coding, and in-loop filtering. Video coding standards, such as High Efficiency Video Coding (HEVC / H.265), Universal Video Coding (VVC / H.266), and AVS, specify concrete video coding formats and are developed by standardization organizations. As more and more video standards adopt advanced video coding technologies, the coding efficiency of new video coding standards is also increasing. Summary of the Invention

[0005] This invention provides a method for processing video content. The method may include: dividing a plurality of blocks associated with an image into a first partition and a second partition along a partitioning edge; performing inter-frame prediction on the plurality of blocks to generate a first prediction signal for the first partition and a second prediction signal for the second partition; and mixing the first prediction signal and the second prediction signal of the edge blocks associated with the partitioning edge.

[0006] This invention provides a video content processing system. The system may include: a memory storing a set of instructions; and at least one processor configured to execute the set of instructions such that the system performs: dividing a plurality of blocks associated with an image into a first partition and a second partition along a partitioning edge; performing inter-frame prediction on the plurality of blocks to generate a first prediction signal for the first partition and a second prediction signal for the second partition; and mixing the first prediction signal and the second prediction signal of the edge blocks associated with the partitioning edge.

[0007] Embodiments of this disclosure provide a non-transitory computer-readable medium storing a set of instructions executable by at least one processor of a computer system, wherein execution of the set of instructions causes the computer system to perform a method comprising: dividing a plurality of blocks associated with an image into a first partition and a second partition along a partitioning edge; performing inter-frame prediction on the plurality of blocks to generate a first prediction signal for the first partition and a second prediction signal for the second partition; and mixing the first prediction signal and the second prediction signal of the edge blocks associated with the partitioning edge. Attached Figure Description

[0008] Embodiments and aspects of this disclosure are illustrated in the following detailed description and accompanying drawings. Various features shown in the figures are not drawn to scale.

[0009] Figure 1 The structure of an exemplary video sequence consistent with embodiments of this disclosure is shown.

[0010] Figure 2A A schematic diagram of an exemplary encoding flow for a hybrid video codec system consistent with embodiments of this disclosure is shown.

[0011] Figure 2B A schematic diagram of another exemplary encoding flow for a hybrid video codec system consistent with embodiments of the present disclosure is shown.

[0012] Figure 3A A schematic diagram of an exemplary decoding process for a hybrid video codec system consistent with embodiments of this disclosure is shown.

[0013] Figure 3B A schematic diagram of another exemplary decoding process for a hybrid video codec system consistent with embodiments of this disclosure is shown.

[0014] Figure 4 This is a block diagram of an exemplary apparatus for encoding or decoding video, consistent with embodiments of this disclosure.

[0015] Figure 5 An exemplary inter-frame prediction based on triangle partitioning, consistent with embodiments of this disclosure, is shown.

[0016] Figure 6 An exemplary table for associating a merged index with a motion vector is shown, consistent with embodiments of this disclosure.

[0017] Figure 7 An exemplary chromaticity weight map and an exemplary luminance weight sample consistent with embodiments of this disclosure are shown.

[0018] Figure 8An example of a 4×4 sub-block for storing motion vectors located in a unidirectional or bidirectional prediction region is shown, consistent with embodiments of this disclosure.

[0019] Figure 9 An exemplary syntax structure for a merging pattern consistent with embodiments of this disclosure is shown.

[0020] Figure 10 Another exemplary syntax structure for the merging pattern consistent with embodiments of this disclosure is shown.

[0021] Figure 11 An exemplary geometric partitioning consistent with embodiments of this disclosure is shown.

[0022] Figure 12 An exemplary lookup table for dis[] is shown, consistent with embodiments of this disclosure.

[0023] Figure 13 (including) Figure 13A and Figure 13B An exemplary lookup table for GeoFilter[] consistent with embodiments of this disclosure is shown.

[0024] Figure 14 (including) Figure 14-1 , Figure 14-2 , Figure 14-3 and Figure 14-4 An exemplary lookup table for angleIdx and distanceIdx, consistent with embodiments of this disclosure, is shown.

[0025] Figure 15A (including) Figure 15A-1 and Figure 15A-2 An exemplary lookup table for stepDis is shown, consistent with embodiments of this disclosure.

[0026] Figure 15B (including) Figure 15B-1 and Figure 15B-2 The following shows another exemplary lookup table for stepDis that is consistent with embodiments of this disclosure.

[0027] Figure 15C (including Figures 15C-1, 15C-2, 15C-3 and 15C-4) shows an exemplary lookup table for angleIdx and distanceIdx when the total number of geometric sub-patterns is set to 140, consistent with embodiments of the present disclosure.

[0028] Figure 15D (including) Figure 15D-1 , Figure 15D-2 and Figure 15D-3 An exemplary lookup table for angleIdx and distanceIdx is shown, consistent with embodiments of this disclosure, when the total number of geometric sub-patterns is set to 108.

[0029] Figure 15E (including) Figure 15E-1 and Figure 15E-2 An exemplary lookup table for angleIdx and distanceIdx is shown, consistent with embodiments of this disclosure, when the total number of geometric sub-patterns is set to 80.

[0030] Figure 15F (including) Figure 15F-1 and Figure 15F-2 An exemplary lookup table for angleIdx and distanceIdx is shown, consistent with embodiments of this disclosure, when the total number of geometric sub-patterns is set to 64.

[0031] Figure 16 An exemplary syntax structure for a geometric partitioning pattern consistent with embodiments of this disclosure is shown.

[0032] Figure 17A Another exemplary syntax structure for a geometric partitioning pattern consistent with embodiments of this disclosure is shown.

[0033] Figure 17B This illustrates yet another exemplary syntax structure for a geometric partitioning pattern consistent with embodiments of the present disclosure.

[0034] Figure 18 An exemplary sub-block transformation of an inter-frame prediction block consistent with embodiments of this disclosure is shown.

[0035] Figure 19 An example of a uniform syntax structure consistent with embodiments of this disclosure is shown.

[0036] Figure 20 Another example of a unified syntax structure consistent with embodiments of this disclosure is shown.

[0037] Figure 21 This illustrates yet another example of a unified syntax structure consistent with embodiments of the present disclosure.

[0038] Figure 22A (including Figures 22A-1, 22A-2, 22A-3 and 22A-4) illustrates an exemplary lookup table for angleIdx and distanceIdx, including triangles and geometric divisions, consistent with embodiments of this disclosure.

[0039] Figure 22B Another exemplary lookup table for angleIdx and distanceIdx, which are consistent with embodiments of this disclosure, is shown.

[0040] Figure 23 The illustration shows an example of an angle that allows only the larger block size to be divided, consistent with embodiments of this disclosure.

[0041] Figure 24 (including Figures 24-1, 24-2, 24-3 and 24-4) shows yet another exemplary lookup table for angleIdx and distanceIdx, which are consistent with embodiments of the present disclosure and include triangles and geometric divisions.

[0042] Figure 25 (including Figures 25-1, 25-2, 25-3 and 25-4) illustrates yet another exemplary lookup table for angleIdx and distanceIdx, which are consistent with embodiments of the present disclosure and include triangles and geometric partitions.

[0043] Figure 26 An exemplary lookup table for Dis[] consistent with embodiments of this disclosure is shown.

[0044] Figure 27 An exemplary coding unit syntax structure consistent with embodiments of this disclosure is shown.

[0045] Figure 28 An example of SBT and GEO partitioning consistent with embodiments of this disclosure is shown.

[0046] Figure 29 Another exemplary coding unit syntax structure consistent with embodiments of this disclosure is shown.

[0047] Figure 30 (including) Figure 30-1 and Figure 30-2 An exemplary lookup table for angleIdx and distanceIdx is shown, consistent with embodiments of this disclosure, when the total number of geometric sub-patterns is set to 80.

[0048] Figure 31 (including) Figure 31-1 and Figure 31-2 An exemplary lookup table for angleIdx and distanceIdx is shown, consistent with embodiments of this disclosure, when the total number of geometric sub-patterns is set to 64.

[0049] Figure 32 An exemplary lookup table for Rho[] consistent with embodiments of this disclosure is shown.

[0050] Figure 33 A table showing the number of each operation for each block, consistent with embodiments of this disclosure, is provided.

[0051] Figure 34 An exemplary lookup table for Rhosubblk[] consistent with embodiments of this disclosure is shown.

[0052] Figure 35A An exemplary mask with an angle of 135° consistent with embodiments of this disclosure is shown.

[0053] Figure 35B An exemplary mask at a 45° angle, consistent with embodiments of this disclosure, is shown.

[0054] Figure 36A An exemplary mask with an angle of 135° consistent with embodiments of this disclosure is shown.

[0055] Figure 36B An exemplary mask at 45° consistent with embodiments of this disclosure is shown.

[0056] Figure 37 Exemplary angles of triangular partitioning patterns of different block shapes consistent with embodiments of this disclosure are shown.

[0057] Figure 38 (including Figures 38-1, 38-2, 38-3 and 38-4) shows an exemplary lookup table for angleIdx and distanceIdx when the total number of geometric sub-patterns is set to 140, consistent with embodiments of the present disclosure.

[0058] Figure 39 This is a flowchart of an exemplary method for processing video content, consistent with embodiments of this disclosure. Detailed Implementation

[0059] Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. All descriptions herein refer to the accompanying drawings, and unless otherwise stated, the same numerals in different drawings denote the same or similar elements. The embodiments set forth in the following description of exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with the aspects of the invention recounted in the appended claims. Unless otherwise specifically stated, the term "or" covers all possible combinations unless impractical. For example, if it is stated that a component may include A or B, then unless otherwise explicitly stated or impractical, the component may include A, or B, or A and B. As a second example, if it is stated that a component may include A, B, or C, then unless otherwise explicitly stated or impractical, the component may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0060] Video encoding systems are commonly used to compress digital video signals, for example, to reduce the storage space consumption associated with such signals or to reduce transmission bandwidth consumption. With the increasing popularity of high-definition (HD) video (e.g., 1920×1080 pixels) in various video compression applications, such as online video streaming, video conferencing, or video surveillance, the need to develop encoding tools that can improve the efficiency of video data compression is constantly growing.

[0061] For example, video surveillance is increasingly widely used in many application scenarios (such as security, traffic, and environmental monitoring), and the number and resolution of monitoring devices are growing rapidly. Many video surveillance applications prefer to provide users with high-definition video to collect more information, as each frame of high-definition video contains more pixels to capture this information. However, high-definition video bitstreams may have high bit rates, requiring high bandwidth transmission and large storage space. For example, a surveillance video stream with an average resolution of 1920×1080 may require up to 4 Mbps of bandwidth for real-time transmission. In addition, video surveillance is generally conducted continuously 24 / 7, and storing video data poses a significant challenge to storage systems. Therefore, the high bandwidth and large storage requirements of high-definition video have become the main constraints on its large-scale deployment in video surveillance.

[0062] Video is a set of still images (or "frames") arranged in chronological order to store visual information. Video capture devices (such as cameras) can be used to capture and store these images in a time-series manner, and video playback devices (such as televisions, computers, smartphones, tablets, video players, or any end-user terminal with a display capability) can be used to display such images in chronological order. Furthermore, in some applications, video capture devices can transmit the captured video in real time to video playback devices (such as computers with monitors) for monitoring, conferencing, live streaming, etc.

[0063] To reduce the storage space and transmission bandwidth required for such applications, video can be compressed before storage and transmission, and decompressed before display. Compression and decompression can be implemented by software executed by a processor (e.g., a processor in a general-purpose computer) or dedicated hardware. The compression module is generally called an "encoder," and the decompression module is generally called a "decoder." Encoders and decoders can be collectively referred to as a "codec." Encoders and decoders can be implemented as any of a variety of suitable hardware, software, or combinations thereof. For example, hardware implementations of encoders and decoders can include circuits such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, or any combination thereof. Software implementations of encoders and decoders can include program code, computer-executable instructions, firmware, or any suitable computer-implemented algorithm or process embedded in a computer-readable medium. Video compression and decompression can be implemented using various algorithms or standards, such as MPEG-1, MPEG-2, MPEG-4, H.26x series, etc. In some applications, a codec can decompress video from a first encoding standard and recompress the decompressed video using a second encoding standard; in this case, the codec can be called a "transcoder."

[0064] Video coding processes can identify and retain useful information that can be used to reconstruct an image, while ignoring unimportant information that cannot be reconstructed. If the ignored, unimportant information cannot be fully reconstructed, such a coding process can be called "lossy." Otherwise, it can be called "lossless." Most coding processes are lossy, a trade-off to reduce required storage space and transmission bandwidth.

[0065] Useful information about the image being encoded (referred to as the "current image") includes changes relative to a reference image (e.g., a previously encoded and reconstructed image). These changes can include variations in pixel position, brightness, or color, with positional changes being the most important. The positional changes of a set of pixels representing an object can reflect the object's movement between the reference and current images.

[0066] An image encoded without referencing another image (i.e., it is its own reference image) is called an "I-image". An image encoded using a previous image as a reference image is called a "P-image". An image encoded using both previous and future images as reference images (i.e., the reference is "bidirectional") is called a "B-image".

[0067] As mentioned earlier, video surveillance using high-definition video faces the challenges of high bandwidth and large storage requirements. To address these challenges, the bit rate of the encoded video can be reduced. Among I-, P-, and B-images, I-images have the highest bit rate. Since the background of most surveillance videos is almost static, one way to reduce the overall bit rate of the encoded video is to use fewer I-images for video encoding.

[0068] However, the improvement from using fewer I-images is likely negligible, as I-images are generally not dominant in encoded video. For example, in a typical video bitstream, the ratio of I-, B-, and P-images can be 1:20:9, where I-images can account for less than 10% of the total bitrate. In other words, in this example, even if all I-images were removed, the bitrate reduction would not exceed 10%.

[0069] Figure 1 The illustration shows the structure of an example video sequence 100 consistent with embodiments of this disclosure. Video sequence 100 may be live video or video that has been captured and archived. Video 100 may be real video, computer-generated video (e.g., computer game video), or a combination thereof (e.g., real video with augmented reality effects). Video sequence 100 may receive video from a video capture device (e.g., a camera), a video archive containing previously captured video (e.g., a video file stored on a storage device), or a video providing interface (e.g., a video broadcast transceiver).

[0070] like Figure 1 As shown, video sequence 100 may include a series of images arranged temporally along a time axis, including images 102, 104, 106, and 108. Images 102-106 are consecutive, and there are more images between images 106 and 108. Figure 1 In this diagram, image 102 is an I-image, and its reference image is image 102 itself. Image 104 is a P-image, and its reference image is image 102, as indicated by the arrow. Image 106 is a B-image, and its reference images are images 104 and 108, as indicated by the arrow. In some embodiments, the reference image of an image (e.g., image 104) may not immediately precede or follow that image. For example, the reference image of image 104 may be an image preceding image 102. It should be noted that the reference images of images 102-106 are merely examples, and this disclosure does not limit their use. Figure 1 The example shown is an embodiment of the reference image.

[0071] Due to the computational complexity of encoding and decoding tasks, video codecs typically do not encode or decode the entire image at once. Instead, they can divide the image into basic segments and encode or decode the image segment by segment. Such basic segments are referred to herein as basic processing units (“BPUs”). For example, Figure 1 Structure 110 illustrates an example structure of an image (e.g., any one of images 102-108) from video sequence 100. In structure 110, the image is divided into 4×4 basic processing units, the boundaries of which are indicated by dashed lines. In some embodiments, the basic processing unit may be referred to as a “macroblock” in some video coding standards (e.g., MPEG series, H.261, H.263, or H.264 / AVC), or as a “coding tree unit” (“CTU”) in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing units in the image can have variable sizes, such as 128×128, 64×64, 32×32, 16×16, 4×8, 16×32, or pixels of arbitrary shape and size. The size and shape of the basic processing units for the image can be chosen based on a balance between coding efficiency and the level of detail to be maintained within the basic processing units.

[0072] A basic processing unit can be a logical unit that may include a set of different types of video data stored in computer memory (e.g., in a video frame buffer). For example, a basic processing unit for a color image may include a luminance component (Y) representing non-color luminance information, one or more chrominance components (e.g., Cb and Cr) representing color information, and associated syntax elements, where the luminance and chrominance components may have the same size as the basic processing unit. In some video coding standards (e.g., H.265 / HEVC or H.266 / VVC), the luminance and chrominance components may be referred to as “code tree blocks” (“CTBs”). Any operation performed on a basic processing unit may be repeated on each of its luminance and chrominance components.

[0073] Video encoding involves multiple operational stages, examples of which will be provided in [the following section]. Figure 2A-2B As detailed in 3A-3B. For each stage, the size of the basic processing unit may still be too large to process, so it can be further divided into segments referred to in this disclosure as "basic processing subunits". In some embodiments, the basic processing subunit may be referred to as a "block" in some video coding standards (e.g., MPEG series, H.261, H.263, or H.264 / AVC), or as a "coding unit" ("CU") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing subunit may have the same size as or smaller than the basic processing unit. Similar to the basic processing unit, the basic processing subunit is also a logical unit that may include a set of different types of video data (e.g., Y, Cb, Cr, and associated syntax elements) stored in computer memory (e.g., in a video frame buffer). Any operation performed on a basic processing subunit may be repeated for each of its luminance and chrominance components. It should be noted that this division can be made to a deeper level as needed for processing. It should also be noted that different schemes can be used to divide the basic processing units for different stages.

[0074] For example, in the pattern decision-making phase (examples will be provided in...) Figure 2B (Detailed in the text) The encoder can decide which prediction mode (e.g., intra-image prediction or inter-image prediction) to use for the basic processing unit, which may be too large to make a decision. The encoder can break down the basic processing unit into multiple basic processing subunits (e.g., CUs in H.265 / HEVC or H.266 / VVC) and determine the prediction type for each individual basic processing subunit.

[0075] For another example, in the prediction phase (where examples will be...) Figure 2AAs detailed in the text, encoders can perform prediction operations at the level of basic processing subunits (e.g., CUs). However, in some cases, the basic processing subunits may still be too large to handle. The encoder can further break down the basic processing subunits into smaller segments (e.g., referred to as "prediction blocks" or "PBs" in H.265 / HEVC or H.266 / VVC), at which prediction operations can be performed.

[0076] For another example, in the transformation phase (the example of which will be...) Figure 2A (Detailed in the text) The encoder can perform transformation operations for residual basic processing subunits (e.g., CUs). However, in some cases, the basic processing subunit may still be too large to process. The encoder can further break down the basic processing subunit into smaller segments (e.g., referred to as "transform blocks" or "TBs" in H.265 / HEVC or H.266 / VVC), at which level transformation operations can be performed. It should be noted that the partitioning scheme of the same basic processing subunit in the prediction and transformation stages can be different. For example, in H.265 / HEVC or H.266 / VVC, the prediction block and the transform block of the same CU can have different sizes and numbers.

[0077] exist Figure 1 In structure 110, the basic processing unit 112 is further divided into 3×3 basic processing sub-units, the boundaries of which are indicated by dashed lines. Different basic processing units of the same image can be divided into basic processing sub-units using different schemes.

[0078] In some implementations, to provide parallel processing and fault tolerance for video encoding and decoding, an image can be divided into multiple regions for processing, so that for one region of the image, the encoding or decoding process does not depend on information from any other region of the image. In other words, each region of the image can be processed independently. By doing so, the codec can process different regions of the image in parallel, thereby improving encoding efficiency. Furthermore, when data in one region is corrupted during processing or lost in network transmission, the codec can correctly encode or decode other regions of the same image without relying on the corrupted or lost data, thus providing fault tolerance. In some video coding standards, an image can be divided into different types of regions. For example, H.265 / HEVC and H.266 / VVC provide two types of regions: "slices" and "tiles". It should also be noted that different images in the video sequence 100 may have different partitioning schemes for dividing the image into multiple regions.

[0079] For example, in Figure 1In the diagram, structure 110 is divided into three regions 114, 116, and 118, whose boundaries are shown as solid lines within structure 110. Region 114 includes four basic processing units. Each of regions 116 and 118 includes six basic processing units. It should be noted that... Figure 1 The basic processing unit, basic processing subunit, and region of structure 110 are merely examples, and this disclosure does not limit its embodiments.

[0080] Figure 2A A schematic diagram of an example encoding flow 200A consistent with embodiments of this disclosure is illustrated. For example, encoding flow 200A may be executed by an encoder. Figure 2A As shown, according to process 200A, the encoder can encode video sequence 202 into video bitstream 228. Similar to... Figure 1 Video sequence 100 and video sequence 202 may include a set of images arranged in chronological order (referred to as "original images"). Similar to... Figure 1 In structure 110, each raw image of video sequence 202 can be divided into basic processing units, basic processing subunits, or regions by an encoder for processing. In some embodiments, the encoder can perform process 200A at the basic processing unit level for each raw image of video sequence 202. For example, the encoder can perform process 200A iteratively, wherein the encoder can encode the basic processing unit in one iteration of process 200A. In some embodiments, the encoder can perform process 200A in parallel for regions (e.g., regions 114-118) of each raw image of video sequence 202.

[0081] exist Figure 2AIn this process, the encoder can provide the basic processing unit (referred to as the "raw BPU") of the original image of video sequence 202 to prediction stage 204 to generate prediction data 206 and prediction BPU 208. The encoder can subtract the prediction BPU from the raw BPU to generate residual BPU 210. The encoder can provide residual BPU 210 to transform stage 212 and quantization stage 214 to generate quantization transform coefficients 216. The encoder can provide prediction data 206 and quantization transform coefficients 216 to binary encoding stage 226 to generate video bitstream 228. Components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 can be referred to as the "forward path". During process 200A, after quantization stage 214, the encoder can provide quantization transform coefficients 216 to inverse quantization stage 218 and inverse transform stage 220 to generate reconstructed residual BPU 222. The encoder can add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224, which is used in the next iteration of process 200A in prediction phase 204. Components 218, 220, 222, and 224 of process 200A can be referred to as the "reconstruction path". The reconstruction path can be used to ensure that the encoder and decoder use the same reference data for prediction.

[0082] The encoder can iteratively execute process 200A to encode each raw BPU (in the forward path) of the original image and generate prediction reference 224 for encoding the next raw BPU (in the reconstruction path) of the original image. After encoding all raw BPUs of the original image, the encoder can continue to encode the next image in the video sequence 202.

[0083] Referring to process 200A, the encoder can receive a video sequence 202 generated by a video acquisition device (e.g., a camera). The term "receive" as used herein can refer to any action of receiving, inputting, acquiring, retrieving, obtaining, reading, accessing, or otherwise inputting data.

[0084] In prediction phase 204, during the current iteration, the encoder can receive the original BPU and prediction reference 224, and perform prediction operations to generate prediction data 206 and prediction BPU 208. Prediction reference 224 can be generated from the reconstruction path of previous iterations in process 200A. The purpose of prediction phase 204 is to reduce information redundancy by extracting prediction data 206, which can be used to reconstruct the original BPU into prediction BPU 208 from prediction data 206 and prediction reference 224.

[0085] Ideally, the predicted BPU 208 should be identical to the original BPU. However, due to non-ideal prediction and reconstruction operations, the predicted BPU 208 is typically slightly different from the original BPU. To record this difference, after generating the predicted BPU 208, the encoder can subtract it from the original BPU to generate the remaining BPU 210. For example, the encoder can subtract the corresponding pixel value in the predicted BPU 208 from the pixel values ​​(e.g., grayscale or RGB values) of the original BPU. Each pixel in the residual BPU 210 has a residual value as the result of subtracting the corresponding pixel from the original BPU and the predicted BPU 208. Compared to the original BPU, the predicted data 206 and the residual BPU 210 can have fewer bits, but they can be used to reconstruct the original BPU without significantly degrading quality. Therefore, the original BPU is compressed.

[0086] To further compress the residual BPU 210, in the transform stage 212, the encoder can reduce its spatial redundancy by decomposing the residual BPU 210 into a set of two-dimensional “fundamental patterns,” each of which is associated with “transform coefficients.” The fundamental patterns can have the same size (e.g., the size of the residual BPU 210). Each fundamental pattern can represent a frequency component of the residual BPU 210 (e.g., the frequency of brightness variation). A fundamental pattern cannot be reproduced from any combination of any other fundamental patterns (e.g., a linear combination). In other words, the decomposition decomposes the variations of the residual BPU 210 into the frequency domain. This decomposition is analogous to the discrete Fourier transform of a function, where the fundamental patterns are analogous to the fundamental functions of the discrete Fourier transform (e.g., trigonometric functions), and the transform coefficients are analogous to the coefficients associated with the fundamental functions.

[0087] Different transform algorithms can use different base modes. Various transform algorithms, such as discrete cosine transform, discrete sine transform, etc., can be used in transform stage 212. The transform at transform stage 212 is reversible. That is, the encoder can recover the residual BPU 210 through the inverse operation of the transform (called the "inverse transform"). For example, to recover the pixels of the residual BPU 210, the inverse transform can be to multiply the values ​​of the corresponding pixels in the base mode by their respective associated coefficients and sum the products to produce a weighted sum. For video coding standards, both the encoder and decoder can use the same transform algorithm (and therefore have the same base mode). Therefore, the encoder can only record the transform coefficients, and the decoder can reconstruct the residual BPU 210 from these coefficients without receiving the base mode from the encoder. Compared to the residual BPU 210, the transform coefficients can have fewer bits, and they can be used to reconstruct the residual BPU 210 without significantly degrading the quality. Therefore, the residual BPU 210 is further compressed.

[0088] The encoder can further compress the transform coefficients in quantization stage 214. In the transform process, different fundamental modes can represent different frequencies of change (e.g., brightness change frequencies). Since the human eye is generally better at recognizing low-frequency changes, the encoder can ignore information about high-frequency changes without significantly degrading the decoding quality. For example, in quantization stage 214, the encoder can generate quantized transform coefficients 216 by dividing each transform coefficient by an integer value (called the "quantization parameter") and rounding the quotient to its nearest integer. Through this operation, some transform coefficients of the high-frequency fundamental modes can be converted to zero, while the transform coefficients of the low-frequency fundamental modes can be converted to smaller integers. The encoder can ignore the zero-value quantized transform coefficients 216, further compressing the transform coefficients. The quantization process is also reversible, where the quantized transform coefficients 216 can be reconstructed into transform coefficients in the inverse operation of quantization (called "inverse quantization").

[0089] Because the encoder ignores the remainder of such division during rounding operations, quantization stage 214 may be lossy. Typically, quantization stage 214 may produce the most information loss in process 200A. The greater the information loss, the fewer bits are needed for the quantization transform coefficients 216. To obtain different levels of information loss, the encoder can use different values ​​for the quantization parameter or any other parameter in the quantization process.

[0090] In the binary encoding stage 226, the encoder can encode the prediction data 206 and the quantization transform coefficients 216 using binary encoding techniques, such as entropy coding, variable-length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless or lossy compression algorithm. In some embodiments, in addition to the prediction data 206 and the quantization transform coefficients 216, the encoder can encode other information in the binary encoding stage 226, such as the prediction mode used in the prediction stage 204, the parameters of the prediction operation, the transform type in the transform stage 212, the parameters of the quantization process (e.g., quantization parameters), encoder control parameters (e.g., bit rate control parameters), etc. The encoder can use the output data of the binary encoding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 can be further packaged for network transmission.

[0091] Referring to the reconstruction path of process 200A, in the inverse quantization stage 218, the encoder can perform inverse quantization on the quantized transform coefficients 216 to generate reconstructed transform coefficients. In the inverse transform stage 220, the encoder can generate a reconstruction residual BPU 222 based on the reconstructed transform coefficients. The encoder can add the reconstruction residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224 that will be used in the next iteration of process 200A.

[0092] It should be noted that other variations of process 200A can be used to encode video sequence 202. In some embodiments, the stages of process 200A may be performed by the encoder in different orders. In some embodiments, one or more stages of process 200A may be combined into a single stage. In some embodiments, a single stage of process 200A may be divided into multiple stages. For example, transform stage 212 and quantization stage 214 may be combined into a single stage. In some embodiments, process 200A may include additional stages. In some embodiments, process 200A may be omitted. Figure 2A One or more stages in the process.

[0093] Figure 2B A schematic diagram of another example encoding flow 200B consistent with embodiments of the present disclosure is illustrated. Flow 200B can be modified from flow 200A. For example, flow 200B can be used by an encoder conforming to a hybrid video coding standard (e.g., H.26x series). Compared to flow 200A, the forward path of flow 200B additionally includes a mode decision stage 230 and divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044. The reconstruction path of flow 200B additionally includes a loop filtering stage 232 and a buffer 234.

[0094] Generally, prediction techniques can be categorized into two types: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-image prediction or "intra-frame prediction") uses pixels from one or more coded neighboring BPUs in the same image to predict the current BPU. That is, the prediction reference 224 in spatial prediction can include neighboring BPUs. Spatial prediction can reduce the inherent spatial redundancy of images. Temporal prediction (e.g., inter-image prediction or "inter-frame prediction") uses regions from one or more coded images to predict the current BPU. That is, the prediction reference 224 in temporal prediction can include coded images. Temporal prediction can reduce the inherent temporal redundancy of images.

[0095] According to reference flow 200B, in the forward path, the encoder performs prediction operations in the spatial prediction phase 2042 and the temporal prediction phase 2044. For example, in the spatial prediction phase 2042, the encoder may perform intra-frame prediction. For the original BPU of the image being encoded, prediction reference 224 may include one or more adjacent BPUs that have been encoded (in the forward path) and reconstructed (in the reconstruction path) in the same image. The encoder can generate the predicted BPU 208 by interpolating the adjacent BPUs. Interpolation techniques may include, for example, linear extrapolation or interpolation, polynomial extrapolation or interpolation, etc. In some embodiments, the encoder may perform interpolation at the pixel level, for example by interpolating the corresponding pixels of each pixel of the predicted BPU 208. The adjacent BPUs used for interpolation can be positioned relative to the original BPU from various directions, such as in the vertical direction (e.g., at the top of the original BPU), the horizontal direction (e.g., to the left of the original BPU), the diagonal direction (e.g., to the lower left, lower right, upper left, upper right), or any direction defined in the video coding standard used. For intra-frame prediction, prediction data 206 may include, for example, the location (e.g., coordinates) of the neighboring BPUs used, the size of the neighboring BPUs used, the interpolation parameters, and the neighboring BPUs used relative to the original orientation BPU.

[0096] In another example, during the temporal prediction phase 2044, the encoder can perform inter-frame prediction. For the original BPU of the current image, the prediction reference 224 can include one or more images (referred to as "reference images") that have been encoded (in the forward path) and reconstructed (in the reconstruction path). In some embodiments, the reference image can be a BPU encoded and reconstructed from the original BPU. For example, the encoder can add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a reconstructed BPU. When all reconstructed BPUs of the same image are generated, the encoder can generate the reconstructed image as the reference image. The encoder can perform a "motion estimation" operation to search for matching regions within the range of the reference image (referred to as a "search window"). The position of the search window in the reference image can be determined based on the position of the original BPU in the current image. For example, the search window can be centered at a position in the reference image that has the same coordinates as the original BPU in the current image and can extend outward by a predetermined distance. When the encoder identifies a region in the search window that is similar to the original BPU (e.g., by using a pixel recursive algorithm, block matching algorithm, etc.), the encoder can determine such a region as a matching region. The matching region can have a different size than the original BPU (e.g., less than, equal to, greater than, or in a different shape). This is because the reference image and the current image are temporally separated on the timeline (e.g., as shown in the image). Figure 1As shown), the matching region can be considered to "move" to the original BPU's location over time. The encoder can record the direction and distance of this motion as a "motion vector." When using multiple reference images (e.g., such as...), Figure 1 In image 106, the encoder can search for matching regions and determine the associated motion vector for each reference image. In some embodiments, the encoder can assign weights to the pixel values ​​of the matching regions of each matching reference image.

[0097] Motion estimation can be used to identify various types of motion, such as translation, rotation, scaling, etc. For inter-frame prediction, prediction data 206 may include, for example, the location (e.g., coordinates) of the matching region, the motion vector associated with the matching region, the number of reference images, the weights associated with the reference images, etc.

[0098] To generate the predicted BPU 208, the encoder can perform a "motion compensation" operation. Motion compensation can be used to reconstruct the predicted BPU 208 based on the predicted data 206 (e.g., motion vectors) and the predicted reference 224. For example, the encoder can move a matching region of the reference image according to the motion vectors, where the encoder can predict the original BPU of the current image. When using multiple reference images (e.g., such as...), Figure 1 In image 106), the encoder can move the matching region of the reference image based on the average pixel value of each motion vector and the matching region. In some embodiments, if the encoder has assigned weights to the pixel values ​​of the matching regions of each matching reference image, the encoder can add the weighted sum of the pixel values ​​of the moved matching regions.

[0099] In some embodiments, inter-frame prediction can be unidirectional or bidirectional. Unidirectional inter-frame prediction can use one or more reference images in the same temporal direction as the current image. For example, Figure 1 The image in the image is a one-way inter-frame prediction image, where the reference image (i.e., image 102) precedes image 104. Two-way inter-frame prediction can use one or more reference images in two temporal directions relative to the current image. For example, Figure 1 Image 106 in the image is a bidirectional inter-frame prediction image, in which the reference images (i.e., images 104 and 108) are in two temporal directions relative to image 104.

[0100] Continuing along the forward path of reference flow 200B, after spatial prediction 2042 and temporal prediction stages 2044, in the mode decision stage 230, the encoder can select a prediction mode (e.g., one of intra-frame prediction or inter-frame prediction) for the current iteration of flow 200B. For example, the encoder can perform rate-distortion optimization techniques, whereby the encoder selects a prediction mode to minimize the value of the cost function based on the bit rate of the candidate prediction modes and the distortion of the reconstructed reference image under the candidate prediction modes. Based on the selected prediction mode, the encoder can generate the corresponding prediction BPU 208 and prediction data 206.

[0101] In the reconstruction path of process 200B, if intra-frame prediction mode is selected in the forward path, the encoder can directly provide prediction reference 224 to spatial prediction stage 2042 for later use (e.g., for interpolation of the next BPU in the current image) after generating prediction reference 224 (e.g., the current BPU that has been encoded and reconstructed in the current image). If inter-frame prediction mode is selected in the forward path, the encoder can provide prediction reference 224 to loop filtering stage 232 after generating prediction reference 224 (e.g., the current image where all BPUs have been encoded and reconstructed), where the encoder can apply loop filters to prediction reference 224 to reduce or eliminate distortions introduced by inter-frame prediction (e.g., block artifacts). The encoder can apply various loop filtering techniques in loop filtering stage 232, such as deblocking, sample adaptive shifting, adaptive loop filtering, etc. The reference image filtered by the loop can be stored in buffer 234 (or "decoded image buffer") for later use (e.g., as an inter-frame prediction reference image for future images of video sequence 202). The encoder can store one or more reference images in buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the encoder can encode the parameters of the loop filter (e.g., loop filter strength), along with the quantization transform coefficients 216, the prediction data 206, and other information in the binary encoding stage 226.

[0102] Figure 3A A schematic diagram of an example decoding process 300A consistent with embodiments of this disclosure is shown. Process 300A may be corresponding to... Figure 2A The compression process 200A in the decompression process is described. In some embodiments, process 300A may be similar to the reconstruction path of process 200A. The decoder may decode the video bitstream 228 into video stream 304 according to process 300A. Video stream 304 may be very similar to video sequence 202. However, due to the compression and decompression processes (e.g., Figure 2A-2B Information loss during the quantization stage 214) typically results in video stream 304 not being entirely identical to video sequence 202. Similar to... Figure 2A-2BIn processes 200A and 200B, the decoder can perform process 300A at the basic processing unit (BPU) level for each image encoded in the video bitstream 228. For example, the decoder can perform process 300A iteratively, where the decoder can decode the basic processing unit in one iteration of process 300A. In some embodiments, the decoder can perform process 300A in parallel for a region (e.g., region 114-118) of each image encoded in the video bitstream 228.

[0103] exist Figure 3A The decoder may provide a portion of the video bitstream 228 associated with a basic processing unit (referred to as an "encoded BPU") of the encoded image to the binary decoding stage 302. In the binary decoding stage 302, the decoder may decode this portion into prediction data 206 and quantization transform coefficients 216. The decoder may provide the quantization transform coefficients 216 to the inverse quantization stage 218 and the inverse transform stage 220 to generate a reconstructed residual BPU 222. The decoder may provide the prediction data 206 to the prediction stage 204 to generate a prediction BPU 208. The decoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction reference 224. In some embodiments, the prediction reference 224 may be stored in a buffer (e.g., a decoded image buffer in computer memory). The decoder may provide the prediction reference 224 to the prediction stage 204 to perform a prediction operation in the next iteration of process 300A.

[0104] The decoder can iteratively execute process 300A to decode each encoded BPU of the encoded image and generate a prediction reference 224 for encoding the next encoded BPU of the encoded image. After decoding all encoded BPUs of the encoded image, the decoder can output the image to video stream 304 for display and continue decoding the next encoded image in video bit stream 228.

[0105] In binary decoding stage 302, the decoder can perform the inverse operation of the binary encoding technique used by the encoder (e.g., entropy coding, variable-length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless compression algorithm). In some embodiments, in addition to the prediction data 206 and quantization transform coefficients 216, the decoder can decode other information in binary decoding stage 302, such as the prediction mode, parameters of the prediction operation, transform type, quantization process parameters (e.g., quantization parameters), encoder control parameters (e.g., bit rate control parameters), etc. In some embodiments, if the video bitstream 228 is transmitted over the network in packets, the decoder can unpack it before sending the video bitstream 228 to binary decoding stage 302.

[0106] Figure 3B A schematic diagram of another example decoding flow 300B consistent with embodiments of the present disclosure is shown. Flow 300B can be modified from flow 300A. For example, flow 300B can be used by a decoder conforming to a hybrid video coding standard (e.g., H.26x series). Compared to flow 300A, flow 300B additionally divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and additionally includes a loop filtering stage 232 and a buffer 234.

[0107] In process 300B, for the encoded basic processing unit (referred to as the "current BPU") of the encoded image to be decoded (referred to as the "current image"), the prediction data 206 decoded by the decoder from the binary decoding stage 302 can include various types of data, depending on the prediction mode used by the encoder to encode the current BPU. For example, if the encoder uses intra-frame prediction to encode the current BPU, the prediction data 206 can include a prediction mode indicator (e.g., a flag value) indicating intra-frame prediction, parameters for the intra-frame prediction operation, etc. Parameters for the intra-frame prediction operation can include, for example, the positions (e.g., coordinates) of one or more neighboring BPUs used as references, the sizes of neighboring BPUs, interpolation parameters, the orientation of neighboring BPUs relative to the original BPU, etc. As another example, if the encoder uses inter-frame prediction to encode the current BPU, the prediction data 206 can include an inter-frame prediction mode indicator (e.g., a flag value), parameters for the inter-frame prediction operation, etc. The parameters of the inter-frame prediction operation may include, for example, the number of reference images associated with the current BPU, the weights associated with each reference image, the positions (e.g., coordinates) of one or more matching regions in each reference image, and one or more motion vectors associated with each matching region.

[0108] Based on the prediction mode indicator, the decoder can determine whether to perform spatial prediction (e.g., intra-frame prediction) in the spatial prediction phase 2042 or temporal prediction (e.g., inter-frame prediction) in the temporal prediction phase 2044. Details regarding the execution of either spatial or temporal prediction are provided in... Figure 2B As described in the previous section, it will not be repeated below. After performing the spatial or temporal prediction, the decoder can generate a prediction BPU 208. The decoder can add the prediction BPU 208 and the reconstruction residual BPU 222 to generate a prediction reference 224, as shown below. Figure 3A As described in [the text].

[0109] In process 300B, the decoder can provide prediction reference 224 to either spatial prediction stage 2042 or temporal prediction stage 2044 for performing prediction operations in the next iteration of process 300B. For example, if the current BPU is decoded using intra-frame prediction in spatial prediction stage 2042, the decoder can directly provide prediction reference 224 to spatial prediction stage 2042 for later use (e.g., for interpolating the next BPU of the current image) after generating prediction reference 224 (e.g., the decoded current BPU). If the current BPU is decoded using inter-frame prediction in temporal prediction stage 2044, the encoder can send prediction reference 224 to loop filtering stage 232 after generating prediction reference 224 (e.g., a reference image where all BPUs have been decoded) to reduce or eliminate distortion (e.g., block artifacts). The decoder can... Figure 2B The loop filter is applied to prediction reference 224 in the manner described herein. The loop-filtered reference image may be stored in buffer 234 (e.g., a decoded image buffer in computer memory) for later use (e.g., as an inter-frame prediction reference image used as a future encoded image of video bitstream 228). The decoder may store one or more reference images in buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the prediction data may further include parameters of the loop filter (e.g., loop filter strength) when the prediction mode indicator of the prediction data 206 indicates that inter-frame prediction is used to encode the current BPU.

[0110] Figure 4 This is a block diagram of an example apparatus 400 for encoding or decoding video, consistent with embodiments of this disclosure. Figure 4 As shown, device 400 may include processor 402. When processor 402 executes the instructions described in this application, device 400 may become a dedicated machine for video encoding or decoding. Processor 402 may be any type of circuit capable of manipulating or processing information. For example, processor 402 may include any combination of any number of central processing units (or “CPU”), graphics processing units (or “GPU”), neural processing units (“NPU”), microcontroller units (“MCU”), optical processors, programmable logic controllers, microcontrollers, microprocessors, digital signal processors, intellectual property (IP) cores, programmable logic arrays (PLAs), programmable array logic (PALs), general-purpose array logic (GALs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), system-on-a-chip (SoCs), application-specific integrated circuits (ASICs), etc. In some embodiments, processor 402 may also be a group of processors grouped into individual logic components. For example, such as Figure 4As shown, processor 402 may include multiple processors, including processor 402a, processor 402b and processor 402n.

[0111] The device 400 may also include a memory 404 configured to store data (e.g., a set of instructions, computer code, intermediate data, etc.). For example, such as Figure 4 As shown, the stored data may include program instructions (e.g., program instructions for implementing stages in processes 200A, 200B, 300A, or 300B) and data for processing (e.g., video sequence 202, video bitstream 228, or video stream 304). Processor 402 can access the program instructions and data for processing (e.g., via bus 410) and execute the program instructions to perform operations or manipulations on the data for processing. Memory 404 may include a high-speed random access memory device or a non-volatile memory device. In some embodiments, memory 404 may include any combination of any number of random access memories (RAM), read-only memories (ROM), optical discs, magnetic disks, hard disks, solid-state drives, flash drives, secure digital cards (SD cards), memory sticks, compact flash (CF) cards, etc. Memory 404 may also be a group of memories grouped into single logical components. Figure 4 (Not shown in the image).

[0112] Bus 410 may be a communication device for transmitting data between components within device 400, such as an internal bus (e.g., CPU-memory bus), an external bus (e.g., a universal serial bus port, a peripheral component high-speed interconnect port), or the like.

[0113] For ease of explanation and to avoid ambiguity, the processor 402 and other data processing circuitry are collectively referred to as "data processing circuitry" in this disclosure. The data processing circuitry can be implemented entirely in hardware or as a combination of software, hardware, or firmware. Furthermore, the data processing circuitry can be a single, independent module or can be wholly or partially integrated into any other component of the device 400.

[0114] The device 400 may also include a network interface 406 to provide wired or wireless communication with a network (e.g., the Internet, intranet, local area network, mobile communication network, etc.). In some embodiments, the network interface 406 may include any combination of any number of network interface controllers (NICs), radio frequency (RF) modules, repeaters, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, near field communication (“NFC”) adapters, cellular network chips, etc.

[0115] In some embodiments, the device 400 may optionally include a peripheral interface 408 to provide connectivity to one or more peripheral devices. Figure 4 As shown, peripheral devices may include, but are not limited to, cursor control devices (such as a mouse, touchpad, or touchscreen), keyboards, displays (such as cathode ray tube displays, liquid crystal displays, or light-emitting diode displays), video input devices (such as cameras or input interfaces coupled to video files), etc.

[0116] It should be noted that the video codec (e.g., the codec for executing processes 200A, 200B, 300A, or 300B) can be implemented as any combination of any software or hardware modules in device 400. For example, some or all stages of processes 200A, 200B, 300A, or 300B can be implemented as one or more software modules of device 400, such as program instructions that can be loaded into memory 404. As another example, some or all stages of processes 200A, 200B, 300A, or 300B can be implemented as one or more hardware modules of device 400, such as dedicated data processing circuitry (e.g., FPGA, ASIC, NPU, etc.).

[0117] This disclosure provides a block partitioning method for use in motion prediction. The method disclosed herein can be executed by an encoder or a decoder.

[0118] Triangle partitioning mode supports inter-frame prediction. Triangle partitioning mode can be applied to blocks of 8×8 or larger and encoded in triangle skip or merge mode. Triangle skip / merge mode is signaled in parallel with regular merge mode, MMVD mode, combined inter-frame and intra-frame prediction (CIIP) mode, or sub-block merge mode.

[0119] When using the triangle partitioning pattern, you can use diagonal partitioning or anti-diagonal partitioning. Figure 5 The block is uniformly divided into two triangular partitions. Each triangular partition in the block uses its own motion for inter-frame prediction. Only unidirectional prediction is allowed per partition. In other words, each partition has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that, as with traditional bidirectional prediction, only two motion-compensated predictions are needed per block. The unidirectional prediction motion for each partition is directly derived from a list of merge candidates constructed for extended merge prediction, and the unidirectional prediction motion is selected from the given merge candidates in the list according to the process described below.

[0120] If the triangular partitioning pattern is used for the current block, it is further signaled with flags indicating the direction of the triangular partitioning (diagonal or anti-diagonal) and two merge indices (one per partition). After predicting each triangular partition, a blending process with adaptive weights is used to adjust the sample values ​​along the diagonal or anti-diagonal edges. This is the prediction signal for the entire block, and the transform and quantization process can be applied to the entire block like other prediction patterns. It is important to note that the Sub-Block Transform (SBT) pattern cannot be applied to blocks encoded using the triangular partitioning pattern. The motion field of a block predicted using the triangular partitioning pattern can be stored in 4×4 sub-blocks.

[0121] The following describes the construction of the one-way prediction candidate list for the triangular partitioning pattern.

[0122] Given a merge candidate index, the unidirectional predicted motion vector is derived from the merge candidate list constructed for the extended merge prediction, such as... Figure 6 As shown. For candidates in the list, their LX (L0 or L1) motion vectors are used as unidirectional predicted motion vectors for the triangle partitioning pattern, where X equals the parity of the merged candidate index value (i.e., X = 0 or 1). These motion vectors in Figure 6 The LX motion vector is marked with "x". When the corresponding LX motion vector does not exist, the L(1-X) motion vector of the same candidate in the expanded and merged prediction candidate list will be used as the unidirectional prediction motion vector of the triangle partitioning mode.

[0123] The following describes the blending along the triangular partition edges.

[0124] After predicting each triangular partition using its own motion, the two predicted signals are blended to derive samples around either the diagonal or anti-diagonal edges. The following weights are used in the blending process: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} represent luminance, and {6 / 8, 4 / 8, 2 / 8} represent chrominance, as shown in the figure. Figure 7 As shown in the image.

[0125] The weights of each luminance and chromaticity sample within a block predicted using the triangular partitioning pattern are calculated using the following equation: -ratio - If the block is split from the first direction (e.g., from the top left to the bottom right), then the split direction splitDir is set to 0. Otherwise, if the block is split from the second direction (e.g., from the top right to the bottom left), then splitDir is set to 1.

[0126] in A weighted graph representing brightness samples. A weighted graph representing chroma samples. The location representing the luminance / chrominance sample. This represents the block width and height in the brightness sample. This indicates the block width and height in the chroma sample.

[0127] Next, the storage of the sports field under the triangle partitioning pattern will be described.

[0128] Motion vectors for blocks encoded using a triangular partitioning pattern are stored in 4×4 sub-blocks. Based on the position of each 4×4 sub-block, either unidirectional or bidirectional prediction motion vectors are stored. Mv1 and Mv2 are represented as follows: Figure 5 The unidirectional predicted motion vectors for partitions 1 and 2. If a 4×4 sub-block is located within a unidirectional predicted region, then Mv1 or Mv2 is stored for that 4×4 sub-block. Otherwise, if the 4×4 sub-block is located within a bidirectional predicted region, then a bidirectional predicted motion vector is stored. The bidirectional predicted motion vector is derived from Mv1 and Mv2 according to the following procedure.

[0129] 1. If Mv1 and Mv2 come from different lists of reference images (one from L0 and the other from L1), then Mv1 and Mv2 are simply combined to form a bidirectional predicted motion vector.

[0130] 2. Otherwise, if Mv1 and Mv2 come from the same list, the bidirectional predicted motion is not stored, and only the unidirectional predicted motion Mv2 is stored.

[0131] It should be noted that when weighting all samples within a 4×4 sub-block, the 4×4 sub-block is considered to be within a bidirectional prediction region. Otherwise, the 4×4 sub-block is considered to be within a unidirectional prediction region. Examples of bidirectional prediction regions (i.e., shaded regions) and unidirectional prediction regions are shown in [the provided text]. Figure 8 As shown in the image.

[0132] An equation can be used to determine whether a 4×4 sub-block is located within a bidirectional prediction region: -ratio - If the block is split from the top left corner to the bottom right corner, then the split direction splitDir is set to 0. Otherwise, if the block is split from the top right corner to the bottom left corner, then splitDir is set to 1.

[0133] -variable -if and , -like Then the 4×4 sub-block is located in the bidirectional prediction region; -Otherwise, the 4×4 sub-block is located in the one-way prediction region; -if and , -like Then the 4×4 sub-block is located in the bidirectional prediction region; -Otherwise, the 4×4 sub-block is located in the one-way prediction region; -if and , -like Then the 4×4 sub-block is located in the bidirectional prediction region; -Otherwise, the 4×4 sub-block is located in the one-way prediction region; -if and , -like Then the 4×4 sub-block is located in the bidirectional prediction region; -Otherwise, the 4×4 sub-block is located in the one-way prediction region; in This indicates the position of the brightness sample at the top left corner of the 4×4 sub-block. This represents the block width and block height in the brightness sample.

[0134] The following describes an exemplary syntax structure for the triangle partitioning pattern.

[0135] exist Figure 9-10 The diagrams show exemplary syntax structures for merging patterns. The CIIP flag shown in the diagram indicates whether the triangular partitioning pattern is used for block prediction.

[0136] The geometric partitioning method conforming to this disclosure is described below.

[0137] In the disclosed embodiments, geometric partitioning patterns can also be used to encode video content. In geometric partitioning patterns, a block is divided into two partitions, which can be rectangular or non-rectangular in shape, such as... Figure 11 As shown. Then, these two partitions perform inter-frame prediction using their own motion vectors. Unidirectional motion prediction uses a reference. Figure 6 The same process described above is derived. After predicting each geometric partition, similar to the process used in the triangular partitioning mode, a hybrid processing method with adaptive weights is used to adjust the sample values ​​along the partition edges. This is the predicted signal for the entire block, which can be transformed and quantized like other prediction modes. It's important to note that the SBT mode can be applied to blocks encoded using the geometric partitioning mode. Finally, the motion field of the block predicted using the geometric partitioning mode can be stored in 4×4 sub-blocks. The advantage of the geometric partitioning mode is that it provides a more flexible partitioning method for motion compensation.

[0138] The geometric partitioning mode applies only to blocks whose width and height are both greater than or equal to 8, whose maximum (width, height) / min (width, height) ratio is less than or equal to 4, and which are encoded using geometric skip or merge modes. The geometric partitioning mode signals each block in parallel with the regular merge mode, MMVD mode, CIIP mode, sub-block merge mode, or triangle partitioning mode. When the geometric partitioning mode is used for the current block, the geometric partitioning mode index indicating which of the 140 partitioning methods (32 quantization angles + 5 quantization distances) is used to split the current block, along with two merge indices, are further signaled. It should be noted that, depending on the settings, the total number of geometric partitioning sub-modes can be one of 140 (16 quantization angles + 9 quantization distances), 108 (16 quantization angles + 7 quantization distances), 80 (12 quantization angles + 7 quantization distances), or 64 (10 quantization angles + 7 quantization distances).

[0139] The blending process along the edges of geometric partitions will be discussed below. After predicting each geometric partition using its own motion, a blending procedure is applied to the two predicted signals to derive samples around the partition edges. In some embodiments, the weight of each luminance sample is calculated using the following equation.

[0140] in This represents the position of each brightness sample. and There are two lookup tables, such as Figure 12-1 Tables 12 and 13A-13B in section 3 are shown respectively.

[0141] The parameters displacement X, displacement Y, and rho are calculated as follows: in ) represents the block width and height in the brightness sample. Set to 32, and Derived from Table 14 in Figure 14. And stepDis is derived from Table 15A in Figure 15A.

[0142] In some embodiments, parameters , and The following calculation can also be performed.

[0143] in (CuW,CuH) These are the block width and height in the brightness sample. NumAngles Set to 32, stepDis Table 15B is from Figure 15B. When the total number of geometric sub-patterns is set to 140, 108, 80, and 64 respectively, and Derive from Tables 15C, 15D, 15E, and 15F of Figures 15C-15F.

[0144] The weights of the chroma samples are derived from the weighted subsample of the top-left luminance sample of each 2×2 luminance subblock, for example, for the YUV 4:2:0 video format.

[0145] The following describes the storage of the sports field under the geometric partitioning pattern.

[0146] Motion vectors for blocks encoded using a geometric partitioning pattern are stored in 4×4 sub-blocks. For each 4×4 sub-block, either a unidirectional or bidirectional predicted motion vector is stored. The derivation process for bidirectional predicted motion is the same as described above. Two methods are proposed to determine whether to store unidirectional or bidirectional predicted motion vectors for each 4×4 sub-block.

[0147] In the first method, for a 4×4 sub-block, the sample weights of its four corners are summed. If the sum is less than threshold 2 and greater than threshold 1, a bidirectional predicted motion vector is stored for that 4×4 sub-block. Otherwise, a unidirectional predicted motion vector is stored. Threshold 1 and threshold 2 are set to... and .

[0148] In the second method, the following equation is used to determine which motion vector is stored in the 4×4 sub-block based on its position.

[0149] in This indicates the position of each 4×4 sub-block. (Variable) , , and Same as the variables mentioned above. When When the value is 2, store the bidirectional predicted motion vector for this 4×4 sub-block. Otherwise, store the unidirectional predicted motion vector for this 4×4 sub-block.

[0150] The three exemplary syntactic structures of the geometric partitioning pattern are respectively in Figure 16-17B As shown in the image.

[0151] In some embodiments, subblock transformation can be used. In subblock transformation, a residual block is divided into two residual subblocks, such as... Figure 18 As shown. Only one of the two residual sub-blocks is encoded. For the other residual sub-block, the residual is set to 0.

[0152] For inter-prediction blocks with residuals, a CU-level flag is signaled to indicate whether sub-block transform is applied. When using sub-block transform mode, parameters are signaled to indicate whether the residual block is symmetrical or asymmetrical, and whether it is divided into two sub-blocks horizontally or vertically.

[0153] Triangular partitioning and geometric partitioning are two partitioning methods to improve the efficiency of motion compensation coding. Triangular partitioning can be considered a subset of geometric partitioning. However, in current implementations, the syntax structure, mixing process, and motion field storage of geometric partitioning differ from those of triangular partitioning. For example, the processing of these two modes is different.

[0154] 1. For blocks encoded in merge mode, two flags (triangle partitioning mode flag and geometric partitioning mode flag) are sent by signaling. Furthermore, triangle partitioning mode can be applied to blocks with a width or height equal to 4. However, geometric partitioning mode cannot be applied to these blocks.

[0155] 2. The calculation formula for the luminance sample weights encoded in the triangular partitioning mode differs from that in the geometric partitioning mode. Furthermore, the weights of chrominance samples encoded in the triangular partitioning mode are calculated separately, while the weights of chrominance samples encoded in the geometric partitioning mode are obtained by secondary sampling from the corresponding luminance samples.

[0156] 3. Motion vectors for blocks encoded using either triangular or geometric partitioning are stored in 4×4 sub-blocks. Furthermore, based on the position of each 4×4 sub-block, either unidirectional or bidirectional predicted motion vectors are stored. However, the process for selecting unidirectional or bidirectional predicted motion vectors for the 4×4 sub-blocks differs between triangular and geometric partitioning modes.

[0157] 4. The SBT pattern is not allowed in the case of triangle partitioning pattern, but it can be applied to geometric partitioning pattern.

[0158] Because the triangular partitioning pattern can be seen as a subset of the geometric partitioning pattern, it can unify all the processes used in the triangular partitioning and geometric partitioning patterns.

[0159] To unify the syntax of triangular and geometric partitioning modes, a single flag can be used to indicate whether a block is divided into two partitions. A flag signal is emitted when the block size is greater than or equal to 64 luminance samples. When the flag is true, a further signal is sent to the partitioning mode index to indicate which partitioning method to use to divide the block.

[0160] In one embodiment, when a block is not encoded using sub-block merging, regular merging, and MMVD mode, a flag is sent (e.g., Figures 19-20 CIIP flag signal.

[0161] In another embodiment, a signal is used to indicate the beginning of the merge syntax structure (e.g., Figure 21 (The triangle / geometric symbol in the image).

[0162] When a block is split into two partitions, the partitioning pattern index is further signaled to indicate which partitioning method to use.

[0163] In one embodiment, two triangular partitioning patterns (e.g., partitioning blocks from top left to bottom right or from top right to bottom left) are placed at the beginning of the partitioning pattern list, followed by the geometric partitioning pattern. In other words, the triangular partitioning pattern is used to partition blocks when the partitioning pattern index is 0 or 1. Otherwise, the geometric partitioning pattern is used.

[0164] In another embodiment, the triangular partitioning pattern is considered one of the geometric partitioning patterns, as shown in Table 22A of Figure 22A. For example, when the partitioning pattern index is 19, the block is partitioned from the top left corner to the bottom right corner. As another example, partitioning pattern index 58 indicates a block partitioned from the top right corner to the bottom left corner.

[0165] In yet another embodiment, the triangular partitioning pattern is also considered as one of the geometric partitioning patterns, such as... Figure 22BAs shown in Table 22, when the partition pattern index is 10, the block is split from the top left to the bottom right. Furthermore, a partition pattern index of 24 indicates that the block is split from the top right to the bottom left.

[0166] Note that the number of block partitioning patterns can depend on block size and / or block shape.

[0167] In one embodiment, if the block's width or height is equal to 4, or the ratio of max(width, height) / min(width, height) is greater than 4, then the block is allowed only two partitioning patterns. Otherwise, 142 partitioning patterns are allowed.

[0168] In another embodiment, the number of partitioning patterns is reduced when the block size exceeds a threshold. For example, when a block size exceeds 1024 luminance samples, only 24 quantization angles and 4 quantization distances are allowed.

[0169] In another embodiment, the number of partitioning patterns is reduced when the block shape is narrow and tall or wide and flat. For example, when the ratio of max(width, height) / min(width, height) is greater than 2, only 24 quantization angles and 4 quantization distances are allowed. Furthermore, only angles that partition the block along the larger dimension are allowed, such as... Figure 23 As shown. For wide, flat blocks, the three angles shown by the dashed lines are not allowed; only the angle shown by the solid lines is allowed.

[0170] In another embodiment, when the block size is greater than a threshold and the block shape is narrow and tall or wide and flat, the number of partitioning patterns can be reduced. For example, when a block size is greater than 1024 luminance samples and the ratio of max(width, height) / min(width, height) is greater than 2, only 24 quantization angles and 4 quantization distances are allowed. This can be further combined with... Figure 23 The restrictions shown are combined.

[0171] The lookup tables for angle indexes and distance indexes can be changed.

[0172] In one embodiment, the partitioning pattern index is used with a first-order distance index instead of a first-order angle index, as shown in Table 24 of Figure 24.

[0173] In another embodiment, the partitioning pattern index order is related to the occurrence rate of the partitioning method. Partitioning methods with higher occurrence rates are placed at the beginning of the lookup table. An example is shown in Table 25 of Figure 25. Angular indices 0, 4, 8, and 12, with a distance of 0, have a higher probability of being used for partitioning blocks.

[0174] As mentioned earlier, the triangular partitioning pattern can be applied to blocks with a size greater than or equal to 64 luminance samples. However, the geometric partitioning pattern can be applied to blocks with both width and height greater than or equal to 8 and a max(width, height) / min(width, height) ratio less than or equal to 4. This unifies the restrictions on block size and shape imposed by the triangular and geometric partitioning patterns.

[0175] In one embodiment, both the triangle partitioning mode and the geometric partitioning mode can be applied to blocks whose width and height are both greater than or equal to 8 and whose max(width, height) / min(width, height) is less than or equal to 4.

[0176] In another embodiment, both the triangular partitioning pattern and the geometric partitioning pattern can be applied to blocks with a size greater than or equal to 64 luminance samples.

[0177] In another embodiment, both the triangular partitioning mode and the geometric partitioning mode can be applied to blocks with a size greater than or equal to 64 luminance samples and max(width, height) / min(width, height) less than or equal to 4.

[0178] This invention also provides a method for calculating the weights of a unified triangle partitioning pattern and a geometric partitioning pattern.

[0179] In one embodiment, the weight calculation process for the brightness samples divided by triangles is replaced with the process used in the geometric division mode (described in the hybrid processing above), with the following two modifications: 1. The value in Dis[] is replaced with Figure 26 The values ​​in Table 26.

[0180] 2. and .

[0181] Additionally, for blocks encoded using a triangular partitioning pattern, if the block is divided from the top left corner to the bottom right corner, then... angleIdx and distanceIdx Set them to 4 and 0 respectively. Otherwise (e.g., the block is split from the top right corner to the bottom left corner). angleIdx and distanceIdx Set them to 12 and 0 respectively.

[0182] In another embodiment, for both the triangular partitioning mode and the geometric partitioning mode, the weights of the chroma samples are second-sampled from the weights of the top-left luminance samples of each 2×2 luminance sub-block.

[0183] In yet another embodiment, for both the triangular partitioning pattern and the geometric partitioning pattern, the weights of the chroma samples are calculated using the same procedure as that used for the luminance samples encoded in the geometric partitioning pattern.

[0184] This disclosure also unifies the sports field storage process used in the triangular partitioning mode and the geometric partitioning mode.

[0185] In one embodiment, motion field storage using a triangular partitioning pattern is replaced with motion field storage using a geometric partitioning pattern. Specifically, the weights of the four brightness samples located at the four corners of a 4×4 sub-block are summed. If the sum is less than threshold 2 and greater than threshold 1, a bidirectional predicted motion vector can be stored for this 4×4 sub-block. Otherwise, a unidirectional predicted motion vector is stored. Threshold 1 and threshold 2 are respectively set to... and .

[0186] In another embodiment, for blocks encoded using a triangular partitioning pattern or a geometric partitioning pattern, the weight of each luminance sample is examined. If the weight of a luminance sample is not equal to 0 or 8, the luminance sample is considered a weighted sample. If all luminance samples in a 4×4 sub-block are weighted, bidirectional predicted motion is stored for the 4×4 sub-block. Otherwise, unidirectional predicted motion is stored.

[0187] To coordinate the interaction between SBT and geometric partitioning patterns, and between SBT and triangular partitioning patterns, SBT for geometric partitioning patterns can be disabled in this disclosure. The combination of SBT and geometric partitioning patterns may result in two intersecting boundaries within a single block, which could lead to subjective quality issues.

[0188] In one embodiment, when geometric partitioning is used, cu_sbt_flag is not signaled, such as Figure 27 Table 27 shows the relevant syntax. Figure 27 It is highlighted in italics and gray.

[0189] In another embodiment, some SBT partitioning patterns are disabled based on the GEO partitioning pattern. When an SBT partitioning edge intersects with a GEO partitioning edge, this SBT partitioning method is not allowed. Otherwise, the SBT partitioning pattern is permitted. Figure 28 Examples of SBT and GEO partitioning patterns are shown. Furthermore, angle and distance indices can be used to determine if there is an intersection between GEO and SBT partitioning edges. In one example, a horizontal SBT partition cannot be applied to the current block when the angle index of the current block is 0 (i.e., a vertical partitioning edge). Additionally, the SBT syntax can be modified as follows: Figure 29 Table 29 is highlighted with italics and gray to emphasize its variations.

[0190] Geometric partitioning divides a block into two geometrically shaped partitions, each with its own motion vector for motion compensation. Geometric partitioning improves the prediction accuracy of inter-frame prediction. However, it can become complex in several ways.

[0191] Firstly, the total number of geometrically partitioned subpatterns is enormous. Therefore, it is impossible to store all the masks used for mixing weights and motion field storage in a practical implementation. With 140 subpatterns, the total number of bits required to store the masks is: - For mixed weights: - For sports field storage: Secondly, if the masks are not stored and instead dynamically computed, the computational complexity increases. Calculating the equations for the masks used to mix weights and store motion fields is complex. More specifically, the number of multiplication (×), shift (<<), addition (+), and comparison operations is enormous. Assuming a block of size W×H, the number of operations per block is: - Multiplication: 5 + 2 × W × H + 2 × (W × H / 16); - Shift: 4 + 3 × width × height + 2 × (width × height / 16); - Bonus: 8 + 6 × width × height + 5 × (width × height / 16); - Comparison: 4 + 2 × W × H + 2 × (W × H / 16).

[0192] See the table below for details. In addition, memory is needed to store four pre-computed tables: Dis[], GeoFilter[], stepDis[], and a lookup table for angleIdx and distanceIdx. The size of each table is as follows: Thirdly, in the current geometric partitioning pattern design, the combination of 45° / 135° and distanceIdx 0 is always prohibited because it assumes that the triangle partitioning pattern in VVC supports these partitioning options. However, for non-square blocks, the partitioning angles in the triangle partitioning pattern are not 45° or 135°, as shown in the table below. Therefore, excluding these two partitioning angles for non-square blocks is meaningless.

[0193] Fourthly, because the geometric partitioning patterns do not support certain combinations of angles and distances, such as using distanceIdx 0 for horizontal partitioning or using distanceIdx 0 for vertical partitioning (this is to avoid redundancy in binary tree partitioning), a lookup table is used to derive the angle and distance for each geometric partitioning subpattern. If the restrictions on combinations of angles and distances are removed, a lookup table may not be needed.

[0194] Fifthly, the mixed processes, motion field storage, and syntax structures used in the triangle partitioning mode and the geometric partitioning mode are inconsistent, meaning that these two modes require two different logics in both software and hardware implementations. Furthermore, the total number of bits required to store the triangle pattern mask is: - For mixed weights: - For sports field storage: Several solutions have been proposed to address the above problems.

[0195] The first solution involves simplifying the geometric partitioning pattern.

[0196] To avoid real-time computation of masks for mixing weights and storing motion fields, it is proposed to derive the mask for each block from several pre-computed masks of size 256×256 or 64×64. The proposed method can reduce the memory used to store the masks.

[0197] In one proposed cropping method, the first and second sets of masks are predefined. The first set of masks... The brackets [] may contain multiple masks, each 256×256 in size, and the blending weights for each block are derived using these masks. (Second set of masks) It may contain multiple masks, each 64×64 in size, used to derive the motion field storage for each block. The number of masks in the first and second groups depends on the number of geometric sub-patterns. For blocks of different sizes, their masks are cropped from one of the masks in the first and second groups.

[0198] In one embodiment, the predefined masks in the first and second groups can be calculated using references to Figures 14 and 15A-15F and equations relating to the sports field storage description. The number of masks in both the first and second groups is N, where N is set to the number of angles supported by the geometric partitioning pattern. The nth mask in the first and second groups, with index n, represents the mask for angle n, where n ranges from 0 to N-1.

[0199] In one example, when the number of geometric subpatterns is set to 140, the variable N is set to 16, which is 16 angles plus 9 distances. In another example, when the number of geometric subpatterns is set to 108, the variable N is set to 16, which is 16 angles plus 7 distances. In other examples, when the number of geometric subpatterns is set to 80 (12 angles plus 7 distances) and 64 (10 angles plus 7 distances) respectively, the variable N is set to 12 and 10.

[0200] For a block of size W×H with a geometric partition index set to K, the mask used to mix the brightness sample weights is derived as follows.

[0201] - Use the geometric partition index K to obtain the variables angleIdx A and distanceIdx D from the lookup table. Examples of lookup tables are shown in Table 15C of Figure 15C and Table 15D of Figure 15D. Figure 22B The figures are shown in Table 22B, Table 30 of Figure 30, and Table 31 of Figure 31.

[0202] The variables offsetX and offsetY are calculated as follows.

[0203] - - The chroma sample weights are derived from the luminance sample weights. That is, the weight of the luminance sample at the top left corner of each corresponding 2x2 luminance sub-block is used as the chroma sample weight for the YUV 4:2:0 video format.

[0204] In addition, the mask used for sports field storage is exported as follows.

[0205] - The variables offsetXmotion and offsetYmotion are calculated as follows: - - The number of bits required to store a predefined mask is as follows: For mixed weights: For sports field storage: In addition, the mask can be calculated instantly using the following simplified equation: in This indicates the position of each brightness sample. and These are the two lookup tables shown in Table 12 and Table 13 respectively. yes Figure 32 The lookup table is shown in Table 32.

[0206] parameter and The calculation is as follows: in The value is set to 32. When the total number of geometric sub-patterns is set to 140, 108, 80, and 64 respectively, Derived from Tables 15C, 15D, 15E, and 15F. angleIdx can also be derived from the lookup tables shown in Tables 22B, 30, and 31.

[0207] Assuming a block of size W×H, the number of operations per block is: - Multiplication: 4 + 2 × W × H + 2 × (W × H / 16); - Shift: 9 + 3 × W × H + 2 × (W × H / 16); - Addition: 7 + 8 × W × H + 6 × (W × H / 16); - Comparison: 6 + 2 × W × H + 2 × (W × H / 16).

[0208] Figure 33 Table 33 lists the details of the number of each operation for each block.

[0209] The memory needs to store five pre-computed tables: Dis[], GeoFilter[], Rho[], Rhosubblk[], and lookup tables for angleIdx and distanceIdx. The lookup table for Rhosubblk[] is located in... Figure 34 The following is an explanation. The size of each table is as follows: The computational complexity of the proposed method is similar to that of the original geometric design. More specifically, compared to the original geometric design.

[0210] - For a W×H block, the number of multiplication operations increases by 1. - For a W×H block, the number of shift operations increases by 5. - For a W×H block, the number of comparison operations increases by 2. - For a W×H block, the number of addition operations increases by 2×W×H + (W×H / 16) -1 - Memory usage increased by 17 bits. The equation used to calculate the mask can be further simplified as follows: The variables angleIdx A and distanceIdx D are obtained from a lookup table using the geometric partition index K. Examples of lookup tables are shown in Tables 15C, 15D, 22B, 30, and 31.

[0211] - When deriving angleIdx and distanceIdx using Tables 15C, 15D, and 22B, the variable N (the number of masks in the first and second groups) is set to 16. On the other hand, when deriving angleIdx and distanceIdx using Tables 30 and 31 respectively, the variable N is set to 12 and 10. The variables offsetX and offsetY are calculated as follows: - - Variables and The calculation is as follows: , in Set it to 32.

[0212] The weights of the brightness samples located at (x, y) are calculated as follows: weightIdx = ( ( ( x + offsetX ) << 1 ) + 1 ) disLut[displacementX] + ( ( ( y + offsetY ) << 1 ) + 1 ) ) disLut[ displacementY ] partFlip = ( angleIdx >= 13&&angleIdx <= 27 ) ? 0 : 1 weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx , in The lookup table is as follows: In another embodiment, the predefined masks in the first and second groups can be calculated using references to Figures 14 and 15A-15F and equations relating to the sports field storage. The number of masks in both the first and second groups is N. reduced ,in N represents the number of angles supported by the geometric sub-pattern. In one example, when the number of geometric sub-patterns is set to 140, the variable N... reduced Set to 9. That is, 16 angles plus 9 distances. In another example, when the number of geometric sub-patterns is set to 108 (16 angles plus 7 distances), 80 (12 angles plus 7 distances), and 64 (10 angles plus 7 distances), the variable N... reduced Set to 9, 7, and 6.

[0213] For angles between 0 and Nreduced–1, their masks are directly cropped from the first and second sets of masks. On the other hand, the masks for angles between Nreduced and N-1 are cropped from the first and second sets of masks and then flipped horizontally. According to this embodiment of the application, Figure 35A An example of a mask at an angle of 135° is shown. Figure 35B An example of a mask with a 45° angle is shown.

[0214] For a block of size W×H with a geometric partition index set to K, the mask used to mix the brightness sample weights is derived as follows.

[0215] - The variables angleIdx A and distanceIdx D are obtained from a lookup table using the geometric partition index K. Examples of lookup tables are shown in Tables 15C, 15D, 22B, 30, and 31.

[0216] The variables offsetX and offsetY can be calculated as follows.

[0217] - - The chroma sample weights are derived from the luminance sample weights by secondary sampling. That is, the weight of the top-left luminance sample of each corresponding 2×2 luminance sub-block is used as the weight of the chroma sample in the YUV 4:2:0 video format.

[0218] On the other hand, the mask for storing the sports field is derived as follows: The variables offsetXmotion and offsetYmotion are calculated as follows: The number of bits required to store a predefined mask is as follows: For mixed weights: (256×256)×9×4=2,359,296 bits=294,912 bytes ≅ 295 KB For sports field storage: (64×64)×9×2=131,072 bits=16,384 bytes≅16 KB In the third embodiment, the predefined masks in the first and second groups can be calculated using the above equation. The mask numbers for both the first and second groups are Nreduced, where... N is the number of angles supported by the geometric partitioning pattern. For angles between 0 and Nreduced – 1, their masks are directly cropped from the masks of the first and second groups. On the other hand, the masks for angles between Nreduced and N-1 are cropped from the masks of the first and second groups and flipped in the vertical direction. According to an embodiment of this application, Figure 36 shows an example of a mask for an angle of 135°. Figure 36B An example of a mask with a 45° angle is shown.

[0219] For a block of size W×H with a geometric partition index set to K, the mask used for luminance sample mixing weights is as follows: The variables angleIdx A and distanceIdx D are obtained from the lookup table using the geometric partition index K. Examples of lookup tables are shown in Tables 15C, 15D, 22B, 30, and 31.

[0220] The variables offsetX and offsetY can be calculated as follows: - The chroma sample weights are derived from the luminance sample weights. That is, the weight of the luminance sample in the upper left corner of each corresponding 2×2 luminance sub-block is used as the weight of the chroma sample in the YUV 4:2:0 video format.

[0221] On the other hand, the mask for storing the sports field is derived as follows: The variables offsetXmotion and offsetYmotion can be calculated as follows: The number of bits required to store a predefined mask is as follows: For mixed weights: (256×256)×9×4=2,359,296 bits=294,912 bytes ≅ 295 KB For sports field storage: (64×64)×9×2=131,072 bits=16,384 bytes≅16 KB Note that the methods used for offset derivation and chromaticity weight derivation can be modified in the foregoing embodiments.

[0222] Different methods for deriving the offset are shown below. Differences compared to the embodiments described above are highlighted in italics and bold.

[0223] The equations for offset derivation can be modified to ensure that the offset is not equal to 0 when distanceIdx is not 0. In one example, the variables offsetX and offsetY are calculated as follows.

[0224] Offset derivation can be based on the number of distances supported in the geometric partitioning pattern. In one example, when 7 distances are supported, the variables offsetX and offsetY are calculated as follows (the same method as in the first embodiment).

[0225] In another example, when 9 distances are supported, the variables offsetX and offsetY are calculated as follows: Offset derivation can be based on angle division. In one example, for angles from 135° to 225° and from 315° to 45°, an offset is added in the vertical direction. Otherwise, for angles from 45° to 135° and from 225° to 315°, an offset is added in the horizontal direction. The variables offsetX and offsetY are calculated as follows.

[0226] The weights of the chroma samples can be directly derived from the first set of masks. The mask for chroma sample mixing weights is exported as follows for a block of size W×H with a geometric partition index set to K.

[0227] - The chroma block size is W'×H' - The variables angleIdx A and distanceIdx D are obtained from the lookup table using the geometric partition index K. Examples of lookup tables are shown in Tables 15C, 15D, 22B, 30, and 31.

[0228] The variables offsetXchroma and offsetYchroma are calculated as follows.

[0229] - in This indicates the position of each chromaticity sample.

[0230] The size of the first predefined mask may not be 256×256. Its size can depend on the maximum block size and the maximum shift offset. Assume the maximum block size is S and the supported distances are N. d The shift offset for each distance is defined as follows: Then, the width and height of the predefined mask can be calculated as follows: In one example, variables S and N d 0 and 128 are set to 128, 9, and 4 respectively. The predefined mask size is set to 192×192. In another example, variables S and N... d 0 is set to 128, 7, and 3 respectively. The predefined mask size is set to 224×224.

[0231] In some embodiments, similar to the cropping method, the first and second sets of masks are predefined. First set of masks It may contain multiple masks of size 256×256, which are used to derive the blending weights for each block. The second set of masks... This may contain several 64×64 masks, which are used to derive masks for motion field storage for each block. For square blocks, the mask is cropped from one of the first and second groups of masks, similar to the cropping method. For non-square blocks, the mask is cropped from one of the first and second groups of masks, followed by an upsampling process.

[0232] In one embodiment, the predefined masks in the first and second groups can be calculated using the equations described above. Both the first and second groups contain N masks, where N is set to the number of angles supported by the geometric partitioning pattern. The nth mask with index n in both the first and second groups represents the mask for angle n, where n ranges from 0 to N-1. For a block of size W×H with a geometric partitioning index set to K, the mask used for brightness sample mixing weights is derived as follows.

[0233] - The variables angleIdx A and distanceIdx D are obtained from a lookup table using the geometric partition index K. Examples of lookup tables are shown in Tables 15C, 15D, 22B, 30, and 31.

[0234] - Set the variable minSize to min(W, H) - The variables ratioWH and ratioHW are set to log2(max(W / H, 1)) and log2(max(H / W, 1)) respectively. - Variable offsets X and Y can be calculated as follows: The chroma sample weights are subsampled from the luminance sample weights. That is, the weight of the top-left luminance sample of each corresponding 2×2 luminance sub-block is used as the weight of the chroma sample in the YUV 4:2:0 video format.

[0235] On the other hand, the mask for storing the sports field is derived as follows: - Set the variable minSubblk to min(W, H)>>2 - The variables offsetXmotion and offsetYmotion can be calculated as follows: In another embodiment, the predefined masks in the first and second groups can be calculated using the equation described above. The number of masks in both the first and second groups is N. reduced ,in N represents the number of angles supported by the geometric partitioning mode. For a block of size W×H with a geometric partitioning index set to K, the mask for luminance sample mixing weights is derived as follows.

[0236] - The variables angleIdx A and distanceIdx D are obtained from a lookup table using the geometric partition index K. Examples of lookup tables are shown in Tables 15C, 15D, 22B, 30, and 31.

[0237] - Set the variable minSize to min(W, H) - The variables ratioWH and ratioHW are set to log2(max(W / H, 1)) and log2(max(H / W, 1)) respectively. The variables offsetX and offsetY can be calculated as follows: The chroma sample weighting is a secondary sampling of the luminance sample weighting. That is, the weight of the top-left luminance sample of each corresponding 2×2 luminance sub-block is used as the weight of the chroma sample in the YUV 4:2:0 video format.

[0238] On the other hand, the mask for storing the sports field is derived as follows: - Set the variable minSubblk to min(W, H)>>2 - The variables offsetXmotion and offsetYmotion can be calculated as follows: - In the third embodiment, the predefined masks in the first and second groups can be calculated using the above equation. The number of masks in both the first and second groups is N. reduced ,in N is the number of angles supported by the geometric partitioning mode. For a block of size W×H with the geometric partition index set to K, the mask used for mixing brightness sample weights is derived as follows.

[0239] - The variables angleIdx A and distanceIdx D are obtained from the lookup table using the geometric partition index K. Examples of lookup tables are shown in Tables 15C, 15D, 22B, 30, and 31.

[0240] - Set the variable minSize to min(W, H) - The variables ratioWH and ratioHW are set to log2(max(W / H, 1)) and log2(max(H / W, 1)) respectively. The variables offsetX and offsetY can be calculated as follows: The chroma sample weighting is a secondary sampling of the luminance sample weighting. That is, the weight of the top-left luminance sample of each corresponding 2×2 luminance sub-block is used as the weight of the chroma sample in the YUV 4:2:0 video format.

[0241] On the other hand, the mask for storing the sports field is derived as follows: - Set the variable minSubblk to min(W, H)>>2 - The variables offsetXmotion and offsetYmotion can be calculated as follows: - It should be noted that the different methods described above for deriving offset and chromaticity weights can also be applied to this paper.

[0242] In the original design of the geometric partitioning pattern, combinations of dividing blocks across the block center at 135° or 45° were always excluded. The main purpose was to remove redundant partitioning options from the triangular partitioning pattern. However, for non-square blocks encoded using the triangular partitioning pattern, the partition angle is neither 135° nor 45°. Therefore, both angles can be selectively excluded based on the block shape.

[0243] In some embodiments of this disclosure, the two excluded angles vary based on the block shape. For square blocks, 135° and 45° are excluded, consistent with the original geometric partitioning design. For other block shapes, the excluded angles are listed below. Figure 37 In Table 37. For example, for a block of size 8×16 (i.e., an aspect ratio of 1:2), angles of 112.5° and 67.5° are excluded, i.e., angleIdx 10 and angleIdx 6. Then, the lookup table for the geometric partition index is modified to Table 38 in Figure 38. The parts of Table 38 related to the excluded angles are highlighted in gray in Figure 38.

[0244] Note that this embodiment can be combined with other embodiments in this disclosure. For example, a mask using a triangulation pattern can be computed with mixed weights of excluded angles and motion field storage. For other angles, a clipping method is used to derive the mask.

[0245] As previously mentioned, the hybrid processes, motion field storage, and syntax structures used in triangle partitioning and geometric patterns are not uniform. This disclosure proposes to unify all processes.

[0246] In one embodiment, the above-described clipping method is used in the process of mixing triangles and geometric partitioning patterns and storing motion fields. Furthermore, the angle and distance lookup tables for each geometric partitioning sub-pattern are removed. The first and second sets of masks are predefined and can be calculated separately using the above equations. The number of masks in both the first and second sets is predefined and can be calculated separately using the above equations. The number of masks in both the first and second sets is N. reduced ,in Let N be the number of angles supported by the geometric partitioning pattern. D This represents the number of distances supported under the geometric partitioning pattern. Therefore, the total number of geometric partitioning sub-patterns is N×N. D In one example, N and N D Set them to 8 and 7 respectively. In another example, N and N D They are set to 12 and 7 respectively. In other examples, N and N D Set them to even and odd numbers respectively.

[0247] For a block of size W×H with a geometric partition index set to K, the mask for luminance sample mixing weights is derived as follows.

[0248] -Variable N halfD Set to N D >>1.

[0249] - Set the variables angleIdx A and distanceIdx D to K % N and K / N, respectively.

[0250] - The variables offsetX and offsetY are calculated as follows: - The chroma sample weights are derived from the weights of the luminance samples. Specifically, the weight of the top-left luminance sample of each corresponding 2×2 luminance sub-block is used as the weight of the chroma sample in the YUV 4:2:0 video format.

[0251] On the other hand, the mask for storing the sports field is derived as follows: - The variables offsetXmotion and offsetYmotion are calculated as follows: The number of bits required to store a predefined mask is In another embodiment, the above-described clipping method is used in the process of mixing triangle and geometric partitioning patterns and storing motion fields. Furthermore, the angle and distance lookup tables for each geometric partitioning sub-pattern are removed. The first and second sets of masks are predefined and can be calculated separately using the above equations. The number of masks in both the first and second sets is N. reduced ,in ), where N is the number of angles supported by the geometric partitioning pattern. Let N D This represents the number of distances supported under the geometric partitioning pattern. Therefore, the total number of geometric partitioning sub-patterns is N×N. D .

[0252] For a block of size W×H with a geometric partition index set to K, the mask used for luminance sample mixing weights is as follows: variable N halfD Set to N D >>1.

[0253] The variables angleIdx A and distanceIdx D are set to K % N and K / N, respectively.

[0254] The variables offsetX and offsetY are calculated as follows: - The chroma sample weights are derived from the luminance sample weights. Specifically, the weight of the top-left luminance sample of each corresponding 2×2 luminance sub-block is used as the weight of the chroma sample in the YUV 4:2:0 video format.

[0255] On the other hand, the mask for the sports field storage is exported as follows.

[0256] - The variables offsetXmotion and offsetYmotion are calculated as follows: In the third embodiment, the proposed upsampling method is used in the hybrid and motion field storage process of triangle and geometric partitioning patterns. Furthermore, the angle and distance lookup tables for each geometric partitioning sub-pattern are removed. The first and second sets of masks are predefined and can be calculated using the above equations respectively. The number of masks in both the first and second sets is N. reduced ,in Let N be the number of angles supported by the geometric partitioning pattern. D This represents the number of distances supported under the geometric partitioning pattern. Therefore, the total number of geometric partitioning sub-patterns is N×N. D For a block of size W×H with a geometric partition index set to K, the mask for the brightness sample mixing weights is derived as follows.

[0257] -Variable N halfD Set to ND>>1.

[0258] - Set the variables angleIdx A and distanceIdx D to K % N and K / N, respectively.

[0259] - Set the variable minSize to min(W, H) - The variables ratioWH and ratioHW are set to log2(max(W / H, 1)) and log2(max(H / W, 1)) respectively. - Variable offset is calculated as follows: - - The chroma sample weights are derived from the luminance sample weights. Specifically, the weight of the top-left luminance sample of each corresponding 2×2 luminance sub-block is used as the weight of the chroma sample in the YUV 4:2:0 video format.

[0260] On the other hand, the mask for storing the sports field is derived as follows: - Set the variable minSubblk to min(W, H)>>2 - The variable offsetmotion is calculated as follows: - Figure 39This is a flowchart of an exemplary method 3900 for processing video content according to some embodiments of the present disclosure. In some embodiments, method 3900 may be provided by a codec (e.g., using...) Figure 2A-2B The encoding process in the 200A or 200B encoder or using Figures 3A-3B The decoding process in the decoder (300A or 300B) is performed. For example, the codec may be implemented as one or more software or hardware components of an apparatus (e.g., apparatus 400) for encoding or transcoding a video sequence. In some embodiments, the video sequence may be an uncompressed video sequence (e.g., video sequence 202) or a decoded compressed video sequence (e.g., video stream 304). In some embodiments, the video sequence may be a surveillance video sequence, which may be performed by a monitoring device (e.g., processor 402) associated with the processor of the apparatus (e.g., processor 402). Figure 4 The device (in a video input device) acquires the video. The video sequence may include multiple images. The device may perform method 3900 at the image level. For example, the device may process one image at a time in method 3900. Alternatively, the device may process multiple images at a time in method 3900. Method 3900 may include the following steps.

[0261] In step 3902, multiple blocks can be divided into a first partition and a second partition along the partitioning edge.

[0262] Multiple blocks are sub-blocks of a first block associated with an image. It is understood that an image can be associated with multiple blocks (including the first block), and each block can be divided into multiple sub-blocks. The first block can be associated with a chroma block and a luma block. Therefore, each of the multiple blocks (e.g., sub-blocks) can be associated with a chroma sub-block and a luma sub-block. A partitioning pattern for the multiple blocks (e.g., sub-blocks) can be determined, and the multiple blocks (e.g., sub-blocks) can be partitioned based on the partitioning pattern. Partitioning can provide improvements to inter-frame prediction of the first block. Exemplary partitioning patterns can include a triangular partitioning pattern or a geometric partitioning pattern.

[0263] As described above, the partitioning pattern can be determined based on at least one indicator signal. For example, return to reference. Figure 19 It provides a first indication signal (e.g., a sub-block merge flag), a second indication signal (e.g., a regular merge / MMVD flag), and a third indication signal (e.g., a CIIP flag) to determine the partitioning mode for the first block. Figure 19As shown, it can be determined whether the first block is encoded using a sub-block merging mode based on a first indication signal (e.g., a sub-block merging flag). In response to determining that the first block is not encoded using a sub-block merging mode, it can be determined whether the first block is encoded using a normal mode or a merging mode with motion vector difference (MMVD) based on a second indication signal (e.g., a normal merging / MMVD flag). In response to the determination that the first block is not encoded using a normal mode or MMVD, a CIIP flag can be used to determine whether the first block uses CIIP encoding.

[0264] In some embodiments, before determining whether the first block uses CIIP encoding based on the third indication signal, step 3902 may further include: determining whether the size of the first block satisfies a given condition; generating a third indication signal in response to determining that the size of the first block satisfies the given condition, or determining that the first block is encoded using CIIP mode in response to determining that the size of the first block does not satisfy the given condition. The given condition may include: the width and height of the first block are both greater than or equal to 8; the ratio of the larger of the width and height to the smaller of the width and height is less than or equal to 4.

[0265] Then, in response to the determination that the first block is not encoded using the CIIP mode, it can be determined that the partitioning mode of the first block is either the triangular partitioning mode or the geometric partitioning mode.

[0266] Once the first block is determined to be divided using either a triangular or geometric partitioning pattern, the target partitioning pattern can be determined based on the partitioning pattern index, angle index, or distance index. Then, the partitioning edges corresponding to the target partitioning pattern can be determined.

[0267] Typically, a partitioning pattern (triangular partitioning pattern or geometric partitioning pattern) can be associated with multiple partitioning methods, and the partitioning pattern index can indicate the number of partitioning methods among these methods. The partitioning pattern index can be associated with an angle index and a distance index. The angle index indicates the angle of the partition edge corresponding to a given partitioning method, and the distance index indicates the distance between the partition edge and the center of the first block.

[0268] In some embodiments, a lookup table (e.g., Table 22A of Figure 22A or ...) is used. Figure 22B Table 22B may include multiple partitioning pattern indices, multiple angle indices, and multiple distance indices associated with various partitioning methods. Given a partitioning pattern index (e.g., K), the angle and distance indices associated with the partitioning method can be determined. The target partitioning method can then be determined in the lookup table based on the partitioning pattern index.

[0269] Among the various partitioning methods, the lookup table may include a first partitioning method associated with a first partitioning pattern index and a second partitioning method associated with a second partitioning pattern index, and the first and second partitioning methods are for a triangular partitioning pattern. For example, referring to table 22B in Figure 22B, when partitioning pattern index "10" is associated with partitioning the block from the top left corner to the bottom right corner, and partitioning pattern index "24" is associated with partitioning the block from the top right corner to the bottom left corner.

[0270] In step 3904, inter-frame prediction can be performed on multiple blocks to generate a first prediction signal for a first partition and a second prediction signal for a second partition. A first motion vector can be applied to the first partition to generate the first prediction signal (e.g., the motion vector of the first partition), and a second motion vector can be applied to the second partition to generate the second prediction signal (e.g., the motion vector of the second partition). The first and second prediction signals can be stored in 4×4 sub-blocks.

[0271] In step 3906, the first and second prediction signals of the edge blocks associated with the partitioning edge can be mixed. Since the first block has been partitioned before applying inter-frame prediction to it, the partitions of the first block can be mixed to complete the encoding of that block. In some embodiments, the edge blocks associated with the partitioning edge can be mixed. It is understood that the prediction signal for each partitioned block is the same. To mix the first and second prediction signals of the edge blocks associated with the partitioning edge, a weight for each edge block can be determined.

[0272] In some embodiments, a set of masks can be generated to determine the blending weights. For example, the first set of masks (e.g., The []) can contain multiple masks, each 256×256 in size, used to derive the blending weights for each block. The number of masks (i.e., N) can be set to the number of angles supported in the geometric partitioning pattern. For example, referring to Table 22B of Figure 22B, the geometric partitioning pattern supports 64 angles; therefore, the number of masks is 16. A first offset (e.g., offsetX) and a second offset (e.g., offsetY) can be determined based on the block size, the angle value of the target partitioning pattern, and the distance value of the target partitioning pattern. For example, the offsetX and offsetY of a block of size W×H can be determined using the following equation.

[0273] Where A and D are the angle index and distance index, respectively. The partitioning pattern index K can be used to look up the table (e.g., ...). Figure 22BFrom Table 22B, we obtain the angle index A and the distance index D. Therefore, the number of masks is determined based on the number of angle indices in the lookup table, the angle value of the target partitioning method is determined based on the angle index of the lookup table corresponding to the target partitioning method, and the distance value of the target partitioning method is determined based on the distance index of the lookup table corresponding to the target partitioning method.

[0274] Angle index A is used to determine whether the segmentation is performed horizontally or vertically. Angle index A is... Figure 22B In the case of the lookup table derived from Table 22B, when the angle index A equals 8 or 24, the block is horizontally split. On the other hand, when the angle index equals 0 or 16, the block is vertically split. Therefore, the condition... Equivalent to .

[0275] As another example, the following equation can be used to determine offsetX and offsetY for a block of size W×H.

[0276] Where N halfD The angle index A and distance index D are determined based on the partitioning pattern index K, which is half the number of distances supported by the geometric partitioning pattern. For example, A = K % N, D = K / N. Thus, the angle value of the target partitioning method is determined based on the partitioning pattern index and the number of angles supported by the geometric partitioning pattern, and the distance value of the target partitioning method is determined based on the partitioning pattern index and the number of angles supported by the geometric partitioning pattern.

[0277] Based on a set of masks (e.g., []), multiple blended weights of edge sub-blocks can be generated using determinable first and second offsets (offsetX and offsetY).

[0278] In some embodiments, the first offset (e.g., offsetX) and the second offset (e.g., offsetY) can be determined without using this set of masks, and the blending weights can be determined on the fly. For example, the weight of each edge block can be calculated using the following equation: ,in, Set to 32 The weights of the brightness samples located at (x, y) (e.g., The following can be calculated: weightIdx = ( ( ( x + offsetX ) << 1 )+1 ) disLut[ displacementX ]+ ( ( ( y + offsetY ) << 1 ) + 1 ) ) disLut[ displacementY ] partFlip = ( angleIdx >= 13&&angleIdx <= 27 ) ? 0 : 1 weightIdxL = partFlip ? 32 + weightIdx : 32- weightIdx Because the first block includes chroma and luma blocks, multiple blending weights can include multiple luma weights for the edge blocks and multiple chroma weights for the edge blocks. Among the multiple chroma weights, the chroma weight is determined based on the luma weight of the top-left corner of the 2×2 sub-block corresponding to that chroma weight. For example, refer to... Figure 7 The luminance weight of the top left corner of the 2×2 luminance sub-block can be used as the chrominance weight of the chrominance block.

[0279] Therefore, mixing the first and second prediction signals of the edge block associated with the segmentation edge may further include: mixing the first and second prediction signals to determine the luminance value of the edge block according to multiple luminance weights of the edge block; and mixing the first and second prediction signals to determine the chrominance value of the edge block according to multiple chrominance weights of the edge block.

[0280] In some embodiments, a non-transitory computer-readable storage medium comprising a plurality of instructions is also provided, and these instructions can be executed by a device for performing the methods described above (e.g., the disclosed encoder and decoder). Common forms of non-transitory media include, for example, floppy disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs and EPROMs, FLASH-EPROMs or any other flash memory, NVRAM, caches, registers, any other memory chips or cassette tapes, and their network versions. The device may include one or more processors (CPUs), input / output interfaces, network interfaces, and / or memory.

[0281] The embodiments may be further described using the following terms: 1. A video content processing method, comprising: The image is divided into a first partition and a second partition along the dividing edges; Inter-frame prediction is performed on the plurality of blocks to generate a first prediction signal for the first partition and a second prediction signal for the second partition; and The first and second prediction signals of the edge blocks associated with the division edge are mixed.

[0282] 2. The method according to Clause 1, wherein dividing multiple blocks along the dividing edge further includes: Determine the partitioning pattern of the plurality of blocks; and The multiple blocks are partitioned based on the partitioning pattern.

[0283] 3. The method according to Clause 2, wherein the plurality of blocks are sub-blocks of the first block, and determining the partitioning pattern of the plurality of blocks further includes: Determine whether the first block uses sub-block merging mode encoding based on the first indication signal; In response to determining that the first block is not encoded using the sub-block merging mode, the second indication signal determines whether the first block is encoded using the normal mode or the merging mode with motion vector difference (MMVD). In response to determining that the first block is not encoded using either the regular mode or MMVD, a third indication signal is used to determine whether the first block is encoded using the Combined Inter-Frame and Intra-Frame Prediction (CIIP) mode; and In response to determining that the first block is not encoded using CIIP, the partitioning mode of the plurality of blocks is determined to be either a triangular partitioning mode or a geometric partitioning mode.

[0284] 4. The method according to clause 2 or 3, wherein dividing the plurality of blocks along the dividing edge further comprises: The target partitioning method is determined based on the partitioning pattern index, angle index, or distance index; and Determine the partitioning edges corresponding to the target partitioning method.

[0285] 5. The method according to Clause 4 further includes: Generate a set of masks; The first offset and the second offset are determined based on the size of the first block, the angle value of the target partitioning method, and the distance value of the target partitioning method; and Based on this set of masks, multiple mixed weights are generated using the first offset and the second offset.

[0286] 6. According to the method in Clause 5, where The number of group masks is determined based on the number of angles supported by the geometric partitioning mode. The angle value of the target partitioning method is determined based on the partitioning mode index and the number of angles supported by the geometric partitioning mode. The distance value of the target partitioning method is determined based on the partitioning mode index and the number of angles supported by the geometric partitioning mode.

[0287] 7. The method according to clause 5 or 6, wherein determining the target partitioning method based on the partitioning mode index, angle index, or distance index further includes: The target partitioning method is determined based on a lookup table, wherein the lookup table includes multiple partitioning pattern indexes, multiple angle indexes, and multiple distance indexes associated with multiple partitioning methods.

[0288] 8. According to the method in Clause 7, where The number of group masks is determined based on the number of angle indices in the lookup table, the angle value of the target partitioning method is determined based on the angle index of the lookup table corresponding to the target partitioning method, and the distance value of the target partitioning method is determined based on the distance index of the lookup table corresponding to the target partitioning method.

[0289] 9. The method according to Clause 7 or 8, wherein the lookup table includes: a first partitioning method associated with a first partitioning pattern index and a second partitioning method associated with a second partitioning pattern index, wherein the first partitioning method and the second partitioning method correspond to the triangular partitioning method.

[0290] 10. The method according to Clause 9, wherein the first partitioning pattern index is equal to 10, and the first partitioning method associated with the first partitioning pattern index corresponds to splitting the block from the upper left corner to the lower right corner; the second partitioning pattern index is equal to 24, and the second partitioning method associated with the second partitioning pattern index corresponds to splitting the block from the upper right corner to the lower left corner.

[0291] 11. The method according to any one of clauses 5-10, wherein the plurality of blending weights includes a plurality of luminance weights for the edge block and a plurality of chrominance weights for the edge block, and blending the first and second prediction signals of the edge block associated with the edge segmentation further includes: The brightness value of the edge block is determined based on multiple brightness weights of the edge sub-block; and The chromaticity value of the edge sub-block is determined based on multiple chromaticity weights of the edge sub-block.

[0292] 12. The method according to Clause 11, wherein, among the plurality of chromaticity weights, the chromaticity weight is determined based on the luminance weight of the upper left corner of the 2×2 block corresponding to a chromaticity weight.

[0293] 13. The method according to Clause 4 further includes: The first offset and the second offset are determined based on the size of the first block, the angle value of the target partitioning method, and the distance value of the target partitioning method; and The first and second offsets are used to generate multiple blending weights for the brightness samples in the first block.

[0294] 14. According to the method of Clause 13, wherein the first offset and the second offset are determined using the following equation: Where W represents the width of the first block, H represents the height of the first block, A represents the angle value of the target division method, and D represents the distance value of the target division method.

[0295] 15. The method according to any one of clauses 3-14, wherein, before determining whether the first block employs combined inter-frame and intra-frame prediction (CIIP) mode coding based on the third indication signal, the method further comprises: Determine whether the size of the first block satisfies a given condition; and In response to determining that the size of the first block satisfies a given condition, the third indication signal is generated, or In response to determining that the size of the block does not meet a given condition, it is determined that the block is encoded using CIIP.

[0296] 16. The method according to Clause 15, wherein the given condition includes: The width and height of the first block are both greater than or equal to 8; and The ratio of the larger of the width and height to the smaller of the width and height is less than or equal to 4.

[0297] 17. A video content processing system, comprising: Memory, the memory storing a set of instructions; and At least one processor, the at least one processor being configured to execute the set of instructions to cause the system to perform: The image is divided into a first partition and a second partition along the dividing edges; Inter-frame prediction is performed on the plurality of blocks to generate a first prediction signal for the first partition and a second prediction signal for the second partition; and The first and second prediction signals of the edge blocks associated with the division edge are mixed.

[0298] 18. The system according to Clause 17, wherein when dividing the plurality of blocks along the partition edge, the at least one processor is configured to execute the group instructions to cause the system to further perform: Determine the partitioning pattern of the plurality of blocks; and The multiple blocks are divided based on the aforementioned partitioning pattern.

[0299] 19. The system according to Clause 18, wherein the plurality of blocks are sub-blocks of a first block, and when determining the partitioning pattern of the plurality of blocks, the at least one processor is configured to execute the group instructions to cause the system to further perform: Determine whether the first block is encoded using a sub-block merging method based on the first indication signal; In response to determining that the first block is not encoded using the sub-block merging mode, the second indication signal determines whether the first block is encoded using either the normal mode or the merging mode with motion vector difference (MMVD). In response to determining that the first block is not encoded using either the regular mode or MMVD, a third indication signal is used to determine whether the first block is encoded using the Combined Inter-Frame and Intra-Frame Prediction (CIIP) mode; and In response to determining that the first block is not encoded using the CIIP method, the partitioning mode of the plurality of blocks is determined to be either a triangular partitioning mode or a geometric partitioning mode.

[0300] 20. A non-transitory computer-readable medium storing an instruction set thereon, the instruction set being executable by at least one processor of a computer system, wherein execution of the instruction set causes the computer system to perform a method comprising: The image is divided into a first partition and a second partition along the dividing edges; Inter-frame prediction is performed on the plurality of blocks to generate a first prediction signal for the first partition and a second prediction signal for the second partition; and The first and second prediction signals of the edge blocks associated with the division edge are mixed.

[0301] It should be noted that the relational terms such as "first" and "second" used in this document are only used to distinguish one entity or operation from another, and do not require or imply any actual relationship or order between these entities or operations. Furthermore, the words "contains," "owns," "includes," and "includes," as well as other similar forms, have equivalent meanings and are open-ended, because one or more items following any of these words do not imply an exhaustive list of such items, or are limited to only the listed items.

[0302] As used herein, unless otherwise specified, the term "or" covers all possible combinations unless impractical. For example, if a database is declared to contain A or B, then unless otherwise explicitly stated or impractical, the database may contain A, or B, or A and B. As a second example, if a database is declared to contain A, B, or C, then unless otherwise explicitly stated or impractical, the database may contain A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0303] It should be understood that the above embodiments can be implemented by hardware, software (program code), or a combination of hardware and software. If implemented by software, it can be stored on the computer-readable medium described above. When executed by a processor, the software can perform the disclosed methods. The computing units and other functional units described in this disclosure can be implemented by hardware, software, or a combination of hardware and software. It will also be understood by those skilled in the art that multiple of the above modules / units can be combined into one module / unit, and each of the above modules / units can be further divided into multiple sub-modules / sub-units.

[0304] In the foregoing specification, numerous specific details have been described with reference to embodiments, which may vary depending on the implementation. Certain modifications and alterations can be made to the described embodiments. Other embodiments will be apparent to those skilled in the art in consideration of the specification and practice of the invention disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the appended claims. The sequence of steps shown in the figures is also intended for illustrative purposes only and is not intended to limit one to any particular order of steps. Therefore, those skilled in the art will understand that these steps may be performed in a different order while implementing the same method.

[0305] Exemplary embodiments have been disclosed in the accompanying drawings and description. However, many variations and modifications can be made to these embodiments. Therefore, although specific terms are used, they are used in a general and descriptive sense only and not for limiting purposes.

Claims

1. A method for processing video content, applied to an encoder, comprising: Determine the partitioning pattern index, angle index, or distance index of the first block encoded using the geometric partitioning pattern; and Perform inter-frame prediction on the first block to generate a first prediction signal and a second prediction signal; The first offset and the second offset are determined based on the size of the first block, the angle index, and the distance index, wherein, The first offset and the second offset are determined based on the following formula: Wherein, "W" represents the width of the first block, "H" represents the height of the first block, "A" represents the angle index, and "D" represents the distance index; Multiple blending weights are generated using the first offset and the second offset for multiple brightness samples in the first block; and Multiple mixing weights of multiple brightness samples from the first block are mixed with the first prediction signal and the second prediction signal.

2. The method according to claim 1, wherein, It also includes dividing the first block associated with the image into a first partition and a second partition along the dividing edge; The process of dividing the first block along the dividing edge further includes: Determining the partitioning pattern of the first block includes: Determine whether the first block is encoded using the sub-block merging mode based on the first indication signal; In response to determining that the first block is not encoded using the sub-block merging mode, a second indication signal is used to determine whether the first block is encoded using either a regular mode or a merging mode with motion vector difference (MMVD); and In response to determining that the first block is not encoded using either the conventional mode or the MMVD, a third indication signal is used to determine whether the first block is encoded using a combined inter-frame and intra-frame prediction CIIP mode; and The first block is divided based on the partitioning pattern.

3. The method according to claim 2, wherein In response to determining that the first block is not encoded using CIIP mode, the partitioning mode of the first block is determined to be geometric partitioning mode.

4. The method according to claim 3, further comprising: Generate a set of masks; and Based on this set of masks, multiple mixed weights are generated using the first offset and the second offset.

5. The method according to claim 4, wherein: The number of group masks is determined based on the number of angles supported in the geometric partitioning pattern; The angle value of the target partitioning method is determined based on the partitioning mode index and the number of angles supported by the geometric partitioning mode; The distance value of the target partitioning method is determined based on the number of angles supported in the partitioning pattern index and the geometric partitioning pattern.

6. The method according to claim 4, wherein The step of determining the target segmentation method based on the segmentation mode index, the angle index, or the distance index further includes: The target partitioning method is determined based on the lookup table, where... The lookup table includes multiple partitioning pattern indexes, multiple angle indexes, and multiple distance indexes associated with multiple partitioning methods.

7. The method according to claim 6, wherein: The number of group masks is determined based on the number of angle indexes in the lookup table. The angle value of the target partitioning method is determined based on the angle index of the lookup table corresponding to the target partitioning method. The distance value of the target partitioning method is determined based on the distance index of the lookup table corresponding to the target partitioning method.

8. The method according to claim 4, wherein The plurality of mixing weights includes a plurality of luminance weights for luminance samples around the segmentation edge and a plurality of chrominance weights for chrominance samples around the segmentation edge, and mixing the first prediction signal and the second prediction signal of the samples associated with the segmentation edge further includes: The brightness values ​​of the multiple brightness samples are determined based on the multiple brightness weights; and The chromaticity values ​​of the multiple chromaticity samples are determined based on the multiple chromaticity weights.

9. The method according to claim 8, wherein Among the plurality of chromaticity weights, a chromaticity weight is determined based on the luminance weight of the top left corner of a 2x2 block corresponding to the chromaticity weight.

10. The method of claim 2, wherein Before determining whether the first block is encoded using a combined inter-frame and intra-frame prediction CIIP mode based on the third indication signal, the method further includes: Determine whether the size of the first block satisfies a given condition; and In response to determining that the size of the first block satisfies the given condition, the third indication signal is generated, or In response to determining that the size of the block does not meet a given condition, it is determined that the block should be encoded using CIIP mode.

11. The method of claim 10, wherein the given conditions include: The width and height of the first block are both greater than or equal to 8; and The ratio of the larger value between the width and the height to the smaller value between the width and the height is less than or equal to 4.

12. A system for processing video content, applied to a decoder, comprising: A memory that stores a set of instructions; and At least one processor is configured to execute the instruction set to enable the system to perform: Determine the target partitioning method by identifying the partitioning pattern index, angle index, or distance index of the first block decoded using the geometric partitioning pattern; and Perform inter-frame prediction on the first block to generate a first prediction signal and a second prediction signal; The first offset and the second offset are determined based on the size of the first block, the angle index, and the distance index, wherein, The first offset and the second offset are determined based on the following formula: Wherein, "W" represents the width of the first block, "H" represents the height of the first block, "A" represents the angle index, and "D" represents the distance index; Multiple blending weights are generated using the first offset and the second offset for multiple brightness samples in the first block; and Multiple mixing weights of multiple brightness samples from the first block are mixed with the first prediction signal and the second prediction signal.

13. The system according to claim 12, wherein, It also includes parsing the partitioning information in the video bitstream that divides the first block associated with the image into a first partition and a second partition; The parsing of the partition information of the first block also includes: The first indication signal is obtained by parsing to determine whether the first block is decoded using the sub-block merging mode; In response to determining that the first block is not decoded using the sub-block merging mode, a second indication signal is parsed to determine whether the first block is decoded using either the normal mode or the merging mode MMVD with motion vector difference; and In response to determining that the first block is not decoded using either the conventional mode or the MMVD, a third indication signal is parsed to determine whether the first block is decoded using a combined inter-frame and intra-frame prediction (CIIP) mode; and The first block is decoded based on the partitioning pattern.

14. The system according to claim 13, wherein Before parsing the third indication signal to determine whether the first block is decoded using a combined inter-frame and intra-frame prediction CIIP mode, the at least one processor is further configured to cause the system to perform: Determine whether the size of the first block satisfies a given condition; and In response to determining that the size of the first block satisfies the given condition, the third indication signal is obtained, or In response to determining that the size of the block does not meet a given condition, it is determined to decode the block using CIIP mode.

15. The system according to claim 14, wherein The given conditions include: The width and height of the first block are both greater than or equal to 8; and The ratio of the larger value between the width and the height to the smaller value between the width and the height is less than or equal to 4.

16. A non-transitory computer-readable storage medium storing an instruction set and a bit stream, the instruction set being executable by one or more processors in a method to generate the bit stream, the method comprising: Determine the partitioning pattern index, angle index, or distance index of the first block encoded in a geometric partitioning pattern; and Perform inter-frame prediction on the first block to generate a first prediction signal and a second prediction signal; A first offset and a second offset are determined based on the size of the first block, the angle index, and the distance index, wherein the first offset and the second offset are determined based on the following formula: Wherein, "W" represents the width of the first block, "H" represents the height of the first block, "A" represents the angle index, and "D" represents the distance index; Multiple blending weights are generated using the first offset and the second offset for multiple brightness samples in the first block; and Multiple mixing weights of multiple brightness samples from the first block are mixed with the first prediction signal and the second prediction signal.

17. The non-transitory computer-readable storage medium according to claim 16, wherein, It also includes dividing the first block associated with the image into a first partition and a second partition along the dividing edge; The process of dividing the first block along the dividing edge further includes: Determining the partitioning pattern of the first block includes: Determine whether the first block is encoded using the sub-block merging mode based on the first indication signal; In response to determining that the first block is not encoded using the sub-block merging mode, a second indication signal is used to determine whether the first block is encoded using either a regular mode or a merging mode with motion vector difference (MMVD); and In response to determining that the first block is not encoded using either the conventional mode or the MMVD, a third indication signal is used to determine whether the first block is encoded using a combined inter-frame and intra-frame prediction CIIP mode; and The first block is divided based on the partitioning pattern.

18. The non-transitory computer-readable storage medium of claim 17, wherein... Before determining whether the first block is encoded using a combined inter-frame and intra-frame prediction CIIP mode based on the third indication signal, the method further includes: Determine whether the size of the first block satisfies a given condition; and In response to determining that the size of the first block satisfies the given condition, the third indication signal is generated, or In response to determining that the size of the block does not meet a given condition, it is determined that the block should be encoded using CIIP mode.

19. The non-transitory computer-readable storage medium according to claim 18, wherein... The given conditions include: The width and height of the first block are both greater than or equal to 8; and The ratio of the larger value between the width and the height to the smaller value between the width and the height is less than or equal to 4.

Images