Video encoding and decoding methods, methods for processing bitstreams, electronic devices and storage media

Abstract

The various embodiments described in this disclosure include methods for video encoding and decoding, methods for processing bitstreams, electronic devices, and storage media. The video decoding method includes acquiring video data comprising a plurality of blocks, the plurality of blocks including a first block. The method further includes identifying a first partitioning pattern for the first block from a plurality of partitioning patterns, wherein the plurality of partitioning patterns includes a first group partitioning pattern and a second group partitioning pattern, each partitioning pattern in the first group partitioning pattern having a single-line boundary and each partitioning pattern in the second group partitioning pattern having multiple-line boundaries. The method further includes: partitioning the first block into a first portion and a second portion according to the first partitioning pattern, wherein the first partitioning pattern includes multiple-line boundaries; and reconstructing the first block, including reconstructing the first portion using a first predictor and reconstructing the second portion using a second predictor.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 348,171, filed June 2, 2022, entitled “GPM and Wedge-Based Prediction Improvements with Polygonal and L-Shape Partitioning,” and is a successor to and claims priority to U.S. Patent Application No. 18 / 128,213, filed March 29, 2023, entitled “Systems and Methods for Partition-Based Predictions,” all of which are incorporated herein by reference in their entirety. Technical Field

[0003] This application relates to video encoding and decoding, and more particularly to a video encoding and decoding method, a method for processing bitstreams, an electronic device, and a storage medium. Background Technology

[0004] Various electronic devices support digital video, such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. Electronic devices send and receive digital video data via communication networks or otherwise transmit digital video data, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video data can be compressed according to one or more video coding standards before transmission or storage.

[0005] Several video codec standards have been developed. For example, video coding standards include AOMedia Video1 (AV1), Multifunctional Video Coding (VVC), Joint Explore Test Model (JEM), High Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), and Moving Picture Experts Group (MPEG) coding. Video coding typically employs predictive methods (e.g., inter-frame prediction, intra-frame prediction, etc.) to utilize the inherent redundancy in video data. Video coding aims to compress video data into a form using a lower bitrate while avoiding or minimizing video quality degradation.

[0006] HEVC (also known as H.265) is a video compression standard designed as part of the MPEG-H project. The ITU-T and ISO / IEC released the HEVC / H.265 standard in 2013 (Revision 1), 2014 (Revision 2), 2015 (Revision 3), and 2016 (Revision 4). Multi-Functional Video Coding (VVC), also known as H.266, is a video compression standard designed to succeed HEVC. The ITU-T and ISO / IEC published the VVC / H.266 standard in 2020 (Revision 1) and 2022 (Revision 2). AV1 is an open video coding format designed as an alternative to HEVC. On January 8, 2019, validation version 1.0.0 and errata 1 of the specification were released. Summary of the Invention

[0007] This disclosure describes various techniques that can be used by a decoder of a video bitstream to improve the quality and / or efficiency of decoding. The video encoder may also implement these techniques during encoding (e.g., to reconstruct encoded frames and / or test hypotheses).

[0008] During the video encoding and decoding process, video data is divided into blocks. In this disclosure, the term "block" can be interpreted as a prediction block, coding block, or coding unit (CU), as detailed below. Geometric partitioning of blocks takes into account the two-dimensional geometry of the video object. After partitioning, different motion predictors can be used for each part of the block. For lossy compression, this partitioning method can improve the quality of more complex objects.

[0009] According to some embodiments, a video decoding method is provided. The method includes: (i) acquiring video data comprising a plurality of blocks, the plurality of blocks including a first block; (ii) identifying a first partitioning pattern for the first block from a plurality of partitioning patterns, wherein the plurality of partitioning patterns include a first group partitioning pattern and a second group partitioning pattern, each partitioning pattern in the first group partitioning pattern having a single-line boundary and each partitioning pattern in the second group partitioning pattern having multiple-line boundaries; (iii) partitioning the first block into a first part and a second part according to the first partitioning pattern, wherein the first partitioning pattern includes multiple-line boundaries; (iv) reconstructing the first block, including reconstructing the first part using a first predictor and reconstructing the second part using a second predictor.

[0010] According to some embodiments, a computing system, such as a streaming system, a server system, a personal computer system, or other electronic device, is provided. The computing system includes a control circuitry system and a memory storing one or more sets of instructions. The set of instructions includes instructions for performing any of the methods described in this disclosure. In some embodiments, the computing system includes encoder components and / or decoder components.

[0011] According to some embodiments, a non-volatile computer-readable storage medium is provided. The non-volatile computer-readable storage medium stores one or more sets of instructions for execution by a computing system. These one or more sets of instructions include instructions for performing any of the methods described in this disclosure.

[0012] Therefore, methods, devices, and systems for video encoding and decoding are disclosed. Such methods, devices, and systems can supplement or replace traditional methods, devices, and systems for video encoding and decoding.

[0013] The features and advantages described in the specification are not necessarily all-encompassing. In particular, some additional features and advantages will be apparent to those skilled in the art from the accompanying drawings, specification, and claims provided in this disclosure. Furthermore, it should be noted that the language used in the specification has been chosen primarily for readability and instructional purposes and is not necessarily intended to define or limit the subject matter described in this disclosure. Attached Figure Description

[0014] To gain a more detailed understanding of this disclosure, reference can be made to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the drawings only illustrate relevant features of this disclosure and are therefore not necessarily limiting, as those skilled in the art will understand upon reading this disclosure that it may contain other effective features.

[0015] Figure 1 This is a block diagram illustrating an example of a communication system according to some embodiments.

[0016] Figure 2A This is a block diagram of example elements of an encoder component according to some embodiments.

[0017] Figure 2B This is a block diagram of example elements of a decoder component according to some embodiments.

[0018] Figure 3 This is a block diagram illustrating an example of a server system according to some embodiments.

[0019] Figures 4A to 4D The illustration shows an example of a coding tree structure according to some embodiments.

[0020] Figure 5A Examples of geometric partitioning pattern predictions based on some embodiments are shown.

[0021] Figures 5B to 5C An example of partition mode mixing according to some embodiments is shown.

[0022] Figure 5D Examples of wedge-based partitioning according to some embodiments are shown.

[0023] Figures 5E to 5H An example of polygon partitioning according to some embodiments is shown.

[0024] Figure 6 This is a flowchart illustrating an example method for decoding video based on some embodiments.

[0025] By convention, the various features illustrated in the accompanying drawings are not necessarily drawn to scale, and similar reference numerals may be used to indicate similar features throughout the specification and the accompanying drawings. Detailed Implementation

[0026] Among other things, this disclosure describes the use of various partitioning techniques to partition video blocks to achieve better motion prediction and higher quality encoding. For example, a straight-line partitioning pattern may not be optimal for more complex video objects. In such cases, an L-shaped or polygonal partitioning pattern, which better represents the shape of the video object, can improve the accuracy of motion prediction, thereby improving the accuracy of video encoding and decoding.

[0027] Example systems and devices

[0028] Figure 1 This is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to 120-m) that are communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is a streaming system, for example, used with video applications such as video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0029] Source device 102 includes a video source 104 (e.g., a camera component or media storage space) and an encoder component 106. In some embodiments, the video source 104 is a digital camera (e.g., used to create an uncompressed video sample stream). The encoder component 106 generates one or more encoded video streams from the video stream. The video stream from video source 104 may have a larger data volume compared to the encoded video stream 108 generated by the encoder component 106. Because the encoded video stream 108 has a smaller data volume (less data) compared to the video stream from the video source, it requires less bandwidth to transmit and less storage space to store compared to the video stream from video source 104. In some embodiments, source device 102 does not include the encoder component 106 (e.g., configured to transmit uncompressed video data to network 110).

[0030] One or more networks 110 represent any number of networks that transmit information between source device 102, server system 112, and / or electronic device 120, including, for example, cable (wired) and / or wireless communication networks. One or more networks 110 can exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet.

[0031] One or more networks 110 include a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is or includes a streaming server (e.g., configured to store and / or distribute video content, such as an encoded video stream from source device 102). The server system 112 includes an encoder component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the encoder component 114 includes an encoder component and / or a decoder component. In various embodiments, the encoder component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the encoder component 114 is configured to decode the encoded video stream 108 and re-encode the video data using different encoding standards and / or methods to generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings based on the encoded video stream 108.

[0032] In some embodiments, server system 112 serves as a media-aware network element (MANE). For example, server system 112 may be configured to trim encoded video streams 108 to customize potentially different streams for one or more electronic devices 120. In some embodiments, the MANE is provided separately from server system 112.

[0033] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode encoded video data 116 to generate an output video stream that can be reproduced on a display or other type of presentation device. In some embodiments, one or more electronic devices 120 do not include a display component (e.g., communicatively coupled to an external display device and / or includes media storage space). In some embodiments, electronic device 120 is a streaming client. In some embodiments, electronic device 120 is configured to access server system 112 to obtain encoded video data 116.

[0034] The source device and / or multiple electronic devices 120 are sometimes referred to as “terminal devices” or “user equipment”. In some embodiments, the source device 102 and / or one or more electronic devices 120 are examples of server systems, personal computers, portable devices (e.g., smartphones, tablets, or laptops), wearable devices, video conferencing equipment, and / or other types of electronic devices.

[0035] In an example operation of communication system 100, source device 102 transmits an encoded video stream 108 to server system 112. For example, source device 102 may encode an image stream acquired by the source device. Server system 112 receives the encoded video stream 108 and may decode and / or encode the encoded video stream 108 using encoder component 114. For example, server system 112 may encode video data in a way more suitable for network transmission and / or storage. Server system 112 may transmit encoded video data 116 (e.g., one or more encoded video streams) to one or more electronic devices 120. Each electronic device 120 may decode the encoded video data 116 to recover and optionally display video images.

[0036] In some embodiments, the above transmission is unidirectional data transmission. Unidirectional data transmission is sometimes used in media service applications, etc. In some embodiments, the above transmission is bidirectional data transmission. Bidirectional data transmission is sometimes used in video conferencing applications, etc. In some embodiments, the encoded video stream 108 and / or the encoded video data 116 are encoded and / or decoded according to any video encoding / compression standard described in this disclosure (e.g., HEVC, VVC, and / or AV1).

[0037] Figure 2AThis is a block diagram illustrating example elements of an encoder component 106 according to some embodiments. The encoder component 106 receives a source video sequence from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives the video sequence from a remote video source (e.g., a video source belonging to a different device component than the encoder component 106). The video source 104 may provide the source video sequence as a digital video sample stream, which may have any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601YCrCb or RGB), and any suitable sampling structure (e.g., YCrCb 4:2:0 or YCrCb 4:4:4). In some embodiments, the video source 104 is a storage device storing previously acquired / prepared video. In some embodiments, the video source 104 is a camera that acquires local image information as a video sequence. The video data may be provided as multiple individual pictures that create a sense of motion when viewed sequentially. An image itself can be organized as a spatial array of pixels, where each pixel can be one or more samples, depending on the sampling structure, color space, etc. Those skilled in the art can readily understand the relationship between pixels and samples. The following description focuses on samples.

[0038] Encoder component 106 is configured to encode and / or compress images of a source video sequence into an encoded video sequence 216 in real time or under other time constraints required by the application. Implementing an appropriate encoding rate is one of the functions of controller 204. In some embodiments, controller 204 controls and is functionally coupled to other functional units described below. Parameters set by controller 204 may include rate control-related parameters (e.g., image skipping, quantizer and / or λ value of rate distortion optimization techniques), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. Other functions of controller 204 will be readily recognizable to those skilled in the art, as they may relate to encoder component 106 optimized for a particular system design.

[0039] In some embodiments, encoder component 106 is configured to operate in an encoding loop. In a simplified example, the encoding loop includes a source encoder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder 210. Decoder 210 reconstructs the symbols in a manner similar to how the (remote) decoder creates sample data to create sample data (when there is lossless compression between the symbols and the encoded video stream). The reconstructed sample stream (sample data) is input to reference image memory 208. Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of reference image memory 208 are also bit-accurate between the local encoder and the remote encoder. Thus, the encoder's prediction portion interprets the same sample values ​​used by the decoder during prediction as reference image samples. The principle of this reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is known to those skilled in the art.

[0040] The operation of decoder 210 can be the same as that of a remote decoder (such as decoder component 122), as described below. Figure 2B Decoder component 122 is described in detail. Brief reference is provided. Figure 2B However, since symbols are available and the entropy encoder 214 and parser 254 can encode / decode symbols into an encoded video sequence in a lossless manner, the entropy decoding portion of decoder component 122 (including buffer memory 252 and parser 254) may not be fully implemented in local decoder 210.

[0041] It can be observed that any decoder technique other than parsing / entropy decoding present in the decoder must also exist in the corresponding encoder in essentially the same functional form. For this reason, this application focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are inverses of the fully described decoder techniques. More detailed descriptions are only required in certain areas, and are provided below.

[0042] As part of its operation, the source encoder 202 can perform motion-compensated predictive coding, predictively encoding the input frame with reference to one or more previously encoded frames from the video sequence designated as reference frames. In this manner, the encoding engine 212 encodes the differences between pixel blocks of the input frame and pixel blocks of the reference frame, which can be selected as the prediction reference for the input frame. The controller 204 can manage the encoding operations of the source encoder 202, including, for example, setting parameters and subgroup parameters for encoding the video data.

[0043] Decoder 210 can decode encoded video data of frames that can be designated as reference frames, based on symbols created by source encoder 202. The operation of encoding engine 212 can advantageously be a lossy process. When encoded video data is processed by video decoder (… Figure 2A During decoding at a location (not shown), the reconstructed video sequence can be a copy of the source video sequence with some errors. Decoder 210 replicates the decoding process, which can be performed on the reference frame by a remote video decoder, and the reconstructed reference frame can be stored in reference image memory 208. In this way, encoder component 106 locally stores copies of the reconstructed reference frames that share common content (no transmission errors) with the reconstructed reference frames to be obtained by the remote video decoder.

[0044] Predictor 206 can perform a prediction search against encoding engine 212. That is, for a new frame to be encoded, predictor 206 can search the reference image memory 208 for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new image. Predictor 206 can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by predictor 206, it can be determined that the input image may have prediction references extracted from multiple reference images stored in the reference image memory 208.

[0045] The outputs of all the aforementioned functional units can undergo entropy encoding in the entropy encoder 214. The entropy encoder 214 converts the symbols generated by the various functional units into an encoded video sequence by performing lossless compression of the symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, and / or arithmetic coding).

[0046] In some embodiments, the output of entropy encoder 214 is coupled to a transmitter. The transmitter can be configured to buffer encoded video sequences, such as those created by entropy encoder 214, in preparation for transmission via communication channel 218, which may be a hardware / software link to a storage device storing the encoded video data. The transmitter can be configured to combine encoded video data from source encoder 202 with other data to be transmitted (e.g., encoded audio data and / or auxiliary data streams (source not shown)). In some embodiments, the transmitter can send additional data and encoded video. Source encoder 202 may include such data as part of the encoded video sequence. Additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data (such as redundant pictures and stripes), auxiliary enhancement information (SEI) messages, fragments of visual usability information (VUI) parameter sets, etc.

[0047] Controller 204 can manage the operation of encoder component 106. During encoding, controller 204 can assign a specific type of encoded picture to each encoded picture, which may affect the encoding technique applied to the corresponding picture. For example, pictures can be assigned as intra-pictures (I-pictures), prediction pictures (P-pictures), or bidirectional prediction pictures (B-pictures). Intra-pictures can be encoded and decoded without using any other frames in the sequence as prediction sources. Some video codecs allow the use of different types of intra-pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are familiar with the variations of I-pictures and their respective applications and characteristics, and therefore they will not be repeated here. Predictive pictures can be encoded and decoded using intra-prediction or inter-prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block. Bidirectional prediction pictures can be encoded and decoded using intra-prediction or inter-prediction, which uses at most two motion vectors and reference indexes to predict sample values ​​for each block. Similarly, multiple prediction pictures can be used to reconstruct a single block using two or more reference pictures and associated metadata.

[0048] Source images are typically spatially subdivided into multiple sample blocks (e.g., each sample is a 4×4, 8×8, 4×8, or 16×16 block) and encoded on a block-by-block basis. These blocks can be predictively encoded with reference to other (already encoded) blocks, which are determined based on the encoding assignment of the corresponding images applied to the blocks. For example, blocks of image I can be encoded unpredictably or predictively with reference to already encoded blocks of the same image (spatial prediction or intra-frame prediction). Pixel blocks of image P can be unpredictably encoded with reference to a previously encoded reference image, either spatially or temporally. Blocks of image B can be unpredictably encoded with reference to one or two previously encoded reference images, either spatially or temporally.

[0049] The acquired video can serve as multiple source images (video images) presented in a time series. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In an embodiment, a specific image being encoded / decoded is segmented into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when multiple reference images are used, the motion vector may have a third dimension that identifies the reference image.

[0050] Encoder component 106 can perform encoding operations according to a predetermined video coding technique or standard (such as any technique or standard described in this disclosure). In its operation, encoder component 106 can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0051] Figure 2B This is a block diagram illustrating example elements of a decoder component 122 according to some embodiments. Figure 2B The decoder component 122 is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to the loop filter unit 256 and configured to transmit data to the display 124 (e.g., via a wired or wireless connection).

[0052] In some embodiments, decoder component 122 includes a receiver coupled to channel 218 and configured to receive data from channel 218 (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more encoded video sequences to be decoded by decoder component 122. In some embodiments, decoding of each encoded video sequence is independent of other encoded video sequences. Each encoded video sequence may be received from channel 218, which may be a hardware / software link to a storage device storing the encoded video data. The receiver may receive encoded video data and other data (e.g., encoded audio data and / or auxiliary data streams), which may be forwarded to their respective user entities (not depicted). The receiver may separate the encoded video sequences from other data. In some embodiments, the receiver receives additional (redundant) data and encoded video. The additional data may be included as part of one or more encoded video sequences. Decoder component 122 may use the additional data to decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or SNR enhancement layers, redundant stripes, redundant pictures, forward error correction codes, etc.

[0053] According to some embodiments, the decoder component 122 includes a buffer memory 252, a parser 254 (sometimes also referred to as an entropy decoder), a scaler / inverse transform unit 258, an intra-frame prediction unit 262, a motion compensation prediction unit 260, an aggregator 268, a loop filter unit 256, a reference image memory 266, and a current image memory 264. In some embodiments, the decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuits. In some embodiments, the decoder component 122 is implemented at least partially in software.

[0054] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., to combat network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, a separate buffer memory is provided outside decoder component 122 (e.g., to combat network jitter) in addition to buffer memory 252 inside decoder component 122 (e.g., buffer memory 252 is configured to handle playback timing). Buffer memory 252 may not be needed when receiving data from a store / forward device with sufficient bandwidth and controllability or from an isochronous network, or buffer memory 252 may be small. Buffer memory 252 may be needed when used on best-effort packet networks (such as the Internet), and buffer memory 252 may be relatively large and may advantageously have an adaptive size, and may be implemented at least partially outside decoder component 122 in an operating system or similar element (not depicted).

[0055] Parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. Symbols may include, for example, information for managing the operation of decoder component 122, and / or information for controlling presentation devices such as display 124. Control information for the presentation device may be in the form of, for example, supplementary enhancement information (SEI) messages or video availability information (VUI) parameter set fragments (not depicted). Parser 254 parses (entropy decodes) the encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow principles known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. Parser 254 may extract a subgroup parameter set from the encoded video sequence for at least one pixel subgroup in the video decoder based on at least one parameter corresponding to a group. Subgroups may include picture groups (GOPs), pictures, tiles, stripes, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. Parser 254 may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0056] The reconstruction of symbol 270 can involve multiple different units, depending on the type of encoded video picture or its portion (such as inter-frame and intra-frame pictures, inter-frame and intra-frame blocks) and other factors. Which units are involved and how they are involved can be controlled by subgroup control information, which is parsed by parser 254 based on the encoded video sequence. For clarity, the flow of this subgroup control information between parser 254 and the multiple units described below is not described.

[0057] In addition to the functional blocks already mentioned, the decoder component 122 can be conceptually subdivided into multiple functional units as described below. In practical implementations operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with one another. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision of the functional units below is retained.

[0058] The scaler / inverse transform unit 258 receives quantization transform coefficients and control information (such as the transform mode to be used, block size, quantization factor, and / or quantization scaling matrix) as one or more symbols 270 from the parser 254. The scaler / inverse transform unit 258 can output blocks including sample values, which can be input into the aggregator 268.

[0059] In some cases, the output samples of the scaler / inverse transform unit 258 belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images, but can use predictive information from previously reconstructed portions of the current image. This predictive information can be provided by the intra-prediction unit 262. The intra-prediction unit 262 can use surrounding reconstructed information obtained from the current (partially reconstructed) image in the current image memory 264 to generate blocks of the same size and shape as the blocks being reconstructed. The aggregator 268 can add the predictive information already generated by the intra-prediction unit 262 to the output sample information provided by the scaler / inverse transform unit 258, based on each sample.

[0060] In other cases, the output samples of the scaler / inverse transform unit 258 belong to the inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit 260 can access the reference image memory 266 to obtain samples for prediction. After motion compensation is performed on the obtained samples according to the symbols 270 belonging to the block, these samples can be added by the aggregator 268 to the output of the scaler / inverse transform unit 258 (referred to as residual samples or residual signals in this case) to generate output sample information. The address in the reference image memory 266 from which the motion compensation prediction unit 260 obtains the predicted samples can be controlled by motion vectors. Motion vectors can be provided to the motion compensation prediction unit 260 in the form of symbols 270, which can have, for example, X, Y, and reference image components. Motion compensation can also include interpolation of sample values ​​obtained from the reference image memory 266 when using subsampled precise motion vectors, motion vector prediction mechanisms, etc.

[0061] The output samples of aggregator 268 can undergo various loop filtering techniques in loop filter unit 256. Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video bitstream and provided to loop filter unit 256 as symbols 270 from parser 254, but may also be in response to metadata obtained during decoding of previous (in decoding order) portions of the encoded picture or encoded video sequence, and in response to previously reconstructed and loop-filtered sample values.

[0062] The output of the loop filter unit 256 can be a sample stream, which can be output to a presentation device such as the display 124, and stored in the reference image memory 266 for subsequent inter-frame image prediction.

[0063] Once a certain encoded image is fully reconstructed, it can be used as a reference image for future predictions. Once an encoded image has been fully reconstructed and has been identified as a reference image (e.g., by parser 254), the current reference image can become part of the reference image memory 266, and a new current image memory can be reallocated before the reconstruction of subsequent encoded images begins.

[0064] Decoder component 122 can perform decoding operations according to a predetermined video compression technique that may be described in a standard (such as any of the standards described in this disclosure). The encoded video sequence may conform to the syntax specified by the video compression technique or standard used; in this sense, it follows the syntax of the video compression technique or standard, as specified in the video compression technique documentation or standard and in the summary document specifically therein. Furthermore, in order to conform to some video compression techniques or standards, the complexity of the encoded video sequence may be within limits defined by the level of the video compression technique or standard. In some cases, the level limits the maximum image size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference image size, etc. In some cases, the level-based limitations may be further restricted by the assumed reference decoder (HRD) specifications and metadata managed by the HRD buffer, which are signaled in the encoded video sequence.

[0065] Figure 3This is a block diagram illustrating a server system 112 according to some embodiments. The server system 112 includes a control circuitry system 302, one or more network interfaces 304, a memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuitry system 302 includes one or more processors (e.g., a CPU, GPU, and / or DPU). In some embodiments, the control circuitry system includes one or more field-programmable gate arrays (FPGAs), hardware accelerators, and / or one or more integrated circuits (e.g., application-specific integrated circuits).

[0066] One or more network interfaces 304 can be configured to interface with one or more communication networks (e.g., wireless, wired, and / or optical networks). Communication networks can be local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), vehicular and industrial networks, real-time networks, latency-tolerant networks, and so on. Examples of communication networks include LANs such as Ethernet and wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial networks including CANbus, and so on. Such communication can be one-way (e.g., broadcast TV), one-way (e.g., CANbus to certain CANbus devices), or bidirectional (e.g., to other computer systems using LANs or WANs). Such communication can include communication to one or more cloud computing networks.

[0067] User interface 306 includes one or more output devices 308 and / or one or more input devices 310. The one or more input devices 310 may include one or more of the following: keyboard, mouse, touchpad, touchscreen, data glove, joystick, microphone, scanner, camera, etc. The one or more output devices 308 may include one or more of the following: audio output devices (e.g., speakers), visual output devices (e.g., displays or monitors), etc.

[0068] Memory 314 may include high-speed random access memory (such as DRAM, SRAM, DDR RAM, and / or other random access solid-state memory devices) and / or non-volatile memory (such as one or more disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state memory devices). Memory 314 may optionally include one or more storage devices located remotely from the control circuitry system 302. Memory 314 or optionally one or more non-volatile solid-state memory devices within memory 314 may include a non-volatile computer-readable storage medium. In some embodiments, memory 314 or the non-volatile computer-readable storage medium of memory 314 stores programs, modules, instructions, and data structures, or subsets or supersets thereof:

[0069] • Operating system 316, which includes processes for handling various basic system services and performing hardware-related tasks;

[0070] • Network communication module 318 is used to connect server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired and / or wireless connections);

[0071] • Encoding module 320 is used to perform various functions related to the encoding and / or decoding of data (such as video data). In some embodiments, encoding module 320 is an example of encoder component 114. Encoding module 320 includes, but is not limited to, one or more of the following:

[0072] ○ Decoding module 322 is used to perform various functions related to the decoding of encoded data, such as the functions previously described related to decoder component 122; and

[0073] ○ Encoding module 340, for performing various functions related to data encoding, such as the functions previously described related to encoder component 106; and

[0074] ● Image memory 352 is used to store images and image data, for example, in conjunction with encoding module 320. In some embodiments, image memory 352 includes one or more of the following: reference image memory 208, buffer memory 252, current image memory 264, and reference image memory 266.

[0075] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform the various functions previously described related to the parser 254), a transformation module 326 (e.g., configured to perform the various functions previously described related to the scaler / inverse transformation unit 258), a prediction module 328 (e.g., configured to perform the various functions previously described related to the motion compensation prediction unit 260 and / or the intra-frame prediction unit 262), and a filter module 330 (e.g., configured to perform the various functions previously described related to the loop filter unit 256).

[0076] In some embodiments, the encoding module 340 includes a code module 342 (e.g., configured to perform the various functions previously described relating to the source encoder 202, encoding engine 212, and / or entropy encoder 214) and a prediction module 344 (e.g., configured to perform the various functions previously described relating to the predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 include Figure 3 The modules shown are subsets. For example, both decoding module 322 and encoding module 340 use a shared prediction module.

[0077] Each of the identified modules stored in memory 314 corresponds to a set of instructions for performing the functions described herein. The identified modules (e.g., multiple sets of instructions) do not need to be implemented as separate software programs, processes, or modules, and therefore various subsets of these modules can be combined or otherwise rearranged in various embodiments. For example, encoding module 320 may optionally not include separate decoding and encoding modules, but instead use the same set of modules to perform both sets of functions. In some embodiments, memory 314 stores a subset of the identified modules and data structures. In some embodiments, memory 314 stores additional modules and data structures not described above, such as audio processing modules.

[0078] In some embodiments, server system 112 includes a web server or Hypertext Transfer Protocol (HTTP) server, File Transfer Protocol (FTP) server, and web pages and applications implemented using Common Gateway Interface (CGI) scripts, PHP Hypertext Preprocessor (PHP), Dynamic Server Pages (ASP), Hypertext Markup Language (HTML), Extensible Markup Language (XML), Java, JavaScript, Asynchronous JavaScript and XML (AJAX), XHP, Javelin, Wireless Universal Resource File (WURFL), etc.

[0079] although Figure 3 The illustration shows a server system 112 according to some embodiments, but... Figure 3 This disclosure is intended more as a functional description of various features that may exist in one or more server systems, and not as a structural schematic diagram of the embodiments described herein. In practice, and as those skilled in the art will recognize, items shown individually may be combined and some items may be separated. For example, Figure 3 Some items shown individually can be implemented on a single server, and a single item can be implemented by one or more servers. The actual number of servers used to implement server system 112, and how features are allocated among them, will vary from one implementation to another, and may optionally depend in part on the amount of data traffic processed by the server system during peak usage periods and during average usage periods.

[0080] Example encoding method

[0081] Figures 4A to 4D The diagram illustrates an example encoding tree structure based on some embodiments. For example... Figure 4A As shown in the first coding tree structure (400), some coding methods (e.g., VP9) use a 4-way partition tree that descends from a 64×64 level to a 4×4 level, with some additional constraints for 8×8 blocks. Figure 4A In this context, a partition designated as R can be called a recursive partition, meaning that the same partition tree is repeated at a lower ratio until the lowest 4×4 level is reached.

[0082] like Figure 4B As shown in the second coding tree structure (402), some coding methods (e.g., AV1) expand the partition tree to a 10-way structure and increase the maximum size (e.g., referred to as the superblock in VP9 / AV1 terminology) to start from 128×128. The second coding tree structure includes 4:1 / 1:4 rectangular partitions that are not present in the first coding tree structure. Figure 4B In the second row, a partition type with 3 sub-partitions is called a T-partition. Rectangular partitions in this tree structure cannot be further subdivided. In addition to the block size, the tree depth can be defined to represent the partitioning depth starting from the root node. For example, the tree depth of the root node (e.g., 128×128) can be set to 0, and the tree depth increases by 1 after each further partition.

[0083] For example, AV1 does not enforce the fixed transform unit size of VP9, ​​but instead allows the luma coding block to be partitioned into transform units of various sizes, which can be represented by partitions recursively down to a maximum of two levels. To accommodate AV1's extended coding block partitioning, square, 2:1 / 1:2, and 4:1 / 1:4 transform sizes are supported, ranging from 4×4 to 64×64. For chroma blocks, only the largest possible transform unit is allowed.

[0084] As an example, a CTU can be partitioned into CUs using a quadtree structure represented as a coding tree to accommodate various local characteristics, such as in HEVC. In some embodiments, at the CU level, it is decided whether to encode a picture region using inter-frame picture (temporal) prediction or intra-frame picture (spatial) prediction. Depending on the PU partitioning type, each CU can be further partitioned into one, two, or four PUs. Within a PU, the same prediction process is applied, and relevant information is transmitted to the decoder on a PU-by-PU basis. After obtaining residual blocks by applying a prediction process based on the PU partitioning type, the CUs can be partitioned into TUs according to another quadtree structure (such as the coding tree used for the CUs). A key feature of the HEVC structure is that it has multiple partitioning concepts including CUs, PUs, and TUs. In HEVC, a CU or TU can only be a square, while a PU can be a square or a rectangle for inter-frame prediction blocks. In HEVC, a coding block can be further partitioned into four square sub-blocks, and a transformation is performed on each sub-block (TU). Each TU can be further recursively partitioned (using quadtree partitioning) into smaller TUs, which is called a residual quadtree (RQT). At image boundaries, such as in HEVC, implicit quadtree segmentation can be used, so that a block will be continuously segmented into quadtrees until its size meets the image boundaries.

[0085] Quadtrees with nested multi-type trees using binary and ternary partitioning segmentation structures (such as the quadtree in VVC) can replace the concept of multiple partition unit types. For example, it eliminates the separation of CU, PU, ​​and TU concepts (except for CUs that are too large to meet the maximum transform length requirement) and supports more flexible CU partition shapes. In the coding tree structure, CUs can have square or rectangular shapes. ACTUs are initially partitioned by a quadtree structure. Quadtree leaf nodes can be further partitioned by multi-type tree structures. Figure 4C As shown in the third coding tree structure (404), the multi-type tree structure includes four partition types. For example, the multi-type tree structure includes vertical binary partition (SPLIT_BT_VER), horizontal binary partition (SPLIT_BT_HOR), vertical ternary partition (SPLIT_TT_VER), and horizontal ternary partition (SPLIT_TT_HOR). The leaf nodes of the multi-type tree are called CUs. Unless the CU is too large to meet the maximum transform length requirement, this segmentation will be used for prediction and transform processing without any further partitioning. This means that in most cases, in a quadtree with a nested multi-type tree coding block structure, the CU, PU, ​​and TU have the same block size. An exception is when the maximum supported transform length is less than the width or height of the color component of the CU. Figure 4D This example shows a block partition of a CTU(406). Figure 4D The illustration shows an example quadtree with a nested multi-type tree coding block structure.

[0086] For example, in VVC, the maximum supported luma transformation size can be 64×64, and the maximum supported chroma transformation size can be 32×32. When the width or height of the CB is greater than the maximum transformation width or height, the CB will be automatically split in the horizontal and / or vertical directions to meet the transformation size limit in that direction.

[0087] In VTM7, for example, the coding tree scheme supports the ability for luma and chroma to have separate block tree structures. In some cases, for P-band and B-band, the luma and chroma CTBs within a single CTU share the same coding tree structure. However, for I-band, luma and chroma can have separate block tree structures. When the separate block tree mode is applied, the luma CTB is partitioned into CUs by one coding tree structure, and the chroma CTB is partitioned into chroma CUs by another coding tree structure. This means that a CU in an I-band can include or consist of a coding block of one luma component or two chroma components, while a CU in a P-band or B-band can always include or consist of coding blocks of all three color components, unless the video is black and white.

[0088] To support extended coded block partitioning, various transformation sizes (e.g., from 4 points to 64 points per dimension) and transformation shapes (e.g., squares or rectangles with width / height ratios of 2:1 / 1:2 and 4:1 / 1:4) can be utilized, as in AV1.

[0089] Motion estimation involves determining motion vectors, which describe the transformation from one image (picture) to another. A reference image (or patch) can be drawn from adjacent frames in a video sequence. Motion vectors can be associated with the entire image (global motion estimation) or a specific patch. Furthermore, motion vectors can correspond to translation or warping models to approximate motion (e.g., 3D rotation, translation, and scaling). In some cases (e.g., more complex video objects), motion estimation can be improved by further partitioning the patch.

[0090] Geometric Partitioning (GPM) can focus on the Cues (Cues) for inter-frame image prediction. When GPM is applied to a Cue, the Cue is divided into two parts by straight partition boundaries. The position of the partition boundaries can be mathematically defined by the angle parameter φ and the offset parameter ρ. These parameters can be quantized and combined into a GPM partition index lookup table. The GPM partition index of the current Cue can be encoded into the bitstream. For example, for a Cue of size w×h=2k×2l (in terms of luminance samples), where k, l∈{3…6}, GPM in VVC supports 64 partitioning modes. For Cues with aspect ratios greater than 4:1 or less than 1:4, GPM can be disabled, for example, because narrow Cues rarely contain geometrically separated patterns.

[0091] Following partitioning, the two GPM portions (partitions) contain individual motion information that can be used to predict the corresponding portion in the current CU. In some embodiments, each portion of the GPM only allows unidirectional motion compensation prediction (MCP), so the memory bandwidth required for MCP in the GPM is equal to the memory bandwidth required for a conventional bidirectional MCP. To simplify motion information encoding and reduce the possible combinations of GPMs, motion information can be encoded using a merging pattern. The GPM merging candidate list can be derived from the conventional merging candidate list to ensure that only unidirectional motion information is included.

[0092] Figure 5A The illustration shows the GPM prediction process according to some embodiments. The current block 510 is divided into a right portion and a left portion by partition 516. The right prediction portion of the current block 510 (e.g., CU) of the current image 502 (e.g., of size w×h) is predicted by MV0 from reference block 512 from reference image 504, while the left portion is predicted by MV1 from reference block 514 from reference image 506.

[0093] Figure 5BThe illustration shows an example blending matrix for partitions (e.g., partition 516) according to some embodiments. In this example, the final GPM prediction (PG) is generated by performing a blending process using integer blending matrices W0 and W1 (e.g., weights containing values ​​in the range of 0 to 8). This can be represented as:

[0094] PG = (W0·P0 + W1·P1 + 4) >> 3

[0095] Where W0 + W1 = 8J

[0096] Equation 1 - Hybrid Prediction

[0097] In Equation 1, J is a matrix of size w×h. The weights of the blending matrix can depend on the displacement between the sample locations and the partition boundaries. The computational complexity of the blending matrix derivation can be extremely low, thus allowing these matrices to be generated on the decoder side.

[0098] The generated GPM prediction (PG) can then be subtracted from the original signal to generate the residual. For example, using a conventional VVC transform, quantization, and entropy coding engine, the residual is transformed, quantized, and encoded into a bitstream. On the decoder side, the signal is reconstructed by adding the residual to the GPM prediction PG. GPM also supports skip modes, such as when the residual is negligible. For example, the encoder discards the residual, and the decoder directly uses the GPM prediction PG as the reconstructed signal.

[0099] GPM can be further enhanced, for example through GPM+TM, GPM+MMVD, and inter-frame + intra-frame GPM. Figure 5C As shown, the blending intensity or blending region width θ can be fixed for all different content. In some embodiments, the weighting values ​​in the blending mask are given by a ramp function:

[0100]

[0101] For example, where θ = 2 is fixed. This ramp function can be quantized as:

[0102] ω m，n =Clip3(0,8,(d(m,n)+32+4)>>3)

[0103] Equation 3 - Quantized ramp function

[0104] This design may not always be optimal, as a fixed blending area width cannot always provide the best blending quality for all types of video content. For example, video content often contains strong textures and sharp edges, which requires a narrow blending area to preserve edge information. For camera-captured content, blending is often necessary; however, the blending area width can depend on several factors, such as the actual boundaries of moving objects and the motion separation between the two sections.

[0105] To address this issue, GPM can employ an adaptive blending scheme, dynamically adjusting the width of the blending region around the GPM partition boundaries. For example, the width (θ) of the blending region can be selected from a set of predetermined values ​​{0, 1, 2, 4, 8}. The encoder can determine the optimal blending region width for each GPM CU and signal this optimal width to the decoder based on the syntax element `merge_gpm_blending_width_idx`. For instance, all predefined blending intensity values ​​can be shifted, while all clipping and shifting operations during the GPM blending process remain unchanged.

[0106] Furthermore, the weight range can be increased from [0,8] to [0,32] to accommodate the increase in the width of the GPM mixing region. Specifically, the weights can be calculated as follows:

[0107]

[0108] Wedge-based prediction is a composite prediction mode similar to GPM (e.g., in AV1). Wedge-based prediction can be used for inter-frame / inter-frame and inter-frame / intra-frame combinations. The boundaries of moving objects are often difficult to approximate using block partitioning on a grid. One solution is to predefine a codebook of 16 possible wedge partitions, and when a coding unit chooses to further partition in this way, it signals the wedge index in the bitstream. Figure 5D As shown, 16 shape codebooks containing horizontal, vertical, or inclined (e.g., slope ±2 or ±0.5) partition orientations were designed for square block 540 and rectangular block 542. To reduce pseudo-high-frequency components that are typically generated by directly juxtaposing two predictors, a 2-D wedge mask with a soft cliff shape can be used to smooth the edges around the expected partitions (e.g., m(i,j) is close to 0.5 near the edge and gradually becomes a binary weight at either end).

[0109] Figure 6This is a flowchart illustrating a method 600 for encoding video according to some embodiments. Method 600 can be executed in a computing system (e.g., server system 112, source device 102, or electronic device 120) having a control circuitry and a memory storing instructions for execution by the control circuitry. In some embodiments, method 600 is executed by executing instructions stored in the memory of the computing system (e.g., memory 314).

[0110] The system acquires (602) video data comprising multiple blocks (including a first block). The system identifies (604) a first partitioning pattern for the first block from multiple partitioning patterns, wherein the multiple partitioning patterns include a first group partitioning pattern and a second group partitioning pattern, each partitioning pattern in the first group partitioning pattern having a single-line boundary and each partitioning pattern in the second group partitioning pattern having multiple-line boundaries. The system partitions (606) the first block into a first part and a second part according to the first partitioning pattern, wherein the first partitioning pattern originates from the second group partitioning pattern and includes multiple-line boundaries. The system reconstructs (608) the first block, including reconstructing the first part using a first predictor and reconstructing the second part using a second predictor. In some embodiments, the system performs a blending operation (610) at the multiple-line boundaries.

[0111] In some embodiments, polygonal or L-shaped partitioning patterns (e.g., second-group partitioning patterns) are constructed using existing line-based partitioning patterns (e.g., first-group partitioning patterns). For example, predefined line partition boundaries, blending processes, and prediction processes remain valid. In some embodiments, two or more line-based partitioning patterns are combined to generate new polygonal or L-shaped partitioning patterns, for example, by reusing existing blending masks and corresponding motion vectors. The newly generated pattern can be either explicitly signaled as an additional pattern via indexing or generated on-the-fly at the decoder side without requiring additional signaling of syntax elements (e.g., implicitly exported).

[0112] In some embodiments, multiple (e.g., two or more) GPM-based and / or wedge-based prediction partitioning patterns (e.g., whose angles sum to 180 degrees) are combined to generate polygonal or L-shaped partitions, or more than one geometric partitioning pattern. Note that an L-shaped pattern is generated if one pattern is 0 degrees and another is 90 degrees; otherwise, a polygonal pattern is generated. Figures 5E to 5G Three examples of combined partitioning patterns according to some embodiments are shown. However, the combined patterns are not limited to these examples.

[0113] In one example, such as Figure 5E As shown, two patterns based on conjugate lines are combined horizontally to generate pattern 556. Figure 5EIn this example, the upper part (shaded area 552) of the generated pattern 556 comes from an existing line-based pattern 1, while the lower part (shaded area 554) of the generated pattern 556 comes from an existing line-based pattern 2. In some embodiments, a blending mask and a soft blending region are also generated accordingly. For example, for a particular mask, one part is fully blended (e.g., weight 8 or 64), while another part is zero blended (e.g., weight 0), then the weights of the soft blending regions around the partition boundaries can range from 0 to fully blended (e.g., 0 to 8 or 0 to 64). In this example, another mask is blended in the opposite manner.

[0114] In another example, such as Figure 5F As shown, two patterns based on conjugate lines are vertically combined to generate pattern 564. Figure 5F In this example, the left portion (shaded area 560) of the generated pattern 564 comes from an existing line-based pattern 1, while the right portion (shaded area 562) comes from an existing line-based pattern 2. In some embodiments, a blending mask and a soft blending region are also generated accordingly. For example, for a particular mask, one part is fully blended (e.g., weight 8 or 64), while another part is zero blended (e.g., weight 0), then the weights of the soft blending regions around the partition boundaries range from 0 to fully blended (e.g., 0 to 8 or 0 to 64). In this example, another mask is blended in the opposite manner.

[0115] In another example, such as Figure 5G As shown, one partitioning pattern (e.g., symmetrical or asymmetrical) is combined with another partitioning pattern to generate an L-shaped partitioning pattern. For example, vertical partitioning pattern 1 (e.g., partition 568) is combined with horizontal partitioning pattern 2 (e.g., partition 570). The mixing and prediction processes can be the same as those described in the previous examples.

[0116] In some embodiments, each of the previously described generated patterns (e.g., Figures 5E to 5G (As illustrated in the diagram) Shifts are performed horizontally or vertically, or both, based on shift values. Shift values ​​can be predefined, adaptively selected, and signaled, or derived from the decoder side based on known information (e.g., the content of the reconstructed predictor and / or the GPM of neighboring blocks / wedge-based prediction patterns).

[0117] In some embodiments, the partitioning lines are encoded according to a line-based predictive partitioning pattern (e.g., based on GPM and / or wedges) based on adjacent blocks (e.g., block 578 and block 580), and their partitioning lines (e.g., Figure 5H The partition lines 582 and 584 in the code are extended (e.g.) Figure 5H (Indicated by dashed lines 586 and 588) to the current coding block 576 to form a new polygon partitioning pattern.

[0118] In some embodiments, the generated pattern is used as an extension of the current pattern and is explicitly signaled. For example, an additional 16 patterns can be generated using several selected angles / offsets, which are signaled along with the GPM pattern index or wedge_index; or conditionally signaled using additional syntax elements such as generated_mode_index. In some embodiments, the generated pattern is derived on the decoder side based on known information such as partition pattern information and reconstructed samples.

[0119] For example, if the current partitioning pattern has already been parsed and reconstructed, a template matching method (e.g., using samples from the top and left sides of the current block) is used to calculate the cost between the parsed pattern and the generated pattern. In this example, the pattern with the lower cost is used for the generation of the final predictor.

[0120] In another example, the derivation may depend on the gradients of two predictors used to generate GPM / wedge-based predictors.

[0121] Although Figure 6 Some logical stages are illustrated in a specific order, but stages that are not dependent on the order can be reordered, and other stages can be combined or split. Reorderings or other groupings not specifically mentioned will be obvious to those skilled in the art, and therefore the orderings and groupings presented in this disclosure are not exhaustive. Furthermore, it should be recognized that the various stages can be implemented in hardware, firmware, software, or any combination thereof.

[0122] Now let’s turn to some example implementations.

[0123] (A1) In one aspect, some embodiments include a method for video decoding (e.g., method 600). In some embodiments, the method is performed in a computing system (e.g., server system 112) having memory and control circuitry. In some embodiments, the method is performed on an encoding module (e.g., encoding module 320). In some embodiments, the method is performed on an entropy encoder (e.g., entropy encoder 214). In some embodiments, the method is performed on a parser (e.g., parser 254). The method includes: (i) acquiring video data comprising a plurality of blocks, the plurality of blocks including a first block (e.g., current block 510); (ii) identifying a first partitioning pattern for the first block from a plurality of partitioning patterns, wherein the plurality of partitioning patterns includes a first group of partitioning patterns (e.g., Figure 5D The pattern shown) and the second group partitioning pattern (e.g., Figures 5E to 5G(iii) The first block is partitioned into a first part and a second part according to the first partitioning pattern, wherein the first partitioning pattern is derived from the second partitioning pattern and includes multi-line boundaries; (iv) The first block is reconstructed, including reconstructing the first part using a first predictor and reconstructing the second part using a second predictor. In some embodiments, the first partitioning pattern includes one or more wedge-shaped partitions (e.g., such as...). Figure 5D (As shown).

[0124] (A2) In some embodiments of A1, the second group partitioning pattern includes a pattern generated by combining two or more patterns from the first group partitioning pattern (e.g., such as...). Figures 5E to 5F (The vertical or horizontal combination pattern shown). For example, the pattern in the second group partitioning pattern includes L-shaped partition boundaries or other polygonal boundaries.

[0125] (A3) In some embodiments of A1 or A2, the method further includes performing a blending operation at the boundaries of multiple straight lines (e.g., as previously described with...). Figures 5B to 5C (Related operations). In some embodiments, the blending operation includes applying a blending mask. In some embodiments, the blending operation includes applying a ramp function (e.g., a ramp function of any one of Equations 2 to 4). In some embodiments, a first blending operation is applied to a first portion, and a second blending operation is applied to a second portion.

[0126] (A4) In some embodiments of A3, the mixing operation includes applying a first mixing operation to a first portion and applying a second mixing operation to a second portion. For example, each portion has a different mixing mask, mixing intensity, and / or mixing area.

[0127] (A5) In some embodiments of any of A1 to A4, the second group partitioning pattern includes horizontally combining two patterns based on conjugate lines (e.g., as shown in the figure). Figure 5E The generated pattern (as illustrated in the diagram). For example, horizontally combining two patterns from the first group of partitioned patterns.

[0128] (A6) In some embodiments of any of A1 to A5, the second group partitioning pattern includes a second pattern generated by vertically combining two patterns based on conjugate lines (e.g., as shown in the image). Figure 5F (As illustrated). For example, vertically combining two patterns in the first group partitioning pattern.

[0129] (A7) In some embodiments of any of A1 to A6, the second grouping pattern includes a second pattern generated by combining a vertical line pattern with a horizontal line pattern (e.g., producing a pattern such as...). Figure 5G(The L-shaped partition pattern shown in the diagram).

[0130] (A8) In some embodiments of any of A1 to A7, partitioning a first block into a first part and a second part according to a first partitioning pattern includes: (i) applying a multiline boundary to the first block; (ii) shifting the multiline boundary; and (iii) identifying the first part and the second part based on the shifted multiline boundary. For example, the boundary may be shifted horizontally and / or vertically. In some embodiments, the boundary may be shifted by a shift value. For example, the shift value may be predefined, adaptively selected and signaled, or derived by a decoder (e.g., based on the content of one or more reconstructed predictors).

[0131] (A9) In some embodiments of any of A1 to A8, the method further includes: (i) identifying a second partitioning pattern for a second block among a plurality of blocks, wherein the second partitioning pattern is determined based on partition lines of two or more adjacent blocks; and (ii) partitioning the second block according to the second partitioning pattern. For example, extending the partition lines of two adjacent blocks to their intersection within the second block to create a multi-line partition for the second block (e.g., as shown in the image). Figure 5H (As shown in the diagram).

[0132] (A10) In some embodiments of any of A1 to A9, multiple partition patterns are explicitly signaled by the encoder. In some embodiments, a second group of partition patterns is conditionally signaled (e.g., using additional syntax elements).

[0133] (A11) In some embodiments of any of A1 through A9, the second group partitioning pattern is derived by the decoder component. For example, the second group partitioning pattern is derived using a template matching operation. Alternatively, the second group partitioning pattern is derived based on the top and left adjacent blocks of the first block.

[0134] (A12) In some embodiments of A11, the second group partitioning pattern is derived based on the gradients of two or more predictors.

[0135] (A13) In some embodiments of any of A1 to A12: (i) for one of a plurality of blocks, the parsed pattern is obtained from an encoder component, while the derived pattern is generated at a decoder component; (ii) the method further includes: (a) determining the cost of the parsed pattern and the derived pattern respectively; (b) when it is determined that the parsed pattern has a lower cost, partitioning the block using the parsed pattern; and (c) when it is determined that the derived pattern has a lower cost, partitioning the block using the derived pattern.

[0136] The methods described in this disclosure can be used alone or in any combination in any order. Each of these methods can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In some embodiments, the processing circuitry executes a program stored on a non-volatile computer-readable medium.

[0137] In another aspect, some embodiments include a computing system (e.g., server system 112) that includes a control circuitry system (e.g., control circuitry system 302) and a memory (e.g., memory 314) coupled to the control circuitry system, the memory storing one or more sets of instructions configured to be executed by the control circuitry system, the set of instructions including instructions for performing any of the methods described in this disclosure (e.g., A1 to A13 above).

[0138] In another aspect, some embodiments include a non-volatile computer-readable storage medium storing one or more sets of instructions for execution by a control circuitry of a computing system, the set of instructions including instructions for performing any of the methods described in this disclosure (e.g., A1 to A13 above).

[0139] It should be understood that although the terms "first," "second," etc., may be used in this disclosure to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another.

[0140] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the claims. As used in the description of embodiments and the appended claims, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context explicitly indicates otherwise. It will also be understood that the term “and / or” as used in this disclosure refers to and covers any and all possible combinations of one or more of the associated listed items. It should also be understood that, when used in this specification, the terms “comprises” and / or “comprising” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0141] As used in this disclosure, the term "if" may be interpreted, depending on the context, as meaning "when the stated prerequisite is true" or "once the stated prerequisite is true" or "in response to determining that the stated prerequisite is true" or "based on determining that the stated prerequisite is true" or "in response to detecting that the stated prerequisite is true". Similarly, depending on the context, the phrases "if it is determined [the stated prerequisite is true]" or "if [the stated prerequisite is true]" or "when [the stated prerequisite is true]" may be interpreted as meaning "a determination that the stated prerequisite is true" or "in response to determining that the stated prerequisite is true" or "based on determining that the stated prerequisite is true" or "a detection that the stated prerequisite is true" or "in response to detecting that the stated prerequisite is true".

[0142] For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in light of the foregoing teachings. The embodiments were chosen and described in order to best explain the principles of operation and practical application, thereby enabling others skilled in the art to understand.

Claims

1. A method for video decoding, characterized in that, The method includes: Acquire a video stream comprising multiple blocks, wherein the multiple blocks include a first block; Identify a first partitioning pattern for the first block from a plurality of partitioning patterns, wherein the plurality of partitioning patterns include a first group partitioning pattern and a second group partitioning pattern, each partitioning pattern in the first group partitioning pattern having a corresponding single-line partitioning boundary, and each partitioning pattern in the second group partitioning pattern having a corresponding partitioning boundary composed of multiple lines, the multiple lines being derived from the plurality of partitioning patterns of the first group partitioning pattern; According to a first partitioning pattern, the first block is divided into a first part and a second part, wherein the first partitioning pattern is derived from the second partitioning pattern and has corresponding partition boundaries composed of a first straight line and a second straight line, the first straight line corresponding to the first partitioning pattern in the first partitioning pattern, and the second straight line corresponding to the second partitioning pattern in the first partitioning pattern; and Reconstructing the first block includes reconstructing the first portion using a first predictor and reconstructing the second portion using a second predictor.

2. The method according to claim 1, characterized in that, The second group partitioning pattern includes a partitioning pattern generated by combining two or more partitioning patterns from the first group partitioning pattern.

3. The method according to claim 1, characterized in that, This further includes performing a mixing operation at the corresponding partition boundary.

4. The method according to claim 3, characterized in that, The mixing operation includes applying a first mixing operation to the first part and applying a second mixing operation to the second part.

5. The method according to any one of claims 1 to 4, characterized in that, The second group partitioning pattern includes a partitioning pattern generated by horizontally combining two partitioning patterns based on conjugate lines.

6. The method according to any one of claims 1 to 4, characterized in that, The second group partitioning pattern includes a second partitioning pattern generated by vertically combining two partitioning patterns based on conjugate lines.

7. The method according to any one of claims 1 to 4, characterized in that, The second group partitioning pattern includes a second partitioning pattern generated by combining a vertical line partitioning pattern with a horizontal line partitioning pattern.

8. The method according to any one of claims 1 to 4, characterized in that, According to the first partitioning pattern, the first block is partitioned into the first part and the second part, including: Apply the corresponding partition boundary to the first block; Shift the corresponding partition boundary; and Identify the first part and the second part based on the corresponding partition boundaries after shifting.

9. The method according to any one of claims 1 to 4, characterized in that, Further includes: Identify a second partitioning pattern for a second block among the plurality of blocks, wherein the second partitioning pattern is determined based on partition lines of two or more adjacent blocks; as well as The second block is partitioned according to the second partitioning pattern.

10. The method according to any one of claims 1 to 4, characterized in that, The multiple partition modes are explicitly signaled by the encoder.

11. The method according to any one of claims 1 to 4, characterized in that, The second group partitioning pattern is derived by the decoder component.

12. The method according to claim 11, characterized in that, The second group partitioning pattern is derived based on the gradients of two or more predictors.

13. The method according to any one of claims 1 to 4, characterized in that, For one of the multiple blocks, the parsed partition pattern is obtained from the encoder component, while the derived partition pattern is generated at the decoder component; The method further includes: Determine the cost of the parsed partition pattern and the exported partition pattern respectively; When it is determined that the parsed partitioning pattern has a low cost, the block is partitioned using the parsed partitioning pattern; and When it is determined that the exported partitioning pattern has a low cost, the block is partitioned using the exported partitioning pattern.

14. The method according to claim 1, characterized in that, The multiple partitioning modes include multiple geometric partitioning modes, and the first partitioning mode is a geometric partitioning mode.

15. The method according to claim 1, characterized in that, The plurality of partitioning patterns include a plurality of wedge-based partitioning patterns, wherein the first partitioning pattern is a wedge-based partitioning pattern.

16. A video encoding method, characterized in that, The method includes: Acquire video data comprising multiple blocks, wherein the multiple blocks include a first block; Identify a first partitioning pattern for the first block from a plurality of partitioning patterns, wherein the plurality of partitioning patterns include a first group partitioning pattern and a second group partitioning pattern, each partitioning pattern in the first group partitioning pattern having a corresponding single-line partitioning boundary, and each partitioning pattern in the second group partitioning pattern having a corresponding partitioning boundary composed of multiple lines, the multiple lines being derived from the plurality of partitioning patterns of the first group partitioning pattern; According to a first partitioning pattern, the first block is divided into a first part and a second part, wherein the first partitioning pattern is derived from the second partitioning pattern and has corresponding partition boundaries composed of a first straight line and a second straight line, the first straight line corresponding to the first partitioning pattern in the first partitioning pattern, and the second straight line corresponding to the second partitioning pattern in the first partitioning pattern; and The first part is compressed using a first predictor, and the second part is compressed using a second predictor.

17. An electronic device, characterized in that, It includes a memory for storing computer-readable instructions; and a processor for reading the computer-readable instructions and performing the method as described in any one of claims 1 to 16 as instructed by the computer-readable instructions.

18. A non-volatile computer-readable storage medium, characterized in that, The non-volatile computer-readable storage medium stores one or more sets of instructions, which, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 16.

19. A method for storing or transmitting a bitstream, characterized in that, The video encoding method of claim 16 is used to generate a bitstream; and the bitstream is stored or transmitted.

Images