Method and apparatus for video coding

Abstract

Aspects of the present application include methods, apparatuses, and non-transitory computer-readable storage media of video coding / decoding. An apparatus includes processing circuitry that decodes prediction information of a coding unit in a video bitstream. The processing circuitry determines, based on the prediction information, whether a luma block and a chroma block associated with the coding unit have different partition trees. When the luma block and the chroma block have different partition trees, the processing circuitry determines, based on a first intra block copy (IBC) flag in the prediction information, whether the luma block is coded in an IBC mode. The processing circuitry determines, based on one of a default mode and the first IBC flag, a second IBC flag in the prediction information, whether the chroma block is coded in the IBC mode. The processing circuitry reconstructs the coding unit based on the luma block and the chroma block.

Description

[0001] References merged

[0002] This application claims priority to U.S. Patent Application No. 17 / 464,255, filed September 1, 2021, entitled "Method and Apparatus for Video Coding," which claims priority to U.S. Provisional Application No. 63 / 126,425, filed December 16, 2020, entitled "Harmonized Scheme Between SDP and IntraBC." The entire disclosure of the earlier application is incorporated herein by reference. Technical Field

[0003] This application describes embodiments that generally relate to video encoding and decoding. Background Technology

[0004] The background description provided herein is intended to provide a general overview of the context of this application. The work of the currently named inventors described in this background section, and aspects of the description that may not constitute prior art at the time of filing, whether express or implied, are not acknowledged as prior art to this application.

[0005] Inter-frame prediction techniques with motion compensation can be used for video encoding and decoding. Uncompressed digital video can comprise a series of images, each with a spatial dimension of, for example, 1920×1080 luminance samples and associated chrominance samples. This series of images can have a fixed or variable image rate (also informally referred to as the frame rate), such as 60 frames per second or 60Hz. Uncompressed video has considerable bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luminance sample resolution at 60Hz frame rate) requires nearly 1.5 Gbits per second of bandwidth. One hour of such video would require over 600 GB of storage space.

[0006] One objective of video encoding and decoding is to reduce redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage requirements, in some cases by two orders of magnitude or more. Lossless compression, lossy compression, and combinations thereof can be used. Lossless compression refers to a technique that allows an exact copy of the original signal to be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. In the case of video, lossy compression is widely used. The amount of distortion tolerated depends on the application; for example, users of some consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio reflects this: higher permissible / acceptable distortion results in a higher compression ratio.

[0007] Video encoders and decoders can utilize techniques from several broad categories, including, for example, motion compensation, transform, quantization, and entropy coding.

[0008] Video codec techniques can include techniques called intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from a previously reconstructed reference image. In some video codecs, the image is spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, the image can be an intra-frame image. Intra-frame images and their derived images (e.g., stand-alone decoder refresh images) can be used to reset the decoder state and therefore can be used as the first frame image in an encoded video stream and video session, or as still images. Samples of an intra-frame block can be transformed, and the transform coefficients can be quantized before entropy coding. Intra-frame prediction can be a technique that minimizes the sample values ​​in the pre-transform domain. In some cases, the smaller the transformed DC value and the smaller the AC coefficients, the fewer bits are needed to represent the block after entropy coding at a given quantization step size.

[0009] Existing intra-frame coding and decoding, such as intra-frame coding and decoding techniques known in MPEG-2, do not use intra-frame prediction. However, some newer video compression techniques include techniques that attempt to access data blocks, for example, from surrounding sample data and / or metadata obtained during the encoding and / or decoding of spatially adjacent data blocks, in the order of decoding. Such techniques are hereinafter referred to as "intra-frame prediction" techniques. Note that in at least some cases, intra-frame prediction uses only reference data from the current image in the reconstruction, and not reference data from a reference image.

[0010] There can be many different forms of intra-frame prediction. When more than one such technique can be used in a given video codec, the techniques used can be encoded in an intra-frame prediction mode. In some cases, a mode can have sub-modes and / or parameters, and these sub-modes and / or parameters can be encoded separately or included in the mode codeword. Which codeword is used for a given combination of mode, sub-mode, and / or parameters can affect the coding efficiency gain through intra-frame prediction, and the entropy coding technique used to convert the codeword into a bitstream can also affect the coding efficiency gain.

[0011] A certain mode of intra-frame prediction was introduced with H.264, improved in H.265, and further refined in newer coding and decoding techniques such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Sets (BMS). Predictor blocks can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of neighboring samples are copied into the predictor block in one direction. The direction used can be encoded into the bitstream or predicted.

[0012] Referring to Figure 1A, a subset of nine predictor directions known from the 33 possible predictor directions of H.265 (corresponding to 33 angular modes of 35 intra-frame modes) is depicted in the lower right. The point (101) where the arrows converge represents the sample being predicted. The arrow indicates the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from one or more samples in the upper right at a 45-degree angle to the horizontal. Similarly, arrow (103) indicates that sample (101) is predicted from one or more samples in the lower left of sample (101) at a 22.5-degree angle to the horizontal.

[0013] Referring again to Figure 1A, a square block (104) of 4×4 samples is depicted in the upper left (indicated by a bold dashed line). The square block (104) comprises 16 samples, each labeled with “S”, its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from top) and the first sample in the X dimension (from left). Similarly, sample S44 is the fourth sample in both the Y and X dimensions of block (104). Since the block size is 4×4 samples, S44 is located in the lower right. Reference samples following a similar numbering scheme are further shown. A reference sample is labeled with R, its Y position (e.g., row index) and X position (column index) relative to block (104). In H.264 and H.265, predicted samples are adjacent to the blocks being reconstructed; therefore, negative values ​​are not required.

[0014] Intra-frame prediction can be achieved by copying reference sample values ​​from neighboring samples indicated by the prediction direction, represented by a signal. For example, suppose an encoded video stream includes signaling that, for a block, indicates a prediction direction consistent with arrow (102)—that is, some samples are predicted using one or more prediction samples at a 45-degree angle to the horizontal direction from the upper right. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Sample S44 is then predicted based on reference sample R08.

[0015] In some cases, the values ​​of multiple reference samples can be combined, for example by interpolation, to calculate the reference sample; especially when the directions cannot be uniformly divided at 45 degrees.

[0016] With the development of video coding technology, the number of possible directions has also increased. In H.264 (2003), nine different directions could be represented. In H.265 (2013), this number increased to 33, and JEM / VVC / BMS, when publicly released, could support up to 65 directions. Experiments have been conducted to identify the most probable directions, and certain techniques from entropy coding have been used to represent those possible directions with a small number of bits, accepting some of the cost of less likely directions. Furthermore, sometimes the direction itself can be predicted based on adjacent directions used in adjacent decoded blocks.

[0017] Figure 1B is a schematic diagram (105) of the 65 intra-frame prediction directions of JEM, used to show the number of prediction directions increasing over time.

[0018] Different video codecs can employ different mapping methods for intra-prediction direction bits, which are used to represent direction in the encoded video bitstream. These mapping methods can range from simple direct mappings from prediction direction to intra-prediction modes, to codewords, to complex adaptive schemes involving the most probable mode, and similar techniques. However, in all cases, there may be some directions that are statistically less likely to appear in the video content compared to certain other directions. Since the goal of video compression is to reduce redundancy, in high-performance video codecs, less likely directions will be represented by more bits than more probable directions.

[0019] Motion compensation can be a lossy compression technique and can involve techniques in which sample data blocks from a previously reconstructed image or portion of an image (the reference image) are spatially shifted along a direction indicated by a motion vector (hereafter referred to as MV) to predict a newly reconstructed image or portion of an image. In some cases, the reference image can be the same as the image currently being reconstructed. MV can have two dimensions, X and Y, or three dimensions, with the third dimension indicating the reference image used (the latter indirectly representing the temporal dimension).

[0020] In some video compression techniques, a video feature (MV) applicable to a region of sample data can be predicted from other MVs, such as those MVs related to sample data from another region spatially adjacent to the region being reconstructed and decoded prior to that MV. This substantially reduces the amount of data required to encode the MV, thereby eliminating redundancy and enhancing compression. MV prediction can work effectively, for example, because when encoding the input video signal derived from the camera (called natural video), there is a statistical probability that a larger region than the region applicable to a single MV moves in similar directions, and therefore, in some cases, similar MVs derived from those of neighboring regions can be used for prediction. This results in the MV found for a given region being similar or identical to the MV predicted from the surrounding MVs, and after entropy encoding, it can be represented with fewer bits than would be needed to directly encode that MV. In some cases, MV prediction can be an example of lossless compression of the signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself may be lossy, for example, due to rounding errors when calculating predictions from several surrounding MVs.

[0021] H.265 / HEVC (ITU-T H.265 Recommendation, “High Efficiency Video Coding”, December 2016) describes various MV prediction mechanisms. Among the various MV prediction mechanisms provided by H.265, this application describes the technique hereinafter referred to as “spatial combining”.

[0022] Referring to Figure 1, the current block (101) includes samples discovered by the encoder during the motion search process, which can be predicted based on previous blocks of the same size that have generated spatial offsets. Alternatively, the MV can be derived from metadata associated with one or more reference images, rather than being directly encoded. For example, using the MV associated with any of the five surrounding samples A0, A1 and B0, B1, B2 (corresponding to 102 to 106 respectively), the MV is derived from the metadata of the nearest reference image (in decoding order). In H.265, MV prediction can use predictions from the same reference image that is also being used in adjacent blocks. Summary of the Invention

[0023] This application provides apparatus for video encoding / decoding. One apparatus includes processing circuitry that decodes prediction information of a coding unit in a current image, which is part of a video bitstream. The processing circuitry determines, based on the prediction information, whether the luma block and chroma block associated with the coding unit have different segmentation trees. When the luma block and chroma block associated with the coding unit have different segmentation trees, the processing circuitry determines whether the luma block is encoded in IBC mode based on a first intra-block copy (IBC) flag in the prediction information. The processing circuitry determines whether the chroma block is encoded in IBC mode based on a default mode, a second IBC flag in the prediction information, and one of the first IBC flags. The processing circuitry reconstructs the coding unit based on the luma block and the chroma block.

[0024] In one embodiment, in response to the luma and chroma blocks associated with the encoding unit having the same segmentation tree, the processing circuit determines whether the luma and chroma blocks are encoded in IBC mode based on a first IBC flag in the prediction information. The processing circuit determines that the luma and chroma blocks have the same block vector based on whether they are encoded in IBC mode.

[0025] In one embodiment, the processing circuitry determines that the chroma block is not encoded in IBC mode based on a default mode that indicates that IBC mode is disabled for chroma blocks that have a different segmentation tree than the luma block.

[0026] In one embodiment, the processing circuitry determines whether the luma block and the chroma block have the same segmentation size. In response to the luma block and the chroma block having the same segmentation size, the processing circuitry determines whether the chroma block is encoded in IBC mode based on a first IBC flag.

[0027] In one embodiment, the luma block size is larger than the chroma block size, and the processing circuitry determines whether the luma and chroma blocks are encoded in IBC mode based on a first IBC flag. In response to the luma and chroma blocks being encoded in IBC mode, the processing circuitry determines that the luma and chroma blocks have the same block vector.

[0028] In one embodiment, if a first subset of chroma block samples is co-located with a first luma block encoded in IBC mode, and a second subset of chroma block samples is co-located with a second luma block encoded in a first intra-prediction mode, then the processing circuit determines that the first subset of chroma block samples is encoded in IBC mode. The processing circuit then determines that the second subset of chroma block samples is encoded in one of the first and second intra-prediction modes in the prediction information.

[0029] In one embodiment, a first subset of the chroma block samples and a first luminance block may have the same block vector.

[0030] This application provides methods for video encoding / decoding. In this method, prediction information of coding units in a current image, which is part of a video bitstream, is decoded. Based on the prediction information, it is determined whether the luma block and chroma block associated with the coding unit have different segmentation trees. When the luma block and chroma block associated with the coding unit have different segmentation trees, it is determined whether the luma block is encoded in Intra-Block Copy (IBC) mode based on a first IBC flag in the prediction information. Based on a default mode, a second IBC flag in the prediction information, and one of the first IBC flags, it is determined whether the chroma block is encoded in IBC mode. The coding unit is reconstructed based on the luma block and chroma block.

[0031] Various aspects of this application also provide a non-volatile computer-readable medium storing instructions that are executed by at least one processor to implement any one or a combination of methods of video decoding. Attached Figure Description

[0032] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0033] Figure 1A is a schematic diagram of an exemplary subset of intra-prediction modes.

[0034] Figure 1B is an illustration of an exemplary intra-frame prediction direction.

[0035] Figure 1C is a schematic diagram of the current block and its surrounding space merge candidates in an example.

[0036] Figure 2 This is a simplified block diagram of a communication system according to an embodiment;

[0037] Figure 3 This is a simplified block diagram of a communication system according to an embodiment;

[0038] Figure 4 This is a simplified block diagram of the decoder according to an embodiment;

[0039] Figure 5 This is a simplified block diagram of the encoder according to an embodiment;

[0040] Figure 6 A block diagram of an encoder according to another embodiment is shown;

[0041] Figure 7 A block diagram of a decoder according to another embodiment is shown;

[0042] Figure 8 Exemplary block segmentation according to some embodiments of this application is shown;

[0043] Figure 9 Exemplary block segmentation according to some embodiments of this application is shown;

[0044] Figure 10 Exemplary block segmentation according to some embodiments of this application is shown;

[0045] Figure 11 An exemplary quadtree with a nested multi-type tree coding block structure is shown according to an embodiment of this application;

[0046] Figure 12 An exemplary block segmentation with a semi-decoupled tree scheme according to an embodiment of this application is shown;

[0047] Figure 13 An exemplary L-shaped (or L-shaped) partition is shown according to an embodiment of this application;

[0048] Figure 14 Four examples of L-shaped segmentation according to some embodiments of this application are shown;

[0049] Figure 15 An exemplary flowchart according to an embodiment is shown; and

[0050] Figure 16 This is a schematic diagram of a computer system according to an embodiment. Detailed Implementation

[0051] I. Video Decoder and Encoder Systems

[0052] Figure 2 This is a simplified block diagram of a communication system (200) according to an embodiment disclosed in this application. The communication system (200) includes a plurality of terminal devices that can communicate with each other via, for example, a network (250). For example, the communication system (200) includes a first terminal device (210) and a second terminal device (220) interconnected via a network (250). Figure 2 In this embodiment, the first terminal device (210) and the second terminal device (220) perform unidirectional data transmission. For example, the first terminal device (210) may encode video data (e.g., a video image stream captured by the terminal device (210)) for transmission over a network (250) to the second terminal device (220). The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device (220) may receive the encoded video data from the network (250), decode the encoded video data to recover the video data, and display video images based on the recovered video data. Unidirectional data transmission is common in applications such as media services.

[0053] In another embodiment, the communication system (200) includes a third terminal device (230) and a fourth terminal device (240) that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For bidirectional data transmission, each of the third terminal device (230) and the fourth terminal device (240) may encode video data (e.g., a video image stream captured by the terminal device) for transmission over a network (250) to the other terminal device. Each of the third terminal device (230) and the fourth terminal device (240) may also receive encoded video data transmitted by the other terminal device and may decode the encoded video data to recover the video data, and may display the video image on an accessible display device based on the recovered video data.

[0054] exist Figure 2 In the embodiments disclosed herein, the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240) may be servers, personal computers, and smartphones, but the principles disclosed herein are not limited thereto. The embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (250) refers to any number of networks that transmit encoded video data between the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240), including, for example, wired (connected) and / or wireless communication networks. The communication network (250) may exchange data in circuit-switched and / or packet-switched channels. The network may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of the network (250) may be irrelevant to the operation of this application.

[0055] As an example, Figure 3 The diagram illustrates the placement of a video encoder and a video decoder in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0056] The streaming system may include an acquisition subsystem (313) that may include a video source (301) such as a digital camera, which creates an uncompressed video image stream (302). In an embodiment, the video image stream (302) includes samples captured by a digital camera. The video image stream (302) is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data (304) (or encoded video bitstream). The video image stream (302) may be processed by an electronic device (320) that includes a video encoder (303) coupled to the video source (301). The video encoder (303) may include hardware, software, or a combination of hardware and software to implement or carry out aspects of the disclosed subject matter as described in more detail below. Compared to the video image stream (302), the encoded video data (304) (or the encoded video bitstream (304)) is depicted as a thin line to emphasize the lower data volume of the encoded video data (304) (or the encoded video bitstream (304)), which can be stored on a streaming server (305) for future use. One or more streaming client subsystems, such as Figure 3 Client subsystems (306) and (308) can access a streaming server (305) to retrieve copies (307) and (309) of encoded video data (304). Client subsystem (306) may include, for example, a video decoder (310) in an electronic device (330). The video decoder (310) decodes the incoming copy (307) of the encoded video data and produces an output video image stream (311) that can be displayed on a display (312) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (304), video data (307), and video data (309) (e.g., video streams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T H.265. In embodiments, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and this application can be used in the context of the VVC standard.

[0057] It should be noted that the electronic devices (320) and (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown), and the electronic device (330) may also include a video encoder (not shown).

[0058] Figure 4This is a block diagram of a video decoder (410) according to an embodiment disclosed in this application. The video decoder (410) may be disposed in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., receiving circuitry). The video decoder (410) may be used in place of... Figure 3 The video decoder (310) in the embodiment.

[0059] The receiver (431) may receive one or more encoded video sequences to be decoded by the video decoder (410); in the same embodiment or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences may be received from a channel (401), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (431) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective user entities (not indicated). The receiver (431) may separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (415) may be coupled between the receiver (431) and the entropy decoder / parser (420) (hereinafter referred to as "parser (420)"). In some applications, the buffer memory (415) is part of the video decoder (410). In other cases, the buffer memory (415) may be located external to the video decoder (410) (not indicated). In other cases, an external buffer (not shown) may be provided for the video decoder (410) to prevent network jitter, for example, and another buffer (415) may be configured internally for, for example, handling broadcast timing. When the receiver (431) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, the buffer (415) may not be necessary, or it may be made smaller. Of course, for use on packet networks such as the Internet, a buffer (415) may be required; this buffer may be relatively large and adaptive in size, and may be at least partially implemented in the operating system or a similar component (not shown) external to the video decoder (410).

[0060] The video decoder (410) may include a parser (420) to reconstruct symbols (421) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (410) and potential information for controlling a display device (412) (e.g., a display screen), which is not part of the electronic device (430) but may be coupled to it, such as... Figure 4As shown in the figure. The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (420) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (420) may extract a subgroup parameter set of at least one subgroup of pixels in the subgroup of pixels in the encoded video sequence for use in the video decoder based on at least one parameter corresponding to a group. The subgroup may include a Group of Pictures (GOP), image, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (420) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0061] The parser (420) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415) to create symbols (421).

[0062] Depending on the type of encoded video image or a portion thereof (e.g., inter-frame and intra-frame images, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (421) may involve multiple different units. Which units are involved and how they are involved can be controlled by the subgroup control information parsed by the parser (420) from the encoded video sequence. For brevity, the flow of such subgroup control information between the parser (420) and the various units described below is not described.

[0063] In addition to the functional blocks already mentioned, the video decoder (410) can be conceptually subdivided into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and can be integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0064] The first unit is the scaler / inverse transform unit (451). The scaler / inverse transform unit (451) receives quantization transform coefficients as symbols (421) and control information from the parser (420), including the transform method used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (451) can output a block containing sample values, which can be input into the aggregator (455).

[0065] In some cases, the output samples of the scaler / inverse transform unit (451) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-image prediction unit (452). In some cases, the intra-image prediction unit (452) uses reconstructed information extracted from the current image buffer (458) to generate surrounding blocks of the same size and shape as the block being reconstructed. For example, the current image buffer (458) buffers partially reconstructed and / or fully reconstructed current images. In some cases, the aggregator (455) adds the predictive information generated by the intra-prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451) based on each sample.

[0066] In other cases, the output samples of the scaler / inverse transform unit (451) may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit (453) can access the reference image memory (457) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (421), these samples can be added by the aggregator (455) to the output of the scaler / inverse transform unit (451) (referred to in this case as residual samples or residual signals) to generate output sample information. The motion compensation prediction unit (453) can obtain the prediction samples from the address in the reference image memory (457) under motion vector control, and the motion vector is available to the motion compensation prediction unit (453) in the form of the symbols (421), which, for example, include X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (457) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0067] The output samples of the aggregator (455) can be employed by various loop filtering techniques in the loop filter unit (456). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video stream), and these parameters can be used as symbols (421) from the parser (420) in the loop filter unit (456). However, in other embodiments, the video compression techniques may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded image or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0068] The output of the loop filter unit (456) can be a sample stream, which can be output to a display device (412) and stored in a reference image memory (457) for subsequent inter-frame image prediction.

[0069] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded image corresponding to the current image has been fully reconstructed and the encoded image (by, for example, the parser (420)) is identified as the reference image, the current image buffer (458) can become part of the reference image memory (457), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0070] The video decoder (410) can perform decoding operations according to a predetermined video compression technique, such as that specified in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the configuration file recorded in the video compression technique or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technique or standard as the only tools available under said configuration file. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further defined by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0071] In this embodiment, the receiver (431) may receive additional (redundant) data along with the encoded video. The additional data may be a portion of the encoded video sequence. The additional data may be used by the video decoder (410) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0072] Figure 5 This is a block diagram of a video encoder (503) according to an embodiment disclosed in this application. The video encoder (503) is disposed in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., a transmission circuit). The video encoder (503) can be used to replace... Figure 3 The video encoder (303) in the embodiment.

[0073] The video encoder (503) can obtain data from the video source (501) (not) Figure 5 In one embodiment, a portion of the electronic device (520) receives video samples, the video source being capable of capturing video images to be encoded by a video encoder (503). In another embodiment, the video source (501) is a portion of the electronic device (520).

[0074] A video source (501) can provide a sequence of source video samples encoded by a video encoder (503) in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (501) can be a storage device storing previously prepared video. In a video conferencing system, the video source (501) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual images, which are given motion when viewed sequentially. The images themselves can be constructed as spatial pixel arrays, where each pixel can include one or more samples depending on the sampling structure, color space, etc., used. Those skilled in the art can easily understand the relationship between pixels and samples. The following focuses on describing samples.

[0075] According to an embodiment, the video encoder (503) can encode and compress images of a source video sequence into an encoded video sequence (543) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (550). In some embodiments, the controller (550) controls and is functionally coupled to other functional units described below. For simplicity, coupling is not shown in the figures. Parameters set by the controller (550) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, reference area allowed for maximum motion vector, etc. The controller (550) may be used with other suitable functions related to the video encoder (503) optimized for a particular system design.

[0076] In some embodiments, the video encoder (503) operates within an encoding loop. As a simplified description, in an embodiment, the encoding loop may include a source encoder (530) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (533) embedded within the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression techniques considered in this application, any compression between the symbols and the encoded video stream is lossless). The reconstructed sample stream (sample data) is input to a reference image memory (534). Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of the reference image memory (534) also correspond bit-accurately between the local encoder and the remote encoder. In other words, the reference image samples "seen" by the encoder's prediction portion are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related techniques.

[0077] The operation of the “local” decoder (533) can be combined with, for example, the above-described method. Figure 4 The video decoder (410) is described in detail as the same as the "remote" decoder. However, a further brief reference is provided. Figure 4 When symbols are available and the entropy encoder (545) and parser (420) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (410), including the buffer (415) and parser (420), may not be fully implemented in the local decoder (533).

[0078] It can be observed that any decoder technique other than parsing / entropy decoding present in the decoder must also exist in the corresponding encoder in essentially the same functional form. For this reason, this application focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are inverses of the fully described decoder techniques. More detailed descriptions are only required in certain areas, and are provided below.

[0079] During operation, in some embodiments, the source encoder (530) may perform motion-compensated predictive coding. The motion-compensated predictive coding predictively encodes the input image with reference to one or more previously encoded images from the video sequence designated as "reference images." In this manner, the encoding engine (532) encodes the differences between pixel blocks of the input image and pixel blocks of the reference image, which may be selected as a predictive reference for the input image.

[0080] The local video decoder (533) can decode encoded video data of an image that can be designated as a reference image, based on symbols created by the source encoder (530). The operation of the encoding engine (532) can be a lossy process. When the encoded video data can be decoded by the video decoder (533), Figure 5 When the source video sequence (not shown) is decoded, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (533) replicates the decoding process, which can be performed by the video decoder on the reference image, and allows the reconstructed reference image to be stored in a reference image cache (534). In this way, the video encoder (503) can locally store a copy of the reconstructed reference image that shares common content (no transmission errors) with the reconstructed reference image to be obtained by the remote video decoder.

[0081] The predictor (535) can perform a prediction search against the encoding engine (532). That is, for a new image to be encoded, the predictor (535) can search in the reference image memory (534) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. The predictor (535) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by the predictor (535), it can be determined that the input image may have prediction references obtained from multiple reference images stored in the reference image memory (534).

[0082] The controller (550) can manage the encoding operations of the source encoder (530), including, for example, setting parameters and subgroup parameters for encoding video data.

[0083] The outputs of all the above-mentioned functional units can be entropy encoded in the entropy encoder (545). The entropy encoder (545) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, and arithmetic coding, thereby converting the symbols into an encoded video sequence.

[0084] The transmitter (540) can buffer the encoded video sequence created by the entropy encoder (545) in preparation for transmission via a communication channel (560), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (540) can combine the encoded video data from the video encoder (503) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0085] The controller (550) manages the operation of the video encoder (503). During encoding, the controller (550) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following image types:

[0086] An intra-frame image (I-image) is an image that can be encoded and decoded without using any other images in the sequence as a prediction source. Some video codecs allow different types of intra-frame images, including, for example, Independent Decoder Refresh (IDR) images. Those skilled in the art will understand variations of I-images and their corresponding applications and characteristics.

[0087] A predictive image (P-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block.

[0088] A bidirectional predictive image (B-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses up to two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0089] The source image is typically spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, determined based on the coding assignment of the corresponding images applied to the blocks. For example, blocks of an I-image can be non-predictively coded, or the blocks can be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be predictively coded with reference to a previously coded reference image via spatial prediction or temporal prediction. Blocks of a B-image can be predictively coded with reference to one or two previously coded reference images via spatial prediction or temporal prediction.

[0090] The video encoder (503) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (503) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0091] In this embodiment, the transmitter (540) may transmit additional data while transmitting encoded video. The source encoder (530) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0092] The acquired video can serve as multiple source images (video images) presented in a time series. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In an embodiment, a specific image being encoded / decoded is segmented into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector may have a third dimension that identifies the reference image.

[0093] In some embodiments, bidirectional prediction techniques can be used in inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly past and future in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. Specifically, the block can be predicted using a combination of the first and second reference blocks.

[0094] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0095] According to some embodiments disclosed in this application, predictions such as inter-frame image prediction and intra-frame image prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video image sequence are segmented into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Furthermore, each CTU can be further subdivided into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be subdivided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In embodiments, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Furthermore, depending on temporal and / or spatial predictability, the CU is divided into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In embodiments, prediction operations in encoding (encoding / decoding) are performed on a per-prediction-block basis. Taking a luma prediction block as an example, a prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0096] Figure 6 This is a diagram of a video encoder (603) according to another embodiment disclosed in this application. The video encoder (603) is used to receive processing blocks (e.g., prediction blocks) of sample values ​​within a current video image in a video image sequence, and to encode said processing blocks into an encoded image that is part of an encoded video sequence. In this embodiment, the video encoder (603) is used instead of Figure 3The video encoder (303) in the embodiment.

[0097] In the HEVC embodiment, the video encoder (603) receives a matrix of sample values ​​for a processing block, such as an 8×8 sample prediction block. The video encoder (603) uses, for example, rate-distortion (RD) optimization to determine whether to use an intra-frame mode, an inter-frame mode, or a bidirectional prediction mode to encode the processing block. When encoding the processing block in intra-frame mode, the video encoder (603) can use intra-frame prediction techniques to encode the processing block into the encoded image; and when encoding the processing block in inter-frame mode or bidirectional prediction mode, the video encoder (603) can use inter-frame prediction or bidirectional prediction techniques to encode the processing block into the encoded image, respectively. In some video coding techniques, the merging mode can be an inter-frame image prediction sub-mode, in which motion vectors are derived from one or more motion vector prediction values ​​without relying on encoded motion vector components outside the prediction values. In some other video coding techniques, motion vector components applicable to the subject block may exist. In the embodiment, the video encoder (603) includes other components, such as a mode decision module (not shown) for determining the processing block mode.

[0098] exist Figure 6 In one embodiment, the video encoder (603) includes, as shown below: Figure 6 The inter-frame encoder (630), intra-frame encoder (622), residual calculator (623), switch (626), residual encoder (624), general controller (621) and entropy encoder (625) are shown coupled together.

[0099] An inter-frame encoder (630) is configured to receive samples of the current block (e.g., the processing block), compare the block with one or more reference blocks in a reference image (e.g., blocks in a previous image and a later image), generate inter-frame prediction information (e.g., a description of redundancy information based on inter-frame coding techniques, motion vectors, merging mode information), and calculate inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some embodiments, the reference image is a decoded reference image based on encoded video information.

[0100] The intra encoder (622) is used to receive samples of the current block (e.g., the processing block), in some cases compare the block with encoded blocks in the same image, generate quantization coefficients after transformation, and in some cases also generate intra prediction information (e.g., based on intra prediction direction information of one or more intra coding techniques). In an embodiment, the intra encoder (622) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and reference blocks in the same image.

[0101] A general-purpose controller (621) determines general-purpose control data and controls other components of the video encoder (603) based on the general-purpose control data. In an embodiment, the general-purpose controller (621) determines the mode of a block and provides control signals to a switch (626) based on the mode. For example, when the mode is an intra-frame mode, the general-purpose controller (621) controls the switch (626) to select an intra-frame mode result for use by the residual calculator (623) and controls the entropy encoder (625) to select intra-frame prediction information and add the intra-frame prediction information to the bitstream; and when the mode is an inter-frame mode, the general-purpose controller (621) controls the switch (626) to select an inter-frame prediction result for use by the residual calculator (623) and controls the entropy encoder (625) to select inter-frame prediction information and add the inter-frame prediction information to the bitstream.

[0102] A residual calculator (623) is used to calculate the difference (residual data) between the received block and the prediction result selected from the intra encoder (622) or the inter encoder (630). A residual encoder (624) is used to operate on the residual data to encode the residual data to generate transform coefficients. In an embodiment, the residual encoder (624) is used to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is used to perform an inverse transform and generate decoded residual data. The decoded residual data can be used appropriately by the intra encoder (622) and the inter encoder (630). For example, the inter encoder (630) can generate a decoded block based on the decoded residual data and inter-frame prediction information, and the intra encoder (622) can generate a decoded block based on the decoded residual data and intra-frame prediction information. The decoded blocks are processed appropriately to generate a decoded image, and in some embodiments, the decoded image may be buffered in a memory circuit (not shown) and used as a reference image.

[0103] An entropy encoder (625) is used to format the bitstream to produce encoded blocks. The entropy encoder (625) generates various information according to a suitable standard such as the HEVC standard. In an embodiment, the entropy encoder (625) is used to obtain general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. It should be noted that, according to the disclosed subject matter, residual information is not present when blocks are encoded in a merged sub-mode of inter-frame mode or bidirectional prediction mode.

[0104] Figure 7This is a diagram of a video decoder (710) according to another embodiment disclosed in this application. The video decoder (710) is used to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed image. In an embodiment, the video decoder (710) is used instead of Figure 3 The video decoder (310) in the embodiment.

[0105] exist Figure 7 In this embodiment, the video decoder (710) includes, as follows: Figure 7 The entropy decoder (771), inter-frame decoder (780), residual decoder (773), reconstruction module (774), and intra-frame decoder (772) are shown coupled together.

[0106] An entropy decoder (771) can be used to reconstruct certain symbols from an encoded image, these symbols representing the syntax elements constituting the encoded image. Such symbols may include, for example, a mode for encoding the block (e.g., intra-frame mode, inter-frame mode, bidirectional prediction mode, a merged sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra-frame prediction information or inter-frame prediction information) that can respectively identify certain samples or metadata used by the intra-frame decoder (772) or the inter-frame decoder (780) for prediction, residual information in the form of, for example, quantized transform coefficients, and so on. In an embodiment, when the prediction mode is inter-frame or bidirectional prediction mode, inter-frame prediction information is provided to the inter-frame decoder (780); and when the prediction type is intra-frame prediction type, intra-frame prediction information is provided to the intra-frame decoder (772). Residual information may be provided to the residual decoder (773) via inverse quantization.

[0107] The inter-frame decoder (780) is used to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0108] The intra-frame decoder (772) is used to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0109] The residual decoder (773) performs inverse quantization to extract the dequantized transform coefficients and processes the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require some control information (to obtain the quantizer parameters QP), and this information may be provided by the entropy decoder (771) (the data path is not indicated because this is only low-level control information).

[0110] The reconstruction module (774) is used to combine the residual output by the residual decoder (773) with the prediction result (which may be output by the inter-frame prediction module or the intra-frame prediction module) in the spatial domain to form a reconstructed block, which may be a part of a reconstructed image, which in turn may be a part of a reconstructed video. It should be noted that other suitable operations, such as deblocking, may be performed to improve visual quality.

[0111] It should be noted that any suitable technology can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In one embodiment, one or more integrated circuits can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In another embodiment, one or more processors executing software instructions can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710).

[0112] II. Block partitioning

[0113] Figure 8 Exemplary block segmentation of some embodiments of this application is illustrated. In one embodiment, Figure 8 The exemplary block segmentation in the example can be applied to VP9 proposed by the Alliance for Open Media (AOMedia). For example... Figure 8 As shown, a 4-way split tree can be used, starting from a 64×64 level and descending to a 4×4 level, with some additional restrictions on 8×8 blocks. It should be noted that a split designated as R can be called a recursive split. That is, the same split tree can be repeated at a lower scale until the lowest 4×4 level is reached.

[0114] Figure 9 Exemplary block segmentation of some embodiments of this application is illustrated. In one embodiment, Figure 9 The exemplary block segmentation in [the example] can be used in AV1 proposed by AOMedia. For example... Figure 9 As shown, the split tree can be expanded to a 10-way structure, and the maximum coded block size (called the superblock in VP9 / AV1 syntax) can be increased to start at 128×128. It should be noted that... Figure 9 The 4:1 / 1:4 rectangular division in the first row does not exist in VP9. Figure 9A segmentation type with three sub-segments in the second row is called a T-segment. Rectangular segments cannot be further subdivided. In addition to the coded block size, a coding tree depth is defined to indicate the segmentation depth from the root node. In one embodiment, the coding tree depth of the root node (e.g., 128×128) can be set to 0. The coding tree depth increases by 1 after each further segmentation of the coded block.

[0115] Instead of being bound by the fixed transform unit size of VP9, ​​AV1 allows luma coding blocks to be divided into transform units of multiple sizes, which can be represented by recursive partitioning down to a maximum of two levels. To incorporate extended coding block partitions into AV1, square transform sizes ranging from 4×4 to 64×64 with ratios of 2:1 / 1:2 and 4:1 / 1:4 are supported. For chroma coding blocks, only the largest possible transform unit is allowed.

[0116] In some relevant examples, such as HEVC, a quadtree structure, represented as a coding tree, can be used to partition a CTU into CUs to accommodate various local characteristics. Whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to encode image regions can be decided at the CU level. Each CU can be further partitioned into one, two, or four PUs based on the PU partitioning type. Within a PU, the same prediction process can be used, and relevant information can be transferred to the decoder on a PU-by-PU basis. After obtaining residual blocks using a prediction process based on the PU partitioning type, the CUs can be partitioned into TUs based on another quadtree structure (such as the coding tree of the CU). A key feature of the HEVC structure is that it has multiple partitioning concepts including CUs, PUs, and TUs. In HEVC, a CU or TU can be simply a square, while a PU, as an inter-frame prediction block, can be either a square or a rectangle. In HEVC, a coding block can be further partitioned into four square sub-blocks, and a transformation process can be performed on each sub-block (i.e., TU). Each TU can be further recursively partitioned into smaller TUs (e.g., using quadtree partitioning). Quadtree partitioning can be called residual quadtree (RQT).

[0117] At the image boundaries, HEVC uses implicit quadtree segmentation, allowing a block to continue undergoing quadtree segmentation until the block size fits the image boundaries.

[0118] In some relevant examples, such as VVC, a quadtree with embedded multi-type trees using binary and ternary segmentation structures can replace the concept of multiple segmentation unit types. That is, except for cases where the CU has a size too large for the maximum transform length, the concepts of CU, PU, ​​and TU are no longer distinguished. Therefore, more flexibility in the shape of CU segmentation can be supported in these examples. In the VVC coding tree structure, the CU can be square or rectangular. The CTU can first be segmented by a quadtree (or quad-tree) structure. Then, the leaf nodes of the quadtree can be further segmented using a multi-type tree structure.

[0119] Figure 10 Exemplary block segmentation methods for multi-type tree segmentation patterns are illustrated in some embodiments of this application. In one embodiment, Figure 10 The example block splitting shown can be used in VVC. For example... Figure 10 As shown, there are four types of partitioning in the multi-type tree structure: vertical binary partitioning (SPLIT_BT_VER), horizontal binary partitioning (SPLIT_BT_HOR), vertical ternary partitioning (SPLIT_TT_VER), and horizontal ternary partitioning (SPLIT_TT_HOR). The leaf nodes of the multi-type tree are called CUs. Unless the CU is too large for the maximum transform length, the multi-type tree structure can be used for both prediction and transform processes without any further partitioning. This means that in most cases, the CU, PU, ​​and TU have the same block size in the quadtree embedded in the multi-type tree coding block structure. An exception occurs when the maximum supported transform length is less than the width or height of the color component of the CU.

[0120] Figure 11 An exemplary quadtree with embedded multi-type tree coding block structures is shown as an embodiment of this application.

[0121] In some relevant examples, such as VVC, the maximum supported luminance transform size is 64×64, and the maximum supported chrominance transform size is 32×32. When the width or height of the CB is greater than the maximum transform width or height, the CB can be automatically segmented along the horizontal and / or vertical directions to meet the transform size limit in that direction.

[0122] In some relevant examples, such as VTM7, the coding tree scheme can support independent block tree structures for the luma CTB and chroma CTB within a single CTU. For instance, for P-slices and B-slices, the luma CTB and chroma CTB within a single CTU share the same coding tree structure. However, for I-slices, the luma CTB and chroma CTB within a single CTU can have independent block tree structures. When using the independent block tree mode, the luma CTB is segmented into CUs by one coding tree structure, while the chroma CTB is segmented into chroma CUs by another coding tree structure. This means that a CU in an I-slice can include a coding block for the luma component or coding blocks for both chroma components, and unless the video is monochrome, a CU in a P-slice or B-slice can always include coding blocks for all three color components.

[0123] In some relevant examples, a semi-decoupled tree (SDT) scheme is employed, also known as a semi-separate tree (SST) or flexible block partitioning for chroma components. In an SDT, luma blocks and chroma blocks within a superblock (SB) can have the same or different block partitioning schemes, depending on the luma block size or luma tree depth. For example, when the luma block size is greater than threshold T1 or the luma block's coding tree split depth is less than or equal to threshold T2, the chroma block associated with the luma block can use the same coding tree structure as the luma block. Otherwise, when the luma block size is less than or equal to T1 or the luma block's luma split depth is greater than T2, the associated chroma block can have a different coding block partitioning scheme than the luma block. Therefore, this scheme is called flexible block partitioning for chroma components. T1 is a positive integer, such as 128 or 256. T2 is a positive integer, such as 1 or 2. Figure 12 An example of block tree splitting is shown, where T2 is set to 1.

[0124] In some relevant examples, an improved semi-decoupled partitioning (SDP) scheme is used, in which the luma block and chroma block can share a partial tree structure starting from the root node of the superblock, and the conditions under which the luma block and chroma block start a separate tree partitioning scheme depend on the partitioning information of the luma block or the high-level syntax in the bitstream.

[0125] In some relevant examples, an L-shaped block partitioning tree scheme is used. Instead of using a rectangular block partitioning scheme, an L-shaped partitioning method is used to divide a block into one or more L-shaped partitions and one or more rectangular partitions.

[0126] Figure 13An exemplary L-shaped (or L-shaped) partition is shown. A rotated L-shaped partition is also considered an L-shaped partition in this application. Several terms related to an L-shaped partition include width, height, shorter width, and shorter height, such as... Figure 13 As shown in the image.

[0127] Figure 14 Four examples of L-shaped segmentation trees according to some embodiments of this application are shown. A block can be divided into two segmentation regions, including an L-shaped segmentation region (segmentation region 1) and a rectangular segmentation region (segmentation region 0).

[0128] III. Intra-block copying

[0129] Intra-block copy (IntraBC or IBC) is a coding tool similar to inter-frame prediction. The main difference is that in IntraBC, the predictor block is formed using reconstructed samples from the current image (e.g., before applying in-loop filters). Therefore, IntraBC can be thought of as "motion compensation" within the current image.

[0130] A block vector (BV) can be encoded to indicate the location of a predictor block. The precision of the BV can be integers. The BV can be represented as a signal in the bitstream to locate the predictor. For the current block, a flag (e.g., an IBC flag) indicating whether the current block is encoded in IntraBC mode is first transmitted in the bitstream. Then, if the current block is encoded in IntraBC mode, the BV difference (diff) can be obtained by subtracting a reference BV from the current BV, and the diff can be classified into one of four types based on the diff values ​​of the horizontal and vertical components. Type information can be transmitted into the bitstream. The diff values ​​of the two components can then be represented as a signal based on the type information.

[0131] In some relevant examples, such as screen content encoding, IntraBC is an effective tool. However, IntraBC can also pose challenges for hardware design. To facilitate hardware design, the following improvements can be adopted.

[0132] (1) When IntraBC is enabled, loop filters, such as deblocking filters, constrained directional enhancement filters (CDEF), and loop restoration filters, are disabled. Accordingly, IntraBC and inter-frame prediction can share the image buffer of reconstructed samples.

[0133] (2) To facilitate parallel decoding, prediction cannot exceed the constrained region. For a super block, if the coordinates of its top-left position are (x0, y0), then if y<y0 and x<x0+2*(y0-y), the prediction at position (x, y) can be read by IntraBC.

[0134] (3) To allow for hardware write-back latency, the currently reconstructed region cannot be accessed through IntraBC prediction. The constrained currently reconstructed region may be in the range of 1 to n super blocks. Therefore, based on the modification of (2), if the coordinates of the top-left position of a super block are (x0, y0), then if y<y0 and x<x0+2*(y0-y)-D, the prediction at position (x, y) can be read by IntraBC, where D represents the constrained currently reconstructed region.

[0135] IV. Improved IBC Prediction

[0136] In some related examples, such as in AV1, luma blocks and chroma blocks can share the same split tree, and the luma block and the corresponding chroma block can also share the same IntraBC flag, which means that the IntraBC mode can be enabled for both luma and chroma blocks, or disabled for both luma and chroma blocks. However, in some cases, such as in SDP, luma blocks and chroma blocks can share the same split tree or have different split trees. Therefore, in some cases such as SDP, it may not be the best practice that luma blocks and chroma blocks always share the IntraBC flag.

[0137] In this application, block size may refer to various dimensional attributes of a block, such as the block width, block height, the maximum value of width and height, the minimum value of width and height, the area size (width multiplied by height), or the aspect ratio (ratio of width to height, or ratio of height to width) of the block. A super block may refer to the largest coding unit (LCU), such as a 128×128 block in AV1. SDP may also be referred to as SDT. The IntraBC flag is a block-level flag used to indicate whether IntraBC is applied to the current block.

[0138] This application includes multiple methods for determining whether to apply the IntraBC mode to a coding unit when the coding unit uses SDP, wherein the luma block in the coding unit and the corresponding one or more chroma blocks can share a partial tree structure. Therefore, these methods can be applied when SDP is enabled for the coding unit.

[0139] According to various aspects of this application, when a luma block and one or more corresponding chroma blocks share (or have) the same segmentation structure (or segmentation tree), the IntraBC flags of the luma block and the chroma block are the same. Otherwise, when a luma block and one or more corresponding chroma blocks have different segmentation trees, the IntraBC flags of the luma block and the one or more chroma blocks may be different.

[0140] In one embodiment, when a luma block and one or more corresponding chroma blocks share the same segmentation structure (or segmentation tree), the IntraBC flag and BV of the luma block and these chroma blocks are identical.

[0141] In one embodiment, when a luma block and one or more corresponding chroma blocks share the same segmentation structure (or segmentation tree), the IntraBC flags of the luma block and the chroma block are identical. Otherwise, when a luma block and one or more corresponding chroma blocks have different segmentation trees, the IntraBC flag of the luma block can be represented by a signal only in the bitstream, and the IntraBC flag of one or more chroma blocks can always be set to zero (or false, disabled).

[0142] In one embodiment, when a luma block and one or more corresponding chroma blocks share the same segmentation structure (or segmentation tree), the IntraBC flags of the luma block and the chroma block are identical. Otherwise, when a luma block and one or more corresponding chroma blocks have different segmentation trees, the IntraBC flags of the luma block and the one or more corresponding chroma blocks can be represented by signals in the bitstream.

[0143] According to various aspects of this application, when luma blocks and chroma blocks have different segmentation tree structures, luma blocks and corresponding multiple chroma blocks can share the same IntraBC flag.

[0144] In one embodiment, when a luma block and its associated chroma blocks have different segmentation tree structures and the same segmentation size, the luma block and its associated chroma blocks can share the same IntraBC flag and / or BV. For example, in the YUV 420 format, if the luma block has a block size of 64×32 and the associated chroma block has a block size of 32×16, then the luma block and the associated chroma block have the same segmentation size. Therefore, the luma block and the associated chroma block can share the same IntraBC flag and / or BV.

[0145] In one embodiment, when a luma block and its associated chroma blocks have different segmentation tree structures and the difference between the luma block size and the chroma block size is equal to or less than a threshold, the luma block and its associated chroma blocks can share the same IntraBC flag and / or BV. For example, when the block size of the associated chroma block is not greater than K times the block size of the luma block and the block size of the associated chroma block is not less than 1 / K times the block size of the luma block, the luma block and its associated chroma blocks can share the same IntraBC flag and / or BV. In one example, K is set to 2 or 4.

[0146] According to various aspects of this application, when encoding a co-occurrence luma block whose block size is larger than the current chroma block, the chroma IntraBC flag and the corresponding BV can be inherited from the co-occurrence luma block.

[0147] According to various aspects of this application, a chroma block can be associated with multiple co-located luma blocks, and the chroma block and the multiple co-located luma blocks can have different segmentation trees. The multiple co-located luma blocks can be partially or completely encoded in IntraBC mode.

[0148] In one embodiment, when multiple co-located luma blocks are fully encoded in IntraBC mode, all multiple co-located luma blocks are encoded in IntraBC mode, and the chroma blocks are also encoded in IntraBC mode. The BV of the chroma block can be derived based on the BV corresponding to the center sample of the multiple co-located luma blocks or the BV corresponding to the corner sample of the multiple co-located luma blocks.

[0149] In one embodiment, when multiple co-located luma blocks are partially encoded in IntraBC mode, a first subset of the multiple co-located luma blocks is encoded in IntraBC mode, and a second subset of the multiple co-located luma blocks is encoded using intra-prediction. First plurality of chroma samples of the chroma block corresponding to the first subset of the multiple co-located luma blocks are encoded in IntraBC mode. The BV of the first plurality of chroma samples can be derived based on the BV corresponding to the center sample of the first subset of the multiple co-located luma blocks or the BV corresponding to the corner sample of the first subset of the multiple co-located luma blocks. Second plurality of chroma samples of the chroma block corresponding to the second subset of the multiple co-located luma blocks are encoded using intra-prediction. The intra-prediction mode of the second plurality of chroma samples can be represented by a signal in the bitstream or derived based on the intra-prediction mode of the second subset of the multiple co-located luma blocks. Then, a prediction block for the chroma block can be generated based on the first plurality of chroma samples and the second plurality of chroma samples.

[0150] V. Flowchart

[0151] Figure 15A flowchart of an exemplary method (1500) according to an embodiment of this application is shown. In various embodiments, the method (1500) is executed by processing circuitry, such as processing circuitry in terminal devices (210), (220), (230), and (240), processing circuitry performing the functions of a video encoder (303), processing circuitry performing the functions of a video decoder (310), processing circuitry performing the functions of a video decoder (410), processing circuitry performing the functions of an intra-frame prediction module (452), processing circuitry performing the functions of a video encoder (503), processing circuitry performing the functions of a predictor (535), processing circuitry performing the functions of an intra-frame encoder (622), processing circuitry performing the functions of an intra-frame decoder (772), and so on. In some embodiments, the method (1500) is implemented as software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry executes the method (1500).

[0152] Method (1500) can typically begin at step (S1510), where method (1500) decodes the prediction information of coding units in the current image, which is part of the video bitstream. Then, method (1500) proceeds to step (S1520).

[0153] In step (S1520), the method (1500) determines, based on the prediction information, whether the luma block and chroma block associated with the coding unit have different segmentation trees. When the luma block and chroma block associated with the coding unit have different segmentation trees, the method (1500) proceeds to step (S1530).

[0154] In step (S1530), the method (1500) determines whether the luma block is encoded in IBC mode based on the first IBC flag in the prediction information. Then, the method (1500) proceeds to step (S1540).

[0155] In step (S1540), method (1500) determines whether the chroma block is encoded in IBC mode based on the default mode, the second IBC flag in the prediction information, and the first IBC flag. Then, method (1500) proceeds to step (S1550).

[0156] In step (S1550), method (1500) reconstructs the coding unit based on the luma block and chroma block. Then, method (1500) terminates.

[0157] In one embodiment, in response to the luma block and chroma block associated with the encoding unit having the same segmentation tree, method (1500) determines whether the luma block and chroma block are encoded in IBC mode based on a first IBC flag included in the prediction information. Based on the luma block and chroma block being encoded in IBC mode, method (1500) determines that the luma block and chroma block have the same block vector.

[0158] In one embodiment, the method (1500) determines that the chroma block is not encoded in IBC mode based on a default mode that indicates that IBC mode is disabled for chroma blocks that have a different segmentation tree than the luma block.

[0159] In one embodiment, method (1500) determines whether the luma block and the chroma block have the same segmentation size. In response to the luma block and the chroma block having the same segmentation size, method (1500) determines whether the chroma block is encoded in IBC mode based on a first IBC flag.

[0160] In one embodiment, the luma block has a larger block size than the chroma block, and the method (1500) determines whether the luma block and the chroma block are encoded in IBC mode based on a first IBC flag. In response to the luma block and the chroma block being encoded in IBC mode, the method (1500) determines that the luma block and the chroma block have the same block vector.

[0161] In one embodiment, a first subset of samples of the chroma block is co-located with a first luma block encoded in IBC mode, and a second subset of samples of the chroma block is co-located with a second luma block encoded in a first intra-prediction mode, and method (1500) determines that the first subset of samples of the chroma block is encoded in IBC mode. Method (1500) determines that the second subset of samples of the chroma block is encoded in one of a first intra-prediction mode and a second intra-prediction mode in the prediction information.

[0162] In one embodiment, the first subset of the chroma block samples and the first luminance block can both have the same block vector.

[0163] VI. Computer System

[0164] The above-described techniques can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 16 A computer system (1600) suitable for implementing some embodiments of this application.

[0165] Computer software can be coded using any suitable machine code or computer language, and can generate instruction code using assembly, compilation, linking, or similar mechanisms. This instruction code can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or through operations such as code interpretation and microcode execution.

[0166] These instructions can be executed in a variety of types of computers or computer components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0167] Figure 16 The components shown for the computer system (1600) are exemplary in nature and are not intended to limit the use or scope of functionality of the computer software used to implement the embodiments of this application. Nor should the configuration of the components be construed as having any dependency or requirement on any part or combination thereof in the exemplary embodiments of the computer system (1600).

[0168] The computer system (1600) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users through, for example, tactile input (such as keystrokes, waving, or movement of a data glove), audio input (such as speech or tapping), visual input (such as gestures), or olfactory input (not shown). The human-machine interface device may also be used to capture certain media that are not necessarily directly associated with conscious human input, such as audio (such as speech, music, or ambient sound), images (such as scanned images or photographic images obtained from a still image camera), and video (such as two-dimensional video, including three-dimensional video, including stereoscopic video).

[0169] Human-machine interface input devices may include one or more of the following (only one of each is depicted): keyboard (1601), mouse (1602), touchpad (1603), touch screen (1610), data glove (not shown), joystick (1605), microphone (1606), scanner (1607), and camera (1608).

[0170] The computer system (1600) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (1610), data gloves (not shown), or joystick (1605), but may also include tactile feedback devices that are not used as input devices), audio output devices (such as speakers (1609), headphones (not shown)), visual output devices, and printers (not shown), wherein the visual output devices include screens (1610), virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown), and screens (1310) include cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, and organic light-emitting diode (OLED) screens, each with or without touchscreen input capability, each with or without tactile feedback capability, and some of these screens are capable of outputting two-dimensional or more than three-dimensional visual outputs in some way (such as stereoscopic image output). These visual output devices (e.g., screens (1610)) can be connected to the system bus (1648) via a graphics adapter (1650).

[0171] The computer system (1600) may also include human-accessible storage devices and their associated media, such as optical media (including CD / DVD ROM / RW with CD / DVD (1620)) or similar media (1621), thumb drives (1622), removable hard disk drives or solid-state drives (1623), conventional magnetic media (such as magnetic tape and floppy disks (not shown)), devices based on dedicated ROM / ASIC / PLD (such as SecureDell chips (not shown)), and so on.

[0172] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0173] The computer system (1600) may also include a network interface (1654) for connecting one or more communication networks (1655). The one or more communication networks (1655) may be, for example, wireless networks, wired networks, optical networks. The one or more communication networks (1655) may also be local area networks, wide area networks, metropolitan area networks, vehicle-to-everything (V2X) and industrial networks, real-time networks, latency-tolerant networks, etc. Examples of one or more communication networks (1655) include local area networks (such as Ethernet, wireless LAN), cellular networks (including Global System for Mobile Communications (GSM), third-generation mobile communication systems (3G), fourth-generation mobile communication systems (4G), fifth-generation mobile communication systems (5G), Long Term Evolution (LTE), etc.), cable or wireless wide-area digital television networks (including cable television, satellite television, and terrestrial broadcast television), vehicle and industrial networks (including CANbus), etc. Some networks typically require an external network interface adapter that connects to a general-purpose data port or peripheral bus (1649) (such as the Universal Serial Bus (USB) port of a computer system (1600); others are typically integrated into the core of the computer system (1600) via a connection to the system bus as described below (e.g., an Ethernet interface to a personal computer system or a cellular network interface to a smartphone computer system). By using any of these networks, the computer system (1600) can communicate with other entities. This communication can be unidirectional, receive-only (e.g., broadcasting TV), transmit-only (e.g., to a CAN bus of a certain CAN bus device), or bidirectional communication to other computer systems using a local area or wide area digital network. Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.

[0174] The aforementioned human-machine interface device, human-accessible storage device, and network interface can be connected to the kernel (1640) of the computer system (1600).

[0175] The kernel (1640) may include one or more central processing units (CPU) (1641), graphics processing units (GPUs) (1642), dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) (1643), task-specific hardware accelerators (1644), graphics adapters (1650), etc. These devices, along with read-only memory (ROM) (1645), random access memory (1646), internal mass storage (1647) (such as internal non-user-accessible hard disk drives, SSDs), etc., may be interconnected via a system bus (1648). In some computer systems, the system bus (1648) may be accessed as one or more physical connectors, allowing for expansion through additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the kernel's system bus (1649), or via a peripheral bus (1648). In one example, a screen (1610) may be connected to a graphics adapter (1650). Peripheral bus architectures include PCI, USB, etc.

[0176] The CPU (1641), GPU (1642), FPGA (1643), and accelerator (1644) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (1645) or RAM (1646). Intermediate data can also be stored in RAM (1646), while permanent data can be stored, for example, in internal mass storage (1647). Fast storage and retrieval to any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1641), GPUs (1642), mass storage (1647), ROM (1645), RAM (1646), etc.

[0177] Computer-readable media may contain computer code on which various computer-executed operations are performed. The media and computer code may be specially designed and constructed for the purposes of this application, or they may be media and code well known and usable by those skilled in the art of computer software.

[0178] By way of example and not limitation, a computer system having an architecture (1600), particularly a kernel (1640), can provide functionality implemented by a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software stored in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage as described above, as well as some storage of the non-transitory kernel (1640), such as internal kernel mass storage (1647) or ROM (1645). Software implementing the embodiments of this application can be stored in such a device and executed by the kernel (1640). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can enable the kernel (1640), particularly the processor therein (including a CPU, GPU, FPGA, etc.), to execute a specific process or a specific portion of a specific process described herein, including defining data structures stored in RAM (1646) and modifying these data structures according to the software-defined process. As a supplement or alternative, a computer system may provide the same functionality as the logic hardwires or other components in a circuit (e.g., accelerator 1644) to replace or operate with the software to perform the specific process or a specific portion of the specific process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry storing the execution of software (such as an integrated circuit (IC)), circuitry including the execution of logic, or both. This application includes any suitable combination of hardware and software.

[0179] While this application has described several exemplary embodiments, various modifications, substitutions, and alternatives to the embodiments are within the scope of this application. Therefore, it should be understood that those skilled in the art can design various systems and methods that, while not explicitly shown or described herein, embody the principles of this application and are thus within its spirit and scope.

[0180] Appendix A: Acronyms

[0181] ALF: Adaptive Loop Filter

[0182] AMVP: Advanced Motion Vector Prediction

[0183] APS: Adaptation Parameter Set

[0184] ASIC: Application-Specific Integrated Circuit

[0185] ATMVP: Alternative / Advanced Temporal Motion Vector Prediction, optional temporal motion vector prediction

[0186] AV1: AOMedia Video 1

[0187] AV2: AOMedia Video 2

[0188] BMS: Benchmark Set

[0189] BV: Block Vector

[0190] CANBus: Controller Area Network Bus

[0191] CB: Coding Block

[0192] CC-ALF: Cross-Component Adaptive Loop Filter

[0193] CD: Compact Disc

[0194] CDEF: Constrained Directional Enhancement Filter

[0195] CPR: Current Picture Referencing.

[0196] CPU: Central Processing Unit

[0197] CRT: Cathode Ray Tube

[0198] CTB: Coding Tree Block

[0199] CTU: Coding Tree Unit

[0200] CU: Coding Unit

[0201] DPB: Decoder Picture Buffer

[0202] DPCM: Differential Pulse-Code Modulation

[0203] DPS: Decoding Parameter Set

[0204] DVD: Digital Video Disc

[0205] FPGA: Field Programmable Gate Area

[0206] JCCR: Joint CbCr Residual Coding.

[0207] JVET: Joint Video Exploration Team

[0208] GOP: Groups of Pictures

[0209] GPU: Graphics Processing Unit

[0210] GSM: Global System for Mobile Communications

[0211] HDR: High Dynamic Range

[0212] HEVC: High Efficiency Video Coding

[0213] HRD: Hypothetical Reference Decoder

[0214] IBC: Intra Block Copy

[0215] IC: Integrated Circuit

[0216] ISP: Intra Sub-Partitions, intra-frame sub-segmentation

[0217] JEM: Joint Exploration Model

[0218] LAN: Local Area Network

[0219] LCD: Liquid-Crystal Display

[0220] LR: Loop Restoration Filter

[0221] LRU: Loop Restoration Unit

[0222] LTE: Long-Term Evolution

[0223] MPM: Most Probable Mode

[0224] MV: Motion Vector

[0225] OLED: Organic Light-Emitting Diode

[0226] PB: Prediction Blocks

[0227] PCI: Peripheral Component Interconnect

[0228] PDPC: Position Dependent Prediction Combination

[0229] PLD: Programmable Logic Device

[0230] PPS: Picture Parameter Set

[0231] PU: Prediction Unit

[0232] RAM: Random Access Memory

[0233] ROM: Read-Only Memory

[0234] SAO: Sample Adaptive Offset

[0235] SCC: Screen Content Coding

[0236] SDR: Standard Dynamic Range

[0237] SEI: Supplementary Enhancement Information

[0238] SNR: Signal Noise Ratio

[0239] SPS: Sequence Parameter Set

[0240] SSD: Solid-state Drive

[0241] TU: Transform Unit

[0242] USB: Universal Serial Bus

[0243] VPS: Video Parameter Set

[0244] VUI: Video Usability Information

[0245] VVC: Versatile Video Coding

[0246] WAIP: Wide-Angle Intra Prediction

Claims

1. A video encoding / decoding method at a decoder, characterized in that, include: Decode the prediction information of the coding units in the current image, which is part of the video stream; Based on the prediction information, it is determined whether the luma block and chroma block associated with the coding unit have different segmentation trees; In response to the fact that the luma block and the chroma block associated with the coding unit have different segmentation trees, it is determined whether the luma block is encoded in IBC mode based on the first intra-block copy IBC flag in the prediction information. In response to the fact that the luma block and the chroma block associated with the encoding unit have different segmentation trees, one of the following is performed for the chroma block: Based on the intra-block copy IBC flags of the luma block and the chroma block respectively represented by signals in the bitstream, it is determined whether the chroma block is encoded in IBC mode. When the luma block and the chroma block have the same segmentation size, it is determined whether the chroma block is encoded in IBC mode based on the first intra-block copy IBC flag. When the block size of the luma block is larger than the block size of the chroma block, it is determined whether the chroma block is encoded in IBC mode based on the first IBC flag; and The coding unit is reconstructed based on the luma block and the chroma block.

2. The method according to claim 1, characterized in that, Further includes: In response to the fact that the luma block and the chroma block associated with the encoding unit have the same segmentation tree, Based on the first IBC flag in the prediction information, determine whether the luma block and the chroma block are encoded in the IBC mode; as well as Based on the luminance block and the chrominance block encoded in the IBC mode, it is determined that the luminance block and the chrominance block have the same block vector.

3. The method according to claim 1, characterized in that, When the size of the luminance block is larger than the size of the chrominance block, In response to the luma block and the chroma block being encoded in the IBC mode It is determined that the luminance block and the chrominance block have the same block vector.

4. The method according to claim 1, characterized in that, When a first subset of samples of the chroma block is co-located with a first luma block encoded in the IBC mode, and a second subset of samples of the chroma block is co-located with a second luma block encoded in the first intra-frame prediction mode, the method further includes: The first subset of samples of the chroma block is determined to be encoded in the IBC mode; and The second subset of samples of the chroma block is determined to be encoded using one of the first intra-frame prediction mode and the second intra-frame prediction mode in the prediction information.

5. The method according to claim 4, characterized in that, The first subset of the chroma block samples and the first luminance block have the same block vector.

6. A video decoding device, characterized in that, include: Processing circuit, the processing circuit being used for: Decode the prediction information of the coding units in the current image, which is part of the video stream; Based on the prediction information, it is determined whether the luma block and chroma block associated with the coding unit have different segmentation trees; In response to the fact that the luma block and the chroma block associated with the coding unit have different segmentation trees, it is determined whether the luma block is encoded in IBC mode based on the first intra-block copy IBC flag in the prediction information. In response to the fact that the luma block and the chroma block associated with the encoding unit have different segmentation trees, one of the following is performed for the chroma block: Based on the intra-block copy IBC flags of the luma block and the chroma block respectively represented by signals in the bitstream, it is determined whether the chroma block is encoded in IBC mode. When the luma block and the chroma block have the same segmentation size, it is determined whether the chroma block is encoded in IBC mode based on the first intra-block copy IBC flag. When the difference between the block size of the luminance block and the block size of the chrominance block is equal to or less than a threshold, it is determined whether the chrominance block is encoded in IBC mode based on the first IBC flag. as well as The coding unit is reconstructed based on the luma block and the chroma block.

7. The apparatus according to claim 6, characterized in that, The processing circuit is further used for: In response to the fact that the luma block and the chroma block associated with the encoding unit have the same segmentation tree, Based on the first IBC flag in the prediction information, determine whether the luma block and the chroma block are encoded in the IBC mode; as well as Based on the luminance block and the chrominance block encoded in the IBC mode, it is determined that the luminance block and the chrominance block have the same block vector.

8. The apparatus according to claim 6, characterized in that, The processing circuit is further used for: When the size of the luminance block is larger than the size of the chrominance block, In response to the luma block and the chroma block being encoded in the IBC mode It is determined that the luminance block and the chrominance block have the same block vector.

9. The apparatus according to claim 7, characterized in that, The processing circuit is further used for: When a first subset of samples of the chroma block is in the same position as a first luma block encoded in the IBC mode, and a second subset of samples of the chroma block is in the same position as a second luma block encoded in the first intra-frame prediction mode, The first subset of samples of the chroma block is determined to be encoded in the IBC mode; and The second subset of samples of the chroma block is determined to be encoded using one of the first intra-frame prediction mode and the second intra-frame prediction mode in the prediction information.

10. The apparatus according to claim 9, characterized in that, The first subset of the chroma block samples and the first luminance block have the same block vector.

11. A non-volatile computer-readable storage medium storing instructions, characterized in that, The instructions are executed by at least one processor to implement the method according to any one of claims 1-5.

Images