On propagating intra prediction mode information of ibc block by using block vector

Abstract

According to one aspect of the present disclosure, a video decoding method performed in a video decoder is provided. In the video decoding method, coded information of a current block within a current picture of a video is received from a coded video bitstream. The coded information indicates that the current block is an intra-block copy (IBC) block. An intra-predicted mode for a subblock of the current block is derived based on a reference subblock within a reference block. The reference subblock corresponds to a subblock of the current block, and the reference block is referenced by a block vector of the IBC block. The intra-predicted mode for a subblock of the current block is further stored. It is determined that a neighbor block of the current block is an IBC block. Using the stored intra-predicted mode, a list of most likely modes (MPM) for the neighbor block is constructed.

Description

Technology Field

[0001] Integration by reference

[0002] The present application claims the benefit of priority to U.S. Provisional Application No. 63 / 261,251, filed September 15, 2021, “On Propagating Intra Prediction Mode Information of IBC Block By Using Block Vector,” and to the benefit of priority to U.S. Patent Application No. 17 / 901,547, filed September 1, 2022, “ON PROPAGATING INTRA PREDICTION MODE INFORMATION OF IBC BLOCK BY USING BLOCK VECTOR.” The disclosures of these prior applications are incorporated herein by reference in their entirety.

[0003] The present disclosure describes embodiments generally related to video coding. Background Technology

[0004] The background description provided in this specification is generally intended to provide the context of the present disclosure. Within the scope of the works described in this background section, the works of the currently named inventors and aspects of the description that may not be recognized as prior art at the time of filing are not recognized as prior art of the present disclosure, either expressly or impliedly.

[0005] Uncompressed digital images and / or videos may contain a series of pictures, each having, for example, a spatial dimension of 20×1080 luminance samples (also called luminance samples) and associated chrominance samples (also called chroma samples). The series of pictures may have, for example, 60 pictures per second or a fixed or variable picture rate of 60 Hz (informally known as the frame rate). Uncompressed images and / or videos have specific bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (20×1080 luminance sample resolution at a 60 Hz frame rate) requires a bandwidth close to 1.5 Gbit / s. One hour of such video requires more than 600 GB of storage space.

[0006] One purpose of image and / or video coding and decoding may be to reduce the redundancy of input image and / or video signals through compression. Compression can help reduce the aforementioned bandwidth and / or storage requirements by more than double orders of magnitude in some cases. While the description in this specification uses video encoding / decoding as an exemplary example, the same techniques may be applied to image encoding / decoding in a similar manner without departing from the spirit of this disclosure. Both lossless compression and lossy compression, as well as combinations thereof, may be used. Lossless compression refers to a technique capable of reconstructing an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original signal and the reconstructed signal is small enough that the reconstructed signal is useful for the intended application. For video, lossy compression is widely employed. The amount of acceptable distortion varies depending on the application. For example, users of certain consumer streaming applications may tolerate higher distortion than users of movie or television distribution applications. The achievable compression ratio may reflect the following: the higher the acceptable distortion, the higher the compression ratio may be.

[0007] Video encoders and decoders can utilize techniques from a wide range of categories, including, for example, motion compensation, transform processing, quantization, and entropy coding.

[0008] Video codec technology may include a technique known as intra coding. In intra coding, sample values ​​are represented without reference to samples of a previously reconstructed reference picture or other data. In some video codecs, a picture is spatially subdivided into blocks of samples. If all blocks of samples are coded in intra mode, the picture may be an intra picture. Derivations such as the intra picture and the independent decoder refresh picture can be used to reset the decoder state and thus serve as the first picture of the coded video bitstream and video session, or as a still image. Samples from an intra block can be derived for a transform, and the transform coefficients can be quantized before entropy coding. Intra prediction may be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value after conversion and the smaller the AC coefficient, the fewer bits are required at a given quantization step size to represent the block after entropy coding.

[0009] For example, traditional intra-coding used in MPEG-2 generative coding techniques does not use intra-prediction. However, some newer video compression techniques include methods that attempt to use, for example, metadata and / or surrounding sample data acquired during the encoding and / or decoding of data blocks. These techniques will henceforth be referred to as "intra-prediction" techniques. Note that, at least in some cases, intra-prediction uses only reference data from the current picture being reconstructed, rather than from the reference picture.

[0010] There may be many different forms of intra-prediction. If one or more of these techniques are available in a given video coding technique, the specific technique in use may be coded in a specific intra-prediction mode that utilizes that technique. In some cases, an intra-prediction mode may have submodes and / or parameters, where the submodes and / or parameters may be coded individually or included in a mode codeword that defines the prediction mode being used. Which codeword is used for a given combination of mode, submode, and / or parameter can affect the coding efficiency gain through intra-prediction, and thus may also affect the entropy coding technique used to convert the codeword into a bitstream.

[0011] Intra prediction of specific modes was introduced with H.264, improved in H.265, and further enhanced in newer coding techniques such as the joint exploration model (JEM), versatile video coding (VVC), and benchmark set (BMS). A predictor block can be formed using neighbor sample values ​​of already available samples. Sample values ​​of neighbor samples can be copied to the predictor block based on direction. References to the direction in use can be encoded in the bitstream or predicted itself.

[0012] Referring to FIG. 1a, a subset of the nine known predictor directions among the 33 possible predictor directions defined in H.265 (corresponding to the 33 angle modes of the 35 intra modes) is shown in the lower right. The point where the arrows converge (101) indicates the predicted sample. The arrows indicate the direction in which the sample is being predicted. For example, arrow (102) indicates that the sample (101) is predicted to the upper right at a 45-degree angle in the horizontal direction from the sample or samples. Likewise, arrow (103) indicates that the sample (101) is predicted to the lower left of the sample (101) at a 22.5-degree angle in the horizontal direction from the sample or samples.

[0013] Referring still to FIG. 1a, a square block (104) of 4×4 samples (indicated by a bold dashed line) is shown in the upper left corner. The square block (104) contains 16 samples, each labeled "S", a position in the Y dimension (e.g., row index) and a position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Likewise, sample S44 is the fourth sample of the block (104) in both the Y and X dimensions. Since the block is 4×4 samples in size, S44 is located in the lower right corner. Additional reference samples following a similar numbering scheme are shown. The reference samples are labeled R, a Y position (e.g., row index) and an X position (column index) relative to the block (104). In both H.264 and H.265, the predicted sample is adjacent to the block being rebuilt, so there is no need to use negative values.

[0014] Intra-picture prediction can be performed by copying a reference sample value from a neighbor sample indicated by a signaled prediction direction. For example, assume that the coded video bitstream includes signaling indicating a prediction direction corresponding to the arrow (102) for this block. That is, the samples are predicted at a 45-degree angle in the horizontal direction from the sample to the upper right. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Then, sample S44 is predicted from reference sample R08.

[0015] In some cases, to calculate reference samples, especially when the direction cannot be divided equally into 45 degrees, the values ​​of multiple reference samples may be combined through interpolation, for example.

[0016] The number of possible directions has increased as video coding technology has advanced. In H.264 (2003), nine different directions could be represented. In H.265 (2013), this increased to 33. Currently, JEM / VVC / BMS can support up to 65 directions. Experiments have been conducted to identify the most likely direction, and specific techniques in entropy coding use such the most likely direction to encode it with a small number of bits, accepting a certain penalty for less likely directions. Additionally, the direction itself can sometimes be predicted from the neighboring direction used in a neighboring block that has already been decoded.

[0017] FIG. 1b illustrates a schematic diagram (110) showing 65 intra-predicted directions according to JEM to explain the number of predicted directions increasing over time.

[0018] The mapping of intra-predicted direction bits, which indicate direction in a coded video bitstream, may vary depending on the video coding technique.

[0019] These mappings vary, for example, from simple direct mapping to complex adaptive methods involving codewords and the most likely mode, and similar techniques. However, in most cases, there may be specific directions in video content that are statistically less likely to occur than other specific directions. Since the goal of video compression is to reduce redundancy, in well-designed video coding techniques, less likely directions can be represented by a larger number of bits than more likely directions.

[0020] Image and / or video coding and decoding may be performed using inter-picture prediction with motion compensation. Motion compensation may be a lossy compression technique and may relate to a technique used for prediction of a newly reconstructed picture or part of a picture after sample data from a previously reconstructed picture or part thereof (reference picture) is spatially shifted in a direction indicated by a motion vector (hereinafter MV). In some cases, the reference picture may be identical to the picture currently being reconstructed. The MV may be two-dimensional in X and Y, or three-dimensional, with the third dimension being an indication of the reference picture in use (the latter may indirectly be the time dimension).

[0021] In some video compression techniques, an MV applicable to a specific area of ​​sample data can be predicted from other MVs, for example, from other MVs associated with other areas of sample data that are spatially adjacent to the area being reconstructed and precede the MV in the decoding order. This significantly reduces the amount of data required to code the MV, thereby eliminating redundancy and increasing compression. For example, when coding an input video signal derived from a camera (known as natural video), MV prediction can work effectively because there is a statistical probability that an area larger than the area where a single MV can be applied moves in a similar direction within the video sequence; thus, in some cases, it can be predicted using similar motion vectors derived from MVs in neighboring areas. As a result, the MV found for a given area is similar or identical to the MV predicted from surrounding MVs, and after entropy coding, it can be represented by fewer bits than would have been used if the MV were coded directly. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., MV) derived from the original signal (i.e., sample stream). In other cases, the MV prediction itself may be lost, for example, because rounding error occurs when calculating the predictor from several surrounding MVs.

[0022] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016). Among the many MV prediction mechanisms provided by H.265, the technique described with reference to Fig. 2 is hereinafter referred to as "spatial merge".

[0023] Referring to FIG. 2, the current block (201) may include samples discovered by an encoder during the motion search process so as to be predictable from a previous block of the same size that is spatially shifted. Instead of directly coding the MV, the MV may be derived from metadata associated with one or more reference pictures, for example from the most recent (in decoding order) reference picture, using an MV associated with one of five neighboring samples denoted as A0, A1 and B0, B1, B2 (202–206, respectively). In H.265, the MV prediction may use a predictor from the same reference picture used by the neighboring block.

[0024] The aspects of the present disclosure provide a method and apparatus for video encoding / decoding. In some examples, the apparatus for video decoding includes a processing circuit.

[0025] According to one aspect of the present disclosure, a video decoding method performed in a video decoder is provided. In the video decoding method, coded information of a current block within a current picture of a video may be received from a coded video bitstream. The coded information may indicate that the current block is an intra-block copy (IBC) block. An intra-predicted mode for a subblock of the current block may be derived based on a reference subblock within a reference block. The reference subblock may correspond to a subblock of the current block, and the reference block may be referenced by a block vector of the IBC block. The intra-predicted mode for a subblock of the current block may be further stored. A neighbor block of the current block may be determined to be an IBC block. Using the stored intra-predicted mode, a list of most probable modes (MPM) for the neighbor block may be constructed.

[0026] The intra prediction mode for the above subblock can be derived as the intra mode of a pixel located at the central position of the reference subblock of the above reference block, and the central position is at row 2, column 2 of the above reference subblock.

[0027] The intra prediction mode for the above subblock can be derived as an intra mode for a pixel located at the center position of the reference subblock of the above reference block, and the center position is at row 2, column 3 of the above reference subblock.

[0028] The intra prediction mode for the above subblock can be derived as an intra mode for a pixel located at the center position of the reference subblock of the above reference block, and the center position is at row 3, column 2 of the above reference subblock.

[0029] The intra prediction mode for the above subblock can be derived as an intra mode for a pixel located at the center position of the reference subblock of the above reference block, and the center position is at row 3, column 3 of the above reference subblock.

[0030] The intra prediction mode for the above subblock can be derived as the intra mode of a pixel located at one of the top-right, top-left, bottom-left, or bottom-right positions of the reference subblock of the above reference block.

[0031] In some embodiments, the current block may be partitioned into a first partition and a second partition based on a geometric partition mode (GPM). The first partition may be predicted by the IBC, and the second partition may be predicted by intra prediction. The subblock may be located in the first partition predicted by the IBC.

[0032] In some embodiments, the subblocks of the current block may be in units of 4×4 pixels.

[0033] In some embodiments, an intra prediction mode for the sub-block in the MPM list for neighbor blocks of the current block is determined as a candidate prediction mode, and the neighbor block can be predicted by the IBC.

[0034] According to another aspect of the present disclosure, an apparatus is provided. The apparatus includes a processing circuit. The processing circuit may be configured to perform either of the video encoding / decoding methods.

[0035] An aspect of the present disclosure also provides a computer-readable, non-transient storage medium for storing instructions, said instructions configured to cause the computer to perform either of the video encoding / decoding methods when executed by a computer for video decoding. Brief explanation of the drawing

[0036] Additional features, nature, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. Figure 1a is a schematic diagram of an exemplary subset of intra-prediction modes. Figure 1b is an example of an exemplary intra-prediction direction. Figure 2 is a schematic diagram of the current block and its surrounding space merge candidate in one example. FIG. 3 is a schematic diagram of a simplified block diagram of a communication system (300) according to one embodiment. FIG. 4 is a schematic diagram of a simplified block diagram of a communication system (400) according to one embodiment. FIG. 5 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. FIG. 6 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. FIG. 7 illustrates a block diagram of an encoder according to another embodiment. FIG. 8 illustrates a block diagram of a decoder according to another embodiment. FIG. 9 illustrates exemplary locations of spatial merge candidates according to another embodiment. Figure 10 illustrates an exemplary redundancy check of spatial merge candidates. FIG. 11 illustrates a schematic diagram of motion vector scaling for time merge candidates according to one embodiment. FIG. 12 illustrates exemplary candidate locations for time merge candidates according to one embodiment. FIG. 13 illustrates an exemplary angular distribution of a geometric partition mode (GPM) according to one embodiment. FIG. 14 illustrates an exemplary partition line for a GPM according to one embodiment. Figure 15 illustrates an example of an intra-block prediction mode for propagation to an intra-block radiation (IBC) block. Figure 16 illustrates an example of intra-predicted mode propagation for an encoded block in a GPM having IBC and intra-mode. FIG. 17 illustrates a flowchart schematically illustrating an exemplary decoding process according to some embodiments of the present invention. FIG. 18 illustrates a flowchart schematically illustrating an exemplary encoding process according to some embodiments of the present invention. FIG. 19 is a schematic diagram of a computer system according to an embodiment. Specific details for implementing the invention

[0037] FIG. 3 illustrates an exemplary block diagram of a communication system (300). The communication system (300) includes a plurality of terminal devices capable of communicating with each other, for example, through a network (350). For example, the communication system (300) includes a first pair of terminal devices (310, 320) interconnected through the network (350). In the example of FIG. 3, the first pair of terminal devices (310, 320) performs unidirectional transmission of data. For example, the terminal device (310) may encode video data (e.g., a stream of video pictures captured by the terminal device (310)) for transmission to another terminal device (320) through the network (350). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. The terminal device (320) may receive the encoded video data from the network (350), decode the encoded video data to restore the video picture, and display the video picture according to the restored video data. Unidirectional data transmission can be common in media serving applications, etc.

[0038] In another example, the communication system (300) includes a second pair of terminal devices (330, 340) that perform bidirectional transmission of coded video data, for example, during a video conference. For bidirectional transmission of data, in one example, each terminal device of the pair of terminal devices (330, 340) may code video data (e.g., a stream of video pictures captured by the terminal device) for transmission to another terminal device of the pair of terminal devices (330, 340) via a network (350). Each terminal device of the pair of terminal devices (330, 340) may also receive coded video data transmitted by another terminal device of the pair of terminal devices (330, 340), decode the coded video data to restore the video, and display the video picture on an accessible display device according to the restored video data.

[0039] In the example of FIG. 3, the terminal devices (310, 320, 330, 340) are exemplified as a server, a personal computer, and a smartphone, respectively, but the principles of the present disclosure may not be so limited. Embodiments of the present disclosure find applications with laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (350) represents any number of networks that transmit coded video data between terminal devices (310, 320, 330, 340), including, for example, wired and / or wireless communication networks. The communication network (350) may exchange data over circuit-switched and / or packet-switched channels. Representative networks include communication networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, the architecture and topology of the network (350) may not be important to the operation of the present disclosure unless described below.

[0040] FIG. 4 illustrates a video encoder and a video decoder in a streaming environment as an example of an application for the disclosed subject. The disclosed subject may be equally applicable to other video enabled applications, such as video conferencing, digital TV, streaming services, and the storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0041] A streaming system may include a capture subsystem (413) which may include, for example, a digital camera, a video source (401) that generates a stream (402) of uncompressed video pictures. In one example, the stream (402) of video pictures includes samples captured by a digital camera. The stream (402) of video pictures, indicated by a bold line to emphasize a high data volume compared to encoded video data (404) (or encoded video bitstream), may be processed by an electronic device (420) comprising a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof capable of enabling or implementing aspects of the disclosed subject matter as described in more detail below. Encoded video data (404) (or encoded video bitstream (404)), indicated by a thin line to emphasize a lower data volume compared to the stream (402) of the video picture, may be stored in a streaming server (405) for later use. One or more streaming client subsystems, such as the client subsystems (406, 408) in FIG. 4, may access the streaming server (405) to retrieve copies (407, 409) of the encoded video data (404). The client subsystem (406) may include a video decoder (410) in, for example, an electronic device (430). The video decoder (410) decodes an incoming copy (407) of the encoded video data and generates an outgoing stream of a video picture (411) that can be rendered on a display (412) (e.g., a display screen) or another rendering device (not shown). In some streaming systems, encoded video data (404, 407, 409) (e.g., video bitstream) may be encoded according to a specific video coding / compression standard.Examples of such standards include ITU-T Recommendation H.265. For instance, a video coding standard under development is informally known as VVC (Versatile Video Coding). The disclosed topics may be used in the context of VVC.

[0042] Note that the electronic devices (420, 430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[0043] FIG. 5 illustrates an exemplary block diagram of a video decoder (510). The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuit). The video decoder (510) may be used instead of the video decoder (410) in the example of FIG. 4.

[0044] A receiver (531) may receive one or more coded video sequences to be decoded by a video decoder (510). In one embodiment, one coded video sequence may be received at a time, wherein the decoding of each coded video sequence is independent of the decoding of other coded video sequences. The coded video sequences may be received from a channel (501) which may be a hardware / software link to a storage device that stores the encoded video data. The receiver (531) may receive the encoded video data along with other data, e.g., coded audio data and / or auxiliary data streams, which may be forwarded to each of them using entities (not shown). The receiver (531) may separate the coded video sequences from the other data. To prevent network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter "parser (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In other applications, it may be outside the video decoder (510) (not shown). In yet another application, the buffer memory (not shown) may be outside the video decoder (510), for example, to prevent network jitter, and additional buffer memory (515) may be inside the video decoder (510), for example, to handle playout timing. When the receiver (531) is receiving data from a storage / forwarding device or an isosynchronous network that has sufficient bandwidth and controllability, the buffer memory (515) may not be needed or may be small.For use in a best-effort packet network such as the Internet, a buffer memory (515) may be required, may be relatively large, and an adaptive size may be advantageous, and may be implemented at least partially in a similar element (not shown) outside of the operating system or video decoder (510).

[0045] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from a coded video sequence. Categories of these symbols include information used to manage the operation of the video decoder (510) and potential information for controlling a rendering device, such as a render device (512) (e.g., a display screen) that can be coupled to the electronic device (530) rather than being an integrated part of the electronic device (530), as illustrated in FIG. 5. The control information for the rendering device(s) may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may be in accordance with video coding techniques or standards and may follow various principles including variable-length coding, Huffman coding, and arithmetic coding with or without context sensitivity. The parser (520) may extract a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to that group. The subgroup may include a Group of Picture (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (520) may also extract from the coded video sequence information, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0046] The parser (520) can generate symbols (521) by performing entropy decoding / parsing operations on a video sequence received from the buffer memory (515).

[0047] The reconstruction of the symbol (521) may be associated with a number of different units depending on the type of the coded video picture or a part thereof (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors. The associated units and how they are associated may be controlled by subgroup control information parsed from the coded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the number of units below it is not shown for simplicity.

[0048] Beyond the previously mentioned functional blocks, the video decoder (510) can be conceptually subdivided into a number of functional units as described below. In actual implementations operating under commercial constraints, many of these units interact closely with one another and may be integrated with one another, at least partially. However, to clearly illustrate the subject of disclosure, the conceptual subdivision into the functional units below is appropriate.

[0049] The first unit is a scaler / inverse transform unit (551). The scaler / inverse transform unit (551) receives quantized transform coefficients as well as control information including the inverse transform to be used, block size, quantization factor, quantization scaling matrix, etc., which are symbol(s) (521) from the parser (520). The scaler / inverse transform unit (551) can output a block containing sample values ​​that can be input to an aggregator (555).

[0050] In some cases, the output sample of the scaler / inverse conversion unit (551) may be related to an intra-coded block. An intra-coded block is a block that does not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed part of the current picture. This prediction information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) uses already reconstructed peripheral information drawn from the current picture buffer (558) to generate a block of the same size and shape as the block being reconstructed. The current picture buffer (558) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. The aggregator (555) may, in some cases, add the prediction information generated by the intra-prediction unit (552) to the output sample information provided by the scaler / inverse conversion unit (551) on a sample basis.

[0051] In other cases, the output samples of the scaler / inverse unit (551) may be intercoded and potentially associated with a motion-compensated block. In such cases, the motion compensation prediction unit (553) may access the reference picture memory (557) to retrieve samples used for inter-picture prediction. After motion-compensating the retrieved samples according to the symbol (521) associated with the block, these samples may be added to the output of the scaler / inverse unit (551) (which may be referred to as residual samples or residual signals in this case) by the aggregator (555) to generate output sample information. The address in the reference picture memory (557) from which the motion compensation prediction unit (553) retrieves the prediction samples may be controlled by a motion vector and may be used by the motion compensation prediction unit (553) in the form of a symbol (521) that may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​drawn from reference picture memory (557) when the accurate motion vector of a subsample is used, motion vector prediction mechanism, etc.

[0052] The output sample of the aggregator (555) may be subject to various loop filtering techniques in the loop filter unit (556). Video compression techniques may include in-loop filter techniques that are controlled by parameters contained in the coded video sequence (also referred to as the coded video bitstream) and made available to the loop filter unit (556) as symbols (521) from the parser (520). Video compression may also respond to meta-information obtained while decoding a previous (in decoding order) portion of the coded picture or coded video sequence, as well as to previously reconstructed and loop-filtered sample values.

[0053] The output of the loop filter unit (556) may be a sample stream that can be output to the render device (512) and stored in the reference picture memory (557) for use in future inter-picture prediction.

[0054] Once a specific coded picture has been completely reconstructed, it can be used as a reference picture for later prediction. For example, when a coded picture corresponding to the current picture is completely reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) can become part of the reference picture memory (557), and the new current picture buffer can be reallocated before starting the reconstruction of the next coded picture.

[0055] A video decoder (510) can perform decoding operations according to a predetermined video compression technology or a standard such as ITU-T Rec. H.265. The coded video sequence may follow the syntax specified by the video compression technology or standard used in that it complies with both the syntax of the video compression technology or standard and the profile documented in the video compression technology. Specifically, the profile may select a specific tool as the only tool available in that profile among all tools available in the video compression technology or standard. Additionally, for compliance, the complexity of the coded video sequence must be within the range defined by the level of the video compression technology or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further limited by HRD specifications and metadata for managing the virtual reference decoder (HRD) buffer signaled in the coded video sequence.

[0056] In one embodiment, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, a temporal layer, a spatial layer or an SNR enhancement layer, a redundant slice, a redundant picture, a forward error correction code, etc.

[0057] FIG. 6 illustrates an exemplary block diagram of a video encoder (603). The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmitting circuit). The video encoder (603) can be used instead of the video encoder (403) in the example of FIG. 4.

[0058] The video encoder (603) can receive video samples from a video source (601) (which is not part of the electronic device (620) in the example of FIG. 6) capable of capturing video image(s) to be encoded by the video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0059] A video source (601) may provide a source video sequence to be coded by a video encoder (603) in the form of a digital video sample stream, which may have any appropriate bit depth (e.g., 8-bit, 10-bit, 12-bit,…), any color space (e.g., BT.601 Y CrCb, RGB,…), and any appropriate sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (601) may be a storage device capable of storing a pre-prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures that impart motion when viewed sequentially. The picture itself may consist of a spatial array of pixels, wherein each pixel may contain one or more samples depending on the sampling structure, color space, etc. being used. A person skilled in the art will easily understand the relationship between pixels and samples. The following description focuses on the sample.

[0060] According to one embodiment, the video encoder (603) can code and compress pictures of a source video sequence into a coded video sequence (643) in real time or under any other required time constraints. Enforcing an appropriate coding rate is one function of the controller (650). In some embodiments, the controller (650) controls other function units and is functionally coupled to other function units as described below. For clarity, the coupling is not shown. Parameters set by the controller (650) may include rate control-related parameters (picture skip, quantizer, lambda value of rate distortion optimization technique, …), picture size, picture group (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured to have other appropriate functions related to the video encoder (603) optimized for a specific system design.

[0061] In some embodiments, the video encoder (603) is configured to operate in a coding loop. For an oversimplified description, in one example, the coding loop may include a source coder (630) (e.g., responsible for generating symbol and reference picture(s), such as a symbol stream, based on the input picture to be coded), and a (local) decoder (633) built into the video encoder (603). The decoder (633) reconstructs the sample to generate sample data in a manner similar to how the (remote) decoder would have generated it. The reconstructed sample stream (sample data) is input into the reference picture memory (634). Since the decoding of the symbol stream leads to a bit-exact result regardless of the decoder location (local or remote), the contents of the reference picture memory (634) are also bit-exact between the local encoder and the remote encoder. In other words, the encoder's prediction part "recognizes" as reference picture samples exactly the same sample values ​​that the decoder "sees" when using prediction during decoding. This fundamental principle of reference picture synchronicity (and the resulting drift when synchronicity cannot be maintained, for example, due to channel error) is also used in some related technologies.

[0062] The operation of the “local” decoder (633) may be the same as the operation of the “remote” decoder, such as the video decoder (510), which has already been described in detail in relation to FIG. 5. Referring briefly back to FIG. 5, since the encoding / decoding of the symbols into a video sequence coded by the entropy coder (645) and the parser (420) may be lossless, the entropy decoding part of the video decoder (510), including the buffer memory (515) and the parser (520), may not be entirely implemented in the encoder of the local decoder (633).

[0063] In one embodiment, all decoder techniques, except for parsing / entropy decoding present in the decoder, exist in the corresponding encoder in the same or substantially the same functional form. Accordingly, the subject matter of the disclosure focuses on decoder operations. Therefore, a description of the encoder techniques may be omitted as it is the opposite of the comprehensively described decoder techniques. In certain areas, a more detailed description is provided below.

[0064] During operation, in some examples, the source coder (630) may perform motion-compensated predictive coding to predictively code an input picture by referencing one or more previously coded pictures from a video sequence designated as a "reference picture." In this way, the coding engine (632) codes the difference between a pixel block of the input picture and a pixel block of the reference picture(s) that can be selected as predictive reference(s) for the input picture.

[0065] The local video decoder (633) can decode the coded video data of a picture that can be designated as a reference picture based on the symbols generated by the source coder (630). The operation of the coding engine (632) may advantageously be a lossy process. When the coded video data can be decoded by a video decoder (not shown in FIG. 6), the reconstructed video sequence may generally be a replica of the source video sequence with some errors. The local video decoder (633) can replicate the decoding process that can be performed by the video decoder on the reference picture and allow the reconstructed reference picture to be stored in the reference picture memory (634). In this way, the video encoder (603) can locally store a copy of the reconstructed reference picture having common content as the reconstructed reference picture to be acquired by the far-end video decoder (no transmission errors).

[0066] The predictor (635) can perform a predictive search for the coding engine (632). That is, for a new picture to be coded, the predictor (635) can search the reference picture memory (634) for sample data (which is a candidate reference pixel block) that can serve as a suitable predictive reference for the new picture, or for specific metadata such as a reference picture motion vector, block shape, etc. The predictor (635) can operate on a sample block-by-pixel block basis to find a suitable predictive reference. In some cases, as determined by the search results obtained by the predictor (635), the input picture may have a predictive reference drawn from a number of reference pictures stored in the reference picture memory (634).

[0067] The controller (650) can manage the coding operation of the source coder (630), including the settings of parameters and subgroup parameters used to encode video data, for example.

[0068] The output of all the aforementioned function units may be subject to entropy coding in the entropy coder (645). The entropy coder (645) converts symbols generated by various function units into a coded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable-length coding, and arithmetic coding.

[0069] The transmitter (640) may buffer the coded video sequence(s) generated by the entropy coder (645) and prepare for transmission through a communication channel (660), which may be a hardware / software link to a storage device for storing the encoded video data. The transmitter (640) may merge the coded video data from the video encoder (603) with other data to be transmitted, e.g., coded audio data and / or an auxiliary data stream (source not shown).

[0070] The controller (650) can manage the operation of the video encoder (603). During coding, the controller (650) can assign a specific coded picture type to each coded picture, which can affect the coding technique that can be applied to each picture. For example, a picture can often be designated as one of the following picture types:

[0071] An Intra Picture (I Picture) may be one that can be coded and decoded without using any other picture within the sequence as a prediction source. Some video codecs allow different types of Intra Pictures, including, for example, Independent Decoder Refresh Picture ("IDR"). Those skilled in the art are aware of these variations of I Pictures and their respective applications and characteristics.

[0072] The predictive picture (P picture) may be coded and decoded using intra-prediction or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0073] A bidirectionally predictive picture (B-picture) may be coded and decoded using intra-prediction or inter-prediction, which utilize up to two motion vectors and reference indices to predict sample values ​​for each block. Similarly, a multiple-predictive picture may use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0074] A source picture is typically subdivided spatially into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks) and can be coded block by block. Blocks can be predictively coded by referencing other (already coded) blocks as determined by the coding assignment applied to each picture in the block. For example, blocks in picture I can be non-predictively coded or predictively coded by referencing already coded blocks (spatial prediction or intra prediction) of the same picture. Pixel blocks in picture P can be predictively coded via spatial prediction or temporal prediction by referencing one previously coded reference picture. Blocks in picture B can be predictively coded via spatial prediction or temporal prediction by referencing one or two previously coded reference pictures.

[0075] The video encoder (603) may perform coding operations according to a predetermined video coding technique or standard, such as ITU-T Rec. H.265. In such operations, the video encoder (603) may perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Thus, the coded video data may follow the syntax specified by the video coding technique or standard used.

[0076] In one embodiment, the transmitter (640) may transmit additional data along with the encoded video. The source coder (630) may include this data as part of the encoded video sequence. The additional data may include other forms of redundant data such as time / space / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0077] Video can be captured as a temporal sequence of multiple source pictures (video pictures). Intra-picture prediction (often abbreviated as intra prediction) utilizes spatial correlations within a given picture, while inter-picture prediction utilizes (temporal or other) correlations between pictures. For example, a specific picture currently being encoded / decoded, referred to as the current picture, can be partitioned into blocks. If a block within the current picture is similar to a reference block in a reference picture that was previously encoded and is still buffered in the video, the block within the current picture can be encoded by a vector referred to as a motion vector. The motion vector points to the reference block within the reference picture and, if multiple reference pictures are in use, may have three dimensions that identify the reference picture.

[0078] In some embodiments, a bi-prediction technique may be used for inter-picture prediction. According to the bi-prediction technique, two reference pictures are used, such as a first reference picture and a second reference picture, both of which precede the current picture in the decoding order in the video (but which may be past or future in the display order, respectively). A block of the current picture may be coded by a first motion vector pointing to a first reference block of the first reference picture and a second motion vector pointing to a second reference block of the second reference picture. A block may be predicted by a combination of the first reference block and the second reference block.

[0079] In addition, coding efficiency can be improved by using merge mode technology in inter-picture prediction.

[0080] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed in blocks. For example, according to the HEVC standard, pictures in a sequence of video pictures are partitioned into coding tree units (CTUs) for compression, and CTUs within a picture may have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU may include a coding tree block (CTB) consisting of one luminance CTB and two chroma CTBs. Each CTU may be repeatedly divided into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU may be divided into one 64×64 pixel CU, four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In one example, each CU may be analyzed to determine a prediction type for the CU, such as an inter-prediction type or an intra-prediction type. A CU may be divided into one or more prediction units (PUs) based on temporal and / or spatial predictability. Typically, each PU includes a luminal prediction block (PB) and two chroma PBs. In one embodiment, the prediction operation in coding (encoding / decoding) is performed on a unit of prediction blocks. If a luminal prediction block is used as an example of a prediction block, the prediction block includes a matrix of values ​​(e.g., luminal values) for pixels, such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0081] FIG. 7 illustrates an exemplary drawing of a video encoder (703). The video encoder (703) is configured to receive a processing block (e.g., a prediction block) of sample values ​​within the current video picture in a sequence of video pictures, and to encode the processing block into a coded picture that is part of a coded video sequence. In one example, the video encoder (703) may be used instead of the video encoder (403) in the example of FIG. 4.

[0082] In the example of HEVC, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a prediction block of 8×8 samples. The video encoder (703) determines whether the processing block is best coded using an intra-mode, inter-mode, or positive-prediction mode, for example, using rate-distortion optimization. If it is determined that the processing block is coded in intra-mode, the video encoder (703) may use an intra-prediction technique to encode the processing block into a coded picture; if it is determined that the processing block is coded in inter-mode or positive-prediction mode, the video encoder (703) may use an inter-prediction or positive-prediction technique, respectively, to encode the processing block into a coded picture. In a particular video coding technique, the merge mode may be a submode of inter-picture prediction in which motion vectors are derived from one or more motion vector predictors without the benefit of the coded motion vector components outside the predictor. In certain other video coding techniques, there may be motion vector components applicable to a subject block. In one example, the video encoder (703) may include components such as a mode determination module (not shown) for determining the prediction mode of a processing block.

[0083] In the example of FIG. 7, the video encoder (703) includes an inter-encoder (730), an intra-encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725) combined together as shown in FIG. 7.

[0084] The inter-encoder (730) is configured to receive a sample of the current block (e.g., processing block), compare the block with one or more reference blocks within a reference picture (e.g., blocks within a previous picture and a subsequent picture), generate inter-predicted information (e.g., description of redundant information according to inter-encoding techniques, motion vectors, merge mode information), and calculate an inter-predicted result (e.g., predicted block) based on the inter-predicted information using any suitable technique. In some examples, the reference picture is a decoded reference picture that is decoded based on encoded video information.

[0085] The intra-encoder (722) is configured to receive a sample of the current block (e.g., processing block), compare the block with a block already coded in the same picture, and generate quantized coefficients after conversion, and, in some cases, also intra-prediction information (e.g., intra-prediction direction information according to one or more intra-encoding techniques). In one example, the intra-encoder (722) also calculates an intra-prediction result (e.g., predicted block) based on the intra-prediction information and reference block of the same picture.

[0086] A general controller (721) is configured to determine general control data and to control other components of the video encoder (703) based on the general control data. In one example, the general controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, if the mode is an intra mode, the general controller (721) controls the switch (726) to select an intra mode result for use in the residual calculator (723) and controls the entropy encoder (725) to select intra prediction information and include the intra prediction information in the bit stream; if the mode is an inter mode, the general controller (721) controls the switch (726) to select an inter prediction result for use in the residual calculator (723) and controls the entropy encoder (725) to select inter prediction information and include the inter prediction information in the bit stream.

[0087] The residual calculator (723) is configured to calculate the difference (residual data) between the received block and the prediction result selected from the intra-encoder (722) or the inter-encoder (730). The residual encoder (724) is configured to operate based on the residual data to encode the residual data and generate transformation coefficients. In one example, the residual encoder (724) is configured to convert the residual data from the spatial domain to the frequency domain and generate transformation coefficients. The transformation coefficients then undergo quantization processing to obtain quantized transformation coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transformation and generate decoded residual data. The decoded residual data can be appropriately used by the intra-encoder (722) and the inter-encoder (730). For example, the inter encoder (730) can generate a decoded block based on decoded residual data and inter prediction information, and the intra encoder (722) can generate a decoded block based on decoded residual data and intra prediction information. The decoded block is appropriately processed to generate a decoded picture, and the decoded picture can be buffered in a memory circuit (not shown) and, in some examples, used as a reference picture.

[0088] The entropy encoder (725) is configured to format the bitstream to include the encoded block. The entropy encoder (725) is configured to include various information in the bitstream according to an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra prediction information or inter prediction information), residual information, and other appropriate information in the bitstream. According to the subject matter disclosed, it is noted that when coding a block in an inter mode or a merged submode of both prediction modes, there may be no residual information.

[0089] FIG. 8 illustrates a diagram of an exemplary video decoder (810). The video decoder (810) is configured to receive a coded picture that is part of a coded video sequence and to decode the coded picture to produce a reconstructed picture. In one example, the video decoder (810) may be used instead of the video decoder (410) in the example of FIG. 4.

[0090] In the example of FIG. 8, the video decoder (810) includes an entropy decoder (871), an inter decoder (880), a residual decoder (873), a reconstruction module (874), and an intra decoder (872) combined together as illustrated in FIG. 8.

[0091] The entropy decoder (871) may be configured to reconstruct, from a coded picture, specific symbols representing syntax elements constituting the coded picture. These symbols may include, for example, the mode in which the block is coded (e.g., intra mode, inter mode, positive-prediction mode, the latter two being merge submodes or other submodes), and prediction information (e.g., intra prediction information or inter prediction information) capable of identifying specific samples or metadata used for prediction by the intra decoder (872) or the inter decoder (880), respectively. The symbols may also include residual information, for example, in the form of quantized transform coefficients. In one example, if the prediction mode is an inter mode or positive-prediction mode, the inter prediction information is provided to the inter decoder (880); if the prediction type is an intra prediction type, the intra prediction information is provided to the intra decoder (872). The residual information may be dequantized and provided to the residual decoder (873).

[0092] The inter decoder (880) is configured to receive inter prediction information and generate an inter prediction result based on the inter prediction information.

[0093] The intra decoder (872) is configured to receive intra prediction information and generate a prediction result based on the intra prediction information.

[0094] The residual decoder (873) is configured to perform inverse quantization to extract inverse quantized transform coefficients and process the inverse quantized transform coefficients to convert residual information from the frequency domain to the spatial domain. The residual decoder (873) may also require specific control information (to include quantizer parameters (QP)), and that information may be provided by the entropy decoder (871) (the data path is not shown as it may contain only a small amount of control information).

[0095] The reconstruction module (874) is configured to form a reconstructed block in the spatial domain by combining residual information output by the residual decoder (873) and a prediction result (output by an inter-prediction module or an intra-prediction module, depending on the case), and the reconstructed block may be part of a reconstructed picture, and the reconstructed picture may consequently be part of a reconstructed video. Note that other appropriate actions, such as deblocking, may be performed to improve visual quality.

[0096] It should be noted that the video encoder (403, 603, 703) and video decoder (410, 510, 810) may be implemented using any suitable technology. In one embodiment, the video encoder (403, 603, 703) and video decoder (410, 510, 810) may be implemented using one or more integrated circuits. In another embodiment, the video encoder (403, 603, 703) and video decoder (410, 510, 810) may be implemented using one or more processors that execute software instructions.

[0097] The present disclosure includes embodiments related to deriving and propagating an intra-predicted mode for an intra-block copy (IBC) block by using an intra-copy block reconstructed through a block vector (BV).

[0098] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) released the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (Version 1), 2014 (Version 2), 2015 (Version 3), and 2016 (Version 4). In 2015, the two standardization organizations jointly formed the Joint Video Exploration Team (JVET) to explore the possibility of developing a next-generation video coding standard beyond HEVC. In October 2017, the two standardization organizations released a Joint Request for Proposals (CfP) for video compression with capabilities beyond HEVC. By February 15, 2018, 22 CfP responses for Standard Dynamic Range (SDR), 12 CfP responses for High Dynamic Range (HDR), and 12 CfP responses for 360 video categories were submitted, respectively. In April 2018, all received CfP responses were evaluated at the 122nd MPEG / 10th JVET meeting. As a result of this meeting, the JVET officially initiated the standardization process for next-generation video coding beyond HEVC; the new standard was named Versatile Video Coding (VVC), and the JVET was renamed the Joint Video Experts Team. In 2020, the ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) released the VVC video coding standard (Version 1).

[0099] In inter-prediction, for each inter-predicted coding unit (CU), motion parameters are required, for example, for the coding features of the VVC to be used to generate the inter-predicted sample. Motion parameters may include motion vectors, reference picture indices, reference picture list usage indices, and / or additional information. Motion parameters may be signaled explicitly or implicitly. If a CU is coded in skip mode, the CU may be associated with a single PU, and significant residual coefficients, coded motion vector deltas, and / or reference picture indices may not be required. If a CU is coded in merge mode, motion parameters for the CU may be obtained from neighboring CUs. Neighboring CUs may include spatial and temporal candidates, and additional schedules (or additional candidates), such as those introduced in the VVC. Merge mode may be applied to all inter-predicted CUs, not just skip mode. An alternative to the merge mode is the explicit transmission of motion parameters, whereby motion vectors, corresponding reference picture indices for each reference picture list, reference picture list usage flags, and / or other necessary information may be explicitly signaled per CU.

[0100] In VVC, the VTM (VVC Test model) reference software may include a number of new and refined inter-predictive coding tools, which may include one or more of the following:

[0101] (1) Extended merge prediction

[0102] (2) Merge motion vector difference (MMVD)

[0103] (3) AMVP mode using symmetric MVD signaling

[0104] (4) Affine motion compensated prediction

[0105] (5) Subblock-based temporal motion vector prediction (SbTMVP)

[0106] (6) Adaptive motion vector resolution (AMVR)

[0107] (7) Motion field storage: 1 / 16 luma sample MV storage and 8×8 motion field compression

[0108] (8) Bi-prediction with CU-level weights (BCW)

[0109] (9) Bi-directional optical flow (BDOF)

[0110] (10) Decoder side motion vector refinement (DMVR)

[0111] (11) Combined inter and intra prediction (CIIP)

[0112] (12) Geometric partitioning mode (GPM)

[0113] The merge candidate list can be constructed to include five types of candidates, as in VTM 4. The merge candidate list can be constructed in the following order:

[0114] 1) Space MVP from Space Neighbor CU,

[0115] 2) Collocated CU Time MVP,

[0116] 3) History-based MVP of the FIFO table,

[0117] 4) Pairwise Average MVP and

[0118] 5) 0(zero) MV.

[0119] The size of the merge list can be signaled in the slice header. The maximum allowable size of the merge list can be 6, as in VTM 4. For each CU coded in merge mode, the index of the best merge candidate can be encoded, for example, using truncated unary binarization. The first bin of the merge index can be coded as context, and bypass coding can be used for other bins.

[0120] In deriving spatial candidates, for example in VVC, the derivation of spatial merge candidates may be the same or similar as the derivation of spatial merge candidates in HEVC. The maximum number of merge candidates (e.g., 4 merge candidates) may be selected from among the candidates located at the positions shown in FIG. 9, for example. As shown in FIG. 9, the current block (901) may include neighboring blocks (902) through (906) located at positions A0, A1, B0, B1, and B2, respectively. The order of deriving spatial merge candidates is B 1, A 1, B 0, A 0,And it can be B2. Position B2 may be considered only if any CU (or block) of positions A0, B0, B1, or A1 is unavailable (e.g., because the CU belongs to a different slice or tile) or is intra-coded. After a candidate (or block) of position A1 is added, the addition of the remaining merge candidates (or blocks) may undergo a duplicate check. The duplicate check can ensure improved coding efficiency by excluding candidates with identical movement information from the merge list. To reduce computational complexity, the duplicate check may not be considered for all possible candidate pairs. Instead, only candidate pairs connected by arrows in Fig. 10 may be considered. For example, the duplicate check may be applied to five candidate pairs, such as the candidate pair of A1 and B1 and the candidate pair of A1 and A0. A candidate may be added to the merge list only if the corresponding candidate used for the duplicate check does not contain identical movement information. For example, candidate B0 may be added to the merge list only if the corresponding candidate B1 does not contain identical movement information.

[0121] In the temporal candidate derivation, only one candidate may be added to the merge list. For example, as illustrated in FIG. 11, in the temporal merge candidate derivation for the current CU (1114), a scaled motion vector may be derived based on the co-located CU (1104) belonging to the co-located reference picture (1112). The reference picture list used for deriving the co-located CU (1104) may be explicitly signaled in the slice header. The scaled motion vector for the temporal merge candidate may be obtained as illustrated by the dashed line (1102) in FIG. 11, which is scaled from the motion vector of the co-located CU (1104) using picture order count (POC) distances tb and td. tb can be defined as the difference in POC between the current picture (e.g., Curr_ref) (1106) and the reference picture of the current picture (e.g., Curr_pic) (1108). td can be defined as the difference in POC between the juxtaposed picture (e.g., Col_ref) (1110) and the reference picture (1112) of the juxtaposed picture (e.g., Col_pic). The reference picture index of the time merge candidate can be set to 0.

[0122] The location for the time candidate can be selected between candidates C0 and C1 as shown in Fig. 12. For example, if the CU at location C0 cannot be used, is intra-coded, or is outside the current row of the CTU, location C1 can be used. Otherwise, location C0 can be used to derive the time merge candidate.

[0123] GPM can be applied to inter-prediction. GPM can be applied only to CUs of a specific size, for example, 8×8 or larger than 8×8. GPM can be signaled using, for example, CU level flags and operates as a type of merge mode. Other merge modes may include normal merge mode, MMVD mode, CIIP mode, and / or subblock merge mode.

[0124] When a GPM is used, the CU can be divided into two geometric partitions using one of a plurality of partitioning schemes. The partitioning schemes can be distinguished by angles and / or edges. In one embodiment, 64 different partitioning schemes can be applied to the GPM. The 64 different partitioning schemes can be divided into 24 angles that are non-uniformly quantized, ranging from 0° to 360° with respect to the center of the CU, up to 4 edges. FIG. 13 illustrates 24 exemplary angles applied to the GPM. FIG. 14 illustrates four exemplary possible partition edges associated with an angle of index 3 in the CU (1402), where each partition edge can be associated with its own distance index. The distance index may indicate a distance from the center of the CU (1402). In the GPM, each geometric partition in the CU can be inter-predicted using its own motion vector. Additionally, only uni-prediction may be allowed for each partition. For example, each partition may have one motion vector and one reference index. The constraint that only single-predicted motions are allowed for each partition can ensure that only two motion-compensated predictions are required for each CU, which also applies to conventional positive-predicted motions.

[0125] When GPM is used for the current CU, a signal including a geometric partition index and two merge indices (one per partition) may be additionally signaled. The maximum GPM candidate size may be explicitly signaled, for example, at the slice level. The maximum GPM candidate size may specify syntax binarization for the GPM merge indices. After each of the two geometric partitions is predicted, sample values ​​may be adjusted along the geometric partition edges using an adaptive weighted blending process. Thus, the prediction signal for the entire CU may be generated after the blending process. Transformation and quantization processes may be additionally applied to the entire CU in different prediction modes. Additionally, the motion field of the CU predicted using GPM may be stored.

[0126] The implementation cost of IBC can be high for the entire area of ​​the picture that has already been reconstructed. For example, a disadvantage of the IBC concept in HEVC is that additional memory is required for the decoded picture buffer (DPB). Therefore, external memory is typically employed in the hardware implementation of IBC. Accessing additional external memory may entail an increase in memory bandwidth.

[0127] In certain implementations, it may be necessary to reduce implementation costs. For example, VVC can significantly reduce memory bandwidth requirements and hardware complexity by using fixed memory, which enables IBC to be realized using on-chip memory. Additionally, the block vector (BV) coding of IBC can employ the concept of a merged list for inter-prediction. The IBC list construction process can consider BVs of two spatial neighbors and five history-based BVs (HBVPs), where when the first HBVP is added to the candidate list (or IBC list), only the first HBVP can be compared with spatial candidates (e.g., BVs of two spatial candidates).

[0128] As in VVC, template matching (TM) can be used to refine motion on the decoder side to further improve compression efficiency. In TM mode, motion is refined by constructing a template from neighboring reconstructed samples on the left and top sides, and the closest match between the template of the current picture and the reference frame can be determined.

[0129] TM can be applied to GPM. When CU is coded as GPM, it can be determined whether TM is applied to refine each movement for geometric partitions. If TM is selected, a template can be constructed using neighbor samples from the left and top, and the movement can be further refined by finding the best match between the current template and a reference frame having the same template pattern. The refined movement can be used to perform movement compensation for geometric partitions and can be additionally stored in the movement field.

[0130] GPM can be applied to support inter and intra prediction to improve coding performance beyond VVC. For example, in a CU where GPM is applied, a predefined intra prediction mode for (or in relation to) the geometric partition line can be selected in addition to the motion vectors from the merge candidate list for each geometric partition. The intra prediction mode or inter prediction mode can be determined for each geometric partition based on a flag. If inter prediction is selected, a single-prediction signal can be generated by the MV from the merge candidate list. Otherwise, if the intra prediction mode is selected, a single-prediction signal can be generated from neighbor samples predicted from a specific index of the intra prediction mode. Possible variations of the intra prediction mode may be limited by the geometric shape. Finally, the two single-prediction signals can be mixed in the same manner as in a conventional GPM.

[0131] To reduce complexity and signaling overhead, variations of possible intra prediction modes may be studied (or defined). For example, the effects of variations of possible intra prediction modes on a GPM with inter and intra predictions were studied for two exemplary configurations. In the first configuration, only parallel and perpendicular intra modes with respect to (or in relation to) geometric partition lines were attempted. In addition to parallel and perpendicular intra angle modes with respect to (or in relation to) geometric partition lines, planar modes were also tested in the second configuration. Two or three possible intra prediction modes were tested for geometric partitions of the GPM with inter and intra predictions.

[0132] In template-based intra mode derivation (TIMD), e.g., JVET-V0098, for each intra mode of the MPM, the sum of absolute transformed differences (SATD) between the predicted sample and the reconstructed sample of the template region of the current CU can be calculated, and the intra mode with the smallest SATD cost for the current CU can be selected. Instead of selecting only the single intra mode with the smallest SATD cost, the first two modes with the smallest SATD costs derived by TIMD, e.g., JVET-W0123, can be selected for the current CU. The two selected modes can be further fused using weights, and the weighted intra prediction can be used as the intra predictor for the current CU. Fusion can be applied if the cost of the selected intra mode with the smallest SATD cost among the two selected modes is greater than half the cost of the other selected intra mode. Otherwise, only the intra mode with the smallest SATD cost can be selected. By applying fusion, weights can be derived from the SATD costs of the two modes.

[0133] When generating an MPM list, if a neighboring CU is encoded as an inter-CU (i.e., a CU coded in an inter-predicted mode), the intra-predicted mode of the neighboring CU can be derived by default as a Planar mode. To improve the accuracy of the MPM list, an intra-mode propagation method for inter-CUs can be applied to the construction of the MPM list. For example, in JVET-V0098, the intra-predicted mode can be propagated to the inter-CU through motion vectors and reference pictures associated with the inter-CU, and the propagated intra-predicted mode of the inter-CU can be used to construct the MPM list. It should be noted that intra-mode propagation for inter-CUs can only be applied to the derivation of TIMD modes.

[0134] To improve the accuracy of constructing the MPM list, the propagated intra mode of neighbor inter-CUs can be adopted for the TIMD mode. The propagated intra mode of neighbor inter-CUs can be derived from the reference picture and motion vector of neighbor inter-CUs. However, if the neighbor CUs are encoded in the IBC prediction mode, the planar mode can still be used for constructing the MPM list.

[0135] In the present disclosure, an intra prediction mode can be derived using the block vector (BV) of an IBC, and intra prediction mode information can be stored for intra mode propagation. For example, when the current block is predicted by the IBC, the propagated intra mode of the current block can be derived from the intra mode of the reference block of the current block. The reference block can be indicated (or referenced) by the BV of the IBC. The propagated intra mode of the current block can be used to construct an MPM list for the neighbor blocks of the current block when the neighbor blocks are IBC blocks (or predicted by the IBC). Thus, when neighbor blocks are IBC coded, the intra prediction mode of the reference block of the current block can be a candidate for constructing the MPM list of the neighbor blocks.

[0136] In one embodiment, the propagated intramode may be stored for the minimum block (also referred to as the minimum subplot or subblock) of the current block being IBC-coded, such as a 4×4 luma block (or subblock) of the current block. Thus, each subblock of the current block may have its own propagated intramode stored. The propagated intramode of the reference block of the current block may be derived from the intramode of the reference block of the current block. The reference block may be indicated (or referenced) by the BV of the IBC.

[0137] In one embodiment, intra-mode information of an IBC block (or a block predicted by the IBC) may be propagated from a reference block referenced by the BV of the IBC. FIG. 15 illustrates an example of intra-predicted mode propagation for an IBC block through a BV. As illustrated in FIG. 15, the BV (1502) may locate the reference block (1506) of the current block (1504) in a reconstructed picture (not shown) applied to the IBC block (or current block) (1504). To obtain the intra-predicted mode from the reference block (1506), the reference block (1506) referenced by the BV (1502) may be divided into a plurality of minimum units (or sub-blocks), such as 4×4 units. For each minimum unit (e.g., (1508)) in the reference block (1506), the respective intra-predicted mode may be derived from a sample located in that respective minimum unit (or respective sub-block). The sample may be located at the center or corner position of the minimum unit (or subblock). For example, the sample may be the top-left sample of the minimum unit. The intra-prediction mode of the minimum unit of the reference block may be additionally stored when the reference block is encoded. The stored intra-prediction mode of the minimum unit of the reference block (1506), which may be a parallel CU of the current block (1504), may be propagated to the current block (1504).

[0138] Referring still to FIG. 15, in some embodiments, the current block (1504) may be divided into a plurality of sub-blocks (e.g., (1510)). Each of the plurality of sub-blocks may have the same size as the minimum unit of the reference block (1506). Thus, each sub-block of the current block (1504) may have a corresponding minimum unit in the reference block (1506). The location of the sub-block within the current block may be the same as the location of the corresponding minimum unit (or sub-block) within the reference block. For example, the sub-block (1510) is in row 1, column 3 of the current block (1504), and the minimum unit (1508) is in row 1, column 3 of the reference block (1506). Thus, the sub-block (1510) of the current block (1504) may correspond to the minimum unit (1508) of the reference block (1506). Accordingly, each sub-block of the current block (1504) may have its own propagated intra-mode derived from the minimum unit of the reference block (1506) corresponding to each sub-block. For example, a sub-block (1510) may have a propagated intra-mode derived from the intra-mode of the minimum unit (1508) of the reference block (1506).

[0139] It should be noted that an intra prediction mode (IPM) map (1512) may be formed based on a reconstructed picture (not shown). The IMP map (1512) may include a plurality of subregions (e.g., (1514)). Each subregion may correspond to its own intra mode associated with the reconstructed picture (not shown).

[0140] In one embodiment, the central position of each minimum unit (e.g., 4×4 unit) within the reference block (1506) may be applied to derive a juxtaposed corresponding intra-prediction mode for the current block (1504). Thus, the intra-prediction mode for a sub-block (e.g., (1510)) of the current block (1504) may be determined by the intra-mode of a pixel (or sample) located at the central position of the minimum unit (e.g., (1508)) of the reference block (1506) corresponding to the sub-block. The central position of the minimum unit (or 4×4 unit) may be one of row 2, column 2 of the minimum unit, row 2, column 3 of the minimum unit, row 3, column 2 of the minimum unit, or row 3, column 3 of the minimum unit. An exemplary drawing of the central position may be the minimum unit (1508) shown in FIG. 15. As illustrated in the minimum unit (1508), the central position may be one of the positions labeled (1,1), (1,2), (2,1), or (2,2). The position labeled (1,1) may correspond to the central position of row 2, column 2 of the minimum unit (1508). The position labeled (2,1) may correspond to the central position of row 2, column 3 of the minimum unit (1508). The position labeled (1,2) may correspond to the central position of row 3, column 2 of the minimum unit (1508). The position labeled (2,2) may correspond to the central position of row 3, column 3 of the minimum unit (1508). Thus, the propagated intra-prediction mode for the sub-block (1510) can be derived from the intra-prediction mode of the pixel at the central position of the minimum unit (1508).

[0141] In one embodiment, corner positions such as the top-left, top-right, bottom-left, or bottom-right positions of each minimum unit (or 4×4 unit) (1508) may be applied to the derivation of a juxtaposed corresponding intra prediction mode. As illustrated in FIG. 15, the top-left position may be labeled (0,0), the top-right position may be labeled (3,0), the bottom-left position may be labeled (0,3), and the bottom-right position may be labeled (3,3). Thus, the propagated intra prediction mode for the subblock (1510) may be derived from the intra prediction mode of a pixel at one of the corner positions of the minimum unit (1508).

[0142] In one embodiment, the intra prediction mode of the reference block may be propagated only to the subblocks of the current block within the geometric partition in IBC mode when the encoded CU (or current block) is encoded in the GPM in IBC and intra modes. As illustrated in FIG. 16, the current block (1604) may be partitioned by the GPM into a first partition (1614) and a second partition (1616). The first partition (1614) may be predicted by intra prediction, and the second partition (1616) may be predicted by IBC. Thus, the intra prediction mode of the reference block (1606) of the current block (1604) may be propagated only to the subblocks of the second partition (1616). Thus, the intra mode of the minimum unit (1608) of the reference block (1606) may be propagated to the corresponding subblock (1610) of the second partition (1616) in the current block (1604). The intra mode of the minimum unit (1612) in the reference block (1606) may not be propagated to the corresponding sub-block (1618) of the first partition (1614) within the current block (1604).

[0143] FIG. 17 illustrates a flowchart schematically illustrating an exemplary decoding process (1700) according to some embodiments of the present disclosure. FIG. 18 illustrates a flowchart schematically illustrating an exemplary encoding process (1800) according to some embodiments of the present disclosure. The proposed processes may be used individually or in combination in any order. Additionally, each process (or embodiment), encoder, and decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a computer-readable non-transient medium.

[0144] The operations of the process (e.g., (1700) and (1800)) may be combined or arranged in any quantity or order as needed. In the embodiments, two or more of the operations of the process (e.g., (1700) and (1800)) may be performed in parallel.

[0145] The process (e.g., (1700) and (1800)) may be used for the reconstruction and / or encoding of a block to generate a prediction block for the block being reconstructed. In various embodiments, the process (e.g., (1700) and (1800)) is executed by a processing circuit, such as a processing circuit of terminal devices (310), (320), (330) and (340), a processing circuit that performs the function of a video encoder (403), a processing circuit that performs the function of a video decoder (410), a processing circuit that performs the function of a video decoder (510), a processing circuit that performs the function of a video encoder (603), etc. In some embodiments, the process (e.g., (1700) and (1800)) is implemented as a software instruction, and thus when the processing circuit executes the software instruction, the processing circuit performs the process (e.g., (1700) and (1800)).

[0146] As illustrated in FIG. 17, the process (1700) can proceed from (S1701) to (S1710). In (S1710), coded information of the current block within the current picture of the video can be received from the coded video bitstream. The coded information may indicate that the current block is an IBC block.

[0147] In (1720), an intra prediction mode for a subblock of the current block can be derived based on a reference subblock within a reference block. The reference subblock may correspond to a subblock of the current block, and the reference block may be referenced by a block vector of an IBC block.

[0148] In (1730), an intra prediction mode for a subblock of the current block can be additionally stored.

[0149] In (1740), the neighbor block of the current block can be determined as the IBC block.

[0150] In (1750), a list of MPMs for neighboring blocks of the current block can be constructed using the stored intra prediction mode.

[0151] The intra prediction mode for a subblock can be derived as the intra mode of a pixel located at the center position of the reference subblock of the reference block, and the center position may be at row 2, column 2 of the reference subblock.

[0152] The intra prediction mode for a subblock can be derived as an intra mode for a pixel located at the center position of the reference subblock of the reference block, and the center position may be at row 2, column 3 of the reference subblock.

[0153] The intra prediction mode for a subblock can be derived as an intra mode for a pixel located at the center position of the reference subblock of the reference block, and the center position may be at row 3, column 2 of the reference subblock.

[0154] The intra prediction mode for a subblock can be derived as an intra mode for a pixel located at the center position of the reference subblock of the reference block, and the center position may be at row 3, column 3 of the reference subblock.

[0155] The intra prediction mode for a subblock can be derived as the intra mode of a pixel located at one of the top-right, top-left, bottom-left, or bottom-right positions of the reference subblock of the reference block.

[0156] In some embodiments, the current block may be partitioned into a first partition and a second partition based on a geometric partition mode (GPM). The first partition may be predicted by an IBC, and the second partition may be predicted by an intra prediction. A subblock may be located in the first partition predicted by the IBC.

[0157] In some embodiments, the subblocks of the current block may be 4×4 pixel units.

[0158] In some embodiments, the intra prediction mode for a subblock may be determined as a candidate prediction mode from a list of MPMs for neighbor blocks of the current block, where the neighbor blocks may be predicted by an IBC.

[0159] After (S1750), the process proceeds to (S1799) and terminates.

[0160] The process (1700) can be appropriately adapted. The step(s) of the process (1700) can be modified and / or omitted. Additional step(s) can be added. Any suitable order of implementation can be used.

[0161] As illustrated in FIG. 18, the process (1800) can proceed from (S1801) to (S1810). In (S1810), an intra-prediction mode for a subblock of the current block in the current picture can be derived based on a reference subblock within a reference block. The reference subblock may correspond to a subblock of the current block. The current block may be an IBC block. The reference block may be referenced by a block vector of the IBC block.

[0162] In step (S1820), the intra prediction mode for the subblock of the current block can be stored.

[0163] In step (S1830), the neighbor block of the current block can be determined as the IBC block.

[0164] In step (S1840), a list of MPMs for neighboring blocks of the current block can be constructed using the stored intra prediction mode.

[0165] In (S1850), coding information for the current block can be generated. The coding information may indicate that the current block is predicted by the IBC, and the intra-predicted information of the subblocks of the current block is based on the corresponding reference subblocks of the reference blocks of the current block.

[0166] Then, proceed to process (S1899) and terminate.

[0167] The process (1800) can be appropriately adapted. The step(s) of the process (1800) can be modified and / or omitted. Additional step(s) can be added. Any appropriate order of implementation can be used.

[0168] The foregoing technology may be implemented as computer software that uses computer-readable instructions and can be physically stored on one or more computer-readable media. For example, FIG. 19 illustrates a computer system (1900) suitable for implementing a specific embodiment of the disclosed subject matter.

[0169] Computer software may be coded using any suitable machine code or computer language capable of generating code containing instructions that can be executed directly, or through interpretation, micro-code execution, etc., by a computer central processing unit (CPU), graphics processing unit (GPU), etc., via assembly, compilation, linking, or similar mechanisms.

[0170] The instruction can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0171] The components of the computer system (1900) illustrated in FIG. 19 are essentially exemplary and are not intended to imply any limitation to the scope of use or function of the computer software implementing the embodiments of the present disclosure. The configuration of the components should not be interpreted as having any dependency or requirement related to any one or combination of the components shown in the exemplary embodiments of the computer system (1900).

[0172] The computer system (1900) may include a specific human interface input device. This human interface input device may respond to input by one or more human users, for example, tactile input (e.g., keystroke, swip, data glove movement), audio input (e.g., voice, clap), visual input (e.g., gesture), and olfactory input (not shown). The human interface device may also be used to capture specific media that are not necessarily directly related to conscious input by a person, such as audio (e.g., voice, music, ambient sound), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0173] The input human interface device may include one or more of a keyboard (1901), a mouse (1902), a trackpad (1903), a touchscreen (1910), a data glove (not shown), a joystick (1905), a microphone (1906), a scanner (1907), and a camera (1908) (each only one is shown).

[0174] The computer system (1900) may include a specific human interface output device. This human interface output device may stimulate the senses of one or more human users, for example, through tactile output, sound, light and smell / taste. These human interface output devices may include haptic output devices (e.g., touch screen (1910), data gloves (not shown), or haptic feedback via a joystick (1905), but there may also be haptic feedback devices that do not serve as input devices), audio output devices (e.g., speakers (1909), headphones (not shown)), visual output devices (e.g., screens (1910) including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without a touch screen (1910) input function and with or without a haptic feedback function—some of which may produce two-dimensional visual output or three-dimensional or higher output through means such as stereographic output, virtual-reality glasses (not shown), holographic display and smoke tank (not shown)—), and printers (not shown).

[0175] The computer system (1900) may also include human-accessible storage devices and associated media, such as optical media including a CD / DVD ROM RW (1920) having a CD / DVD media (1921), a thumb drive (1922), a removable hard drive or solid-state drive (1923), legacy magnetic media such as tape and floppy disk (not shown), and specialized ROM / ASIC / PLD-based devices such as a security dongle (not shown).

[0176] Those skilled in the art should also understand that the term “computer-readable medium” as used in connection with the subject matter disclosed herein does not include a transmitting medium, a carrier wave, or other transient signals.

[0177] The computer system (1900) may also include an interface (1954) to one or more communication networks (1955). The network may be, for example, a wireless, wired, optical network. The network may also be a local, wide-area, metropolitan, automotive and industrial, real-time, latency-tolerant network. Examples of networks include cellular networks including Ethernet, wireless LAN, GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, automotive and industrial networks including CANBus, etc. A specific network generally requires an external network interface adapter attached to a specific general-purpose data port or peripheral bus (1949) (e.g., a USB port of the computer system (1900)); others are generally integrated into the core of the computer system (1900) by attaching to a system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1900) can communicate with another network. This communication may be unidirectional, receive-only (e.g., TV broadcasting), unidirectional transmit-only (e.g., from a CANbus to a specific CANbus device), or bidirectional (e.g., to another computer system using a local or wide-area digital network). Specific protocols and protocol stacks may be used for each of the networks and network interfaces as described above.

[0178] The aforementioned human interface device, human-accessible storage device, and network interface can be attached to the core (1940) of the computer system (1900).

[0179] The core (1940) may include one or more central processing units (CPUs) (1941), graphics processing units (GPUs) (1942), specialized programmable processing units in the form of a field programmable gate array (FPGA) (1943), hardware accelerators (1944) for specific tasks, graphics adapters (1950), etc. Along with read-only memory (ROM) (1945), random access memory (1946), and internal mass storage devices (1947) such as internal hard drives or SSDs that are not accessible to the user, these devices may be connected via a system bus (1948). In some computer systems, the system bus (1948) may be accessible in the form of one or more physical plugs that enable expansion by additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (1948) or via a peripheral bus (1949). For example, the screen (1910) can be connected to a graphics adapter (1950). Architectures for peripheral buses include PCI, USB, etc.

[0180] The CPU (1941), GPU (1942), FPGA (1943), and accelerator (1944) can be combined to execute specific instructions that can construct the aforementioned computer code. The computer code may be stored in ROM (1945) or RAM (1946). Transitional data may also be stored in RAM (1946), while persistent data may be stored, for example, in an internal mass storage device (1947). Fast storage and retrieval of any of the memory devices may be made possible through the use of a cache memory that may be closely associated with one or more CPUs (1941), GPUs (1942), mass storage devices (1947), ROM (1945), RAM (1946), etc.

[0181] A computer-readable medium may have computer code for performing various computer-implemented operations. The medium and the computer code may be specially designed and constructed for the purposes of this disclosure, or they may be of a type well known and available to those skilled in the art of computer software.

[0182] As an example, not limited to, the architecture (1900), specifically the core (1940), may provide functionality as a result of processor(s) (including CPU, GPU, FPGA, accelerator, etc.) executing software implemented on one or more types of computer-readable media. Such computer-readable media may be media associated with a mass storage device accessible to a user as described above, as well as specific storage devices of the core (1940) of a non-transient nature, such as a mass storage device (1947) or ROM (1945) within the core. Software implementing various embodiments of the present disclosure may be stored on such devices and executed by the core (1940). The computer-readable media may include one or more memory devices or chips as needed. Software may enable the core (1940) and, in particular, the internal processor (including a CPU, GPU, FPGA, etc.) to execute a specific process or a specific part of a specific process described herein, including defining data structures stored in RAM (1946) and modifying such data structures according to a process defined by the software. Additionally or alternatively, the computer system may provide a function that is otherwise implemented in a circuit (e.g., accelerator (1944)) as a result of logic hardwired, which can operate instead of or with the software to execute a specific process or a specific part of a specific process described herein. References to software may include logic, and where appropriate, vice versa. References to computer-readable media may include circuits that store software for execution (e.g., integrated circuits (ICs)), circuits that implement logic for execution, or both, where appropriate. The present disclosure includes any suitable combination of hardware and software.

[0183] [부록 A: 약어]

[0184] JEM: joint exploration model

[0185] VVC: versatile video coding

[0186] BMS: benchmark set

[0187] MV: Motion Vector

[0188] HEVC: High Efficiency Video Coding

[0189] SEI: Supplementary Enhancement Information

[0190] VUI: Video Usability Information

[0191] GOPs: Groups of Pictures

[0192] TUs: Transform Units,

[0193] PUs: Prediction Units

[0194] CTUs: Coding Tree Units

[0195] CTBs: Coding Tree Blocks

[0196] PBs: Prediction Blocks

[0197] HRD: Hypothetical Reference Decoder

[0198] SNR: Signal Noise Ratio

[0199] CPUs: Central Processing Units

[0200] GPUs: Graphics Processing Units

[0201] CRT: Cathode Ray Tube

[0202] LCD: Liquid-Crystal Display

[0203] OLED: Organic Light-Emitting Diode

[0204] CD: Compact Disc

[0205] DVD: Digital Video Disc

[0206] ROM: Read-Only Memory

[0207] RAM: Random Access Memory

[0208] ASIC: Application-Specific Integrated Circuit

[0209] PLD: Programmable Logic Device

[0210] LAN: Local Area Network

[0211] GSM: Global System for Mobile communications

[0212] LTE: Long-Term Evolution

[0213] CANBus: Controller Area Network Bus

[0214] USB: Universal Serial Bus

[0215] PCI: Peripheral Component Interconnect

[0216] FPGA: Field Programmable Gate Areas

[0217] SSD: solid-state drive

[0218] IC: Integrated Circuit

[0219] CU: Coding Unit

[0220] Although the present disclosure describes some exemplary embodiments, there are modifications, permutations, and various alternative equivalents that fall within the scope of the present disclosure. Accordingly, those skilled in the art will understand that numerous systems and methods can be devised that implement the principles of the present disclosure and thus fall within the spirit and scope of the present disclosure, even though they are not expressly illustrated or described herein.

Claims

Claim 1 A video decoding method performed in a video decoder, comprising: receiving coded information of a current block within a current picture of a video from a coded video bitstream, wherein the coded information indicates that the current block is an intra block copy (IBC) block and the current block is partitioned into multiple partitions based on a geometric partition mode (GPM); deriving an intra prediction mode for one subblock among the multiple partitions as an intra mode of a pixel located at the position of a reference subblock within a reference block, wherein the reference subblock corresponds to the subblock of the current block and is located at the same relative position within the reference block as the subblock within the current block, and the reference block is referenced by a block vector of the IBC block; storing the intra prediction mode for the subblock of the current block; and determining that a neighbor block of the current block is an IBC block. A video decoding method comprising the step of constructing a list of most probable modes (MPMs) for neighboring blocks using the stored intra-predicted mode for the sub-block of the current block. Claim 2 A video decoding method according to claim 1, wherein the sub-block of the current block is in units of 4×4 pixels. Claim 3 A video decoding method according to claim 1, further comprising the step of determining the intra prediction mode for the sub-block in the MPM list for the neighbor block of the current block as a candidate prediction mode, wherein the neighbor block is predicted by the IBC. Claim 4 A video decoding method according to claim 1, wherein the position is a central position in one of (i) row 2, column 2 of the reference subblock, (ii) row 2, column 3 of the reference subblock, (iii) row 3, column 2 of the reference subblock, and (iv) row 3, column 3 of the reference subblock. Claim 5 A video decoding method according to claim 1, wherein the sub-block of the current block is a 4×4 luma block. Claim 6 A video decoding method according to claim 1, wherein the current block is partitioned into a plurality of partitions including a first partition and a second partition based on the GPM, the first partition is predicted by an IBC and the second partition is predicted by an intra prediction, and the subblock is located in the first partition predicted by the IBC. Claim 7 A video encoding method performed in a video encoder, comprising: determining an intra prediction mode for a subblock of a current block as the intra mode of a pixel located at the position of a reference subblock within a reference block, wherein the reference subblock corresponds to the subblock of the current block and is located at the same relative position within the reference block as the subblock within the current block, wherein the current block is an intra block copy (IBC) block, wherein the reference block is referenced by a block vector of the IBC block, and the current block is partitioned into a plurality of partitions based on a geometric partition mode (GPM); storing the intra prediction mode for the subblock of the current block; determining that a neighbor block of the current block is an IBC block; and constructing a list of most probable modes (MPM) for the neighbor block using the stored intra prediction mode for the subblock of the current block. Claim 8 A video encoding method according to claim 7, wherein the sub-block of the current block is in units of 4×4 pixels. Claim 9 A video encoding method according to claim 7, further comprising the step of determining the intra prediction mode for the sub-block as a candidate prediction mode in the MPM list for the neighbor block of the current block, wherein the neighbor block is predicted by the IBC. Claim 10 A video encoding method according to claim 7, wherein the position is a central position in one of (i) row 2, column 2 of the reference subblock, (ii) row 2, column 3 of the reference subblock, (iii) row 3, column 2 of the reference subblock, and (iv) row 3, column 3 of the reference subblock. Claim 11 A video encoding method according to claim 7, wherein the subblock of the current block is a 4×4 luma block. Claim 12 A video encoding method according to claim 7, wherein the current block is partitioned into a plurality of partitions including a first partition and a second partition based on the GPM, the first partition is predicted by IBC and the second partition is predicted by intra prediction, and the subblock is located in the first partition predicted by the IBC. Claim 13 A video decoder comprising a processing circuit, wherein the processing circuit is configured to implement the method claimed in any one of claims 1 to 6. Claim 14 A video encoder comprising a processing circuit, wherein the processing circuit is configured to implement the method claimed in any one of claims 7 to 12. Claim 15 A computer-readable non-transient storage medium for storing instructions, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform the method claimed in any one of claims 1 to 6. Claim 16 A computer-readable non-transient storage medium for storing instructions, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform the method claimed in any one of claims 7 through 12.

Images