A system and method for improving composite inter- and intra-prediction.

Abstract

To provide a method and an apparatus for hybrid inter- and intra-prediction (CIIP) for video coding.SOLUTION: A method includes obtaining a first reference image and a second reference image associated with a current prediction block, generating a first prediction L0 on the basis of a first motion vector MV0 from the current prediction block to a reference block in the first reference image, generating a second prediction L1 on the basis of a second motion vector MV1 from the current prediction block to a reference block in the second reference image, determining whether a bidirectional optical flow (BDOF) operation is applied, and calculating a dual prediction of the current prediction block on the basis of the first prediction L0 and the second prediction L1 and the first gradient value and the second gradient value.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] This application claims priority based on Provisional Application No. 62 / 790,421, filed on January 9, 2019, and incorporates the entire contents thereof herein by reference. The entire contents thereof are incorporated herein by reference.

[0002] This application relates to video coding and compression. More specifically, this application relates to methods and devices for composite inter and intra prediction (CIIP) methods for video decoding.

Background Art

[0003] To compress video data, various video coding techniques can be used. Video coding is performed in accordance with one or more video coding standards. For example, video coding standards include Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Coding (H.265 / HEVC), High Definition Video Coding (H.264 / AVC), Moving Picture Experts Group (MPEG) coding, and the like. Video coding generally utilizes prediction methods (e.g., inter prediction, intra prediction, etc.) that exploit redundancies present in video images or sequences. An important goal of video coding techniques is to compress video data into a form that uses a lower bitrate while avoiding or minimizing a degradation in video quality. Video coding is performed in accordance with one or more video coding standards. For example, video coding standards include Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Coding (H.265 / HEVC), High Definition Video Coding (H.264 / AVC), Moving Picture Experts Group (MPEG) coding, and the like. Video coding generally utilizes prediction methods (e.g., inter prediction, intra prediction, etc.) that exploit redundancies present in video images or sequences. An important goal of video coding techniques is to compress video data into a form that uses a lower bitrate while avoiding or minimizing a degradation in video quality. Video coding generally utilizes prediction methods (e.g., inter prediction, intra prediction, etc.) that exploit redundancies present in video images or sequences. An important goal of video coding techniques is to compress video data into a form that uses a lower bitrate while avoiding or minimizing a degradation in video quality.

Summary of the Invention

[0004] Examples of the present disclosure provide a method for improving the efficiency of syntax signaling of merge-related modes.

[0005] JPEG0007833594000001.jpg106161

[0006] According to a second aspect of this disclosure, within the list of reference images associated with the current prediction block Obtain a reference image and a first motion vector from the current image to the first reference image. Based on this, generating an interpretation and associating the current prediction block with To obtain the intra prediction mode and, based on the intra prediction, the current prediction block To generate an intra prediction for the buck, and to average the intra prediction with the said intra prediction. This generates the final prediction for the current prediction block, and the current prediction block The buck, compared to the most likely mode (MPM) based intra-mode prediction, Identifying whether it will be treated as a terminal mode or intra-mode, and making a compromise. How to do video coding.

[0007] JPEG0007833594000002.jpg87161

[0008] According to a fourth aspect of this disclosure, a non-temporary computer that stores instructions A readable storage medium is provided. When executed by one or more processors, the current Retrieve the reference image in the list of reference images associated with the prediction block, and the current image To generate an interpretation based on a first motion vector from the image to a first reference image. This involves obtaining the intra prediction mode associated with the current prediction block, and the previous Based on the intra prediction, generate the intra prediction for the current prediction block. By averaging the inter-prediction and the intra-prediction, the current prediction block is To generate the final prediction, and to determine that the current prediction block is in the most likely mode (MP) For M) base intra-mode prediction, either inter-mode or intra-mode Identifying how to treat something and implementing actions including these on a computing device To make them do it.

[0009] Both the general description above and the detailed description below are merely examples and do not limit this disclosure. Please understand that this is not the case. [Brief explanation of the drawing]

[0010] The accompanying drawings incorporated herein and constituting part thereof illustrate examples consistent with this disclosure. This, along with the explanation, helps to illustrate the principles of this disclosure. [Figure 1] This is a block diagram of an encoder according to an example of the disclosure. [Figure 2] This is a block diagram of a decoder, as an example of the disclosure. [Figure 3] This flowchart shows a method for generating composite inter-intra predictions (CIIP) according to an example of the disclosure. [Figure 4] This flowchart shows a method for generating a CIIP according to an example of this disclosure. [Figure 5A] This figure shows a block partition in a multi-type tree structure, as an example of the disclosure. [Figure 5B] This figure shows a block partition in a multi-type tree structure, as an example of the disclosure. [Figure 5C] This figure shows a block partition in a multi-type tree structure, as an example of the disclosure. [Figure 5D] This figure shows a block partition in a multi-type tree structure, as an example of the disclosure. [Figure 5E]A diagram showing block partitioning in a multi-type tree structure according to an example of the present disclosure. [Figure 6A] A diagram showing composite inter and intra prediction (CIIP) according to an example of the present disclosure. [Figure 6B] A diagram showing composite inter and intra prediction (CIIP) according to an example of the present disclosure. [Figure 6C] A diagram showing composite inter and intra prediction (CIIP) according to an example of the present disclosure. [Figure 7A] A flowchart of an MPM candidate list generation process according to an example of the present disclosure. [Figure 7B] A flowchart of an MPM candidate list generation process according to an example of the present disclosure. [Figure 8] A diagram showing the workflow of an existing CIIP design in VVC according to an example of the present disclosure. [Figure 9] A diagram showing the workflow of a proposed CIIP method by removing BDOF according to an example of the present disclosure. [Figure 10] A diagram showing the workflow of a single prediction-based CIIP that selects a prediction list based on POC distance according to an example of the present disclosure. [Figure 11A] A flowchart of a method for enabling a CIIP block for MPM candidate list generation according to an example of the present disclosure. [Figure 11B] A flowchart of a method for disabling a CIIP block for MPM candidate list generation according to an example of the present disclosure. [Figure 12] A diagram showing a computing environment coupled with a user interface according to an example of the present disclosure.

Mode for Carrying Out the Invention

[0011] Here, examples of the present disclosure are referred to in detail, and the examples are shown in the accompanying drawings. The following description is otherwise Unless otherwise stated, the same number in different drawings represents the same or similar elements in the attached drawings. This refers to the surface. The embodiments described below in the examples of this disclosure are the same as those of this disclosure. This does not represent all possible embodiments. Instead, they are described in the attached claims. These are merely examples of apparatus and methods that are consistent with the embodiments related to this disclosure described within the scope of this disclosure. ru.

[0012] The terms used in this disclosure are for the sole purpose of describing specific embodiments. This disclosure is not intended to limit the scope of this disclosure and the attached claims. As used, the singular forms "a," "an," and "the" are not explicitly indicated in the context. As far as is, it is intended to include plural forms. The word is any or all possible combination of one or more of the related listed items. It should be understood that this means "combination" and also implies the intention to include.

[0013] Here, we use terms such as "first," "second," and "third" to explain various pieces of information. While this is possible, please understand that the information should not be limited by these terms. These terms are used solely to distinguish information in one category from information in another. For example, without departing from the scope of this disclosure, the first information may be referred to as the second information. This is possible, and similarly, the second piece of information can also be called the first piece of information. In such cases, the term "if" can mean "when" or "on the occasion" or "when" depending on the context. This can be understood as meaning "at the discretion of the judge."

[0014] The first version of the HEVC standard was completed in October 2013, and this was a successor to the previous generation of video. Compared to the H.264 / MPEG AVC decoding standard, it has approximately 50% less bitrate. It offers cost savings or equivalent perceived quality. The HEVC standard offers significantly better performance than its predecessors. While it offers improvements to coding, adding coding tools to HEVC would be even better. There is evidence that coding efficiency can be achieved. Based on this, VCEG and MPE Both G are new coding technologies for future video coding standards. We have begun the investigation into advanced technologies that will enable a significant improvement in coding efficiency. In October 2015, ITU-TVECG and ISO / IE were established to initiate important research on this topic. A Joint Video Exploration Team (JVET) was formed through C MPEG. One reference software called the Joint Exploration Model (JEM) is used for HEVC test models. By integrating several additional coding tools on top of Dell (HM), JVE It was maintained by T.

[0015] In October 2017, a joint call for proposals was issued for video compression with features exceeding those of HEVC. The (CfP) was issued by the ITU-T and ISO / IEC. In April 2018, the first At 10 JVET meetings, 23 CfP responses were received and evaluated, approximately 40% better than HEVC. The compression efficiency gain was demonstrated. Based on these evaluation results, JVET decided to Versatil To develop a new generation video coding standard called e Video Coding (VVC) We launched a new project. In the same month, to demonstrate a reference implementation of the VVC standard A reference software codebase called the VVC Test Model (VTM) was established. It was done.

[0016] Similar to HEVC, VVC is a block-based hybrid video coding framework. It is constructed on a floorwork. Figure 1 (explained below) shows a typical block-based high A block diagram of a hybrid video coding system is given. The input video signal is in the block ( It is called a loading unit (CU). It is processed for each CU. In VTM-1.0, C U can be up to 128x128 pixels. However, it is based only on quad trees. Unlike HEVC, which divides into blocks, VVC uses a quad / binary / ternary approach. To adapt to various local characteristics based on the tree, one coding tree unit The nit (CTU) is divided into CUs. Furthermore, multiple partitions in HEVC. The concept of unit types has been eliminated, meaning CU, Prediction Unit (PU), and Transformation Unit. The separation of (TU) no longer exists in VVC, and instead, each CU always has an additional partition Used as the basic unit for both prediction and transformation without modification. Multi-type tree structure First, one CTU is divided by a quad-tree structure. Next, each quad Tree leaf nodes can be further subdivided into binary and ternary tree structures. As shown in Figures 5A, 5B, 5C, 5D, 5D, and 5E (explained below), These are quaternary partitioning, horizontal two-way partitioning, and vertical two-way partitioning, respectively. The five types of partitioning are: partitioning, horizontal three-way partitioning, and vertical three-way partitioning. There is a split type available.

[0017] Figure 1 (described below) allows for the performance of spatial and / or temporal predictions. Spatial prediction (or "intra prediction") is already performed in the same video image / slice. Pixels from a sample of an adjacent block that has been read (called a reference sample) Use this to predict the current video block. Spatial prediction is based on the spatial characteristics inherent in the video signal. Reduce redundancy. Time prediction (also called "interpretation" or "motion-compensated prediction"). ) uses reconstructed pixels from already coded video images, Predicts the current video block. Time prediction reduces the temporal redundancy inherent in the video signal. The time prediction signal for a specific CU is typically the time between the current CU and its time reference. Signaled by one or more motion vectors (MVs) indicating the amount and direction of movement. Also, if multiple reference images are supported, one reference image index is It is sent in addition. This is because the time prediction signal is from which reference image in the reference image store it is coming from. It is used to identify whether it is true or false. After spatial and / or temporal predictions, the encoder is used. The mode determination block, for example, determines the optimal predictive mode based on a rate distortion optimization method. Select a block. Next, the predicted block is subtracted from the current video block, and the predicted remaining The difference is decorrelated using transformation and quantization.

[0018] The quantized residual coefficients are then inversely quantized and inversely transformed to form the reconstructed residuals, and then It is added to the prediction block to form the reconfigured signal of the CU. Deblocking fill Sample Adaptive Offset (SAO), Adaptive In-Loop Filter Further in-loop filtering, such as (ALF), will be placed in the reference image store in the future. It can be applied to the reconfigured CU before being used for coding the video block. To form a video bitstream, the coding mode (interface or inface) The tiger), predictive mode information, motion information, and quantized residual coefficients are all entropy - The bitstream is sent to the coding unit, where it is further compressed and packed. To form.

[0019] Figure 2 (described below) shows a typical block diagram of a block-based video decoder. The video bitstream is first processed by an entropy decode unit. Coding mode and prediction information are provided by the spatial prediction unit (intracoder). (If it is being coded) or time forecasting unit (if it is being coded) It is sent to one of the following to form a prediction block. The residual transformation coefficient is the inverse quantization unit. The data is then sent to the inverse transform unit to reconstruct the residual block. Next, the predicted block and the residual block are reconstructed. The difference blocks are added together. The reconstructed blocks are stored in the reference image store. Before that, it can go through in-loop filtering again. Next, the reference image stock The reconstructed video in A is sent out to drive the display device, It is also used to predict future video blocks.

[0020] Figure 1 shows a typical encoder 100. Encoder 100 has a video input 110. Motion compensation 112, motion estimation 114, intra / intermode determination 116, block prediction 140 units, 128 adders, 130 transformers, 132 quantizers, 142 prediction-related information, intraprediction Measurement 118, image buffer 120, inverse quantization 134, inverse transform 136, adder 126, memory 124, in-loop filter 122, entropy coding 138, and bit It has trim 144.

[0021] Figure 2 shows a block diagram of a typical decoder 200. Decoder 200 is a bitst Reem 210, Entropy Decode 212, Inverse Quantization 214, Inverse Transform 216, Adder 2 18, Intra / Intermode selection 220, Intra prediction 222, Memory 230, In Loop filter 228, motion compensation 224, image buffer 226, prediction-related information 234, It also has video output 232.

[0022] Figure 3 shows an example of generating composite inter-intra prediction (CIIP) according to this disclosure. A schematic method 300 is shown.

[0023] In step 310, the first reference image and the second reference image associated with the current prediction block are used. Get the reference image. Here, the first reference image is before the current image in display order, and the second The reference image is displayed after the current image in the display order.

[0024] In step 312, from the current prediction block to the reference block in the first reference image Based on the first motion vector MV0, the first prediction L0 is obtained.

[0025] In step 314, move from the current prediction block to the reference block in the second reference image. Based on the second motion vector MV1, a second prediction L1 is obtained.

[0026] JPEG0007833594000003.jpg55161

[0027] JPEG0007833594000004.jpg32161

[0028] Figure 4 shows an exemplary method for generating a CIIP according to this disclosure. For example, The method uses single-prediction-based interprediction and MPM-based to generate CIIP. This includes intranet predictions.

[0029] In step 410, in the list of reference images associated with the current prediction block, Obtain the reference image.

[0030] In step 412, based on the first motion vector from the current image to the first reference image Next, we generate interpretation predictions.

[0031] In step 414, the intra prediction mode associated with the current prediction block is To obtain.

[0032] In step 416, based on the intra prediction, the intra prediction of the current prediction block Generate a measurement.

[0033] In step 418, by averaging the inter-prediction and intra-prediction, the current Generate the final prediction for the prediction block.

[0034] In step 420, the current predicted block is the most likely mode (MPM). Regarding the base intra-mode prediction, is it inter-mode or intra-mode? Determine how it will be handled.

[0035] Figure 5A shows a block quaternary partition in a multi-type tree structure according to an example of the present disclosure. A diagram illustrating the concept is shown.

[0036] Figure 5B shows a block vertical binary in a multi-type tree structure according to an example of the present disclosure. A diagram illustrating the arrangement is shown.

[0037] Figure 5C shows a block horizontal binary pattern in a multi-type tree structure according to an example of the present disclosure. A diagram illustrating the arrangement is shown.

[0038] Figure 5D shows a block vertical ternary pattern in a multi-type tree structure according to an example of the present disclosure. A diagram illustrating the arrangement is shown.

[0039] Figure 5E shows a block horizontal ternary pattern in a multi-type tree structure according to an example of the present disclosure. A diagram illustrating the arrangement is shown.

[0040] Composite Inter and Intra Prediction As shown in Figures 1 and 2, the inter- and intra-prediction methods are hybrid videoco Used in a reading scheme. Here, each PU is either in the time domain or the spatial domain. Furthermore, to utilize correlation, you are allowed to choose between interpretation or intraprediction. Therefore, it is not possible to do both. However, as pointed out in previous literature, interpretation The residual signals generated by the lock and intra-prediction blocks have very different characteristics from each other. This may indicate that... Therefore, if we can efficiently combine the two types of predictions... To reduce the energy of the prediction residual and improve coding efficiency, another positive Accurate predictions can be expected. Furthermore, in natural video content, the movement of moving objects is clearly visible. This can become complicated. For example, older content (for example, previously coded (objects included in the image) and new content (for example, previously coded There is a possibility that there are areas containing both (objects excluded in the filtered image). In this scenario, both interpretation and intrapretation are one of the current blocks. We cannot provide an accurate prediction.

[0041] To further improve prediction efficiency, the VVC standard includes a merge mode for coding. A composite interface and interface that combines the intra-prediction and inter-prediction of a single CU. The CIIP prediction method is used. Specifically, one additional flag is added for each merge CU. The signal indicates whether CIIP is enabled for the current CU. It is applied. For the luminance component, CIIP is in planar mode, DC mode, and horizontal. Supports four frequently used intra modes, including vertical mode. Saturation For the component, DM (that is, saturation is the same intramolecular as the luminance component) Reusing the code is always applied without additional signaling. Furthermore, existing CI In the IP design, a weighted average is applied to the interpretation sample of one CIIP CU. The intra-predicted samples are then combined. Specifically, either planar mode or DC mode is selected. If this is the case, equal weights (i.e., 0.5) are applied. Otherwise, (That is, either horizontal mode or vertical mode is applied.) The current CU is first Four areas of the same size, either horizontally (in horizontal mode) or vertically (in vertical mode). It is divided into.

[0042] JPEG0007833594000005.jpg60161

[0043] Furthermore, in the current VVC operating specifications, the intra-mode of a single CIIP CU is the most The input of its adjacent CIIP CU via the Most Probable Mode (MPM) mechanism It can be used as a predictor to predict the tramode. Specifically, each CI Regarding an IP CU, if its adjacent block is also a CIIP CU, The intra-modes of these adjacent blocks are, first, planar mode, DC mode, horizontal mode, And rounded to the nearest mode within the vertical mode, then the MPM candidate list for the current CU. It is added. However, when configuring the MPM list for each intraCU, its adjacent One of the blocks is considered unavailable if it is coded in CIIP mode. In other words, the intra mode of one CIIP CU is the same as the intra mode of its adjacent intra CUs. Predicting the intra-mode is not permitted. Figures 7A and 7B (explained below) This compares the MPM list generation processes of IntraCU and CIIP CU.

[0044] JPEG0007833594000006.jpg72162

[0045] JPEG0007833594000007.jpg84163

[0046] JPEG0007833594000008.jpg137163

[0047] JPEG0007833594000009.jpg61163

[0048] Here, shift and o offset These are 15-BD and 1≪(14-BD)+2·(1≪13 ) is equal to the right-shifted value applied to combine the L0 and L1 prediction signals of the dual prediction. And the offset value.

[0049] Figure 6A shows a diagram illustrating a composite intermediate and intra-prediction of horizontal modes according to an example of this disclosure. show.

[0050] Figure 6B shows a diagram illustrating a composite intermediate and intra-prediction of vertical modes according to an example of this disclosure. show.

[0051] Figure 6C shows an example of a planar mode and DC mode composite interface and intraprevention according to the present disclosure. A diagram illustrating the measurement is shown.

[0052] Figure 7A shows an example of the process for generating an MPM candidate list in intraCUS according to this disclosure. A low chart is shown.

[0053] Figure 7B shows the process of generating an MPM candidate list for a CIIP CU according to an example of the disclosure. A low chart is shown.

[0054] Improvements to CIIP CIIP can improve the efficiency of conventional motion compensation prediction, but its design is further This can be improved. Specifically, in the existing CIIP design in VVC The following issues are identified in this disclosure.

[0055] First, as explained in the section on "Composite Inter and Intra Prediction," CIIP is, To combine samples of inter and intra predictions, each CIIP CU is restructured It is necessary to generate a predicted signal using the generated adjacent samples. This is done using one CII This means that decoding a PCU depends on the complete reconstruction of its adjacent blocks. Due to this interdependence, in actual hardware implementations, CIIP is adjacent to This needs to be done during the reconstruction phase so that the configured samples can be used for intra-prediction. Yes. The decoding of CUs during the reconstruction phase must be performed sequentially (i.e., one by one). Therefore, the calculation operations included in the CIIP process (e.g., multiplication, addition, bit shifts) are not included. The number of ) should not be too high in order to ensure sufficient throughput for real-time decoding. It is not possible to do so.

[0056] As mentioned in the "Bidirectional Optical Flow" section, BDOF is forward and From two reference blocks from both time directions backward, one is intercoded. When a CU is predicted, it is enabled to improve prediction quality. (Figure 8, explained below) As shown in (A) the current VVC also uses BDOF and CIIP mode for predictive screening. It is involved in generating pulls. Given the further complexity with BDOF, this The design is such that when CIIP is enabled, the hardware codec encoding / Decode throughput may decrease significantly.

[0057] Next, in the current CIIP design, one CIIP CU is one that is double-predicted When referencing merge candidates, generate motion compensation prediction signals for both lists L0 and L1. It is necessary to do so. If one or more MVs are not integer precision, partial sampling To interpolate the sample at the specified location, an additional interpolation process must be invoked. Such processes not only increase computational complexity but also require more input from external memory. If you need to access light samples, increase the memory bandwidth as well.

[0058] Then, as discussed in the section on "Composite Inter and Intra Prediction," the current CI In IP design, the intra mode of the CIIP CU and the intra mode of the intra CU are They are treated differently when constructing the MPM list for those adjacent blocks. Specifically, If one current CU is coded in CIIP mode, then its adjacent A CIIP CU is considered an intranet, meaning that the intranet of an adjacent CIIP CU is considered an intranet. The code can be added to the MPM candidate list. However, if the current CU is intramode If coded in [code], then its adjacent CIIP CU is considered an interface. Therefore, the intra-mode of the adjacent CIIP CU is excluded from the MPM candidate list. This is the case. Such inconsistent designs are ideal for the final version of the VVC standard. There's a possibility that it's not the case.

[0059] Figure 8 shows the workflow of an existing CIIP design in VVC, as an example of this disclosure. A diagram illustrating this is shown.

[0060] Simplification of CIIP This disclosure uses existing CIIP designs to facilitate hardware codec implementation. A method for simplifying is provided. Generally, the main aspects of the technology proposed in this disclosure The conclusion can be summarized as follows:

[0061] First, in order to improve CIIP coding / decode throughput, CIIP mode It is proposed to exclude BDOF from the generation of interpretation prediction samples in the system.

[0062] Next, in order to reduce computational complexity and memory bandwidth consumption, one CIIP When CU is double predicted (i.e., has both L0 and L1 MV): Next, to generate interpretation samples, we convert the blocks from double prediction to single prediction. A method is proposed.

[0063] Then, the two methods, when forming MPM candidates for adjacent blocks, intra This is proposed to harmonize the intra-modes of CU and CIIP.

[0064] CIIP without BDOF As noted in the "Problem Statement" section, BDOF is currently C When U is predicted twice, we generate interpretation samples for the CIIP mode. Therefore, it is always enabled. Due to the further complexity of BDOF, the existing CIIP design This can significantly reduce encoding / decoding throughput, especially in real Time decoding may be difficult for VVC decoders. On the other hand, CIIP For CU, the final prediction sample is the inter-prediction sample and the intra-prediction sample. It is generated by averaging the values. In other words, it is an improved predictive score using BDOF. The pull signal is not used directly as the prediction signal for CIIP CU. Therefore, conventional dual prediction Compare with measured CU (where BDOF is applied directly to generate predictive samples) Consequently, the corresponding improvements derived from BDOF become less efficient in CIIP CU. Therefore, based on the above circumstances, when generating interpretation samples in CIIP mode It is proposed to disable BDOF. Figure 9 (explained below) shows the result after removing BDOF. This shows the corresponding workflow for the proposed CIIP process.

[0065] Figure 9 shows a proposed CIIP method by removing BDOF, as an example of the present disclosure. A diagram illustrating the workflow is shown.

[0066] CIIP based on a single forecast As described above, merge candidates referenced by one CIIP CU are double-predicted. Sometimes, both L0 and L1 prediction signals are generated to predict samples within the CU. (Note) To reduce bandwidth and interpolation complexity, in one embodiment of the present disclosure, (current CU Interpretation samples generated using single prediction (even when double prediction is performed) This will be used to combine with intra-predicted samples in CIIP mode. In terms of the current CIIP CU, in the case of single prediction, the interpretation samples are: It is directly coupled with the intra-predicted sample. Otherwise (i.e., the current CU is a duplicate prediction) (If measured) the interpretation sample used by CIIP is one prediction Generated based on a single prediction from a list (L0 or L1). Select a prediction list. Various methods can be applied to this. In the first method, the prediction is made using two reference images. For any CIIP block, always select the first prediction (i.e., list L0). This has been proposed.

[0067] In the second method, for any CIIP block predicted by the two reference images Therefore, it is proposed to always select the second prediction (i.e., list L1). In the third method One adaptation method is to choose the image with the smallest point of image (POC) distance from the current image. This applies when the prediction list associated with the reference image is selected. (Figure 10 below) (Explained in [reference]) is a single-prediction-based CIIP that selects a list of predictions based on POC distance. The workflow is shown.

[0068] Finally, in the last method, CIIP mode is only used if the current CU is a single predicted value. It has been proposed to enable it. Furthermore, in order to reduce overhead, CII The signaling of the P enable / disable flag depends on the current CIIP CU prediction direction. When the current CU is predicted as a single, the CIIP flag is in the bitstream. Signaling is performed to indicate whether CIIP is enabled or disabled. Otherwise (i.e., currently) If the CU is predicted twice, the CIIP flag signaling is skipped. This is presumed to be false, meaning CIIP is always disabled.

[0069] Figure 10 shows a single prediction that selects a list of predictions based on the POC distance, according to an example of this disclosure. The diagram shows the workflow of the measurement-based CIIP.

[0070] Harmonizing intra-CU and CIIP intra-mode for MPM candidate list construction As described above, the current CIIP design is an intra CU and an intra CIIP CU. Regarding the method of forming a list of MPM candidates for those adjacent blocks using the mode, They are not synchronized. Specifically, in both intra-CU and CIIP CU intra-mode. This allows us to predict the intra-mode of adjacent blocks coded in CIIP mode. However, with only the intra mode of the intraCU, the intra mode of the intraCU is predictable. Yes, it is. To achieve another unified design, there are two methods: MPM list Harmonizing the use of intra-CU and CIIP intra-mode for configuration, this section It will be proposed.

[0071] In the first method, the CIIP mode is treated as the intermode for MPM list configuration. It has been proposed that, specifically, one CIIP CU or one intraCU When generating any of the MPM lists, adjacent blocks are coded in CIIP mode. If this is the case, the intra-mode of the adjacent block will be marked as unavailable. This method uses the intra-mode of the CIIP block to configure the MPM list. It is not possible. Conversely, in the second method, CIIP mode is used for MPM list configuration. It has been proposed to treat it as an intra-mode. Specifically, in this method, CII In the P CU's intra mode, both adjacent CIIP blocks and intra blocks The intra-mode can be predicted. Figures 11A and 11B (explained below) show the two methods described above. This shows the MPM candidate list generation process when the following applies.

[0072] Other embodiments of this disclosure may take into account the specifications and practices of this disclosure disclosed herein. This will be obvious to those skilled in the art. This application follows the general principles known in the art. Any modification or use of this disclosure, including deviations from this disclosure that fall within the scope of customary practice, Or intended to cover conformity. The true scope and spirit of this disclosure are the following patents. As indicated by the claims, the specification and examples are intended to be considered merely illustrative. It is being done.

[0073] This disclosure is not limited to the specific examples described above and shown in the attached drawings, It was understood that various modifications and changes could be made without deviating from the scope. The scope of this disclosure is intended to be limited only by the attached claims. ru.

[0074] Figure 11A shows an example of the present disclosure in which a CIIP block is used to generate an MPM candidate list. A flowchart showing how to enable it is provided.

[0075] Figure 11B shows an example of the CIIP block used to generate an MPM candidate list according to this disclosure. A flowchart showing how to disable it is provided.

[0076] Figure 12 shows the computing environment 12 coupled with the user interface 1260. This indicates 10. The computing environment 1210 may be part of a data processing server. Computing environment 1210 consists of processor 1220, memory 1240, and I / Includes O interface 1250.

[0077] Processor 1220 is typically associated with display, data acquisition, data communication, and image processing. It controls the overall operation of the computing environment 1210, including operations such as those performed by the processor. 1220 executes instructions to perform all or some of the steps in the above method. It may include one or more processors. Furthermore, processor 1220 is processor 1 It may include one or more circuits that facilitate interaction between 220 and other components. Processors include central processing units (CPUs), microprocessors, and single-chip machines. This could be a GPU, for example.

[0078] Memory 1240 is various to support the operation of computing environment 1210. It is configured to store data of a certain type. An example of such data is computer Instructions used in any application or method operating in the 1210 environment, video Includes data, image data, etc. Memory 1240 is of any type volatile or non-volatile Static memory devices, or combinations thereof, for example, static random access memory. (SRAM), electrically erasable programmable read-only memory (EEPROM) Erasable programmable read-only memory (EPROM), programmable read Promo memory (PROM), read-only memory (ROM), magnetic memory, flash memory This can be achieved using Mori, magnetic disks, or optical disks.

[0079] The I / O interface 1250 connects to the processor 1220, and also to the keyboard and clickholes. Provides an interface between the device, buttons, and other peripheral interface modules. The buttons include a home button, a scan start button, and a scan stop button. These are rare, but not limited to them. The I / O interface 1250 is an encoder. It can also be coupled with a decoder.

[0080] In one embodiment, in order to carry out the above method, within the computing environment 1210 Multiple such as those contained in memory 1240, which are executable by processor 1220. Non-temporary computer-readable storage media containing programs are also provided. For example, non-temporary Computer-readable storage media include ROM, RAM, CD-ROM, magnetic tape, and floppy disks. - This could include disks, optical data storage devices, etc.

[0081] Non-temporary computer-readable storage media are computers having one or more processors. It stores multiple programs within it for execution by the routing device. When multiple programs are executed by one or more processors, the computer The testing device performs the method for predicting the operation described above.

[0082] In one embodiment, the computing environment 1210 performs the method described above. , one or more application-specific integrated circuits (ASICs), digital signal processors (DS P), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PL) D) Field-programmable gate array (FPGA), graphical processing Graphics processing unit (GPU), controller, microcontroller, microprocessor —or other electronic components can be used to achieve this.

Claims

【Request Item 1】

2. The method according to claim 1, further comprising specifying that the BDOF operation is applied on the condition that CIIP is not applied to generate the final prediction of the current coding block.

3. The method according to claim 1, further comprising calculating a double prediction for the current coding block based on averaging the first prediction and the second prediction.

4. Calculating the double prediction of the current coding block is: The BDOF value is calculated based on the first horizontal gradient value, the first vertical gradient value, the second horizontal gradient value, and the second vertical gradient value. The dual prediction of the current coding block is calculated based on the first prediction, the second prediction, and the BDOF value. The method according to claim 1, further comprising:

5. The method according to claim 1, wherein the bit depth of the video data is equal to 12.

6. The method according to claim 1, wherein the bit depth of the video data is greater than 12.

7.

8. The video coding apparatus according to claim 7, further comprising specifying that the operation is applied on the condition that CIIP is not applied to generate the final prediction of the current coding block.

9. The video coding apparatus according to claim 7, wherein the bit depth of the video data is equal to 12.

10. The video coding apparatus according to claim 7, wherein the bit depth of the video data is greater than 12.

11. The video coding apparatus according to claim 7, wherein the operation further comprises calculating a double prediction for the current coding block based on averaging the first prediction and the second prediction.

12. Calculating the double prediction of the current coding block is: The BDOF value is calculated based on the first horizontal gradient value, the first vertical gradient value, the second horizontal gradient value, and the second vertical gradient value. The dual prediction of the current coding block is calculated based on the first prediction, the second prediction, and the BDOF value. The video coding apparatus according to claim 7, further comprising:

13. A non-temporary computer-readable storage medium for storing a plurality of programs executed by a computing device having one or more processors, A non-temporary computer-readable storage medium wherein the plurality of programs, when executed by one or more processors, cause the computing device to perform the operation according to any one of claims 1 to 6.

14. A method for storing a bitstream, A bitstream is generated by performing the method according to any one of claims 1 to 4, The bitstream mentioned above is stored, A method that includes this.

15. The method according to claim 14, wherein the bit depth of the video data is equal to 12.

16. The method according to claim 14, wherein the bit depth of the video data is greater than 12.

Images