Encoding and decoding method, code stream, encoder, decoder and storage medium
Patent Information
- Application Number
- CN202380096983.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-14
- Publication Date
- 2025-11-18
AI Technical Summary
In existing video encoding and decoding technology, intra-frame block copy and intra-frame template matching prediction modes have redundancy in the encoding process, which affects video encoding efficiency.
By removing duplicate block vector information in intra-frame merging mode, a deredundant block vector candidate list is constructed to improve the expression efficiency of header information.
It improves the efficiency of video encoding and decoding, reduces the redundancy of block vectors, and improves the performance during encoding and decoding.
Smart Images

Figure CN120982097A_ABST
Abstract
Description
Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field
[0001] The present application relates to the field of video coding and decoding technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art
[0002] Intra Block Copy (IBC), a block-level coding mode, performs a motion search to find the optimal block vector (BV) for each coding block. This optimal block vector is found by searching the reconstructed area of the frame containing the current coding block (i.e., the current coding frame). Intra Template Matching Prediction (Intra TMP) is also a coding mode that uses BV for intra-frame prediction, and its motion search process uses template matching as the metric.
[0003] However, in actual encoding, the IBC mode and the IntraTMP mode share many similarities, such as using BV to obtain prediction values and being used for encoding natural and screen video content. However, the organization used in related technologies is redundant with the search ranges of other prediction modes, hindering efficient encoding and thus affecting video coding efficiency.
[0004] Summary of the Invention
[0005] The present application provides a coding and decoding method, a code stream, an encoder, a decoder and a storage medium, which removes the redundant BV information between different modes, can improve the expression efficiency of header information, and thus improve the overall video coding efficiency.
[0006] The technical solution of this application can be implemented as follows:
[0007] In a first aspect, an embodiment of the present application provides a decoding method, comprising:
[0008] Decode the code stream and determine the prediction mode of the current block;
[0009] If the prediction mode of the current block indicates that the current block uses a prediction mode based on intra block merging to determine a prediction value of the current block, determining a first block vector candidate list of the current block;
[0010] Determine a first area range, and determine a second block vector candidate list based on the first block vector candidate list and the first area range;
[0011] Based on the second block vector candidate list, a prediction value of the current block is determined.
[0012] In a second aspect, an embodiment of the present application provides an encoding method, including:
[0013] Determine the prediction mode of the current block;
[0014] If the prediction mode indicates that the current block uses a prediction mode based on intra-frame block merging, determining a first block vector candidate list for the current block;
[0015] Determine a first area range, and determine a second block vector candidate list based on the first block vector candidate list and the first area range;
[0016] Based on the second block vector candidate list, a prediction value of the current block is determined.
[0017] In a third aspect, an embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:
[0018] The prediction difference value of the current block, the candidate block vector index information, the extended candidate block vector index information, the first syntax identification information, the second syntax identification information and the third syntax identification information;
[0019] Among them, the first syntax identification information is used to indicate whether the current block uses the prediction mode of intra-frame block copy, the second syntax identification information is used to indicate whether the current block uses the prediction mode of intra-frame block merging, and the third syntax identification information is used to indicate whether the current block uses the MBVD mode.
[0020] In a fourth aspect, an embodiment of the present application provides a decoder, including:
[0021] A decoding part, configured to decode the bitstream and determine a prediction mode for the current block;
[0022] A first determining part is configured to determine a first block vector candidate list of the current block if the prediction mode of the current block indicates that the current block uses a prediction mode based on intra block merging to determine a prediction value of the current block; and determine a first area range;
[0023] a first removing portion configured to determine a second block vector candidate list based on the first block vector candidate list and the first area range;
[0024] The first prediction part is configured to determine a prediction value of the current block based on the second block vector candidate list.
[0025] In a fifth aspect, an embodiment of the present application provides a decoder, comprising a first memory and a first processor; wherein,
[0026] a first memory for storing a computer program capable of running on the first processor;
[0027] The first processor is configured to execute the method according to the first aspect when running a computer program.
[0028] In a sixth aspect, an embodiment of the present application provides an encoder, including:
[0029] A second determining part is configured to determine a prediction mode of a current block; and if the current block uses a prediction mode based on intra-frame block merging, determine a first block vector candidate list of the current block; and determine a first area range;
[0030] a second removing portion configured to determine a second block vector candidate list based on the first block vector candidate list and the first area range;
[0031] The second prediction part is configured to determine a prediction value of the current block based on the second block vector candidate list.
[0032] In a seventh aspect, an embodiment of the present application provides an encoder, comprising a second memory and a second processor; wherein,
[0033] a second memory for storing a computer program capable of running on the second processor;
[0034] The second processor is configured to execute the method according to the second aspect when running a computer program.
[0035] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed, implements the method described in the first aspect or the method described in the second aspect.
[0036] An embodiment of the present application provides a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium. Whether it is an encoding end or a decoding end, in the process of block coding and decoding, for the scenario of intra-frame merging mode prediction, based on the first block vector candidate list of the current block, the block vectors or block vectors with overlap within a first area, such as in the intra-frame template matching prediction mode, can be removed to determine a second block vector candidate list, and the prediction value of the current block is determined based on the de-redundant block vectors in the second block vector candidate list. In this way, since the redundancy of the block vectors is reduced, the expression efficiency of the header information can be improved when the de-redundant block vectors or block vectors are used for coding and decoding, thereby improving the video coding and decoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The accompanying drawings herein are incorporated into and constitute a part of this specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, serve to illustrate the technical solutions of the present application. Obviously, the drawings described below are merely some embodiments of the present application. Those skilled in the art can, without inventive effort, derive other drawings from these drawings.
[0038] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.
[0039] FIG1 is a schematic diagram of an optional process for a decoding end to obtain a reconstructed pixel value according to an embodiment of the present application;
[0040] FIG2a is a schematic diagram of an optional template type provided in an embodiment of the present application;
[0041] FIG2 b is a second schematic diagram of an optional template type provided in an embodiment of the present application;
[0042] FIG2c is a third schematic diagram of an optional template type provided in an embodiment of the present application;
[0043] FIG2 d is a fourth schematic diagram of an optional template type provided in an embodiment of the present application;
[0044] FIG2e is a fifth schematic diagram of an optional template type provided in an embodiment of the present application;
[0045] FIG2f is a sixth schematic diagram of an optional template type provided in an embodiment of the present application;
[0046] FIG3 is a schematic diagram of an optional adjacent block position provided in an embodiment of the present application;
[0047] FIG4 is a schematic diagram of an optional BVP candidate provided in an embodiment of the present application;
[0048] FIG5 is a schematic diagram of an optional template cost calculation provided in an embodiment of the present application;
[0049] FIG6 is a schematic diagram of an optional candidate item clustering provided in an embodiment of the present application;
[0050] FIG7a is a first flow chart of an optional candidate item search process provided in an embodiment of the present application;
[0051] FIG7 b is a second flow chart of an optional candidate item search process provided in an embodiment of the present application;
[0052] FIG8 is a schematic diagram of an optional method for determining a block vector according to an embodiment of the present application;
[0053] FIG9 a is a first schematic diagram of an optional IBC reference area provided in an embodiment of the present application;
[0054] FIG9 b is a second schematic diagram of an optional IBC reference area provided in an embodiment of the present application;
[0055] FIG9 c is a third schematic diagram of an optional IBC reference area provided in an embodiment of the present application;
[0056] FIG9 d is a fourth schematic diagram of an optional IBC reference area provided in an embodiment of the present application;
[0057] FIG10 is a fifth schematic diagram of an optional IBC reference area provided in an embodiment of the present application;
[0058] FIG11 is a sixth schematic diagram of an optional IBC reference area provided in an embodiment of the present application;
[0059] FIG12 is a schematic diagram of an optional Intra TMP prediction process provided in an embodiment of the present application;
[0060] FIG13 is a schematic diagram of an optional template search area provided in an embodiment of the present application;
[0061] FIG14 is a schematic diagram of a prediction process of an optional IntraTMP mode provided in an embodiment of the present application;
[0062] FIG15 is a schematic diagram of a detailed process of an optional IntraTMP mode provided in an embodiment of the present application;
[0063] FIG16 is a first schematic diagram of an optional search process of the Intra TMP Fusion mode provided in an embodiment of the present application;
[0064] FIG17 is a second schematic diagram of an optional search process of the Intra TMP Fusion mode provided in an embodiment of the present application;
[0065] FIG18 is a schematic diagram of a process for determining a prediction value in an optional Intra TMP Fusion mode provided in an embodiment of the present application;
[0066] FIG19a is a first structural diagram of an optional filter provided in an embodiment of the present application;
[0067] FIG19 b is a second structural diagram of an optional filter provided in an embodiment of the present application;
[0068] FIG19c is a third structural diagram of an optional filter provided in an embodiment of the present application;
[0069] FIG19d is a fourth structural diagram of an optional filter provided in an embodiment of the present application;
[0070] FIG19e is a fifth structural diagram of an optional filter provided in an embodiment of the present application;
[0071] FIG19f is a sixth structural diagram of an optional filter provided in an embodiment of the present application;
[0072] FIG20 is a schematic diagram of an optional TM-FLM prediction provided in an embodiment of the present application;
[0073] FIG21 is a schematic flow chart of an optional decoding method provided in an embodiment of the present application;
[0074] FIG22 is a flowchart diagram of an optional encoding method provided in an embodiment of the present application;
[0075] FIG23 is a second flow chart of an optional encoding method provided in an embodiment of the present application;
[0076] FIG24 is a third flow chart of an optional encoding method provided in an embodiment of the present application;
[0077] FIG25 is a fourth flow chart of an optional encoding method provided in an embodiment of the present application;
[0078] FIG26 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;
[0079] FIG27 is a schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present application;
[0080] FIG28 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;
[0081] FIG29 is a schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;
[0082] Figure 30 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0083] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.
[0084] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.
[0085] In the following description, reference is made to "some embodiments," which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first, second, and third" in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first, second, and third" may be interchanged in a specific order or sequential order where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0086] It should also be pointed out that if similar descriptions of "first / second" appear in the application documents, the following explanation shall be added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0087] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:
[0088] Video coding standard, Versatile Video Coding (H.266 / VVC);
[0089] Intra Block Copy (IBC);
[0090] Intra Template Matching Prediction (Intra TMP);
[0091] Block Vector (BV);
[0092] Template Matching (TM);
[0093] Advanced Motion Vector Prediction (AMVP)
[0094] Advanced Block Vector Prediction (ABVP)
[0095] Combined inter and intra prediction (CIIP);
[0096] Geometric Partitioning Mode (GPM);
[0097] Local Illumination Compensation (LIC).
[0098] To facilitate understanding of the embodiments of the present application, the following briefly introduces the concepts related to the BV-based prediction technology involved in the embodiments of the present application:
[0099] Currently, BV-based prediction technologies mainly include intra block copy (IBC)-based prediction and intra template matching prediction (IntraTMP). The following describes these two BV-based prediction technologies in detail.
[0100] Intra Block Copy (IBC) is an extension tool of VVC (Video Coding Standard) for coding video sequences of screen content type (Screen Content Coding), which can significantly improve the coding efficiency of screen content sequences.
[0101] IBC is a block-level coding mode. Similar to inter-frame technology, the encoder performs motion search to find the best block vector (Block Vector, BV) for each coding block. The motion search here can be understood as block matching, and the block vector here can also be called a motion vector (Motion Vector, MV). The block vector is a vector pointing from the current block to the reference block. The difference from inter-frame technology is that the best block vector of IBC is obtained by searching in the reconstructed area of the frame where the current coding block is located (that is, the current coding frame), while the inter-frame motion vector is obtained by searching the adjacent reference frames of the current coding frame in the time domain.
[0102] IBC technology is used as a key tool for screen video coding in coding standards such as H.265 / HEVC SCC, H.266 / VVC, and AV1, AVS3, and EVC. The exploratory model ECM for the next-generation H.266 / VVC coding standard also continues to use IBC technology as a screen video coding tool, expanding it into a tool set with multiple modes. It has also been introduced to the coding of general natural video, making it a more versatile video coding tool.
[0103] Intra-frame template-based prediction (IntraTMP) is also a coding mode that uses BV block vectors for intra-frame prediction. Unlike IBC, its motion search block matching process is performed at both the decoder and encoder ends. The matching criterion in the search process is not based on the matching degree of the coding block itself, but on template matching (TM).
[0104] On the decoding side, the basic decoding process for the IBC mode or the IntraTMP mode includes obtaining BV, using BV to obtain the predicted value, and determining the reconstructed value.
[0105] It should be noted that the decoder can determine the decoding mode corresponding to the current block, such as the IBC mode or the IntraTMP mode, according to the corresponding syntax element information (mode identification information) by parsing the bitstream.
[0106] Referring to FIG. 1 , FIG. 1 shows a schematic diagram of an optional process for a decoding end to obtain a reconstructed pixel value provided by an embodiment of the present application. As shown in FIG. 1 , the process for the decoding end to obtain the reconstructed pixel value includes S11 to S13:
[0107] S11. Obtain BV.
[0108] At the decoding end, the decoder obtains the block vector corresponding to the current block by parsing the bitstream.
[0109] Specifically, the decoder obtains BV through syntax parsing. In actual implementation, BV can be obtained in multiple ways, so the code stream can contain mode information for the BV acquisition method, as well as corresponding parameter information. For example, if the IBC merge mode is obtained in the syntax element, the decoder constructs a merge list, parses the merge list option index, and then obtains the specific information of BV from the corresponding option in the merge list according to the index; for another example, if it is a normal IBC mode (i.e., IBC-ABVP mode), the decoder obtains the specific information of BV by parsing the value of the syntax element that describes BV or BV prediction difference in the code stream; for another example, if it is an IntraTMP mode, the decoder directly obtains an optimal BV by searching for BV within a certain range, or constructs a candidate BV list, and then determines the specific information of the BV finally used by the syntax elements in the code stream.
[0110] S12. Use BV to obtain prediction samples.
[0111] At the decoding end, the decoder uses the BV to obtain the prediction value corresponding to the prediction sample. Here, using the BV to obtain the prediction value usually involves copying the corresponding area pointed to by the BV to obtain the prediction value of the current block.
[0112] S13. Obtain a reconstruction sample.
[0113] At the decoding end, if there is a prediction residual, the decoder needs to parse the bitstream to obtain the prediction residual, and then add the prediction residual to the predicted value to obtain the final reconstructed value of the current block.
[0114] Considering that both IBC merge mode and IntraTMP mode involve template matching and template matching cost during the encoding and decoding process, the following is a unified description of template matching:
[0115] Referring to Figures 2a, 2b, 2c, 2d, 2e and 2f, Figures 2a to 2f are schematic diagrams of an optional template type provided in an embodiment of the present application. As shown in Figures 2a to 2f, the template types include multiple types, including but not limited to the template types shown in Figures 2a to 2f. Among them, Figures 2a, 2b and 2c are three commonly used template types. The template type shown in Figure 2a is the area on the upper and left sides of the current block (Current Block); the template type shown in Figure 2b is the area on the left side of the current block; and the template type shown in Figure 2c is the area on the upper side of the current block. It should be noted that different template types can be valid at the same time, or they can be used separately (for example, only using the template type shown in Figure 2a). The available template types are specifically specified by the syntax elements.
[0116] In addition, the template type used by the current block can be determined by parsing the bitstream and the corresponding syntax element information. Here, the current block can use a single template type for template matching to determine the BV of the current block; the current block can use a combination of multiple template types for template matching to determine the BV of the current block. This application does not impose any restrictions on this.
[0117] The process of template matching cost is to use the difference in pixel values between the template of the reference block and the template of the current block through a preset cost function to express the cost value of the reference block and the current block.
[0118] Among them, the preset cost function can be the sum of absolute differences (SAD), the sum of absolute transformation differences (SATD), the mean square error (MSE), the sum of squared errors (SSD), the mean absolute difference (MAD), the mean sum of squared errors (MSD), the normalized correlation coefficient (NCC), etc.
[0119] It should be noted that the preset cost function may be any one of the above cost functions, or any combination of the above multiple cost functions, and this application does not impose any limitation on this.
[0120] For example, the cost function of absolute error and SAD can be expressed by formula (1):
[0121] In formula (1), T i is the template in the search process, M represents the number of pixels in the template, T m represents the mth pixel in the template of the current block, T i,m represents the mth pixel in the i-th template of the reference block.
[0122] Regarding S11 (obtaining BV, i.e. deriving BV), the steps of S11 are introduced in detail below:
[0123] For S11, the block vector bvL of luminance and the block vector bvC of chrominance are introduced respectively.
[0124] (1) Brightness component:
[0125] The luminance block vector (bvL) of the current block is determined according to the luminance position information of the current block, the width and the height of the current block.
[0126] Input: luma position (xCb, yCb), which specifies the top-left corner sample of the current coding block (current block) relative to the top-left corner luma sample of the current picture, a variable cbWidth, which specifies the width of the current coding block in luma samples, and a variable cbHeight, which specifies the height of the current coding block in luma samples.
[0127] It should be noted that, by parsing the bitstream, the decoder can obtain the size information (ie, width and height) of the current block in the header information.
[0128] Output: Block vector bvL (Block Vector Luma) of luminance.
[0129] Among them, the IBC mode can be roughly divided into two major modes: IBC merge mode and IBC ABVP mode (similar to the merge and AMVP modes of the inter-frame mode in VVC). The process of obtaining BV can include S21 to S23:
[0130] S21. When deriving bvL, it is necessary to establish an IBC block vector candidate list bvCandList.
[0131] For the IBC mode, in the process of obtaining the bvL of the current block, it is necessary to establish a block vector candidate list bvCandList, which includes multiple candidate items.
[0132] It should be noted that the number of block vectors (i.e., the number of candidates) in the block vector candidate lists bvCandList of different modes may be different. For example, the number of candidates that can be included in the bvCandList of the IBC merge mode is 6, and the number of candidates that can be included in the bvCandList of the IBC ABVP mode is 2. Of course, in an embodiment of the present application, the number of candidates that can be included in the bvCandList may be N, where N is a positive integer greater than or equal to 1. The number of candidates N is pre-set, and the decoder and the encoder agree or specify the value of N corresponding to the block vector candidate list bvCandList corresponding to the current block. For example, the decoder and the encoder agree or specify that the value of N corresponding to the block vector candidate list bvCandList corresponding to the current block is 8. Then, in the process of encoding or decoding the current block, the number of candidates contained in the established IBC block vector candidate list bvCandList is 8.
[0133] S22. Determine candidate items in the selected list according to the code stream.
[0134] By parsing the code stream, the index value of the bvL corresponding to the current block is determined, and the candidate corresponding to the current block is determined in the block vector candidate list according to the index value.
[0135] For the IBC mode and the IntraTMP mode, the candidate information (i.e., availability conditions) in the block vector candidate list includes at least one of the following information:
[0136] 1) Prediction direction (L0 or L1, generally L0 is the default);
[0137] 2) BV information (horizontal and vertical components);
[0138] 3) Reference frame (the default is the current image), the current image is the current reference frame;
[0139] 4) Flip type (e.g., no flip, horizontal flip, and vertical flip);
[0140] 5) Whether to use LIC (linear model).
[0141] It should be noted that the availability condition mentioned above is only an example, and can be any one of the availability conditions mentioned above, or a combination of the above multiple availability conditions. Of course, other availability conditions can also be included in practice, and this application does not impose any restrictions on this.
[0142] S23. Determine the final BV based on the candidate.
[0143] Furthermore, S21 includes S211 to S215:
[0144] S211. Derivation of airspace candidates.
[0145] When the usage condition is met (for example, the size condition IsGt4by4 is equal to TRUE: the variable IsGt4by4 is TRUE when the luma width multiplied by the height is greater than 16), the derivation process of the spatial block vector candidates from the adjacent coding units specified in the decoding specification is called with the luma coding block position (xCb, yCb), the luma coding block width cbWidth and the height cbHeight as input, and the output is the availability flags such as availableFlagA1, availableFlagB1 and the block vectors bvA1 and bvB1; wherein the size condition IsGt4by4 indicates the block size restriction. Among them, availableFlagA1 indicates the availability flag corresponding to the block vector bvA1, and availableFlagB1 indicates the availability flag corresponding to the block vector bvB1.
[0146] Exemplarily, when the value of the availability flag availableFlagA1 is a first preset value, it indicates that the block vector bvA1 can be used as a candidate; when the value of the availability flag availableFlagA1 is a second preset value, it indicates that the block vector bvA1 cannot be used as a candidate.
[0147] It should be noted that the first preset value is different from the second preset value, and the first preset value and the second preset value can be in parameter form or in digital form. Specifically, the availability flag availableFlagA1 can be a parameter written in the profile or a flag value, which is not specifically limited here.
[0148] Exemplarily, for the first preset value and the second preset value, the first preset value can be set to 1, and the second preset value can be set to 0; or, the first preset value can be set to 0, and the second preset value can be set to 1; or, the first preset value can be set to true, and the second preset value can be set to false; or, the first preset value can be set to false, and the second preset value can be set to true; but this is not specifically limited here.
[0149] Furthermore, the judgment condition for the availability check of each candidate item (i.e., the availability condition) includes at least one of the following:
[0150] Condition 1: The offset position obtained by adding the current block position to the BVP does not exceed the picture boundary;
[0151] Condition 2: The current block position plus the block position pointed to by the BVP does not overlap the current block;
[0152] Condition 3: The offset position obtained by adding the current block position to the BVP does not exceed the IBC available area;
[0153] Condition 4: Whether the current block position plus the block position pointed to by BVP has been reconstructed.
[0154] It should be noted that the reference block is a reconstructed block, so the reference block has BV information. On this basis, it needs to meet conditions 1, 2, 3 and 4 at the same time before it can be determined as a candidate.
[0155] In addition, the availability conditions mentioned above (condition 1, condition 2, condition 3 and condition 4) are only examples. The availability conditions include but are not limited to the above-mentioned conditions 1, condition 2, condition 3 and condition 4. It can be any one of the above-mentioned availability conditions, or any combination of multiple conditions. In practice, other availability conditions may also be included. The embodiments of this application do not impose any restrictions on this.
[0156] It should be noted that the availability conditions include but are not limited to the above-mentioned conditions 1, 2, 3, and 4. The availability conditions are specified or agreed upon by both the encoder and the decoder. When performing availability detection on the block vector of the current block, consistent detection results can be obtained by performing the detection according to the availability conditions specified by both parties.
[0157] In addition, a block vector is considered as a candidate only if it satisfies all the availability conditions simultaneously.
[0158] Exemplarily, when the codec and the decoder agree that the availability conditions for block vector availability detection include: condition 1, condition 2, condition 3 and condition 4, if the block vector bvA1 only meets condition 1, condition 2, condition 3, and does not meet condition 4, the block vector bvA1 will not be considered as a candidate; if the block vector bvA1 meets condition 1, condition 2, condition 3 and condition 4 at the same time, the block vector bvA1 will be considered as a candidate.
[0159] In addition, in the process of determining the candidate items, it is necessary to traverse the adjacent blocks of the current block in a preset order to determine the candidate items. Among them, the preset order of traversing the adjacent blocks is specified by both the encoder and the decoder. During the encoding or decoding process, the adjacent blocks of the current block are traversed according to the established preset order. Referring to Figure 3, Figure 3 is a schematic diagram of an optional adjacent block position provided by an embodiment of the present application. As shown in Figure 3, the adjacent blocks of the current block include: A0, A1, B0, B1, B2, and the preset order (i.e., the traversal order) can be set to: A1→B1→B0→A0→B2. The preset order is also predetermined or agreed upon by both the encoder and the decoder. Of course, the preset order here is only an example, and this application does not limit this.
[0160] S212. Airspace candidates are added to the candidate list.
[0161] When the usage condition is met (for example, the size condition IsGt4by4 is equal to TRUE), that is, for the candidates that meet the availability condition, the block vector candidate list bvCandList can be constructed as follows:
[0162] i=0
[0163] if(availableFlagA1)
[0164] bvCandList[i++]=bvA1
[0165] if(availableFlagB1)
[0166] bvCandList[i++]=bvB1
[0167] For each block vector of the current block, determine whether the block vector meets the availability condition by judging the availability flag of the block vector. For block vectors that meet the availability condition, add the block vector to the block vector candidate list bvCandList, and update the number of candidates in the block vector candidate list bvCandList.
[0168] Exemplarily, when traversing the adjacent blocks of the current block in the preset order of A1→B1→B0→A0→B2, it is determined whether the availability flag availableFlagA1 of the adjacent block A1 is true (or equal to 1) in the preset order of A1→B1→B0→A0→B2. If the availability flag availableFlagA1 of the adjacent block A1 is true (or equal to 1), the block vector bvA1 corresponding to the adjacent block A1 is used as the i-th candidate in the block vector candidate list bvCandList, and i is increased by 1; and the availability flag availableFlagB1 of the adjacent block B1 is determined to be true (or equal to 1). Otherwise, if the availability flag availableFlagB1 of the adjacent block B1 is true (or equal to 1), the block vector bvB1 corresponding to the adjacent block B1 is used as the i-th candidate in the block vector candidate list bvCandList, and i is increased by 1; continue to judge whether the availability flag availableFlagB0 of the adjacent block B0 is true (or equal to 1); if the availability flag availableFlagB0 of the adjacent block B0 is false (or equal to 0), continue to judge whether the availability flag availableFlagA0 of the adjacent block A0 is true (or equal to 1), and so on.
[0169] S213: Check the number of valid items in the candidate list.
[0170] Use numCurrCand to represent the number of valid items in the candidate list. The derivation process of the variable numCurrCand (the number of candidates currently obtained) is as follows:
[0171] If the usage condition is met (eg, the size condition IsGt4by4 is equal to TRUE), numCurrCand is set equal to the number of candidates in bvCandList; otherwise, numCurrCand is set to 0.
[0172] S214. If the candidate list does not reach the specified number of items, continue with historical candidate item derivation, availability detection, and addition.
[0173] If the candidate list does not reach the specified number of items (for example, the specified number of items in the IBC merge mode is 6, and the specified number of items in the IBC ABVP mode is 2), the historical candidate item derivation, availability detection, and addition are continued. That is, the BV that has appeared previously in the row where the current CTU is located is selected as the historical candidate item and selected from the historical list. The historical candidate item list can be updated by filling in the items from the back to the front.
[0174] Specifically, when numCurrCand is less than MaxNumIbcMergeCand (the maximum number of candidates in IBC merge mode) and NumHmvpIbcCand (the maximum number of candidates for the historical optimal block vector (Hmvp) in IBC mode) is greater than 0, the derivation process of the history-based IBC block vector candidates specified in the decoding specification is called with bvCandList and numCurrCand as input and the modified bvCandList and numCurrCand as output.
[0175] S215. Continue checking the number of valid items in the candidate list and adding other available candidate items until the specified number of items is reached.
[0176] Available candidates include at least one of the following: a pairwise average candidate, a zero-valued BV candidate.
[0177] For example, the pairwise average candidate can be constructed by using the first and second candidates, which can be expressed as: mvAvgLX=(mvCand0LX+mvCand1LX+1)>>1
[0178] For example, the zero value BV can be directly set to:
[0179] bvCandList[numCurrCand][0] is set equal to 0. (horizontal component of bv)
[0180] bvCandList[numCurrCand][1] is set equal to 0. (vertical component of bv)
[0181] As another example, a group of BVP candidates located in the IBC reference region can also be used as candidates that can be added. Referring to Figure 4, Figure 4 is a schematic diagram of an optional BVP candidate provided in an embodiment of the present application. As shown in Figure 4, the coordinates of a group of BVP candidates are determined by the width and height of the current block and the ΔX and ΔY parameters, a reference region (Reference Region) is determined in the reconstructed region (Reconstructed Region) of the IBC, and a group of BVP candidates (Proposed Candidates) are determined in the reference region as candidates that can be added (shown as A, B and C in Figure 4).
[0182] Each time an item is added to the block vector candidate list bvCandList, the number of valid items in the candidate list numCurrCand increases by 1. It should be noted that the number of valid items in the candidate list numCurrCand can be written into the bitstream.
[0183] In summary, through S211 to S215, the establishment of the basic block vector candidate list bvCandList can be completed.
[0184] Based on the block vector candidate list bvCandList, if the current block mode is IBC merge mode, the order of the block vector candidate list bvCandList can also be calculated using a template to perform cost calculation to reorder the candidates in bvCandList and adjust the order of the candidates in the candidate list. This can fully utilize the high spatial correlation, reduce the coding bit transmission, and effectively improve coding efficiency.
[0185] Exemplarily, for the IBC merge mode, after constructing the intermediate candidate list (i.e., bvCandList) according to the above list construction method (note that the length of the intermediate candidate list can be greater than or equal to the maximum number of candidates in the IBC merge mode), all candidate items in the list are reordered using the template, and sorted in order from small to large according to the template matching cost, and the first N (for example, N = 6) candidates in the sorted list are selected.
[0186] Refer to Figure 5, which is a schematic diagram of an optional template cost calculation provided in an embodiment of the present application. As shown in Figure 5, in the IBC merge mode, the SAD of the template position of the reference block pointed to by each candidate and the template position of the current block is calculated, that is, the cost value between the template (Template, T) of the current block (Current Block) and the reference sample template (Reference samples of template, RT) of the reference block (Reference Block) pointed to by the BV candidate (BV of an IBC merge candidate) is calculated, and the cost values corresponding to each reference block are arranged in ascending order, and the top 6 candidates (N) are selected as the candidate list for IBC merge.
[0187] Based on the block vector candidate list bvCandList, if the current block mode is IBC ABVP, redundant candidates can be removed based on their distance. Specifically, the number of candidates remains unchanged, and candidate lists are created for integer pixel and 4-pixel precision, respectively.
[0188] In the case where the mode of the current block is IBC ABVP mode, if the current block corresponds to non-RRIBC mode (not flipped), if the number of valid BVP candidates exceeds two, up to six BVP candidates in the candidate list are clustered according to the Euclidean distance between them, and the radius (R) is a set of block vectors determined as a logarithmic function of the width (cbWidth) and height (cbHeight) of the current block. If the Euclidean distance between the reference positions pointed to by several BV candidates is less than R, they are clustered, where the radius (R) can be determined by formula (2): R = log2((cbWidth·cbHeight)>>MIN_PU_SIZE) (2)
[0189] In formula (2), cbWidth represents the width of the current block, cbHeight represents the height of the current block, and MIN_PU_SIZE represents the maximum PU (Prediction Unit) size, R. The prediction unit PU here is the candidate.
[0190] Referring to Figure 6, Figure 6 is a schematic diagram of an optional candidate clustering provided in an embodiment of the present application. As shown in Figure 6, when the current block (CB) corresponds to a non-RRIBC mode, if the number of valid BVP candidates exceeds two, a maximum of six BVP candidates in the candidate list (shown as BVP1, BVP2, BVP3, BVP4, BVP5, and BVP6 in Figure 6) are clustered according to the Euclidean distance between the reference blocks (PB) corresponding to each BVP. As shown in Figure 6, up to six BVP candidates are grouped according to the distance between the reference blocks (PB) pointed to by each BVP. As shown in Figure 6, the distance (L2) between the PB pointed to by BVP1 and the PB pointed to by BVP2, as well as the distance (L2) between the PB pointed to by BVP1 and the PB pointed to by BVP5 are similar and are both less than the radius R (Radius), then the PB pointed to by BVP1, the PB pointed to by BVP2, and the PB pointed to by BVP5 can be clustered and divided into Group 1 (Group 1). If the distance (L2) between the PB pointed to by BVP3 and the PB pointed to by BVP6 is less than the radius R, the PB pointed to by BVP3 and BVP6 can be clustered and divided into Group 2. The PB pointed to by BVP4 is far away from the other five PBs, so the PB pointed to by BVP4 can be clustered and divided into Group 3.
[0191] The clustering method is performed in the order of the candidate list. In each class, the BVP with the lowest TM cost is selected as the representative candidate of the group. The representative candidates of all classes are then template-sorted, and the representative candidates of the first two groups are selected for the motion estimation process.
[0192] Exemplarily, taking the candidate clustering shown in FIG6 as an example, first, the cost values of the templates of the PB pointed to by BVP1, the PB pointed to by BVP2, and the PB pointed to by BVP5 in group 1 and the template of the current block (CB) are calculated and sorted, and the PB with the smallest cost value in group 1 is selected as the representative candidate of group 1. The cost values of the templates of the PB pointed to by BVP3 and the PB pointed to by BVP6 in group 2 and the template of the current block (CB) are calculated and sorted, and the PB with the smallest cost value in group 2 is selected as the representative candidate of group 2. The cost value of the PB pointed to by BVP4 in group 3 and the template of the current block (CB) is calculated and sorted, and the PB with the smallest cost value in group 2 is selected as the representative candidate of group 2. Subsequently, the representative candidates of group 1, the representative candidates of group 2, and the representative candidates of group 3 are sorted, and the first two representative candidates with the smallest cost values are selected as the final candidate.
[0193] When the mode of the current block is the IBC ABVP mode, if the current block corresponds to the Reconstruction-Reordered IBC (RRIBC) mode, the candidate items are adjusted to point to the boundary of the valid IBC search area according to the horizontal or vertical direction of the RRIBC mode.
[0194] Among them, the flipped IBC mode is a method of copying the reference block in the order of row or column coordinates and using it as the prediction value when the BV specifies the reference block to predict the current block due to the symmetry of some graphics and texts.
[0195] For S22, in the case of IBV merge mode (general_merge_flag[xCb][yCb] is true) and IBC ABVP mode (general_merge_flag[xCb][yCb] is false), the candidate index bvIdx is derived as follows, where general_merge_flag indicates whether it is IBC merge mode. The above process can be expressed as: bvIdx = general_merge_flag[xCb][yCb]? merge_idx[xCb][yCb]: bvp_L0_flag[xCb][yCb]
[0196] The decoder parses the bitstream and obtains the index value of the candidate item of the current block through the bitstream when the general_merge_flag of the current block is true (equal to 1).
[0197] For S23, in IBC merge mode, the specific bvL can be obtained according to the index bvIdx and the block vector candidate list bvCandList. The above process can be expressed as: bvL[0]=bvCandList[bvIdx][0] bvL[1]=bvCandList[bvIdx][1]
[0198] This bvL is the final BV.
[0199] When the current block is in IBC MBVD (Merge Mode with Block Vector Difference) mode, similar to MMVD in VVC's inter-frame technology, starting from a candidate in the IBC merge list, a candidate is selected from a set of candidate points corresponding to a predefined distance and direction set, and its corresponding block vector is used as the final BV. That is, an offset is performed with the current candidate as the center point to obtain a set of candidate points, and another candidate is selected from this set of candidate points as the final BV.
[0200] For example, when the current block is in IBC MBVD mode, the distance set is defined as {1pel, 2pel, 4pel, 8pel, 12pel, 1-pel, 24pel, 32-pel, 40pel, 48pel, 56pel, 64pel, 72pel, 80pel, 88pel, 96pel, 104pel, 112pel, 120pel, 128pel}, and the BVD directions are positive and negative horizontal directions and positive and negative vertical directions.
[0201] The base candidate is selected from the first five candidates in the reordered IBC merge list, and all possible MBVD refinement positions (i.e., 20×4 candidates) of each base candidate are reordered based on the SAD cost between the template and its reference at each refinement position. Finally, the first 8 refinement positions with the smallest template SAD are retained for MBVD index encoding. IBC-MBVD candidates do not inherit the flip type from RR-IBC encoded neighboring blocks. The MBVD index is binarized using a Rice code with parameter equal to 1.
[0202] Furthermore, in IBC merge mode, after obtaining the bvL based on the above information, the BV can be locally refined using TM. Specifically, a search is performed within a small range centered on the obtained bvL, and the optimal BV within this range is selected as the final BV based on the minimum template matching cost. In other words, each of the eight obtained candidates (bvL) is further locally refined to obtain the optimal BV within the local range of the candidate as the final BV.
[0203] For example, the TM refinement is performed on the candidate items in the candidate list of the IBC merge mode. A specific implementation process is as follows:
[0204] When building a candidate, the flip type defaults to no flip;
[0205] In the case of IBC TM merge mode, the transmission syntax element specifies whether to perform integer-pixel precision TM refinement. The location of the refined motion vector and the template used in each refinement step must comply with the reference area constraint.
[0206] Search near the candidate's pointed position, using the SAD between the reference block's template and the current block's template to determine the optimal position. This includes, but is not limited to, the following search methods: The search range is [-8, 8], and a whole-pixel diamond search is first performed, searching eight points near the center, as shown in the upper left corner of the figure below. The maximum number of searches is 375. After the optimal position is determined for the first time, the search continues, searching five points at even-numbered positions and three points at odd-numbered positions.
[0207] Exemplarily, referring to FIG7a, FIG7a is a flowchart of an optional candidate search provided by an embodiment of the present application. As shown in FIG7a, when the optimal position (shown as a grid circle in FIG7a) is determined to be an even point for the first time, the five points shown by the black circle in FIG7a are continued to be searched. Referring to FIG7b, FIG7b is a flowchart of an optional candidate search provided by an embodiment of the present application. As shown in FIG7b, when the optimal position (shown as a grid circle in FIG7a) is determined to be an odd point for the first time, the three points shown by the black circle in FIG7b are continued to be searched. After the above-mentioned diamond search process, a whole-pixel cross search is performed, which is only performed once. That is, after the diamond search finds the current optimal position, four positions below, right, above, and left of the current optimal position, which are one pixel away, are checked to update the final optimal position, that is, to update the refined candidate list.
[0208] When the current block is in IBC ABVP mode, the bvL obtained by indexing bvIdx and the block vector candidate list bvCandList is the predicted bvL. The actual bvL also needs to be added with the block vector difference (BVD). The generalized specific process includes S31 to S33:
[0209] S31. Get the horizontal and vertical components of BVD, where MvdL0 is the forward motion vector difference. The above process can be expressed as: bvd[0] = MvdL0[xCb][yCb][0] bvd[1] = MvdL0[xCb][yCb][1]
[0210] S32. Round the predicted bvL obtained above, where the right shift parameter AmvrShift is rounded and the left shift parameter AmvrShift is used to improve the resolution. The above process can be expressed as: offset=(AmvrShift==0)? 0:((1<<(AmvrShift-1))-1) bvL[0]=Sign(bvL[0])*(((Abs(bvL[0])+offset)>>AmvrShift)<<AmvrShift) bvL[1]=Sign(bvL[1])* (((Abs(bvL[1])+offset)> >AmvrShift)< <AmvrShift)
[0211] S33, the derivation of the real bvL is as follows, and its range needs to be controlled between -217 and 217 –1 The above process can be expressed as: u[0]=(bvL[0]+bvd[0]+218)%218 bvL[0]=(u[0]>=217)? (u[0]-218):u[0] u[1]=(bvL[1]+bvd[1]+218)%218 bvL[1]=(u[1]>=217)? (u[1]-218):u[1]
[0212] Refer to Figure 8, which is a schematic diagram of an optional block vector determination provided by an embodiment of the present application. As shown in Figure 8, after obtaining the horizontal and vertical components of the BVD of the current PU, the final BV is determined according to the BVP and BVD corresponding to the current PU.
[0213] It should be noted that the method for obtaining BVD in S31 can also be inferred from syntax elements obtained through other encoding methods. The BVD of IBC ABVP uses sub-pixels, whole pixels, or 4 pixels as units. During encoding, its sign can be predicted, and the suffix of the exponential Golomb code obtained after its binarization can also be predicted. Therefore, its syntax elements can be obtained from the BVD value: whether it is 0, prefix, sign, and suffix multiple information definitions. These information combinations and analysis can obtain the actual value of BVD.
[0214] A specific embodiment is as follows: context coding is used to identify whether BVD is 0. The value of (absolute value - 1) is binarized using first-order exponential Columbus, the first 5 bins of the EG1 prefix are context coded, and the remaining prefixes are bypass coded. Up to 4 bins of the EG1 suffix use context coding to transmit the prediction index, and the other bins of the EG1 suffix use bypass coding. The two bins of the sign bit use context coding to transmit the symbol prediction index. The prediction of the suffix bins that transmit the prediction index horizontally and vertically, and the prediction of the bins that transmit the symbol index horizontally and vertically, are sorted using the current block template and the template at the corresponding BV to derive the prediction index of the suffix and the symbol.
[0215] The final derived BV should be within the specified range (coordinate range in rows and columns).
[0216] For example, in VVC, the reference area of IBC is 128x128. For a CTU of 128x128 size, the reference area can refer to Figures 9a-9d. Figure 9a is a schematic diagram of an optional IBC reference area provided in an embodiment of the present application. As shown in Figure 9a, the reference blocks in the reference area of the current block may include three reference blocks in the shaded part, of which one block marked with an "×" will not be used as a reference block; Figure 9b is a schematic diagram of an optional IBC reference area provided in an embodiment of the present application. As shown in Figure 9b, the reference blocks in the reference area of the current block may include three reference blocks in the shaded part. 9c is a third schematic diagram of an optional IBC reference area provided in an embodiment of the present application. As shown in FIG9c , the reference blocks in the reference area of the current block may include the three reference blocks in the shaded part, of which the three blocks marked with “×” will not be used as reference blocks. FIG9d is a fourth schematic diagram of an optional IBC reference area provided in an embodiment of the present application. As shown in FIG9d , the reference blocks in the reference area of the current block may include the three reference blocks in the shaded part, of which the four blocks marked with “×” will not be used as reference blocks.
[0217] For example, in ECM7.0, when the CTU size is 128, the reference area of IBC can be referred to as shown in Figure 10. As shown in Figure 10, a box represents a CTU, the grid box represents the CTU where the current block is located, and the black dot box represents its reference area. Specifically, for the current block to be encoded, assuming it is located in CTU (m, n), the reference area includes CTUs with indices of (m–2, n–2)…(W, n–2)…(0, n–1)…(W, n–1), (0, n)…(m, n). Where W represents the maximum horizontal index within the current Tile, Slice, or Picture.
[0218] For example, when the CTU size is 256, the reference area of IBC can refer to FIG11 . As shown in FIG11 , the reference area is adjusted, a box represents a CTU, a grid box represents the CTU where the current block is located, and a black dot box represents its reference area.
[0219] The range of block vector search (or local search) for each block is restricted to horizontally [–(C<<1), C>>2] and vertically [–C, C>>2] to accommodate the reference area extension, where C represents the CTU size.
[0220] The above is the method for obtaining brightness BV in IBC correlation mode. In IntraTMP mode, the input and output of obtaining brightness BV are basically the same, but the specific methods are different. The main difference is that the search process is performed at the decoding end.
[0221] Referring to FIG12 , FIG12 is a schematic diagram of an optional Intra TMP prediction process provided in an embodiment of the present application. As shown in FIG12 , in the IntraTMP mode, the decoding end searches within a certain range near the zero-value BV, and searches for the optimal BV (BV_best) with the minimum template matching cost as the criterion, or constructs an optimal BV list. It should be noted that the zero-value BV is the current block (Current Block), that is, the decoder searches within a certain range near the current block with the minimum template matching cost as the goal, obtains the reference frame (Ref Block) corresponding to the optimal template (T-BEST) with the minimum template matching cost, and uses the optimal BV (BV-BEST) corresponding to the reference frame as the final BV.
[0222] Refer to Figure 13, which is a schematic diagram of an optional template search area provided in an embodiment of the present application. As shown in Figure 13, the dotted box represents the searchable area of the template, and the middle box of the dotted box is the current block. It can be seen that the searchable area of the template of the current block is a square area. Compared with the reference area of the IBC mode shown in Figures 10 and 11, the reference area of the Intra TMP mode shown in Figure 13 is more square, while the reference area of the IBC mode is slender.
[0223] Referring to FIG. 14 , FIG. 14 is a schematic diagram of a prediction process of an optional IntraTMP mode provided in an embodiment of the present application. As shown in FIG. 14 , the prediction process of the IntraTMP mode includes S41 to S43:
[0224] S41, determining the current template type;
[0225] S42, obtaining the current template pixel;
[0226] S43. Determine a block vector within a predefined search range.
[0227] In IntraTMP mode, the template type is first obtained. Based on this, template pixels are retrieved around the current block and searched within a predefined search range. If the final BV is determined directly from the template matching cost, the entry with the lowest template matching cost is the final BV. If the template matching cost is used to create a BV candidate list, the list stores multiple BV candidates in ascending order of template matching cost. The corresponding syntax elements in the bitstream determine the final BV candidate. This BV candidate is used as the final BV.
[0228] (2) Chroma component:
[0229] In the IBC mode or IntraTMP mode of the chroma component, the BV of the chroma can be derived based on the luma BV, and then prediction and reconstruction can be achieved based on this BV. The process of deriving the chroma BV based on the luma BV is as follows:
[0230] Input: bvL of brightness (1 / 16 pixel accuracy);
[0231] Output: bvC (Block Vector Chroma) of chroma (1 / 32 pixel accuracy).
[0232] The derivation process can be directly scaling, or using TM to refine after scaling. The scaling operation example is as follows: bvC[0] = ((bvL[0]>>(3+SubWidthC))*32) bvC[1] = ((bvL[1]>>(3+SubHeightC))*32)
[0233] The variables SubWidthC and SubHeightC depend on the chroma format sampling structure specified by sps_chroma_format_idc. Table 1 shows the corresponding relationship between the chroma format sampling structure. The specific correspondence is as follows:
[0234] Table 1
[0235] The detailed operation example can be as follows:
[0236] Refer to Figure 15, which is a schematic diagram of the refinement process of an optional IntraTMP mode provided in an embodiment of the present application. As shown in Figure 15, the current block is refined using TM template matching, that is, after obtaining the luminance BV, the offset position is found using the position of the chrominance block and the BV, and the template is used to perform refinement near the offset position, and the optimal BV (refined BV) is obtained with the minimum TMcost as the standard.
[0237] Regarding S12 (using BV to obtain prediction samples), the steps of S12 are introduced in detail below:
[0238] Input: Luma position (xCb, yCb) specifies the top left corner sample of the current coding block relative to the top left corner luma sample of the current picture, a variable cbWidth specifies the width of the current coding block in luma samples, a variable cbHeight specifies the height of the current coding block in luma samples, a block vector bv and a variable cIdx specifies the color component index of the current block.
[0239] Output: array predSamples of predicted samples.
[0240] An example of the derivation process of directly copying the prediction block is as follows:
[0241] When cIdx is equal to 0, that is, the brightness component, for x = xCb..xCb + cbWidth-1 and y = yCb..yCb + cbHeight-1:
[0242] xVb=(x+(bvL[0]>>4))&(IbcBufWidthY-1) / / Restore to the position of the component with integer pixel precision
[0243] yVb=(y+(bvL[1]>>4))&(CtbSizeY-1)
[0244] predSamples[x][y]=ibcVirBuf[0][xVb][yVb] / / prediction value of a pixel
[0245] IbcBufWidthY is the width of the brightness pixels of the reconstructed buffer stored in IBC, CtbSizeY is the size of the CTU (Coding Tree Unit), and ibcVirBuf is the reconstructed pixels stored in IBC.
[0246] When cIdx is not equal to 0, that is, the chrominance component, for x=xCb / SubWidthC..xCb / SubWidthC+cbWidth / SubWidthC-1 and y=yCb / SubHeightC..yCb / SubHeightC+cbHeight / SubHeightC-1:
[0247] xVb=(x+(bvC[0]>>4))&(IbcBufWidthC-1)
[0248] yVb=(y+(bvC[1]>>4))&((CtbSizeY / subHeightC)-1)
[0249] predSamples[x][y]=ibcVirBuf[cIdx][xVb][yVb]
[0250] In addition to the basic acquisition methods above, there is also an IBC flip mode method that requires horizontally or vertically flipping the prediction area to obtain the predicted value. For example, a syntax element indicates whether to flip the area and, if so, whether to flip horizontally or vertically. The decoder then reversely rearranges the pixels in the reference area horizontally or vertically according to the syntax to obtain the predicted pixel values of the coding block.
[0251] In addition to the above acquisition process, a template can also be used to establish a model between the current block and the predicted area. The predicted block is then processed according to the model to obtain the predicted value of the current block. For example, the IBC LIC mode, applied to IBC merge and IBC ABVP, uses a linear equation to compensate for local illumination changes. Similar to the LIC for inter-frame prediction in VVC, the parameters of the linear equation can be expressed as a scaling parameter α and an offset parameter β, that is, α*p[x]+β to compensate for illumination changes, where p[x] is the reference sample pointed to by the BV at position x in the current image. The linear model parameters are derived using the least squares method.
[0252] On the basis of the above-obtained prediction values, weighted prediction with other intra-frame prediction methods can also be implemented, and the result after weighted prediction is used as the final prediction result.
[0253] The weighted prediction method includes the inter-frame and intra-frame hybrid prediction CIIP method borrowed from VVC, that is, the result obtained by the above prediction process is weightedly combined with the prediction result obtained by the ordinary intra-frame directional prediction mode at each pixel position; it also includes the inter-frame geometric prediction mode GPM method borrowed from VVC, that is, based on a wedge division, the results obtained by using different prediction modes in different wedge areas, and weighted mixing in accordance with certain rules near the wedge division line.
[0254] An example of the operation process in IBC CIIP mode: the IBC prediction value of the current block is weightedly fused with the prediction value of a certain intra-frame mode. The IBC prediction part can be obtained by applying the conventional merge, TM merge, MBVD and ABVP modes.
[0255] For the case where the IBC prediction part is the conventional IBC merge, TM merge, and MBVD mode, the weight ratio of IBC prediction and intra prediction is 13:3. The intra mode includes the TIMD mode of the current block and the intra prediction mode at the candidate BV. If the second prediction mode in the intra mode is the same as the first prediction mode, it is determined whether the first prediction mode is the PLANAR mode. If so, the second prediction mode is replaced with the horizontal prediction mode, otherwise the second prediction mode is replaced with the PLANAR mode.
[0256] For the case where the IBC prediction part is the ABVP mode, the weight ratio of the IBC prediction and the intra-frame prediction is 1:1, and the TIMD mode is obtained as the first prediction mode of the intra-frame prediction mode. If the derived prediction mode is the horizontal prediction mode, PLANAR is obtained as the second prediction mode of the intra-frame prediction mode. Otherwise, the horizontal prediction mode is obtained as the second prediction mode of the intra-frame prediction mode.
[0257] Example of the operation process in IBC GPM mode: Conventional merge and TM merge can be applied to the IBC prediction part. With only one partition set to IBC and one to INTRA mode, the intra prediction mode (IPM) candidate list is constructed using the same method as for inter GPM, and the IPM candidate list size is predefined as 3.
[0258] In a specific implementation, there are 48 geometric partitioning modes, which can be divided into two geometric partitioning mode sets. Table 1 is a first geometric partitioning model set, and Table 2 is a second geometric partitioning model set.
[0259] Table 2
[0260] Table 3
[0261] When using IBC GPM, the IBC GPM geometric partitioning mode set flag is transmitted to indicate whether the first or second geometric partitioning mode set is selected, followed by the geometric partitioning mode index. The IBC-GPM frame partition flag is then transmitted to indicate whether intra prediction is used for the first sub-partition. The intra prediction portion requires the transmission of the intra prediction mode index, and the IBC prediction portion requires the transmission of the MEGRE index.
[0262] In the IntraTMP mode, in addition to the basic copying method to obtain the predicted value, there is also a method of using the corresponding positions of multiple BVs to perform filtering and fusion to obtain the predicted value, as well as a method of fusion prediction with the ordinary Intra mode.
[0263] For example, after obtaining a BV candidate list by template matching during the decoding end region search process, the first N items (eg, N=3) are selected for weighted fusion. This method may be referred to as the IntraTMP Fusion mode.
[0264] Alternatively, after obtaining an optimal BV, multiple points are taken around the BV and their corresponding predicted values are weighted and fused to obtain the predicted value. This method is called the IntraTMP FLM mode.
[0265] Alternatively, after obtaining an optimal BV, the predicted value obtained using this BV is weighted and combined with the predicted value obtained using a common intra mode, such as the planar mode or a directional mode (the directional mode can be inferred from the TIMD or located in the intra candidate table using the index transmitted in the bitstream). This method is called the IntraTMP CIIP mode.
[0266] Referring to FIG. 16 , FIG. 16 is a schematic diagram 1 of an optional search process of the Intra TMP Fusion mode provided in an embodiment of the present application. As shown in FIG. 16 , the search process of the Intra TMP Fusion mode includes S51 to S53:
[0267] S51, initialization;
[0268] S52, determining a search area of the template in the current frame;
[0269] S53 : Search and determine block vectors corresponding to N matching templates in the search area.
[0270] In the IntraTMP Fusion mode, the Intra TMP Fusion technology searches for and determines block vectors corresponding to N matching templates in the search area.
[0271] Further, referring to FIG17 , FIG17 is a second schematic diagram of an optional search process of the Intra TMP Fusion mode provided in an embodiment of the present application. As shown in FIG17 , S53 includes S531 to S532:
[0272] S531: Determine the value of the number N of candidate templates.
[0273] Intra TMP Fusion technology first requires determining the value of N. N can be preset to a constant, such as 4, or within a certain range, such as any integer between 2 and 8. With this preset range, the encoder can determine the optimal N value using a coarse selection of cost 1, a coarse selection of cost 2, or a coarse selection of cost 3, a fine selection of cost 4, and transmit this optimal N value to the decoder in the form of a bitstream. Costs 1, 2, 3, and 4 can be any of several cost functions used to measure patterns, such as SAD, SATD, MSE, MAD, and RDO.
[0274] S532, determine the matching template comparison criteria and record the block vectors BV corresponding to the N matching templates.
[0275] After determining the number N of candidate templates, it is necessary to select the block vectors BV corresponding to the N more matching candidate templates using a certain comparison criterion.
[0276] The operations for searching for matching templates and recording block vectors BV are similar to those in the prior art Intra TMP. The difference is that Intra TMP Fusion filters and records multiple block vectors BVn based on these operations. Based on the block vectors BVn, that is, based on the template offsets pXn and pYn, a matching template is obtained. The template matching costs are then calculated and the BVs corresponding to the N matching templates with the lowest costs are recorded. These N matching templates are referred to as candidate templates.
[0277] The template matching cost may be one of the cost functions used to measure the pattern, such as SAD, SATD, MSE, MAD, RDO, and correlation coefficient.
[0278] The matching cost comparison criterion used in this main solution is MAD (Mean Absolute Difference), and the calculation formula is as follows:
[0279] In formula (3), M represents the number of pixels in the template, curT m Represents the template of the current block, refT m Represents the template of the reference block.
[0280] In the Intra TMP Fusion technology of the present application, after obtaining the block vectors BV corresponding to N candidate templates, N candidate reconstructed blocks are obtained through the BV, and then the N candidate reconstructed blocks are weightedly fused to obtain the prediction block of the current coding block. Referring to Figure 18, Figure 18 is a schematic diagram of a process for determining a prediction value in an optional Intra TMP Fusion mode provided in an embodiment of the present application. As shown in Figure 18, the process for determining a prediction value in the Intra TMP Fusion mode includes S61 to S63:
[0281] S61. Obtain N candidate reconstruction blocks.
[0282] After obtaining the block vector BV corresponding to N candidate templates, directly according to BV n Get the candidate reconstruction block RefBlock in the current image n , among which BV n The horizontal offset is pX n , the vertical offset is pY n , where n=0,1…,N-1.
[0283] It is realized by simple translation copy. The specific operation is: for x=0…nTbW-1, y=0…nTbH-1,
[0284] RefBlock n [x][y]=recSamples[x+pXn][y+pYn]
[0285] Among them, recSamples represents the reconstructed pixels of the current frame.
[0286] S62: Determine the weighted fusion weight.
[0287] After obtaining N candidate reconstructed blocks RefBlock, it is necessary to calculate the weight W of the weighted fusion of the N candidate reconstructed blocks. The weight can be a predefined value or a value adaptively calculated using a cost value, pixel value, etc.
[0288] The Intra TMP Fusion technology in the embodiment of the present application utilizes the candidate template refT n The reconstruction value and the template to be predicted refpredT n The weighted fusion weights are derived by minimizing the MSE of the pixel values.
[0289] Specifically, the MSE minimization process takes the autocorrelation matrix of the first P matching reference samples refT, the cross-correlation vector of the first P matching reference samples refT and the adjacent template samples curT of the current coding block as input, and outputs the weight of the reconstructed block corresponding to each matching reference item.
[0290] S63. Weighted fusion generates a prediction value.
[0291] The prediction block is calculated based on each candidate reconstructed block and its corresponding weighted fusion weight. Specifically, the value of each candidate reconstructed block and its corresponding weight are multiplied and accumulated to obtain the current prediction block (i.e., weighted prediction). The calculation formula is as follows:
[0292] For x = 0 ... nTbW-1, y = 0 ... nTbH-1, the predicted value is calculated as follows:
[0293] Each predicted value predSamples x,y The spatial storage is the output prediction block of Intra TMP Fusion.
[0294] In IntraTMP FLM mode, the linear filter model is built using the best matching template found in the previous step and the current coding block template. This mainly involves the following two steps: determining the reconstruction area for calculating the filter coefficients and calculating the filter coefficients.
[0295] The number of filter taps is defined as nTap. nTap can be a constant or a variable; for example, the number of filter taps can be dynamically adjusted based on the block size. Referring to Figures 19a through 19f, Figures 19a through 19f illustrate six optional fixed filter shapes provided in an embodiment of the present application. In this embodiment of the present application, filters of various shapes can be selected.
[0296] The available reconstructed pixels are determined based on the template type refTemplateType. The reconstruction region used to calculate the filter coefficients is determined based on the available reconstructed pixels. This region is called R and typically includes all available reconstructed pixels in the template. Within region R, the filter coefficients are derived by minimizing the MSE.
[0297] For each current block to be encoded, a set of filter coefficients can be obtained by the above method. The filter coefficients are c0, c1, c2...c nTap-1 Then, the best matching reconstructed block is filtered according to the filter shape and filter coefficients, that is, the filter coefficients and the reference pixels at the corresponding position are weighted and summed to obtain the current pixel to be predicted. Let ref be the best matching reconstructed block, which is defined as follows: ref[i][j] = recSamples[i+pX_BEST][j+pY_BEST] (5)
[0298] Wherein, i=0,…,nTbW-1, j=0,…,nTbH-1, recSamples are the reconstructed pixels of the current image or the reference image, and pX_BEST and pY_BEST are the horizontal and vertical components of the optimal motion vector.
[0299] The specific prediction value calculation process is as follows:
[0300] Assuming that the number of filter taps nTap is 5, the shape of the filter is shown in the left figure below. c0 to c4 are the tap coefficients of the filter, where the yellow dot corresponding to the tap coefficient C0 is the current pixel to be predicted Y pred [i][j] are the reconstructed pixels ref[i][j] at the corresponding positions in the best matching block; the remaining brown dots are the reconstructed pixels in the best matching reconstructed block that are adjacent to the current spatial position. Referring to FIG20 , FIG20 is a schematic diagram of an optional TM-FLM prediction provided by an embodiment of the present application. As shown in FIG20 , the points in the prediction block are the predicted pixels Y pred [i][j].
[0301] The specific prediction value calculation process is as follows:
[0302] For each current pixel to be predicted (i, j), the pixel position in the filter template is defined as (k, l), then the corresponding reconstructed pixel in the best matching block during filtering is defined as ref[i+k][j+l], and each filter coefficient at the position (k, l) in the filter template is defined as c k,l : c k,l =c n (6)
[0303] Wherein, n=0,…,nTap-1, k and l are between -1 and 1.
[0304] For i=0, ..., nTbW-1, j=0, ..., nTbH-1: Y pred [i][j]=∑ k ∑ l ref[i+k][j+l]×C ,,k,l (7)
[0305] The final predicted pixels are: predSamples[i][j]=Clip3(0,(1<<BitDepth)-1,Ypred[i][j]) (8)
[0306] in,
[0307] In summary, in the related art, there is a problem of overlapping BV ranges expressed between the IBC mode and the IntraTMP mode, which leads to low expression efficiency and affects the coding efficiency.
[0308] The following describes in detail a decoding method proposed in an embodiment of the present application in conjunction with the intra-frame prediction part.
[0309] A decoding method is provided in an embodiment of the present application. Referring to FIG21 , a flowchart of an optional decoding method provided in an embodiment of the present application is shown. As shown in FIG21 , the method may include:
[0310] S101: Decode a code stream and determine a prediction mode for a current block.
[0311] It should be noted that the decoding method of the embodiment of the present application is applied to a decoder. In addition, the decoding method can refer to a video intra-frame prediction method; it can also be a decoding method based on an intra-frame block copy mode, which mainly improves the intra-frame prediction algorithm in related technologies. It can avoid the problem of overlapping candidate BV ranges corresponding to the current block during video decoding based on the IBC mode and video decoding based on non-IBC mode, thereby achieving better intra-frame prediction effects and improving video decoding efficiency.
[0312] In an embodiment of the present application, the prediction mode based on intra-frame block copy is a block-level encoding and decoding mode. Similar to the inter-frame technology, the encoding end performs motion search to find the best block vector (BV) for each coding block, where the motion search here can be understood as block matching, and the block vector here can also be called a motion vector (MV). The block vector is a vector pointing from the current block to the reference block. The difference from the inter-frame technology is that the best block vector of IBC is obtained by searching in the reconstructed area of the frame where the current coding block is located (that is, the current coding frame), while the inter-frame motion vector is obtained by searching the adjacent reference frames of the current coding frame in the time domain.
[0313] In an embodiment of the present application, the decoder parses the bitstream to determine first syntax identification information, wherein the first syntax identification information is used to indicate whether the current block uses a prediction mode based on intra block copy (IBC).
[0314] Exemplarily, the first syntax identification information may be represented by pred_mode_ibc_flag.
[0315] In an embodiment of the present application, if the first syntax identification information is a first value, it is determined that the first syntax identification information indicates that the current block uses a prediction mode based on an intra-frame block copy; if the first syntax identification information is a second value, it is determined that the first syntax identification information indicates that the current block does not use a prediction mode based on an intra-frame block copy.
[0316] It should be noted that in the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the first syntax identification information can be a parameter written in the profile or a flag value, which is not specifically limited here.
[0317] For example, for the first value and the second value, the first value can be set to 1 and the second value can be set to 0; or, the first value can be set to 0 and the second value can be set to 1; or, the first value can be set to true and the second value can be set to false; or, the first value can be set to false and the second value can be set to true; but this is not specifically limited here.
[0318] Exemplarily, when pred_mode_ibc_flag is true or 1, it is determined that the current block uses the intra block copy based prediction mode.
[0319] In the embodiment of the present application, taking the flag written into the bitstream as an example, assuming that the first value is set to 0 (false) and the second value is set to 1 (true), if the value of the first syntax identification information is 0 (false), then it can be determined that the current block uses the prediction mode based on the intra-frame block copy, that is, there is no need to execute the decoding method described in the embodiment of the present application; if the value of the first syntax identification information is 1 (true), then it can be determined that the current block does not use the prediction mode based on the intra-frame block copy, that is, it is necessary to execute the decoding method described in the embodiment of the present application.
[0320] It should be noted that the embodiments of the present application do not limit the expression forms of the first value and the second value.
[0321] In an embodiment of the present application, when the first syntax identification information indicates that the current block uses a prediction mode based on intra block copy, the code stream is parsed to determine the second syntax identification information, wherein the second syntax identification information is used to indicate whether the current block uses a prediction mode based on intra block merging (IBC merge).
[0322] Exemplarily, the second syntax identification information may be represented by general_merge_flag.
[0323] In an embodiment of the present application, if the second syntax identification information is the third value, it is determined that the second syntax identification information indicates that the current block uses a prediction mode based on intra-frame block merging; if the second syntax identification information is the fourth value, it is determined that the first syntax identification information indicates that the current block uses a prediction mode based on intra-frame block merging.
[0324] It should be noted that in the embodiment of the present application, the third value is different from the fourth value, and the third value and the fourth value can be in parameter form or in digital form. Specifically, the first syntax identification information can be a parameter written in the profile or a flag value, which is not specifically limited here.
[0325] Exemplarily, for the third value and the fourth value, the third value can be set to 1 and the fourth value can be set to 0; or, the third value can be set to 0 and the fourth value can be set to 1; or, the third value can be set to true and the fourth value can be set to false; or, the third value can be set to false and the fourth value can be set to true; but this is not specifically limited here.
[0326] In the embodiment of the present application, taking the flag written into the bitstream as an example, assuming that the third value is set to 1 (true) and the fourth value is set to 0 (false), if the value of the second syntax identification information is 0 (false), then it can be determined that the current block does not use the prediction mode based on intra-frame block merging, that is, there is no need to execute the decoding method described in the embodiment of the present application; if the value of the second syntax identification information is 1 (true), then it can be determined that the current block uses the prediction mode based on intra-frame block merging, that is, it is necessary to execute the decoding method described in the embodiment of the present application.
[0327] Exemplarily, when general_merge_flag is true or 1, it is determined that the current block uses a prediction mode based on intra-frame block merging.
[0328] It should be noted that the embodiments of the present application do not limit the forms of expression of the third value and the fourth value.
[0329] The prediction mode indicated by the second syntax identification information may be the prediction mode of the current block.
[0330] S102: If the prediction mode of the current block indicates that the current block uses a prediction mode based on intra-frame block merging to determine a prediction value of the current block, determine a first block vector candidate list for the current block.
[0331] In the embodiment of the present application, when the second syntax identification information (prediction mode of the current block) indicates that the current block uses a prediction mode based on intra-block merging, a first block vector candidate list (merge list) for the current block is determined. That is, when the decoder parses and finds that the prediction mode of the current block is based on intra-block merging, the decoder needs to construct a first block vector candidate list for the current block.
[0332] Exemplarily, for the IBC mode, in the process of obtaining the bvL (luminance block vector) of the current block, it is necessary to establish a block vector candidate list bvCandList (ie, the first block vector candidate list), which includes multiple candidates.
[0333] In an embodiment of the present application, the first block vector candidate list includes Q first candidate block vectors, where Q is a positive integer greater than or equal to 1.
[0334] It should be noted that the number of block vectors in the block vector candidate list bvCandList (first block vector candidate list) of different modes (i.e., the number of first candidate block vectors) may be different. For example, the number of first candidate block vectors that may be included in the first block vector candidate list of the IBC merge mode is 6, and the number of first candidate block vectors that may be included in the first block vector candidate list of the IBC ABVP mode is 2. Of course, in an embodiment of the present application, the number of first candidate block vectors that may be included in the first block vector candidate list may be Q, where Q is a positive integer greater than or equal to 1. The number Q of first candidate block vectors is pre-set, and the decoder and the encoder agree or specify the value of N corresponding to the number of first candidate block vectors corresponding to the current block. For example, the decoder and the encoder agree or specify that the value of N corresponding to the number of first candidate block vectors corresponding to the current block is 8. Then, in the process of encoding or decoding the current block, the number of candidate items contained in the first block vector candidate list established by the decoder or the encoder is 8.
[0335] Furthermore, determining the first block vector candidate list for the current block may include: performing spatial candidate derivation; adding the spatial candidate to the candidate list, and after adding the spatial candidate to the candidate list, checking the number of valid items in the candidate list; if the candidate list does not reach the specified number of items, continuing historical candidate derivation, availability detection, and addition; continuing to check the number of valid items in the candidate list and adding other available candidates until the specified number of items is reached, thereby obtaining the first block vector candidate list for the current block. The steps of determining the first block vector candidate list for the current block may refer to S211 to S215 described above and will not be repeated here.
[0336] S103: Determine a first area range, and determine a second block vector candidate list based on the first block vector candidate list and the first area range.
[0337] In the embodiment of the present application, the decoder removes the block vectors belonging to the first region based on the first block vector candidate list to obtain a second block vector candidate list.
[0338] In an embodiment of the present application, when the second syntax identification information indicates that the current block uses a prediction mode based on intra-frame block merging, the decoder removes the block vectors belonging to the first area in the current frame where the current block is located according to each first candidate block vector in the first block vector candidate list, thereby obtaining a second block vector candidate list.
[0339] In some embodiments of the present application, the first area range includes one of the following: the size of the current coding unit; the size of the reconstructed area in the current frame; the size of the image boundary of the current block; the search range of the Intra TMP mode, and the present application does not impose any restrictions on this.
[0340] It should be noted that the first area range is only an example, and the first area range can be any one of the above, or a combination of multiple first area ranges. The decoder can first determine the first area range. This application does not impose any limitation on this.
[0341] It can be understood that when the first area range is the search range of the Intra TMP mode, the decoder removes the block vectors in the search range of the Intra TMP mode based on the first block vector candidate list to obtain the final second block vector candidate list. At this time, the second block vector candidate list does not contain block vectors in the search range of the Intra TMP mode. Therefore, the second block vector candidate list does not contain block vectors in the candidate list corresponding to the Intra TMP mode. This effectively avoids the problem of overlapping block vectors expressed in the candidate lists of the IBC mode and the Intra TMP mode, thereby saving bit stream bits and improving video decoding efficiency.
[0342] It can be understood that since there are no overlapping block vectors in the second block vector candidate list and the candidate list corresponding to the Intra TMP mode, when using the same bit stream, the embodiment of the present application can diversify the block vectors and thus improve the efficiency of video decoding.
[0343] For example, in an embodiment of the present application, based on the first block vector candidate list, the block vectors belonging to the search range of the Intra TMP mode are removed to obtain the second block vector candidate list. This can remove the BV information redundantly represented between different modes, improve the expression efficiency of the header information, and thus improve the overall video decoding efficiency.
[0344] S104. Determine a prediction value of the current block based on the second block vector candidate list.
[0345] In an embodiment of the present application, after determining the second block vector candidate list, the decoder determines a prediction value of the current block according to the second block vector candidate list.
[0346] In some embodiments of the present application, S104 may include the following steps:
[0347] Decode the code stream and determine the index information of the extended candidate block vector;
[0348] Determine a first block vector according to the extended candidate block vector index information and the second block vector candidate list;
[0349] A reference block of the current block is determined according to the first block vector, and a prediction value of the current block is determined according to the reference block.
[0350] In the embodiment of the present application, the decoder can determine the extended candidate block vector index information of the current block by parsing the bitstream.
[0351] In the embodiment of the present application, the extended candidate block vector index information is a unique identifier of the first block vector in the second block vector candidate list.
[0352] In the embodiment of the present application, the extended candidate block vector index information is a preset index value of the block vector, that is, the extended candidate block vector index information is an index value of the block vector specified or agreed upon by both the encoder and the decoder.
[0353] In the embodiment of the present application, each block vector included in the second block vector candidate list has unique index information.
[0354] In the embodiment of the present application, the extended candidate block vector index information may be a parameter written in a profile, or may be a flag value, which is not specifically limited here.
[0355] Exemplarily, the extended candidate block vector index information may be represented in digital form. For example, the extended candidate block vector index information 0 may represent the first block vector in the second block vector candidate list. Furthermore, the extended candidate block vector index information may be represented in different digital bases, which is not limited in this application.
[0356] In an embodiment of the present application, the decoder determines the first block vector in the second block vector candidate list based on the extended candidate block vector index information of the current block, and then uses the decoded block pointed to by the first block vector as the reference block of the current block, and determines the prediction value of the current block based on the reference block.
[0357] In some embodiments of the present application, the method further comprises:
[0358] Decode the code stream and determine the prediction difference of the current block;
[0359] A reconstructed value of the current block is determined according to the predicted value of the current block and the predicted difference value of the current block.
[0360] In the embodiment of the present application, if the current block has a prediction residual, the decoder parses the bitstream to determine the prediction residual of the current block, and performs weighting or other processing on the prediction value of the current block and the prediction difference of the current block to obtain a reconstructed value of the current block. If the current block does not have a prediction residual, the decoder directly uses the prediction value of the current block as the reconstructed value of the current block.
[0361] In an embodiment of the present application, the decoder first parses the code stream to determine the first syntax identification information; then, if the first syntax identification information indicates that the current block uses a prediction mode based on intra block copy, the decoder determines the second syntax identification information; if the second syntax identification information indicates that the current block uses a prediction mode based on intra block merging, the decoder determines a first block vector candidate list for the current block; subsequently, the decoder removes the block vectors belonging to the first area based on the first block vector candidate list to obtain a second block vector candidate list; finally, the decoder obtains a prediction value of the current block based on the second block vector candidate list.
[0362] On the one hand, when the decoder removes the block vectors in the first area range according to the first block vector candidate list, the final second block vector candidate list is obtained. At this time, the block vectors in the first area range do not exist in the second block vector candidate list. In this way, the problem of the overlap between the candidate list of the IBC mode and the block vectors expressed in the first area range can be effectively avoided, thereby saving code stream bits and improving the video decoding efficiency. On the other hand, since there are no overlapping block vectors in the second block vector candidate list and the first area range, when using the same bit stream, the embodiment of the present application can increase the diversity of block vectors and thus improve the efficiency of video decoding. On the other hand, since the second block vector candidate list effectively removes the block vector information in the first area range, thereby improving the expression efficiency of the header information in the code stream, the overall video decoding efficiency can be improved.
[0363] In some embodiments of the present application, S103 includes S1031 to S1033:
[0364] S1031: If it is determined that the current block uses a prediction mode based on merging with block vector differences within a frame, determine candidate block vector index information.
[0365] In an embodiment of the present application, when the current block uses a prediction mode based on intra-frame merged block vector difference (IBC MBVD), the decoder determines the candidate block vector index information corresponding to the current block by parsing the bitstream.
[0366] In the embodiment of the present application, the candidate block vector index information is used to indicate the first candidate block vector (basic candidate) in the first block vector candidate list.
[0367] For example, for the IBC MBVD intra block copy mode, assuming the first block vector candidate list includes eight candidates, the first five of the eight candidates are used as basic candidates. The decoder parses the bitstream to obtain candidate block vector index information for these five first candidate block vectors. Furthermore, based on the candidate block vector index information, the decoder obtains the five first candidate block vectors indicated by the candidate block vector index information from the first block vector candidate list.
[0368] S1032. Determine, based on the first block vector candidate list, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information.
[0369] In an embodiment of the present application, after determining the first candidate block vector indicated by the candidate block vector index information based on the first block vector candidate list, the decoder determines at least one first extended candidate block vector corresponding to each first candidate block vector.
[0370] In the embodiment of the present application, the first extended candidate block vector is determined according to the distance set and the block vector prediction direction.
[0371] In some embodiments of the present application, S1032 may include S10321 to S10323:
[0372] S10321. Determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1.
[0373] In an embodiment of the present application, before determining the first candidate block vector using the candidate block vector index information, it is necessary to sort the candidate block vectors in the first block vector candidate list to obtain a sorted first block vector candidate list, and then determine N first candidate block vectors with the minimum template cost in the sorted first block vector candidate list.
[0374] In an embodiment of the present application, during the sorting process of the candidate block vectors in the first block vector candidate list, a template matching cost may be calculated for each candidate block vector in the first block vector candidate list. The template matching cost values of the candidate block vectors are then re-sorted, and the sorting is determined based on the template matching cost values. In other words, the candidates in the sorted IBC merge list are the first block vector candidate list sorted based on the template matching cost values.
[0375] It should be noted that, in the process of sorting the candidate block vectors in the first block vector candidate list, the candidate block vectors can be sorted in descending order according to the template matching cost value, and the candidate block vectors can be sorted in ascending order according to the template matching cost value. This application does not impose any restrictions on this.
[0376] Exemplarily, when the candidate block vectors are sorted in ascending order of template matching cost values, the template matching cost value of the first candidate block vector in the sorted first block vector candidate list is the smallest.
[0377] In the embodiment of the present application, N is pre-set, that is, the decoder and the encoder can pre-determine or agree on the number of first candidate block vectors.
[0378] In the embodiment of the present application, N can also be obtained by parsing the code stream. In this case, N is determined by the encoder.
[0379] In an embodiment of the present application, when the candidate block vectors in the first block vector candidate list can be sorted in ascending order according to the template matching cost value, the N first candidate block vectors with the smallest template cost are determined by the first N candidate block vectors in the sorted first block vector candidate list; when the candidate block vectors in the first block vector candidate list can be sorted in descending order according to the template matching cost value, the N first candidate block vectors with the smallest template cost are determined by the last N candidate block vectors in the sorted first block vector candidate list.
[0380] For example, in the IBC MBVD mode, N may be 8.
[0381] S10322. Determine a first candidate block vector from N first candidate block vectors based on the candidate block vector index information.
[0382] In an embodiment of the present application, the decoder determines a first candidate block vector from N first candidate block vectors based on the acquired candidate block vector index information.
[0383] Exemplarily, the decoder determines one first candidate block vector that needs to be refined from eight first candidate block vectors according to the candidate block vector index information.
[0384] It should be noted that what is shown here is that the decoder determines one first candidate block vector from N first candidate block vectors according to one candidate block vector index information.
[0385] Therefore, when the decoder obtains multiple candidate block vector index information by parsing the bitstream, for each candidate block vector index information, the first candidate block vector corresponding to each candidate block vector index information is determined from the N first candidate block vectors.
[0386] That is, assuming that the decoder determines that N is 8 by parsing the bitstream and obtains 5 candidate block vector index information, the decoder determines 5 candidate block vector index information among the 8 first candidate block vectors based on the 5 candidate block vector index information.
[0387] S10323. Refine the first candidate block vector based on the distance set and the block vector prediction direction, and determine at least one first extended candidate block vector corresponding to the first candidate block vector.
[0388] In an embodiment of the present application, after the decoder determines the first candidate block vector based on the candidate block vector index information, it refines each first candidate block vector based on a preset distance set and block vector prediction direction to determine at least one first extended candidate block vector corresponding to each first candidate block vector.
[0389] In the embodiment of the present application, multiple refined positions of the first candidate block vector may be determined according to a predefined distance set and a predefined direction set, thereby achieving refinement of the first candidate block vector.
[0390] In the embodiment of the present application, the distance set and the block vector prediction direction are preset, that is, the distance set and the block vector prediction direction are specified or agreed upon by both the decoder and the encoder.
[0391] For example, in the IBC MBVD mode, the distance set can be defined as {1pel, 2pel, 4pel, 8pel, 12pel, 16pel, 24pel, 32pel, 40pel, 48pel, 56pel, 64pel, 72pel, 80pel, 88pel, 96pel, 104pel, 112pel, 120pel, 128pel}, where the distance set includes multiple pixel distances (such as 2pel).
[0392] For example, in the IBC MBVD mode, block vector prediction directions can be defined as two positive and negative horizontal directions and two positive and negative vertical directions. Therefore, the block vector prediction directions include positive and negative horizontal directions and positive and negative vertical directions, for a total of four directions. Furthermore, using 20 distances and four directions, 20×4 refinement positions (i.e., the first extended candidate block vector) can be determined.
[0393] S1033: Remove the block vectors belonging to the first region from the at least one first extended candidate block vector to obtain a second block vector candidate list.
[0394] In an embodiment of the present application, after determining at least one first extended candidate block vector corresponding to each first candidate block vector, the decoder removes the block vectors belonging to the first area in the at least one first extended candidate block vector, thereby obtaining a second block vector candidate list.
[0395] It can be understood that during the decoding process, during the block decoding process, for the scenario of intra-frame merging mode prediction, the decoder can remove the overlapping block vectors or block vectors within the first area, such as in the intra-frame template matching prediction mode, based on the first block vector candidate list of the current block, and determine the second block vector candidate list, and determine the prediction value of the current block based on the de-redundant block vector in the second block vector candidate list. In this way, since the redundancy of the block vector is reduced, the expression efficiency of the header information can be improved when the de-redundant block vector or block vector is used for decoding, thereby improving the video decoding efficiency.
[0396] In some embodiments of the present application, S1033 may include:
[0397] For each first extended candidate block vector, block vectors corresponding to reference blocks belonging to the first region are removed in sequence to obtain at least one filtered first extended candidate block vector;
[0398] A second block vector candidate list is determined based on the filtered at least one first extended candidate block vector.
[0399] In an embodiment of the present application, for each first extended candidate block vector, if the first extended candidate block vector belongs to the block vector corresponding to the reference block within the first area range, the first extended candidate block vector is removed; if the first extended candidate block vector does not belong to the block vector corresponding to the reference block within the first area range, the first extended candidate block vector is added to the second block vector candidate list.
[0400] Exemplarily, the 20×4 first extended candidate block vectors corresponding to each first candidate block vector are screened in sequence, and the first extended candidate block vectors belonging to the first area range are removed to obtain at least one screened first extended candidate block vector, and the at least one screened first extended candidate block vector is added to the second block vector candidate list to determine the second block vector candidate list.
[0401] In some embodiments of the present application, sequentially removing block vectors corresponding to reference blocks belonging to the first region from each first extended candidate block vector to obtain at least one filtered first extended candidate block vector may include:
[0402] If the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector falls within the horizontal range of the preset range, and the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector falls within the vertical range of the preset range, then the first candidate reference block pointed to by any first extended candidate block vector belongs to the reference block of the first area range;
[0403] The any first extended candidate block vector is removed, and the removal process is continued on other first extended candidate block vectors to obtain at least one filtered first extended candidate block vector.
[0404] In some embodiments of the present application, if the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the horizontal range of a preset range, or the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the vertical range of a preset range, then the first candidate reference block pointed to by any first extended candidate block vector does not fall within the reference block of the first area range; and the any first extended candidate block vector is a filtered first extended candidate block vector corresponding to each candidate block vector.
[0405] In the embodiment of the present application, the first area range may be a preset search range of the template in the IntraTMP mode.
[0406] In an embodiment of the present application, the horizontal range of the preset range can be expressed as -searchRangeWidth and searchRangeWidth. Among them, -searchRangeWidth represents the left boundary of the horizontal range of the preset range, and searchRangeWidth represents the left and right boundaries of the horizontal range of the preset range. The vertical range of the preset range can be expressed as -searchRangeHeight and searchRangeHeight. Among them, -searchRangeHeight represents the lower boundary of the horizontal range of the preset range, and searchRangeHeight represents the upper boundary of the horizontal range of the preset range.
[0407] In an embodiment of the present application, the horizontal boundary position and the vertical boundary position of the first candidate reference block pointed to by the first extended candidate block vector are judged respectively. If the horizontal boundary position and the vertical boundary position of the first candidate reference block fall within the horizontal range and the vertical range of the preset range, the first extended candidate block vector is removed, that is, it does not participate in the subsequent sorting process.
[0408] In some embodiments of the present application, if the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the horizontal range of a preset range, or the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the vertical range of a preset range, then the first candidate reference block pointed to by any first extended candidate block vector does not fall within the reference block of the first area range; and the any first extended candidate block vector is a filtered first extended candidate block vector corresponding to each candidate block vector.
[0409] Exemplarily, the process of determining the horizontal boundary position and the vertical boundary position of the first candidate reference block pointed to by the first extended candidate block vector can be expressed as: (bvCand[x0][y0][disIdx][dirIdx][0]>-searchRangeWidth&& bvCand[x0][y0][disIdx][dirIdx][0])<searchRangeWidth&&bvCand[x0][y0][disIdx][dirIdx][1]> -searchRangeHeight&&bvCand[x0][y0][disIdx][dirIdx][1] <searchRangeHeight)。
[0410] Among them, bvCand represents the first candidate reference block pointed to by the first extended candidate block vector, -searchRangeWidth represents the left boundary of the horizontal range of the preset range, searchRangeWidth represents the left and right boundaries of the horizontal range of the preset range, -searchRangeHeight represents the lower boundary of the horizontal range of the preset range, and searchRangeHeight represents the upper boundary of the horizontal range of the preset range.
[0411] Exemplarily, in an embodiment of the present application, if (bvCand[x0][y0][disIdx][dirIdx][0] > -searchRangeWidth && bvCand[x0][y0][disIdx][dirIdx][0] < searchRangeWidth && bvCand[x0][y0][disIdx][dirIdx][1] > -searchRangeHeight && bvCand[x0][y0][disIdx][dirIdx][1] < searchRangeHeight) corresponding to the first candidate reference block of the first extended candidate block vector is true, then the first extended candidate block vector is removed. If (bvCand[x0][y0][disIdx][dirIdx][0] > -searchRangeWidth && bvCand[x0][y0][disIdx][dirIdx][0] < searchRangeWidth && bvCand[x0][y0][disIdx][dirIdx][1] > -searchRangeHeight && bvCand[x0][y0][disIdx][dirIdx][1] < searchRangeHeight) corresponding to the first candidate reference block of the first extended candidate block vector is false, then the reference blocks of the first extended candidate block vector that do not belong to the first region range are selected, and the first extended candidate block vector is used as a filtered first extended candidate block vector corresponding to the candidate block vector.
[0412] In some embodiments of the present application, based on at least one filtered first extended candidate block vector, a second block vector candidate list is determined, including:
[0413] Determine M first extended candidate block vectors corresponding to the minimum template cost from at least one filtered first extended candidate block vector; where M is an integer greater than or equal to 1;
[0414] Based on the M first extended candidate block vectors, a second block vector candidate list is constructed.
[0415] In an embodiment of the present application, after determining at least one filtered first extended candidate block vector, at least one first extended candidate block vector is further sorted to obtain M sorted first extended candidate block vectors.
[0416] It should be noted that, in the process of sorting the first extended candidate block vectors, each first extended candidate block vector can be sorted in descending order according to the template matching cost value, and each first extended candidate block vector can be sorted in ascending order according to the template matching cost value. This application does not impose any restrictions on this.
[0417] Exemplarily, when the first extended candidate block vectors are sorted in ascending order of template matching cost values, the template matching cost value of the first extended candidate block vector in the sorted first extended candidate block vectors is the smallest.
[0418] In the embodiment of the present application, M is pre-set, that is, the decoder and the encoder can pre-define or agree on the number of first extended candidate block vectors.
[0419] In the embodiment of the present application, M can also be obtained by parsing the code stream. In this case, M is determined by the encoder.
[0420] In an embodiment of the present application, when the first extended candidate block vectors are sorted in ascending order of template matching cost values, M first extended candidate block vectors with minimum template costs are determined by the first N candidate block vectors in the sorted first extended candidate block vectors; when the first extended candidate block vectors are sorted in descending order of template matching cost values, M first extended candidate block vectors with minimum template costs are determined by the last M candidate block vectors in the sorted first extended candidate block vectors.
[0421] In an embodiment of the present application, M first extended candidate block vectors are added to the second block vector candidate list to obtain the second block vector candidate list.
[0422] In some embodiments of the present application, when the number of the at least one first extended candidate block vector after screening is less than M, other candidate block vectors are determined from the first block vector candidate list; the other candidate block vectors are different from the N candidate block vectors;
[0423] Based on the other candidate block vectors and the screened at least one first extended candidate block vector, M first extended candidate block vectors are determined.
[0424] In an embodiment of the present application, when the number of at least one first extended candidate block vector after screening is less than M, other candidate block vectors are continuously determined from the first block vector candidate list until the number of other candidate block vectors and the at least one first extended candidate block vector after screening is M.
[0425] It can be understood that while removing the redundant representation of BV information, the decoder determines other first candidate vectors from the first block vector candidate list; the other first candidate vectors are different from the N first candidate vectors, that is, optional options that were originally unexpressible are introduced into the reordered list, thereby improving the representation range of the BV candidate list in the corresponding mode, increasing the diversity of options, and also improving video decoding efficiency.
[0426] In some embodiments of the present application, S1032 may further include:
[0427] Determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1;
[0428] Removing the block vectors belonging to the first region from the N first candidate block vectors to obtain a third block vector candidate list;
[0429] Based on the third block vector candidate list, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information is determined.
[0430] In an embodiment of the present application, before determining the first candidate block vector using the candidate block vector index information, it is necessary to sort the candidate block vectors in the first block vector candidate list to obtain a sorted first block vector candidate list, and then determine N first candidate block vectors with the minimum template cost in the sorted first block vector candidate list.
[0431] In an embodiment of the present application, during the sorting process of the candidate block vectors in the first block vector candidate list, a template matching cost may be calculated for each candidate block vector in the first block vector candidate list. The template matching cost values of the candidate block vectors are then re-sorted, and the sorting is determined based on the template matching cost values. In other words, the candidates in the sorted IBC merge list are the first block vector candidate list sorted based on the template matching cost values.
[0432] In an embodiment of the present application, when the candidate block vectors in the first block vector candidate list can be sorted in ascending order according to the template matching cost value, the N first candidate block vectors with the smallest template cost are determined by the first N candidate block vectors in the sorted first block vector candidate list; when the candidate block vectors in the first block vector candidate list can be sorted in descending order according to the template matching cost value, the N first candidate block vectors with the smallest template cost are determined by the last N candidate block vectors in the sorted first block vector candidate list.
[0433] In the embodiment of the present application, each of the N first candidate block vectors is judged in turn, and the first candidate block vectors belonging to the first area are removed to obtain a third block vector candidate list.
[0434] In the embodiment of the present application, according to the candidate block vector index information, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information is determined in the third block vector candidate list.
[0435] It is understood that during the decoding process, when performing block decoding, for a scenario based on a prediction mode with intra-frame merging with block vector differences, the decoder can perform deduplication based on at least one extended candidate block vector corresponding to N first candidate block vectors in the first block vector candidate list of the current block. Overlapping block vectors or block vectors of the at least one extended candidate block vector within a first region, such as in an intra-frame template matching prediction mode, are removed to determine a second block vector candidate list. A prediction value of the current block is determined based on the de-redundant block vectors in the second block vector candidate list. This reduces block vector redundancy, thereby improving header information expression efficiency when using the de-redundant block vectors or block vectors for decoding, thereby improving video decoding efficiency.
[0436] In some embodiments of the present application, determining, based on the third block vector candidate list, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information includes:
[0437] determining a first candidate block vector from a third block vector candidate list based on the candidate block vector index information;
[0438] Based on the distance set and the block vector prediction direction, the first candidate block vector is refined to determine at least one first extended candidate block vector corresponding to the first candidate block vector.
[0439] In an embodiment of the present application, after the decoder determines the first candidate block vector based on the candidate block vector index information, it refines each first candidate block vector based on a preset distance set and block vector prediction direction to determine at least one first extended candidate block vector corresponding to each first candidate block vector.
[0440] In the embodiment of the present application, multiple refined positions of the first candidate block vector may be determined according to a predefined distance set and a predefined direction set, thereby achieving refinement of the first candidate block vector.
[0441] In the embodiment of the present application, the distance set and the block vector prediction direction are preset, that is, the distance set and the block vector prediction direction are specified or agreed upon by both the decoder and the encoder.
[0442] For example, in the IBC MBVD mode, the distance set can be defined as {1pel, 2pel, 4pel, 8pel, 12pel, 16pel, 24pel, 32pel, 40pel, 48pel, 56pel, 64pel, 72pel, 80pel, 88pel, 96pel, 104pel, 112pel, 120pel, 128pel}, where the distance set includes multiple pixel distances (such as 2pel).
[0443] For example, in the IBC MBVD mode, block vector prediction directions can be defined as two positive and negative horizontal directions and two positive and negative vertical directions. Therefore, the block vector prediction directions include positive and negative horizontal directions and positive and negative vertical directions, for a total of four directions. Furthermore, using 20 distances and four directions, 20×4 refinement positions (i.e., the first extended candidate block vector) can be determined.
[0444] In some embodiments of the present application, the decoding method further includes:
[0445] Decode the code stream and determine the index information of the extended candidate block vector;
[0446] determining a first block vector from at least one first extended candidate block vector according to the extended candidate block vector index information;
[0447] A reference block of the current block is determined according to the first block vector, and a prediction value of the current block is determined according to the reference block.
[0448] In the embodiment of the present application, the decoder determines the extended candidate block vector index information corresponding to the current block by parsing the bitstream.
[0449] In an embodiment of the present application, the decoder determines, based on the extended candidate block vector index information, a first block vector corresponding to the extended candidate block vector index information from at least one first extended candidate block vector.
[0450] In the embodiment of the present application, if the current block has a prediction residual, the decoder parses the bitstream to determine the prediction residual of the current block, and performs weighting or other processing on the prediction value of the current block and the prediction difference of the current block to obtain a reconstructed value of the current block. If the current block does not have a prediction residual, the decoder directly uses the prediction value of the current block as the reconstructed value of the current block.
[0451] In some embodiments of the present application, block vectors belonging to the first region are removed from the N first candidate block vectors to obtain a third block vector candidate list, which may include:
[0452] For each of the N first candidate block vectors, block vectors corresponding to the reference block belonging to the first region are removed in sequence to obtain at least one filtered first candidate block vector;
[0453] A third block vector candidate list is determined based on the filtered at least one first candidate block vector.
[0454] In an embodiment of the present application, for each first candidate block vector, if the first candidate block vector belongs to the block vector corresponding to the reference block within the first area range, the first candidate block vector is removed; if the first candidate block vector does not belong to the block vector corresponding to the reference block within the first area range, the first candidate block vector is added to the third block vector candidate list.
[0455] Exemplarily, each first candidate block vector is screened in turn, and the first candidate block vectors belonging to the first area range are removed to obtain at least one screened first candidate block vector, and the at least one screened first candidate block vector is added to the third block vector candidate list to determine the third block vector candidate list.
[0456] In some embodiments of the present application, for each of the N first candidate block vectors, the block vector corresponding to the reference block belonging to the first region is removed in sequence to obtain at least one filtered first candidate block vector, which may include:
[0457] offsetting each of the N first candidate block vectors in sequence to determine at least one second candidate block vector corresponding to each first candidate block vector;
[0458] If the maximum horizontal boundary position and the minimum horizontal boundary position in at least one second candidate reference block pointed to by the at least one second candidate block vector are within the horizontal range of the preset range, and the maximum vertical boundary position and the minimum vertical boundary position in at least one second candidate reference block are within the vertical range of the preset range, then the first candidate block vector corresponding to the at least one second candidate block vector belongs to the block vector of the first area range;
[0459] The first candidate block vector is removed, and the removal process is continued on other first candidate block vectors to obtain at least one first candidate block vector after screening.
[0460] It is understood that during the decoding process, during the block decoding process, for a scenario based on a prediction mode with intra-frame block vector difference merging, the decoder can deduplicate the N first candidate block vectors in the first block vector candidate list of the current block. Within a first region of the N first candidate block vectors, for example, in an intra-frame template matching prediction mode, overlapping block vectors or block vectors are removed to determine a third block vector candidate list, and a prediction value of the current block is determined based on the de-redundant block vectors in the third block vector candidate list. This reduces redundancy in block vectors, thereby improving the efficiency of header information expression when decoding using the de-redundant block vectors or block vectors, thereby improving video decoding efficiency.
[0461] In some embodiments of the present application, the decoding method further includes:
[0462] If the maximum horizontal boundary position or the minimum horizontal boundary position in at least one second candidate reference block pointed to by at least one second candidate block vector does not fall within the horizontal range of the preset range, or the maximum vertical boundary position or the minimum vertical boundary position in at least one second candidate reference block does not fall within the vertical range of the preset range, then the candidate block vector corresponding to the at least one second candidate block vector does not fall within the block vector of the first area; the first candidate block vector corresponding to the at least one second candidate block vector is used as a screened first candidate block vector;
[0463] The removal process is continued on other first candidate block vectors until the processing of N first candidate block vectors is completed, or N filtered first candidate block vectors are obtained.
[0464] In the embodiment of the present application, the first area range may be a preset search range of the template in the IntraTMP mode.
[0465] In an embodiment of the present application, the horizontal range of the preset range can be expressed as -searchRangeWidth and searchRangeWidth. Among them, -searchRangeWidth represents the left boundary of the horizontal range of the preset range, and searchRangeWidth represents the left and right boundaries of the horizontal range of the preset range. The vertical range of the preset range can be expressed as -searchRangeHeight and searchRangeHeight. Among them, -searchRangeHeight represents the lower boundary of the horizontal range of the preset range, and searchRangeHeight represents the upper boundary of the horizontal range of the preset range.
[0466] In this embodiment of the present application, the second candidate block vector is obtained by offsetting the first candidate block vector. The first candidate block vector is offset by a preset offset pixel, where the preset offset pixel is a pre-set pixel. For example, the first candidate block vector is offset horizontally or vertically by a preset offset pixel (128 pel).
[0467] In an embodiment of the present application, the horizontal boundary position and the vertical boundary position of the second candidate reference block pointed to by the second candidate block vector are judged respectively. If the horizontal boundary position and the vertical boundary position of the second candidate reference block fall within the horizontal range and the vertical range of the preset range, the second candidate block vector is removed, that is, it does not participate in the subsequent sorting process.
[0468] In some embodiments of the present application, if the horizontal boundary position of the second candidate reference block pointed to by any second candidate block vector does not fall within the horizontal range of a preset range, or the vertical boundary position of the second candidate reference block pointed to by any second candidate block vector does not fall within the vertical range of a preset range, then the first candidate reference block pointed to by any second candidate block vector does not fall within the reference block of the first area range; and the any second candidate block vector is a filtered first candidate block vector corresponding to each candidate block vector.
[0469] Exemplarily, the process of determining the horizontal boundary position and the vertical boundary position of the second candidate reference block pointed to by the second candidate block vector can be expressed as:
[0470] (bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx][dirHorIdx_left][0]>-searchRangeWidth&&bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx][dirHorIdx_right][0]<searchRangeWidth&&bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx_up][dirHorIdx][1]> -searchRangeHeight&&bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx_down][dirHorIdx][1] <searchRangeHeight)
[0471] Among them, bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx][dirHorIdx_left][0] represents the horizontal boundary position of the first candidate reference block pointed to by the second candidate block vector, bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx_up][dirHorIdx][1] represents the vertical boundary position of the first candidate reference block pointed to by the second candidate block vector, -searchRangeWidth represents the left boundary of the horizontal range of the preset range, searchRangeWidth represents the left and right boundaries of the horizontal range of the preset range, -searchRangeHeight represents the lower boundary of the horizontal range of the preset range, and searchRangeHeight represents the upper boundary of the horizontal range of the preset range.
[0472] Exemplarily, if (bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx][dirHorIdx_left][0] > -searchRangeWidth && bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx][dirHorIdx_right][0] < searchRangeWidth && bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx_up][dirHorIdx][1] > -searchRangeHeight && bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx_down][dirHorIdx][1] < searchRangeHeight) corresponding to the candidate reference block pointed to by the second candidate block vector is true, then the first candidate block vector corresponding to the second candidate block vector belongs to the block vector within the first region range; and the first candidate block vector is removed, and the removal process continues for other first candidate block vectors to obtain at least one filtered first candidate block vector.
[0473] Exemplarily, if (bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx][dirHorIdx_left][0] > -searchRangeWidth && bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx][dirHorIdx_right][0] < searchRangeWidth && bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx_up][dirHorIdx][1] > -searchRangeHeight && bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx_down][dirHorIdx][1] < searchRangeHeight) corresponding to the candidate reference block pointed to by the second candidate block vector is false, then the candidate block vector corresponding to this second candidate block vector does not belong to the block vector within the first region range; and the first candidate block vector corresponding to this second candidate block vector is used as a filtered first candidate block vector.
[0474] It can be understood that during the decoding process of the decoder, in the process of block decoding, for the scenario of the prediction mode based on the merge with block vector difference within a frame, duplicate removal can be performed on the N first candidate block vectors in the first block vector candidate list of the current block. For the N first candidate block vectors within the first region range, for example, in the intra-frame template matching prediction mode, the overlapping block vectors or block vectors are removed to determine the third block vector candidate list, and the predicted value of the current block is determined based on the non-redundant block vectors in the third block vector candidate list. And for each first candidate block vector in the third block vector candidate list, at least one corresponding first extended candidate block vector is removed again to obtain the second block vector candidate list, and the predicted value of the current block is determined based on the second block vector candidate list. For at least one first extended candidate block vector within the first region range, for example, in the intra-frame template matching prediction mode, the overlapping block vectors or block vectors are removed, further reducing the redundancy of the block vectors. Then, when using the non-redundant block vectors or block vectors for decoding, the expression efficiency of the header information can be improved, and thus the video decoding efficiency can be improved.
[0475] In some embodiments of the present application, removing the block vectors within the first region range based on the first block vector candidate list to obtain the second block vector candidate list may include:
[0476] If it is determined that the current block uses a prediction mode based on template matching merging, K third candidate block vectors with minimum template costs are determined from the first block vector candidate list; K is an integer greater than or equal to 1;
[0477] Block vectors belonging to the first area are removed from the K third candidate block vectors to obtain a second block vector candidate list.
[0478] In an embodiment of the present application, when the current block uses the IBC merge prediction mode, the decoder determines K third candidate block vectors with minimum template costs from the first block vector candidate list.
[0479] In an embodiment of the present application, before determining the first candidate block vector using the candidate block vector index information, it is necessary to sort the candidate block vectors in the first block vector candidate list to obtain a sorted first block vector candidate list, and then determine K first candidate block vectors with the minimum template cost in the sorted first block vector candidate list.
[0480] It should be noted that, in the process of sorting the candidate block vectors in the first block vector candidate list, the candidate block vectors can be sorted in descending order according to the template matching cost value, and the candidate block vectors can be sorted in ascending order according to the template matching cost value. This application does not impose any restrictions on this.
[0481] Exemplarily, when the candidate block vectors are sorted in ascending order of template matching cost values, the template matching cost value of the first candidate block vector in the sorted first block vector candidate list is the smallest.
[0482] In the embodiment of the present application, K is pre-set, that is, the decoder and the encoder can pre-determine or agree on the number of first candidate block vectors.
[0483] In the embodiment of the present application, K can also be obtained by parsing the code stream. In this case, K is determined by the encoder.
[0484] In an embodiment of the present application, when the candidate block vectors in the first block vector candidate list can be sorted in ascending order according to the template matching cost value, the K third candidate block vectors with the minimum template cost are determined by the first K candidate block vectors in the sorted first block vector candidate list; when the candidate block vectors in the first block vector candidate list can be sorted in descending order according to the template matching cost value, the K third candidate block vectors with the minimum template cost are determined by the last K candidate block vectors in the sorted first block vector candidate list.
[0485] In an embodiment of the present application, after determining K third candidate block vectors, the decoder removes block vectors belonging to the first area range from the K third candidate block vectors in at least one first extended candidate block vector, thereby obtaining a second block vector candidate list.
[0486] In some embodiments of the present application, removing the block vectors belonging to the first region from the K third candidate block vectors to obtain a second block vector candidate list may include:
[0487] If, among the K third candidate block vectors, the horizontal boundary position of the third candidate reference block pointed to by any third candidate block vector falls within the horizontal range of the preset range, and the vertical boundary position of the third candidate reference block pointed to by any third candidate block vector falls within the vertical range of the preset range, then the third candidate reference block pointed to by any third candidate block vector belongs to the reference block of the first area range;
[0488] Removing any third candidate block vector of the reference block belonging to the first area, and continuing to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained;
[0489] A second block vector candidate list is determined based on the filtered at least one third candidate block vector.
[0490] It is understood that during the decoding process, when performing block decoding, in the scenario of a template matching-based merge prediction mode, the decoder can perform deduplication based on the K third candidate block vectors in the first block vector candidate list of the current block. Within a first region, for example, in an intra-frame template matching prediction mode, overlapping block vectors or block vectors of the K third candidate block vectors are removed to determine a second block vector candidate list. Local refinement is then performed based on the de-redundant block vectors in the second block vector candidate list to determine the prediction value of the current block. This reduces block vector redundancy, thereby improving the efficiency of header information expression when using the de-redundant block vectors or block vectors for decoding, thereby improving video decoding efficiency.
[0491] In the embodiment of the present application, the first area range may be a preset search range of the template in the IntraTMP mode.
[0492] In an embodiment of the present application, the horizontal range of the preset range can be expressed as -searchRangeWidth and searchRangeWidth. Among them, -searchRangeWidth represents the left boundary of the horizontal range of the preset range, and searchRangeWidth represents the left and right boundaries of the horizontal range of the preset range. The vertical range of the preset range can be expressed as -searchRangeHeight and searchRangeHeight. Among them, -searchRangeHeight represents the lower boundary of the horizontal range of the preset range, and searchRangeHeight represents the upper boundary of the horizontal range of the preset range.
[0493] In an embodiment of the present application, the horizontal boundary position and the vertical boundary position of the third candidate reference block pointed to by the third candidate block vector are judged respectively. When the horizontal boundary position and the vertical boundary position of the third candidate reference block belong to the horizontal range and the vertical range of the preset range, the third candidate block vector is removed, that is, it does not participate in the subsequent sorting process.
[0494] Exemplarily, the process of determining the horizontal boundary position and the vertical boundary position of the third candidate reference block pointed to by the third candidate block vector can be expressed as follows:
[0495] (bvCand[x0][y0][merge_idx][0]>-searchRangeWidth&&bvCand[x0][y0][merge_idx][0]<searchRangeWidth&&bvCand[x0][y0][merge_idx][1]> -searchRangeHeight&&bvCand[x0][y0][merge_idx][1] <searchRangeHeight)
[0496] Among them, bvCand[x0][y0][merge_idx][0] represents the horizontal boundary position of the third candidate reference block pointed to by the third candidate block vector, bvCand[x0][y0][merge_idx][1] represents the vertical boundary position of the third candidate reference block pointed to by the third candidate block vector, -searchRangeWidth represents the left boundary of the horizontal range of the preset range, searchRangeWidth represents the left and right boundaries of the horizontal range of the preset range, -searchRangeHeight represents the lower boundary of the horizontal range of the preset range, and searchRangeHeight represents the upper boundary of the horizontal range of the preset range.
[0497] Exemplarily, if (bvCand[x0][y0][merge_idx][0] > -searchRangeWidth && bvCand[x0][y0][merge_idx][0] < searchRangeWidth && bvCand[x0][y0][merge_idx][1] > -searchRangeHeight && bvCand[x0][y0][merge_idx][1] < searchRangeHeight) is true, then the third candidate reference block pointed to by the third candidate block vector belongs to the reference block within the first region range; and the third candidate block vector for the reference block within the first region range is removed, and the removal process continues for other third candidate block vectors until the processing of K third candidate block vectors is completed or K filtered third candidate block vectors are obtained.
[0498] Exemplarily, if (bvCand[x0][y0][merge_idx][0] > -searchRangeWidth && bvCand[x0][y0][merge_idx][0] < searchRangeWidth && bvCand[x0][y0][merge_idx][1] > -searchRangeHeight && bvCand[x0][y0][merge_idx][1] < searchRangeHeight) is false, then the third candidate reference block pointed to by the third candidate block vector does not belong to the reference block within the first region range, and the third candidate block vector is used as the filtered third candidate block vector.
[0499] In an embodiment of the present application, at least one filtered third candidate block vector is added to the second block vector candidate list to determine the second block vector candidate list.
[0500] In some embodiments of the present application, removing the block vectors within the first region range from the K third candidate block vectors to obtain the second block vector candidate list may include:
[0501] If, among the K third candidate block vectors, the first extended horizontal boundary position of the third candidate reference block pointed to by any one of the third candidate block vectors belongs to the horizontal range within the preset range, and the first extended vertical boundary position of the third candidate reference block pointed to by any one of the third candidate block vectors belongs to the vertical range within the preset range, then the third candidate reference block pointed to by any one of the third candidate block vectors belongs to the reference block within the first region range;
[0502] The first extended horizontal boundary position is the horizontal boundary position of the third candidate reference block obtained by extending the first horizontal search range by the horizontal boundary of the third candidate reference block; the first extended vertical boundary position is the vertical boundary position of the third candidate reference block obtained by extending the first vertical search range by the vertical boundary of the third candidate reference block; the first horizontal search range and the first vertical search range are search ranges specified in the IBC™ merge mode;
[0503] Removing any third candidate block vector of the reference block belonging to the first area, and continuing to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained;
[0504] A second block vector candidate list is determined based on the filtered at least one third candidate block vector.
[0505] In the embodiment of the present application, the first area range may be a preset search range of the template in the IntraTMP mode.
[0506] In an embodiment of the present application, the horizontal extent of a preset range can be expressed as -searchRangeWidth and searchRangeWidth. Where -searchRangeWidth represents the left horizontal extent of the preset range, and searchRangeWidth represents the left and right horizontal extents of the preset range. The vertical extent of a preset range can be expressed as -searchRangeHeight and searchRangeHeight. Where -searchRangeHeight represents the lower horizontal extent of the preset range, and searchRangeHeight represents the upper horizontal extent of the preset range.
[0507] In an embodiment of the present application, the first extended horizontal boundary position and the first extended vertical boundary position of the third candidate reference block pointed to by the third candidate block vector are judged respectively. If the first extended horizontal boundary position and the first extended vertical boundary position of the third candidate reference block belong to the horizontal range and the vertical range of the preset range, the third candidate block vector is removed, that is, it does not participate in the subsequent sorting process.
[0508] Exemplarily, the process of determining the first extended horizontal boundary position and the first extended vertical boundary position of the third candidate reference block pointed to by the third candidate block vector can be expressed as follows:
[0509] (bvCand[x0][y0][merge_idx][0]-searchRangeWidthIBCTM>-searchRangeWidth&&bvCand[x0][y0][merge_idx][0]+searchRangeWidthIBCTM<searchRangeWidth&&bvCand[x0][y0][merge_idx][1]-searchRangeHeightIBCTM> -searchRangeHeight&&bvCand[x0][y0][merge_idx][1]+searchRangeHeightIBCTM <searchRangeHeight)
[0510] Wherein, bvCand[x0][y0][merge_idx][0]-searchRangeWidthIBCTM represents the first extended horizontal boundary position of the third candidate reference block pointed to by the third candidate block vector, bvCand[x0][y0][merge_idx][1]-searchRangeHeightIBCTM represents the first extended vertical boundary position of the third candidate reference block pointed to by the third candidate block vector, -searchRangeWidth represents the left boundary of the horizontal range of the preset range, searchRangeWidth represents the left and right boundaries of the horizontal range of the preset range, -searchRangeHeight represents the lower boundary of the horizontal range of the preset range, and searchRangeHeight represents the upper boundary of the horizontal range of the preset range. searchRangeWidthIBCTM and searchRangeHeightIBCTM refer to the search range specified in the IBC™ merge mode (for example, both searchRangeWidthIBCTM and searchRangeHeightIBCTM are 8 (for example, [-8,8]).
[0511] Exemplarily, if (bvCand[x0][y0][merge_idx][0] - searchRangeWidthIBCTM > -searchRangeWidth && bvCand[x0][y0][merge_idx][0] + searchRangeWidthIBCTM < searchRangeWidth && bvCand[x0][y0][merge_idx][1] - searchRangeHeightIBCTM > -searchRangeHeight && bvCand[x0][y0][merge_idx][1] + searchRangeHeightIBCTM < searchRangeHeight) is true, then the third candidate reference block pointed to by the third candidate block vector belongs to the reference block within the first region range. Remove the third candidate block vector for the reference block within the first region range, and continue to remove other third candidate block vectors until the processing of K third candidate block vectors is completed or K filtered third candidate block vectors are obtained.
[0512] Exemplarily, if (bvCand[x0][y0][merge_idx][0] - searchRangeWidthIBCTM > -searchRangeWidth && bvCand[x0][y0][merge_idx][0] + searchRangeWidthIBCTM < searchRangeWidth && bvCand[x0][y0][merge_idx][1] - searchRangeHeightIBCTM > -searchRangeHeight && bvCand[x0][y0][merge_idx][1] + searchRangeHeightIBCTM < searchRangeHeight) is false, then the third candidate reference block pointed to by the third candidate block vector does not belong to the reference block within the first region range. Take the third candidate block vector as the filtered third candidate block vector.
[0513] In an embodiment of the present application, add at least one filtered third candidate block vector to the second block vector candidate list, thereby determining the second block vector candidate list.
[0514] In some embodiments of the present application, based on the second block vector candidate list, the predicted value of the current block may include:
[0515] Perform template matching refinement on the second block vector candidate list to determine the fourth block vector candidate list;
[0516] Decode the code stream and determine the candidate block vector index information;
[0517] Determine a first block vector according to the candidate block vector index information and the fourth block vector candidate list;
[0518] A reference block of the current block is determined according to the first block vector, and a prediction value of the current block is determined according to the reference block.
[0519] In the embodiment of the present application, template matching refinement is performed on each second block vector candidate in the second block vector candidate list, thereby determining a fourth block vector candidate list.
[0520] In the embodiment of the present application, the decoder determines, from the fourth block vector candidate list according to the candidate block vector index information, a first block vector corresponding to the candidate block vector index information.
[0521] In the embodiment of the present application, if the current block has a prediction residual, the decoder parses the bitstream to determine the prediction residual of the current block, and performs weighting or other processing on the prediction value of the current block and the prediction difference of the current block to obtain a reconstructed value of the current block. If the current block does not have a prediction residual, the decoder directly uses the prediction value of the current block as the reconstructed value of the current block.
[0522] It is understood that during the decoding process, when performing block decoding, in scenarios based on a prediction mode with intra-frame merging with block vector differences, the decoder can perform deduplication based on the N first candidate block vectors in the first block vector candidate list of the current block. Overlapping block vectors or block vectors within a first region, such as in an intra-frame template matching prediction mode, are removed from the N first candidate block vectors to determine a third block vector candidate list. A prediction value for the current block is determined based on the de-redundant block vectors in the third block vector candidate list. At least one first extended candidate block vector corresponding to each first candidate block vector in the third block vector candidate list is further removed to obtain a second block vector candidate list. A prediction value for the current block is determined based on the second block vector candidate list. Overlapping block vectors or block vectors within the first region, such as in an intra-frame template matching prediction mode, are removed from at least one first extended candidate block vector to further reduce block vector redundancy. Consequently, when decoding using the de-redundant block vectors or block vectors, header information expression efficiency can be improved, thereby improving video decoding efficiency.
[0523] In some embodiments of the present application, in the process of determining the first block vector candidate list of the current block, a duplicate checking operation is performed on the candidate block vectors in the first block vector candidate list.
[0524] In an embodiment of the present application, for the IBC mode, in the process of determining the first block vector candidate list of the current block, a duplicate checking operation is performed on the candidate block vectors in the first block vector candidate list.
[0525] In some embodiments of the present application, performing a duplicate checking operation on the candidate block vectors in the first block vector candidate list may include:
[0526] Determine the current candidate block vector to be added;
[0527] Comparing the current candidate block vector with the candidate block vectors in the first block vector candidate list;
[0528] If the current candidate block vector does not coincide with a candidate block vector in the first block vector candidate list, then the current candidate block vector is added to the first block vector candidate list;
[0529] If the current candidate block vector coincides with a candidate block vector in the first block vector candidate list, the current candidate block vector is skipped and the step of determining the next candidate block vector to be added is continued.
[0530] In an embodiment of the present application, for a current candidate block vector to be added to the first block vector candidate list, it is determined whether the current candidate block vector coincides with a candidate block vector in the first block vector candidate list. If the current candidate block vector coincides with a candidate block vector in the first block vector candidate list, the current candidate block vector is skipped, that is, the current candidate block vector is not added to the first block vector candidate list, and the step of determining the next candidate block vector to be added is continued. If the current candidate block vector coincides with a candidate block vector in the first block vector candidate list, the current candidate block vector is added to the first block vector candidate list.
[0531] In some embodiments of the present application, the current candidate block vector coincides with a candidate block vector in the first block vector candidate list, which may include:
[0532] The current candidate block vector is exactly the same as the candidate block vector in the first block vector candidate list, or,
[0533] A difference between the current candidate block vector and a candidate block vector in the first block vector candidate list satisfies a first preset condition.
[0534] In an embodiment of the present application, two dimensions can be considered to determine whether the current candidate block vector is completely identical to the candidate block vectors in the first block vector candidate list. The first dimension is: the current candidate block vector is completely identical to the candidate block vector, for example, the position information of the current candidate block vector and the candidate block vector is completely identical. The second dimension is: the difference between the current candidate block vector and the candidate block vector satisfies a first preset condition, for example, the difference in position information between the current candidate block vector and the candidate block vector can be within a certain range, wherein the first preset condition here is pre-set, that is, the first preset condition is stipulated or agreed upon by both the encoder and the decoder.
[0535] Exemplarily, the current candidate block vector and the candidate block vector are considered to be overlapping when they are exactly the same (both horizontal and vertical components are the same); or it may be that the difference between the current candidate block vector and the candidate block vector is too small after comparison with a certain threshold and they are considered to be overlapping (for example, the absolute values of the differences in the horizontal and vertical components are both less than or equal to 2, or the absolute value of the horizontal component difference is less than 2, and the absolute value of the vertical component difference is less than 3).
[0536] It can be understood that deduplication can be performed when making the first block vector candidate list, removing some overlapping BVs, which can reduce the redundancy of block vectors during decoding, and then when using the de-redundant block vector or block vector for encoding and decoding, the expression efficiency of header information can be improved, thereby improving the video decoding efficiency.
[0537] An encoding method is provided in an embodiment of the present application. Referring to FIG22 , a flowchart of an optional decoding method provided in an embodiment of the present application is shown. As shown in FIG22 , the method may include:
[0538] S501, determining a prediction mode for a current block;
[0539] In an embodiment of the present application, in the scenario of encoding the current block, the encoder can traverse different block prediction modes to perform template matching estimation and determine the prediction mode with the minimum template cost, which is the prediction mode of the current block.
[0540] It should be noted that in the embodiment of the present application, the prediction mode may include: intra-frame block prediction mode and inter-frame block prediction mode, among which the intra-frame block vector prediction mode may also include IBC mode and IntraTMP mode, etc., which is not limited in the embodiment of the present application.
[0541] S502: If the prediction mode indicates that the current block uses a prediction mode based on intra-frame block merging, determine a first block vector candidate list for the current block;
[0542] S503: Determine a first region range, and determine a second block vector candidate list based on the first block vector candidate list and the first region range;
[0543] In an embodiment of the present application, the encoder removes the block vectors belonging to the first region based on the first block vector candidate list to obtain a second block vector candidate list.
[0544] In this embodiment of the present application, the first area range includes one of the following:
[0545] The size of the current encoding unit;
[0546] The size of the reconstructed area in the current frame;
[0547] The size of the image border of the current block;
[0548] Search range for Intra TMP mode.
[0549] For example, in an embodiment of the present application, based on the first block vector candidate list, the block vectors belonging to the search range of the Intra TMP mode are removed to obtain the second block vector candidate list. This can remove the BV information redundantly represented between different modes, improve the expression efficiency of the header information, and thus improve the overall video coding efficiency.
[0550] S504: Determine a prediction value of the current block based on the second block vector candidate list.
[0551] In this embodiment of the present application, the encoder determines a first block vector based on the second block vector candidate list, determines a reference block for the current block based on the first block vector, and determines a prediction value for the current block based on the reference block. The candidate block vector index information corresponding to the first block vector is written into the bitstream.
[0552] In some embodiments of the present application, the encoder determines an original value of the current block; and determines a prediction difference value of the current block according to the original value of the current block and a prediction value of the current block.
[0553] In the embodiment of the present application, the encoder writes the prediction difference into the bitstream.
[0554] It should be noted that the specific implementation of S502-504 is consistent with the implementation principle of the decoder and will not be repeated here.
[0555] It can be understood that during the encoding process, during the block encoding process, for the scenario of intra-frame merging mode prediction, the encoder can remove the overlapping block vectors or block vectors within the first area, such as in the intra-frame template matching prediction mode, based on the first block vector candidate list of the current block, and determine the second block vector candidate list, and determine the prediction value of the current block based on the de-redundant block vector in the second block vector candidate list. In this way, since the redundancy of the block vector is reduced, the expression efficiency of the header information can be improved when the de-redundant block vector or block vector is used for encoding, thereby improving the video coding efficiency.
[0556] In some embodiments of the present application, an encoding method is further provided, as shown in FIG23 , including:
[0557] S501, determining a prediction mode for a current block;
[0558] S502: If the prediction mode indicates that the current block uses a prediction mode based on intra-frame block merging, determine a first block vector candidate list for the current block;
[0559] S5031. Determine the scope of the first area;
[0560] S5032: If it is determined that the current block uses a prediction mode based on merging with block vector differences within the frame, determine, based on the first block vector candidate list, at least one first extended candidate block vector corresponding to each first candidate block vector;
[0561] In some embodiments of the present application, third syntax identification information is determined and written into the bitstream, where the third syntax identification information indicates whether the current block uses a prediction mode based on intra-frame merging with block vector differences.
[0562] In some embodiments of the present application, the implementation of the encoder determining at least one first extended candidate block vector corresponding to each first candidate block vector based on the first block vector candidate list includes: determining N first candidate block vectors with minimum template cost from the first block vector candidate list; N is an integer greater than or equal to 1; based on the distance set and the block vector prediction direction, refining each first candidate block vector in the N first candidate block vectors to determine at least one first extended candidate block vector corresponding to each first candidate block vector.
[0563] In some embodiments of the present application, the encoder removes, for each first candidate block vector, block vectors corresponding to reference blocks belonging to the first region from each first extended candidate block vector in sequence, to obtain at least one filtered first extended candidate block vector as follows:
[0564] If the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector falls within the horizontal range of a preset range, and the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector falls within the vertical range of a preset range, then the first candidate reference block pointed to by any first extended candidate block vector belongs to the reference block of the first area range; the any first extended candidate block vector is removed, and the removal process is continued on other first extended candidate block vectors corresponding to each first candidate block vector to obtain at least one first extended candidate block vector after screening.
[0565] In some embodiments of the present application, if the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the horizontal range of a preset range, or the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the vertical range of a preset range, then the first candidate reference block pointed to by any first extended candidate block vector does not fall within the reference block of the first area range; and the any first extended candidate block vector is a filtered first extended candidate block vector corresponding to each candidate block vector.
[0566] It should be noted that the encoder determines the first extended candidate block vector of the N first candidate block vectors after reordering based on the first block vector candidate list to determine whether BV removal is required.
[0567] S5033: Remove the block vectors belonging to the first region from at least one first extended candidate block vector corresponding to each first candidate block vector to obtain a second block vector candidate list;
[0568] In some embodiments of the present application, the encoder removes the block vectors belonging to the first area range from at least one first extended candidate block vector corresponding to each first candidate block vector to obtain the second block vector candidate list. The implementation may include: for each first candidate block vector, removing the block vectors corresponding to the reference block belonging to the first area range from each of its first extended candidate block vectors in turn to obtain at least one filtered first extended candidate block vector; and determining the second block vector candidate list based on the at least one filtered first extended candidate block vector corresponding to each first candidate block vector.
[0569] In some embodiments of the present application, the encoder determines, for each first candidate block vector, M first extended candidate block vectors corresponding to the minimum template cost from at least one first extended candidate block vector after screening; where M is an integer greater than or equal to 1; and constructs a second block vector candidate list based on the M first extended candidate block vectors of each first candidate block vector.
[0570] In some embodiments of the present application, for each first candidate block vector, when the number of at least one first extended candidate block vector after screening is less than M, other first candidate vectors are determined from the first block vector candidate list; the other first candidate vectors are different from the N first candidate vectors; and M first extended candidate block vectors are determined based on the other first candidate vectors and the at least one first extended candidate block vector after screening.
[0571] It can be understood that while removing the redundant BV information, the encoder determines other first candidate vectors from the first block vector candidate list; the other first candidate vectors are different from the N first candidate vectors, that is, optional options that were originally unexpressible are introduced into the reordered list, thereby improving the representation range of the BV candidate list in the corresponding mode, increasing the diversity of options, and also improving video encoding efficiency.
[0572] For example, in the MBVD mode, when constructing the MBVD candidate list, the BVs within the IntraTMP search range are deleted. The specific method is as follows:
[0573] In the IBC MBVD intra-frame block copy mode, similar to the MMVD in VVC's inter-frame technology, a candidate in the IBC merge list is used as the starting point, and a candidate is selected from the candidate point set corresponding to the predefined distance and direction set, and its corresponding block vector is used as the final BV.
[0574] For example, in IBC MBVD, the distance set is defined as {1-pel, 2-pel, 4-pel, 8-pel, 12pel, 16pel, 24pel, 32pel, 40pel, 48pel, 56pel, 64pel, 72pel, 80pel, 88pel, 96pel, 104pel, 112pel, 120pel, 128pel}, and the BVD directions are the positive and negative horizontal directions and the positive and negative vertical directions.
[0575] The base candidate is selected from the first five candidates in the re-ranked IBC merge list, and the refinement positions of all possible MBVDs of each base candidate (i.e., 20×4 candidates) are re-ranked based on the SAD cost between the template and its reference at each refinement position. In this process, the BV of the 20×4 candidates must first be judged whether it is within the IntraTMP search range.
[0576] Finally, the first N refined positions with the smallest template SAD are retained, where N can be preset artificially, for example, to 8. If there are less than N, the BV in the IBC candidate BV list can be used to supplement it for MBVD index encoding. If N is 0, IBC-MBVD is not performed. The candidates for IBC-MBVD do not inherit the flip type from the adjacent blocks encoded by RR-IBC. The MBVD index (candidate block vector index information) is binarized using a Rice code with a parameter equal to 1, and the embodiment of the present application does not limit its encoding method.
[0577] S504: Determine a prediction value of the current block based on the second block vector candidate list.
[0578] It should be noted that S502, S5031-S503, and S504 in the embodiment of the present application are consistent with the implementation principle of the decoder in determining that the current block uses a prediction mode based on merging with block vector differences within the frame, and will not be repeated here.
[0579] The difference is that the encoder constructs and removes the first extended candidate block vector for each of the N first candidate block vectors, while the decoder constructs and removes the first extended candidate block vector for the first candidate block vector corresponding to the candidate block vector index information among the N first candidate block vectors. However, the principles of constructing the first extended candidate block vector for each first candidate block vector and removing the block vectors belonging to the first region are the same.
[0580] It is understood that during the encoding process, during block encoding, in a scenario based on a prediction mode with intra-frame merging with block vector differences, the encoder can deduplicate at least one extended candidate block vector corresponding to N first candidate block vectors in the first block vector candidate list of the current block. Overlapping block vectors or block vectors of the at least one extended candidate block vector within a first region, such as in an intra-frame template matching prediction mode, are removed to determine a second block vector candidate list. A prediction value of the current block is determined based on the de-redundant block vectors in the second block vector candidate list. This reduces redundancy in block vectors, thereby improving header information expression efficiency when encoding using the de-redundant block vectors or block vectors, thereby improving video coding efficiency.
[0581] In some embodiments of the present application, an encoding method is further provided, as shown in FIG24 , including:
[0582] S501, determining a prediction mode for a current block;
[0583] S502: If the prediction mode indicates that the current block uses a prediction mode based on intra-frame block merging, determine a first block vector candidate list for the current block;
[0584] S505. Determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1;
[0585] S506: Remove the block vectors belonging to the first region from the N first candidate block vectors to obtain a third block vector candidate list;
[0586] In some embodiments of the present application, the encoder removes the block vector corresponding to the reference block belonging to the first area range from each first candidate block vector of N first candidate block vectors in turn to obtain at least one filtered first candidate block vector; and determines a third block vector candidate list based on the at least one filtered first candidate block vector.
[0587] In some embodiments of the present application, the encoder sequentially offsets each of the N first candidate block vectors to determine at least one second candidate block vector corresponding to each first candidate block vector; if the maximum horizontal boundary position and the minimum horizontal boundary position in at least one second candidate reference block pointed to by the at least one second candidate block vector are within a preset horizontal range, and the maximum vertical boundary position and the minimum vertical boundary position in at least one second candidate reference block are within a preset vertical range, then the first candidate block vector corresponding to the at least one second candidate block vector belongs to a block vector within the first area range; the first candidate block vector is removed, and the removal process is continued on other first candidate block vectors to obtain at least one filtered first candidate block vector.
[0588] In some embodiments of the present application, if the maximum horizontal boundary position or the minimum horizontal boundary position in at least one second candidate reference block pointed to by at least one second candidate block vector does not fall within the horizontal range of a preset range, or the maximum vertical boundary position or the minimum vertical boundary position in at least one second candidate reference block does not fall within the vertical range of a preset range, the encoder determines that the candidate block vector corresponding to the at least one second candidate block vector does not fall within the block vector of the first area range; the first candidate block vector corresponding to the at least one second candidate block vector is used as a screened first candidate block vector; and continues to perform removal processing on other first candidate block vectors until the processing of N first candidate block vectors is completed, or N screened first candidate block vectors are obtained.
[0589] It should be noted that in MBVD mode, when judging the options in the merge list, merge candidates whose all options are within the IntraTMP search range after offset are ignored. The specific method is as follows:
[0590] In IBC MBVD mode, similar to the MMVD in VVC's inter-frame technology, a candidate in the IBC merge list is used as the starting point, and a candidate is selected from the candidate point set corresponding to the predefined distance and direction set, and its corresponding block vector is used as the final BV.
[0591] For example, in IBC MBVD, the distance set is defined as {1-pel, 2-pel, 4-pel, 8-pel, 12pel, 16pel, 24pel, 32pel, 40pel, 48pel, 56pel, 64pel, 72pel, 80pel, 88pel, 96pel, 104pel, 112pel, 120pel, 128pel}, and the BVD directions are the positive and negative horizontal directions and the positive and negative vertical directions.
[0592] The basic candidates are selected from the first N candidates (N first candidate block vectors) in the reordered IBC merge list. First, the first N candidates are offset to determine whether the options are all within the IntraTMP search range. If they are all within the IntraTMP search range, the candidate (one of the N) is removed from the basic candidates. At this time, the merge_idx syntax element no longer corresponds to the sequence number in the original IBC merge list, but the sequence number of the reordered list after removing the options whose offset options are all within the IntraTMP search range. At this time, a new candidate list MBVD_merge_cand_list dedicated to MBVD needs to be created, and the first N items in the original IBC merge candidate list IBC merge_cand_list are checked one by one. If the condition that the offset options are all within the IntraTMP search range is not met, the candidate is placed in the MBVD_merge_cand_list, otherwise it is not placed, and this continues until there are no candidates or the predefined number of candidates (N) is met.
[0593] S507 : Determine at least one first extended candidate block vector corresponding to each first candidate block vector based on the third block vector candidate list.
[0594] In some embodiments of the present application, the encoder refines each first candidate block vector in the third block vector candidate list based on the distance set and the block vector prediction direction, and determines at least one first extended candidate block vector corresponding to each first candidate block vector.
[0595] S508. Determine a first block vector from at least one first extended candidate block vector;
[0596] S509 : Determine a reference block for the current block according to the first block vector, and determine a prediction value for the current block according to the reference block.
[0597] In some embodiments of the present application, the encoder writes the candidate block vector index information and the extended candidate block vector index information corresponding to the first block vector into the bitstream.
[0598] It is understandable that during the encoding process, during the block encoding process, the encoder can deduplicate the N first candidate block vectors in the first block vector candidate list of the current block in a scenario based on a prediction mode with a merge of block vector differences within a frame. Within a first region of the N first candidate block vectors, for example, in an intra-frame template matching prediction mode, overlapping block vectors or block vectors are removed to determine a third block vector candidate list, and a prediction value of the current block is determined based on the de-redundant block vectors in the third block vector candidate list. This reduces the redundancy of the block vectors, and thus, when the de-redundant block vectors or block vectors are used for encoding, the efficiency of expressing header information can be improved, thereby improving video coding efficiency.
[0599] S5033: Remove the block vectors belonging to the first region from at least one first extended candidate block vector corresponding to each first candidate block vector to obtain a second block vector candidate list;
[0600] S504: Determine a prediction value of the current block based on the second block vector candidate list.
[0601] It is understood that during the encoding process, during block encoding, in scenarios based on intra-frame prediction modes with block vector difference merging, the encoder can perform deduplication on N first candidate block vectors in the first block vector candidate list for the current block. Overlapping block vectors or block vectors within a first region, such as in an intra-frame template matching prediction mode, are removed from the N first candidate block vectors to determine a third block vector candidate list. A prediction value for the current block is determined based on the de-redundant block vectors in the third block vector candidate list. At least one first extended candidate block vector corresponding to each first candidate block vector in the third block vector candidate list is further removed to obtain a second block vector candidate list. A prediction value for the current block is determined based on the second block vector candidate list. Overlapping block vectors or block vectors within the first region, such as in an intra-frame template matching prediction mode, are removed from at least one first extended candidate block vector to further reduce block vector redundancy. Consequently, when encoding using the de-redundant block vectors or block vectors, header information expression efficiency can be improved, thereby improving video coding efficiency.
[0602] It should be noted that S502-S509, as well as S502-S507, S5033 and S504 in the embodiment of the present application are consistent with the implementation principle of the decoder in determining that the current block uses a prediction mode based on merging with block vector differences within the frame, and will not be repeated here.
[0603] The difference is that in S5033 and S504, the encoder constructs and removes the first extended candidate block vector for each of the N first candidate block vectors, while the decoder constructs and removes the first extended candidate block vector for the first candidate block vector corresponding to the candidate block vector index information among the N first candidate block vectors. However, the principles for constructing the first extended candidate block vector and removing the block vectors belonging to the first region for each first candidate block vector are the same.
[0604] It should be noted that, in the embodiment of the present application, after S507, S508-S509 and S5033-S504 are two optional implementations, and the embodiment of the present application does not limit them.
[0605] In some embodiments of the present application, an encoding method is further provided, as shown in FIG25 , including:
[0606] S501, determining a prediction mode for a current block;
[0607] S510: If it is determined that the current block uses a prediction mode based on template matching merging, K third candidate block vectors with minimum template costs are determined from the first block vector candidate list; K is an integer greater than or equal to 1;
[0608] S511, removing the block vectors belonging to the first region from the K third candidate block vectors to obtain a second block vector candidate list;
[0609] It should be noted that, in the embodiment of the present application, the encoder removes the block vectors belonging to the first region from the K third candidate block vectors to obtain the second block vector candidate list in the following two ways:
[0610] Method 1
[0611] If, among the K third candidate block vectors, the horizontal boundary position of the third candidate reference block pointed to by any third candidate block vector falls within the horizontal range of the preset range, and the vertical boundary position of the third candidate reference block pointed to by any third candidate block vector falls within the vertical range of the preset range, then the third candidate reference block pointed to by any third candidate block vector belongs to the reference block of the first area range;
[0612] Removing any third candidate block vector of the reference block belonging to the first area, and continuing to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained;
[0613] A second block vector candidate list is determined based on the filtered at least one third candidate block vector.
[0614] Method 2
[0615] If, among the K third candidate block vectors, the first extended horizontal boundary position of the third candidate reference block pointed to by any third candidate block vector falls within the horizontal range of the preset range, and the first extended vertical boundary position of the third candidate reference block pointed to by any third candidate block vector falls within the vertical range of the preset range, then the third candidate reference block pointed to by any third candidate block vector belongs to the reference block of the first area range;
[0616] The first extended horizontal boundary position is the horizontal boundary position of the third candidate reference block obtained by extending the first horizontal search range by the horizontal boundary of the third candidate reference block; the first extended vertical boundary position is the vertical boundary position of the third candidate reference block obtained by extending the first vertical search range by the vertical boundary of the third candidate reference block; the first horizontal search range and the first vertical search range are search ranges specified in the IBC™ merge mode;
[0617] Removing any third candidate block vector of the reference block belonging to the first area, and continuing to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained;
[0618] A second block vector candidate list is determined based on the filtered at least one third candidate block vector.
[0619] S504: Determine a prediction value of the current block based on the second block vector candidate list.
[0620] It should be noted that S502-S509, as well as S502-S507, S5033 and S504 in the embodiment of the present application are consistent with the implementation principle of the decoder in determining the prediction mode of the current block using template matching-based merging, and will not be repeated here.
[0621] It is understood that during the encoding process, when performing block encoding, in the scenario of a template matching-based merge prediction mode, the encoder can deduplicate the K third candidate block vectors in the first block vector candidate list of the current block. Within a first region, for example, in an intra-frame template matching prediction mode, overlapping block vectors or block vectors are removed from the K third candidate block vectors to determine a second block vector candidate list. Local refinement is then performed based on the de-redundant block vectors in the second block vector candidate list to determine the prediction value of the current block. This reduces block vector redundancy, and when encoding using the de-redundant block vectors or block vectors, the efficiency of header information expression can be improved, thereby improving video coding efficiency.
[0622] It should be noted that, in the process of determining the first block vector candidate list of the current block in the present application, the encoding method further includes: in the process of determining the first block vector candidate list of the current block, performing a duplicate checking operation on the candidate block vectors in the first block vector candidate list.
[0623] In an embodiment of the present application, the encoder performs a duplicate checking operation on the candidate block vectors in the first block vector candidate list, including: determining a current candidate block vector to be added; comparing the current candidate block vector with the candidate block vectors in the first block vector candidate list; if the current candidate block vector does not overlap with the candidate block vectors in the first block vector candidate list, adding the current candidate block vector to the first block vector candidate list; if the current candidate block vector overlaps with the candidate block vector in the first block vector candidate list, skipping the current candidate block vector and continuing to execute the step of determining the next candidate block vector to be added.
[0624] In some embodiments of the present application, the current candidate block vector is completely identical to the candidate block vector in the first block vector candidate list.
[0625] In some embodiments of the present application, a difference between the current candidate block vector and a candidate block vector in the first block vector candidate list satisfies a first preset condition.
[0626] It can be understood that deduplication can be performed when making the first block vector candidate list, removing some overlapping BVs, which can reduce the redundancy of block vectors during encoding, and then when using the de-redundant block vector or block vector for encoding, the expression efficiency of header information can be improved, thereby improving video encoding efficiency.
[0627] In some embodiments of the present application, if the current block uses a prediction mode based on intra-frame block merging, the first syntax identification information is determined, and the second syntax identification information is determined and written into the bitstream; the first syntax identification information indicates whether the current block uses a prediction mode of intra-frame block copy; the second syntax identification information indicates whether the current block uses a prediction mode of intra-frame block merging.
[0628] The above encoding and decoding methods are explained in detail below in some specific embodiments:
[0629] (1) Example 1:
[0630] In this embodiment of the present application, the current block is mainly in IBC MBVD mode (block vector merge mode with BVD). IBC MBVD mode is similar to MMVD in VVC's inter-frame technology. Starting from a candidate in the IBC merge list, a candidate is selected from a set of candidate points corresponding to a predefined distance and direction set, and its corresponding block vector is used as the final BV. That is, the current candidate is used as the center point for offset to obtain a candidate point set, and a candidate is further selected from this candidate point set as the final BV.
[0631] Specifically, when the current block is in IBC MBVD mode, in the process of constructing the MBVD candidate list, the BV within the IntraTMP frame template matching prediction search range is deleted. The specific method is as follows:
[0632] In the intra-frame block copy mode where the current block is IBC MBVD, similar to the MMVD in the inter-frame technology of VVC, a candidate in the IBC merge list (the first block vector candidate list) is used as the starting point, and a candidate is selected from the candidate point set corresponding to the predefined distance and direction set, and its corresponding block vector is used as the final BV.
[0633] According to the predefined distance set and the predefined direction set, a plurality of refined positions of the basic candidate items can be determined, thereby achieving refinement of the basic candidate items.
[0634] For example, in the IBC MBVD mode, the distance set is defined as {1-pel, 2-pel, 4-pel, 8-pel, 12pel, 16pel, 24pel, 32pel, 40pel, 48pel, 56pel, 64pel, 72pel, 80pel, 88pel, 96pel, 104pel, 112pel, 120pel, 128pel}, and the BVD directions are two positive and negative horizontal directions and two positive and negative vertical directions. The direction set includes positive and negative horizontal directions and positive and negative vertical directions, for a total of four directions. Using 20 distances and four directions, 20×4 refinement positions (here, the refinement positions of the base candidate) can be determined.
[0635] First, it is necessary to determine the base candidate among the candidates in the sorted IBC merge list corresponding to the IBC MBVD mode. Among them, the number of base candidates is preset, that is, the decoder and the encoder can pre-specify or agree on the number of base candidates through a protocol.
[0636] It should be noted that the sorted IBC merge list is a list re-sorted after calculating the template matching cost for each candidate. That is to say, each candidate in the sorted IBC merge list is sorted according to the cost value of template matching. Each candidate can be sorted in descending order or ascending order according to the cost value of template matching. This application does not make any restrictions on this.
[0637] Exemplarily, when each candidate can be sorted in ascending order according to the cost value of template matching, the cost value of template matching of the first candidate in the sorted IBC merge list is the smallest.
[0638] The base candidate (the first candidate block vector) is selected from the first five candidates in the re-sorted IBC merge list (the first block vector candidate list), and based on the SAD cost between the template and the reference of the refined position of each of its base candidates, the refined positions of all possible MBVDs of each base candidate (that is, 20×4 candidates) are re-sorted. In this process, it is necessary to first determine whether the BV of the 20×4 candidates of each base candidate is within the IntraTMP search range.
[0639] Exemplarily, if (bvCand[x0][y0][disIdx][dirIdx][0] > -searchRangeWidth && bvCand[x0][y0][disIdx][dirIdx][0] < searchRangeWidth && bvCand[x0][y0][disIdx][dirIdx][1] > -searchRangeHeight && bvCand[x0][y0][disIdx][dirIdx][1] < searchRangeHeight) is true, it indicates that the BV is within the IntraTMP range and does not participate in the re-sorting. Here, searchRangeWidth and searchRangeHeight refer to the preset search range of the template in the IntraTMP mode.
[0640] bvCand represents the location of the candidate, searchRangeWidth and searchRangeHeight represent the preset search range. The preset search range includes at least one of the following: the size of the CTU, the range of the reconstructed area, the current image area where the current block is located, etc. This application does not impose any restrictions on this.
[0641] After determining the IBC MBVD list for IBC MBVD mode, the top N refinement positions with the smallest template SAD are retained. N can be preset, for example, to 8. If there are fewer than N, BVs from the IBC candidate BV list are used to supplement them for MBVD index encoding. If N is 0, IBC-MBVD is not performed. IBC-MBVD candidates do not inherit the flip type from neighboring blocks encoded in RR-IBC. The MBVD index is binarized using a Rice code with parameter equal to 1.
[0642] In this embodiment, by removing redundant BV information between different modes, the efficiency of header information expression can be improved, thereby improving overall video coding efficiency. While removing redundant BV information, it also introduces options that were not originally represented in the reordered list, expanding the range of BV candidate lists in the corresponding mode, increasing the diversity of options, and further improving video coding efficiency.
[0643] In an embodiment of the present application, in the IBC mode, the options within the BV representation range that can be covered by the IntraTMP mode are removed from the candidate list of modes selected according to the template cost sorting, thereby improving the expression efficiency of the header information, increasing the diversity of options, and improving the video encoding efficiency.
[0644] (II) Example 2:
[0645] Based on Example 1, when the current block is in IBC MBVD mode, the options in the merge list are judged, and the merge candidates whose all options are within the IntraTMP search range after the offset are ignored. The specific method is as follows:
[0646] In IBC MBVD mode, similar to the MMVD in VVC's inter-frame technology, a candidate in the IBC merge list is used as the starting point, and a candidate is selected from the candidate point set corresponding to the predefined distance and direction set, and its corresponding block vector is used as the final BV.
[0647] For example, in the IBC MBVD mode, the distance set is defined as {1-pel, 2-pel, 4-pel, 8-pel, 12pel, 16pel, 24pel, 32pel, 40pel, 48pel, 56pel, 64pel, 72pel, 80pel, 88pel, 96pel, 104pel, 112pel, 120pel, 128pel}, and the BVD directions are positive and negative horizontal directions and positive and negative vertical directions.
[0648] The basic candidate items are selected from the first N candidate items in the reordered IBC merge list. First, the first N candidate items are judged whether the offset options (such as the candidate points after the offset of 128pel) are all within the IntraTMP search range. If they are all within the IntraTMP search range, the candidate items are removed from the basic candidate items. At this time, the merge_idx syntax element no longer corresponds to the sequence number in the original IBC merge list, but the sequence number of the reordered list after removing the options whose offset options are all within the IntraTMP search range. At this time, it is necessary to create a new candidate list MBVD_merge_cand_list dedicated to MBVD, and check the first N items in the original IBC merge candidate list IBC merge_cand_list one by one. If the condition that the offset options are all within the IntraTMP search range is not met, the candidate item is placed in the MBVD_merge_cand_list, otherwise it is not placed, and this continues until there are no candidates or the predefined number of candidates (such as 5) is met.
[0649] Examples of conditions for determining that all offset options are within the IntraTMP search range:
[0650] If (bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx][dirHorIdx_left][0] > -searchRangeWidth && bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx][dirHorIdx_right][0] < searchRangeWidth && bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx_up][dirHorIdx][1] > -searchRangeHeight && bvCand[x0][y0][merge_idx][disIdx_max][dirVerIdx_down][dirHorIdx][1] < searchRangeHeight) is true.
[0651] After determining the IBC MBVD list for the IBC MBVD mode, the basic candidates for subsequent operations are selected from the MBVD_merge_cand_list. At the same time, the meaning of the syntax element merge_idx also becomes the sequence number in the MBVD_merge_cand_list. Here, searchRangeWidth and searchRangeHeight refer to the search range using the TM refinement option in the TM merge mode, for example, set to the constant 8.
[0652] Based on the SAD cost between the template and the reference at each refined position, the refined positions of all possible MBVDs for each basic candidate (i.e., 20×4 candidates) are re - sorted. During this process, the BV of the 20×4 candidates must first be judged whether it is within the IntraTMP search range, for example:
[0653] If (bvCand[x0][y0][merge_idx][disIdx][dirVerIdx][dirHorIdx][0] > -searchRangeWidth && bvCand[x0][y0][merge_idx][disIdx][dirVerIdx][dirHorIdx][0] < searchRangeWidth && bvCand[x0][y0][merge_idx][disIdx][dirVerIdx][dirHorIdx][1] > -searchRangeHeight && bvCand[x0][y0][merge_idx][disIdx][dirVerIdx][dirHorIdx][1] < searchRangeHeight) is true, then this BV is within the IntraTMP range and does not participate in reordering. Here, searchRangeWidth and searchRangeHeight refer to the preset search range of the template in the IntraTMP mode.
[0654] Finally, retain the top N refined positions with the smallest template SAD. N can be preset manually, for example, as 8. If there are less than N, BVs in the IBC candidate BV list can be used for supplementation for MBVD index coding. The candidates of IBC-MBVD do not inherit the flip type from the adjacent blocks of RR-IBC coding. The MBVD index is binarized by the Rice code with a parameter equal to 1.
[0655] (III). Embodiment 3:
[0656] In the case where the current block is in the IBC TM merge mode, based on the constructed merge list, ignore the BVs within the IntraTMP search range in the list, that is, establish a candidate list specifically for the IBC TM merge mode, in which some BVs within the IntraTMP search range are excluded. A specific implementation method is exemplified as follows:
[0657] In the IBC TM merge mode, after obtaining bvL according to the above information, TM can also be used to locally refine the BV. The specific operation is to search within a small range centered on the obtained bvL, and select the optimal BV within this range as the final BV with the smallest template matching cost as the criterion.
[0658] Perform TM refinement on the candidate list of the IBC merge mode. A specific implementation process is as follows:
[0659] Construct a candidate list IBC_TM_merge_cand_list dedicated to the IBC TM merge mode. Check each of the first N items in the original IBC merge candidate list IBC merge_cand_list one by one. If it does not meet the condition of being within the IntraTMP search range, put this candidate into IBC_TM_merge_cand_list; otherwise, do not put it in, until there are no more candidates or the predefined number of candidates (such as 6) is reached.
[0660] An example of the condition for judging that the offset options are all within the IntraTMP search range:
[0661] If (bvCand[x0][y0][merge_idx][0] > -searchRangeWidth && bvCand[x0][y0][merge_idx][0] < searchRangeWidth && bvCand[x0][y0][merge_idx][1] > -searchRangeHeight && bvCand[x0][y0][merge_idx][1] < searchRangeHeight) is true. Here, searchRangeWidth and searchRangeHeight refer to the predefined search range of the template in the IntraTMP mode.
[0662] After determining the IBC TM merge list in the IBC TM merge mode, the basic candidates for subsequent operations are selected from IBC_TM_merge_cand_list. At the same time, the meaning of the syntax element merge_idx also becomes the serial number in IBC_TM_merge_cand_list.
[0663] It should be noted that when constructing candidates, the default flip type is non - flip.
[0664] In the IBC TM merge mode, the transport syntax element specifies whether to perform TM refinement with integer - pixel accuracy. The refined motion vectors and the positions of the templates used in each refinement step must comply with the constraints of the reference region.
[0665] A search is performed near the candidate's pointed location, using the SAD between the templates of the reference block and the current block to determine the optimal location. This includes, but is not limited to, the following search methods: Within the search range [-8, 8], a whole-pixel diamond search is first performed, searching eight points near the center, as shown in the upper left figure below. The maximum number of searches is 375. After the initial optimal location is determined, the search continues, searching five points at even-numbered locations and three points at odd-numbered locations. Referring to Figures 7a and 7b, using Figure 7b as an example, a counterclockwise circle is formed from the network dots, with indices 0 to 7. Specifically, if the red location is selected as the optimal location, the search continues for five or three points at the purple location. Following this diamond search, a whole-pixel cross search is performed only once. After the diamond search determines the current optimal location, four locations, one pixel below, to the right, above, and to the left of the current optimal location, are then checked to update the final optimal location, thereby refining the candidate list.
[0666] (four), embodiment 4:
[0667] Similar to Example 3, the only difference is that the candidate items whose search range is completely included in the IntraTMP search range are removed, so as to ensure that the search range completely includes the range of IntraTMP. The specific implementation method is as follows:
[0668] In the IBC TM merge mode, after obtaining the bvL based on the above information, the TM can also be used to locally refine the BV. The specific operation is to search within a small range centered on the obtained bvL and select the optimal BV within the range as the final BV based on the minimum template matching cost.
[0669] The TM refinement of the candidate list for the IBC merge mode is carried out. The specific implementation process is as follows:
[0670] A candidate list IBC_TM_merge_cand_list dedicated to the IBC TM merge mode is constructed. The first N items in the original IBC merge candidate list IBC merge_cand_list are checked one by one. If the candidate does not meet the condition of being within the IntraTMP search range, the candidate is added to the IBC_TM_merge_cand_list. Otherwise, it is not added until there are no more candidates or the predefined number of candidates (such as 6) is met.
[0671] Examples of conditions for determining that all offset options are within the IntraTMP search range:
[0672] If (bvCand[x0][y0][merge_idx][0] - searchRangeWidthIBCTM > -searchRangeWidth && bvCand[x0][y0][merge_idx][0] + searchRangeWidthIBCTM < searchRangeWidth && bvCand[x0][y0][merge_idx][1] - searchRangeHeightIBCTM > -searchRangeHeight && bvCand[x0][y0][merge_idx][1] + searchRangeHeightIBCTM < searchRangeHeight) is true. Here, searchRangeWidth and searchRangeHeight refer to the preset search range of the template in IntraTMP mode, and searchRangeWidthIBCTM and searchRangeHeightIBCTM refer to the search range specified in IBC TM merge mode (for example, both searchRangeWidthIBCTM and searchRangeHeightIBCTM take the value of 8, such as [-8, 8] as exemplified below).
[0673] After determining the IBC TM merge list in IBC TM merge mode, the basic candidates for subsequent operations are selected from IBC_TM_merge_cand_list. At the same time, the meaning of the syntax element merge_idx also becomes the sequence number in IBC_TM_merge_cand_list.
[0674] It should be noted that when constructing candidates, the flip type is defaulted to non - flip
[0675] In IBC TM merge mode, the transmission syntax element specifies whether to perform integer-pixel TM refinement. The location of the refined motion vector and the template used in each refinement step must adhere to the constraints of the reference region. A search is performed near the candidate's pointed location, using the SAD between the templates of the reference block and the current block to determine the optimal location. This includes, but is not limited to, the following search methods: A search range of [-8, 8] is initially performed, searching eight points near the center in an integer-pixel diamond pattern, as shown in the upper left corner of the figure below. The maximum number of searches is 375. After the optimal location is determined for the first time, the search continues, with five points searched at even-numbered locations and three at odd-numbered locations. Referring to Figures 7a and 7b, using Figure 7b as an example, a counterclockwise circle is formed from the network point, with indices 0 to 7. Specifically, if the red location is selected as the optimal location, the search continues with five or three points at the purple location. After the above diamond search process, a whole-pixel cross search is performed only once. That is, after the diamond search finds the current optimal position, four positions with a distance of one pixel below, to the right, above, and to the left of the current optimal position are checked to update the final optimal position, that is, to update the refined candidate list.
[0676] (V) Example 5:
[0677] The above prediction techniques related to IBC™ merge and MBVD can be used in combination.
[0678] That is, Example 1 and Example 2, Example 2 and Example 4, Example 2 and Example 3, and Example 1 and Example 4 can all be used in combination.
[0679] (Six), embodiment 6:
[0680] When building the TM merge candidate list or the MBVD candidate list, you can further check and remove duplicates:
[0681] Compare the new BV to be added to the list with the BV already in the list. If they coincide, skip the BV and continue to check the next candidate BV; if they do not coincide, add the BV to the candidate list.
[0682] The criteria for coincidence can be:
[0683] They are considered to be coincident when they are exactly the same (both horizontal and vertical components are the same);
[0684] Alternatively, the BV difference may be compared with a certain threshold and the difference may be too small to be considered as overlap (for example, the absolute values of the differences between the horizontal and vertical components are both less than or equal to 2, or the absolute value of the horizontal component difference is less than 2, and the absolute value of the vertical component difference is less than 3).
[0685] An embodiment of the present application provides a code stream, wherein the code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:
[0686] The prediction difference value of the current block, the candidate block vector index information, the extended candidate block vector index information, the first syntax identification information, the second syntax identification information and the third syntax identification information;
[0687] Among them, the first syntax identification information is used to indicate whether the current block uses the prediction mode of intra block copy, the second syntax identification information is used to indicate whether the current block uses the prediction mode of intra block merging, and the third syntax identification information is used to indicate whether the current block uses the MBVD mode.
[0688] In yet another embodiment of the present application, based on the same inventive concept as the above embodiment, see Figure 26, which shows a schematic diagram of the composition structure of a decoder 1 provided in an embodiment of the present application. As shown in Figure 26, the decoder 1 may include:
[0689] The decoding part 10 is configured to decode the code stream and determine the prediction mode of the current block;
[0690] A first determining part 11 is configured to determine a first block vector candidate list of the current block if the prediction mode of the current block indicates that the current block uses a prediction mode based on intra block merging to determine a prediction value of the current block; and determine a first area range;
[0691] A first removing part 12 is configured to determine a second block vector candidate list based on the first block vector candidate list and the first area range;
[0692] The first prediction part 13 is configured to determine a prediction value of the current block based on the second block vector candidate list.
[0693] In some embodiments of the present application, the first removing part 12 is further configured to remove block vectors belonging to the first area based on the first block vector candidate list to obtain a second block vector candidate list.
[0694] In some embodiments of the present application, the first removing part 12 is further configured to determine candidate block vector index information if it is determined that the current block uses a prediction mode based on merging with block vector differences within a frame;
[0695] determining, based on the first block vector candidate list, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information;
[0696] Block vectors belonging to the first area range are removed from the at least one first extended candidate block vector to obtain the second block vector candidate list.
[0697] In some embodiments of the present application, the first determining part 11 is further configured to determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1;
[0698] Determine a first candidate block vector from the N first candidate block vectors based on the candidate block vector index information;
[0699] Based on the distance set and the block vector prediction direction, the first candidate block vector is refined to determine at least one first extended candidate block vector corresponding to the first candidate block vector.
[0700] In some embodiments of the present application, the first removing part 12 is further configured to sequentially remove, from each first extended candidate block vector, block vectors corresponding to reference blocks belonging to the first region, to obtain at least one filtered first extended candidate block vector;
[0701] The second block vector candidate list is determined based on the filtered at least one first extended candidate block vector.
[0702] In some embodiments of the present application, the first removing portion 12 is further configured to: if a horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector falls within a horizontal range of a preset range, and a vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector falls within a vertical range of a preset range, then the first candidate reference block pointed to by any first extended candidate block vector belongs to the reference block of the first area range;
[0703] The any first extended candidate block vector is removed, and the removal process is continued on other first extended candidate block vectors to obtain at least one filtered first extended candidate block vector.
[0704] In some embodiments of the present application, the first removal part 12 is further configured to, if the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the horizontal range of a preset range, or the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the vertical range of a preset range, then the first candidate reference block pointed to by any first extended candidate block vector does not belong to the reference block within the first area range; and the any first extended candidate block vector is a filtered first extended candidate block vector corresponding to each candidate block vector.
[0705] In some embodiments of the present application, the first determining part 11 is further configured to determine M first extended candidate block vectors corresponding to the minimum template cost from the at least one first extended candidate block vector after screening; wherein M is an integer greater than or equal to 1;
[0706] The second block vector candidate list is constructed based on the M first extended candidate block vectors.
[0707] In some embodiments of the present application, the first determining part 11 is further configured to determine other candidate block vectors from the first block vector candidate list when the number of the at least one first extended candidate block vector after screening is less than M; the other candidate block vectors are different from the N candidate block vectors;
[0708] Based on the other candidate block vectors and the screened at least one first extended candidate block vector, M first extended candidate block vectors are determined.
[0709] In some embodiments of the present application, the decoding part 10 is further configured to decode the code stream and determine the extended candidate block vector index information;
[0710] The first determining part 11 is further configured to determine a first block vector according to the extended candidate block vector index information and the second block vector candidate list; and determine a reference block of the current block according to the first block vector;
[0711] The first prediction part 13 is further configured to determine a prediction value of the current block according to the reference block.
[0712] In some embodiments of the present application, the first determining part 11 is further configured to determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1;
[0713] removing the block vectors belonging to the first region from the N first candidate block vectors to obtain the third block vector candidate list;
[0714] Based on the third block vector candidate list, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information is determined.
[0715] In some embodiments of the present application, the first determining part 11 is further configured to determine a first candidate block vector from the third block vector candidate list based on the candidate block vector index information;
[0716] Based on the distance set and the block vector prediction direction, the first candidate block vector is refined to determine at least one first extended candidate block vector corresponding to the first candidate block vector.
[0717] In some embodiments of the present application, the decoding part 10 is further configured to decode the code stream and determine the extended candidate block vector index information;
[0718] determining a first block vector from the at least one first extended candidate block vector according to the extended candidate block vector index information;
[0719] A reference block of the current block is determined according to the first block vector, and a prediction value of the current block is determined according to the reference block.
[0720] In some embodiments of the present application, the first removing portion 12 is further configured to sequentially remove, from each of the N first candidate block vectors, a block vector corresponding to a reference block belonging to the first region, to obtain at least one filtered first candidate block vector;
[0721] The third block vector candidate list is determined based on the filtered at least one first candidate block vector.
[0722] In some embodiments of the present application, the first removing part 12 is further configured to sequentially offset each of the N first candidate block vectors to determine at least one second candidate block vector corresponding to each first candidate block vector;
[0723] If the maximum horizontal boundary position and the minimum horizontal boundary position in the at least one second candidate reference block pointed to by the at least one second candidate block vector are within the horizontal range of the preset range, and the maximum vertical boundary position and the minimum vertical boundary position in the at least one second candidate reference block are within the vertical range of the preset range, then the first candidate block vector corresponding to the at least one second candidate block vector belongs to the block vector of the first area range;
[0724] The first candidate block vector is removed, and the removal process is continued on other first candidate block vectors to obtain at least one first candidate block vector after screening.
[0725] In some embodiments of the present application, the first removing portion 12 is further configured to: if the maximum horizontal boundary position or the minimum horizontal boundary position in the at least one second candidate reference block pointed to by the at least one second candidate block vector does not fall within a preset horizontal range, or the maximum vertical boundary position or the minimum vertical boundary position in the at least one second candidate reference block does not fall within a preset vertical range, then the candidate block vector corresponding to the at least one second candidate block vector does not fall within the block vector of the first region; and the first candidate block vector corresponding to the at least one second candidate block vector is used as a filtered first candidate block vector;
[0726] The removal process is continued on other first candidate block vectors until the processing of N first candidate block vectors is completed, or N filtered first candidate block vectors are obtained.
[0727] In some embodiments of the present application, the first removing part 12 is further configured to, if it is determined that the current block uses the prediction mode of template matching-based merging, determine K third candidate block vectors with minimum template costs from the first block vector candidate list; K is an integer greater than or equal to 1;
[0728] Block vectors belonging to the first area are removed from the K third candidate block vectors to obtain the second block vector candidate list.
[0729] In some embodiments of the present application, the first removing portion 12 is further configured to: if, among the K third candidate block vectors, a horizontal boundary position of a third candidate reference block pointed to by any third candidate block vector falls within a preset horizontal range, and a vertical boundary position of a third candidate reference block pointed to by any third candidate block vector falls within a preset vertical range, then the third candidate reference block pointed to by any third candidate block vector belongs to the reference block within the first area range;
[0730] removing any third candidate block vector of the reference block belonging to the first area, and continuing to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained;
[0731] The second block vector candidate list is determined based on the filtered at least one third candidate block vector.
[0732] In some embodiments of the present application, the first removing portion 12 is further configured to: if, among the K third candidate block vectors, a first extended horizontal boundary position of a third candidate reference block pointed to by any third candidate block vector falls within a preset horizontal range, and a first extended vertical boundary position of a third candidate reference block pointed to by any third candidate block vector falls within a preset vertical range, then the third candidate reference block pointed to by any third candidate block vector belongs to a reference block within the first area range;
[0733] The first extended horizontal boundary position is the horizontal boundary position of the third candidate reference block obtained by extending the first horizontal search range by the horizontal boundary of the third candidate reference block; the first extended vertical boundary position is the vertical boundary position of the third candidate reference block obtained by extending the first vertical search range by the vertical boundary of the third candidate reference block; the first horizontal search range and the first vertical search range are search ranges specified in the prediction mode based on template matching merging;
[0734] removing any third candidate block vector of the reference block belonging to the first area, and continuing to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained;
[0735] The second block vector candidate list is determined based on the filtered at least one third candidate block vector.
[0736] In some embodiments of the present application, the first determining part 11 is further configured to perform template matching refinement on the second block vector candidate list to determine a fourth block vector candidate list;
[0737] The decoding part 10 is further configured to decode the code stream and determine the candidate block vector index information;
[0738] The first determining part 11 is further configured to determine a first block vector according to the candidate block vector index information and the fourth block vector candidate list; and determine a reference block of the current block according to the first block vector;
[0739] The first prediction part 13 is further configured to determine a prediction value of the current block according to the reference block.
[0740] In some embodiments of the present application, the first removing part 12 is further configured to perform a duplicate checking operation on the candidate block vectors in the first block vector candidate list during the process of determining the first block vector candidate list of the current block.
[0741] In some embodiments of the present application, the first removing part 12 is further configured to determine a current candidate block vector to be added;
[0742] comparing the current candidate block vector with candidate block vectors in the first block vector candidate list;
[0743] If the current candidate block vector does not overlap with a candidate block vector in the first block vector candidate list, adding the current candidate block vector to the first block vector candidate list;
[0744] If the current candidate block vector coincides with a candidate block vector in the first block vector candidate list, the current candidate block vector is skipped and the step of determining the next candidate block vector to be added is continued.
[0745] In some embodiments of the present application, the first removing part 12 is further configured to ensure that the current candidate block vector is completely identical to the candidate block vector in the first block vector candidate list.
[0746] In some embodiments of the present application, the first removing part 12 is further configured to ensure that a difference between the current candidate block vector and a candidate block vector in the first block vector candidate list satisfies a first preset condition.
[0747] In some embodiments of the present application, the decoding part 10 is further configured to decode the code stream to determine the prediction difference value of the current block;
[0748] The first prediction part 13 is further configured to determine a reconstructed value of the current block according to the predicted value of the current block and the prediction difference value of the current block.
[0749] In some embodiments of the present application, the first area range includes one of the following:
[0750] The size of the current encoding unit;
[0751] The size of the reconstructed area in the current frame;
[0752] The size of the image border of the current block;
[0753] Search range for Intra TMP mode.
[0754] In some embodiments of the present application, the decoding part 10 is further configured to decode the code stream and determine first syntax identification information; the first syntax identification information is used to indicate whether the current block uses a prediction mode based on intra block copy;
[0755] If the first syntax identification information indicates that the current block uses a prediction mode based on intra block copy, the second syntax identification information is determined; the second syntax identification information indicates whether the current block uses a prediction mode based on intra block merging to determine a prediction value of the current block.
[0756] In practical applications, the embodiment of the present application further provides a decoder, as shown in Figure 27, which shows a schematic diagram of the specific hardware structure of the decoder 1 provided in the embodiment of the present application. As shown in Figure 27, the decoder 1 may include:
[0757] The first communication interface 14, the first memory 15, and the first processor 16; each component is coupled together via a first bus system 17. It is understood that the first bus system 17 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 17 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in FIG27, all buses are labeled as the first bus system 17.
[0758] The first communication interface 14 is used to receive and send signals during the process of sending and receiving information with other external network elements;
[0759] A first memory 15, for storing computer programs that can be run on the first processor 16;
[0760] The first processor 16 is configured to execute a decoding method on the decoder side when running the computer program.
[0761] It can be understood that during the decoding process, during the block decoding process, for the scenario of intra-frame merging mode prediction, the decoder can remove the overlapping block vectors or block vectors within the first area, such as in the intra-frame template matching prediction mode, based on the first block vector candidate list of the current block, and determine the second block vector candidate list, and determine the prediction value of the current block based on the de-redundant block vector in the second block vector candidate list. In this way, since the redundancy of the block vector is reduced, the expression efficiency of the header information can be improved when using the de-redundant block vector or block vector for encoding and decoding, thereby improving the video decoding efficiency.
[0762] It is understood that the first memory 15 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 15 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0763] The first processor 16 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 16. The above-mentioned first processor 16 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 15. The first processor 16 reads the information in the first memory 15 and, in conjunction with its hardware, completes the steps of the above method.
[0764] In another embodiment of the present application, based on the same inventive concept as the above embodiment, see Figure 28, which shows a schematic diagram of the composition structure of an encoder 2 provided in an embodiment of the present application. As shown in Figure 28, the encoder 2 may include:
[0765] The second determining part 20 is configured to determine a prediction mode of a current block; and if the current block uses a prediction mode based on intra block merging, determine a first block vector candidate list of the current block; and determine a first area range;
[0766] A second removing part 21 is configured to determine a second block vector candidate list based on the first block vector candidate list and the first area range;
[0767] The second prediction part 22 is configured to predict the value of the current block based on the second block vector candidate list.
[0768] In some embodiments of the present application, the second removing part 21 is further configured to remove block vectors belonging to the first area based on the first block vector candidate list to obtain a second block vector candidate list.
[0769] In some embodiments of the present application, the second removing part 21 is further configured to, if it is determined that the current block uses a prediction mode based on merging with block vector differences within a frame, determine at least one first extended candidate block vector corresponding to each first candidate block vector based on the first block vector candidate list;
[0770] The block vectors belonging to the first area range are removed from the at least one first extended candidate block vector corresponding to each first candidate block vector to obtain the second block vector candidate list.
[0771] In some embodiments of the present application, the second determining part 20 is further configured to determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1;
[0772] Based on the distance set and the block vector prediction direction, each of the N first candidate block vectors is refined to determine at least one first extended candidate block vector corresponding to each first candidate block vector.
[0773] In some embodiments of the present application, the second removing portion 21 is further configured to, for each first candidate block vector, sequentially remove block vectors corresponding to reference blocks belonging to the first region for each first extended candidate block vector, to obtain at least one filtered first extended candidate block vector;
[0774] The second block vector candidate list is determined based on the filtered at least one first extended candidate block vector corresponding to each first candidate block vector.
[0775] In some embodiments of the present application, the second removing portion 21 is further configured to: if a horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector falls within a horizontal range of a preset range, and a vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector falls within a vertical range of a preset range, then the first candidate reference block pointed to by any first extended candidate block vector belongs to the reference block of the first area range;
[0776] Any first extended candidate block vector is removed, and the removal process is continued for other first extended candidate block vectors corresponding to each first candidate block vector to obtain at least one filtered first extended candidate block vector.
[0777] In some embodiments of the present application, the second removal part 21 is further configured to, if the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the horizontal range of a preset range, or the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the vertical range of a preset range, then the first candidate reference block pointed to by any first extended candidate block vector does not fall within the reference block of the first area range; and the any first extended candidate block vector is a filtered first extended candidate block vector corresponding to each candidate block vector.
[0778] In some embodiments of the present application, the second determining portion 20 is further configured to determine, for each first candidate block vector, M first extended candidate block vectors corresponding to the minimum template cost from the at least one first extended candidate block vector after screening; wherein M is an integer greater than or equal to 1;
[0779] The second block vector candidate list is constructed based on the N first extended candidate block vectors of each first candidate block vector.
[0780] In some embodiments of the present application, the second determining portion 20 is further configured to, for each of the first candidate block vectors, determine another first candidate vector from the first block vector candidate list if the number of at least one first extended candidate block vector after screening is less than M; the other first candidate vector is different from the N first candidate vectors;
[0781] M first extended candidate block vectors are determined based on the other first candidate vectors and the screened at least one first extended candidate block vector.
[0782] In some embodiments of the present application, the second determining part 20 is further configured to determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1;
[0783] removing the block vectors belonging to the first region from the N first candidate block vectors to obtain the third block vector candidate list;
[0784] Based on the third block vector candidate list, at least one first extended candidate block vector corresponding to each first candidate block vector is determined.
[0785] In some embodiments of the present application, the second determination part 20 is further configured to refine each first candidate block vector in the third block vector candidate list based on the distance set and the block vector prediction direction, and determine at least one first extended candidate block vector corresponding to each first candidate block vector.
[0786] In some embodiments of the present application, the second removing portion 21 is further configured to sequentially remove, from each of the N first candidate block vectors, the block vector corresponding to the reference block belonging to the first region, to obtain at least one filtered first candidate block vector;
[0787] The third block vector candidate list is determined based on the filtered at least one first candidate block vector.
[0788] In some embodiments of the present application, the second removing part 21 is further configured to sequentially offset each of the N first candidate block vectors to determine at least one second candidate block vector corresponding to each first candidate block vector;
[0789] If the maximum horizontal boundary position and the minimum horizontal boundary position in the at least one second candidate reference block pointed to by the at least one second candidate block vector are within the horizontal range of the preset range, and the maximum vertical boundary position and the minimum vertical boundary position in the at least one second candidate reference block are within the vertical range of the preset range, then the first candidate block vector corresponding to the at least one second candidate block vector belongs to the block vector of the first area range;
[0790] The first candidate block vector is removed, and the removal process is continued on other first candidate block vectors to obtain at least one first candidate block vector after screening.
[0791] In some embodiments of the present application, the second removing portion 21 is further configured to: if the maximum horizontal boundary position or the minimum horizontal boundary position in the at least one second candidate reference block pointed to by the at least one second candidate block vector does not fall within a preset horizontal range, or the maximum vertical boundary position or the minimum vertical boundary position in the at least one second candidate reference block does not fall within a preset vertical range, then the candidate block vector corresponding to the at least one second candidate block vector does not fall within the block vector of the first area; and the first candidate block vector corresponding to the at least one second candidate block vector is used as a filtered first candidate block vector;
[0792] The removal process is continued on other first candidate block vectors until the processing of N first candidate block vectors is completed, or N filtered first candidate block vectors are obtained.
[0793] In some embodiments of the present application, the second removing part 21 is further configured to, if it is determined that the current block uses the prediction mode of template matching-based merging, determine K third candidate block vectors with minimum template costs from the first block vector candidate list; K is an integer greater than or equal to 1;
[0794] Block vectors belonging to the first area are removed from the K third candidate block vectors to obtain the second block vector candidate list.
[0795] In some embodiments of the present application, the second removing portion 21 is further configured to: if, among the K third candidate block vectors, a horizontal boundary position of a third candidate reference block pointed to by any third candidate block vector falls within a horizontal range of a preset range, and a vertical boundary position of a third candidate reference block pointed to by any third candidate block vector falls within a vertical range of a preset range, then the third candidate reference block pointed to by any third candidate block vector belongs to a reference block within the first area range;
[0796] removing any third candidate block vector of the reference block belonging to the first area, and continuing to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained;
[0797] The second block vector candidate list is determined based on the filtered at least one third candidate block vector.
[0798] In some embodiments of the present application, the second removing portion 21 is further configured to: if, among the K third candidate block vectors, a first extended horizontal boundary position of a third candidate reference block pointed to by any third candidate block vector falls within a preset horizontal range, and a first extended vertical boundary position of a third candidate reference block pointed to by any third candidate block vector falls within a preset vertical range, then the third candidate reference block pointed to by any third candidate block vector belongs to a reference block within the first area range;
[0799] The first extended horizontal boundary position is the horizontal boundary position of the third candidate reference block obtained by extending the first horizontal search range by the horizontal boundary of the third candidate reference block; the first extended vertical boundary position is the vertical boundary position of the third candidate reference block obtained by extending the first vertical search range by the vertical boundary of the third candidate reference block; the first horizontal search range and the first vertical search range are search ranges specified in the prediction mode based on template matching merging;
[0800] removing any third candidate block vector of the reference block belonging to the first area, and continuing to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained;
[0801] The second block vector candidate list is determined based on the filtered at least one third candidate block vector.
[0802] In some embodiments of the present application, the second removing part 21 is further configured to perform a duplicate checking operation on the candidate block vectors in the first block vector candidate list during the process of determining the first block vector candidate list of the current block.
[0803] In some embodiments of the present application, the second removing part 21 is further configured to determine a current candidate block vector to be added;
[0804] comparing the current candidate block vector with candidate block vectors in the first block vector candidate list;
[0805] If the current candidate block vector does not overlap with a candidate block vector in the first block vector candidate list, adding the current candidate block vector to the first block vector candidate list;
[0806] If the current candidate block vector coincides with a candidate block vector in the first block vector candidate list, the current candidate block vector is skipped and the step of determining the next candidate block vector to be added is continued.
[0807] In some embodiments of the present application, the second removing part 21 is further configured so that the current candidate block vector is completely identical to the candidate block vector in the first block vector candidate list.
[0808] In some embodiments of the present application, the second removing part 21 is further configured to ensure that a difference between the current candidate block vector and a candidate block vector in the first block vector candidate list satisfies a first preset condition.
[0809] In some embodiments of the present application, the encoder 2 further includes: a writing part 23; the second determining part 20 is further configured to determine the first syntax identification information and the second syntax identification information if the current block uses the prediction mode based on intra-frame block merging;
[0810] The writing part 23 is further configured to write first syntax identification information and second syntax identification information into the code stream; the first syntax identification information indicates whether the current block uses the prediction mode of intra-frame block copy; the second syntax identification information indicates whether the current block uses the prediction mode of intra-frame block merging.
[0811] In some embodiments of the present application, the second determining portion 20 is further configured to determine third grammar identification information;
[0812] The writing part 23 is further configured to write third syntax identification information into the bitstream, where the third syntax identification information indicates whether the current block uses a prediction mode based on intra-frame merging with block vector differences.
[0813] In some embodiments of the present application, the second determining part 20 is further configured to determine a first block vector according to the second block vector candidate list; determine a reference block of the current block according to the first block vector;
[0814] The second prediction part 22 is further configured to determine a prediction value of the current block according to the reference block.
[0815] In some embodiments of the present application, the writing part 23 is further configured to write the candidate block vector index information corresponding to the first block vector into the code stream.
[0816] In some embodiments of the present application, the writing part 23 is further configured to write the candidate block vector index information and the extended candidate block vector index information corresponding to the first block vector into the code stream.
[0817] In some embodiments of the present application, the second prediction part 22 is further configured to determine the original value of the current block; and determine the prediction difference value of the current block according to the original value of the current block and the predicted value of the current block.
[0818] In some embodiments of the present application, the first area range includes one of the following:
[0819] The size of the current encoding unit;
[0820] The size of the reconstructed area in the current frame;
[0821] The size of the image border of the current block;
[0822] Search range for Intra TMP mode.
[0823] In practical applications, the embodiment of the present application further provides an encoder, as shown in Figure 29, which shows a schematic diagram of the specific hardware structure of the encoder 2 provided in the embodiment of the present application. As shown in Figure 29, the encoder 2 may include:
[0824] The second communication interface 24, the second memory 25, and the second processor 26 are coupled together via a second bus system 27. It is understood that the second bus system 27 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 27 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in FIG29, all buses are labeled as the second bus system 27.
[0825] The second communication interface 24 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;
[0826] A second memory 25 for storing computer programs that can be run on the second processor 26;
[0827] The second processor 26 is configured to execute the encoding method on the encoder side when running the computer program.
[0828] It can be understood that during the encoding process, during the block encoding process, for the scenario of intra-frame merging mode prediction, the encoder can remove the overlapping block vectors or block vectors within the first area, such as in the intra-frame template matching prediction mode, based on the first block vector candidate list of the current block, and determine the second block vector candidate list. The prediction value of the current block is determined based on the de-redundant block vector in the second block vector candidate list. In this way, since the redundancy of the block vector is reduced, the expression efficiency of the header information can be improved when the de-redundant block vector or block vector is used for encoding, thereby improving the video coding efficiency.
[0829] An embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a first processor, it implements a decoding method such as a decoder, or when the computer program is executed by a second processor, it implements an encoding method corresponding to an encoder.
[0830] In yet another embodiment of the present application, referring to FIG30 , a schematic diagram of the structure of a coding and decoding system provided by an embodiment of the present application is shown. As shown in FIG30 , the coding and decoding system 3 may include an encoder 2 and a decoder 1 .
[0831] In the embodiment of the present application, the encoder 2 may be the encoder described in any one of the aforementioned embodiments, and the decoder 1 may be the decoder described in any one of the aforementioned embodiments.
[0832] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.
[0833] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0834] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0835] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.
[0836] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0837] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability
[0838] In an embodiment of the present application, whether it is the encoding end or the decoding end, in the process of block encoding and decoding, for the scenario of intra-frame merging mode prediction, based on the first block vector candidate list of the current block, the block vectors or block vectors with overlap within the first area, such as in the intra-frame template matching prediction mode, can be removed to determine the second block vector candidate list, and the prediction value of the current block is determined based on the de-redundant block vectors in the second block vector candidate list. In this way, since the redundancy of the block vector is reduced, the expression efficiency of the header information can be improved when the de-redundant block vector or block vector is used for encoding and decoding, thereby improving the video encoding and decoding efficiency.
Claims
1. A decoding method, applied to a decoder, the method comprising: Decode the bitstream and determine the prediction mode of the current block; If the prediction mode of the current block indicates that the current block uses a prediction mode based on intra-block merging to determine a prediction value of the current block, determining a first block vector candidate list of the current block; Determine a first area range, and determine a second block vector candidate list based on the first block vector candidate list and the first area range; Based on the second block vector candidate list, a prediction value of the current block is determined.
2. The method according to claim 1, wherein: The determining, based on the first block vector candidate list and the first area range, a second block vector candidate list includes: Based on the first block vector candidate list, block vectors belonging to the first area are removed to obtain a second block vector candidate list.
3. The method according to claim 2, wherein: The step of removing the block vectors within the first region based on the first block vector candidate list to obtain a second block vector candidate list includes: If it is determined that the current block uses a prediction mode based on a merge with a block vector difference within a frame, then determining candidate block vector index information; Based on the first block vector candidate list, determine at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information; The block vectors belonging to the first area range are removed from the at least one first extended candidate block vector to obtain the second block vector candidate list.
4. The method according to claim 3, wherein: The determining, based on the first block vector candidate list, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information comprises: Determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1; Determine a first candidate block vector from the N first candidate block vectors based on the candidate block vector index information; Based on the distance set and the block vector prediction direction, the first candidate block vector is refined to determine at least one first extended candidate block vector corresponding to the first candidate block vector.
5. The method according to claim 4, wherein: The block vectors belonging to the first area range are removed from the at least one first extended candidate block vector to obtain the second block vector candidate list, including: For each first extended candidate block vector, the block vector corresponding to the reference block belonging to the first area is removed in turn to obtain at least one first extended candidate block vector after screening; Based on the screened at least one first extended candidate block vector, the second block vector candidate list is determined.
6. The method according to claim 5, wherein: The method of removing the block vectors corresponding to the reference blocks belonging to the first area from each first extended candidate block vector in turn to obtain at least one first extended candidate block vector after screening includes: If the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector is within the horizontal range of the preset range, and the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector is within the vertical range of the preset range, then the first candidate reference block pointed to by any first extended candidate block vector belongs to the reference block of the first area range; Any of the first extended candidate block vectors is removed, and the removal process is continued on other first extended candidate block vectors to obtain at least one first extended candidate block vector after screening.
7. The method according to claim 5 or 6, wherein: The method further comprises: If the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the horizontal range of the preset range, or the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the vertical range of the preset range, then the first candidate reference block pointed to by any first extended candidate block vector does not fall within the reference block of the first area; the any first extended candidate block vector is a screened first extended candidate block vector corresponding to each candidate block vector.
8. The method according to any one of claims 5 to 7, wherein: The determining the second block vector candidate list based on the at least one first extended candidate block vector after screening includes: Determining M first extended candidate block vectors corresponding to the minimum template cost from the at least one first extended candidate block vector after screening; wherein M is an integer greater than or equal to 1; Based on the M first extended candidate block vectors, the second block vector candidate list is constructed.
9. The method according to claim 8, wherein: The method further comprises: When the number of at least one first extended candidate block vector after screening is less than M, determine other candidate block vectors from the first block vector candidate list; the other candidate block vectors are different from the N candidate block vectors; Based on the other candidate block vectors and the screened at least one first extended candidate block vector, M first extended candidate block vectors are determined.
10. The method according to any one of claims 1 to 9, wherein: The step of determining the prediction value of the current block based on the second block vector candidate list includes: Decode the code stream and determine the index information of the extended candidate block vector; Determine a first block vector according to the extended candidate block vector index information and the second block vector candidate list; A reference block of the current block is determined according to the first block vector, and a prediction value of the current block is determined according to the reference block.
11. The method according to any one of claims 3 to 10, wherein: The determining, based on the first block vector candidate list, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information comprises: Determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1; Removing the block vectors belonging to the first area from the N first candidate block vectors to obtain the third block vector candidate list; Based on the third block vector candidate list, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information is determined.
12. The method according to claim 11, wherein: The determining, based on the third block vector candidate list, at least one first extended candidate block vector corresponding to the first candidate block vector indicated by the candidate block vector index information comprises: Determine a first candidate block vector from the third block vector candidate list based on the candidate block vector index information; Based on the distance set and the block vector prediction direction, the first candidate block vector is refined to determine at least one first extended candidate block vector corresponding to the first candidate block vector.
13. The method according to claim 11 or 12, wherein: The method further comprises: Decode the code stream and determine the index information of the extended candidate block vector; Determine a first block vector from the at least one first extended candidate block vector according to the extended candidate block vector index information; A reference block of the current block is determined according to the first block vector, and a prediction value of the current block is determined according to the reference block.
14. The method according to any one of claims 11 to 13, wherein: The step of removing the block vectors belonging to the first region from the N first candidate block vectors to obtain the third block vector candidate list includes: For each first candidate block vector among the N first candidate block vectors, the block vector corresponding to the reference block belonging to the first area is removed in sequence to obtain at least one first candidate block vector after screening; The third block vector candidate list is determined based on the screened at least one first candidate block vector.
15. The method according to claim 14, wherein: The method of removing the block vector corresponding to the reference block belonging to the first area from each of the N first candidate block vectors in turn to obtain at least one first candidate block vector after screening includes: Sequentially shifting each of the N first candidate block vectors to determine at least one second candidate block vector corresponding to each first candidate block vector; If the maximum horizontal boundary position and the minimum horizontal boundary position in the at least one second candidate reference block pointed to by the at least one second candidate block vector belong to the horizontal range of the preset range, and the maximum vertical boundary position and the minimum vertical boundary position in the at least one second candidate reference block belong to the vertical range of the preset range, then the first candidate block vector corresponding to the at least one second candidate block vector belongs to the block vector of the first area range; The first candidate block vector is removed, and the removal process is continued on other first candidate block vectors to obtain at least one first candidate block vector after screening.
16. The method according to claim 14 or 15, wherein: The method further comprises: If the maximum horizontal boundary position or the minimum horizontal boundary position in the at least one second candidate reference block pointed to by the at least one second candidate block vector does not belong to the horizontal range of the preset range, or the maximum vertical boundary position or the minimum vertical boundary position in the at least one second candidate reference block does not belong to the vertical range of the preset range, then the candidate block vector corresponding to the at least one second candidate block vector does not belong to the block vector of the first area; the first candidate block vector corresponding to the at least one second candidate block vector is used as a screened first candidate block vector; The removal process is continued on other first candidate block vectors until the processing of N first candidate block vectors is completed, or N screened first candidate block vectors are obtained.
17. The method according to claim 2, wherein: The step of removing the block vectors within the first region based on the first block vector candidate list to obtain a second block vector candidate list includes: If it is determined that the current block uses a prediction mode based on template matching merging, then determining K third candidate block vectors with minimum template costs from the first block vector candidate list; K is an integer greater than or equal to 1; The block vectors belonging to the first area range are removed from the K third candidate block vectors to obtain the second block vector candidate list.
18. The method according to claim 17, wherein: The step of removing the block vectors belonging to the first region from the K third candidate block vectors to obtain the second block vector candidate list includes: If, among the K third candidate block vectors, the horizontal boundary position of the third candidate reference block pointed to by any third candidate block vector is within the horizontal range of the preset range, and the vertical boundary position of the third candidate reference block pointed to by any third candidate block vector is within the vertical range of the preset range, then the third candidate reference block pointed to by any third candidate block vector belongs to the reference block of the first area range; Remove any third candidate block vector of the reference block belonging to the first area, and continue to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained; Based on the screened at least one third candidate block vector, the second block vector candidate list is determined.
19. The method according to claim 17, wherein: The step of removing the block vectors belonging to the first region from the K third candidate block vectors to obtain the second block vector candidate list includes: If, among the K third candidate block vectors, a first extended horizontal boundary position of a third candidate reference block pointed to by any third candidate block vector belongs to the horizontal range of the preset range, and a first extended vertical boundary position of a third candidate reference block pointed to by any third candidate block vector belongs to the vertical range of the preset range, then the third candidate reference block pointed to by any third candidate block vector belongs to the reference block of the first area range; The first extended horizontal boundary position is the horizontal boundary position of the third candidate reference block after the first horizontal search range is extended; the first extended vertical boundary position is the vertical boundary position of the third candidate reference block after the first vertical search range is extended; the first horizontal search range and the first vertical search range are search ranges specified in the prediction mode based on template matching merging; Remove any third candidate block vector of the reference block belonging to the first area, and continue to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained; Based on the screened at least one third candidate block vector, the second block vector candidate list is determined.
20. The method according to any one of claims 17 to 19, wherein: The step of determining the prediction value of the current block based on the second block vector candidate list includes: Performing template matching refinement on the second block vector candidate list to determine a fourth block vector candidate list; Decode the bitstream and determine the candidate block vector index information; Determine a first block vector according to the candidate block vector index information and the fourth block vector candidate list; A reference block of the current block is determined according to the first block vector, and a prediction value of the current block is determined according to the reference block.
21. The method according to any one of claims 1 to 20, wherein: The method further comprises: In the process of determining the first block vector candidate list of the current block, a duplication checking operation is performed on the candidate block vectors in the first block vector candidate list.
22. The method according to claim 21, wherein: The performing a duplication check operation on the candidate block vectors in the first block vector candidate list includes: Determine the current candidate block vector to be added; comparing the current candidate block vector with candidate block vectors in the first block vector candidate list; If the current candidate block vector does not overlap with the candidate block vector in the first block vector candidate list, adding the current candidate block vector to the first block vector candidate list; If the current candidate block vector coincides with a candidate block vector in the first block vector candidate list, the current candidate block vector is skipped, and the step of determining the next candidate block vector to be added is continued.
23. The method according to claim 22, wherein: The current candidate block vector coincides with a candidate block vector in the first block vector candidate list, including: The current candidate block vector is completely identical to the candidate block vector in the first block vector candidate list.
24. The method according to claim 22, wherein: The current candidate block vector coincides with a candidate block vector in the first block vector candidate list, including: A difference between the current candidate block vector and a candidate block vector in the first block vector candidate list satisfies a first preset condition.
25. The method according to any one of claims 1 to 22, wherein: The method further comprises: Decoding a bitstream to determine a prediction difference value of the current block; A reconstructed value of the current block is determined according to the predicted value of the current block and the predicted difference value of the current block.
26. The method of claim 1, wherein: The first area includes one of the following: The size of the current encoding unit; The size of the reconstructed area in the current frame; The size of the image border of the current block; Search range for Intra TMP mode.
27. The method of claim 1, wherein: The method further comprises: Decoding a bitstream to determine first syntax identification information; the first syntax identification information is used to indicate whether a current block uses a prediction mode based on an intra-frame block copy; If the first syntax identification information indicates that the current block uses a prediction mode based on intra block copy, the second syntax identification information is determined; the second syntax identification information indicates whether the current block uses a prediction mode based on intra block merging to determine a prediction value of the current block.
28. A coding method, applied to an encoder, the method comprising: Determine the prediction mode of the current block; If the prediction mode indicates that the current block uses a prediction mode based on intra-frame block merging, determining a first block vector candidate list of the current block; Determine a first area range, and determine a second block vector candidate list based on the first block vector candidate list and the first area range; Based on the second block vector candidate list, a prediction value of the current block is determined.
29. The method according to claim 28, wherein: The determining, based on the first block vector candidate list and the first area range, a second block vector candidate list includes: Based on the first block vector candidate list, block vectors belonging to the first area are removed to obtain a second block vector candidate list.
30. The method of claim 29, wherein: The step of removing the block vectors within the first region based on the first block vector candidate list to obtain a second block vector candidate list includes: If it is determined that the current block uses a prediction mode based on merging with block vector differences within a frame, then determining at least one first extended candidate block vector corresponding to each first candidate block vector based on the first block vector candidate list; The block vectors belonging to the first area range are removed from the at least one first extended candidate block vector corresponding to each first candidate block vector to obtain the second block vector candidate list.
31. The method according to claim 30, wherein: The determining, based on the first block vector candidate list, at least one first extended candidate block vector corresponding to each first candidate block vector comprises: Determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1; Based on the distance set and the block vector prediction direction, each of the N first candidate block vectors is refined to determine at least one first extended candidate block vector corresponding to each first candidate block vector.
32. The method according to claim 31, wherein: The step of removing the block vectors belonging to the first region from the at least one first extended candidate block vector corresponding to each first candidate block vector to obtain the second block vector candidate list includes: For each first candidate block vector, sequentially remove the block vectors corresponding to the reference blocks belonging to the first area from each first extended candidate block vector, to obtain at least one first extended candidate block vector after screening; The second block vector candidate list is determined based on the filtered at least one first extended candidate block vector corresponding to each first candidate block vector.
33. The method of claim 32, wherein: The method of removing, for each first extended candidate block vector, the block vectors corresponding to the reference blocks belonging to the first area range from each first extended candidate block vector in turn to obtain at least one first extended candidate block vector after screening includes: If the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector is within the horizontal range of the preset range, and the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector is within the vertical range of the preset range, then the first candidate reference block pointed to by any first extended candidate block vector belongs to the reference block of the first area range; Any first extended candidate block vector is removed, and other first extended candidate block vectors corresponding to each first candidate block vector are removed to obtain at least one first extended candidate block vector after screening.
34. The method according to claim 32 or 33, wherein: The method further comprises: If the horizontal boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the horizontal range of the preset range, or the vertical boundary position of the first candidate reference block pointed to by any first extended candidate block vector does not fall within the vertical range of the preset range, then the first candidate reference block pointed to by any first extended candidate block vector does not fall within the reference block of the first area; the any first extended candidate block vector is a screened first extended candidate block vector corresponding to each candidate block vector.
35. The method according to any one of claims 32 to 34, wherein: The determining the second block vector candidate list based on the at least one first extended candidate block vector after screening corresponding to each first candidate block vector comprises: For each first candidate block vector, determine M first extended candidate block vectors corresponding to the minimum template cost from the at least one first extended candidate block vector after screening; wherein M is an integer greater than or equal to 1; The second block vector candidate list is constructed based on the M first extended candidate block vectors of each first candidate block vector.
36. The method of claim 35, wherein: The method further comprises: For each of the first candidate block vectors, when the number of at least one first extended candidate block vector after screening is less than M, determine other first candidate vectors from the first block vector candidate list; the other first candidate vectors are different from the N first candidate vectors; Based on the other first candidate vectors and the screened at least one first extended candidate block vector, M first extended candidate block vectors are determined.
37. The method according to any one of claims 30 to 36, wherein: The determining, based on the first block vector candidate list, at least one first extended candidate block vector corresponding to each first candidate block vector comprises: Determine N first candidate block vectors with minimum template costs from the first block vector candidate list; N is an integer greater than or equal to 1; Removing the block vectors belonging to the first area from the N first candidate block vectors to obtain the third block vector candidate list; Based on the third block vector candidate list, at least one first extended candidate block vector corresponding to each first candidate block vector is determined.
38. The method according to claim 37, characterized in that The step of determining at least one first extended candidate block vector corresponding to each first candidate block vector based on the third block vector candidate list comprises: Based on the distance set and the block vector prediction direction, each first candidate block vector in the third block vector candidate list is refined to determine at least one first extended candidate block vector corresponding to each first candidate block vector.
39. The method according to claim 37 or 38, characterized in that The step of removing the block vectors belonging to the first region from the N first candidate block vectors to obtain the third block vector candidate list includes: For each first candidate block vector among the N first candidate block vectors, the block vector corresponding to the reference block belonging to the first area is removed in sequence to obtain at least one first candidate block vector after screening; The third block vector candidate list is determined based on the screened at least one first candidate block vector.
40. The method of claim 39, wherein: The method of removing the block vector corresponding to the reference block belonging to the first area from each of the N first candidate block vectors in turn to obtain at least one first candidate block vector after screening includes: Sequentially shifting each of the N first candidate block vectors to determine at least one second candidate block vector corresponding to each first candidate block vector; If the maximum horizontal boundary position and the minimum horizontal boundary position in the at least one second candidate reference block pointed to by the at least one second candidate block vector belong to the horizontal range of the preset range, and the maximum vertical boundary position and the minimum vertical boundary position in the at least one second candidate reference block belong to the vertical range of the preset range, then the first candidate block vector corresponding to the at least one second candidate block vector belongs to the block vector of the first area range; The first candidate block vector is removed, and the removal process is continued on other first candidate block vectors to obtain at least one first candidate block vector after screening.
41. The method according to claim 39 or 40, wherein: The method further comprises: If the maximum horizontal boundary position or the minimum horizontal boundary position in at least one second candidate reference block pointed to by the at least one second candidate block vector does not fall within the horizontal range of the preset range, or the maximum vertical boundary position or the minimum vertical boundary position in the at least one second candidate reference block does not fall within the vertical range of the preset range, then the candidate block vector corresponding to the at least one second candidate block vector does not fall within the block vector of the first area; the first candidate block vector corresponding to the at least one second candidate block vector The quantity is used as a first candidate block vector after screening; The removal process is continued on other first candidate block vectors until the processing of N first candidate block vectors is completed, or N screened first candidate block vectors are obtained.
42. The method of claim 29, wherein: The step of removing the block vectors within the first region based on the first block vector candidate list to obtain a second block vector candidate list includes: If it is determined that the current block uses a prediction mode based on template matching merging, then determining K third candidate block vectors with minimum template costs from the first block vector candidate list; K is an integer greater than or equal to 1; The block vectors belonging to the first area range are removed from the K third candidate block vectors to obtain the second block vector candidate list.
43. The method of claim 42, wherein: The step of removing the block vectors belonging to the first region from the K third candidate block vectors to obtain the second block vector candidate list includes: If, among the K third candidate block vectors, the horizontal boundary position of the third candidate reference block pointed to by any third candidate block vector is within the horizontal range of the preset range, and the vertical boundary position of the third candidate reference block pointed to by any third candidate block vector is within the vertical range of the preset range, then the third candidate reference block pointed to by any third candidate block vector belongs to the reference block of the first area range; Remove any third candidate block vector of the reference block belonging to the first area, and continue to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained; Based on the screened at least one third candidate block vector, the second block vector candidate list is determined.
44. The method of claim 42, wherein: The step of removing the block vectors belonging to the first region from the K third candidate block vectors to obtain the second block vector candidate list includes: If, among the K third candidate block vectors, a first extended horizontal boundary position of a third candidate reference block pointed to by any third candidate block vector belongs to the horizontal range of the preset range, and a first extended vertical boundary position of a third candidate reference block pointed to by any third candidate block vector belongs to the vertical range of the preset range, then the third candidate reference block pointed to by any third candidate block vector belongs to the reference block of the first area range; The first extended horizontal boundary position is the horizontal boundary position of the third candidate reference block after the first horizontal search range is extended; the first extended vertical boundary position is the vertical boundary position of the third candidate reference block after the first vertical search range is extended; the first horizontal search range and the first vertical search range are search ranges specified in the prediction mode based on template matching merging; Remove any third candidate block vector of the reference block belonging to the first area, and continue to remove other third candidate block vectors until the processing of K third candidate block vectors is completed, or K screened third candidate block vectors are obtained; Based on the screened at least one third candidate block vector, the second block vector candidate list is determined.
45. A method according to any one of claims 28 to 44, wherein: The method further comprises: In the process of determining the first block vector candidate list of the current block, a duplication checking operation is performed on the candidate block vectors in the first block vector candidate list.
46. The method of claim 45, wherein: The performing a duplication check operation on the candidate block vectors in the first block vector candidate list includes: Determine the current candidate block vector to be added; comparing the current candidate block vector with candidate block vectors in the first block vector candidate list; If the current candidate block vector does not overlap with the candidate block vector in the first block vector candidate list, adding the current candidate block vector to the first block vector candidate list; If the current candidate block vector coincides with a candidate block vector in the first block vector candidate list, the current candidate block vector is skipped, and the step of determining the next candidate block vector to be added is continued.
47. The method of claim 46, wherein: The current candidate block vector coincides with a candidate block vector in the first block vector candidate list, including: The current candidate block vector is completely identical to the candidate block vector in the first block vector candidate list.
48. The method of claim 46, wherein: The current candidate block vector coincides with a candidate block vector in the first block vector candidate list, including: A difference between the current candidate block vector and a candidate block vector in the first block vector candidate list satisfies a first preset condition.
49. The method of claim 28, wherein: The method further comprises: If the current block uses a prediction mode based on intra-frame block merging, the first syntax identification information is determined, and the second syntax identification information is determined and written into the bitstream; the first syntax identification information represents whether the current block uses a prediction mode of intra-frame block copy; the second syntax identification information represents whether the current block uses a prediction mode of intra-frame block merging.
50. The method of claim 30 or 42, wherein: The method further comprises: Determine third syntax identification information and write it into the bitstream, wherein the third syntax identification information indicates whether the current block uses a prediction mode based on merging with block vector differences within a frame.
51. A method according to any one of claims 28 to 50, wherein: The step of determining the prediction value of the current block based on the second block vector candidate list includes: Determine a first block vector according to the second block vector candidate list; A reference block of the current block is determined according to the first block vector, and a prediction value of the current block is determined according to the reference block.
52. The method of claim 51, wherein: The method further comprises: The candidate block vector index information corresponding to the first block vector is written into the bitstream.
53. The method of claim 30, wherein: The method further comprises: The candidate block vector index information and the extended candidate block vector index information corresponding to the first block vector are written into the bitstream.
54. A method according to any one of claims 29 to 53, wherein: The method further comprises: Determining an original value of the current block; A prediction difference value of the current block is determined according to the original value of the current block and the prediction value of the current block.
55. The method of claim 28, wherein: The first area includes one of the following: The size of the current encoding unit; The size of the reconstructed area in the current frame; The size of the image border of the current block; Search range for Intra TMP mode.
56. A code stream, wherein The code stream is generated by bit encoding according to the information to be encoded; wherein the information to be encoded includes at least one of the following: The prediction difference value of the current block, the candidate block vector index information, the extended candidate block vector index information, the first syntax identification information, the second syntax identification information and the third syntax identification information; Among them, the first syntax identification information is used to indicate whether the current block uses the prediction mode of intra-frame block copy, the second syntax identification information is used to indicate whether the current block uses the prediction mode of intra-frame block merging, and the third syntax identification information is used to indicate whether the current block uses the MBVD mode.
57. A decoder comprising: A decoding part, configured to decode the bitstream and determine a prediction mode of a current block; A first determining part is configured to determine a first block vector candidate list of the current block if the prediction mode of the current block indicates that the current block uses a prediction mode based on intra-frame block merging to determine a prediction value of the current block; and determine a first area range; A first removing part is configured to determine a second block vector candidate list based on the first block vector candidate list and the first area range; The first prediction part is configured to determine the prediction value of the current block based on the second block vector candidate list.
58. A decoder comprising a first memory and a first processor; wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 1 to 27 when running the computer program.
59. An encoder comprising: A second determining part is configured to determine a prediction mode of a current block; and if the current block uses a prediction mode based on intra-block merging, determining a first block vector candidate list of the current block; Determine the scope of the first area; A second removing part is configured to determine a second block vector candidate list based on the first block vector candidate list and the first area range; The second prediction part is configured to determine the prediction value of the current block based on the second block vector candidate list.
60. A codec comprising a second memory and a second processor; wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 28 to 55 when running the computer program.
61. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 27 is implemented, or the method according to any one of claims 28 to 55 is implemented.