Image decoding method, image encoding method, device and computer storage medium
By generating an extended motion vector angle prediction mode and building a candidate list, the problem of insufficient motion vector angle prediction in the prior art is solved, and image decoding efficiency and video encoding compression efficiency are improved.
Patent Information
- Application Number
- CN202410996952.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-07-24
AI Technical Summary
The existing motion vector angle prediction mode has limited angle prediction direction in video encoding and large sub-blocks, which leads to insufficient MV angle prediction, affecting compression efficiency.
An image decoding method is proposed, by generating an extended motion vector angle prediction mode, increasing the angle prediction direction, and building a candidate list based on block information and motion vector angle prediction mode, selecting the optimal prediction mode for decoding, improving the flexibility of motion vector angle prediction and accuracy.
By increasing the angle prediction direction and sub-block processing, the efficiency and accuracy of image decoding are improved, and a more flexible motion vector angle prediction method is provided, which improves the compression efficiency of video encoding.
Smart Images

Figure CN118540496B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image / video coding and decoding technology, and in particular to an image decoding method, an image coding method, an image coding and decoding device, and a computer storage medium. Background Art
[0002] The amount of video image data is relatively large, and it is usually necessary to compress the video pixel data (RGB, YUV, etc.). The compressed data is called a video stream, which is transmitted to the user end through a wired or wireless network and then decoded for viewing. The entire video encoding process includes block division, prediction, transformation, quantization, encoding and other processes.
[0003] The Motion Vector Angle Prediction (MVAP) mode in AVS3 uses the motion vector (MV) of the adjacent decoded block to predict the MV of the current block. However, the current MVAP has a limited number of angle prediction directions and a large MVAP sub-block, which is not conducive to more precise MV angle prediction. Summary of the invention
[0004] In order to solve the above technical problems, the present application proposes an image decoding method, an image encoding method, an image encoding and decoding device and a computer storage medium.
[0005] In order to solve the above technical problems, the present application proposes an image decoding method, which includes:
[0006] Obtaining the intra-frame prediction mode and prediction mode sequence number of the coded block based on the coding syntax of the image code stream;
[0007] Based on the original motion vector angle prediction mode, generating an extended motion vector angle prediction mode;
[0008] Based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode, obtaining a candidate motion vector angle prediction mode;
[0009] deriving a prediction mode candidate list based on the intra prediction mode of the coded block and the candidate motion vector angle prediction mode;
[0010] Based on the prediction mode sequence number, obtaining a target prediction mode in the prediction mode candidate list;
[0011] The image code stream is decoded according to the target prediction mode to obtain a reconstructed image.
[0012] The step of generating an extended motion vector angle prediction mode based on the original motion vector angle prediction mode includes:
[0013] Extending at least one extended angular prediction direction between adjacent angular prediction directions of the original motion vector angular prediction mode;
[0014] Based on the extended angular prediction direction, the extended motion vector angular prediction mode is generated.
[0015] The acquiring of a candidate motion vector angle prediction mode based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode includes:
[0016] Obtain block information of the block to be decoded;
[0017] In response to the value of the block information being greater than or equal to a preset threshold, taking the original motion vector angle prediction mode and the extended motion vector angle prediction mode as the candidate motion vector angle prediction mode;
[0018] In response to a value of the block information being less than a preset threshold, the original motion vector angle prediction mode is used as the candidate motion vector angle prediction mode.
[0019] The block information is one or more of width and height characteristics, the number of pixels, and aspect ratio.
[0020] wherein the prediction mode candidate list is derived based on the intra prediction mode of the coded block and the candidate motion vector angle prediction mode,
[0021] Obtaining a valid number according to the candidate motion vector angle prediction mode and the adjacent block motion vectors of the block to be decoded;
[0022] Based on the effective number, determining the length of the candidate list;
[0023] A prediction mode candidate list is derived based on the intra prediction mode of the encoded block and the candidate list length.
[0024] The obtaining of the effective number according to the candidate motion vector angle prediction mode and the adjacent block motion vectors of the block to be decoded includes:
[0025] Acquire adjacent block motion vectors of the block to be decoded, wherein the adjacent block motion vectors include a plurality of minimum coding unit motion vectors;
[0026] Traversing each candidate motion vector angle prediction mode, and obtaining a minimum coding unit motion vector corresponding to each candidate motion vector angle prediction mode;
[0027] When the minimum coding units corresponding to the candidate motion vector angle prediction mode are all available and the motion vectors are different, the candidate motion vector angle prediction mode is valid;
[0028] Count the valid number of all candidate motion vector angle prediction modes.
[0029] The step of deriving the prediction mode candidate list based on the intra prediction mode of the coded block and the candidate list length includes:
[0030] Get a list of valid modes;
[0031] Deriving a target mode list from the intra prediction mode of the encoded block according to a preset candidate derivation method;
[0032] The prediction mode candidate list is constructed using the valid mode list and the target mode list.
[0033] The preset candidate derivation method includes one or more of intra-frame prediction mode derivation, motion vector angle prediction mode inheritance, gradient-based derivation, and fixed candidate derivation.
[0034] Wherein, after deriving the target mode list from the intra prediction mode of the encoded block according to the preset candidate derivation method, the image decoding method further includes:
[0035] In response to the presence of an intra non-angle prediction mode in the target mode list, converting the intra non-angle prediction mode into a preset candidate motion vector angle prediction mode;
[0036] In response to the existence of an intra-frame angle prediction mode in the target mode list, the angle direction of the intra-frame angle prediction mode is converted to a candidate motion vector angle prediction mode with a closest angle.
[0037] Wherein, after deriving the target mode list from the intra prediction mode of the encoded block according to the preset candidate derivation method, the image decoding method further includes:
[0038] In response to the preset candidate derivation method being inheritance of the original motion vector angle prediction mode, and the intra-frame prediction mode of the encoded block being an extended motion vector angle prediction mode, according to the preset mapping relationship and the principle of the nearest angle prediction direction, the extended motion vector angle prediction mode of the encoded block is mapped to the original motion vector angle prediction mode.
[0039] Wherein, the image decoding method further includes:
[0040] Obtain predicted pixels of the block to be decoded;
[0041] Calculate the gradient direction and gradient magnitude of each predicted pixel;
[0042] Accumulating the gradient magnitudes of the predicted pixels having the same angle prediction mode as the candidate motion vector corresponding to the gradient direction;
[0043] The candidate motion vector angle prediction mode with the largest gradient magnitude is taken as one of the prediction mode candidate list of the block to be decoded.
[0044] Wherein, after deriving a prediction mode candidate list based on the intra-frame prediction mode of the coded block and the candidate motion vector angle prediction mode, the image decoding method further includes:
[0045] Dividing the block to be decoded into a plurality of sub-blocks to be decoded;
[0046] The motion vector angle prediction mode of each sub-block to be decoded is determined based on the prediction mode candidate list of the block to be decoded.
[0047] The step of determining the motion vector angle prediction mode of each sub-block to be decoded based on the prediction mode candidate list of the block to be decoded includes:
[0048] Determine a motion vector angle prediction mode for each sub-block to be decoded based on the prediction mode candidate list of the block to be decoded and the individual sub-block prediction mode syntax;
[0049] Alternatively, based on the prediction mode candidate list of the block to be decoded and the prediction mode syntax of the first sub-block, the motion vector angle prediction mode of the first sub-block to be decoded is determined, and the motion vector angle prediction modes of the remaining sub-blocks to be decoded are determined using the motion vector angle prediction mode and relative offset of the first sub-block to be decoded.
[0050] In order to solve the above technical problems, the present application also proposes an image encoding method, which includes:
[0051] Based on the original motion vector angle prediction mode, generating an extended motion vector angle prediction mode;
[0052] Based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode, obtaining a candidate motion vector angle prediction mode;
[0053] Derivation of a prediction mode candidate list based on the intra prediction mode of the coded block and the candidate motion vector angle prediction mode;
[0054] Obtaining a coding cost value for each prediction mode in the prediction mode candidate list;
[0055] The target prediction mode with the optimal coding cost value is used to encode the coding block to obtain a coded bitstream.
[0056] Wherein, the image encoding method further includes:
[0057] Setting the motion vector angle prediction mode switch syntax based on whether the image coding method is enabled;
[0058] Based on the motion vector angle prediction mode scheme, setting the motion vector angle prediction mode scheme syntax;
[0059] The motion vector angle prediction mode scheme syntax includes a scheme execution syntax and a scheme index syntax.
[0060] In order to solve the above-mentioned technical problems, the present application also proposes an image coding and decoding device, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the image decoding method and / or image encoding method as mentioned above.
[0061] In order to solve the above technical problems, the present application also proposes a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the above image decoding method and / or image encoding method.
[0062] Compared with the prior art, the beneficial effects of the present application are as follows: the image encoding and decoding device obtains the intra-frame prediction mode and the prediction mode sequence number of the encoded block based on the coding syntax of the image code stream; generates the extended motion vector angle prediction mode based on the original motion vector angle prediction mode; obtains the candidate motion vector angle prediction mode based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode; derives the prediction mode candidate list based on the intra-frame prediction mode of the encoded block and the candidate motion vector angle prediction mode; obtains the target prediction mode in the prediction mode candidate list based on the prediction mode sequence number; decodes the image code stream according to the target prediction mode to obtain a reconstructed image. Through the above-mentioned image decoding method, more flexible motion vector angle prediction methods are provided, which effectively improves the efficiency of image decoding. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0064] Figure 1 is a schematic diagram of the MVAP angle provided in this application;
[0065] Figure 2 It is a schematic diagram of obtaining MVs of adjacent blocks of MVAP provided by this application;
[0066] Figure 3 is a schematic diagram of the horizontal MAVP mode provided by the present application;
[0067] Figure 4 It is a flowchart of EMVAP decoding provided by this application;
[0068] Figure 5 It is a flowchart of an embodiment of an image encoding method provided by the present application;
[0069] Figure 6 is a schematic diagram of the EMVAP angle prediction mode provided by the present application;
[0070] Figure 7 is a schematic diagram of the candidate locations of the MVAP airspace provided by this application;
[0071] Figure 8 is a schematic diagram of an intra-frame angle prediction mode provided by the present application;
[0072] Fig. 9 is a pixel schematic diagram of the gradient calculation provided by this application;
[0073] Fig.10 yes Figure 5 The specific flow chart of step S15 of the image encoding method shown;
[0074] Fig.11 It is a schematic diagram of the EMVAP inter-frame sub-block division provided by the present application;
[0075] Fig.12 is a schematic diagram of the sub-block MV provided by the present application;
[0076] Fig.13 It is a flowchart of an embodiment of an image decoding method provided by the present application;
[0077] Fig.14 It is a structural schematic diagram of an embodiment of an image encoding and decoding device provided by the present application;
[0078] Fig.15 It is a structural diagram of an embodiment of a computer storage medium provided by the present application. DETAILED DESCRIPTION
[0079] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0080] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0081] The MVAP involved in this application supports five angle prediction modes, namely horizontal (mode 0, Horizontal), vertical (mode 1, Vertical), horizontal up (mode 2, Horizontal_Up), horizontal down (mode 3, Horizontal_Down), vertical right (mode 4, Vertical_Right), as shown in detail. Figure 1 As shown, Figure 1 It is a schematic diagram of the MVAP angle provided in this application.
[0082] The specific process of MVAP can be divided into the following four steps:
[0083] (1) Get the MV of the adjacent block: Figure 2 As shown, Figure 2 This is a schematic diagram of MVAP adjacent block MV acquisition provided by this application. Wherein, m=W / 4, n=H / 4, W, H are the width and height of the coding block respectively. Starting from the upper left corner, obtain the motion vectors of (H+W) / 4 scu (minimum coding unit) of adjacent rows and columns, as well as the motion vector of the upper left corner scu. The specific position is as follows Figure 2 A0 to A 2m+2n Among them, the motion vector can only be obtained when scu exists, has been decoded, and is in inter-frame prediction mode; otherwise, the motion vector cannot be obtained.
[0084] (2) MVAP mode validity check: Only MVAP modes that have been verified to be valid will be used. The verification of each angle of MVAP is as follows:
[0085] (Horizontal) Mode 0: If A m-1+H / 8 With A m+n-1 If the SCUs at the positions are all available and the MVs are different, mode 0 is valid; otherwise, mode 0 is invalid.
[0086] (Vertical) Mode 1: If A m+n+1+W / 8With A m+n+1 If the SCUs at the positions are all available and the MVs are different, Mode 1 is valid; otherwise, Mode 1 is invalid.
[0087] (Horizontal Upward) Mode 2: If Am+n-1 With A m+n The SCUs at the location are all available and the MVs are different, or A m+n and A m+n+1 If the SCUs at the positions are all available and the MVs are different, Mode 2 is valid; otherwise, Mode 2 is invalid.
[0088] (Horizontal downward) Mode 3: If A W / 8-1 With A m-1 The SCUs at the location are all available and the MVs are different, or A m-1 With A m-1+H / 8 If the SCUs at the locations are all available and the MVs are different, Mode 3 is valid; otherwise, Mode 3 is invalid.
[0089] (Vertical right) Mode 4: If A m+n+1+W / 8 With A 2m+n+1 The SCUs at the location are all available and the MVs are different, or A 2m+n+1 With A 2m+n+1+H / 8 If the SCUs at the positions are all available and the MVs are different, Mode 4 is valid; otherwise, Mode 4 is invalid.
[0090] (3) Adjacent MV filling: traverse adjacent blocks A in sequence i scu of position, i=0~2m+2n.
[0091] When i=0, if the A0 position scu is not available, the corresponding motion vector is filled with a zero vector; otherwise, the A0 position scu is filled with an actual motion vector;
[0092] When i≠0, if the scu at the corresponding position is available, then A i Position is filled with the actual motion vector; otherwise, A i Position filling A i-1 The motion vector of the position.
[0093] 4) Current MV derivation: According to the angle prediction mode of MVAP, the MV of each 8*8 sub-block of the current CU is derived. The specific process of each mode is described as follows:
[0094] m=W>>2
[0095] n= H>>2
[0096] if (curMvapMode == 0) {
[0097] MotionArray[i][j]= neighborMotions[m + n - 1 – (j<<1)]
[0098] }
[0099] else if (curMvapMode == 1) {
[0100] MotionArray[i][j]= neighborMotions[m + n + 1 + (i<<1)]
[0101] }
[0102] else if (curMvapMode == 2) {
[0103] MotionArray[i][j]= neighborMotions[m + n - (j<<1) + (i<<1)]
[0104] }
[0105] else if (curMvapMode == 3) {
[0106] MotionArray[i][j]= neighborMotions[m + n - (j<<1) - (i<<1) - 3]
[0107] }
[0108] else if (curMvapMode == 4) {
[0109] MotionArray[i][j]= neighborMotions[m + n + (j<<1) + (i<<1) + 3]
[0110] }
[0111] Among them, MotionArray is the MV matrix of the 8*8 sub-block, i and j are the vertical and horizontal indexes of the 8*8 sub-block in the CU, and neighborMotions is the MV of the neighboring block.
[0112] Taking the horizontal prediction mode (mode 0) as an example, the prediction process of MVAP of 16*16 CU is as follows Figure 3 As shown, Figure 3 It is a schematic diagram of the horizontal MAVP mode provided by the present application. The MVs of the two upper sub-blocks are the MVs corresponding to the A1 position, and the MVs of the two lower sub-blocks are the MVs corresponding to the A3 position.
[0113] In the Skip / direct inter-frame prediction mode of the AVS3 standard, the motion vector (MV) candidate list is shown in Table 1 below, where NumOfMvapCand is the maximum number of MVAP candidates (5 when the MVAP tool is turned on, otherwise 0), and NumOfHmvpCand is the maximum number of historical motion vector (HMVP, History-based Motion Vector Prediction) candidates allowed in SPS (the value range is 0~8).
[0114] In addition, if the Skip / direct candidate list is still not full after all candidates are added, it is filled with the last HMVP candidate or the last airspace candidate (when the current HMVP is closed).
[0115] Table 1 Skip / direct general candidate build list
[0116]
[0117] However, the current MVAP has a limited number of angle prediction directions and a large MVAP sub-block, which is not conducive to more precise MV angle prediction; although MVAP assumes that MV has correlation in the upper spatial direction, it does not fully utilize the spatial correlation and texture information of the image content; MVAP in the CU (coding unit) uses the same prediction mode, which has poor flexibility and limits the prediction effect of MVAP to a certain extent.
[0118] In this regard, the present application provides an enhanced motion vector angle prediction (EMVAP) method, which improves the MV accuracy of the CU and thus the compression efficiency by introducing more angle prediction directions and reducing the size of the MVAP sub-block; uses spatial correlation, texture and other information to screen MVPA candidates and improve the prediction efficiency of MVAP; in order to improve the prediction flexibility of MVAP, the CU is divided into multiple subCUs, each subCU has an independent MVAP mode.
[0119] The purpose of the EMVAP method provided in this application is to determine the motion vector of the current coding block, and its decoding process is as follows: Figure 4 As shown, Figure 4 The flowchart of EMVAP decoding provided by the present application is as follows: First, the relevant syntax of EMVAP is decoded from the bitstream, then a candidate list of EMVAP prediction modes is constructed, the EMVAP prediction mode is determined by combining the two, and finally the EMVAP prediction process is executed.
[0120] The design of EMVAP in this proposal includes the following four aspects:
[0121] 1) EMVAP prediction technology: EMVAP prediction technology supports multiple angle prediction directions to predict the MV of the current block.
[0122] 2) EMVAP candidate construction technology: The EMVAP candidate list technology uses information such as spatial correlation and texture content to construct the EMVAP mode candidate list.
[0123] 3) EMVAP inter-frame sub-block partitioning prediction technology: This technology allows the current CU to be divided into sub-blocks to obtain subCUs, and the EMVAP prediction mode is determined in units of subCUs.
[0124] 4) EMVAP syntax design: In conjunction with the EMVAP candidate construction technology, the corresponding syntax is designed to determine the EMVAP prediction mode that is ultimately used.
[0125] Please refer to Figure 5 , Figure 5 It is a flowchart of an embodiment of an image encoding method provided by the present application.
[0126] The image coding method of the present application is applied to an image coding and decoding device, wherein the image coding and decoding device of the present application can be a server, a terminal device, or a system in which a server and a terminal device cooperate with each other. Accordingly, the various parts included in the image coding and decoding device, such as various units, sub-units, modules, and sub-modules, can all be set in the server, can all be set in the terminal device, or can be set in the server and the terminal device respectively.
[0127] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules used to provide distributed servers, or it can be implemented as a single software or software module, which is not specifically limited here.
[0128] like Figure 5 As shown, the specific steps are as follows:
[0129] Step S11: Generate an extended motion vector angle prediction mode based on the original motion vector angle prediction mode.
[0130] In the embodiments of the present application, the EMVAP prediction mode expansion method in the EMVAP prediction technology provided by the present application includes but is not limited to:
[0131] 1) Based on the original MVAP angle prediction direction, one or more angle prediction directions are extended between two adjacent MVAP angles. The extended angle prediction direction can be the center angle, quarter angle, etc. of the two adjacent angles.
[0132] 2) Based on the range of the prediction angle, a number of prediction angles with equal intervals are divided.
[0133] In the first specific implementation, it is assumed that the EMVAP prediction model is as follows Figure 6 As shown, Figure 6 It is a schematic diagram of the EMVAP angle prediction mode provided in this application. Figure 6 The EMVAP in the MPEGAP includes 9 angle prediction directions, and the EMVAP mode numbers are 0 to 8. Among them, 0 to 4 are conventional MVAP angle prediction modes, that is, the original motion vector angle prediction modes, and modes 5 to 8 are based on the center angle of the original adjacent angle prediction direction as the direction, and are extended to obtain the MVAP angle prediction mode, that is, the extended motion vector angle prediction mode.
[0134] Step S12: Based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode, a candidate motion vector angle prediction mode is obtained.
[0135] In an embodiment of the present application, the image encoding and decoding device determines the applied EMVAP prediction mode, that is, the candidate motion vector angle prediction mode, according to the application condition of the EMVAP prediction mode in the EMVAP prediction technology.
[0136] Specifically, the present application determines the use of the angle prediction mode in EMVAP based on the CU size, including but not limited to:
[0137] 1) Based on the relationship between the width and height features of the CU and the set feature threshold, determine whether to use the conventional MVAP prediction mode, the extended MVAP prediction mode, or all MVAP prediction modes.
[0138] 2) Based on the relationship between the number of pixels of the CU and the pixel number threshold, determine whether to use the conventional MVAP prediction mode, the extended MVAP prediction mode, or all MVAP prediction modes.
[0139] 3) Based on the relationship between the aspect ratio of the CU and the set aspect ratio threshold, determine whether to use the conventional MVAP prediction mode, the extended MVAP prediction mode, or all MVAP prediction modes.
[0140] In the second specific implementation, it is assumed that the conventional MVAP prediction mode, the extended MVAP prediction mode and all MVAP prediction modes are determined based on the relationship between the width and height features of the CU and the set feature thresholds, and the width and height feature thresholds are both set to 32, as follows:
[0141] When the width W of the CU is greater than or equal to 32 and the height H of the CU is greater than or equal to 32, the MVAP prediction modes allowed by EMVAP are modes 0 to 8; otherwise, EMVAP allows the use of MVAP prediction modes 0 to 4.
[0142] For another example, when the number of pixels of the CU is greater than or equal to a pixel number threshold, the MVAP prediction modes allowed by EMVAP are modes 0 to 8; otherwise, EMVAP allows the use of MVAP prediction modes 0 to 4.
[0143] For another example, when the aspect ratio of the CU is greater than or equal to the aspect ratio threshold, the MVAP prediction modes allowed by EMVAP are modes 0 to 8; otherwise, EMVAP allows the use of MVAP prediction modes 0 to 4.
[0144] In other specific embodiments, other methods may be used to determine the candidate prediction mode, including but not limited to:
[0145] When the width W of the CU is greater than or equal to 32 and the height H of the CU is greater than or equal to 32, the MVAP prediction modes allowed by EMVAP are modes 0 to 4; otherwise, EMVAP allows the use of MVAP prediction modes 0 to 8.
[0146] When the number of pixels of the CU is greater than or equal to the pixel number threshold, the MVAP prediction modes allowed by EMVAP are modes 0 to 4; otherwise, EMVAP allows the use of MVAP prediction modes 0 to 8.
[0147] When the aspect ratio of the CU is greater than or equal to the aspect ratio threshold, the MVAP prediction modes allowed by EMVAP are modes 0 to 4; otherwise, EMVAP allows the use of MVAP prediction modes 0 to 8.
[0148] Furthermore, the image encoding and decoding device determines the sub-block size of the EMVAP prediction according to the EMVAP sub-block granularity in the EMVAP prediction technology.
[0149] Specifically, the sub-block size predicted by EMVAP only needs to be smaller than the CU size, and the sub-block size can be determined based on features, including but not limited to:
[0150] 1) Determine the EMVAP sub-block size based on the relationship between the CU width, height, aspect ratio, number of pixels, and the corresponding thresholds.
[0151] 2) All scale CUs use EMVAP sub-blocks of the same size.
[0152] In a third specific implementation, it is assumed that, based on the relationship between the width and height of the CU and the corresponding thresholds, the EMVAP sub-block size is determined to be 4*4 or 8*8, and the width and height thresholds are both set to 32, as follows:
[0153] When CU width W>=32 and CU height H>=32, the sub-block width subW=4 and the sub-block height subH=4; otherwise, the sub-block width subW=8 and the sub-block height subH=8.
[0154] Step S13: derive a prediction mode candidate list based on the intra prediction mode of the encoded block and the candidate motion vector angle prediction mode.
[0155] In the implementation of this application, the EMVAP prediction process is similar to the MVAP prediction process, but the mode verification process and the sub-block MV derivation process are different from the prior art. The specific process is as follows:
[0156] (1) Acquisition of adjacent block MV (same as acquisition of adjacent block MV in MVAP): Figure 2 As shown, starting from the upper left corner, the motion vectors of (H+W) / 4 scus of adjacent rows and columns and the motion vector of the upper left corner scu are obtained. The MV can only be obtained when the scu exists, has been decoded, and is in inter-frame prediction mode; otherwise, the MV cannot be obtained.
[0157] (2) Mode check (similar to the mode check of MVAP, but with different prediction modes): For a given angle prediction mode, the mode prediction process requires adjacent block MVs, and selects some of these MVs for duplication check. If the MVs in the selected samples are inconsistent, the mode is valid; otherwise, the mode is invalid.
[0158] Specifically, the image coding and decoding device obtains the motion vectors of the adjacent blocks of the block to be encoded, and the adjacent block motion vectors include the motion vectors of several scu. Then, the image coding and decoding device determines the motion vector of the corresponding scu according to each candidate motion vector angle prediction mode, and performs a mode check. That is, if the corresponding scu is available and the motion vectors are different, it proves that the candidate motion vector angle prediction mode is valid. Finally, the image coding and decoding device counts the valid number of all candidate motion vector angle prediction modes for subsequent EMVAP candidate construction.
[0159] (3) Adjacent MV filling (same as adjacent MV filling of MVAP):
[0160] 1. If the A0 position scu is not available, the corresponding motion vector is filled with the forward predicted zero vector; otherwise, the A0 position scu is filled with the actual motion vector.
[0161] 2. Traverse adjacent blocks A one by one i scu of position, i=1~2m+2n.
[0162] If available, then A i position is filled with the actual motion vector; otherwise, A i Position filling A i-1 The motion vector of the position.
[0163] (4) Sub-block MV derivation (similar to the sub-block MV derivation of MVAP, but with different prediction modes): The CU is divided into sub-blocks, and the MV of each sub-block is derived based on the MVAP prediction angle.
[0164] In the fourth specific implementation, the specific process of mode verification is as follows:
[0165] Mode 0: If A m-1+H / 8 With A m+n-1 If the SCUs at the positions are all available and the MVs are different, mode 0 is valid; otherwise, mode 0 is invalid.
[0166] Mode 1: If A m+n+1+W / 8 With A m+n+1 If the SCUs at the positions are all available and the MVs are different, Mode 1 is valid; otherwise, Mode 1 is invalid.
[0167] Mode 2: If A m+n-1 With A m+n The SCUs at the location are all available and the MVs are different, or A m+n and A m+n+1 If the SCUs at the positions are all available and the MVs are different, Mode 2 is valid; otherwise, Mode 2 is invalid.
[0168] Mode 3: If A W / 8-1 With A m-1 The SCUs at the location are all available and the MVs are different, or A m-1 With A m-1+H / 8 If the SCUs at the locations are all available and the MVs are different, Mode 3 is valid; otherwise, Mode 3 is invalid.
[0169] Mode 4: If A m+n+1+W / 8 With A 2m+n+1 The SCUs at the location are all available and the MVs are different, or A 2m+n+1 With A 2m+n+1+H / 8 If the SCUs at the positions are all available and the MVs are different, Mode 4 is valid; otherwise, Mode 4 is invalid.
[0170] Mode 5: If Am+n-1+H / 16 With A m+n-1+H / 16+H / 8 If the SCUs at the positions are all available and the MVs are different or Mode 2 is valid, Mode 5 is valid; otherwise, Mode 5 is invalid.
[0171] Mode 6: If A m+n+1+W / 16 With A m+n-1+W / 16+W / 8 If the SCUs at the positions are all available and the MVs are different or Mode 2 is valid, Mode 6 is valid; otherwise, Mode 6 is invalid.
[0172] Mode 7: If A W / 8-1+W / 16 With A m-1+H / 16 The SCUs at the location are all available and the MVs are different, or A m-1+H / 16 With A m-1+H / 8+H / 16 If the SCUs at the positions are all available and the MVs are different, Mode 7 is valid; otherwise, Mode 7 is invalid.
[0173] Mode 8: If A m+n+1+W / 8-W / 16 With A 2m+n+1-W / 16 The SCUs at the location are all available and the MVs are different, or A 2m+n+1-W / 16 With A 2m+n+1+H / 8-H / 16 If the SCUs at the positions are all available and the MVs are different, Mode 8 is valid; otherwise, Mode 8 is invalid.
[0174] The specific process of sub-block MV derivation is as follows:
[0175] Based on the angular prediction mode of EMVAP, the MV of each sub-block can be calculated as follows: Where subH is the sub-block height, subW is the sub-block width, logScuSize is log2(scu_size), scu_size is the scu size, and i and j are the horizontal and vertical indexes of the sub-block.
[0176] m=W>>2
[0177] n= H>>2
[0178] if (curMvapMode == 0) {
[0179] MotionArray[i][j]= neighborMotions[m + n - 1 – (j<<(subH>>logScuSize))]
[0180] }
[0181] else if (curMvapMode == 1) {
[0182] MotionArray[i][j]= neighborMotions[m + n + 1 + (i<<(subW>>logScuSize))]
[0183] }
[0184] else if (curMvapMode == 2) {
[0185] MotionArray[i][j]= neighborMotions[m + n - (j<<(subH>>logScuSize)) +(i<<(subW>>logScuSize))]
[0186] }
[0187] else if (curMvapMode == 3) {
[0188] MotionArray[i][j]= neighborMotions[m + n - (j<<(subH>>logScuSize)) -(i<<(subW>>logScuSize)) - 3]
[0189] }
[0190] else if (curMvapMode == 4) {
[0191] MotionArray[i][j]= neighborMotions[m + n + (j<<(subH>>logScuSize)) +(i<<(subW>>logScuSize)) + 3]
[0192] else if (curMvapMode == 5) {
[0193] MotionArray[i][j]= neighborMotions[m + n – n>>2 - (j<<(subH>>logScuSize))<<1 + (i<<(subW>>logScuSize))]
[0194] }
[0195] else if (curMvapMode == 6) {
[0196] MotionArray[i][j]= neighborMotions[m + n + m>>2 - (j<<(subH>>logScuSize))>>1 + (i<<(subW>>logScuSize))]
[0197] }
[0198] else if (curMvapMode == 7) {
[0199] MotionArray[i][j]= neighborMotions[m + n – (n>>2)*3 - (j<<(subH>>logScuSize))<<1- (i<<(subW>>logScuSize))]
[0200] }
[0201] else if (curMvapMode == 8) {
[0202] MotionArray[i][j]= neighborMotions[m + n + (m>>2)*3 + (j<<(subH>>logScuSize))>>1+ (i<<(subW>>logScuSize))]
[0203] }
[0204] In the embodiment of the present application, the image encoding and decoding device uses the EMVAP candidate construction technology to construct a candidate list, which specifically includes two parts:
[0205] (1) EMVAP candidate list construction: How to construct the EMVAP candidate list.
[0206] (2) EMVAP candidate list application: Application of EMVAP candidates in the encoding and decoding process.
[0207] Among them, the EMVAP candidate list construction technology constructs one or several EMVAP candidate lists. The design of the candidate list mainly includes: 1) candidate list length design; 2) candidate derivation method; 3) candidate list construction.
[0208] Specifically, the candidate list length design part confirms the candidate list length N that needs to be constructed, including but not limited to the valid number of conventional MVAP angle prediction modes, the valid number of extended MVAP angle prediction modes, the valid number of all MVAP angle prediction modes, and several fractions of these valid numbers, as well as a fixed number, etc.
[0209] In an embodiment of the present application, the candidate derivation method part describes how the candidates in the candidate list are derived, and the derivation methods include but are not limited to: intra-frame prediction mode derivation, EMVAP inheritance, gradient-based derivation, fixed candidates and the like.
[0210] In addition, the derived EMVAP angle prediction mode may belong to all MVAP angle prediction modes or only belong to the conventional MVAP angle prediction mode or only belong to the extended MVAP angle prediction mode. The above derivation methods are described in detail below:
[0211] Intra-frame prediction mode derivation: The EMVAP angle prediction mode is derived based on the intra-frame prediction mode of spatially adjacent blocks or non-adjacent blocks.
[0212] EMVAP inheritance: The EMVAP angle prediction mode is derived based on the EMVAP angle prediction modes obtained from spatially adjacent or non-adjacent blocks.
[0213] Based on gradient derivation: Gradient calculation is performed based on spatially adjacent reconstructed pixels or non-adjacent reconstructed pixels or predicted pixels, and the gradient amplitude of the EMVAP angle mode corresponding to each gradient direction is counted, and one or several angle modes with the largest gradient amplitude are selected.
[0214] Fixed candidate derivation: From a pre-set set of prediction modes, the EMVAP angle prediction mode is derived in a fixed order.
[0215] The candidate list construction section only describes the method of constructing the candidate list, and does not limit the specific construction process of the candidate list. The method of constructing the candidate list includes but is not limited to:
[0216] 1) Conventional candidate list: The candidate list is filled based on one or more derivation methods to form a candidate list; the number and order of each derivation method added to the candidate list are arbitrary and are not limited here.
[0217] 2) Complete the candidate list: If one or more candidate lists already exist, exclude one or more candidates in the existing candidate lists from the valid EMVAP candidate set to obtain a completed candidate list; wherein, the candidates in the valid EMVAP candidate set can be arranged in a fixed order, ascending order, descending order, etc.
[0218] In the fifth specific implementation, the candidate list length is: assuming the valid number M of conventional MVAP angle prediction modes, M=4, assuming the valid modes are {0, 1, 3, 4}.
[0219] Candidate derivation method: fixed candidates. Assuming that the pre-set set is all regular MVAP prediction modes, the order of the fixed candidates is from small to large according to the MVAP angle prediction mode number, then the pre-set set is {0, 1, 2, 3, 4}.
[0220] Candidate list construction: The candidate list is constructed based on a fixed derivation method, and {0,1,3,4} is inserted into the candidate list in the order of {0,1,2,3,4}.
[0221] In the sixth specific implementation, the length of the candidate list: the length of the candidate list is a fixed number of 2.
[0222] Candidate derivation method: intra-frame prediction mode derivation, fixed candidate derivation, and conventional MVAP angle prediction mode are obtained.
[0223] Candidate list construction (conventional candidate list): Construct the candidate list based on intra prediction mode derivation and fixed candidate derivation.
[0224] The specific process is as follows:
[0225] (1) Acquisition of intra-frame modes of adjacent blocks: in the order of F, G, C, A, B, D, the specific positions are as follows Figure 7 As shown, Figure 7 : is a schematic diagram of MVAP spatial candidate positions provided by the present application, where E is the current CU. If the adjacent block exists and is in intra-frame prediction coding mode, the adjacent block intra-frame prediction mode is obtained; otherwise, the adjacent block intra-frame angle prediction mode is not obtained.
[0226] (2) The intra-frame prediction mode is converted to the conventional MVAP angle prediction mode. Figure 8 FIG. 1 is a schematic diagram of an intra-frame angle prediction mode. Figure 8 It is a schematic diagram of the intra-frame angle prediction mode provided by the present application, where 0, 1, and 2 are non-angle prediction modes.
[0227] For intra non-angular prediction mode, it is directly converted to the regular MVAP prediction mode 0 or no conversion is performed.
[0228] For the intra-frame angle prediction mode, it is converted to the conventional MVAP prediction mode with the closest angle to the conventional MVAP prediction mode based on the corresponding angle direction. A possible conversion is shown in Table 2 below:
[0229] Table 2 Conversion table between intra-frame angle prediction mode and conventional MVAP angle prediction mode
[0230]
[0231] In addition, when the adjacent block angle prediction mode is converted to the conventional MVAP prediction mode, if the converted conventional MVAP prediction mode already exists in the list, the mode will not be added to the list again. If the list does not reach the required length, it is derived in a fixed candidate manner, that is, the valid conventional MVAP prediction mode is added to the candidate list in the order of conventional MVAP prediction mode 0, 1, 2, 3, 4 until the required list length is reached.
[0232] In the seventh specific implementation, the length of the candidate list: the length of the candidate list is a fixed number of 2.
[0233] Candidate derivation method: EMVAP inheritance, only inherits the conventional MVAP angle prediction mode.
[0234] Candidate list construction (regular candidate list): Build the candidate list based on the EMVAP inheritance method.
[0235] The specific process is as follows: Figure 7 In the order of F, G, C, A, B, D, traverse the adjacent blocks. If the adjacent block exists and is in EMVAP prediction mode, obtain the EMVAP prediction mode of the adjacent block as the inherited EMVAP mode. Due to the restriction that only the conventional MVAP angle prediction mode is inherited, if the EMVAP prediction mode of the adjacent block is an extended MVAP angle prediction mode, based on the principle of the closest angle prediction direction, the extended MVAP prediction mode is converted to the conventional MVAP prediction mode. A possible mapping table is shown in Table 3 below:
[0236] Table 3 Conversion table between conventional EMVAP angle prediction mode and EMVAP angle prediction mode
[0237]
[0238] In an eighth specific implementation, the length of the candidate list: the length of the candidate list is a fixed number 1.
[0239] Candidate derivation method: derive a regular MVAP angle prediction mode based on gradient.
[0240] Candidate list construction (regular candidate list): Build a candidate list based on gradient inference.
[0241] The specific process is as follows:
[0242] (1) Gradient calculation: Sobel operator gradient calculation is performed based on the predicted pixels of the current block, i.e., the block to be encoded. The gradient magnitude and gradient direction (corresponding to the conventional MVAP angle mode) of each predicted pixel are counted. The conventional MVAP angle prediction mode with the largest gradient magnitude is taken as the final derived conventional MVAP angle prediction mode.
[0243] The expression for gradient calculation is as follows:
[0244] Gh=2*f(x+1,y)+f(x+1,y-1)+f(x+1,y+1)-2*f(x-1,y)-f(x-1,y-1)-f(x-1,y+1)
[0245] Gv=2*f(x,y+1)+f(x-1,y+1)+f(x+1,y+1)-2*f(x,y-1)-f(x-1,y-1)-f(x+1,y-1)
[0246] Among them, Gh and Gv are the horizontal gradient and vertical gradient respectively, and x and y are the horizontal and vertical coordinates of the predicted pixel.
[0247] Among them, the predicted pixels for gradient calculation only include the pixels inside the prediction block, such as Fig. 9 The white area shown is shown, where Fig. 9 It is a pixel schematic diagram of the gradient calculation provided by this application.
[0248] (2) Count the gradient direction (corresponding regular MVAP angle pattern) and gradient amplitude of each pixel. If the regular MVAP angle patterns of different pixels are the same, the corresponding gradient amplitudes are accumulated.
[0249] Specifically, the gradient direction is calculated as follows:
[0250] =atan(Gv / Gh)
[0251] Among them, atan() is the inverse tangent function.
[0252] Switch to the prediction mode that is closest to the conventional MVAP prediction direction, such as (-pi / 8, pi / 8), then it corresponds to mode 0, [-pi / 8, -pi*3 / 8] corresponds to mode 3.
[0253] The gradient magnitude is calculated as follows:
[0254] G=|Gh|+|Gv|
[0255] (3) Obtain the final conventional MVAP pattern. Based on the above steps, the conventional MVAP pattern with the largest gradient amplitude is statistically obtained. It is assumed that the conventional MVAP pattern with the largest gradient amplitude is pattern 2.
[0256] In the ninth specific implementation, the candidate list length is: the effective number M of conventional MVAP angle prediction modes, assuming M=4.
[0257] Candidate derivation methods: gradient-based derivation, intra-frame prediction mode derivation, EMVAP inheritance, fixed candidate derivation conventional MVAP angle prediction mode.
[0258] Candidate list construction (conventional candidate list): The candidate list is constructed in the order of gradient-based derivation, intra-frame prediction mode derivation, EMVAP inheritance, and fixed candidate derivation.
[0259] Assume that the number of candidates allowed to be added in the list is 1, 2, and 1 respectively based on gradient derivation, intra-frame prediction mode derivation, and EMVAP inheritance. When the first three derivations are completed, only when the candidate list does not reach the required length (in a single derivation method, when the number of allowed additions is not reached, no addition is made), addition is made in a fixed manner. In addition, during the candidate addition process, if there is a prediction mode to be added in the candidate list, it will not be added repeatedly.
[0260] For example, suppose the valid modes of conventional MVAP are 0, 1, 2, and 4. Mode 2 is derived based on the gradient, mode 4 is derived based on the intra prediction mode, and EMVAP inherits no mode. In the candidate list, mode 2 is added first, then mode 4, and then modes 0 and 1.
[0261] Furthermore, the EMVPA candidate list application provided by this application includes but is not limited to the following three application modes:
[0262] (1) Compatible merge mode: insert several EMVAP candidate lists into any position of the Skip / direct candidate list.
[0263] (2) Independent of merge mode: The EMVAP candidate list is not inserted into the Skip / direct candidate list and is used as an independent prediction mode.
[0264] (3) Mixed mode: There is a compatible merge mode, as well as an independent and merge mode.
[0265] In the tenth specific implementation, a compatible merge mode is adopted: an EMVAP candidate list is inserted into the Skip / direct candidate list, and the insertion position is after the spatial domain candidate; it is assumed that the EMVAP candidate list is constructed in the same manner as in the ninth specific implementation.
[0266] In the eleventh specific implementation, a compatible merge mode is adopted: two EMVAP candidate lists are inserted into the Skip / direct candidate list, and the insertion position is after the spatial candidate and after the HMVP.
[0267] Assume that the first candidate list is constructed in the same manner as in the eighth specific implementation manner, and the candidate mode of the EMVAP candidate list is mode 2.
[0268] The second candidate list is constructed in the manner of completing the candidate list. Assuming that only modes 0, 1, 2, and 4 are valid in the EMVAP candidates, the candidate modes of the second EMVAP candidate list are 0, 1, and 4.
[0269] Combined with the existing Skip / direct candidate list construction technology, candidates in the second EMVAP candidate list will be inserted into the Skip / direct candidate list one by one only when the list length does not reach the maximum length after the HMVP is inserted.
[0270] In the twelfth specific implementation, a prediction mode independent of the merge mode is adopted: the Skip / direct candidate list does not contain any EMVAP, and the EMVAP candidate list is an independent inter-frame prediction mode candidate list.
[0271] In the thirteenth specific implementation, a hybrid mode is adopted: one EMVAP candidate list is inserted into the Skip / direct candidate list after the spatial candidates; and another EMVAP candidate list is used as an independent EMVAP inter-frame prediction mode.
[0272] Assume that the first candidate list is constructed in the same manner as in the eighth specific implementation manner, and the candidate mode of the EMVAP candidate list is mode 2.
[0273] The second candidate list is constructed based on the completion candidate list method. Assuming that only modes 0, 1, 2, and 4 are valid in the EMVAP candidates, the candidate modes of the second EMVAP candidate list are 0, 1, and 4.
[0274] Insert the second EMVAP candidate list into the Skip / direct candidate list and use the first EMVAP candidate list as an independent inter-frame EMVAP prediction mode.
[0275] Step S14: Obtain the encoding cost value of each prediction mode in the prediction mode candidate list.
[0276] In the embodiment of the present application, the encoding cost value types used to select a prediction mode with the best cost from several prediction mode candidates in the prediction mode candidate list include but are not limited to: rate distortion cost, coarse selection cost, absolute difference value and the like.
[0277] Step S15: Encode the block to be encoded using the target prediction mode with the optimal encoding cost value to obtain an encoded bitstream.
[0278] In an embodiment of the present application, the image coding and decoding device performs motion estimation according to the prediction mode candidate list determined in the above steps, obtains the optimal MV of the block to be encoded, thereby determining the prediction block of the block to be encoded, and finally uses the prediction block to encode the block to be encoded to obtain the encoded bitstream.
[0279] Furthermore, this application also proposes an EMVAP inter-frame sub-block partitioning prediction technology, for details, please refer to Fig.10 , Fig.10 yes Figure 5 The specific flow chart of step S15 of the image coding method is shown. The EMVAP inter-frame sub-block division technology provided by the present application includes but is not limited to the following three aspects: inter-frame sub-block division method, inter-frame sub-block EMVAP mode, and sub-block MV inheritance.
[0280] like Fig.10 As shown, the specific steps are as follows:
[0281] Step S151: Divide the block to be encoded into a number of sub-blocks to be encoded.
[0282] In the embodiment of the present application, regarding the inter-frame sub-block division method, the image coding and decoding device divides the CU into several subCUs, and the division method includes but is not limited to: dividing the CU into several subCUs of the same size along horizontal, vertical, diagonal and other angles.
[0283] Step S152: Determine the motion vector angle prediction mode of each sub-block to be encoded based on the prediction mode candidate list of the block to be encoded.
[0284] In the embodiment of the present application, regarding the EMVAP mode part of the inter-frame sub-block, the image encoding and decoding device determines the EMVAP prediction mode of each subCU, including but not limited to:
[0285] 1) Independent EMVAP mode: Each subCU determines the angle prediction mode based on an independent EMVAP candidate list and corresponding syntax.
[0286] 2) Offset EMVAP mode: The EMVAP angle prediction mode of each subCU is offset based on the EMVAP angle prediction mode of the first subCU. The EMVAP angle prediction mode of the first subCU is determined based on the candidate list technology and the corresponding syntax.
[0287] Regarding the sub-block MV inheritance, during the EMVAP prediction process, the image coding and decoding device can use the MV of the subCU in the previous decoding order for prediction for the subCU in the later decoding order.
[0288] In the fourteenth specific implementation manner, the inter-frame sub-block division method is as follows: Fig.11 As shown, Fig.11 It is a schematic diagram of the EMVAP inter-frame sub-block division provided in this application. The CU can be divided into 4 subCUs, i.e., coding sub-units, in the horizontal and vertical directions.
[0289] EMVAP mode of inter-frame sub-blocks (independent EMVAP mode): Assume that the EMVAP list of CU contains modes {0,1,2,3}. The EMVAP mode of subCU1 is 1, the EMVAP mode of subCU2 is 1, the EMVAP mode of subCU3 is 0, and the EMVAP mode of subCU4 is 3.
[0290] Or, the EMVAP mode of the inter-frame sub-block (offset EMVAP mode): Assuming that the EMVAP prediction mode of subCU1 is idx1, the EMVAP prediction mode of subCU2 is idx1+idxOffset2, the EMVAP prediction mode of subCU3 is idx1+idxOffset3, and the EMVAP prediction mode of subCU3 is idx1+idxOffset4. Assuming that the maximum allowed mode offset is 1, idxOffset2, idxOffset3, idxOffset4 are {-1, 0, 1}.
[0291] Use sub-block MV: Fig.12 The 16*16 CU is divided into four 8*8 subCUs. Fig.12 This is a schematic diagram of the sub-block MV provided by this application. For subCU4, its adjacent block MV is A' i , where A'0, A'1, A'7, A'8 come directly from A2, A3, A 13 , A 14 , the MVs of A'2, A'3, A'4, A'5, and A'6 come from the MVs of the corresponding positions of the subCU.
[0292] Furthermore, the present application also provides an EMVAP syntax design, including but not limited to: 1) EMVAP switch syntax; 2) EMVAP scheme syntax; 3) EMVAP syntax encoding.
[0293] (1) EMVAP switch syntax: used to express whether the technology proposed in this proposal is enabled in the codec. When multiple technologies are involved, different technologies are allowed to have independent switch syntax. The switch syntax can be transmitted in syntax structures including but not limited to video parameter sets, sequence parameter sets, picture parameter sets, picture headers, coding units, etc.
[0294] (2) EMVAP scheme syntax: used to express how to execute a specific EMVAP scheme in the decoder. The scheme syntax includes but is not limited to:
[0295] 1) Scheme execution syntax: When there is an EMVAP candidate list independent of the merge mode, it is used to express whether the independent EMVAP candidate list scheme is executed.
[0296] 2) Scheme index syntax: For an EMVAP candidate list, when the candidate list length is greater than 1, it is used to express the execution of a candidate in the EMVAP candidate list.
[0297] The scheme syntax can be transmitted in syntax structures including but not limited to video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), photo head (PH), coding unit, etc.
[0298] (3) EMVAP syntax coding: The syntax coding methods include but are not limited to advanced entropy coding, signed fixed-length coding, unsigned fixed-length coding, exponential Golomb coding, etc.; among them, the coding methods are described in the standard text in the form of descriptors, and the specific meanings refer to the corresponding standard text.
[0299] In the fifteenth specific embodiment, it is assumed that the codec enables the conventional MVAP prediction mode by default.
[0300] (1) EMVAP switch syntax:
[0301] For the extended EMVAP mode, the switch syntax emvap_mode_extend_flag is used to express whether the codec allows the EMVPA mode to be enabled; when emvap_mode_extend_flag=0, it means that the extended EMVAP mode is prohibited, and NumOfMvapCand=5; when emvap_mode_extend_flag=1, it means that the extended EMVAP mode is allowed, and NumOfMvapCand=9.
[0302] For EMVAP candidate construction technology, the switch syntax emvap_cand_flag is used to express whether the codec allows the enabling of EMVAP candidate construction technology; when emvap_cand_flag=0, it means that EMVAP candidate construction technology is prohibited; when emvap_cand_flag=1, it means that EMVAP candidate construction technology is allowed.
[0303] For the EMVAP inter-frame sub-block partitioning technology, the switch syntax emvap_isp_flag is used to express whether the codec allows the EMVAP inter-frame sub-block partitioning technology to be enabled. When emvap_isp_flag=0, it means that the EMVAP inter-frame sub-block partitioning technology is prohibited; when emvap_isp_flag=1, it means that the EMVAP inter-frame sub-block partitioning technology is allowed.
[0304] In addition, an EMVPA master switch syntax emvap_enable_flag can be designed to express whether the above three technologies are allowed to be enabled, and then further determine whether the corresponding technologies are allowed to be enabled based on the switch syntax of these technologies. When emvap_enable_flag=0, it means that all EMVAP technologies are prohibited; when emvap_enable_flag=1, it means that EMVAP technologies are allowed to be enabled.
[0305] The following is an example of EMVAP switch syntax in SPS:
[0306]
[0307] (2) EMVAP scheme syntax
[0308] Assume that when the extended MVAP prediction mode is turned off, that is, emvap_mode_extend_flag=0, the EMVAP candidate list technology is turned on, that is, emvap_cand_flag=1, and the EMVAP inter-frame sub-block partitioning prediction technology is turned off, that is, emvap_isp_flag=0:
[0309] Assume that the EMVAP candidate construction technology is the technology in the eleventh specific implementation mode, assume that HMVP has 4 candidates, EMVAPi and HMVPj represent the i-th EMVAP mode and the j-th HMVP mode respectively, when NumOfMvapCand=5, NumOfHmvpCand=8, taking P frame as an example, the skip / direct candidate list is as follows:
[0310]
[0311] Therefore, the semantics of the skip / direct candidate list index cu_subtype_index (also the EMVAP scheme index syntax) in the coding unit syntax changes. When cu_subtype_index = 2, the candidate is the EMVPA mode based on gradient derivation in the eleventh specific implementation; when cu_subtype_index is located after HMVP, cu_subtype_index corresponds to the valid EMVAP candidate obtained based on the candidate list completion method in the eleventh specific implementation.
[0312]
[0313] In the sixteenth specific implementation, the switch syntax is the same as the switch syntax of the fifteenth specific implementation.
[0314] (1) EMVAP scheme syntax
[0315] Assume that the extended MVAP prediction mode is turned off, that is, emvap_mode_extend_flag=0, the EMVAP candidate list technology is turned on, that is, emvap_cand_flag=1, and the EMVAP inter-frame sub-block partitioning prediction technology emvap_isp_flag=0 is turned off.
[0316] Assuming that the EMVAP candidate construction technology is the technology in the thirteenth specific implementation mode, assuming that there are 4 HMVPs, taking P frames as an example, the skip / direct candidate list is as follows.
[0317]
[0318] Therefore, the index cu_subtype_index of the skip / direct candidate list in the coding unit syntax (also the EMVAP scheme index syntax) changes semantically and is transmitted only when the independent inter-frame EMVAP prediction mode is not executed. When cu_subtype_index is located after the spatial candidate and before the HMVP, the candidate is a valid EMVAP candidate obtained based on the candidate list completion method in the thirteenth specific implementation method.
[0319] In addition, the emvap_dimd_flag scheme execution syntax is used to express an independent inter-frame EMVAP prediction mode (since the length of the mode candidate list is only 1, there is no need to further transmit the scheme index syntax), as shown in the following table. When emvap_dimd_flag=0, it means that the EMVAP prediction mode derived based on gradient in the thirteenth specific implementation manner is not executed; when emvap_dimd_flag=1, it means that the EMVAP prediction mode derived based on gradient in the thirteenth specific implementation manner is executed.
[0320]
[0321] Please continue reading Fig.13 , Fig.13 It is a flowchart of an embodiment of an image decoding method provided by the present application.
[0322] like Fig.13 As shown, the specific steps are as follows:
[0323] Step S21: Obtain the intra-frame prediction mode and prediction mode sequence number of the encoded block based on the coding syntax of the image code stream.
[0324] In an embodiment of the present application, an image encoding and decoding device extracts the intra-frame prediction mode and the prediction mode sequence number of the encoded block from the syntax of the image code stream, wherein the prediction mode sequence number refers to the sequence number of the target prediction mode confirmed from the prediction mode candidate list according to the optimal cost value in the encoding stage.
[0325] Step S22: Generate an extended motion vector angle prediction mode based on the original motion vector angle prediction mode.
[0326] Step S23: Based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode, obtain a candidate motion vector angle prediction mode.
[0327] Step S24: derive a prediction mode candidate list based on the intra prediction mode of the encoded block and the candidate motion vector angle prediction mode.
[0328] In the embodiment of the present application, the candidate mode list construction technology in steps S22 to S25 in the image decoding method is the same as the candidate mode list construction technology in steps S11 to S14 in the image encoding method, and will not be repeated here.
[0329] Step S25: Based on the prediction mode sequence number, obtain the target prediction mode in the prediction mode candidate list.
[0330] In an embodiment of the present application, the image coding and decoding apparatus selects a target prediction mode from a prediction mode candidate list using a prediction mode sequence number.
[0331] Step S26: Decode the image code stream according to the target prediction mode to obtain a reconstructed image.
[0332] In an embodiment of the present application, the image encoding and decoding device obtains the intra-frame prediction mode and prediction mode sequence number of the encoded block based on the coding syntax of the image code stream; generates an extended motion vector angle prediction mode based on the original motion vector angle prediction mode; obtains a candidate motion vector angle prediction mode based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode; derives a prediction mode candidate list based on the intra-frame prediction mode of the encoded block and the candidate motion vector angle prediction mode; obtains a target prediction mode in the prediction mode candidate list based on the prediction mode sequence number; decodes the image code stream according to the target prediction mode to obtain a reconstructed image. Through the above-mentioned image decoding method, more flexible motion vector angle prediction methods are provided, which effectively improves the efficiency of image decoding.
[0333] This application provides more flexible motion vector angle prediction methods, including more angle prediction directions, more refined MVAP sub-block processing, etc.; utilizes space-time domain correlation to design a candidate list construction technology, which can effectively improve the efficiency of motion vector angle prediction; and also proposes CU sub-block division technology to further explore the compression potential of motion vector angle prediction.
[0334] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0335] In order to implement the above-mentioned image decoding method and / or image encoding method, the present application also proposes an image encoding and decoding device, for details, please refer to Fig.14 , Fig.14 It is a structural diagram of an embodiment of an image encoding and decoding device provided by the present application.
[0336] The image coding and decoding apparatus 400 of this embodiment includes a processor 41 , a memory 42 , an input and output device 43 , and a bus 44 .
[0337] The processor 41, memory 42, and input / output device 43 are respectively connected to a bus 44. The memory 42 stores program data. The processor 41 is used to execute the program data to implement the image decoding method and / or image encoding method described in the above embodiments.
[0338] In the embodiment of the present application, the processor 41 may also be referred to as a CPU (Central Processing Unit). The processor 41 may be an integrated circuit chip having the ability to process signals. The processor 41 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or the processor 41 may also be any conventional processor, etc.
[0339] This application also provides a computer storage medium, please continue to refer to Fig.15 , Fig.15 It is a structural diagram of an embodiment of a computer storage medium provided in the present application. The computer storage medium 600 stores a computer program 61. When the computer program 61 is executed by a processor, it is used to implement the image decoding method and / or image encoding method of the above-mentioned embodiment.
[0340] When the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
[0341] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. An image decoding method, characterized in that: The image decoding method comprises: Obtaining the intra-frame prediction mode and prediction mode sequence number of the coded block based on the coding syntax of the image code stream; Based on the original motion vector angle prediction mode, generating an extended motion vector angle prediction mode; Based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode, obtaining a candidate motion vector angle prediction mode; Derivation of a prediction mode candidate list based on the intra prediction mode of the coded block and the candidate motion vector angle prediction mode; Based on the prediction mode sequence number, obtaining a target prediction mode in the prediction mode candidate list; The image code stream is decoded according to the target prediction mode to obtain a reconstructed image.
2. The image decoding method according to claim 1, characterized in that: The step of generating an extended motion vector angle prediction mode based on the original motion vector angle prediction mode includes: Extending at least one extended angular prediction direction between adjacent angular prediction directions of the original motion vector angular prediction mode; Based on the extended angular prediction direction, the extended motion vector angular prediction mode is generated.
3. The image decoding method according to claim 1 or 2, characterized in that: The acquiring a candidate motion vector angle prediction mode based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode comprises: Obtain block information of the block to be decoded; In response to the value of the block information being greater than or equal to a preset threshold, taking the original motion vector angle prediction mode and the extended motion vector angle prediction mode as the candidate motion vector angle prediction mode; In response to a value of the block information being less than a preset threshold, the original motion vector angle prediction mode is used as the candidate motion vector angle prediction mode.
4. The image decoding method according to claim 3, characterized in that: The block information is one or more of width and height characteristics, the number of pixels, and aspect ratio.
5. The image decoding method according to claim 1, characterized in that: The intra-frame prediction mode of the coded block and the candidate motion vector angle prediction mode are used to derive a prediction mode candidate list, Obtaining a valid number according to the candidate motion vector angle prediction mode and the adjacent block motion vectors of the block to be decoded; Based on the effective number, determining the length of the candidate list; A prediction mode candidate list is derived based on the intra prediction mode of the encoded block and the candidate list length.
6. The image decoding method according to claim 5, characterized in that: The obtaining of the effective number according to the candidate motion vector angle prediction mode and the adjacent block motion vector of the block to be decoded includes: Acquire adjacent block motion vectors of the block to be decoded, wherein the adjacent block motion vectors include a plurality of minimum coding unit motion vectors; Traversing each candidate motion vector angle prediction mode, and obtaining a minimum coding unit motion vector corresponding to each candidate motion vector angle prediction mode; When the minimum coding units corresponding to the candidate motion vector angle prediction mode are all available and the motion vectors are different, the candidate motion vector angle prediction mode is valid; Count the valid number of all candidate motion vector angle prediction modes.
7. The image decoding method according to claim 5, characterized in that: The derivation of the prediction mode candidate list based on the intra prediction mode of the coded block and the candidate list length comprises: Get a list of valid modes; Deriving a target mode list from the intra prediction mode of the encoded block according to a preset candidate derivation method; The prediction mode candidate list is constructed using the valid mode list and the target mode list.
8. The image decoding method according to claim 7, characterized in that: The preset candidate derivation method includes one or more of intra-frame prediction mode derivation, motion vector angle prediction mode inheritance, gradient-based derivation, and fixed candidate derivation.
9. The image decoding method according to claim 7 or 8, characterized in that: After deriving the target mode list from the intra prediction mode of the encoded block according to the preset candidate derivation method, the image decoding method further includes: In response to the presence of an intra non-angle prediction mode in the target mode list, converting the intra non-angle prediction mode into a preset candidate motion vector angle prediction mode; In response to the existence of an intra-frame angle prediction mode in the target mode list, the angle direction of the intra-frame angle prediction mode is converted to a candidate motion vector angle prediction mode with a closest angle.
10. The image decoding method according to claim 7 or 8, characterized in that: After deriving the target mode list from the intra prediction mode of the encoded block according to the preset candidate derivation method, the image decoding method further includes: In response to the preset candidate derivation method being inheritance of the original motion vector angle prediction mode, and the intra-frame prediction mode of the encoded block being an extended motion vector angle prediction mode, according to the preset mapping relationship and the principle of the nearest angle prediction direction, the extended motion vector angle prediction mode of the encoded block is mapped to the original motion vector angle prediction mode.
11. The image decoding method according to claim 7, characterized in that: The image decoding method further includes: Obtain predicted pixels of the block to be decoded; Calculate the gradient direction and gradient magnitude of each predicted pixel; Accumulating the gradient magnitudes of the predicted pixels having the same angle prediction mode as the candidate motion vector corresponding to the gradient direction; The candidate motion vector angle prediction mode with the largest gradient magnitude is taken as one of the prediction mode candidate list of the block to be decoded.
12. The image decoding method according to claim 1, characterized in that: After deriving the prediction mode candidate list based on the intra prediction mode of the coded block and the candidate motion vector angle prediction mode, the image decoding method further includes: Dividing the block to be decoded into a number of sub-blocks to be decoded; The motion vector angle prediction mode of each sub-block to be decoded is determined based on the prediction mode candidate list of the block to be decoded.
13. The image decoding method according to claim 12, characterized in that: The step of determining the motion vector angle prediction mode of each sub-block to be decoded based on the prediction mode candidate list of the block to be decoded comprises: Determine a motion vector angle prediction mode for each sub-block to be decoded based on the prediction mode candidate list of the block to be decoded and the individual sub-block prediction mode syntax; Alternatively, based on the prediction mode candidate list of the block to be decoded and the prediction mode syntax of the first sub-block, the motion vector angle prediction mode of the first sub-block to be decoded is determined, and the motion vector angle prediction modes of the remaining sub-blocks to be decoded are determined using the motion vector angle prediction mode and relative offset of the first sub-block to be decoded.
14. An image encoding method, characterized in that: The image encoding method comprises: Based on the original motion vector angle prediction mode, generating an extended motion vector angle prediction mode; Based on the original motion vector angle prediction mode and the extended motion vector angle prediction mode, obtaining a candidate motion vector angle prediction mode; Derivation of a prediction mode candidate list based on the intra prediction mode of the coded block and the candidate motion vector angle prediction mode; Obtaining a coding cost value for each prediction mode in the prediction mode candidate list; The target prediction mode with the optimal coding cost value is used to encode the coding block to obtain a coded bitstream.
15. The image encoding method according to claim 14, characterized in that: The image encoding method further comprises: Setting the motion vector angle prediction mode switch syntax based on whether the image coding method is enabled; Based on the motion vector angle prediction mode scheme, setting the motion vector angle prediction mode scheme syntax; The motion vector angle prediction mode scheme syntax includes a scheme execution syntax and a scheme index syntax.
16. An image encoding and decoding device, characterized in that: The image coding and decoding device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the image decoding method according to any one of claims 1 to 13, and / or the image encoding method according to any one of claims 14 to 15.
17. A computer storage medium, characterized in that The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the image decoding method according to any one of claims 1 to 13 and / or the image encoding method according to any one of claims 14 to 15.
Citation Information
Patent Citations
Video signal processing method and device
CN109716773A
Decoding and coding method, device and equipment
CN113709457A
